Hadoop

“Most things, Hadoop, Spark, Storm, they think they're running by themselves. And so they compete for resources in really interesting ways.”

“We don't necessarily get smarter over time. It's just that certain technologies get cheaper. They get, they become more available. So machine learning algorithms have always been around. The data that exists that you could collect has always been around. But i…”

“I was a PhD student at UC Berkeley, and we actually started working with Hadoop users back in 2007. And I did, for example, an internship at Facebook when Facebook was only about 300 people and they were just starting to set up Hadoop.”

“So in particular you know, one of the things we saw is many of the projects that were built on top of Hadoop, such as Hive, which is a SQL processing at scale and Pig and Mahout for machine learning are starting to run on top of Spark as well, so that users of…”

“Some of the most frequently used kind of data analytics tools extremely basic, because they actually look and feel like sheet of paper, like two-dimensional sheet of paper, and you know, so that's the problem with analytics, that on one hand, we have, you know…”

“Spark doesn't subsume data warehousing. Hadoop doesn't subsume, you know, streaming. So they're just like different technologies for different jobs.”

“With all the investment in Hadoop and this and Hadoop that, you know, most companies are still data bankrupt. You know, Hadoop or Data Warehouse or whatever is a place where data goes to die”

“Well, you can say maybe we should use Hadoop or BI. Unfortunately, these are kind of older generation tools. They require specialized skills and abilities to be able to use them. And so they need armies of analysts to use. These are really only affordable for …”