Nguyen: Apache Spark Would Have Failed Earlier Due to Memory Costs
Christopher Nguyen · a16z Podcast | Making Sense of Big Data, Machine Learning, and Deep Learning · Jan 2, 2019 · at 14:45
Christopher Nguyen, CEO of Adatao, discusses how falling hardware memory costs were critical to the timing and market success of Apache Spark.
0:00 / 0:10exact quote · 10.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Now Spark, if it was created six, five, six years before its time would have completely failed because memory was so much more expensive.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Christopher Nguyen
Nguyen: Machine Learning Is the Primary Purpose of Big Data
“So it turns out the reason for big data is machine learning.”
Nguyen: Big Data Is Defined by Learning Thresholds, Not Volume
“So I like something that Peter Norvig, the director of research at Google said when he referred to big data, he says, big data is not just quantitatively different, but it's qualitatively different. In other words, there's something that happens when you have …”
Nguyen: Machine Learning Mirrors Human Learning From Experience
“And the way I think about big data is when machines learn from big data is very much like human beings learn from life experiences.”
Nguyen: Human Intuition Functions Like Parameters in a Machine Learning Model
“Well, what we think of as intuition are actually, you can think of as parameters inside a machine learning model.”
Nguyen: Modern Business Intelligence Uses ML to Predict Unknowns
“You can think of business intelligence going forward as the ability to apply machine learning algorithms to big data, and not just look at past questions, but also future questions, or asking to predict the unknowns from the knowns.”
Nguyen: Big Data Progress Is Driven by Cheaper Tech, Not Smarter People
“We don't necessarily get smarter over time. It's just that certain technologies get cheaper. They get, they become more available. So machine learning algorithms have always been around. The data that exists that you could collect has always been around. But i…”