Elprin: Apache Spark still generates more industry hype than actual business value
Nick (Domino Data Lab) · Lessons Learned from Advanced Data Science Orgs // Domino Data Lab [FirstMark's Data Driven] · Nov 9, 2016 · at 15:56
Nick Elprin, CEO of Domino Data Lab, discusses data science tooling trends at FirstMark's Data Driven NYC in 2016.
0:00 / 0:05exact quote · 5.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I think there's still more hype around Spark than actual value extraction from it.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Nick (Domino Data Lab)
Elprin: Elite data science organizations prioritize collective knowledge over solo practitioners
“The best organizations we've seen think of their work as contributing to collective knowledge.”
Elprin: Data science progress relies on compounding small insights, not epiphanies
“Progress actually comes from lots of little insights that compound over time”
Elprin: Agility to experiment with new tools beats single-platform lock-in
“That's never the answer. It's much more about having agility to let people rapidly experiment with whatever the next thing is that comes out”
Elprin: True data science reproducibility requires code, data, results, and environment
“What we've learned is that for this kind of work, this advanced analytics quantitative research work, reproducibility is more than just, hey, we have a snapshot of the code. You know, like, that's, you know, it's great for software engineering. GitHub does tha…”
Elprin: Regulatory compliance drives enterprise demand for data science reproducibility tools
“One of the really interesting things about reproducibility we've learned is that it's been extremely resonant in industries where there are regulatory and compliance concerns, and we're sort of seeing that more and more, especially in financial services, they'…”
Elprin: Drive organizational best practices by packaging them inside individual productivity tools
“What those people seem to want is the ability to test more ideas faster, that experimental agility some ways to sort of expose their work more out into the business. But let's package that in a way that automates or incentivizes best practices.”