Elprin: True data science reproducibility requires code, data, results, and environment
Nick (Domino Data Lab) · Lessons Learned from Advanced Data Science Orgs // Domino Data Lab [FirstMark's Data Driven] · Nov 9, 2016 · at 6:11
Nick Elprin is the co-founder and CEO of Domino Data Lab. Speaking at Data Driven NYC, he discusses structural and technical requirements for enterprise data science teams.
0:00 / 0:44exact quote · 44.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“What we've learned is that for this kind of work, this advanced analytics quantitative research work, reproducibility is more than just, hey, we have a snapshot of the code. You know, like, that's, you know, it's great for software engineering. GitHub does that really well. What we found is that if you actually want to reproduce what you did in some of these quantitative research experiments, you might need the data sets as well. You really want the results as well, not just sort of the source code you ran, because You're not building a binary artifact. You're building a you know, results of some analysis and rock curves or whatever. You also need the environment in which it ran. What were the versions of packages you had? What version of R was deployed? Because if you don't have that, then yeah, you can check out the source code again, but you can't actually reuse it. You can't actually reproduce the result.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Nick (Domino Data Lab)
Elprin: Apache Spark still generates more industry hype than actual business value
“I think there's still more hype around Spark than actual value extraction from it.”
Elprin: Elite data science organizations prioritize collective knowledge over solo practitioners
“The best organizations we've seen think of their work as contributing to collective knowledge.”
Elprin: Data science progress relies on compounding small insights, not epiphanies
“Progress actually comes from lots of little insights that compound over time”
Elprin: Agility to experiment with new tools beats single-platform lock-in
“That's never the answer. It's much more about having agility to let people rapidly experiment with whatever the next thing is that comes out”
Elprin: Regulatory compliance drives enterprise demand for data science reproducibility tools
“One of the really interesting things about reproducibility we've learned is that it's been extremely resonant in industries where there are regulatory and compliance concerns, and we're sort of seeing that more and more, especially in financial services, they'…”
Elprin: Drive organizational best practices by packaging them inside individual productivity tools
“What those people seem to want is the ability to test more ideas faster, that experimental agility some ways to sort of expose their work more out into the business. But let's package that in a way that automates or incentivizes best practices.”