Dec 5, 2013 · 58m · mad
Panel: Metamarkets, Kaggle and Quid // Data Driven NYC #4 // Mar 2012
Mike Driscoll · 16m spoken Matt Turck · 2m spoken Alex Jacqueline · 58s spoken Michael Selick · 53s spoken Carter Schoenwald · 50s spoken Garrett Vidal · 39s spoken
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
At Data Driven NYC #4, panel moderator Matt Turck hosts industry leaders Mike Driscoll, Jeremy Howard, and Sean Gourley to discuss early-stage startup data infrastructure, talent acquisition strategies, enterprise hype, and the evolving landscape of competitive machine learning.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 4.6% of the talking time here. How this is scored →
Matt as informed peer 1.9 Guest teaching 5.6 Guest disagreement 2.4 Matt pushing back 1.4
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 35:10 Direct dismissal over chemoinformatics literature
Jeremy Howard dismisses Mike Driscoll's speculation on pharma R&D progress by directly stating 'You don't know' regarding recent machine learning literature in chemoinformatics.
Hardest push from Matt ▶ 46:00 Host shuts down audience promotional pitchHost Matt Turck immediately cuts off an audience questioner who starts giving a marketing pitch for her company, instructing her to skip the intro and ask a question directly.
Biggest teaching moment ▶ 19:40 Correcting the misapplication of Norvig's data thesisJeremy Howard corrects Mike Driscoll on Peter Norvig's quote regarding data overriding algorithms, clarifying that Norvig specifically referred to human language translation rather than standard tabular machine learning problems.
Matt holds his own ▶ 5:06 Host demonstrates industry knowledge via Strata debateHost Matt Turck demonstrates domain knowledge by citing a recent Strata conference debate to formulate a structured question on domain expertise versus pure machine learning skill.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Hiring Data Science Talent: Domain vs. Machine Learning Expertise | 4 | 5 | 2 | 3 | Host Matt Turck cites a recent Strata conference debate to frame a question about hiring machine learning generalists versus domain experts. The guests expand on the premise, using Kaggle data and practical coding tests to explain how pragmatism outweighs traditional academic backgrounds. | |
| Kaggle Competitions and Private Market Liquidity | 1 | 6 | 1 | 0 | An audience member asks why Kaggle prize money seems low given the business value generated. Jeremy Howard educates the room on unpublicized private competitions, market liquidity, and participant motivations. | |
| Consumer Algorithms, Data Conglomerates, and Targeting Limits | 1 | 7 | 5 | 1 | Jeremy Howard vigorously challenges Mike Driscoll's claim that more data always beats better algorithms, pointing out that Peter Norvig's famous quote applied specifically to complex natural language translation rather than standard tabular machine learning problems. | |
| Data Team Ratios, Infrastructure Contracts, and Coding Skills | 1 | 5 | 3 | 0 | An audience member asks about data team support ratios. Driscoll and Howard debate the technical boundaries for data scientists, discussing whether writing MapReduce in Java is necessary or if SQL and high-level scripting suffice. | |
| Industry Hype, Enterprise Sales, and the Hadoop Mandate | 4 | 6 | 4 | 3 | An audience question about biotech hype leads to direct friction when Howard bluntly tells Driscoll he does not know the current chemoinformatics literature. Host Matt Turck steers the conversation toward enterprise sales challenges and the top-down mandate for Hadoop. | |
| The Evolution from Data Mining to Data Science | 1 | 5 | 2 | 0 | An audience member asks what truly changed between the era of data mining and data science. The guests reframe the issue, emphasizing open-source tooling, lower barriers to entry, and community democratization. | |
| Core Purpose of Data Science and Automation of Models | 3 | 5 | 2 | 6 | Host Matt Turck forcefully intervenes when an audience questioner turns her introduction into an explicit company sales pitch. Jeremy Howard then draws an analogy between 1940s human computers and the inevitable automation of data science tasks. | |
| Integrating Machine Learning Algorithms into Production Architecture | 1 | 6 | 2 | 0 | An audience member asks how model design accounts for production backend architecture. The guests explain a two-step process where optimal statistical models are simplified or translated into production code like PHP. | |
| Data Quality, Probabilistic Mindset, and Panel Conclusion | 1 | 5 | 1 | 0 | In response to a query about messy enterprise data quality, the panelists explain that organizations must adopt a probabilistic mindset rather than expecting deterministic perfection from real-world datasets. |
Statements from this episode (10)
Mike Driscoll: Solving real data problems beats taking data science courses
“I think that a lot of people who are interested in data science will, Decide that, you know, I gotta, I should go take a bunch of courses and read a bunch of books, and check a series of boxes, you know and I think that's actually the wrong approach. I think t…”
Mike Driscoll: Hire ML talent for structured problems, domain experts otherwise
“If your data is already structured well enough, and you've got a set of criteria to define the success of your machine learning algorithm, then you should hire the strongest pure machine learner you can find. You should find someone who would, you know, who's …”
Mike Driscoll: Reject data science candidates who fail basic command-line tasks
“If someone can't get an answer to that, cannot pull that column off and turn it into a human readable date string, In the order of minutes, you do not want that person as a data scientist.”
Mike Driscoll: Data-holding entities will win as algorithms move to data
“Analytics and algorithms are light. Data is very heavy. So the natural course of things will be that the algorithms will move to where the data lives. And the entities that have the data will ultimately win because of that strategic advantage”
Mike Driscoll: Telcos hold the world's most powerfully predictive social graphs
“If you want to know who has the most powerfully predictive social graphs on the planet, it's the telcos, because who you call and who you text is an extraordinarily strong indicator of who you actually are connected to in a social way.”
Mike Driscoll: Installing Hive on a Hadoop cluster increases usage by 10x
“Once you install Hive onto Hadoop cluster, usage of that cluster typically goes up by a factor of 10 because you lower the friction of getting data out of that system.”
Mike Driscoll: Big pharma R&D cuts create data science recruiting opportunities
“Most big pharma actually have turned, reduced their investment in R&D, and many of, it's a great place to recruit data scientists is out of these pharma R&D labs.”
Mike Driscoll: Klout is a bad company due to poor algorithms
“Clout's maybe a crappy company because their algorithms are bad”
Mike Driscoll: Running algorithms via Apache Mahout on Hadoop is too slow
“I think the problem with Mahoot is that anything, it's, many of these things are, if you run in Hadoop, you're slow. You need to be able to run in an environment that's fast”
Mike Driscoll: Combining human data labeling with machine learning outperforms pure automation
“People make this sort of distinction of either it's machine learning or it's people. Either you've got a team in India or, you know, Romania working through the data, trying to make sense of it or you've got some algorithm working on it, but I think there is a…”