StarZero All shows Try the podcast clipper →

Mar 15, 2021 · 26m · mad

Fireside Chat: Chip Huyen with Matt Turck (Partner, FirstMark)

Chip Huyen · 19m spoken Matt Turck · 3m spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

In this Data Driven NYC fireside chat, Chip Huyen and Matt Turck discuss machine learning system design, the evolution of MLOps tooling, real-time online learning architectures, and global data infrastructure trends.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 17.4% of the talking time here. How this is scored →

Matt as informed peer 2.7 Guest teaching 5.7 Guest disagreement 0.5 Matt pushing back 1.7
05100:0010:0020:000:11–4:12 · Matt as informed peer 2/10 Teaching Machine Learning System Design at Stanford Matt sets up the interview with friendly open-ended questions about Chip's Stanford course on ML system design. Chip educates the host on why taking a systematic architectural approach is necessary rather than piecing together tutorials.4:12–7:24 · Matt as informed peer 2/10 Evolution of the Machine Learning Tooling Landscape Matt prompts Chip on her survey of the ML tooling ecosystem. Chip explains tracking 284+ tools and details the historical transition from pre-deep learning frameworks to the PyTorch and TensorFlow consolidation.7:24–10:08 · Matt as informed peer 3/10 Current State of ML Tools: Low-Hanging Fruit vs. Core Challenges Matt asks Chip to clarify what she means by the landscape being both crowded and underdeveloped. Matt pushes on specific definitions around data generation versus data movement when Chip highlights low-hanging fruit like data labeling.10:08–18:09 · Matt as informed peer 3/10 Online Machine Learning vs. Offline Batch Predictions Chip presents a detailed breakdown contrasting batch offline predictions like Netflix with real-time online learning like TikTok. Matt asks clarifying questions about whether the difference is algorithmic or infrastructural while acknowledging his limited ML coursework background.18:09–21:05 · Matt as informed peer 4/10 ML Infrastructure Differences Between East and West Matt introduces a well-informed topic on how ML infrastructure trends diverge between China and the West. Chip explains how Chinese tech companies adopted online learning faster because they were not burdened by legacy batch systems.21:05–24:46 · Matt as informed peer 2/10 Audience Q&A: Online Learning Strategies and Applications Matt reads audience questions about online learning strategies and practical applications. Chip candidly shares insights on recommendation systems and customer support while humbly acknowledging the limits of her hands-on experience building full online learning setups.0:11–4:12 · Guest teaching 5/10 Teaching Machine Learning System Design at Stanford Matt sets up the interview with friendly open-ended questions about Chip's Stanford course on ML system design. Chip educates the host on why taking a systematic architectural approach is necessary rather than piecing together tutorials.4:12–7:24 · Guest teaching 5/10 Evolution of the Machine Learning Tooling Landscape Matt prompts Chip on her survey of the ML tooling ecosystem. Chip explains tracking 284+ tools and details the historical transition from pre-deep learning frameworks to the PyTorch and TensorFlow consolidation.7:24–10:08 · Guest teaching 6/10 Current State of ML Tools: Low-Hanging Fruit vs. Core Challenges Matt asks Chip to clarify what she means by the landscape being both crowded and underdeveloped. Matt pushes on specific definitions around data generation versus data movement when Chip highlights low-hanging fruit like data labeling.10:08–18:09 · Guest teaching 7/10 Online Machine Learning vs. Offline Batch Predictions Chip presents a detailed breakdown contrasting batch offline predictions like Netflix with real-time online learning like TikTok. Matt asks clarifying questions about whether the difference is algorithmic or infrastructural while acknowledging his limited ML coursework background.18:09–21:05 · Guest teaching 6/10 ML Infrastructure Differences Between East and West Matt introduces a well-informed topic on how ML infrastructure trends diverge between China and the West. Chip explains how Chinese tech companies adopted online learning faster because they were not burdened by legacy batch systems.21:05–24:46 · Guest teaching 5/10 Audience Q&A: Online Learning Strategies and Applications Matt reads audience questions about online learning strategies and practical applications. Chip candidly shares insights on recommendation systems and customer support while humbly acknowledging the limits of her hands-on experience building full online learning setups.0:11–4:12 · Guest disagreement 1/10 Teaching Machine Learning System Design at Stanford Matt sets up the interview with friendly open-ended questions about Chip's Stanford course on ML system design. Chip educates the host on why taking a systematic architectural approach is necessary rather than piecing together tutorials.4:12–7:24 · Guest disagreement 0/10 Evolution of the Machine Learning Tooling Landscape Matt prompts Chip on her survey of the ML tooling ecosystem. Chip explains tracking 284+ tools and details the historical transition from pre-deep learning frameworks to the PyTorch and TensorFlow consolidation.7:24–10:08 · Guest disagreement 1/10 Current State of ML Tools: Low-Hanging Fruit vs. Core Challenges Matt asks Chip to clarify what she means by the landscape being both crowded and underdeveloped. Matt pushes on specific definitions around data generation versus data movement when Chip highlights low-hanging fruit like data labeling.10:08–18:09 · Guest disagreement 1/10 Online Machine Learning vs. Offline Batch Predictions Chip presents a detailed breakdown contrasting batch offline predictions like Netflix with real-time online learning like TikTok. Matt asks clarifying questions about whether the difference is algorithmic or infrastructural while acknowledging his limited ML coursework background.18:09–21:05 · Guest disagreement 0/10 ML Infrastructure Differences Between East and West Matt introduces a well-informed topic on how ML infrastructure trends diverge between China and the West. Chip explains how Chinese tech companies adopted online learning faster because they were not burdened by legacy batch systems.21:05–24:46 · Guest disagreement 0/10 Audience Q&A: Online Learning Strategies and Applications Matt reads audience questions about online learning strategies and practical applications. Chip candidly shares insights on recommendation systems and customer support while humbly acknowledging the limits of her hands-on experience building full online learning setups.0:11–4:12 · Matt pushing back 1/10 Teaching Machine Learning System Design at Stanford Matt sets up the interview with friendly open-ended questions about Chip's Stanford course on ML system design. Chip educates the host on why taking a systematic architectural approach is necessary rather than piecing together tutorials.4:12–7:24 · Matt pushing back 1/10 Evolution of the Machine Learning Tooling Landscape Matt prompts Chip on her survey of the ML tooling ecosystem. Chip explains tracking 284+ tools and details the historical transition from pre-deep learning frameworks to the PyTorch and TensorFlow consolidation.7:24–10:08 · Matt pushing back 3/10 Current State of ML Tools: Low-Hanging Fruit vs. Core Challenges Matt asks Chip to clarify what she means by the landscape being both crowded and underdeveloped. Matt pushes on specific definitions around data generation versus data movement when Chip highlights low-hanging fruit like data labeling.10:08–18:09 · Matt pushing back 2/10 Online Machine Learning vs. Offline Batch Predictions Chip presents a detailed breakdown contrasting batch offline predictions like Netflix with real-time online learning like TikTok. Matt asks clarifying questions about whether the difference is algorithmic or infrastructural while acknowledging his limited ML coursework background.18:09–21:05 · Matt pushing back 2/10 ML Infrastructure Differences Between East and West Matt introduces a well-informed topic on how ML infrastructure trends diverge between China and the West. Chip explains how Chinese tech companies adopted online learning faster because they were not burdened by legacy batch systems.21:05–24:46 · Matt pushing back 1/10 Audience Q&A: Online Learning Strategies and Applications Matt reads audience questions about online learning strategies and practical applications. Chip candidly shares insights on recommendation systems and customer support while humbly acknowledging the limits of her hands-on experience building full online learning setups.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 26.8% · guest 73.2%0:00 · Matt 26.8% · guest 73.2%3:00 · Matt 19.8% · guest 80.2%3:00 · Matt 19.8% · guest 80.2%6:00 · Matt 7.1% · guest 92.9%6:00 · Matt 7.1% · guest 92.9%9:00 · Matt 16.3% · guest 83.7%9:00 · Matt 16.3% · guest 83.7%12:00 · Matt 8% · guest 92%12:00 · Matt 8% · guest 92%15:00 · Matt 14% · guest 86%15:00 · Matt 14% · guest 86%18:00 · Matt 16.4% · guest 83.6%18:00 · Matt 16.4% · guest 83.6%21:00 · Matt 30% · guest 70%21:00 · Matt 30% · guest 70%24:00 · Matt 18.3% · guest 81.7%24:00 · Matt 18.3% · guest 81.7%
Sharpest disagreement ▶ 3:14 Rejecting solution-first ML approach

Chip forcefully criticizes the conventional R&D mindset where engineers pick fancy models before identifying real business problems, calling it fundamentally the wrong approach for production systems.

 Hardest push from Matt ▶ 8:37 Pressing on data management definitions

Matt refuses to accept a vague description of data bottlenecks and explicitly interrupts to ask whether Chip means generating data, moving data, or specific pipeline operations.

 Biggest teaching moment ▶ 16:39 Reframing machine learning convergence for online systems

Chip corrects standard textbook assumptions about ML training by explaining how traditional concepts like multiple epochs and stationary convergence fail when applied to real-time online learning.

 Matt holds his own ▶ 18:08 Introducing East vs West ML infrastructure divergence

Matt shows domain awareness by raising the geopolitical split in ML infrastructure tooling between China and Western tech ecosystems, prompting Chip to share niche industry observations.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
Teaching Machine Learning System Design at Stanford 2511 Matt sets up the interview with friendly open-ended questions about Chip's Stanford course on ML system design. Chip educates the host on why taking a systematic architectural approach is necessary rather than piecing together tutorials.
Evolution of the Machine Learning Tooling Landscape 2501 Matt prompts Chip on her survey of the ML tooling ecosystem. Chip explains tracking 284+ tools and details the historical transition from pre-deep learning frameworks to the PyTorch and TensorFlow consolidation.
Current State of ML Tools: Low-Hanging Fruit vs. Core Challenges 3613 Matt asks Chip to clarify what she means by the landscape being both crowded and underdeveloped. Matt pushes on specific definitions around data generation versus data movement when Chip highlights low-hanging fruit like data labeling.
Online Machine Learning vs. Offline Batch Predictions 3712 Chip presents a detailed breakdown contrasting batch offline predictions like Netflix with real-time online learning like TikTok. Matt asks clarifying questions about whether the difference is algorithmic or infrastructural while acknowledging his limited ML coursework background.
ML Infrastructure Differences Between East and West 4602 Matt introduces a well-informed topic on how ML infrastructure trends diverge between China and the West. Chip explains how Chinese tech companies adopted online learning faster because they were not burdened by legacy batch systems.
Audience Q&A: Online Learning Strategies and Applications 2501 Matt reads audience questions about online learning strategies and practical applications. Chip candidly shares insights on recommendation systems and customer support while humbly acknowledging the limits of her hands-on experience building full online learning setups.

Statements from this episode (8)

Insight
Chip Huyen: Machine learning projects should start with problems, not models
“So so the main idea is you go backward from the problems. So I think a lot of approach machine is saying it's like, you start with the solutions and it tries to have like five problems when machine can I be applied. So, and it's like, so tend to be like, oh, H…”
Chip Huyen Mar 15, 2021 ▶ 3:15
Opinion
Huyen: Snorkel AI is one of the better machine learning startups
“Snuggle is, is a great tool. And I think like it's one of the, like one of the better ML startups out there.”
Chip Huyen Mar 15, 2021 ▶ 5:00
Disclosure
Huyen: ML tooling survey tracked 284 tools after several shut down
“And I think the last version has about 284. It was supposed to be almost 300 and then a few of them died between like December like December, like two years ago and now December last year.”
Chip Huyen Mar 15, 2021 ▶ 5:19
Assertion Partly supported
Chip Huyen: Netflix generates recommendation batches offline rather than in real time
“All these recommendations actually generate offline.”
Chip Huyen Mar 15, 2021 ▶ 11:05
Insight
Chip Huyen: MLOps startups ignore real-time learning for low-hanging fruit
“We don't have tools for it yet. And I see very, very, very little tools focusing on it because most people are like targeting on focusing on low hanging fruit.”
Chip Huyen Mar 15, 2021 ▶ 17:58
Assertion Not checkable as stated
Huyen: Alibaba and ByteDance lead US firms in online ML scale
“When I was looking into online learning and I realized it's like on the examples I felt were by Chinese companies. And it could be I think I've heard some American companies doing that, but they are doing a much smaller scale, like a lot less complex models th…”
Chip Huyen Mar 15, 2021 ▶ 18:45
Insight
Huyen: Legacy systems prevent US companies from matching Chinese online learning
“So a lot of American, American internet companies are like a lot older than the average, like the new, like Chinese internet company. So it means it's like American internet companies have legacy systems that you from like, 20 years ago. And just have to build…”
Chip Huyen Mar 15, 2021 ▶ 20:39
Assertion Not checkable as stated
Chip Huyen highlights customer support as a massive untapped online ML market
“Another use case, I think it's like heavily underexplored and I don't see a lot of companies doing it except for a few, like I think except for very few, few big companies. So it's a customer service support.”
Chip Huyen Mar 15, 2021 ▶ 23:40
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more