Chip Huyen: Machine learning projects should start with problems, not models
Chip Huyen · Fireside Chat: Chip Huyen with Matt Turck (Partner, FirstMark) · Mar 15, 2021 · at 3:15
Chip Huyen, instructor for Stanford's ML System Design course, discusses system architecture methodology with Matt Turck of FirstMark Capital.
0:00 / 0:41exact quote · 41.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So so the main idea is you go backward from the problems. So I think a lot of approach machine is saying it's like, you start with the solutions and it tries to have like five problems when machine can I be applied. So, and it's like, so tend to be like, oh, Hey, this is fancy model coming out like bird or whatsoever. And then let's try to like, see, like let's try to run it and see like whether it help us. And I think it's just like, it's very interesting in R and D, but I think it's just like a wrong approach when you try to slow actual problems. So I was trying to encourage my students to like, look at the problems. Like what problems is this? Like, what is the easiest, simplest solutions? And it doesn't have to be machine learning. It can be like non-machine learning or if it's machine learning, it can be a very simple model.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Chip Huyen
Chip Huyen argues post-training is what differentiates frontier AI models
“So, so I do think that post-training is what makes this, like, really big lab models are, like, different.”
Evaluation is the single biggest bottleneck holding back enterprise AI adoption
“So, so I do think that evaluation is the biggest bottleneck for AI adoptions, because unless, like, if we can, like, if we can, like, develop a more reliable way to evaluate the application, that application is not going to get adopted. Like, or maybe, maybe i…”
Chip Huyen warns million-token context capacity does not imply efficient processing
“Just because a model, I think the second reason may be actually more important at least for now is that just because a model can fit in a million con token context doesn't mean that it can process that million token efficiently.”
Chip Huyen argues human-generated plans are poor training data for AI agents
“When we ask humans to generate like what they consider the best plan for an actions, for a task, it's actually like not quite the best plan for AI, because what is what is easy or efficient for humans is not the same as easy and efficient for AI, right?”
Huyen: Legacy systems prevent US companies from matching Chinese online learning
“So a lot of American, American internet companies are like a lot older than the average, like the new, like Chinese internet company. So it means it's like American internet companies have legacy systems that you from like, 20 years ago. And just have to build…”