Finn: Pre-trained VLMs let robots perform tasks with unseen internet concepts
Chelsea Finn · No Priors Ep. 107 | With Physical Intelligence Co-Founder Chelsea Finn · Mar 20, 2025 · at 7:05
Chelsea Finn, co-founder of Physical Intelligence and Stanford professor, discusses how foundation models transfer internet knowledge to robotic manipulation.
0:00 / 0:10exact quote · 10.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We had a research result a couple years ago where we showed that if you leverage vision language models, then you could actually get the robot to do tasks that require concepts that were never in the robot's training data, but were in the internet.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Chelsea Finn
Finn: Humanoids are overrated because collecting teleoperation data is too difficult
“On the other hand, I think that they're a little overrated, and one way it kind of to practically look at it is, I think that we're generally fairly bottlenecked on data right now, and some people argue that with humanoids, you can maybe collect data more easi…”
Finn: Robot training data transfers across different physical hardware embodiments
“We've seen a lot of evidence that you could actually transfer a lot of rich information across these different embodiments and allows you to use data. And also if you iterate on your robot platform, you don't have to throw all your data away.”
Finn: Biggest risk in generalist robotics is technical failure, not competition
“The last thing that I'll mention is that I think the biggest risk with this bet is that it won't work. Like, I'm not really worried about competitors. I'm more worried that no one will solve the problem.”
Finn: Wrist-mounted RGB cameras capture much of what tactile sensors provide
“And we found that actually that mounting RGB cameras to the wrists ends up being very, very helpful and probably giving you a lot of the same information that tactile sensors can give you.”
Finn: Embodied AI and motor control are underrated compared to language models
“I feel like actually people underestimate how much intelligence goes into motor control. Many, many years of evolution is what led to us being able to use our hands the way that we do. And there are many animals that they can't do it even though they had so mu…”
Finn: General multi-robot models outperformed research labs' custom single-robot policies
“We actually found that we could take a checkpoint, send that model checkpoint to another lab halfway across the country, and the grad student at that lab could run the checkpoint on the robot, and it would actually More often than not do better than the model …”