Levine: Multimodal LLMs hold broad knowledge but lack physical grounding
Sergey Levine · World's Top Researcher on AI, LLMs, and Robot Intelligence · Invest Like The Best · Mar 31, 2026 · at 12:00
Sergey Levine, co-founder of Physical Intelligence, discusses using multimodal LLMs to give robots common-sense reasoning in unusual physical situations.
0:00 / 0:11exact quote · 11.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Multimodal language models are really good at pulling in knowledge and trying to articulate that knowledge. They're not very good at, like, grounding that knowledge in physical situations, but they know stuff.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Sergey Levine
Levine: General Robotics Models May Ultimately Be Easier Than Narrow Ones
“And part of the thesis of this company is that we believe that doing it at the full level of generality might actually in the long run be easier than trying to special case very specific narrow application domains.”
Levine: Multi-robot data will enable foundational physical models for rapid deployment
“So if we can draw on data from many sources, many applications, many robots, then we can have a model that has a physical understanding, and it'll be much, much easier to put new applications on top of that platform.”
Levine: Future Robots Won't Just Be Humanoid 'Metal People'
“And I think, you know, we, sometimes we think that, like, robots are going to be, like, one thing. Like, it's just like, you know, there's people, and now we're going to make, like, metal people, and that'll be, like, robots. But I don't think that's how it's …”
Levine: Robotic foundation models will adapt across diverse physical form factors
“And I think that in the future we'll have A robotic foundation model, which can then be adapted to all sorts of applications, and they might really run the gamut from like, you know, like bulldozers or something, to humanoids, to robotic arms like this thing, …”
Levine: Chain-of-thought reasoning allows robots to handle edge cases
“So the way you get common sense is by essentially using chain of thought. So the robot enters a scene and instead of directly starting to move, it thinks about what it was asked to do. So if it was told to clean up the kitchen, looks at the scene and says, lik…”