Levine: Text Representations in LLMs Are Suboptimal for Embodied AI
Sergey Levine · World's Top Researcher on AI, LLMs, and Robot Intelligence · Invest Like The Best · Mar 31, 2026 · at 1:06:16
Robotics researcher Sergey Levine highlights the structural differences needed between standard text LLMs and embodied foundation models.
0:00 / 0:26exact quote · 26.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So LLMs make certain kinds of representations very convenient. They make it very convenient to basically turn text into other text. But that's not necessarily the best representation for what an embodied system needs to do. Like sometimes it needs to think about things more spatially, sometimes semantically, sometimes other representations, and trying to figure out exactly how to structure that kind of internal thinking process might be a very important question, and the answer to that question might be different. In the world of embodied foundation models than it is in the world of LLMs.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Sergey Levine
Levine: General Robotics Models May Ultimately Be Easier Than Narrow Ones
“And part of the thesis of this company is that we believe that doing it at the full level of generality might actually in the long run be easier than trying to special case very specific narrow application domains.”
Levine: Multi-robot data will enable foundational physical models for rapid deployment
“So if we can draw on data from many sources, many applications, many robots, then we can have a model that has a physical understanding, and it'll be much, much easier to put new applications on top of that platform.”
Levine: Future Robots Won't Just Be Humanoid 'Metal People'
“And I think, you know, we, sometimes we think that, like, robots are going to be, like, one thing. Like, it's just like, you know, there's people, and now we're going to make, like, metal people, and that'll be, like, robots. But I don't think that's how it's …”
Levine: Robotic foundation models will adapt across diverse physical form factors
“And I think that in the future we'll have A robotic foundation model, which can then be adapted to all sorts of applications, and they might really run the gamut from like, you know, like bulldozers or something, to humanoids, to robotic arms like this thing, …”
Levine: Chain-of-thought reasoning allows robots to handle edge cases
“So the way you get common sense is by essentially using chain of thought. So the robot enters a scene and instead of directly starting to move, it thinks about what it was asked to do. So if it was told to clean up the kitchen, looks at the scene and says, lik…”