Chen: Simulating robotic manipulation is harder than autonomous driving due to contact
Peter Chen · No Priors Ep. 48 | With Covariant CEO Peter Chen · Jan 24, 2024 · at 31:05
Covariant CEO Peter Chen discusses why simulation alone is insufficient for robotics manipulation compared to self-driving vehicles.
0:00 / 0:35exact quote · 35.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“When you think about simulation in self-driving car, like we are really mostly thinking about systems that hopefully don't physically interact with each other, right? Like if two cars get in contact with each other, that's a really terrible thing, right? And so the simulation there is more about simulation of avoidance, multi-agent behaviors, like avoidance of contact. But if you think about like manipulation. Like if you never contact something, that's also a big problem. Like, because like, then you actually don't do any work. And whenever you involve contact, simulation of those things become very, very difficult.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Peter Chen
Chen: Scaling robotics foundation models requires commercial robot fleets, not labs
“And the only way to collect enough data is to build fleets of robots that are actually creating value for customers that, so that you can collect those data in production. Because even if you try to scale up data collection in a lab environment, There's a limi…”
Chen: Robotics value does not require emergent scaling due to domain data coverage
“You kind of don't need that when you are in a more Restricted domain, like robotics because like you actually could have so much data coverage that your test scenarios are just part of your training scenario. So to some degree, like we actually don't need to r…”
Chen: Next-token LLM training is mimicry, imposing a natural capability ceiling
“For people that study reinforcement learning, we call it behavior cloning, which means you're just asking the AI to clone the behavior of another agent. And that is like one of the most primitive way possible to train this type of systems. Like, because if you…”
Chen: Humanoids are the universal hardware form factor for physical AI
“Humanoid is the universal hardware form factor that can be dropped into any place in our world.”
Peter Chen: All Covariant robots feed learning into a single foundation model
“All of these customers, all of these different robots are networked together. Like it's one single foundation model. And everything that they learn come back and make this central model better.”
Chen: AI scaling laws reliably reduce training loss in robotic manipulation
“The most technical definition of scaling law does apply and we have seen it apply in this domain. And it's somewhat not surprising because like if you think about like the scaling law in the most technical sense, which is if you scale up data and you scale up …”