Jan 24, 2024 · 40m · no-priors
No Priors Ep. 48 | With Covariant CEO Peter Chen
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of No Priors, Covariant co-founder and CEO Peter Chen joins Sarah Guo to discuss how embodied foundation models, real-world physical data flywheels, and reinforcement learning are revolutionizing robotic manipulation and industrial automation.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 14.7% of the talking time here. How this is scored →
The hosts as informed peer 3.9 Guest teaching 4.0 Guest disagreement 0.4 The hosts pushing back 0.8
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 25:54 Peter reframes scaling law definitions
Peter politely corrects the premise of Sarah's question, distinguishing between strict technical loss-reduction scaling laws and popular notions of emergent general intelligence.
Hardest push from the hosts ▶ 30:41 Sarah challenges simulation insufficiency using TeslaSarah directly questions Peter's real-world data thesis by pointing out Tesla's successful use of high-quality simulation for autonomous training.
Biggest teaching moment ▶ 22:55 Peter educates on physical grounding vs internet videoPeter breaks down why YouTube videos and multimodal LLMs fail at physical embodiment, citing their lack of sub-millimeter precision and action-force feedback loops.
The host holds their own ▶ 5:39 Sarah analyzes autonomy deployment models from venture portfolioSarah demonstrates deep sector expertise by comparing Covariant's data flywheel to autonomy architectures she backed in companies like Aurora, Nuro, and Kodiak.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Founding Covariant and the Physical Data Flywheel | 6 | 2 | 0 | 2 | Sarah demonstrates strong domain knowledge by drawing parallels to her early autonomous vehicle investments in Aurora, Nuro, and Kodiak, comparing their standalone brain strategy to Tesla's iterative data flywheel. Peter agrees with her framing and elaborates on Covariant's commercial product deployment strategy. | |
| Overcoming the Limitations of Rigid Industrial Robotics | 3 | 4 | 0 | 0 | Sarah prompts Peter to lay out the baseline landscape and operational limits of contemporary warehouse robotics. Peter provides an educational overview contrasting rigid repetitive automation with adaptive e-commerce fulfillment needs. | |
| Demystifying Put Walls and Manipulation Complexity | 4 | 4 | 1 | 1 | Sarah asks Peter to explain physical put walls and probes whether grasping is fundamentally harder than sorting or identification. Peter clarifies that while grasping is complex, modern AI also fundamentally reimagines barcode identification and routing. | |
| Scaling the Covariant Brain and Hardware Generalization | 3 | 2 | 0 | 0 | Sarah inquires about Covariant's roadmap regarding new tasks, humanoid form factors, and enterprise footprint. Peter details their unified foundation model approach across diverse warehouse domains. | |
| Physical Grounding and the Limits of Internet Data | 4 | 5 | 1 | 1 | Sarah asks what physical foundation models lack that cannot be learned from internet videos and images. Peter explains that video data lacks sub-millimeter precision and action-feedback force dynamics required for manipulation. | |
| Scaling Laws and Domain-Specific Foundation Models | 4 | 6 | 1 | 1 | Sarah asks whether scaling laws predict robotics emergence. Peter reframes her question by distinguishing formal loss-function scaling laws from broad emergent reasoning, noting domain-specific models rely on bounded data coverage rather than out-of-domain extrapolation. | |
| Real-World Data Primacy vs. Simulation Constraints | 5 | 5 | 1 | 2 | Sarah pushes Peter on Covariant's core scientific bet and brings up Tesla's heavy reliance on high-fidelity simulation. Peter explains why contact dynamics and 100,000 distinct warehouse SKUs make pure simulation inadequate compared to real-world data collection. | |
| Defining the ChatGPT Moment for Industrial Robotics | 3 | 4 | 0 | 0 | Sarah asks what a ChatGPT moment entails for robotics and whether future facilities will be fully lights-out. Peter points out that physical robotics requires near-total reliability over generative creativity and predicts human-in-the-loop co-piloting fleets. | |
| Consumer Robotics Feasibility and Operational Safety | 3 | 4 | 0 | 0 | Sarah asks about near-term consumer robotics and physical AI safety. Peter explains why non-manipulation consumer devices arrive first and how industrial settings bound alignment risks using physical safety cages and certified controllers. |
Statements from this episode (16)
Chen: Robotics will propel AI forward through grounded, embodied physical data
“And so we found robotics to be such a great way to both utilize the advances in AI, but also we think of it as a way to also propel AI forward. Like this is where you get the grounded data. This is where you get that embodied data of not just AI that is traine…”
Chen: Scaling robotics foundation models requires commercial robot fleets, not labs
“And the only way to collect enough data is to build fleets of robots that are actually creating value for customers that, so that you can collect those data in production. Because even if you try to scale up data collection in a lab environment, There's a limi…”
Chen: Over 99% of deployed robots worldwide are rigidly pre-programmed
“99 plus percent of the robots that are deployed in the world,
Are dumb robots. Like these robots are pre-programmed to do the same thing again and again, and they don't really have any kinds of intelligence that can adapt to new circumstances, communicate with…”
Chen: Covariant's average warehouse customer has over 100% annual turnover
“An average warehouse that we serve have typically more than a hundred percent year over year turnover rate.”
Chen: Robotic grasping requires far more AI than routing and identification
“I would say identification and routings are typically more considered a more solved problem than grasping. Like, because if you, there are other, like, more mechanical way to solve those problems. Like, you can design a piece of conveyor that, like, if you alw…”
Chen: Humanoids are the universal hardware form factor for physical AI
“Humanoid is the universal hardware form factor that can be dropped into any place in our world.”
Peter Chen: All Covariant robots feed learning into a single foundation model
“All of these customers, all of these different robots are networked together. Like it's one single foundation model. And everything that they learn come back and make this central model better.”
Chen: Robotics foundation models diverge from multimodal models on precision
“There's really no, very, no precise grounding. And there's no precise understanding of the physical world that's naturally occurring on the internet. So that's, like, one of the first thing that you'll find Kind of the departure of robotics foundation models f…”
Chen: AI scaling laws reliably reduce training loss in robotic manipulation
“The most technical definition of scaling law does apply and we have seen it apply in this domain. And it's somewhat not surprising because like if you think about like the scaling law in the most technical sense, which is if you scale up data and you scale up …”
Chen: Robotics value does not require emergent scaling due to domain data coverage
“You kind of don't need that when you are in a more Restricted domain, like robotics because like you actually could have so much data coverage that your test scenarios are just part of your training scenario. So to some degree, like we actually don't need to r…”
Chen: Simulating robotic manipulation is harder than autonomous driving due to contact
“When you think about simulation in self-driving car, like we are really mostly thinking about systems that hopefully don't physically interact with each other, right? Like if two cars get in contact with each other, that's a really terrible thing, right? And s…”
Chen: The Bar for a Robotics ChatGPT Moment Is Higher Than Pure Software
“So I would say, like, the bar for the chat GPT moment for robotics is higher. Like, you need to solve the generality, like, which is the same kind of problem, but you need to solve it with high level of reliability.”
Chen: The robotics ChatGPT moment will happen in industrial settings before consumer
“And before humanoids are fully widespread I think we will see that the chat GBT of a moment for robotics being articulated in the industrial settings earlier than in the commercial settings, like, because those are the places that can actually justify the hard…”
Chen: Warehouses won't be lights-out; workers will oversee 30 robots each
“I don't think it would be fully lights out and no human, at least in the near future, but I think of it as would be very robotics augmented. Like, so think of one person would be able to oversee 10, 20, 30 robots.”
Chen: First consumer robots will focus on navigation rather than physical manipulation
“If I have to guess, it probably would be a home robot that don't involve much manipulation. So think of it as like a home robot that might be like a Roomba. It can follow you around, like you can talk to it. So like it has that navigation of movement aspects o…”
Chen: Next-token LLM training is mimicry, imposing a natural capability ceiling
“For people that study reinforcement learning, we call it behavior cloning, which means you're just asking the AI to clone the behavior of another agent. And that is like one of the most primitive way possible to train this type of systems. Like, because if you…”