Physical Intelligence

“It turns out to be like pretty straightforward to go in and like find all those pauses and remove them. And you can speed things up further, so you can get a task where a person demonstrates what it means to succeed, and then you can have the robot practice th…”

“So if we can draw on data from many sources, many applications, many robots, then we can have a model that has a physical understanding, and it'll be much, much easier to put new applications on top of that platform.”

“And part of the thesis of this company is that we believe that doing it at the full level of generality might actually in the long run be easier than trying to special case very specific narrow application domains.”

“What was surprising is that we could also get these systems to perform very dexterous behaviors without really doing anything particularly special for that.”

“Multimodal language models are really good at pulling in knowledge and trying to articulate that knowledge. They're not very good at, like, grounding that knowledge in physical situations, but they know stuff.”

“And I think that in the future we'll have A robotic foundation model, which can then be adapted to all sorts of applications, and they might really run the gamut from like, you know, like bulldozers or something, to humanoids, to robotic arms like this thing, …”

“So, I think the big challenge, and this is kind of what I'm leaning up to, and what I hope to, ah, that we'll figure out here at Physical Intelligence is how to combine those threads. How to bring in all of that knowledge that you get with generative AI, but a…”

“So without, without, like, giving too much away what I can say is that a big focus for us right now is actually better understanding this kind of, like, mid-level reasoning part of the problem. Because we think that we have a pretty good sense for how to acqui…”

“Where we could get our models to work on all sorts of other robots, including robots with multi-fingered hands robots with different numbers of degrees of freedom, and obviously we needed to get data, and we needed to fine-tune the model, but the model itself …”

“So what that means is that the bottleneck had actually shifted from the lowest level, meaning the robot's ability to physically do the task, to this, like, middle level, where now the system is more bottlenecked by its ability to interpret the scene and select…”

“I think the place where I would see the biggest technical risk is dealing with the breadth of different situations. So, I think if we were talking about a well-defined, but, you know, But slightly chaotic environment like cleaning hotel rooms, or working, ah, …”

“So we had some demos that we released last April where we showed our robot cleaning kitchens, and like, you know, I think it's kind of cool, but if you watch an individual video out of context, it's just like, okay, it's like picking up plates, like anybody ca…”

“And I think increasingly what we'll see is a shift where domains where collecting data is straightforward They actually end up falling into the easy bucket over time, even if they are physically intricate. But there will be domains where collecting data is dif…”

“So, I think that the key is not so much to quantify, like, here is exactly the price tag of getting the ultimate robot data set. The key is to get a system that can go into the world that's useful enough That does a wide variety of different things and they ca…”

“So in my mind, the most important thing to get right is To get the system to be general, and in particular, to get it to be general with respect to how it can be improved, right? So, for example, hand-designed robotic controllers are not very general with resp…”

“And for example, we had this demo on, ah, making espresso. That system practiced making those espressos many, many times and used that to improve robustness, improve speed, improve throughput.”

“And we tried these things, and it actually turned out that we could solve almost all of them. We didn't get there's one we couldn't do, which was turning a dress shirt inside out, because the grippers on this thing wouldn't fit inside the sleeve, so we probabl…”

“I think changing a child's diaper will be really, really hard.”

“Well, that's not often the best assumption because you need the right kind of data. Like, Maybe some data is easy, like, it's easy to get, like, videos of people doing something, but that doesn't mean that's the right kind of data, and it might be domain depen…”

“We, we're so primed to interact with the physical world, so primed to have physical intelligence that you can use it in everyday speech by saying that company has a lot of momentum, and you can use it when advancing fundamental theoretical physics.”

“A good learning method can actually like compensate for deficient sensing fairly well.”

“Where there's a bootstrap challenge, like getting to a particular level of usefulness so that, ah, robots can be deployed so they can do useful tasks, so they can start collecting data from open world settings at scale, and because that's such a, like a sudden…”

“And I think that's like, there's something interesting there, because it suggests the power of generality, that when you have this kind of general system, you can really just, like, onboard all these crazy tasks without really doing anything particularly sophi…”

“So the way you get common sense is by essentially using chain of thought. So the robot enters a scene and instead of directly starting to move, it thinks about what it was asked to do. So if it was told to clean up the kitchen, looks at the scene and says, lik…”