Physical Intelligence

“So we're trying to build a big neural network model that could ultimately control any robot to do anything in any scenario.”

“And in terms of the architecture, we're using transformers and we are using pre-trained models, pre-trained vision language models, and that allows you to leverage all of the rich information on the internet.”

“The last thing that I'll mention is that I think the biggest risk with this bet is that it won't work. Like, I'm not really worried about competitors. I'm more worried that no one will solve the problem.”

“So for that release that we had in late October last year, we collected data in three buildings, technically.”

“The other thing that I'll mention is actually right now we're most like, our policies right now do not have any memory. They only look at the current image frame. They can't remember even half a second prior.”

“Unfortunately, a lot of the tactile sensors that are out there Are either far less robust than skin, far more expensive or very, very low resolution.”

“If you need to do like a longer horizon task, meaning a task that might take minutes to do, then if you just train a single policy to like output actions based on images like if you're trying to make a sandwich and you train a policy that's just outputting the…”

“We've seen a lot of evidence that you could actually transfer a lot of rich information across these different embodiments and allows you to use data. And also if you iterate on your robot platform, you don't have to throw all your data away.”

“And we found that actually that mounting RGB cameras to the wrists ends up being very, very helpful and probably giving you a lot of the same information that tactile sensors can give you.”

“We've not only have we open-sourced some of the weights and released details in, in technical papers, we've actually also been working with hardware companies and giving designs of robots to hardware companies.”

“We had a research result a couple years ago where we showed that if you leverage vision language models, then you could actually get the robot to do tasks that require concepts that were never in the robot's training data, but were in the internet.”

“As one example, taking a robot off campus was like almost a non-starter just for code security reasons. And if you want to collect diverse data, taking robots off campus is, is valuable.”

“I feel like actually people underestimate how much intelligence goes into motor control. Many, many years of evolution is what led to us being able to use our hands the way that we do. And there are many animals that they can't do it even though they had so mu…”

“That's one of the things we're kind of optimizing for, and so we're using cheap robots. We're using robots that we can very easily develop teleoperation interfaces for, in which you can do teleoperation very quickly and collect diverse data, collect lots of da…”

“We actually found that we could take a checkpoint, send that model checkpoint to another lab halfway across the country, and the grad student at that lab could run the checkpoint on the robot, and it would actually More often than not do better than the model …”