Robotics Foundation Models

“What this looks like more concretely, I would say, is, like, we just train a model on hundreds of thousands or millions of hours of data of Every single type of possible physical interaction in the world we can think of, and then we now have a model that we ca…”

“Another one is the ability to have some, like, ingenuity around how to use tools in a way that was also not trained for the task. So basically we can ask the robot to do a certain type of task where we've only trained it with one type of tool. We can give it a…”

“We think that it's going to be more like a peeling an audience analogy, where you start from a really strong base model that have all sorts of common sense knowledge and already works to some extent on your robot, and you have then a Mixed autonomy system. Ver…”

“And I personally don't see any reason why we couldn't have this, these be this, the same model.”

“I think that data can have a lot of value, but I think that by itself, it won't get you very far and I think that there's actually some really nice analogies you can make where for example, if you watch, like, an Olympic swimmer, swimmer race even if you had t…”

“And then I also think that autonomous experience will play a huge role, just like we've seen in language models. After you get an initial language model, if you can use reinforcement learning to have the robot, the language model bootstrap on its own experienc…”

“There's really no, very, no precise grounding. And there's no precise understanding of the physical world that's naturally occurring on the internet. So that's, like, one of the first thing that you'll find Kind of the departure of robotics foundation models f…”