The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 29 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Finn: Generalist robotics models may outperform purpose-built models
“And we think that this sort of generalist model may work better and be easier to use than purpose-built models, just like we've seen in the development of foundation, foundation models for language and other applications.”
Finn: Scale is necessary but not sufficient for open-world robotics models
“And so I think the lesson here is that scale is necessary for developing these models that can generalize in open world conditions, but they're subordinate to actually solving the problem. So you need scale, but it's not sufficient for the entire problem.”
Finn: Diverse Home Training Matches Custom Target-Environment Performance
“And we find that if we actually increase the amount of homes, the amount of locations that are represented in the data, The performance increases, which is great. And it actually gets to the same level of performance as if we train on data from that target env…”
Finn: Frontier models struggle with visual understanding for robotics
“In general, we found that these frontier models generally struggle with visual understanding as it pertains to robotics. Which makes sense because in general, these models aren't kind of really targeting, ah, many physical applications and have very little dat…”
Chelsea Finn: Real robot data cannot be replaced by synthetic data
“I think that at the end of the day, there's going to be no replacement for real data. And so we're like large amounts of real robot data. It's going to be a necessary component of any like system that's going to work in a generalizable way.”
Finn: Physical Intelligence is building a general-purpose model for all robots
“And in particular, we're trying to develop a general purpose model that can enable any robot to do any task in any environment.”
Finn: Human video data is constrained by robot-human embodiment gaps
“Alternatively, maybe we look at data from YouTube, which has also a massive data source and many videos of humans doing tasks that can be useful for training robots. But at the same time, we don't learn how to write by watching other people write, and we don't…”
Finn: Pre-training and curated fine-tuning unlocked robotic laundry folding
“And this was actually to take some inspiration from the world of language modeling to actually instead of just training a policy on all of our data, can we pre-train on all the data? And then fine tune on a highly, on a curated, consistent, high quality set of…”
Finn: Physical Intelligence adapted its model to an unseen third-party robot
“We're also able to apply that same recipe to robots at other companies. This is a robot that I've actually never seen in person before. They collected data. They sent the data to us. We fine tuned our model on their data. We actually didn't even know exactly h…”
Finn: Full Pre-Training Mixture Boosts Robot Performance Over 20% in Novel Homes
“And we find that these kind of bars on the right, which are excluding data from static robots in labs and environments and so forth reduces performance significantly. So the performance goes down to less than 60% when you exclude that data when evaluated in no…”
Chelsea Finn: Synthetic data's robotic analog is RL, not simulation
“I think that the analog of synthetic data in language models is actually not necessarily simulation in robotics, but closer to something like reinforcement learning.”
Finn: Solving robotics applications traditionally requires building separate companies from scratch
“If you want to truly solve a robotics application, you essentially need to build an entire company around that application. Ah, you need to build a different company for logistics, for wet lab automation, For robots and kitchens, for surgical robots, and so on…”
Finn: Industrial automation data lacks behavioral diversity for general robotics
“So for example, we might look at data from industrial automation and you get tons and tons of data of robots doing tasks over and over again like this, but the sort of data isn't going to allow robots to go into disaster zones or to make a sandwich or to bag g…”
Finn: Current Robot Datasets Are Minuscule Compared to Future Scale
“I should mention this is large scale by today's robot standards and arguably a minuscule amount of data compared to the sorts of robot data that we should have in the years to come.”
Folding laundry is the most impressive physical robot feat Finn has seen
“And to date, I think this is the most impressive thing that I've seen A robot do in the physical world.”
Finn: Physical Intelligence uses PaliGemma 3B with flow matching diffusion head
“We took an open source vision language model, a three billion parameter model called polygema. Previously we were using, the previous videos were all with like a hundred to three hundred million parameters that we're iterating on. This model takes as input ima…”
Finn: Architectural fix boosted robot language following rate from 20% to 80%
“And second, it also followed language far better an 80% follow rate rather than a 20% follow rate which suggests that we're able to preserve the kind of pre-training in the vision language model backbone.”
Finn: Real-world human-robot interaction data is hard to scale
“It's going to be challenging to collect a large number of human robot interactions with the real robot in the loop. And this is also going to be fairly hard to scale.”
Finn: LLMs can generate synthetic prompts to relabel robot data
“We can use language models to relabel and generate hypothetical human prompts for the scenarios that the robots are in.”
Finn: Reinforcement Learning Outperforms Pure Imitation Learning in Robotics
“I think that reinforcement learning can play a very large role in it actually, in post-training. I think that online data from the robots which reinforcement learning allows you to use, Can allow robots to have a much higher success rate and also be faster tha…”
Finn: Physical Intelligence Has Not Struggled to Raise Capital
“We ourselves haven't had a lot of challenge with fundraising, and I think that a lot of robotics companies recently have also done a great job and found that there's actually a lot of excitement around this sort of technology, because I think things are actual…”
Finn: World models hallucinate success when evaluating suboptimal actions
“You might train it on demonstration data of successful data of completing the task, and then evaluate it on to try to actually use it to evaluate actions that are not optimally completing the task, and then the world model will hallucinate a video of completin…”
Finn: Retrieval systems struggle because models frequently ignore retrieved content
“So in my experience working on like retrieval based systems is that it actually is a little bit tricky to first figure out what should be offloaded versus actually done by the model. And second sometimes the model will ignore the retrieved content and try to g…”
Finn: Robot-side infrastructure is an underworked opportunity for builders
“There's some open source code for that sort of thing, but there's a lot of opportunities to make robot infrastructure better. And not a lot of people I think are working on that aspect of the problem.”
Chelsea Finn: Abundant resources can lead to wasteful compute usage
“Sometimes when you have a lot of resources, you don't actually think as carefully and as critically about what runs are going to be doing and so forth, and you end up being sometimes more wasteful of compute than if you were kind of more compute constrained.”
Finn: Data curation reduced five-item folding time to 12 minutes
“We selected and worked on our curation strategy for curating a higher quality set of demonstration data. We got it from 20 minutes down to 12 minutes for these five items.”
Physical Intelligence collected robot manipulation data across over 100 unique rooms
“And in total, we had more than a hundred unique rooms represented in the dataset.”