ARC Prize

“Over the course of twenty-twenty-four, I believe the ArcPrize like, completion rate went from, I believe, 33% to about fifty-something percent, and a human can do, like, a hundred percent of these.”

“And I think arc shows that like, yeah, there actually are frontier problems. That are unsolved, that individual people and individual teams can actually make a difference on today.”

“We do think that efficiency is actually a really, really important aspect of intelligence. You know, you can brute force your way up to intelligence, but we really do want to be shooting for like human targets and efficiency for this stuff.”

“Opening eye situation was a little different because they had reached out to us and said, Hey, we think we've got a really impressive result on the public eval set. And we'd like your help to verify. On the semi-private set, which is what we created that data …”