Laskin: Humanity's Last Exam Benchmark Barely Matters to End Users
Misha Laskin · No Priors Ep. 123 | With ReflectionAI Co-Founder and CEO Misha Laskin · Jul 17, 2025 · at 15:53
Reflection AI co-founder and CEO Misha Laskin critiques standard frontier AI benchmarks when discussing specialized tool training.
0:00 / 0:16exact quote · 16.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Now, that's great, but I think the downside of that is that does humanity's last exam actually matter in any meaningful way for an end user? And I would argue that some weak correlation, but the answer is most likely no.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Misha Laskin
Laskin: Enterprise AI coding tool productivity impact is negligible or negative
“Within enterprises, when you know, they're adopting coding tools and you see the impact that this is having on their actual productivity. And I think it's much lower than people expect. So it's in fact, it's sometimes negative, sometimes negligible.”
Laskin: Teaching AI agents to take action is mostly solved
“To me, it seems like really, 20% of the problem is teaching these agents how to act, and it's more or less solved.”
Laskin: New frontier labs can succeed without cloud provider ownership
“Our thought was that this was the time where you can actually start a you know, a generational frontier lab that does not need to be coupled to a, you know, to a big cloud provider because if you do it right, you'll actually be able to generate you know, suffi…”
Laskin: Machine learning generalization is just bringing test distribution into training
“There's no such thing as generalization. There's just bringing the test distribution into train.”
Laskin: Scaling RL on LLMs is the final paradigm before ASI
“The next paradigm, and effectively the final paradigm that we need to have in place before a, you know, what people used to call AGI, or now I think the goalposts have shifted to ASI, is reached, is just figuring out how to scale reinforcement learning on top …”
Laskin: AI Apps Without Custom Model Training Are Fundamentally Limited
“The important part, I think, is to be able to tweak every part of the system from, you know, the product features to the agent design to the model training in order to build the best overall system. And if you are capped in which parts you can change, like if …”