Hendrycks: Saturating Humanity's Last Exam benchmark will signal superhuman STEM AI
Dan Hendrycks · No Priors Ep. 105 | With Director of the Center of AI Safety Dan Hendrycks · Mar 5, 2025 · at 32:09
Dan Hendrycks, director of the Center for AI Safety, explains the scope of his benchmark 'Humanity's Last Exam' and what near-ceiling performance will signify.
0:00 / 0:16exact quote · 16.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“When performance is near the ceiling, I think that'd basically be an indication that, like, you have something like a superhuman mathematician or a superhuman STEM scientist for, in many ways, for when they're, when closed-ended questions are very useful, such as in math.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Dan Hendrycks
Hendrycks: U.S. and China may launch preemptive cyberattacks on data centers
“I think that later on, it becomes so destabilizing that China just says, we're going to do something preemptive, like do a cyber attack on your data center. And the U.S. Might do that to China.”
Hendrycks: Extreme AI export controls make a Taiwan invasion more likely
“If you turn the pain dial all the way up for China in export controls and if AI chips are the currency of economic power in the future, then this increases the probability that they want to invade Taiwan.”
Hendrycks: Voluntary AI Pauses Without Enforcement Only Benefit Bad Actors
“If you do it voluntarily, you just make yourself less powerful and you let the worst actors get ahead of you. You could say, well, we'll try and try to sign a treaty. We will not assume that the treaty will be followed. Like that would be very imprudent. You w…”
Hendrycks: AI Safety Is a Geopolitical Problem, Not Primarily a Technical One
“Safety isn't, as I've been I'm trying to reinforce not really that much of a technical problem. This is more of a complex geopolitical problem with technical aspects.”
Hendrycks: China will steal model weights even if chip controls succeed
“Even so, I still think if you really tighten the export controls, you made it so that China can't get any of those chips at all, and this is your, one of your biggest priorities, they're just going to steal the weights anyway.”
Hendrycks: US-China race will force rapid, high-risk military AI integration
“China can have AIs that are totally aligned with them. The U S can have AIs that are totally aligned with them. You still are going to have a strategic competition between the two. This is going to they're going to need to integrate it in their militaries. The…”