Hendrycks: Loss-of-Control Risk Stems from Competitive Pressure to Fully Automate AI R&D
Dan Hendrycks · AI's Rising Risks: Hacking, Virology, Loss of Control — With Dan Hendrycks · Mar 28, 2025 · at 7:25
Dan Hendrycks discusses structural and competitive dynamics that could lead to loss of control over autonomous AI systems.
0:00 / 0:41exact quote · 41.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Loss of control risks, which I think primarily stem from people, an AI company trying to automate all of AI research and development, and they can't have humans check in on that process because that would slow them down too much. If you have a human do a week review every month of what's been going on and trying to interpret what's happening, this is this would slow them down substantially, and the competitor that doesn't do that will end up getting ahead. What that would mean is that you'd have basically AI development going very rapidly where there's nobody really checking what's going on or hardly checking. And I think a loss of control in that scenario is, is more likely.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Dan Hendrycks
Hendrycks: Superintelligence race will incentivize preemptive sabotage of rival AI projects
“I think that form of competition would be very dangerous, and because there's a risk of loss of control, and because it might incentivize states to engage in preventive sabotage or preemptive sabotage to disable these sorts of projects.”
Hendrycks: Russia and China may threaten US data centers over superintelligence progress
“If the US were pulling ahead both Russia and China may have a substantial interest in saying, hey, cut this out. Pulling ahead to develop super intelligence, which could give it a huge advantage and an ability to crush crush them. They'd say, you don't get to …”
Hendrycks: Automated AI R&D could compress a decade of progress into one year
“If you could have one AI do that, then you could make, you know, a 100,000 copies of these and have them perform research simultaneously. So that could lead to some very substantial acceleration in the rate of development. You might get a decade's worth of AI …”
Hendrycks: Fast-moving automated AI R&D loops are nearly impossible to de-risk
“Meanwhile, some automated AI research and development loop that goes extremely quickly with very little human oversight, that seems hard to de-risk and get the risks to be at a negligible level, just because it's so fast and so ah, and there's so little human …”
Hendrycks: AI poses no immediate extinction risk because it cannot make PowerPoints
“So, I don't think AI poses a risk of extinction like today, ok? I don't think that they're powerful enough to do that. They, because they can't make PowerPoints yet, right? They don't have Agential skills. They can't accomplish tasks that require many hours to…”
Hendrycks: Cyberattacks and bioweapons are top malicious AI risks within two years
“In the shorter term when AIs get more agential, I'd be concerned about AIs causing cyber attacks on critical infrastructure, possibly by as directed by a rogue actor. There'd also be the risk of AIs facilitating the development of bioweapons in particular pand…”