White: Writing bulletproof RL verifiers is far harder than supervised training
Andrew White · 🔬 From Red Teaming GPT-4 to Automating Drug Discovery: The Future of AI in Science — Andrew White · Jan 28, 2026 · at 1:12:03
AI researcher Andrew White discusses the EtherZero project, which applied reinforcement learning and verifiable rewards to chemistry and molecular design.
0:00 / 0:26exact quote · 26.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Pre-training or training transformers, you know on just data, like just supervised training where you just have the inputs and the outputs directly, very nice, relaxing, you know, like things are always robust, you know, things go pretty smoothly. When we do these verifiable rewards where you have to like write a bulletproof verifier, it is really difficult. And we had so many models trained only to find out they were hacking some other like random thing in our setup. It's really hard.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Andrew White
White: Molecular dynamics and DFT are overrated in materials science
“So, I think molecular dynamics is overrated. And DFT is overrated. In fact, DFT may be even more overrated than like the dynamics.”
White: AI for Science Is Difficult in Academia and Demands Bigger Bets
“AI over science is just, I think, A, difficult to do in academia, and B, so exciting, but I think you can take bigger bets, and I think having a tenured position and writing research grants is maybe not the biggest bet you can take on, on a field.”
White: Existing LLMs Are Already Capable of Automating Much of Science
“We can actually automate so much of the scientific method, because it turns out, especially in a field like biology, which is very empirical limited, you know, the top one percent guesser of, you know, what they think will happen in experiment that, you know, …”
White: AI Science Agents Do Not Require Bespoke Automated Labs
“We don't actually have to hold their hands so much anymore, or like they don't actually need to necessarily have an automated lab. They can like write an email to a CRO or something, or they can like tell you what experiment to do. And you can take a video of …”
White: Simulating biological systems hits fundamental limits; empirical lab measurement is unavoidable
“Basically the best model whatever, Opus seven or GPT 10, like it really can only propose the first experiment, maybe slightly more clever, but at a certain point you just need information, right? Like some little calculations you can do that, like, there's mor…”
Logistics Data Matters More Than Frontier LLM Intelligence in Lab Science
“I think whether GPT, 5.2 codex max or Opus 4.5 is going to do better. It's probably doesn't matter. It's just a matter of like, which one's going to have all the information about what's in the lab and how much will it cost? How long will it take?”