White: RLHF fails on scientific hypotheses by ignoring impact and information gain
Andrew White · 🔬 From Red Teaming GPT-4 to Automating Drug Discovery: The Future of AI in Science — Andrew White · Jan 28, 2026 · at 20:35
Andrew White (Future House) discusses experiments using human feedback to train AI agents to generate scientific hypotheses.
0:00 / 0:30exact quote · 30.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We learned a lot about how bad our LHF is with people, just like people pay really attention to the tone, to the details, to like how many specific facts or figures on the hypothesis, right? Like actionability about like if the experiment is feasible, but what people didn't really pay attention to is like, I don't know how to describe this, but like if this hypothesis is true, how does it change the world? If the hypothesis is false, how does it change the world? This like how much information do you gain? It's not really information, but like impact or something. And that really didn't come through from those things.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Andrew White
White: Molecular dynamics and DFT are overrated in materials science
“So, I think molecular dynamics is overrated. And DFT is overrated. In fact, DFT may be even more overrated than like the dynamics.”
White: AI for Science Is Difficult in Academia and Demands Bigger Bets
“AI over science is just, I think, A, difficult to do in academia, and B, so exciting, but I think you can take bigger bets, and I think having a tenured position and writing research grants is maybe not the biggest bet you can take on, on a field.”
White: Existing LLMs Are Already Capable of Automating Much of Science
“We can actually automate so much of the scientific method, because it turns out, especially in a field like biology, which is very empirical limited, you know, the top one percent guesser of, you know, what they think will happen in experiment that, you know, …”
White: AI Science Agents Do Not Require Bespoke Automated Labs
“We don't actually have to hold their hands so much anymore, or like they don't actually need to necessarily have an automated lab. They can like write an email to a CRO or something, or they can like tell you what experiment to do. And you can take a video of …”
White: Simulating biological systems hits fundamental limits; empirical lab measurement is unavoidable
“Basically the best model whatever, Opus seven or GPT 10, like it really can only propose the first experiment, maybe slightly more clever, but at a certain point you just need information, right? Like some little calculations you can do that, like, there's mor…”
Logistics Data Matters More Than Frontier LLM Intelligence in Lab Science
“I think whether GPT, 5.2 codex max or Opus 4.5 is going to do better. It's probably doesn't matter. It's just a matter of like, which one's going to have all the information about what's in the lab and how much will it cost? How long will it take?”