Douglas: AI will use long-horizon reward signals within two years
Sholto Douglas · Weekly Recap - Elon Vs Trump, Ukraine's Drone Attack, Cluely Update & OpenAI CRO · Jun 7, 2025 · at 1:35:46
Anthropic researcher Sholto Douglas discusses moving beyond RLHF toward objective and long-horizon empirical reinforcement learning environments.
0:00 / 0:07exact quote · 7.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We've got very little rewards right now, but pretty quickly over the next year or two, you're going to start to see much more meaningful and long horizon rewards.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Sholto Douglas
Douglas: Generalist models will obsolete specialized fine-tuned models
“I really do think that similar to how we saw with large pre-trained models before with small fine-tuned models made it like, had gains over the sort of GPT-II era, but then were obsoleted by GPT-IV being generally good at everything. I think, to be honest, you…”