Fedus: High-compute reinforcement learning is essential for AI tool use
Liam Fedus · Building an AI Physicist: ChatGPT Co-Creator’s Next Venture · Sep 30, 2025 · at 40:35
Liam Fedus, co-founder of Periodic Labs and ChatGPT co-creator, details why high-compute reinforcement learning is critical for creating sophisticated AI systems that effectively use tools.
0:00 / 0:12exact quote · 12.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“High compute reinforcement learning is really effective. This is how you should think about the strategies it's using. This is how you create effective tool using towards those problems, and this is how you optimize it effectively.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Liam Fedus
Fedus: Pre-training on domain data outperforms retrieval-augmented generation
“However, as we've seen with things like ChatGPT and other things, when you pre-train on the data, when you actually encode the knowledge into the weights, it's not just a retrieval system, you have a richer, deeper understanding of the material.”
Fedus: Physics and chemistry demonstrate scaling laws similar to AI
“On the material science side, we're seeing scaling laws within physics, within chemistry both with respect to simulations, with respect to experiment, and it's like the same kind of principles at play and ML.”
Fedus: Physics provides ideal verifiable reward functions for AI
“Physics is very verifiable. It's a great reward function, fairly fast iteration loop. You have simulators for large classes of physical systems.”
Fedus: Periodic Labs uses physical experiments as RL reward functions
“And what we're doing, and by having the lab, is we create a physically grounded reward function. That becomes the basis on which we're optimizing against. And so, If a simulator has some deficiencies or some issues, we always error correct, because for us, the…”
Fedus: Early ChatGPT was mathematically weak due to friendliness rewards
“The reward functions that we were using originally couldn't determine whether you were mathematically correct or not. So early versions of Chachapiti were mathematically not particularly strong, and it sort of results from the reward function. What did you opt…”
Fedus: AI physics requires generating new experimental data, not web scrapes
“The technology that we think is necessary to do it has really just emerged in the last couple of years, and this data Isn't like on a Reddit forum or something like you need to actually go produce experimental data, simulation data.”