Oct 23, 2025 · 1h 10m · mad
Are We Misreading the AI Exponential? Julian Schrittwieser on Move 37 & Scaling RL (Anthropic)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of The MAD Podcast, host Matt Turck interviews Anthropic researcher Julian Schrittwieser about the exponential growth trajectory of frontier AI, the evolution of reinforcement learning from AlphaGo to LLM agents, and the technical and societal implications of autonomous AI systems.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 25.5% of the talking time here. How this is scored →
Matt as informed peer 3.9 Guest teaching 3.8 Guest disagreement 0.3 Matt pushing back 1.2
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 21:26 Pushing back on Richard Sutton's RL-from-scratch stance
Julian explicitly rejects the premise put forward by AI pioneer Richard Sutton, arguing that abandoning pre-training in favor of pure from-scratch RL is practically inefficient and bad for safety alignment.
Hardest push from Matt ▶ 7:44 Challenging benchmark performance vs real-world messinessMatt directly challenges the optimism around capability benchmarks by asking how well GDPval or Meter predict actual economic utility once messy data, compliance, liability, and friction are factored in.
Biggest teaching moment ▶ 48:39 Explaining why pre-trained text cannot create effective agentsJulian clearly articulates why raw base models make poor agents, explaining that static pre-training corpora contain static knowledge rather than interactive action loops or self-correction patterns.
Matt holds his own ▶ 20:55 Citing Richard Sutton's recent podcast argument on pure RLMatt displays deep context and industry familiarity by bringing up Richard Sutton's recent argument on Dwarkesh Patel's podcast regarding whether future models should replace pre-training entirely with RL.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| The Viral Blog Post: Failing to Understand the Exponential | 2 | 3 | 1 | 1 | Matt opens by referencing Julian's viral blog post on exponential growth in AI. Julian elaborates on why people struggle to extrapolate exponential progress, drawing parallels to early COVID-19 perception. | |
| Unpacking Predictions for 2026–2027 AI Capabilities | 3 | 2 | 0 | 0 | Julian breaks down baseline extrapolations for AI autonomous task length into 2026 and 2027. Matt accurately summarizes task length and time-to-complete as core evaluation metrics. | |
| Evaluating AI Progress: GDPval vs. Real-World Friction | 5 | 3 | 1 | 4 | Matt offers a sharp pushback question, asking how benchmark metrics account for real-world friction like compliance and messy data. Julian explains how duration benchmarks naturally force models to deal with complexity. | |
| Can AI Be Creative? Move 37 and LLM Novelty | 3 | 4 | 1 | 1 | Matt asks about AlphaGo's Move 37 and whether LLMs can demonstrate true novelty. Julian details Move 37 context and explains that LLM novelty is easy, but making novel outputs useful is the real challenge. | |
| AI in Scientific Discovery and the Timeline to a Nobel Prize | 5 | 3 | 0 | 1 | Matt cites specific DeepMind and biomedical research breakthroughs (AlphaCode, AlphaTensor). Julian outlines a timeline leading to potential Nobel-prize-tier AI scientific discoveries by 2027. | |
| The Singularity Debate: Discontinuity vs. Scientific Scaling | 4 | 5 | 1 | 2 | Matt questions whether self-improving AI leads to a technological singularity or faces diminishing returns. Julian reframes the singularity concept using pharmacology and scientific scaling laws, arguing against a sudden discontinuity. | |
| Is Pre-Training + RL Sufficient to Achieve AGI? | 6 | 4 | 2 | 3 | Matt demonstrates high domain familiarity by citing Richard Sutton's podcast appearance arguing for pure RL from scratch. Julian explicitly rejects Sutton's premise, defending the practical and alignment benefits of pre-training. | |
| Julian Schrittwieser's Personal Journey: From Austria to DeepMind | 1 | 1 | 0 | 0 | Matt invites Julian to share his personal career trajectory. Julian discusses growing up in Austria, building game engines, interning at Google, and deciding to join DeepMind after seeing Demis Hassabis speak. | |
| The Evolution of Reinforcement Learning: AlphaGo to MuZero | 5 | 5 | 0 | 1 | Matt guides the chronology of AlphaGo, AlphaGo Zero, AlphaZero, and MuZero, while clarifying technical concepts like tree search versus database search. Julian explains the step-by-step removal of human priors. | |
| World Models and Translating Game RL to LLM Agents | 4 | 4 | 0 | 1 | Matt asks how game RL translated into LLM agents and modern world models. Julian explains why MuZero learned an environment predictor to handle tasks without explicit simulators. | |
| Implicit World Models in Language Models vs. MuZero | 4 | 4 | 0 | 1 | Matt probes implicit versus explicit world models. Julian explains that language models naturally build implicit world representations to predict future text tokens without reconstructing full high-dimensional state space. | |
| Why Combining Pre-training and RL Took Time | 4 | 5 | 0 | 2 | Matt asks why combining pre-training and RL took so long industry-wide. Julian educates on the engineering difficulty of RL feedback loops compared to the stability of supervised pre-training targets. | |
| Scaling Laws and Compute Allocation in Reinforcement Learning | 4 | 4 | 0 | 1 | Matt asks about compute allocation and scaling laws in RL. Julian notes that RL scales predictably with compute and discusses flexible reward signals like Constitutional AI. | |
| Training Data for Modern RL and Model-Generated Data | 4 | 5 | 0 | 1 | Matt asks whether quality, quantity, or recency matters most for RL data. Julian contrasts the high stability of AlphaZero planning search data with the stability challenges of direct LLM RL sampling. | |
| The Intersection of Reinforcement Learning and Agentic AI | 3 | 5 | 0 | 0 | Matt asks Julian to explain the connection between RL and agentic AI for a general tech audience. Julian highlights how pre-training text lacks action trajectories and error recovery, which RL provides. | |
| Building Agentic Applications: Frontier Models vs. Custom RL | 4 | 4 | 0 | 1 | Matt asks whether app developers should train custom RL models or rely on frontier APIs. Julian advises relying on base frontier models while investing effort into tool harnesses and execution context. | |
| Goodhart's Law and the Challenges of Public AI Benchmarks | 4 | 3 | 0 | 1 | Matt introduces Goodhart's Law and leaderboard gaming. Julian confirms the benchmark optimization phenomenon and recommends held-out internal benchmarks to avoid gaming metrics. | |
| Internal Evaluation Practices and Evaluating Complex Model Capabilities | 3 | 4 | 0 | 1 | Matt inquires about internal evaluation practices at DeepMind and Anthropic. Julian points out the difficulty of building evals that are simultaneously cheap, reliable, and representative of complex work. | |
| Mechanistic Interpretability and the Golden Gate Claude Feature | 5 | 4 | 0 | 1 | Matt asks about mechanistic interpretability and Anthropic's alignment commitments. Julian highlights the Golden Gate Claude feature discovery and explains how interpretability validates model alignment. | |
| Comparative Advantage and the Economic Impact of AI on Jobs | 4 | 4 | 0 | 1 | Matt asks about economic impact on employment and inequality. Julian discusses comparative advantage, noting AI elevates productive capacity rather than executing flat human replacement. |
Statements from this episode (27)
AI autonomous task duration doubles every three to four months
“We are seeing this very consistent improvement over many, many years where every say like, you know, three, four months is able to like do a task that is twice as long as before completely on its own.”
Top AI models will work autonomously for full days within two years
“In a year from now, maybe two years from now, it's the top models are going to be able to work completely on their own for like a whole day or more”
Valuations for OpenAI, Anthropic, and Google are fairly conservative
“If you look at OpenAI, if you look at Anthropic, if you look at Google, those evaluations, those revenue numbers are actually fairly conservative.”
Schrittwieser: Wider AI ecosystem may face bubble while frontier labs thrive
“There may simultaneously be like some sort of bubble in, you know, the wider ecosystem, while at the same time, the frontier labs on a very solid trajectory, having a lot of revenue, making a lot of money.”
Schrittwieser: Task duration dictates how much work can be delegated to AI
“The reason I think why task length specifically is interesting is because that's What allows you to delegate more and more work to language models, to agents. Now, even if you have a very clever model, but if it needs feedback or the interaction with you very …”
Schrittwieser: OpenAI's GDPval is a strong benchmark for economic impact
“I think that GDP is like a super cool evaluation from OpenAI where they collected a lot of, like, you know, real world tasks from real domain experts to make sure that it is actually representative of what you might do in the economy.”
Schrittwieser: AI language models are clearly capable of novel output
“For me, as somebody who has been doing research a long time, I think it's pretty clear that these models can do novel things.”
AI will make Nobel Prize-level scientific discoveries by 2027 or 2028
“I think my guess for that level of capability might be maybe 2027. I think we're probably not going to find out for quite some time afterwards because of the delay in getting prices. But I think by 20, 27, 20, 28, I think extremely likely that the models will …”
A sudden AI singularity or intelligence explosion is extremely unlikely
“Yeah, I think a true discontinuity is extremely unlikely from, you know, obviously AI researchers are already using AI to accelerate themselves. And so what's, what's already happening and like what is likely to continue to happening is that we see like a smoo…”
Schrittwieser: Current AI paradigm likely to achieve human-level performance in productivity tasks
“I think if you're thinking of, oh, we want some kind of system that can perform at roughly human level in basically all tasks that we care about. Productivity wise. Then I think, yeah, it's extremely likely that the current approach, pre-training RL, you know,…”
Future AI models will continue to rely on pre-training data
“Personally, I think that's unlikely. Not, not because pre-training is strictly necessary. I think we may well be able to train something completely from scratch, as we've been able to do in other domains, but more because pre-training on this vast data sets th…”
Pre-training aids AI alignment by implicitly instilling human values
“I definitely think we would keep using pre-training data, not just from an efficiency point of view as well, but also I think there is interesting safety angles, because by pre-training and, you know, all this human knowledge, we're implicitly creating an agen…”
AlphaGo would have probably lost to Lee Sedol if played earlier
“And I think if we had done it a few months earlier, we would have probably lost.”
Schrittwieser: Language models possess an implicit world model
“So I think, yes, I would say that language models have an, not an explicit world model, but they do have an implicit model of the world.”
Schrittwieser: AI pre-training risks over-restricting an agent's exploration search space
“I think the main, you know, the main challenge or the main thing you need to watch out for is that you don't over encode or you don't restrict your search space too much. If your pre-training, if your prior knowledge prevents you from exploring something that …”
Reinforcement learning scaling yields returns on compute similar to pre-training
“If you look at all the RL literature over time, we see very similar returns on compute in pre-training and in RL, where we can invest exponentially more compute in RL and keep getting benefits.”
Schrittwieser: Reliable reward sources will be key to scaling reinforcement learning
“Figuring out what are the best reward sources, and how do we scale it up, and how do we, you know, get more rewards, more reliable rewards. That will be one of the key ingredients in scaling up RL further.”
AI research still lacks scaling laws for training data quality
“I think we don't have any good scaling laws yet. That tell us the trade off, especially I think because it's very hard to measure what is the quality of a data point, right? Like how good is this example compared to this other example without being able to mea…”
Schrittwieser: Adding model reasoning improves RL training stability and scaling
“One direction of scaling RL and making it more stable is by improving this by, for example, putting more reasoning into your language model to generate much more high quality training data. That can then give us training that is much more stable, and then we c…”
Schrittwieser: Raw pre-trained AI models make poor agents without RL
“Our pre-training data is not very agent-like. If you think of the pre-training data, right, there is like websites and books and, you know, all kinds of recent text that has a lot of information, but it doesn't have a lot of actions. It doesn't really capture …”
AI app developers do not need custom fine-tuning for top models
“I think nowadays, with the capabilities of, like, you know, top, probably cloud models, top OpenAI, GPT models, You don't need to do any fine tuning. You can take the model as is, ride your own tools, your own harness, and benefit from that agentic training. B…”
Schrittwieser: AI progress will remain smooth and incremental without a single bottleneck
“There's probably, yeah, not one individual blocker. And that's why we will continue to see sort of smooth incremental progress over model releases”
Internal benchmarks are the most accurate way to select AI models
“Just make your own internal benchmark that really represents what you care about, and then measure on that. And I think that's likely to be the most objective, most accurate way of measuring.”
Using chain-of-thought as an RL reward destroys model interpretability
“If you're not careful with RL, you can make interpretability harder. For example, one Common thing with modern models is they do reasoning with the chain of thought. You could look at the chain of thoughts to, you know, see what are the model internal thoughts…”
Schrittwieser: AI safety must span the entire stack, not just RL
“Yeah, I wouldn't view it alignment adjust like an RL problem. I think it sort of, it goes throughout the whole stack. You might, you know, for example, filter the pre-training data in some way. You might, after training, you might have classifiers that, you kn…”
Schrittwieser: Distributing AI productivity gains is a political problem, not technological
“I think it's much more like a political, social problem of, like, figuring out how do we actually benefit from all these improvements, and, like, you know, bring the increases in wealth and productivity to everybody, and it's much less a technological problem.…”