Becker: Operational Long Tail Will Delay Full AI R&D Automation
Joel Becker · Measuring Exponential Trends Rising (in AI) — Joel Becker, METR · Feb 27, 2026 · at 31:52
Joel Becker of METR discusses why paper-writing and code benchmarks capture only a fraction of what is required to fully automate AI research and development.
0:00 / 0:24exact quote · 24.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“There's this, Very long tail of things potentially involved in in R&D that would perhaps need to be fully automated in order to lead to capabilities explosion. I expect we're measuring, you know, in some ways only, only a small proportion of, only a small proportion of those capabilities. And so I expect, you know, the capabilities needed for the full loop to close to come somewhat later.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Joel Becker
Becker: Overly Bullish AI Developer Speedup Estimates Are Inflated
“I do think that very bullish estimates of speed up today are, you know, to some extent inflated by what we document in that original paper, that people's expectations of speed up tend to be too optimistic, it seems. They also tend to be inflated, I think, by n…”
Becker: Algorithmic Progress in AI Is Strictly a Function of Compute
“The suggestion in this paper is that if you think that algorithmic progress, you know, that, that is coming up with the transformer, coming up with RLHF, you know, MOEs, all of this stuff, better learning rate schedules is, is is itself a function of compute b…”
Becker: Halving AI Compute Growth Halves Algorithmic Progress and Milestones
“And both of them both of those components half when compute halves sort of trivially, because compute is halving, and algorithmic progress halves because compute is this important input, and compute halves, then you might expect time horizon growth to half. An…”
Becker: AI Scaffolding Value Does Not Persist Across Model Generations
“Within model generation, it's valuable, and across model generations, it's not so valuable.”