Becker: Overly Bullish AI Developer Speedup Estimates Are Inflated
Joel Becker · Measuring Exponential Trends Rising (in AI) — Joel Becker, METR · Feb 27, 2026 · at 20:08
Joel Becker of METR analyzes why developer self-reported productivity gains from AI tools often exceed measurable real-world economic value.
0:00 / 0:24exact quote · 24.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I do think that very bullish estimates of speed up today are, you know, to some extent inflated by what we document in that original paper, that people's expectations of speed up tend to be too optimistic, it seems. They also tend to be inflated, I think, by not quite grokking that the value of the additional tasks that they're able to complete, a lower value Than you might think, you know, that there's a reason that they weren't doing them previously.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Joel Becker
Becker: Operational Long Tail Will Delay Full AI R&D Automation
“There's this, Very long tail of things potentially involved in in R&D that would perhaps need to be fully automated in order to lead to capabilities explosion. I expect we're measuring, you know, in some ways only, only a small proportion of, only a small prop…”
Becker: Algorithmic Progress in AI Is Strictly a Function of Compute
“The suggestion in this paper is that if you think that algorithmic progress, you know, that, that is coming up with the transformer, coming up with RLHF, you know, MOEs, all of this stuff, better learning rate schedules is, is is itself a function of compute b…”
Becker: Halving AI Compute Growth Halves Algorithmic Progress and Milestones
“And both of them both of those components half when compute halves sort of trivially, because compute is halving, and algorithmic progress halves because compute is this important input, and compute halves, then you might expect time horizon growth to half. An…”
Becker: AI Scaffolding Value Does Not Persist Across Model Generations
“Within model generation, it's valuable, and across model generations, it's not so valuable.”