Becker: Time Horizon Metric Measures AI Task Difficulty in Human Time
Joel Becker · Measuring Exponential Trends Rising (in AI) — Joel Becker, METR · Feb 27, 2026 · at 11:59
Joel Becker of METR discusses how AI evaluation benchmarks work on Latent Space, clarifying a common industry misunderstanding regarding model execution duration versus task difficulty.
0:00 / 0:05exact quote · 5.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“You know, instead we're just plotting what's the difficulty of tasks they can do over time, and that difficulty is measured in human time.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Joel Becker
Becker: Overly Bullish AI Developer Speedup Estimates Are Inflated
“I do think that very bullish estimates of speed up today are, you know, to some extent inflated by what we document in that original paper, that people's expectations of speed up tend to be too optimistic, it seems. They also tend to be inflated, I think, by n…”
Becker: Operational Long Tail Will Delay Full AI R&D Automation
“There's this, Very long tail of things potentially involved in in R&D that would perhaps need to be fully automated in order to lead to capabilities explosion. I expect we're measuring, you know, in some ways only, only a small proportion of, only a small prop…”
Becker: Algorithmic Progress in AI Is Strictly a Function of Compute
“The suggestion in this paper is that if you think that algorithmic progress, you know, that, that is coming up with the transformer, coming up with RLHF, you know, MOEs, all of this stuff, better learning rate schedules is, is is itself a function of compute b…”
Becker: Halving AI Compute Growth Halves Algorithmic Progress and Milestones
“And both of them both of those components half when compute halves sort of trivially, because compute is halving, and algorithmic progress halves because compute is this important input, and compute halves, then you might expect time horizon growth to half. An…”