Mike Merrill
Co-Creator, Terminal-Bench. On 1 show, 2 appearances. The Shows tab opens the full record on each.
Mike Merrill is the co-creator of Terminal-Bench, a coding agent benchmark. He developed Terminal-Bench 2.0 to evaluate frontier AI models on verified terminal tasks.
1shows
2appearances
13statements
3resolved
2supported
1contradicted
67%fully supported
Everything Mike Merrill said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Merrill: AI terminal interfaces will outpace GUI computer use development
“The bet here that we made was that GUI based computer use was going to be much slower to come into its own. Than terminal based use. And there's a few reasons for this. I mean, as I mentioned earlier, text is just the modality that works best with these models…”
Merrill: AI evals will shift to observing real jobs over 2-3 years
“I think like the future of evals does look much more like this is like observing people who are doing their real jobs and then translating those real jobs into a format that allows you to evaluate language models and harnesses on them. And it's probably going …”
Merrill: Frontier AI labs will center operations around vertical products
“And with the Claude codes and the codec CLIs and the deep researchers, researchers, you starting to see some evidence that the products are going to be a much more central part of how these frontier labs operate.”
Merrill: Terminus-2 harness yields highest agent performance on Terminal-Bench 2.0
“Terminus-II, which is our harness for measuring language models, is the highest performing agent for many of these models.”
Merrill: Text-based agents are currently the frontier of performance
“We like text-based agents. We think right now they're the frontier of agentic performance. This very well may change.”
Merrill: AI agents could ace Terminal-Bench with $1,000 compute per task
“Like, I think there's probably a way in which you spend a thousand dollars on tokens for each terminal bench task. And like get them all right just by scaling inference time compute.”
Merrill: Launching benchmarks is the best way to steer AI development
“And so, if you have an opinion about how AI should work, what's the best way to get other people to work on it? And that's by launching a benchmark.”
Merrill: Terminal-Bench is used by every frontier AI lab
“It's used by all Frontier Labs in some capacity.”
Merrill: Dario Amodei highlighted Terminal-Bench on the Claude model card
“I think one of the really key moments for us was getting onto the Claude IV model card. Being one of two benchmarks that Dario actually mentioned while releasing the model.”
Merrill: AI models now reliably solve Terminal-Bench's ML training task
“Unfortunately we are getting to the point where models do reliably get this one.”
Merrill: 60% to 70% of SWE-bench Verified tasks come from Django
“If you go look at sweet bench verified, I think like 60, 70% of the tasks in there are from Django.”
Terminal-Bench 2.0 tasks are based on real paid workflows
“Every task in the benchmark is based off of real work that people do out in the world and are paid for.”
Merrill: Terminal-Bench team spent 300+ hours auditing 89 benchmark tasks
“So I estimate we've spent at least 300 hours digging through each one of these 89 tasks, trying to figure out whether or not they're broken, whether or not they're some way to cheat, whether or not they're reproducible, and really obsessing over quality while …”
One line per show, most statements first. The link opens Mike's full record on that show: the calibration, argument clarity, speaking style and every statement made there.
| Show | Role there | Eps | Statements | Record | |
|---|---|---|---|---|---|
 LATENT SPACELEDGER | Co-Creator, Terminal-Bench | 2 | 13 | 67% 2/3 | full record on Latent Space → |