StarZero All shows Try the podcast clipper →

SWE-bench

includes SWE Bench Verified

3 statements across 2 episodes · 1 bullish · 0 bearish · 2 people on the record · first statement Feb 6, 2025 by Amjad Masad · said 3 times in 1 episodes since 2025 · across every show →

Mentions by year, the whole family

brought up most by Amjad Masad (3)

tap a year for its mentions
0021312025episodesmentions
0112025episodes it came up in
001.50.5312025episodesmentions per episode
2025 3 mentions in 1 episode

every mention, scene by scene, with the transcript →

Everything said about SWE-bench, oldest first

Feb 6, 2025 neutral
Disclosure
Masad: Replit does not compete on standard SWE-bench benchmark
“There's a, there's this benchmark called SWE bench. So software engineering bench. And that's what everyone's competing on. We don't really compete on that. And the reason is because we are building for a different audience.”
Amjad Masad Feb 6, 2025 ▶ 59:04 The AI Coding Agent Revolution, The Future of Software, Techno-Optimism | Amjad Masad, CEO, Replit
Oct 2, 2025
Assertion Partly supported
Douglas: The entire AI industry scored under 20% on SWE-bench last year
“As recently as a year ago, I think we were under 20% or something like that as a field.”
Sholto Douglas Oct 2, 2025 ▶ 33:05 Sonnet 4.5 & the AI Plateau Myth — Sholto Douglas (Anthropic)
Oct 2, 2025 positive
Assertion Supported
Douglas: Sonnet 4.5 pushed SWE-bench scores from roughly 72% to 78%
“We moved recently from roughly 72 to roughly 78 in Sweepbench”
Sholto Douglas Oct 2, 2025 ▶ 32:58 Sonnet 4.5 & the AI Plateau Myth — Sholto Douglas (Anthropic)
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more