SWE-bench Pro, every mention
5 scenes · ← back to SWE-bench Pro
tap a year for its mentions
every year anyone John Yang 3Mia Glaese 2
Verbatim, from the transcripts: the passages where SWE-bench Pro comes up
The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals
- ▶ 1:33 unnamed speaker Like SweetBenchPro. 2 times in the scene
- ▶ 10:48 unnamed speaker And then, uh, CBench Pro will, will be some of the next one, which is an effort from scale. 3 times in the scene
- ▶ 24:24 Mia Glaese Like Sweebench Pro, we're like, yes, but that's a better eval now. 2 times in the scene
[State of Code Evals] After SWE-bench, Code Clash & SOTA Coding Benchmarks recap — John Yang
Terminal-Bench: Pushing Claude Code, OpenAI Codex, Factory Droid, et al to the limits
- ▶ 16:12 unnamed speaker You don't have sweet bench pro like are all of those things. 2 times in the scene