SWE-bench Pro
part of SWE Bench
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Feb 23, 2026 by Olivia Watkins · said 12 times in 3 episodes since 2025 · across every show →
Mentions by year
brought up most by John Yang (3), Mia Glaese (2)
tap a year for its mentions
2026 7 mentions in 1 episode
Everything said about SWE-bench Pro, oldest first
Feb 23, 2026 negative
Watkins: SWE-bench Verified is saturated, contaminated, and should be retired
“SweetBenchVerified has been one of the Northstar coding benchmarks that the field has looked at to measure coding progress. But recently we've seen that progress has kind of stalled and this, we realized that this is because the eval is effectively saturated a…”