SWE-bench
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Mar 13, 2025 by Thomas Dohmke · said 7 times in 6 episodes since 2024 · across every show →
Mentions by year
brought up most by Zach Lloyd (2), Sarah Guo (2), Thomas Dohmke (1), Brendan Foody (1), Aaron Levie (1)
tap a year for its mentions
2025 5 mentions in 4 episodes 1 per episode
2024 2 mentions in 2 episodes 1 per episode
Everything said about SWE-bench, oldest first
Mar 13, 2025 bullish
Dohmke: Improved model reasoning will push SWE-bench scores near 100%
“As the models get better in reasoning we're going to get closer to a hundred percent of this VBench, which is that benchmark out of 12 repos open source Python repos a team in Princeton identified 2200 or so issue pull request pairs. Effectively, all the model…”