SWE-bench, every mention
4 scenes · ← back to SWE-bench
tap a year for its mentions
every year anyone Mike Krieger 2Nick Turley 1Brendan Foody 1
Verbatim, from the transcripts: the passages where SWE-bench comes up
Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody
- ▶ 6:58 Brendan Foody If you look at just how fast people were able to saturate Olympiad math once they focused on it, how fast we've been saturating Sweebench once we focus on it.
Inside ChatGPT: The fastest growing product in history | Nick Turley (OpenAI)
- ▶ 6:45 Nick Turley I'm especially excited about its performance on coding, um, whether or not that's SWE bench, which is a common benchmark or
Anthropic's CPO on what comes next | Mike Krieger (co-founder of Instagram)
- ▶ 6:34 Mike Krieger And he's like, not all of them are going to be right, you know, but even I think as of last year, he was talking about, you know, we're at 50% on sweet bench, which is just like, you know, benchmark around how well the models are at coding.
- ▶ 53:10 Mike Krieger And part of that was because there's this real, like, you know, we can hill climb on these evaluations and talk about sweet bench and towel bench and terminal bench, whatever, but customers ultimately know, like, you know,