SWE-bench Multimodal, every mention
4 scenes · ← back to SWE-bench Multimodal
tap a year for its mentions
every year anyone Shawn Wang 1Jesse Hu 1Erik Schluntz 1
Verbatim, from the transcripts: the passages where SWE-bench Multimodal comes up
[State of Code Evals] After SWE-bench, Code Clash & SOTA Coding Benchmarks recap — John Yang
- ▶ 1:47 unnamed speaker but yeah, uh, multimodal. 3 times in the scene
2024 Year in Review: The Big Scaling Debate, the Four Wars of AI, Top Themes and the Rise of Agents
- ▶ 1:07:42 Shawn Wang Uh, we, we care about the Sweebench multimodal.
The new Claude 3.5 Sonnet, Computer Use, and Building SOTA Agents — with Erik Schluntz, Anthropic
- ▶ 24:04 Erik Schluntz I will say, um, Sweetbench just released Sweetbench multimodal, um, which I believe is either entirely JavaScript or largely JavaScript.