SimpleQA, every mention
4 scenes across 2 shows · ← back to SimpleQA
tap a year for its mentions
the a16z Podcast 4
Latent Space 4
every year every show Latent Space 4 the a16z Podcast 4
Verbatim, from the transcripts: passages where SimpleQA comes up on Latent Space, the a16z Podcast
Is AI Slowing Down? Nathan Labenz Says We're Asking the Wrong Question
- ▶ 8:39 Nathan Labenz 4.5 on this one benchmark called simple QA, which is really just a super long tail trivia benchmark. 2 times in the scene
- ▶ 39:39 Nathan Labenz I mean, it also wasn't going back to the simple QA thing. 2 times in the scene
The RLVR Revolution — with Nathan Lambert (AI2, Interconnects.ai)
- ▶ 22:59 Nathan Lambert You look at all the evals from reasoning models, and one of the trends is that, like, um, simple QA numbers all drop. 3 times in the scene
The Agent Reasoning Interface: Claude, ChatGPT Canvas, Tasks, Operator — with Karina Nguyen, OpenAI
- ▶ 15:56 Shawn Wang And then OpenAI put out simple eval, then nobody uses that.