Humanity's Last Exam, every mention
5 scenes · ← back to Humanity's Last Exam
tap a year for its mentions
every year anyone Harry Stebbings 3Mike Krieger 1Brendan Foody 1Aatish Nayak 1
Verbatim, from the transcripts: the passages where Humanity's Last Exam comes up
Mercor CEO: Why Application Layer Companies Have No Moat & The Cost of Hiring AI Researchers · 20VC with Harry Stebbings
- ▶ 48:52 Brendan Foody Humanity's last exam for this long tail of academic problems no one really cares about.
Cohere's Chief AI Officer, Joelle Pineau: Why Scaling Laws Will Continue & Future of Synthetic Data · 20VC with Harry Stebbings
- ▶ 46:51 Harry Stebbings I've had some people on the show suggest that evals are, to put it delicately, bullshit, uh, and that they don't actually mean anything anymore, and like, you know, humanity's last test, like, what does that really even mean?
Mercor CEO & Co-Founder, Brendan Foody: How They Grew from $1M to $500M in 17 Months · 20VC with Harry Stebbings
- ▶ 24:14 Harry Stebbings We train or we assess the effectiveness or efficiency of models based on, you know, humanity's last test and all this other crap, which doesn't actually determine practical usage in society. 2 times in the scene
20Product: How Scale AI and Harvey Build Product | Why PMs Are Wrong: They are not the CEOs of the Product | How to do Pre and Post Mortems Effectively and How to Nail PRDs | The Future of Product Management in a World of AI with Aatish Nayak
- ▶ 42:02 Aatish Nayak A lot of public legal benchmarks, and then even things like, you know, scales, uh, humanities last exam, they're all multiple choice.
Mike Krieger, Instagram CoFounder & Anthropic CPO: Where Will Value Be Created in an AI World?|E1265 · 20VC with Harry Stebbings
- ▶ 12:12 Mike Krieger Uh, evals like humanities last exam, which like very much like, okay, multi-step reasoning, but there's yet to be the, the sort of, uh, I show up to a new job.