MMLU, every mention
3 scenes · ← back to MMLU
tap a year for its mentions
every year anyone Ion Stoica 1Anjney Midha 1
Verbatim, from the transcripts: the passages where MMLU comes up
Beyond Leaderboards: LMArena’s Mission to Make AI Reliable
- ▶ 0:43 Anjney Midha And so while benchmarks like MMLU and the idea of these static exams were useful three years ago, the future is about real-time evaluation, real-time systems, real-time testing in the wild.
- ▶ 11:53 Ion Stoica MMLEO is not probably as good as reflecting the preferences of the users.
The Quest for Community-Trained Open Source AI Models
- ▶ 31:26 unnamed speaker There's like benchmark, like, uh, Hela Swag, MMLU, and those type of question answering benchmarks.