LMSYS Chatbot Arena
part of LMSYS
4 statements across 4 episodes · 3 bullish · 0 bearish · 4 people on the record · first statement Jan 11, 2024 by Nathan Lambert · said 87 times in 25 episodes since 2023 · across every show →
Mentions by year
brought up most by Shawn Wang (43), Anastasios Angelopoulos (13), Nathan Lambert (9), Alessio Fanelli (2), Vibhu Sapra (1), Vasek Mlejnsky (1), Tri Dao (1), Thomas Scialom (1)
tap a year for its mentions
2026 3 mentions in 2 episodes 2 per episode
2025 57 mentions in 11 episodes 5 per episode
- [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
- The RLVR Revolution — with Nathan Lambert (AI2, Interconnects.ai)
- [State of Research Funding] Beyond NSF, Slingshots, Open Frontiers — Andy Konwinski, Laude Institute
- ⚡️Ranking Agentic LLMs — Pratik Bhavsar, Galileo
- [State of Code Evals] After SWE-bench, Code Clash & SOTA Coding Benchmarks recap — John Yang
- [State of Context Engineering] Agentic RAG, Context Rot, MCP, Subagents — Nina Lopatina, Contextual
- Greg Brockman on OpenAI's Road to AGI
- ⚡️Multi-Turn RL for Multi-Hour Agents — with Will Brown, Prime Intellect
- 3 more episodes that year, every mention in 2025 →
2024 26 mentions in 11 episodes 2 per episode
- The Origin and Future of RLHF: the secret ingredient for ChatGPT - with Nathan Lambert
- In the Arena: How LMSys changed LLM Benchmarking Forever
- Training Llama 2, 3 & 4: The Path to Open Source AGI — with Thomas Scialom of Meta AI
- The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka
- Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]
- The State of AI Startups in 2024 [LS Live @ NeurIPS]
- [Paper Club] Molmo + Pixmo + Whisper 3 Turbo - with Vibhu Sapra, Nathan Lambert, Amgadoz
- Personal benchmarks vs HumanEval - with Nicholas Carlini of DeepMind
- 3 more episodes that year, every mention in 2024 →
2023 1 mention in 1 episode
Everything said about LMSYS Chatbot Arena, oldest first
Jan 11, 2024 positive
Lambert: GPT-4 Turbo Showed a Noticeable Jump on LMSYS Chatbot Arena
“GPT-IV Turbo is also notably ahead of the other GPT-IVs, which it kind of showed up immediately once they added it to the leaderboard, or to the arena, and I was like, all the GPT-IV memes aside, it seems like this is effectively a bump in the model.”
Jan 1, 2025 positive
Swyx: Iso-ELO LLM token costs dropped nearly 1,000x in 2024
“For the same amount of ELO, what you used to pay at the start of 24 you know, let's say, you know, 50, 40 to 50 dollars Per million tokens now is available approximately at, with Amazon Nova approximately at, I don't know, 0.075 dollars per token. So like seve…”
Jan 19, 2025 bullish
Dec 31, 2025 neutral
Yang: Evaluating Human-AI Code Interaction Requires Compelling Products or Simulators
“From an academic standpoint, it feels like there's two difficult approaches to resolving that. Either you build, like, a really compelling product, like Elmarina, that people have people use consistently, which is, I mean, really tricky in and of itself. Or yo…”