LM arena
also referred to as: lmarena
14 statements across 2 episodes · 5 bullish · 1 bearish · 2 people on the record · first statement Jul 14, 2025 by Pratik Bhavsar · across every show →
Everything said about LM arena, oldest first
Jul 14, 2025 neutral
General LLM benchmark rankings do not translate to agent performance
“What we have understood from our previous experiments is that it's not necessary that what you see as top models, In let's say LM arena or other specific evaluation or general evaluations, they might not be also the same ranking for other tasks like agentic ta…”
Dec 31, 2025 positive
Dec 31, 2025
Dec 31, 2025 positive
Dec 31, 2025 positive
Dec 31, 2025
Dec 31, 2025
Dec 31, 2025
Arena processes tens of millions of conversations monthly, totaling 250 million
“We have probably two hundred and fifty million conversations that happen over the course of the platform. We're on the order of, you know, mid tens of millions of conversations every month that are happening on the platform.”
Dec 31, 2025
Dec 31, 2025 bullish
Dec 31, 2025 neutral
Arena sampled open-source models at 60/40, debunking Leaderboard Illusion paper
“But, you know, there, for example said that we were, that we only sampled, like, nine percent open source models and, like, you know, 60%, like, closed source models, and this created a gap between open and closed source. But in reality, we're actually really …”
Dec 31, 2025 neutral
Dec 31, 2025 positive
Arena's organic user prompts provide realism that Artificial Analysis lacks
“They have arenas, but the arenas are not based on organic usage. Like the thing that distinguishes our platform versus theirs is that the users are actually inputting their own use case. They're actually asking their own question. And that gives a level of rea…”
Dec 31, 2025 negative
Arena's public leaderboard will never adopt Gartner-style pay-to-play models
“You can't pay to get on the public leaderboard. It's not like a Gartner in that sense. It's not like any of these, like you know, pay to play systems, never going to be like that. Models are going to be listed on the leaderboard, whether or not the providers p…”