LM arena
also referred to as: lmarena
16 statements across 1 episodes · 13 bullish · 1 bearish · 4 people on the record · first statement May 29, 2025 by Anastasios Angelopoulos · said 11 times in 1 episodes since 2025 · across every show →
Mentions by year
brought up most by Anastasios Angelopoulos (10), Anjney Midha (1)
tap a year for its mentions
2025 11 mentions in 1 episode
Everything said about LM arena, oldest first
May 29, 2025 positive
Angelopoulos: LMArena conducts pre-release model testing for AI developers
“One of the things that we help everybody to do is pre-release testing of their models. Okay. So it's not just that, you know, we work together to evaluate the models are released, but we also try to be their release partners and say, Hey, can we help you guys …”
May 29, 2025 positive
Angelopoulos: Bradley-Terry models converge for AI evaluation, unlike Elo scores
“Okay, let's move from Elo to Bradley Terry because we're actually performing an estimate here instead of just like You know, and the ELO score moves over time. It doesn't converge, but Rally Terry models converge and how do we then construct confidence interva…”
May 29, 2025 positive
May 29, 2025 positive
Angelopoulos: LMArena prompt router yields double the performance per dollar
“Now, if you trace the performance, the best performance that, you know, any individual model can give you as part of the router as a function of cost. That's like two X worse than the router. In other words, the router is giving you double the bang for your bu…”
May 29, 2025 positive
May 29, 2025 negative
Angelopoulos: Believing chatbot leaderboards are easily gameable is naive
“I have to say, I also just like completely disagree with the foundation of the question. The like implicit assumption is that like chat is easy or that it's even easier than web dev. That's completely false. It's a completely naive perspective that people have…”
May 29, 2025 bullish
May 29, 2025 positive
Chiang: Personalized leaderboards will improve LMArena data quality
“And in that case, we align the interests of individuals and the platform as a whole, because you don't want to mess up your personal leaderboard. Just like how people these days, when they use social media, They don't like a random post because if they do that…”
May 29, 2025 positive
May 29, 2025 positive
Stoica: Over 70% of daily LMArena prompts are completely unique
“And basically measures out how many more, you know, fresh prompts you have in one day compared to what you've seen in the past three months, right? And by a similarity score of something like 70, 75%, you have over 70 of these prompts are fresh.”
May 29, 2025 positive
May 29, 2025
May 29, 2025 positive
Angelopoulos: LMArena router model outperforms all constituent models on Chatbot Arena
“When you train a prompt to leaderboard model, which is like, let's say a seven billion parameter model, and then you use it to route on just questions on the arena and everybody's questions, that model does better than any of the constituent models that were u…”
May 29, 2025 positive
May 29, 2025 neutral
May 29, 2025 positive
Angelopoulos: Industry AI evaluation platforms face skepticism over bias
“The fact that we come from Berkeley and from a university really speaks to our scientific approach in neutrality. I think if it came from an industrial lab, people would always have questions about, oh, well, these people are they also training a model and wha…”