LM arena

also referred to as: lmarena

14 statements across 2 episodes · 5 bullish · 1 bearish · 2 people on the record · first statement Jul 14, 2025 by Pratik Bhavsar · across every show →

Everything said about LM arena, oldest first

Jul 14, 2025 neutral
Insight
General LLM benchmark rankings do not translate to agent performance
“What we have understood from our previous experiments is that it's not necessary that what you see as top models, In let's say LM arena or other specific evaluation or general evaluations, they might not be also the same ranking for other tasks like agentic ta…”
Pratik Bhavsar Jul 14, 2025 ▶ 2:22 ⚡️Ranking Agentic LLMs — Pratik Bhavsar, Galileo
Dec 31, 2025 positive
Assertion Not checkable as stated
Angelopoulos: LMArena has released more real-world AI data than almost anyone
“We've probably released more data than basically anybody on the real world use cases of AI.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 16:32 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025
Assertion Not checkable as stated
LMArena funds all model inference running on its platform
“We fund all of the inference on the platform.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 4:13 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025 positive
Assertion Supported
Gradio scaled Arena to 1 million monthly active users before migration
“Gradio scaled us to a million Mal.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 8:48 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025 positive
Assertion Not checkable as stated
Arena's anonymous Nano Banana test moved Google's stock and product roadmap
“I mean, that moment alone changed Google's like roadmap. Market share. Seriously. I mean, Google stock, billions of dollars are moving because of Nano.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 13:20 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025
Assertion Not checkable as stated
Angelopoulos: 25% of LMArena platform users write software for a living
“25% of the people on our platform, for example, do software for a living.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 5:12 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025
Assertion Not checkable as stated
Angelopoulos: About half of LMArena users are authenticated
“About half of our users now are login.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 5:34 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025
Assertion Not checkable as stated
Arena processes tens of millions of conversations monthly, totaling 250 million
“We have probably two hundred and fifty million conversations that happen over the course of the platform. We're on the order of, you know, mid tens of millions of conversations every month that are happening on the platform.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 4:46 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025
Disclosure
Angelopoulos: LMArena receives only standard enterprise inference discounts
“No, no, we get discounts, but they're, but they are standard enterprise discounts. The same that would be given to any other customer.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 4:19 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025 bullish
Disclosure
Angelopoulos: LMArena to launch video evaluations by early next year
“Video we're soon to launch on the site at some point, you know, later this year or early next.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 18:58 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025 neutral
Assertion Supported
Arena sampled open-source models at 60/40, debunking Leaderboard Illusion paper
“But, you know, there, for example said that we were, that we only sampled, like, nine percent open source models and, like, you know, 60%, like, closed source models, and this created a gap between open and closed source. But in reality, we're actually really …”
Anastasios Angelopoulos Dec 31, 2025 ▶ 11:51 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025 neutral
Disclosure
Angelopoulos: LMArena's top expenses are free-tier inference, hiring, and SF office
“Primarily inference that funds the free usage of the platform and then also hiring, of course, headcount. We have an office, you know. That's an SF.”
Anastasios Angelopoulos Dec 31, 2025 ▶ 9:57 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025 positive
Opinion
Arena's organic user prompts provide realism that Artificial Analysis lacks
“They have arenas, but the arenas are not based on organic usage. Like the thing that distinguishes our platform versus theirs is that the users are actually inputting their own use case. They're actually asking their own question. And that gives a level of rea…”
Anastasios Angelopoulos Dec 31, 2025 ▶ 7:24 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Dec 31, 2025 negative
Disclosure
Arena's public leaderboard will never adopt Gartner-style pay-to-play models
“You can't pay to get on the public leaderboard. It's not like a Gartner in that sense. It's not like any of these, like you know, pay to play systems, never going to be like that. Models are going to be listed on the leaderboard, whether or not the providers p…”
Anastasios Angelopoulos Dec 31, 2025 ▶ 17:26 [State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.