The Ledger, every show
Every statement that passed quotation and attribution checks, across all 44 shows. Pick shows below, then mix any filter with any other.
shows 
every show 44 of 44
Angelopoulos: Kimi K3 Beat Top US Closed Models in Web Development
“And for the first time ever, we saw a couple of weeks ago that Kimi K three actually beat the best closed source American models on a, you know, pretty important subset of tasks, for example, front end coding, like web development, which a huge fraction of dev…”
Angelopoulos: Most global AI inference spend remains on proprietary first-party APIs
“If you look at the whole space of all inference, most of it is still being consumed on first party APIs and on proprietary models. That's why anthropic revenue has been just a total hockey stick. It's not, you know, it's not like they're being completely canni…”
Angelopoulos: Thinking Machines' Inkling ranks #1 in US open source, behind nine Chinese models
“And within that time, they'd become the number one American open source model. But then the less generous take would be the companies existed for a year and a half. And then they've come up with, yes, the number one American open source, but there's nine Chine…”
Angelopoulos: Arena has 30M+ monthly visitors, bigger than Hugging Face and xAI
“People don't know this, but Arena's one of the largest consumer AI apps in the world. We're bigger than, like, XAI. We're bigger than, like, Hugging Face, and Manus, and GenSpark, where it's so massive, like if you, like outside in, it's like 30 plus million m…”
Arena sampled open-source models at 60/40, debunking Leaderboard Illusion paper
“But, you know, there, for example said that we were, that we only sampled, like, nine percent open source models and, like, you know, 60%, like, closed source models, and this created a gap between open and closed source. But in reality, we're actually really …”
Angelopoulos: LMArena makes style control the default AI evaluation method
“That's why we're making style control default.”
Angelopoulos: LMArena router model outperforms all constituent models on Chatbot Arena
“When you train a prompt to leaderboard model, which is like, let's say a seven billion parameter model, and then you use it to route on just questions on the arena and everybody's questions, that model does better than any of the constituent models that were u…”
Angelopoulos: LMArena prompt router yields double the performance per dollar
“Now, if you trace the performance, the best performance that, you know, any individual model can give you as part of the router as a function of cost. That's like two X worse than the router. In other words, the router is giving you double the bang for your bu…”
Angelopoulos: OpenAI o1 crushed Chatbot Arena, proving the benchmark isn't saturated
“So there's this model and it crushed the benchmark. You know, it's just like really like a big gap. And what that's telling us is that it's not saturated yet. And so it's still measuring some signal that was encouraging point.”
Angelopoulos: The Chatbot Arena leaderboard is currently not an apples-to-apples comparison
“None of the leaderboard currently is apples to apples, because you have, like, Gemini Flash, you have, you know, all sorts of tiny models, like Llama Like, eight B and four or five B are not apples to apples.”
Angelopoulos: Chinese AI labs face severe hardware constraints and rely on black-market chips
“So they're way hardware constrained over there. And they've been trying to like black market import chips because of this. And you see this in the news, right? The information just reported on this.”
Angelopoulos: China Has Already Restricted American AI Models Domestically
“And by the way, it's worth noting that China has already restricted the use of American models within China, right? So if you look at the two by two matrix of US China restrict, not restrict, you know, like export import stuff They have already restricted the …”
Angelopoulos: Google's Gemma sits on the performance-versus-cost Pareto curve
“Gemma, by the way, is pretty good in terms of efficiency. If you look at arena, you'll see the, on the Pareto curves of like performance versus cost. Gemma's on there.”
Angelopoulos: Harvey's CEO views model labs as his biggest competitive worry
“Harvey, the CEO of Harvey himself is saying that, you know, his biggest competitive worry is the model labs.”
Angelopoulos: Bradley-Terry models converge for AI evaluation, unlike Elo scores
“Okay, let's move from Elo to Bradley Terry because we're actually performing an estimate here instead of just like You know, and the ELO score moves over time. It doesn't converge, but Rally Terry models converge and how do we then construct confidence interva…”
Angelopoulos: LMArena will remain open-source as a commercial company
“We're going to keep publishing papers. We're going to keep releasing open source. We're going to keep releasing open data.”
Angelopoulos: Arena received grants from Sequoia and a16z before incorporating
“He was not, you know, A-sixteen was not the only one to do this. We also had a great grant from Sequoia, but Ansh was in particular quite, quite supportive of us and, you know, gave us some resources in order to continue building out Arena before we even We're…”
Gradio scaled Arena to 1 million monthly active users before migration
“Gradio scaled us to a million Mal.”
Angelopoulos: LMArena will launch Data-Driven Debugging within months
“So we're building a project now that we call data-driven debugging D three. It's, you know, it's a little farther out. It'll come in a couple months.”
Angelopoulos: Chatbot Arena scores are calculated via logistic regression
“The arena score that we show on our leaderboard is a particular type of linear model, right? It's a linear model that takes, it's a logistic regression that takes model identities and fits them against human preference, right? So it regresses human preference …”