Chatbot Arena
also referred to as: chat bot arena
3 statements across 2 episodes · 1 bullish · 1 bearish · 2 people on the record · first statement May 15, 2024 by Dwarkesh Patel · said 4 times in 1 episodes since 2024 · across every show →
Mentions by year
brought up most by Alex Kantrowitz (4)
tap a year for its mentions
2024 4 mentions in 1 episode
Everything said about Chatbot Arena, oldest first
May 15, 2024 negative
Chatbot Arena fails to properly evaluate long-form conversational context
“And in fact, I think even chatbot arena has some deficiencies in terms of evaluation because from what I understand, you're doing these pairwise comparisons, but you're doing them you ask a question and two of them respond. And what I'm more curious about is w…”
May 15, 2024 neutral
The AI industry treats Chatbot Arena pairwise comparisons almost as gospel
“And in fact, like the thing that people always look at in terms of model performance is this chat bot arena where they put two answers from different chat bots side by side and say, okay, well, which one is the best? And that's kind of, it seems janky, but it'…”
Nov 15, 2024 positive
Kantrowitz: Google's experimental Gemini 1114 model currently tops the Chatbot Arena leaderboard
“They have a new experimental Gemini model. It's called Gemini exp one, four. And Do you know about, ah, Chatbot Arena where they test which is the best LLMs? It's currently sitting at the top of Chatbot Arena and kicking the butt of ChatGPT for O, O-one Previe…”