The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 3 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Chatbot Arena fails to properly evaluate long-form conversational context
“And in fact, I think even chatbot arena has some deficiencies in terms of evaluation because from what I understand, you're doing these pairwise comparisons, but you're doing them you ask a question and two of them respond. And what I'm more curious about is w…”
Kantrowitz: Google's experimental Gemini 1114 model currently tops the Chatbot Arena leaderboard
“They have a new experimental Gemini model. It's called Gemini exp one, four. And Do you know about, ah, Chatbot Arena where they test which is the best LLMs? It's currently sitting at the top of Chatbot Arena and kicking the butt of ChatGPT for O, O-one Previe…”
The AI industry treats Chatbot Arena pairwise comparisons almost as gospel
“And in fact, like the thing that people always look at in terms of model performance is this chat bot arena where they put two answers from different chat bots side by side and say, okay, well, which one is the best? And that's kind of, it seems janky, but it'…”