Static benchmarks
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement May 29, 2025 by Anastasios Angelopoulos · across every show →
Everything said about Static benchmarks, oldest first
May 29, 2025 positive
Angelopoulos: Chatbot Arena is immune to model overfitting by design
“Static benchmarks overfit. Why? It's because as Jan said earlier, you're giving the student the same test over and over. You have a model, you test it, you know, you look at whether or not it's improved on a static data set. Then you find another model, you te…”