benchmarks
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Jan 10, 2025 by Mario Rodriguez · across every show →
Everything said about benchmarks, oldest first
Jan 10, 2025 negative
GitHub CPO: Every Existing Public AI Benchmark Can Be Gamed
“I don't like benchmarks out there by the way, because you could game every single one of them. In my opinion, but what I do like about benchmarking is that it gives you a view into a set of scenarios that then you could then figure out, are you getting better …”