general benchmarks
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Sep 25, 2025 by Hamel Husain · across every show →
Everything said about general benchmarks, oldest first
Sep 25, 2025 neutral
Husain: General LLM benchmarks do not correlate with product-specific evals
“Up until now, a lot of the big labs understandably focused on general benchmarks, like MMLU score, human eval, things like that, which are very important for foundation models. And, you know, those not very related to product specific evals, like the ones we t…”