GSM-1K
0 statements across 0 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement May 22, 2024 by Alexandr Wang · across every show →
Everything said about GSM-1K, oldest first
May 22, 2024
Alexandr Wang: Held-out benchmarks reveal several AI models underperform their reported scores.
“So we, one of the things we did is we published DSM-I-K, which was a held out eval. So we basically produced a new evaluation of the math capabilities of models. That there's no way it would ever exist in the training data set to really see how much of the, ho…”