AI evaluation
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Nov 16, 2023 by Kanjun Qiu · across every show →
Everything said about AI evaluation, oldest first
Nov 16, 2023 negative
Qiu: Evaluating AI models solely on binary correctness loses critical evaluation data
“Part of why a lot of teams try to work on just math or code reasoning is because those are the easiest to evaluate and like the clearest Answers, but just relying on, like, is the output correct or not, that loses a lot of information in the evaluation.”