pairwise preferences
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Sep 28, 2024 by Eugene Yan · across every show →
Everything said about pairwise preferences, oldest first
Sep 28, 2024 negative
Yan: Pairwise evaluation fails for objective metrics like factuality
“The reason why pairwise preferences cannot work is that if you give two things that are both factual or if you give two things that are both non-factual, you would say that one is better than the other, but it still doesn't meet the bar of being factual enough…”