rubrics
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Sep 25, 2025 by Shreya Shankar · across every show →
Everything said about rubrics, oldest first
Sep 25, 2025 neutral
Shreya Shankar: AI eval rubrics cannot be defined upfront without data
“What's new here is that you can't figure out your rubrics upfront. People's opinions of good and bad change as they review more outputs. They think of failure modes only after seeing 10 outputs they would never have dreamed of in the first place.”