offline evals
2 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Dec 7, 2025 by Ankur Goyal · across every show →
Everything said about offline evals, oldest first
Dec 7, 2025 negative
Goyal: Creating Golden Datasets for AI Evals Is Wasted Effort
“People don't really want to create golden data sets. It's, I think it's often a wasted effort to the point that you're making. I think the best teams view offline evals as a mechanism of reconciling what they see in production with real users who are using the…”
Dec 7, 2025
Goyal: Offline Evals Are Hardest to Build but Most Efficient Feedback Loop
“Offline evals, which I think is what a lot of the debate was about are both the most challenging to build feedback loop and also the most efficient once built. And then I think AB tests are a little bit less challenging to build and a little bit less efficient…”