evals
also referred to as: eval
2 statements across 2 episodes · 0 bullish · 0 bearish · 2 people on the record · first statement Feb 20, 2025 by Adarsh Hiremath · across every show →
Everything said about evals, oldest first
Feb 20, 2025 neutral
Hiremath: AI model evaluation definitionally requires human-created datasets
“I think a lot of it will be human data going forward, and I think a great example of this is evals, right? Evals for models definitionally have to be outside of model capability, right? In order to see whether model is doing well at a particular task, You need…”
Sep 15, 2025
If an AI model is the product, its eval is the PRD
“And if we think about the model as the product, then the eval is the PRD. And so many people have been sort of just like, you know, vibe spending on AI without actually writing the PRD of what do they want to implement and how do they measure that it's going t…”