EvalGen
2 statements across 1 episodes · 2 bullish · 0 bearish · 1 people on the record · first statement Sep 28, 2024 by Eugene Yan · said 11 times in 2 episodes since 2024 · across every show →
Mentions by year
brought up most by Eugene Yan (8), Shreya Shankar (3)
tap a year for its mentions
Everything said about EvalGen, oldest first
Sep 28, 2024 positive
Yan: Annotators must update past grades when evaluation criteria drift
“If you find that your criteria has drifted, instead of trying to maintain the same criteria, And aligning to the previous grades. Instead, what we should do is we should revisit those previous grades and fix it because it's an iterative process.”
Sep 28, 2024 positive
Yan: EvalGen requires fewer assertions than SPADE for comparable evaluation
“Evalgen only needed three assertions, be quote assertions, or LLM prompts. So essentially, it needed less than spade, which needed five to get comparable results. Now, when we look at the product pipeline we see that evalgen only requires four assertions, whic…”