Eugene Yan
9 statements across 2 episodes · 4 bullish · 2 bearish · 1 people on the record · first statement Sep 28, 2024 by Eugene Yan · said 7 times in 3 episodes since 2024 · across every show →
On the record as a speaker too: Eugene Yan's record, appearances and statements → this page counts the times other people say the name.
Mentions by year
brought up most by Shawn Wang (6)
tap a year for its mentions
2026 1 mention in 1 episode
2025 5 mentions in 1 episode
Everything said about Eugene Yan, oldest first
Sep 28, 2024 neutral
Sep 28, 2024 negative
Yan: Pairwise evaluation fails for objective metrics like factuality
“The reason why pairwise preferences cannot work is that if you give two things that are both factual or if you give two things that are both non-factual, you would say that one is better than the other, but it still doesn't meet the bar of being factual enough…”
Sep 28, 2024 positive
Yan: Annotators must update past grades when evaluation criteria drift
“If you find that your criteria has drifted, instead of trying to maintain the same criteria, And aligning to the previous grades. Instead, what we should do is we should revisit those previous grades and fix it because it's an iterative process.”
Sep 28, 2024 positive
Nov 29, 2024 neutral
Nov 29, 2024 negative
Eugene Yan: Verifying LLM outputs is often harder than generating them
“And Shreya also has an interesting point, that it's much easier to verify the output and generate it. I actually observe the opposite. Or maybe it depends on the task. Like, for classification tasks, yes, it's easy. For, like, factuality, or comprehensiveness,…”
Nov 29, 2024 positive
Eugene Yan: DocETL pipeline optimizer costs roughly $100 and 30 minutes
“Running the optimizer, right, they used the optimizer. I don't know how many plans the optimizer generated, but it cost approximately a hundred dollars and less than half an hour. Just go get lunch and you come back and you get your optimized pipeline. And the…”
Nov 29, 2024 bullish