The Ledger

Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.

clear all ✕

why aren't all 4 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable

Assertion Supported
Hu: OpenAI o1-preview surpasses human Kaggle Grandmasters with seven gold medals
“Since a grandmaster requires five gold medals and oh, and preview gets an average of eight or sorry, seven gold medals. They're out competing even capital grandmasters.”
Jesse Hu Oct 19, 2024 ▶ 47:29 [Paper Club] SWE-Bench [OpenAI Verified/Multimodal] + MLE-Bench with Jesse Hu
Assertion Supported
Hu: MLE-bench authors found obfuscating competition details did not show overfitting
“They do a lot of checks against overfitting on the Kaggle tasks themselves, and so they do something where they obfuscate some of the details of the Of the competitions, and then they rerun it. And I guess if they were overfitting on the competitions themselve…”
Jesse Hu Oct 19, 2024 ▶ 52:19 [Paper Club] SWE-Bench [OpenAI Verified/Multimodal] + MLE-Bench with Jesse Hu
Assertion Supported
Sanseviero: Kaggle launched an exam-based benchmark leaderboard for AI agents
“Last week, they released a new system for agent evaluation. It's like a very, like, experimental initial benchmark, but pretty much allowing agents to take an exam and compete in a leaderboard, which is always fun.”
Omar Sanseviero May 24, 2026 ▶ 28:31 ⚡️ Google's Open AI Strategy — Omar Sanseviero, Google DeepMind
Assertion Supported
Hu: OpenAI o1-preview achieves bronze medals in 17% of MLE-bench competitions
“Their final results with a one preview and this a scaffolding from a different company was that they got a bronze medal. I don't think I've ever achieved once but I haven't competed that much in. 17% of competitions.”
Jesse Hu Oct 19, 2024 ▶ 45:06 [Paper Club] SWE-Bench [OpenAI Verified/Multimodal] + MLE-Bench with Jesse Hu
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.