The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 8 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Chi: Bundling model evaluation with data consulting produces pay-to-win benchmarks
“If you look at auditing as an industry, you end up with issues like Enron, where if you have the same group who's responsible for doing the audit, as well as also consulting and supporting the company, you have a mixed incentive structure, and then it just bec…”
Chi: Fragmented sovereign AI infrastructure is extremely capital inefficient
“If I was taking a God's eye view, it would be extremely inefficient to build all of these data centers and replicate this data engineering process and train these very large models when in fact you could probably consolidate a lot of these efforts but it seems…”
Chi: Legible evaluation methods are a primary driver of model capability
“What was very clear to me was the very tight relationship between what it takes to build new systems for generation, and actually new mechanisms for evaluation. In fact, in order to get one, you often need to get better at the other. And actually one of the bi…”
Chi: Trillion-Dollar Industries Require Independent Testing and Auditing Groups
“Every time a new trillion dollar industry emerges there, there's a need for this independent testing group”
Ryan Chi: Direct recursive self-improvement testing is too slow and expensive
“In an ideal world, what you want to do is actually take a frontier model and have it train the next version of itself and see where the delta comes from. But obviously that's very expensive and slow.”
Chi: Retiring AI benchmarks is necessary to reflect current real-world knowledge
“There's another component of retiring benchmarks, which I think is, is underappreciated which is that benchmark should also be reflective of the current state of the world.”