why aren't all 853 resolved? a statement only gets an assessment when the public
record can support or contradict it. opinions and what-ifs never can, and 0 checkable
ones are still open, waiting for their date. predictions held up or didn't;
assertions are supported or contradicted. on every card:
▮▮▮▮▮ certainty ·
▮▮▮▮▮ debate potential. speakers are clickable
Assertion Contradicted
Cherny: Anthropic Can No Longer Demonstrate Prompt Injection on Opus 5
“So essentially, if you combine a well-aligned model, so this is, like, essentially three years of research into alignment, With a prompt injection classifier, which we run for all traffic, and what this is doing is it's based on Chrysola's mechanistic interpre…”
Prediction Didn’t hold up
Musk in 2016: Human travel to Mars could happen by 2026
“It's a meaningful number of people going to Mars I. I think this is potentially something that can be accomplished in about 10 years maybe sooner , maybe nine years.”
Assertion Supported
Cherny: Bun codebase was rewritten to Rust in 11 days using Claude
“And he had the model rewrite it from Zig to Rust. It was one prompt. It was a dynamic workflow, and a dynamic workflows are a feature in quad code that essentially let you orchestrate, you know, dozens, 100,000 of agents to do work productively. And it ran for…”
Assertion Partly supported
Srinivas: Google possessed only fourth or fifth best AI models in 2023-2024
“And 2024, 20 23 especially, and large part of 2024 too, Google had like, maybe a fourth or fifth best models at any moment. So as a startup outside Google, you had access to AI that was better than what Google internally had.”
Assertion Partly supported
Elon Musk emailed OpenAI saying it had zero percent success chance
“I can say this one because it got publicized in early, not early, a few years into OpenAI, where Elon sent us this really mean email. We'd been working through that for a while and said we had a zero percent chance of success. Like, not 0.10, that we were tota…”
Assertion Supported
Iorns: Replication studies show most published scientific results are not reproducible
“So the reality is that the results that have been generated show that most published results are not reproducible.”
Assertion Supported
Altman: Almost no top 30 software startup had remote early co-founders
“The data on this is look at the, say, like, 30 most successful software startups of all time, and try to point to a single example where the co-founders were in different locations. It's really, really tough.”
Assertion Partly supported
Wang: Meta rebuilt frontier AI lab from scratch after Llama 4 stalled
“Lama four wasn't on the trajectory that was needed for Meta. And so I got in there and we kind of did a zero based build of how do you know, build an entire frontier lab you know, in some ways kind of from scratch, obviously using a lot of what we had and move…”
Assertion Partly supported
Ries: Foundation-owned companies are 6x more likely to reach year 50
“Companies with this structure are six times more likely to live to year 50. 10% versus 60% probability.”
Assertion Supported
Tan: Jeff Lawson was ousted from Twilio 199 days after voting protections expired
“Our friend Jeff Lawson at Twilio built that company from nothing. To four billion dollars in, you know, actual revenue, like stock up 390% since IPO. I mean, by all accounts, you know, smash rip roaring success. And then his super voting shares expired after 1…”
Assertion Supported
Ries: Philip Morris generates $600B in annual US external costs
“They have something like eight billion dollars a year in net income. But there's been all these studies. They create six hundred billion dollars a year in costs just in the U.S. That have to be borne by others. I think it's three hundred billion in direct heal…”
Assertion Supported
Ries: Shareholder primacy was never enacted by statutory law or referendum
“In the history of the world. Shareholder primacy has never been subject to any referendum, any legislative action, nothing. So it's weird. If you learned in school, how a bill becomes a law, There's no law for shareholder primacy.”
Assertion Partly supported
Chaubard: HRM scored 70% on ARC Prize 1 without pre-training
“There is no pre-training at all. This starts from, like, literally Tagula-Rasa weights, and it can outperform at that time, if we go back, you know, we had O-three, if you remember back, way back when. And it, O-three gets zero. Literally zero, and this got, l…”
Assertion Supported
Tan: Boom Supersonic is using jet engines to power AI data centers
“Boom supersonic, instead of making supersonic jets right now, is on this good quest to create enough power for a bunch of these
AI data centers that are being built right now.
They use jet engines and even those like are so bad, you know, the supply chain for …”
Assertion Contradicted
Chas Englander: AI models outperform humans at structuring public filing data
“You know, if you look at some of these data providers, they've got humans reading information from say public filings and put it in that structured format. You know, the bulk of the work that we're doing, what we're seeing is models are already more accurate t…”
Assertion Supported
Finn: Diverse Home Training Matches Custom Target-Environment Performance
“And we find that if we actually increase the amount of homes, the amount of locations that are represented in the data, The performance increases, which is great. And it actually gets to the same level of performance as if we train on data from that target env…”
Assertion Supported
Jumper: Equivariance explains only 2-3 GDT of AlphaFold 2's 30-point gain
“The sixth row there, no IPA, invariant point attention, that removes all the equivariance in alpha fold, and it hurts a bit, but only a bit. Alpha fold itself on this GDT scale that you can see on the left graph, alpha fold two was about 30 GDT better than alp…”
Prediction Held up
Srinivas: Phone Makers Will Offer Multiple AI Search Alternatives
“The phone makers will start offering all of them as alternatives. It's not going to be like a locked-in default search option.”
Assertion Supported
Srinivas: OpenAI's built-in search features in ChatGPT did not kill Perplexity
“OpenAI has perplexity within ChatGPT that did not kill any of these companies.”
Assertion Supported
Garry Tan: CS majors face higher unemployment than art history majors
“The craziest stat, I think this came out of the New York fed in February of this year computer science majors You know, obviously this is not the people in this room. This is just, like, out of, like, you know, a normal distribution of all computer science maj…”
Assertion Supported
Hu: Cursor scaled from $1M to $100M revenue in one year
“We've seen crazy growth unlike anything, only possible with, right now with AI, like all these companies that we work with, zero to twelve million in 12 months. The cursor story where they went zero to one in one year, the next one to a hundred. This is, like,…”
Assertion Supported
Chollet: Scaling LLMs 50,000x Only Lifted ARC Accuracy to 10%
“And from at the time, back in 2019 to now, with a model like GPT 4.5, for instance, there's been a roughly 50,000 X scale up of basal alarms. And we went from zero percent accuracy on that benchmark to roughly 10%, which is not a lot.”
Assertion Supported
Chollet: Base LLMs score 0% and static reasoning scores 1-2% on ARC-2
“Well, if you take Bazel Alums, model Slack, GPT-IV-IV-V, LAMA-IV, it's simple, they get zero percent. There is simply no way to do these tasks simply via memorization. Next, if you look at static reasoning systems, so systems that use a single chain of tasks t…”
Assertion Supported
Chollet: SOTA test-time adaptation takes thousands in compute to solve ARC-1
“Even the latest set of the art CTA techniques they still need thousands of dollars of compute to solve arc one at human level. And that doesn't even scale to arc two.”
Assertion Partly supported
Nadella: Microsoft achieved physics breakthrough to fabricate Majorana quantum chip
“Finally we've had a physics breakthrough, and we were able to actually fabricate that particle, and so therefore that's what has led to this chip.”
Assertion Supported
Tom Blomfield says software startups are reaching revenue milestones faster than ever
“So getting to a 1,000,010 million, a hundred million of revenue with a software idea had is just happening faster and faster than ever.”
Assertion Supported
Thompson: Four Years After $42B Broadband Act, Effectively None Built
“We are four calendar years after the rural broadband provision was passed. No rural broadband effectively has been built. And of the 56 states and localities that applied for funding, only three of them even made it all the way through that 14 step process.”
Assertion Supported
McGrew: AI impact is not visible in productivity statistics
“I mean, yes, AI has had some effects, you know, particularly on people who write code, but, you know, I don't think you can see it in the productivity statistics, unless it's about how big the data centers are that we're building.”
Assertion Supported
Tan: OpenAI Uses Fake Model to Hide Raw o1 Chain of Thought
“If you use O-one in ChatGPT, it looks like it will tell you what's really going on, but apparently they have a fake model that just spits out things to give you the impression that it's breaking it up into steps. And they've actually You know, hidden it, becau…”
Assertion Supported
Saez-Gil: A trillion trees could draw down 200 gigatons of carbon
“There is about one billion hectares in the planet that are available for forest restoration without competing with agriculture, In that land, you can plant about a trillion trees, and that trillion trees has the potential to draw down about 200 gigatons of car…”
Assertion Supported
Doctorow: Facebook allowed non-members to scrape group membership lists
“She discovered that you could enumerate the membership of any Facebook group. Even if you weren't a member of it. And when she reported it to Facebook, they said that is a feature request, not a bug report. We will send it off to the product team. And when she…”
Assertion Supported
Doctorow: Big Tech and entertainment companies secretly colluded on anti-poaching agreements
“In California, we have no enforceable non-competes, so the big tech companies all got together in secret with the big entertainment companies and agreed that they wouldn't hire each other's employees, top employees.”
Assertion Supported
Doctorow: Ninth Circuit ruled LinkedIn cannot use CFAA to block scrapers
“HiQ was scraping LinkedIn to produce employer analytics. LinkedIn turned a blind eye to it until they launched their own analytics product. Then they tried to block them out and sue them under the Computer Fraud and Abuse Act. Not only is hiQ won in court and …”
Prediction Didn’t hold up
Best: Newsletters will replace social feeds for news within five years
“I think the answer is it won't replace the social network aspect of it. The thing that Facebook and the social networks are great at is having the relationship between you and like the people, you know, in real life... The part of it that should absolutely be …”
Assertion Contradicted
Allred: Lambda School graduates more than 85% of its starting students
“Now we're graduating, you know, more than 85% of students that start day one, and a lot of the dropout is because of reasons outside of our control.”
Assertion Contradicted
Caldwell: Consumer startups make up only about 5% of YC's portfolio
“People think that YC funds consumer companies, and this is what our bread and butter is. And like, Yeah, like, I think it's like five percent consumer.”
Assertion Supported
Authoritarian regimes weaponize Germany's content moderation laws to silence dissent
“We're already also seeing authoritarian governments adopt this law into their own into their own legal systems to silence dissent and to go after journalists who are publishing so-called fake news and whatnot.”
Assertion Supported
Americans shared junk news at a one-to-one ratio in 2016
“In 2016 we found that Americans on average were sharing about a one to one ratio of junk news to professionally produced information.”
Assertion Supported
The junk news sharing ratio worsened to 1.3:1 during 2018 midterms
“In 2018, when we redid this analysis during the midterm, the ratio of junk news actually went up. And so it went to like 1.2 or 1.3 to one.”
Assertion Supported
US users share vastly more junk news than European democracies
“During the UK elections, and Germany, and France, and Sweden, Mexico, all of those countries have much lower levels of junk news to professionally produced news shares than the US.”
Assertion Supported
Hua: Legalization pushed medical cannabis patients into the illicit market
“The irony of legalization is that there's a lot of medical patients have gone and turned to the illicit market for their products.”
Assertion Contradicted
Graham: Robert Morris invented the buffer overflow
“He invented buffer overflow.”
Assertion Contradicted
Capturing 20% of global freight makes a company bigger than Saudi Aramco
“Our goal is to get to 20 or 40% market share. If you do that, you're the biggest company on planet earth. Like full stop, like bigger than Saudi Aramco.”
Prediction Didn’t hold up
Preskill: Validated topological qubit may appear as soon as 2019
“And it's very ambitious, because at this point, it's not even clear they have a single qubit, but if that approach is successful, and we're, it's making progress, so I think we will see a validated Qubit of this type soon, maybe next year.”
Prediction Didn’t hold up
Garg: The biggest crypto winners will likely re-centralize in Silicon Valley
“I think you could have the inspiration for ideas come from underserved markets, but I wouldn't be surprised if the really big winners in crypto actually sort of re-centralize around Silicon Valley.”
Assertion Supported
A major humanitarian group raised $500M but built only six houses
“It was one of the largest humanitarian organizations, like a bunch of big headlines came out that they raised basically like half a billion dollars and within like six years had only built, and this is not an exaggeration, I think like six houses. That's like …”
Assertion Supported
Watts: Psilocybin's antidepressant effects typically last two to three months
“It seems to, for most people last for about two or three months and then the depression starts to come back.”
Prediction Held up
Bromberg: Regulators Will Classify Almost All Token Sales as Securities
“I think that what will happen is that the regulators are going to say, these are almost all securities. You can't just sell these tokens to anyone on the public markets.”