why aren't all 684 resolved? a statement only gets an assessment when the public
record can support or contradict it. opinions and what-ifs never can, and 0 checkable
ones are still open, waiting for their date. predictions held up or didn't;
assertions are supported or contradicted. on every card:
▮▮▮▮▮ certainty ·
▮▮▮▮▮ debate potential. speakers are clickable
Assertion Supported
Cherny: Bun codebase was rewritten to Rust in 11 days using Claude
“And he had the model rewrite it from Zig to Rust. It was one prompt. It was a dynamic workflow, and a dynamic workflows are a feature in quad code that essentially let you orchestrate, you know, dozens, 100,000 of agents to do work productively. And it ran for…”
Assertion Supported
Iorns: Replication studies show most published scientific results are not reproducible
“So the reality is that the results that have been generated show that most published results are not reproducible.”
Assertion Supported
Altman: Almost no top 30 software startup had remote early co-founders
“The data on this is look at the, say, like, 30 most successful software startups of all time, and try to point to a single example where the co-founders were in different locations. It's really, really tough.”
Assertion Supported
Tan: Jeff Lawson was ousted from Twilio 199 days after voting protections expired
“Our friend Jeff Lawson at Twilio built that company from nothing. To four billion dollars in, you know, actual revenue, like stock up 390% since IPO. I mean, by all accounts, you know, smash rip roaring success. And then his super voting shares expired after 1…”
Assertion Supported
Ries: Philip Morris generates $600B in annual US external costs
“They have something like eight billion dollars a year in net income. But there's been all these studies. They create six hundred billion dollars a year in costs just in the U.S. That have to be borne by others. I think it's three hundred billion in direct heal…”
Assertion Supported
Ries: Shareholder primacy was never enacted by statutory law or referendum
“In the history of the world. Shareholder primacy has never been subject to any referendum, any legislative action, nothing. So it's weird. If you learned in school, how a bill becomes a law, There's no law for shareholder primacy.”
Assertion Supported
Tan: Boom Supersonic is using jet engines to power AI data centers
“Boom supersonic, instead of making supersonic jets right now, is on this good quest to create enough power for a bunch of these
AI data centers that are being built right now.
They use jet engines and even those like are so bad, you know, the supply chain for …”
Assertion Supported
Finn: Diverse Home Training Matches Custom Target-Environment Performance
“And we find that if we actually increase the amount of homes, the amount of locations that are represented in the data, The performance increases, which is great. And it actually gets to the same level of performance as if we train on data from that target env…”
Assertion Supported
Jumper: Equivariance explains only 2-3 GDT of AlphaFold 2's 30-point gain
“The sixth row there, no IPA, invariant point attention, that removes all the equivariance in alpha fold, and it hurts a bit, but only a bit. Alpha fold itself on this GDT scale that you can see on the left graph, alpha fold two was about 30 GDT better than alp…”
Prediction Held up
Srinivas: Phone Makers Will Offer Multiple AI Search Alternatives
“The phone makers will start offering all of them as alternatives. It's not going to be like a locked-in default search option.”
Assertion Supported
Srinivas: OpenAI's built-in search features in ChatGPT did not kill Perplexity
“OpenAI has perplexity within ChatGPT that did not kill any of these companies.”
Assertion Supported
Garry Tan: CS majors face higher unemployment than art history majors
“The craziest stat, I think this came out of the New York fed in February of this year computer science majors You know, obviously this is not the people in this room. This is just, like, out of, like, you know, a normal distribution of all computer science maj…”
Assertion Supported
Hu: Cursor scaled from $1M to $100M revenue in one year
“We've seen crazy growth unlike anything, only possible with, right now with AI, like all these companies that we work with, zero to twelve million in 12 months. The cursor story where they went zero to one in one year, the next one to a hundred. This is, like,…”
Assertion Supported
Chollet: Scaling LLMs 50,000x Only Lifted ARC Accuracy to 10%
“And from at the time, back in 2019 to now, with a model like GPT 4.5, for instance, there's been a roughly 50,000 X scale up of basal alarms. And we went from zero percent accuracy on that benchmark to roughly 10%, which is not a lot.”
Assertion Supported
Chollet: Base LLMs score 0% and static reasoning scores 1-2% on ARC-2
“Well, if you take Bazel Alums, model Slack, GPT-IV-IV-V, LAMA-IV, it's simple, they get zero percent. There is simply no way to do these tasks simply via memorization. Next, if you look at static reasoning systems, so systems that use a single chain of tasks t…”
Assertion Supported
Chollet: SOTA test-time adaptation takes thousands in compute to solve ARC-1
“Even the latest set of the art CTA techniques they still need thousands of dollars of compute to solve arc one at human level. And that doesn't even scale to arc two.”
Assertion Supported
Tom Blomfield says software startups are reaching revenue milestones faster than ever
“So getting to a 1,000,010 million, a hundred million of revenue with a software idea had is just happening faster and faster than ever.”
Assertion Supported
Thompson: Four Years After $42B Broadband Act, Effectively None Built
“We are four calendar years after the rural broadband provision was passed. No rural broadband effectively has been built. And of the 56 states and localities that applied for funding, only three of them even made it all the way through that 14 step process.”
Assertion Supported
McGrew: AI impact is not visible in productivity statistics
“I mean, yes, AI has had some effects, you know, particularly on people who write code, but, you know, I don't think you can see it in the productivity statistics, unless it's about how big the data centers are that we're building.”
Assertion Supported
Tan: OpenAI Uses Fake Model to Hide Raw o1 Chain of Thought
“If you use O-one in ChatGPT, it looks like it will tell you what's really going on, but apparently they have a fake model that just spits out things to give you the impression that it's breaking it up into steps. And they've actually You know, hidden it, becau…”
Assertion Supported
Saez-Gil: A trillion trees could draw down 200 gigatons of carbon
“There is about one billion hectares in the planet that are available for forest restoration without competing with agriculture, In that land, you can plant about a trillion trees, and that trillion trees has the potential to draw down about 200 gigatons of car…”
Assertion Supported
Doctorow: Facebook allowed non-members to scrape group membership lists
“She discovered that you could enumerate the membership of any Facebook group. Even if you weren't a member of it. And when she reported it to Facebook, they said that is a feature request, not a bug report. We will send it off to the product team. And when she…”
Assertion Supported
Doctorow: Big Tech and entertainment companies secretly colluded on anti-poaching agreements
“In California, we have no enforceable non-competes, so the big tech companies all got together in secret with the big entertainment companies and agreed that they wouldn't hire each other's employees, top employees.”
Assertion Supported
Doctorow: Ninth Circuit ruled LinkedIn cannot use CFAA to block scrapers
“HiQ was scraping LinkedIn to produce employer analytics. LinkedIn turned a blind eye to it until they launched their own analytics product. Then they tried to block them out and sue them under the Computer Fraud and Abuse Act. Not only is hiQ won in court and …”
Assertion Supported
Authoritarian regimes weaponize Germany's content moderation laws to silence dissent
“We're already also seeing authoritarian governments adopt this law into their own into their own legal systems to silence dissent and to go after journalists who are publishing so-called fake news and whatnot.”
Assertion Supported
Americans shared junk news at a one-to-one ratio in 2016
“In 2016 we found that Americans on average were sharing about a one to one ratio of junk news to professionally produced information.”
Assertion Supported
The junk news sharing ratio worsened to 1.3:1 during 2018 midterms
“In 2018, when we redid this analysis during the midterm, the ratio of junk news actually went up. And so it went to like 1.2 or 1.3 to one.”
Assertion Supported
US users share vastly more junk news than European democracies
“During the UK elections, and Germany, and France, and Sweden, Mexico, all of those countries have much lower levels of junk news to professionally produced news shares than the US.”
Assertion Supported
Hua: Legalization pushed medical cannabis patients into the illicit market
“The irony of legalization is that there's a lot of medical patients have gone and turned to the illicit market for their products.”
Assertion Supported
A major humanitarian group raised $500M but built only six houses
“It was one of the largest humanitarian organizations, like a bunch of big headlines came out that they raised basically like half a billion dollars and within like six years had only built, and this is not an exaggeration, I think like six houses. That's like …”
Assertion Supported
Watts: Psilocybin's antidepressant effects typically last two to three months
“It seems to, for most people last for about two or three months and then the depression starts to come back.”
Prediction Held up
Bromberg: Regulators Will Classify Almost All Token Sales as Securities
“I think that what will happen is that the regulators are going to say, these are almost all securities. You can't just sell these tokens to anyone on the public markets.”
Assertion Supported
Baidu's Deep Speech engine achieves superhuman accuracy on short queries
“The speech engine that we've built at Baidu called Deep Speech it's actually superhuman for these short queries.”
Prediction Held up
Greene: Enterprise tech will consolidate into four clouds immune to startup disruption
“Everything's going to be in just a few, maybe four clouds or so, you know, there's not going to be that many of them, because like, we spend about ten billion a year in infrastructure, and so it's not going to Get disrupted by a startup, unless they invent qua…”
Prediction Held up
Benet: Decentralized storage nodes will beat AWS on unit economics
“Our bet there is that yes, that there's a whole bunch of places and cases where Certain individuals or groups in the world have access to either really cheap storage or storage that's positioned well in the network that is kind of like, you know, somewhere bet…”
Assertion Supported
Altman: Chesky asked if candidates would work with one year to live
“He used to ask people before he hired them at Airbnb if they would take the job if they got a medical diagnosis that they had one year left to live.”
Assertion Supported
Stuart: Multi-GPU compilers often produce kernels slower than baseline
“Two is to use compiler-based approaches, but we find these compilers, compilers to be quite suboptimal. They produce kernels that are sometimes slower than non-overlapped baselines”
Assertion Supported
Stuart: Bypassing NCCL intermediate buffers speeds up all-reduce by 80%
“For example, Nickel's default mode forces intermediate buffers which adds extra data movement between the sender and the receiver. And for fine-grained communication, this overhead really accumulates, and by stripping it out, you can speed up an operation as s…”
Assertion Supported
Stuart: ParallelKittens matches hand-optimized kernels in 50 to 100 lines
“And what we find is that with roughly 50 to 100 lines of device code, PK is able to surpass or match hand-optimized kernels that are often hundreds to thousands lines of code.”
Assertion Supported
Scholl: XB-1 Broke Sound Barrier Six Times With No Ground Boom
“XB one broke the sound barrier on two flights, six times through the sound barrier. Each time, no audible sonic boom in the ground.”
Assertion Supported
Anthropic Deleted 80% of Claude Code System Prompt for Opus 5
“So yeah, we deleted 80% of the system prompt.”
Assertion Supported
Cherny: Claude Code runs in production on Bun's AI-rewritten Rust codebase
“This is in production now. This is what quad code uses now when you're running it.”
Assertion Supported
Huang: Radiology jobs grew 20% over recent years despite AI scan automation
“The task of reading radiology scans has been automated, but the number of radiology jobs has increased some 20% in the last several years, even though AI has taken over the whole field.”
Assertion Supported
Pincus: Early backers sold Facebook stock because none foresaw its ultimate scale
“I don't think Reed or I Ever imagined how, I know none of us ever imagined how big this, Peter Thiel, all of us, we all sold our stock in Facebook, so we voted with our shares, you know, none of us could have ever described how big this all got.”
Assertion Supported
Ries: Twilio fired Jeff Lawson despite revenue growth since IPO
“So at the time he was fired, the stock was down like 80% from the peak. And it's like, oh, well case closed. But if you measure from the IPO or even from the pandemic peak, revenue was up. It's like, did the business go down? Was revenue down? Was there some k…”
Assertion Supported
Tan: Professional engineers average only 30 to 50 lines of code daily
“If you look at the literature about software engineering, going back to like 2000. 1990. I mean, it's pretty clear that the average number of lines of code that a professional software engineer that's like tested and production ready, it's not like a hundred l…”
Assertion Supported
Tan: GStack has more GitHub stars than Ruby on Rails in three weeks
“I built GStack to encode this three weeks ago, and now it has more GitHub stars than Ruby on Rails.”
Assertion Supported
Vuong: OpenX generalist robot model outperformed embodiment-specific specialists by 50%
“You can compare it to the specialist that has been optimized to work well on a particular embodiment. How does it compare? And the interesting result from OpenX is it was 50% better.”