The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 2,445 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 100 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Hou: 99% of AI editor rules file contents will be automatically inferred
“We strongly believe that having a rules file, you know, we do allow users to add a rules file, we strongly believe that a rules file is a crutch. You know, by the end of twenty-twenty-five, 99% of the things that you're gonna put in a rules file will be interp…”
Kamradt Predicts ARC-AGI-3 Benchmark Will Remain Unbeaten For 3 Years
“And then V three, our durability estimate for that is three years. And that's what we're aiming for is 36 months for V three.”
Bergum: Pinecone will not endure like MongoDB because it is too narrow
“So there's always like this convergence, but MongoDB kind of, it sticks, but I don't think that for Pinecoin that was originally leading that movement, it won't like stick in the same way.
It's too narrow.
It's too, too narrow.”
Conrad: Hyperscalers will probably lose significant money reselling Nvidia GPUs
“My intuition is that the hyperscalers are probably going to lose a lot of money, and they know they're going to lose a lot of money on reselling NVIDIA GPUs at least.”
Conrad: Decentralized compute networks will never beat co-located InfiniBand clusters
“I just don't really think this is gonna ever be more efficient than a fully interconnected cluster with Infiniband, or, you know, whatever sort of next spec might be. Like, I could be completely wrong, but Speedolite is really hard to beat. And regardless of w…”
Conrad: The AI VC bubble will pop and fail to return capital
“So what you've done by not having a future is you've inflated the venture capital market. And that is a bubble that's totally going to pop at some point. Like a lot of the companies are not going to work. And the valuations are not going to work. And what's go…”
Beauchamp: Timeline for AGI has certainly been pushed out
“I think the odds, the timeline for AGI has certainly been pushed out, right?”
Zhang: Meta Failed at Training MoE Models for Llama Series
“The reason why Lama open-sourced the MOE model, because I think they tried to train our MOE model, but they failed. So that, that's why they didn't open source MOE mode for Lama series.”
Bryk: Perplexity and ChatGPT Search rely on legacy Google and Bing APIs
“So these systems, there are a few of them now they basically rely on like traditional search engines like Google or Bing, and then they combine them with like LLMs at the end to, you know, output some power graphics answering your question. So they, Like, Sear…”
Swyx: AI models hit a 2T parameter wall, won't reach 10T
“As far as everyone is concerned, Claude, you know, Opus 3.5 is not coming out. GPT 4.5 is not coming out. And Gemini two, like we don't have pro whatever we've hit that wall, whatever that wall is. Maybe I'll call it like the two trillion parameter wall. Like …”
Ben Allal: Recent web dumps improve model benchmarks despite synthetic data
“So what we did is we trained different models on these different dumps, and we then computed their performance on popular like NLP benchmarks, and then we computed the aggregated score. And surprisingly, you can see that the latest dumps are actually even bett…”
Fu: Embedding model quality barely matters for final RAG performance
“We had this experience over and over again where you could have any, an embedding model of any quality, so you could have a really, really bad embedding model, or you could have a really, really good one by, and by any measure of good, and for the final RAG ap…”
Specialized open-source expert models will outperform one-size-fits-all closed-source models
“And that's our prediction is With specialization, there will be a lot of expert models, really, really good, and even better than, like, one size fits all open source closed source model.”
Crivello: Lindy AI agents perform better than humans for most use cases
“I think the bar is it needs to be better than a human. And for most use cases we serve today, it is better than the human, especially if you put it on rails.”
Crivello: Every major remote success story lost to an in-person competitor
“In every one of these examples, you have a co-located counterfactual that is sometimes orders of magnitude bigger.”
Goyal: Agent control flow and graph routing will move into models
“It feels very clear to me that this type of logic is going to be built into the model. Anytime there is control flow complexity or uncertainty complexity, I think the history of AI has been to push more and more into the model.”
Goyal: OpenAI o1 will make agentic frameworks obsolete
“And I think O-one is going to do that to agentic frameworks as well. Hey, I think To me, it seems very unlikely that the, you know, you and me sort of like sipping an espresso and thinking about how, like, different personified roles of people should interact …”
Shunyu Yao predicts training models on human computer trajectories achieves AGI
“The simplest way to achieve AGI is literally just record the re-actuatory of every human being and just put them together, you know, like what do you have thought about? What do you have done? Let's say on the computer, right? Imagine like solid experiment. Li…”
Joscha Bach: Only a Tiny Fraction of Wikimedia's Budget Goes to Servers
“The Wikimedia Foundation is publishing what they are paying the money for, and a very tiny fraction on this goes into running the servers, and the editors are working for free.”
Chintala: George Hotz's TinyGrad requires major breakthroughs to match PyTorch
“There's no, like, I don't think, like, unless we have, like, great breakthroughs, like, George's vision is achievable, like, or, like, he should be thinking about a narrower problem, such as, I'm only gonna make this for, like, work for self-driving car con ne…”
Chintala: Apple's MLX will fail server-side due to lack of differentiation
“If they end up expanding onto the server side, and they'll probably build something like PyTorch as well, right? Like, eventually, that'll where it will land. And I think there, they will kind of fail on the, like, lack of differentiation. Like, it wouldn't be…”
Patel: OpenAI and Microsoft partnership will likely collapse within years
“Yeah, I expect in the next few years that the OpenAI and Microsoft probably falls apart too.”
Royzen: Fine-tuned open source models will beat proprietary in 2024
“So I think that even if a delta exists, in twenty-twenty-four, the delta between proprietary and open source won't be large enough that a startup like us, with a lot of data that we've collected, can take the data that we have, fine-tune an open source model, …”
Royzen: GPT-4 was trained on HumanEval, proving data contamination
“GPT-IV itself has been trained on human eval, and we know this because GPT-IV is able to predict the exact doc string in many of the problems. I've seen it predict, like, the specific example values in the doc string, which is extremely improbable for it to ju…”
Howard: Alec Radford Built OpenAI's GPT After Reading ULMFiT
“I organized a chat for both of us with Kate Metz in the New York Times, and Kate Metz answered, sorry, and Alec answered this question for Kate, and Kate just like, so how did, you know, GPT come about? And he said, well, I was pretty sure that pre-training on…”
Howard: JAX was a grassroots Google reaction against TensorFlow 2
“But I mean, in the meantime, I will say, you know, Google now does have a backup plan. You know, they have JAX, which was never a strategy. It was just a bunch of people who also recognized TensorFlow two as shit, and they just decided to build something else.”
Hotz: Tinybox will be 5x faster than H100 systems per dollar
“For 90% of most companies model training use cases, the tiny box will be five X faster for the same price.”
Hotz: Machines will replace all human labor in about 20 years
“I'm a believer that machines are gonna replace everything in about 20 years.”
Jenik: Accelerated Understanding achieved 5-trillion token inference context length
“Like we're able to train up to a trillion context input. We're able to train With like, even inputs, outputs, both trillion context length, we are able to do inference at five trillion contexts.”
Anandkumar: Multi-physics models outperform single-physics models of equivalent parameter size
“And in fact, I was going to add that it turns out that having the model of the same size with multiple areas of physics does better than giving all of those parameters to each single physics. So if you had separate models and made them big enough as the origin…”
Lie: Cerebras runs OpenAI's flagship model 14x faster than GPUs
“We're running you know, frontier level, one of the most intelligent models, right? OpenAI's largest, most capable, most intelligent model right now at 14 times faster than their normal, you know, GPU speeds.”
Sean Lie: Next Cerebras Chip Will Run Frontier AI at 5,000 TPS
“And in particular, we designed it together with our next generation wafer chip that will be coming next year. And with that chip, We'll be pushing the performance even further. So we just saw a two X improvement this year with CS four. We're going to push it e…”
Lie: Cerebras chips have 100x more memory than Groq LPUs
“One of our chips has, You know, order a hundred times more memory than one of their chips, right? So you got two orders of magnitude difference in scale kind of for free, right?”
Sean Lie: Cerebras has solved yield at scale and 3D packaging
“And so, you know, we already have solved yield at scale. For example, we've already solved how you can actually package in a three-dimensional way. And that's, You know, problems that Samsung, that D-Matrix, and everybody else are also gonna have to solve over…”
Transformers will never scale to high-resolution 4D physics simulations
“So forget ever having a transformer for anything of this scale. All of the world's compute will not be enough. And first of all, they all have to be co-located to be able to ever do this. So that's why we need other architectures.”
Neural operators are the only AI architecture that works for climate emulation
“This is where the Allen AI Institute has now built climate models based on our neural operator architecture. And that's the only one that works As an AI emulator, right? None of the other architectures work for climate because climate requires us to assume the…”
AI models predict fusion reactor plasma disruption one million times faster
“You know, I talk about plasma and fusion reactor. You know, we barely have a few thousand samples, but we are able to accurately predict events like disruption very well. And we are able to do that a million times faster than what traditional simulations were …”
Park: Generative agent digital twins replicate human behavior at 85% accuracy
“And this is where we basically could replicate people's behaviors and attitudes, 85% as accurately as people would replicate their own. So that actually was the first really paper that gave this validated results that we can actually model individuals in an ac…”
Park: Frontier models hit only 20-30% accuracy predicting niche human behavior
“Where in some cases, the model performance of frontier models go all the way down to 20, 30%. Especially if you go into that more niche population on topics that our customers will actually care about. On more gen pop, it might be around 50 to 60%.”
Park: Future simulations will cost as much to run as training models
“My hunch here is I do think in the next Some number of years, we will start creating simulations that will actually cost as much as training a foundation model.”
Krentsel: AI takeoff flywheel will emerge from agents iterating on harness code
“There's a moment like that happening now, and this is the moment that's going to be a flywheel, I feel like, because this takeoff moment will come from being iterating in the same layer that you are producing, I think.”
McPartlon: AI models are nearing direct output of viable drug molecules
“We're kind of at the inflection point now. We're really seeing this internally at CHI, where the models are getting pretty close to, like, producing Molecules that could eventually, or like are very close to drugs.”
Patil: Biomolecular AI models will be as massive and impactful as LLMs
“And, you know, I think this class of models is going to be like just as big, just as impactful as LLMs, but it's almost like the compute market, like kind of doesn't realize that yet, both in the capacity sense, but also in like the software stack sense.”
Companies developing fused mega kernels rarely run them in production
“Even the companies that have worked at or people that have spoken to who work at companies that do fused mega kernels, they, Very, very often don't end up running those in production because the TRTL and modular kernels that will launch are faster because you …”
NVIDIA's Vera Rubin GPU architecture effectively renders mega kernel research obsolete
“The GPU is, is It's designed in such a way that it basically kills megakernels. You don't need to use megakernels that much anymore. So it seems like that entire research field goes into, like, won't be continued, but yeah.”
Long-form AI video generation must switch to autoregressive architectures over diffusion
“Autoregressive video seems to me like that is the bet that the future is going to be making. But there are no good open source autoregressive video models out there today. And that seems to be, if you want to get like an hour movie, if you want to see video mo…”
Kant: Major AI labs did not prioritize RL for LLMs three years ago
“And the second was that reinforcement learning was going to be the biggest driver for LLM capabilities. Today, very obvious three years ago was not an opinion held or direction held at either OpenAI or Google or Anthropic or others.”