The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 1,824 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Slack: AI agents eliminate the need to hire low-agency staff
“And compared to a few years ago, You do not have the need to go and hire people that you do not trust, that you do not trust to have high agency or the right skills, because you can use an agent to do those things.”
Slack says hiring early PMs or marketers is an outdated practice
“And I see some other companies that are around our size starting to go hire a PM or a marketer or something like that. And that just feels like the old way of building a software business. And ultimately that's gonna lead to something where 10% of the people a…”
Azhnyuk: Nuclear Weapons Are Now the Only Credible Deterrence
“Someone, someone someone could argue that if a country currently doesn't work on their own nuclear program, They're doing a disservice to their country and the government should be fired. Like, because it seems like from the recent world history that, that, th…”
Casado: Frontier AI Labs Could Outspend and Consume Application Layer Startups
“It literally becomes an issue of like raise capital, turn that directly into growth, use that to raise three times more. And if you can keep doing that, you literally can outspend any company that's built. Not any company. You can outspend the aggregate of com…”
Yegge: Rewriting code from scratch with LLMs beats trying to fix it
“And now we've discovered that it is for a larger and larger and larger class of Piece of bodies of code. It is better to just start over and rewrite it from scratch than it is to try to fix it. The LLM will do a better job.”
Pliny: Latent-space AI safety guardrails fail every single time
“They tried to solve this on the latent space level. I think I've It's shown every single time that that doesn't work.”
Morcos: Post-training alignment is ineffective long-term compared to pre-training alignment
“Like fundamentally, I think alignment and post training doesn't really make sense as a long-term solution. If you can easily align a model through post training, you can easily misalign a model through post training. If it's easy to put it in, it's easy to tak…”
Optimal future AI coding interfaces will not evolve from traditional IDEs
“Our take is that it is very unlikely that the optimal UI or the optimal interaction pattern for this new software development where humans spend much less time writing code. I think it's very unlikely that that optimal interaction pattern will be found by iter…”
Embiricos: Scaffolding-heavy AI agents are limited by developers' mental capacity
“A lot of, like, agents that I see are really impressive, but it's basically, like, part of what's impressive is it's like a bunch of developers building this, like, really bespoke state machine around a bunch of, like, short model calls, and so then the upper …”
Beauchamp: ML model performance plateaus near human-level on an S-curve
“With machine learning, first of all, you see that the performance of the models follows an S-curve. So it's not like it just goes off to infinity, right? And the S curve, it kind of plateaus around human level performance.”
Anthropic's Schluntz: Avoid agent frameworks and start from scratch with raw prompts
“I think with agent frameworks in general, they can certainly save you some like boilerplate, but I think there's actually this like downside of making agents too easy, where you end up very quickly, like building a much more complex system than you need. And s…”
Crivello: Remote work optimizes for cost; building creative software requires in-person collaboration
“If you're optimizing for cost, absolutely be remote. If you're optimizing for creativity, which I think that software and product building is a creative endeavor, if you're optimizing for creativity, it's kind of like composing an album. You can't do it on the…”
Tay: Frontier AI researchers cannot maintain standard nine-to-five work-life balance
“You cannot be, like, checking out on, like, Friday, Saturday, Sunday, and, like, work at, like, nine to five if you want to, like, Make progress, or like, some people are just so good at detaching, like, ok, like, you know, like, eight pm, I'm not going to, my…”
Albrecht: LLM emergence is an artifact of non-linear evaluation metrics
“This emergent behavior that you're seeing, Is not really emergent behavior, but is really a function of the evaluation metrics that we're using.”
Translating code proves LLMs possess causal, functional understanding
“If you ask the LLM to translate a bit of Python into a little bit of C, and it's performing this task, obviously it is understanding in the sense that it has a, The causal, functional model it implements.”
Bach: Claude is implemented as an invariant pattern similarly to consciousness
“Claude exists only as a pattern. It's something that is a pattern in the activation of the transistors. And even transistors don't actually exist. They are A pattern in the atoms that we are able to see as an invariance because we tune the atoms in a particula…”
Liu: LLM observability startups ignore full systems; just use Postgres
“The issue really is the fact that these observability companies isn't actually doing observability for the system, it's just doing the LLM thing. Like I still end up using like Datadog, right? Or like, you know, Sentry to do, like, latency. And so I just have …”
Liu: Companies abandon LLM frameworks to regain control over prompts
“So much of it is changing that if you give control of these systems away too early, you end up ultimately wanting them back. Like many companies I know that I reach out or ones were like, oh, we're going off of the frameworks because now that we know what the …”
AI Productivity Gains Will Ultimately Increase Demand for Software Engineers
“Like, I think we can, you know, 10 x the amount of developers, and still, you know, have a lot of people making a lot of money, you know, building amazing software, and also being, while at the same time being more productive. Like, I never understood this, li…”
Hsu: Higher valuation for the same quality company is always worse
“Higher valuation, given the same company quality, is always worse.”
Liu: RAG is fundamentally just an algorithmic prompt-stuffing hack
“RAG is basically just a hack, but it turns out it's a very good hack because what is RAG? RAG is you keep the model fixed, and you just figure out a good way to, like, stuff stuff into the prompt of the language model. Everything that we're doing nowadays in t…”
Hotz: Maximally compressed human brain represents only a couple gigabytes
“Quantization is a poor man's compression. I think we're only talking really here about, like, maybe a couple gigabytes, right? And then if you have, like, a couple gigabytes of true information of yourself up there, cool man. Like, what does it mean for me to …”
Hotz: Real AI alignment problem is corporate and government misalignment
“I think it's actually not a question of whether the computer is aligned with the company who owns the computer. It's a question of whether that company's aligned with you or that government's aligned with you. And the answer is no. And that's how you end up de…”
Slack: AI agents and code are the ultimate settings screen
“Really an agent is the ultimate settings screen for any software and code is the ultimate setting screen for any software.”
Anandkumar: Standard Transformers cannot scale to 5-trillion context lengths for physics
“On the other hand, if you think about using transformer architectures that have worked so well for language, that just wouldn't be able to support a five trillion context length. No matter all the compute in the world is thrown at it. So that kind of quadratic…”
Jenik: PDE feedback pushes physics AI models to exceed training data
“You can use those PDEs both for numerical simulators to generate data, but if you're clever about it, you can even use them as a training signal. You can check how well is my model actually doing on the PDEs themselves. And use that as an additional training s…”
Anandkumar: Dense physics feedback enables better AI self-improvement than sparse LLMs
“And the difference there is compared to language where self-improvement needs something like human feedback or other reward signals that are very sparse. They just tell you yes or no, thumbs up or down. We have dense feedback because the physics laws, there's …”
Park: Training is required for new world physics, while prompting suffices for reactions
“My intuition behind the actual, when do you train or even post train a model versus just prompt a model is if the model has to learn the underlying physics of the world that it's operating in. So it has to learn new social physics. The places where it doesn't …”
Park: Agent simulations beat multi-year field trials via instant repeated execution
“This is the reason why you want to run a simulation. You spend five years, forty million dollars on this one study and have one finding. But if you can run simulation many, many times instantly, then that's the value.”
Krentsel: AI self-improvement requires collapsing outer optimization loops
“You have some outer system that's optimizing some inner system. What if you want to optimize the way you're doing? You're optimizing, then you need some outer, outer loop, and it's this infinite recursion out. And the only way I think out of that is to collaps…”
McPartlon: Multi-specific antibody modalities must be designed computationally from first principles
“There are drug modalities that you just can't discover with immunization. Like you're not gonna design your like crazy multi-specific Warheaded, super intense formats. These are really things where you kind of have to design these from first principles. Even j…”
Quantizing more layers can actually improve model fidelity via error cancellation
“It is possible that the model in which I quantized more information is going to perform better because the quantization errors have canceled out. And so what Joshua showed in his mathematical proof where he had like a verifier in is that you can predict which …”
Optimal inference parallelism cannot be mathematically calculated; it must be auto-tuned
“And with training, it's more of like a math, like you can run the math and see the flops and maximize it. With inference, it's more of like an auto-tuning, like GPU kernel auto-tuning... You shadow the same traffic, like real traffic, and you just see which co…”
Custom $50B ASICs become economically rational for $500B AI training runs
“If you have a trillion dollar or five hundred billion dollar training run, then take fifty billion of that and make an ASIC. Like, it's fine. Like you will get more than 10% efficiency from the ASIC. And like, that makes sense.”
Ultra-fast network interface cards could deliver 100x speedups in disaggregated AI inference
“If you were to somehow be able to, in like this theoretical dreamland, have extremely fast NICs, you could, in theory, spare that HBM and you could just transfer KVCache trans, like directly from one node to another. This would give you like almost a hundred X…”
Kant: Open weights alone do not allow developers to recreate AI models
“Weights are a binary. Let's call them what they are. Yes, we can modify it and we can change them, but like, Giving someone the weights does not allow them ultimately to recreate what you're doing.”
Kant: Industry will squeeze far more capability from smaller models via behavior
“We are going to be able to squeeze so much more out of smaller models than I think we had imagined in the industry. Because yes, there's intelligence and larger models are more intelligent. Like no doubt about it. We should continue to scale up. But the behavi…”
Kant: Pre-training compute runs are not the expensive part of AI
“The training run is not the expensive part. The training run is a very anticlimactic event, right?”
Kant: RL compute cannot scale like pre-training due to task batch constraints
“And RL is batch size constraint, right? So like you are ultimately in your batch size constraint because you don't have infinite tasks, right? When you've got the entire web, you can be much more flexible in scaling up your batch size because you've got the en…”
Wang: Data Quality and Scale Matter More Than Architecture or Priors
“Overall, if we have to give an order, my order would be the quality among scale of the datasets, and then the architecture, and then the prior knowledge.”
Beam: Science acts as an infinite token generator for AI
“Science is as an infinite token generator to train models at scale.”
Beam: Chain of thought is an unreliable narrator of model computation
“It actually thinks in latent space, it emits tokens. So, like, the chain of thought is often an unreliable narrator for what the model, the computation of the model is actually doing.”
Beam: Verified scientific reasoning traces lift models despite parameter disadvantages
“So like we have just seen incredible lift from showing the model that even if we're at like a parameter disadvantage relative to the frontier models, just showing it an experimentally verified reasoning, reasoning trace, you see just immediate lift when we do …”
Biderman: Manually partitioning LLM memory vs retrieval becomes unmanageable whack-a-mole
“And now the thing is, if you start manually, heuristically saying this is in, this is out, then it becomes a whack-a-mole. Every, every person in every enterprise has different data, and you can really very easily pick and choose what goes in and what goes out…”
Biderman: AI efficiency and frontier intelligence cannot be decoupled
“The point for me is, the principle is, any kind of, like, efficiency and intelligence, they cannot really be decoupled. Sometimes people think if you're building something that's more efficient, that can save you dollars, therefore you're not in the premium ca…”
Perszyk: Mitigating AI Homogenization Requires a Diverse Society of Models
“The only way to counter that, and again, this is throughout human history, throughout human evolution, is to increase the diversity of ideas, the size of the ideas, the size of the population, and the interconnectivity of the ideas. So rather than having indiv…”
Bubna: Speculative decoding accept length delivers multiplicative speedups over kernel tuning
“People talk a lot about, we made these kernels faster and whatnot, but improving kernel only give you like a few percentage points of improvement and increasing except length literally is a multiplicative decrease.”
Feinberg: High-throughput screening exhibits a shockingly low R-squared to actual synthesis
“The reality is that the translation of high throughput screens, whether it's Dell, DNA encoded libraries, or more traditional screens, the R-squared of those predictions to, like, the actual business of resynthesizing a molecule de novo and doing a Low through…”