The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 1,824 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Feinberg: Robotic lab automation trades off molecule novelty and quality for speed
“The kinds of chemistries that today can be automated are fairly constrained. And so your ability to search chemical space broadly for those really top top Pareto optimal compounds is, is actually very limited. So the benefits you get in speed have a very harsh…”
Cohen: Semantic retrieval cannot answer broad weekly focus queries for agents
“There's no retrieval-based search, semantic search, keyword matching search that's going to be able to give your agent that information.”
Matei Zaharia: AI agents need stateful contextual security policies, not binary permissions
“A lot of coding agents today have very basic things like you can tell me which tool patterns I'll allow or disallow or whatever. It's like yes or no, but that puts you in a very tough spot. So just as an example, like, should my agent be able to read, you know…”
Xin: Single HTAP database engines compromise ecosystem compatibility and performance
“This is sort of the holy grail of database engineering is, why not build a single system that can do both of this? But it ends up just being a lot of compromises. And one, I think one of the first issue is that, hey, each, they say Postgres has a massive ecosy…”
Xin: Unifying storage delivers 99% of HTAP database benefits
“HTAP wants to build a single engine for both. We think you can get 99% of what you need by unifying the storage and just have a single storage layer.”
Xin: Overfitting to Initial Customers Has Far Lower Downside Than Boiling the Ocean
“I think the industry has a sense of, hey, maybe if you overfit to like one or two customers, it's going to be really bad for you. But I think the downside overfitting is much smaller than the upside itself. And if you sort of try to be too ambitious and boil t…”
Zaharia: Scaling down from bulk ingestion to serving is easier than scaling up
“It turned out that, you know, it's easier to go from that bod thing that's really good at the scale and ingesting and super low cost and create versions in it that have the speed and features of the, you know, super easy to use, like smaller data for business …”
Kolter: Scaling model size does not automatically improve AI safety or red teaming
“Traditionally this has been an area where both in terms of safety models don't get better by just being bigger, unlike most other areas where models do get better by being bigger. Safety has not been like that traditionally. You know, you have to train them ex…”
Kolter: AI is an alien intelligence with completely distinct failure modes from humans
“It is clearly a different form of intelligence than people. It's some alien intelligence that is vastly different, and that difference is actually often brought out to a large degree by things like adversarial attacks and red teaming, because there are certain…”
Kolter: Full experimental observability has not produced fundamental understanding of AI
“It's like we could kind of run experiments on the brain, observe every neuron in it, reset its state to prior states, and run counterfactuals, none of which we can do with humans, and yet we still understand neither very well. Even with that, all that ability,…”
Kolter: Adversarial Red Teaming Is Essential for True Capability Elicitation
“One of the most effective ways of doing capability elicitation is actually through some amount of what you would call red teaming, right? So if a model refuses a task because it thinks it's being evaluated, but it knows how to complete that task, getting it to…”
Fredrikson: Agent Guardrails Should Block Policy Violations, Not Injection Payloads
“If you parse some untrusted content and there is like a prompt injection, you know, something that's clearly trying to get the model to do a bad thing, you might be interested in knowing about that, but you don't necessarily like want your cloud code that you …”
AI diagnostic adoption is blocked because legal liability remains with doctors
“The problem remains then and now is regulatory because you actually can't shift the burden Of the wrong clinical diagnoses from the physician to the AI system.”
Midha: AI Labs Raising Too Much Money Too Fast Develop Fragile Cultures
“And I think teams who can raise too much money too fast, too early, who don't have to define what the P zero is, because that's the only thing when you have scarce resources, you gotta invest in. Those cultures end up being the most fragile and brittle, and th…”
Krause: Materials industry is constrained by experiments, not compute
“We're not compute constraint in the materials industry. We're experiment constraint.”
Awais: There is only one type of AI agent, the coding agent
“There's only one type of agent, and that is a coding agent. It can do it all, right?”
Awais: Open LLM coding failures are harness issues, not model issues
“So I feel like this always ends up being a tool call, a hardness issue. Then, you know, an actual model issue.”
Awais: Interactive permission prompts degrade coding agent model performance
“If you run any coding agent with permissions on, The models are actually number. And if you run them without, you know, the complete bypass of permissions, they do much better. Even if you like sit through those yes, yes, yes, accept or whatnot, you will see t…”
Awais: AI design slop is a contract gap, not a model capability deficit
“Feels like you can fix 90% of a design slop, which is not a capability gap. It's more like a contract gap in what your harness is telling an LLM to do versus what your user is saying.”
Petersson: Percentage-based AI benchmarks saturate with noise above 92%
“Even when you're not at a hundred, I think a lot of these evals have a lot of problems in them. So, like, actually, it's, like, if you get to, like, 92 or something like that, many of them, it's, like, then there's, like, there's no, really no difference betwe…”
Petersson: Multi-agent conversations inevitably converge to default helpfulness over time
“My hypothesis is that like deep down, they are still helpful assistants. That's what they're trained to be. And even if we prompt it super hard, that's what they are. And when they spend like a few hours just back and forth talking with each other then like, B…”
Hong: Structured and formal data enables broad horizontal transfer learning
“If you have more structured and formal data, it's going to be a lot more horizontal than the specific vertical we are tackling.”
Hong: Formal verification in AI is about scaling superintelligence, not bug fixes
“It is not about, like, formal verification or verified AI to us. It's not just about handling or, like, kicking out the lousiness, the hallucinations, the mistakes. It's about scaling brilliance. It's about super intelligence.”
Hong: Scaling inference for formal math has almost no wall
“I think that we found scaling inference to have almost no wall recursively decomposing you know, approved goal into many sub goals and then learning to backtrack as well.”
Hong: Lean and Rust yield superior reinforcement learning convergence over Python
“If you want proof to be informal math, It's very annoying, because then that's, like, just makes objective function. Your code is something like Python, your proof is, say, natural language, math proof. You will not have very strong RL kind of performance, rig…”
Nadella: Managing multi-agent coding requires completely rebuilding IDE interfaces beyond chat
“Coding has worked so well that we now have to rebuild the IDE, right? I mean, it's kind of nuts to see what we launched is like, oh my God, I have these hundred agent sessions. I, the cognitive load, it transfers back to me as a human is so excessive that now …”
Nadella says private evals and execution traces are a company's biggest IP
“So that's why I would say every company having private evals may be the biggest IP, right? I think about it like what's that private eval that you can then use even a frontier model to hill climb on and not leak the traces. Maybe one of the biggest drivers of …”
Nadella: SaaS companies must unbundle data and logic to find new models
“So I think the challenge of the SAS business model is we packaged one way. We now have to learn how to unbundle these things and rebundle in new ways and discover new business models, right?”
Nadella: Customers reject outcome-based pricing once outcomes materialize
“Most people love outcomes until they have an outcome. Because once you have an outcome, it's like giving away royalty, right? I mean, I've talked to customers who love, you know, outcome-based pricing, and I say, I'm all in until they, oh my God, like, what ar…”
Daigle: Replacing pull requests is hard because code review codifies social trust
“I think the reason why there's not a single answer is ultimately we're trying to codify trust. We're trying to say like, okay, if Sean reviews this, I'm going to trust it because you're Sean or you're the senior dev or you're the whatever. And right now when w…”
Ethan He: Pipeline bug fixes drive more model gains than new algorithms
“And often I find that this is kind of boring, but like a lot of the improvements does not come from new algorithms. It comes from finding small bugs here and there in the data pipeline, in the model training pipeline. Those gave the biggest boost to the model …”
Ethan He: Training video models costs roughly the same as medium-scale LLMs
“So surprisingly video models is like the cost is very, is comparable to language models. And obviously the largest scale is language model. Maybe like a medium scale language models.”
Ethan He: Visual intelligence in video generation models stems primarily from language models
“The visual intelligence are actually mostly coming from language. Like, these video models, especially from now, since the diffusion model technology is more mature, the, like, every time you see there, there's some improvement on these models, I would say mos…”
Murray: Standalone agents cannot compete against sandbox and model layers
“I don't want to compete for like 20 dollars a seat. I think that that is just a really difficult business. I think it's very easy to copy the main pieces of it. I mean, again, like I built this fairly quickly, and I think because you are not owning, I guess, t…”
Yan: Getting agent integrations right often requires bespoke builds over MCP
“The thing we found is, so like MCPs obviously it has been like, it's like really big explosion of, oh, you can go like integrate it with all these different things. But to actually get the integration right and get the right experience, oftentimes we found tha…”
Yan: Practical multi-agent setups require isolated sandboxes without shared machines
“We've actually given Devon an MCP so they can just go arbitrarily message other Devons and create new Devons, et cetera. But I guess like it somehow creates like a really chaotic world in that sense. And so we, we've still found that most practical use on a da…”
Murray: Codebases regress to worst engineer as AI multiplies bad patterns
“The meme that I have is that your code base regresses to your worst engineer because that engineer who is, you know, very gung ho about AI and is not auditing their code, their pattern starts cementing into the code. And now the AI is referencing their pattern…”
Rives: Curing disease requires personalized computational models, not conventional pills
“What is, you know, what is the cure to disease look like, right? It's not a pill, right? It's not a medicine in the conventional sense. You know, it's going to have to be a system that is capable of modeling and understanding, you know, the underlying physiolo…”
Burazin: Infrastructure built for human developers fails for AI agents
“Most people thought it was the same infrastructure for humans and agents. We understood a quarter ago. It's not, we just didn't know what was the right primitive.”
Burazin: AI agents need persistent, stateful computers like human laptops
“Agents will be like humans in the sense of you don't want your laptop to be shut down until you're done with work. Like, and you want to close the lid and open the lid. It's the same state. So agents would want that like pause and come back. They want those tw…”
Burazin: GitHub introduces too much overhead for AI agent inner loops
“What we saw from our customers was that they were all trying to figure out how to do versioning. Everyone is doing it in different ways. There were some really weird ways where people were doing that. And the reason was that GitHub as is Was an overhead. Like …”
Cooper: AI coding agents prefer complex CLIs with hundreds of flags
“Things that were prohibitively annoying to humans are not actually prohibitively annoying to agents. They're really, really nice, right? And so, if I wanted to hand you a CLI and I said, hey, guess what? The CLI has 40 arguments and. Right. 600 flags. You'd be…”
Cooper: Temporal requires holding full workflow states mentally, risking replay bugs
“Temporal was always like really, really, really great in theory. And it was really, really great when you got it working the way that you wanted to in production. It's just, it required you to like model that entire journey in your head. And if you didn't have…”
Cooper: Developers can treat servers as pets with instantaneous cloning
“I think you can move towards having pets so long as you have a, and this is gonna be a jump, so long as you have a cloning machine for your pets. If you can snapshot every single thing at every frame, then, like, It actually doesn't matter if, you know, that t…”
Cooper: Two is the worst number of co-founders due to deadlocks
“And this is why I was saying like two is the worst number of co-founders is because you have no tie break, right? You basically are like, well, I disagree on this thing and I disagree on this thing, right? I was like, well, how do you resolve that, right?”
Azhnyuk: Wartime survival demands building weapons before calculating ethics edge cases
“When you're in a situation where you're in a forest in front of a wolf, you know, you first gonna deal with the wolf that wants to eat you, and then you're gonna go consult Greenpeace. So that's kind of situation that Ukraine is in.”
Azhnyuk: Full autonomy increases FPV drone capability by 10,000 times
“Such full autonomy increases the capabilities of an FPV drone, which is already, like, three orders more powerful than an artillery shell. Full autonomy increases its capabilities by four orders of magnitude, because now you can have a hundred times as many pe…”
Azhnyuk: Warfare has shifted from multi-billion platforms to rapidly iteratable mass
“It's no longer about an aircraft carrier that costs whatever, fourteen billion dollars, and takes forever to build. It's about mass, ah, that is, you can iterate on very quickly, you can upgrade it, everyone can operate it, and then that mass, when it is combi…”