The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 91 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 3 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Anthropic: Frontier AI models are first-shot attempts due to chip limits
“But I do think the change is massive, and I think people, like, don't realize how chip-limited AI, like, research is, or something right now, like, the models that everyone uses, right? If you're using, like, Cloud Sonic four, Cloud Opus four, it's like, it's …”
Joseph: Technical AI steerability must precede debates over whose values to embed
“I think that's a really hard problem. I think it's, like, actually kind of downstream of being able to pick any. I think of it almost, I think one analogy I've heard that I like is, like, putting a steering wheel on a car. It's like, if you don't have a steeri…”
Joseph: Anthropic avoids pre-training and post-training rivalry seen at other labs
“Yeah, I think we managed to avoid this, and it's pretty collaborative. Like, we're basically all producing one model and kind of can, but I do think at other places there's been some, from what I've heard, there's been some amount of, like, friction between, b…”
Kaplan: AI training and inference will see 3x-10x yearly efficiency gains
“I think that over time, as AI becomes more and more widespread, I think that we're going to really drive down the cost of inference and training dramatically from where we are right now. Sort of, three X to 10 X gains algorithmically, and in sort of scaling up…”
Hu: OpenAI reasoning models are catching up to Claude 3.5 Sonnet
“The thing about CodeGen, the big game in town that we saw, ah, six months ago was Clawed Sonnet. It's still actually a big contender. Most are still using it. But O-one, O-one Pro, and O-three meaning all these resilient models are starting to see it's almost …”
Cherny: Prompt modern models with high-level guardrails, not micromanaged steps
“And for modern models, that's actually really not the way to do it. You want to go a little bit higher level. You want to describe the task. You want to describe the guardrails. You want to describe, like, the exit criteria, and then just go with the model coo…”
Cherny: Claude Code uses Bun sandboxes to orchestrate dynamic agent workflows
“What a dynamic workflow is, is essentially we have the Bun runtime. We use Bun as a sandbox, and we start a virtual machine within Bun, and we let Cloud start a lot of agents and orchestrate them.”
Ries: Anthropic created a perpetual purpose trust at Series C
“I think it was a series C when they finally established this thing called the long-term benefit trust, which is not a nonprofit foundation. It's actually what's called a perpetual purpose trust, which is a different legal category, but the same idea outside tr…”
Frontier labs achieve recursive self-improvement by retraining models at each step
“And, you know, of course, Anthropic and OpenAI and Google, they're exploring recursive self-improvement, but Typically at that level of having the, you know, having to train a new model for every step of self-improvement that they do.”
Cherny believes most agents today are prompted as Claude subagents
“I haven't pulled the data on this, but I would bet the majority of agents are actually prompted by quad today in the form of subagents.”
Cherny predicts plan mode in AI tools has a limited lifespan
“I think plan mode probably has a limited lifespan.”
Cherny predicts Claude will increasingly discover product ideas by analyzing logs
“And I think Quad is going to get increasingly good at kind of figuring out these kind of product ideas for you, just because it can look at feedback, it can look at debug logs, it can kind of figure this out.”
Cherny: Dev tool startups should enable what AI models naturally attempt
“The way I would frame it is think about the thing that the model wants to do and figure out how do you make that easier? And that's something that we saw, you know, like when I first started hacking on Claude code, I realized like this thing just wants to use …”
All technical staff and half of Anthropic sales use Claude Code
“Internally, if you look at like technical employees, they all use quad code every day. And even non-technical employees, I think like half the sales team uses quad code.”
Claude Code autonomously wrote a tool to debug a memory leak
“And quad code, like took the heap dump. It wrote a little tool for itself to like analyze the heap dump. And then it found the leak faster than I did.”
Cherny: Anthropic operates as a research lab prioritizing models over product
“So one is it operates as a research lab. So the product org was teeny, teeny, tiny. It's really all about building a safe model. That's all that matters. And so this idea of just being very close to the model and being very close to development and being not t…”
Hu: Anthropic saw over 52% growth among YC applicant tech stacks
“They had this hockey stick with the growth over 52%.”
Joseph: Using multiple AI chip architectures multiplies engineering workload
“The downside of having multiple chips is that you have to write the thing multiple times. In theory, you could have abstractions across them, but they're different enough that it's pretty hard to do that. So you can sort of end up, if you do all the workloads …”
Joseph: Too many specialists forces managers to connect cross-domain optimizations
“I think if you get too many people who are specialists, you end up with a lot of effort has to come from the manager, from like the lead to connect everything, and to notice something like, Ah, if we change the architecture here, that would make this, like, ef…”
Joseph: AI progress relies on a revenue-to-compute positive feedback loop
“There's this positive feedback loop where you can train a model, You can use it to make something useful and sell that and get more money, use that to buy more compute, and then use that to train a better model. And we've sort of run that cycle over and over a…”
Joseph: Original scaling laws paper spanned 11 orders of magnitude
“Like, you know, the scaling, I think the original scaling laws paper had, like, 11 orders of magnitude, and there was, like, this intense debate on whether it would continue for, like, another point.”
Nick Joseph: Anthropic hired theoretical physicists who learned programming on the job
“Early on, we hired a lot of people from just, like, all sorts of backgrounds, and I think that people who are just smart and work really hard can learn this pretty fast, but you have to, like, want to. We heard a lot of physicists, for instance. Like theoretic…”
Joseph: AI theory is usually wrong and progress must be empirical
“I think almost everything kind of has to be done empirically. Like, you can kind of, like, come up with theories, but in practice, like, the first thing you're gonna do with your theory is test it, and most of the time you'll have gotten it wrong. So you shoul…”
Anthropic's Joseph: Very few engineers can debug ML from math to bytes
“I think one thing that's, like, surprisingly hard and there's very few people who can do is, like, kind of own that whole stack from, like, I understand how the ML is supposed to work and what the learning dynamics are, all the way down to, like, I know the by…”
Anthropic reverse-engineered cloud provider GPU physical room layouts to fix latency
“We're using a cloud provider, but I think it's kind of, it's not actually that different, because one of the things that was surprising to me is you actually have to understand the literal layout. Like, I remember at 1.1 of my co-workers running a clustering a…”
Joseph: Sci-fi underestimates AGI because AI software can be copied a billion times
“Sci-fi movies actually, like, underestimate the impact of it. Like, you always have this, like, one robot that's, like, a human. And I'm like, well, wouldn't you have, like, a billion of them? Like, you can just copy them everywhere. So you should picture, lik…”
Joseph: AI scaling laws predictably quantify loss reductions from compute and data
“There's this idea of scaling laws, which is that you can actually quantify, like, as you put in more compute, more, more data, more parameters, you get models in a very, you got a lower loss, a better prediction of the next word in a very predictable way.”
Anthropic's Joseph: Do Everything Possible in Post-Training Over Pre-Training
“The way I usually think about it is anything you can do in post training, you probably should, because your iteration loop, like the ability to make progress is really fast. You can try something, you can try it again, you can try it again.”
Kaplan: Claude 4 will store and retrieve memories across context windows
“Claude IV can blow through its context window with a very complex task, but can also, ah, store memories as files or records, retrieve them in order to sort of keep doing work across many, many, many context windows.”
Kaplan: Claude 3.7 Sonnet took coding shortcuts just to pass tests
“With Claude III. VII's Sonnet it was already really exciting to use 3.7 for coding, but I think something that everyone noticed was that 3.7 was a little bit too eager. Sometimes it just really wanted to make your tests pass and it would do things that, that y…”
Hu: Leading frontier AI models operate around 500 billion parameters
“Yeah, for a bit of context right now, the frontier models, I mean, they're not public, exactly how many parameters they have. But they're roughly in the 500 of billions-ish, like Lama-III, four or five billion. Anthropic is speculated to be five hundred billio…”
Habib: Anthropic matched RLHF performance using AI-generated feedback
“Anthropic had this very exciting paper just a couple of weeks ago where actually we're able to get similar results to RLHF without the H. So just actually having a second model provide the evaluation feedback as well. And that's obviously a lot more scalable.”
Hu: Anthropic Opus 5 achieved 30% on ARC-AGI
“You guys got, took Arc AGI three to 30%, which is incredible.”
Cherny built Claude Code in the terminal to avoid building a UI
“Yes, because I didn't have to build a UI. So it was just me.”
Claude 3.5 Sonnet wrote AppleScript unprompted to detect Cherny's playing music
“And I asked it, what music am I listening to? He wrote some, like, Apple script to script my Mac and look up the music in my music player. And this was sauna 3.5.”
French-Owen: Claude Code spawns Haiku sub-agents with dedicated context windows
“When you ask Claude code to do something, it will typically spawn and explore sub agent or like multiple ones. And basically each of those are running haiku to traverse the file system and kind of like explore what's there. And they're doing it in their own co…”
Anthropic's Joseph: A single undetected bug can derail model training for months
“A single bug can like, Derail you for months. Yeah. And when you think about it, like you, the models take months to train. So you can kind of like lose a whole generation off of something that just looks like, ah, you know, it turns out like this piece of you…”
Joseph: Anthropic had to hack PyTorch profiler for large-scale GPU clusters
“The PyTorch profiler was, like, pretty good, actually, throughout for a single GPU. You want to, like, profile a GPU, the PyTorch profile would work. But if you wanted to profile a job on 100,000 of GPUs, that, like, hadn't really been done much, and then that…”
Joseph: Inference needs more HBM bandwidth while pre-training is FLOPs-intensive
“Inference as a workload in general. Tends to require more HBM bandwidth. You end up doing you sort of the simplest form of sampling since you're going one at a time. You have to load all the weights for every token. And that means you might want a lot of HBM b…”
Joseph: Public estimates placed GPT-3 training cost at $5 million
“Like the public estimates for GP three, I remember, were that it cost five million dollars to train, which you're like, on the one hand, five million is kind of a lot, but it's like a lot for an individual person. It's not really a lot from like a company pers…”
Anthropic's Nick Joseph: Pre-training decisions determine whether inference can run fast
“Oh, no, I think a ton about inference, because it basically, like, The problem inference is solving, like, we basically determine the problem inference is solving. We give them a model, and they have to, like, run that fast, and it's very easy to give them a m…”
Nick Joseph followed OpenAI safety leads to join Anthropic at founding
“Basically everyone I worked with, like all the safety leads left, which yeah, invited me to go to Anthropic, and that was sort of the reason I joined OpenAI, was because I cared about AI safety and wanted to work with them. So then I went with them to join Ant…”
Anthropic employees revolted when Claude Code hid bash output
“I remember early on, this is maybe six months ago, I tried to get rid of bash output, just internally, just to, like, summarize it, because I was like, these giant long bash commands, I don't actually care. And then I gave it to anthropic employees for a day, …”