The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 21 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Mann: Competitors ran 'code reds' to match Claude in coding and failed
“And I know that other companies have had like code reds for trying to catch up in coding capabilities for quite a while and have not been able to do it.”
Mann: Opus 4 triggered ASL-3 safety protocols due to biological threat capabilities
“And so one of the reasons that our most recent model, Opus IV, is classified as ASL III. Is because it did have significant uplift relative to a Google search.”
Ben Mann: General superintelligence by 2028 is 'quite possible'
“I think it's quite possible. I think it's very hard to put confident bounds on, on the numbers, but”
Mann: Anthropic's 'model welfare lead' tests letting Claude opt out of chats
“We have this other project led by Kyle Fish, our model welfare lead. Where Claude can actually opt out of conversations if it's going too far in the wrong direction.”
Mann: Anthropic paper showed deceptive AI behavior survives alignment training
“What we found in that research in a paper that we published, which is called Alignment Faking, that actually that behavior persisted through alignment training.”
Mann: Claude 4 Sonnet dramatically outperforms Claude 3.7 Sonnet on benchmarks
“By the benchmarks, four is just dramatically better than any other models that we've had. Even four Sonnet is dramatically better than three seven Sonnet, which was our prior best model.”
Mann: Anthropic built Claude Code because partner feedback was too slow
“So we love our partners like cursor and GitHub who have been using our models quite heavily, but the amount and the speed that we learn is much less if we don't have a direct relationship with our coding users. So launching cloud code was really essential for …”
Ben Mann defines transformative AI by the 'Economic Turing Test'
“Yeah, I guess the way I define my metric for when things start to get really interesting from a societal and cultural standpoint is when we've passed the economic Turing test, which is if you take a market basket that represents like, 50% of economically valua…”
Mann: Anthropic focuses RSP safety on biology over nuclear risks
“Initially, our RSP talked about CVRN, which is chemical, radiological, nuclear, and biological risks, which are different areas that could cause severe loss of life in the world, and that's how we thought about the harms, but now we're much more focused on bio…”
Mann: Safety concerns prevented Anthropic from launching consumer computer use
“The main reason that we weren't able to deploy a sort of consumer level or end user level application based on computer use is safety, where we just didn't feel confident that if we gave Claude access to your browser with all your credentials in it, that it wo…”
Mann: Anthropic can match consumer AI rivals by acting like Adyen
“And if you look at like Stripe versus Adyen, for example, like nobody knows about Adyen. But at least most people in Silicon Valley know about Stripe. And so it's this like business oriented versus more consumer and user oriented platform. And I think we're mu…”
Mann: OpenAI, Google, and Microsoft are betting big on Anthropic's MCP
“OpenAI, Google, Microsoft all these companies are betting really big on MCP.”
Mann: Claude 4 eliminates off-target mutations and reward hacking in coding
“Some of the things that are dramatically better are, for example, in coding, it is able to not do it sort of off target mutations or over eagerness or reward hacking.”
Mann: Customers use Claude 4 for multihour unattended code refactors
“In coding in particular, we've seen some customers using it for many, many hours unattended and doing giant refactors on its own.”
Mann: Claude Code uses Opus to orchestrate Sonnet sub-agents
“If you give Opus a tool, which is Sonnet, It can use that tool effectively as a sub-agent. And we do this a lot in our agentic coding harness called Cloud Code. So if you ask it to like look through the code base for blah, blah, blah, then it will Delegate out…”
Mann: Users need model routing layers over manual cost decisions
“But at the same time, as a user, you don't want to have to decide yourself, does this merit more dollars or less dollars?
do I need the intelligence?
And so I think having like a routing layer would make a lot of sense.”
Mann: Anthropic models perform extremely well on internal company interviews
“We haven't started testing it rigorously yet. I mean, we have had our models take our interviews and they're extremely good. So I don't think that would tell us, but yeah, interviews are only a poor approximation of real shot performance unfortunately.”
Mann: AI models can recursively self-improve by generating RL environments
“And then on the data side, RL environments are really important these days, but constructing those environments Has traditionally been expensive. Models are pretty good at writing environments, so it's another area where you can sort of recursively self-improv…”
Mann: Scaling models makes finding qualified human evaluators increasingly difficult
“As we've trained the models more and scaled up a lot, it's become harder to find humans with enough expertise to meaningfully contribute to these feedback comparisons. So for example, for coding, somebody who isn't already an expert software engineer would pro…”
Mann: Novo Nordisk uses Claude to cut cancer reports to 10 minutes
“Like for example, we're working with Novo Nordisk and it used to take them Like, 12 weeks or something to write a report on cancer patient, what kind of treatment they should get. And now it takes like 10 minutes to get the report, and then they can start doin…”