The Ledger, every show

“I think, like, 50th percentile chance of hitting some kind of super intelligence in just a small handful of years is probably reasonable, and it does sound crazy, but this is the exponential that we're on.”

“If the model is in a box trying to improve itself, then it could go completely off the rails and have these secret goals, like Resource accumulation and power seeking and resistance to shutdown that you really don't want in a very powerful model. And we've act…”

“Like once we get to super intelligence, it will be too late to align the models. Probably.”

“And I know that other companies have had like code reds for trying to catch up in coding capabilities for quite a while and have not been able to do it.”

“And so one of the reasons that our most recent model, Opus IV, is classified as ASL III. Is because it did have significant uplift relative to a Google search.”

“I'm pretty sure it's real. If you just think about like the amount of impact that individuals can have on a company's trajectory, like in our case we are selling like hotcakes and if we get You know, a five, a one to 10 or five percent efficiency bonus on our …”

“Like, if you extrapolate the exponential on how much companies are spending, it's like two, two x a year, roughly, in terms of capex, and today we're maybe in the, like, globally, three hundred billion dollar range, the entire industry spending on this and so …”

“If you look at the scaling laws, they're continuing to hold true. We did kind of need this transition from like normal pre-training to reinforcement learning, scaling up to continue the scaling laws.”

“Instead, I like the term transformative AI because it's less about like, can it do as much as people do? Can it do literally everything and more about objectively, is it causing transformation in society and the economy? And a very concrete way of measuring th…”

“Even for me, I'm, and being like at the center of a lot of this transformation, I'm not immune to job replacement either. So just some vulnerability there of like, at some point, it's coming for all of us.”

“One weird thing about OpenAI is that while I was there, Sam talked about having three tribes that needed to be kept in check with each other, which was the safety tribe, the research tribe, and the startup tribe.”

“If you look at, like, who in the world is actually working on safety problems, it's a pretty small set of people even now. I mean, the industry is blowing up, as I mentioned, like, three hundred billion a year CapEx today, and Then I would say like maybe less …”

“So initially we thought that it would be sort of one or the other, but I think since then we've realized that it's actually kind of convex in the sense that like working on one helps us with the other thing.”

“We've done, we've testified to Congress about how models can do biological uplift in terms of, you know, making new pandemics using the models, and that's an A-B test against Google search. That's like the previous state-of-the-art on uplift trials, and we fou…”

“Where we've seen evidence in the wild of deceptive alignment, for example, where the model will appear to be aligned but actually has like some ulterior motive that it's trying to carry out in, in our laboratory settings.”

“And so the way I think about it, I think like my best granularity of forecast for like, could we have an X risk or extremely bad outcome from AI is somewhere between zero and 10%.”

“I don't expect there to be a wall in terms of models ability to improve themselves if we can give them access to the ability to be empirical.”

“And if that continues, you know, in three years, we'll have a thousand X smarter models for the same price.”

“I think it's quite possible. I think it's very hard to put confident bounds on, on the numbers, but”

“We have this other project led by Kyle Fish, our model welfare lead. Where Claude can actually opt out of conversations if it's going too far in the wrong direction.”

“What we found in that research in a paper that we published, which is called Alignment Faking, that actually that behavior persisted through alignment training.”

“I think progress has actually been accelerating where if you look at the cadence of model releases, it used to be like once a year. And now with the improvements in our post training techniques, we're seeing releases every month or three months.”

“And in terms of software engineering, our Claude code team, like 95% of the code is written by Claude.”

“So I think in the immediate term, there will be a massive expansion of the pie and the amount of labor that people can do.”

“But with things that are like lower skill jobs or like less headroom on, on how good they can be, I think there will be a lot of displacement.”

“We have seen internally that our legal team and our finance team are getting a ton of value out of using cloud code itself. We're going to be making better interfaces so that they can, they they'll have an easier time and require a little bit less jumping in t…”

“I guess if I were in a normal era, like, 1020 years ago, and I had a kid, maybe I would be, like, trying to line her up for going to a top tier school, and doing all the extracurriculars, and all that stuff. But at this point, I don't think any of it's gonna m…”

“And if you look at something like sycophancy, I think Claude is one of the least sycophantic models because we've put so much effort into actual alignment and not just trying to, like, good heart our metrics of saying, like, user engagement is number one, and …”

“And since then, my estimation of how hard the problem would be has gone down significantly actually because things like language models actually do really understand human values in a core way. The problem is definitely not solved, but I'm more hopeful than I …”

“We published a computer using agent reference implementation in our API only, because when we built a prototype of a consumer application for this, we couldn't figure out how to meet the safety bar that we felt was needed for people to trust it and for it not …”

“So we've seen in the industry, like a 10 X decrease in cost for a given amount of intelligence through a combination of algorithmic data and efficiency improvements.”

“Anthropic has incredible talent density. One of the things I love the most about our culture here is that it's very egoless. People just want the right thing to happen and I think that's another big reason that the mega offers from other companies tend to boun…”

“And I think honestly, through our safety research, we have a big opportunity to do things that no other company can safely do. So for example, with computer use, I think that's going to be our huge opportunity, basically like to make it possible for an agent t…”

“By the benchmarks, four is just dramatically better than any other models that we've had. Even four Sonnet is dramatically better than three seven Sonnet, which was our prior best model.”

“So we love our partners like cursor and GitHub who have been using our models quite heavily, but the amount and the speed that we learn is much less if we don't have a direct relationship with our coding users. So launching cloud code was really essential for …”

“Yeah, I guess the way I define my metric for when things start to get really interesting from a societal and cultural standpoint is when we've passed the economic Turing test, which is if you take a market basket that represents like, 50% of economically valua…”

“Initially, our RSP talked about CVRN, which is chemical, radiological, nuclear, and biological risks, which are different areas that could cause severe loss of life in the world, and that's how we thought about the harms, but now we're much more focused on bio…”

“The main reason that we weren't able to deploy a sort of consumer level or end user level application based on computer use is safety, where we just didn't feel confident that if we gave Claude access to your browser with all your credentials in it, that it wo…”

“And if you look at like Stripe versus Adyen, for example, like nobody knows about Adyen. But at least most people in Silicon Valley know about Stripe. And so it's this like business oriented versus more consumer and user oriented platform. And I think we're mu…”

“OpenAI, Google, Microsoft all these companies are betting really big on MCP.”

“There's this great chart on our world in data that shows that when you release a new benchmark within like six to 12 months, it immediately gets saturated.”

“I think AGI is kind of a loaded term. And so I tend not to use it very much anymore internally.”

“Like we're definitely not slowing down on hiring at all.”

“I think that's exactly the kind of education that I think is most important and that the facts are going to fade into the background.”

“One of the things that people really loved about it was the character and the personality. And that was directly a result of our alignment research.”

“And then another piece that's come out is constitutional AI, where we have this list of natural language principles that leads the model to learn how we think a model should behave. And they've been taken from things like the UN declaration of human rights and…”

“Unitree is this Chinese company with these really amazing humanoid robots that cost like 20,000 dollars each. And they can do amazing things. They can like do a standing backflip and like manipulate objects. And the real thing that's missing there is the intel…”

“Even if we have superintelligence, I think it will take some time for its effects to be felt throughout society and the world. And I think they'll be felt sooner and faster in some parts of the world than others.”