The Ledger, every show

“My fundamental view is that AI has been on an exponential for the last 10 years and as part of a sort of Moore's law for intelligence, and that we are now well advanced on that curve, and there are only a small number of years for AI models surpassing the cogn…”

“I actually think that, you know, as the revenues, as the economic business of AI ramps up and it's ramping up exponentially, you know, if I'm right in a couple years, it'll be the biggest source of revenue in the world, right? It'll be the biggest industry in …”

“I do think when our AI system, when our AI systems get advanced enough, I suspect they'll have something that, you know, resembles what we would call consciousness or moral significance. I do think it'll happen at some point.”

“A lot of these models, particularly the ones that come from China, are optimized for benchmarks and are distilled from, ah, you know, from kind of the big US labs.”

“So we see the progress as being very fast, and the exponential is continuing, and we don't see any diminishing returns.”

“Anthropic is actually the fastest growing software company in history at the scale that it's at. So we grew from zero to a hundred million in 2023, a hundred million to a billion in 2024, and this year we've grown from one billion to, I think I said this befor…”

“I've said nothing that, that any, that anywhere near resembles the idea that this company should be the only one to build the technology. I don't know how anyone could ever derive that from anything that I've said. It's just, yeah, yeah, it's just a, it's just…”

“I think that next year we're probably going to see from multiple players models on the order of one billion dollars, and in 2025 we're going to see models on the order of Several billion, I don't know, perhaps even ten billion dollars.”

“If you scale up models, you give them more data, more compute. Again, there are a few modifications like RL, but not really very much. It's pretty close to pure scaling. You find that, you know, when you do that, you find, you know, incredible increases in per…”

“I think coding is going away first, or coding is being, you know, done by the AI models first, and then the broader task of software engineering will take longer, but I think that is, you know, that doing that end to end, I think that is going to happen as wel…”

“Static data is becoming less important and what we might call like dynamic data that the model creates itself is, you know, for reinforcement learning is becoming more important. So, you know, I don't think data is, is, is quite the most central thing anymore,…”

“Yeah, so the, again, the gross margins right now are very positive.”

“When we go out, when we're you know, competing against other companies for enterprise contracts, We see, just honestly and candidly, we see Google, and we see OpenAI. Every once in a while, we see a couple other US players. I have almost never lost a deal, los…”

“You know, I think there's maybe I don't know, 20 or 25% chance that sometime in the next two years, the models just start getting, stop getting better for reasons we don't understand, or maybe reasons we do understand, like, you know, data or compute availabil…”

“And from a, from an, from a machine learning perspective, from an AI perspective, there's no reason we can't make the context length a hundred million words today, right? Which is roughly what a human hears in their lifetime.”

“Without being too specific you know, I think, and we already have maybe some, you know, some evidence to suggest that this is another of those problems that is, is not as difficult as it seems. That will fall to scale Plus a slightly different way of thinking …”

“Our view is that if anything, the enterprise use of AI is going to be greater even than the consumer use of AI, right?”

“There's the cost of running, there's the cost of running the model, right? So, so for every dollar the model makes, it costs a certain amount. That is actually already fairly profitable.”

“Actually, the original reason for building GPT-II and GPT-III, it was an outgrowth of the kind of AI alignment work that we were doing, right? Where myself and Paul Cristiano and some of the Anthropic co-founders had invented this technique called RL from huma…”

“I think we are on track, even if there were no algorithmic improvements from here, even if we just scaled up what we had so far, ah, I think the scaling laws are going to continue, and I think that's going to lead to amazing improvements.”

“I think my basic view is that inference will not get that much more expensive. The simple, you know, the basic logic of the scaling laws is that if you increase compute by a factor of n, you need to increase data by a factor of square root of n. And size the m…”

“One broad way that we've been thinking about it from the beginning and are probably going to work more on formalizing it over the coming months and years is this idea of sort of safe scaling or checkpointing. So there could be this kind of alternating step whe…”

“Really the main thing holding back infinite context windows is just, you know, as you make the context window longer and longer, of course, the majority of the compute starts to be in the context window. So at some point it just becomes too, too expensive in t…”

“We have an unusual governance structure, something called the long-term benefit Trust. you know, it's a body that, that kind of ultimately appoints, you know, the majority of the board members for Anthropic and is, you know, made up of financially disinterest…”

“We've said that there should be regulation of AI when all the other companies and the administration have said there shouldn't be regulation of AI.”

“The regulation we've advocated for example, SB in California exempted everyone who makes under five hundred million dollars a year in, in in, in, in revenue, right? SB, SB was a transparency law, which you know basically requires companies to you know, to show…”

“You know, we did some studies around code and showed that, you know, depending on how you use the model, you know, we can see, De-skilling in terms of writing code, right? There are different ways to use the model, and some of them don't cause de-skilling, and…”

“I, I'm positive on, like, I think biotech is about to have a renaissance. Like, ultimately we'll be driven by AI. You know, I'm not going to name a particular company, but like you know, nor will I say whether I think it's better to bet on the big pharma compa…”

“We've seen more progress on say math and code where, where the models are, you know, getting pretty close to like a high professional Level and less on more subjective tasks. But I think that is very much a temporary obstacle.”

“I think the actual majority of code that's written, written at Anthropic is, you know, at this point written by, or at least with the involvement of one, you know, one of the quad models”

“And I would also say, if you look at the size of the data centers that we're building with, for example, Amazon I don't think our data center scaling is substantially smaller than that of any of the other companies in the space.”

“Two years from now, my guess is that we'll have models that cost of the same order of magnitude that they cost today, except they'll be much more capable of doing work, much more autonomously, much more broadly than they are capable of today.”

“We're getting more compute. We can scale up the models more and we can greatly increase the amount of data that, that is programming. So, so, so we have so many ways that we can amplify this. And so Of course it's gonna work. It's just a matter of time.”

“The most expensive models made today cost about a hundred million dollars, say plus or minus a factor of two.”

“I was one of the, like, co-inventors of that at OpenAI, but since then it's been, you know, improved to power ChatGPT”

“We've been able to find, you know, neurons that correspond to very specific concepts, neural circuits that correspond to, you know, keep track of how to do rhymes in poetry, and so we're starting to understand what these models Do, right?”

“So we raised, I think at this point, a little short of twenty billion dollars.”