The Ledger, every show

“I got this question a bunch with deep seek when deep seek came out, like, all right, what does deep seek mean for you? And I think there's things that we learned from on the tech side, just looking at what they were doing, but from a go to market and place in …”

“At this point, I would be shocked if it wasn't 95% plus.”

“I've had so many conversations with Alex Wang about this because he has this whole thing about how in the future most friends will be AI friends. And you know, I don't think he's wrong.”

“We found that there's not a lot of correlation between [10288] Mike Krieger: Like the person who's using the most tokens and like the person that I like, it's an interesting thought. I just made you at your companies like write down your 10 most productive peo…”

“We found that there's not a lot of correlation between Like the person who's using the most tokens and like the person that I like, it's an interesting thought. I just made you at your companies like write down your 10 most productive people that you think a…”

“Hey, I'm in the discord, the, you know, the, Cloud anthropic discord. I'm in the user for I'm on X and I'm reading things and like, here's what's emergent. That's step one. Models can do that today. Step two, which the models probably can do today. We just hav…”

“I think models over time get more different rather than more similar.”

“So I think it absolutely has to be a mix. And I think the best models will come from that combination of great, like for code it's, you know, being, having good foundational understanding of code and good examples, but then also being able to explore a really …”

“We are going to be building things that are general purpose as a rule with maybe some specialization at the, like, user level, but not at the, I don't anticipate us building a lot of verticalized experiences that are, like, fairly bespoke to a given workflow o…”

“So when I think about the coding space and where we can play and add value, it really is on the agentic side. It's not on the ID side.”

“Recognize that they're not yet at the place where for many use cases, you can let them kind of run free for hours. You need that more human in the loop piece.”

“How do we evolve from being mostly code writers to mostly Delegators to the models and code reviewers. That's what I think the work looks like three years from now. It's coming up with the right ideas, doing the right user interaction design, figuring out a de…”

“That, that's still a very human problem that I think we're at least three years away from the models being, being solving at that level of abstraction.”

“We have a feature called dynamic workflows where you can have it like also break down the task into like a lot of subtasks. And I trusted it to sort of not just do the individual action, but here's like a whole language conversion of millions of hundreds of th…”

“I moved much more to before going to bed, making sure that I had, like, queued up for Fable, like, enough chunky work to last. I would call it the whole night and I would check in later and it got it done in an hour and it was like, I guess, hanging out for th…”

“And I trusted it to sort of not just do the individual action, but here's like a whole language conversion of millions of hundreds of thousands of lines of code at that point. Go off and do it, go plan it, go execute it, go verify the work, double verify the w…”

“And then also out of that first incarnation came computer use. Computer use was different, though, because when we built it was really bad. Like we tried a bunch of products with it, and this was around, you know, Sonic three five and be like, Claude, can you …”

“So, 4.5 sine 4.5 basically outdoes Opus, our largest model in effectively every category, but does so while running faster and at a fifth the cost. So if you think about where we were in May at, you know, code with Claude, we were announcing, announcing Opus f…”

“In the long run, like overall, these products like might not, or probably will be doing more sort of automation or even replacement of work.”

“I still think there's like a tremendous amount of economic opportunity throughout. It just might be, you know, more smaller companies rather than fewer, bigger monolithic companies.”

“Rather than treat it as a sort of substitute for how the model might otherwise access information or sort of a system built on top of the model, we actually have trained it deeply into the model. And so the model knows about the concept of memory, which I know…”

“So this, you know, we had a customer and internally, we also got like a 30 hour plus kind of execution versus I think Opus four was seven hours.”

“He's like, I think we'll be at 90% by the end of 20, 25 or something like that. And sure enough, we're at about 72 now with the new models and. We're at 50% when you made that prediction and it's like continued to scale pretty much like as predicted.”

“More labs either choose to not show or otherwise obscure the chain of thought down the line.”

“The Claude three, we were training a model at the frontier that was state of the art with a team that was much, much, much smaller. Than any other lab.”

“Claude AI and probably ChatGPT.com were, like, very much, like, initially just built to be sort of showcases of the models and not really built in a lot of ways to be the right, like, The sort of foundational for like a much more complex sort of multi product …”

“That's like theme one that I'm extremely excited about. And I think if we do it right, should actually like transform all of our products like from head to toe.”

“I think if I pointed at something between Sonnet four and four or five, a lot of it really has been on the engineering side to just be able to scale up especially a lot of the post-training work.”

“We had one customer had it work for 30 hours. Of course, that's not going to be every task, but like, that's the kind of upper bound that we're starting to see.”

“I think people end up feeling more like managers of AI than just users of AI.”

“Our best engineers are managing three or four cloud code instances running at once.”

“Even when it was already outperforming Opus, for example, on sweet bench, people still didn't feel it was better, but then it continued to train and it was like now better than Opus and people don't want to switch back.”

“We really rapidly became bottlenecked on other things like our merge queue, which is the sort of sort of get in line to get your change accepted by you know, the system that then deploys it to production. We had to completely re architect it because so much mo…”

“I would expect that like a year from now, the way that we are like conceiving of building and shipping software just changes a lot because it's gonna be very painful to do it the current way.”

“I think things that are going to, I can't promise this as like a five to 10 year thing, but at least like one to three years, things that feel defensible or durable. One is understanding of a particular market.”

“I still think we are in, like, day one around, is AI an indispensable part of most people's work? And I think the answer is no.”

“Even, like, let's take within the labs, like, I assume every single one of the labs is using, like, even within themselves, like, it is very valuable to be able to take, you know, the knowledge of your highest-end model and then be able to make it higher, you …”

“People love talking about the, like the super app and we chat, and there was some technical challenges solved by those at scale that were of the same scale of challenges that Facebook was challenged was doing.”

“As a really, you know, specific example with cloud code within, you know, a week of it being deployed internally, we had found a way in which one of the sort of tools that it has access to the model wasn't using as well as it could have, and that made its way …”

“On Novo Nordisk used to take, I think it was something like 15 weeks to do their clinical trial reports, and now they use cloud and get it done in 20 minutes, and like, that's a step change.”

“In day one of Instagram, you know, there wasn't really a network there. You were there because the filters made your, you know, photos from your iPhone three G. Or three GS look a little better or a lot better”

“When we launched Explore and made it personalized rather than just like, most popular photos, like, that was a two X delta in usage.”

“We were the first company that Facebook acquired and sort of kept running, and then since then you've seen Oculus and WhatsApp, and, ah, I think there were a lot of things that we figured out early on in terms of the relationship, the office space, hiring and …”

“I started the labs, the original labs team with Ben Mann, who's one of the co-founders of Anthropic in my third week at Anthropic. And it had been something that I've been bubbling under. And at the time, the reason was really different. It was all of our prod…”

“So if you look at our model cards, for example, one of the ways we look at risk in the bio domain is comparing the uplift from sort of lay a lay person using the model versus, you know, an expert or a lay person just using the Internet and seeing what the comp…”

“These entities, these agents are going to start showing up in all places where you do work, whether that's your Slack or your Teams, for example.”

“We asked every Claude from Claude one to Claude, you know, 4.5 to recreate Claude.ai. So like our flagship AI products. And 4.5 was really the first one that was able to do it end to end and actually produce something of, you know, quality. It actually works. …”

“We're 13 at sale and 16 at close.”