The Exchanges

Q Now, what if a parent were going to say, you know, Andy, this sounds good, but, uh, we all know that Sort of to be competitive or relevant today. Um, you know, you're living in a Google world, and so would it restrict my kid to, like, not give them access to all that Google tooling?

A Well, uh, it's actually giving them the choice. Uh, you know, uh, Life is long and kids take a lot of time to grow, right? Uh, take some, uh, 18 years before they become fully formed adults in, in many cases. Uh, once you put them in the Google ecosystem, you can't take them out, right? They're there for, for good. Uh, there's nothing that, you know, prevents your child in 15 years from making a decision that actually they don't prefer the perfect ecosystem, they want to be in Google instead. Uh, but they can make that decision on their own, fully conscious of the risk that, that entails. So it's not either or. It's just to give an additional option to children. So if they decide that they want to live outside of big tech in the future and have an independent, let's say, future away from big tech, they have that possibility. And I think this is a powerful option that you're giving your children for the future.

AI assessment note: “Well, uh, it's actually giving them the choice.”

Q Do people use those apps within ChatGPT? Like, remember, like a couple years ago, there was all this hype, like, you'll be able to order an Uber right from ChatGPT. I don't know anyone that's done that.

A Yeah. I think the usage has been pretty minimal so far, and I think the implementation, Has been slightly awkward. Like, I think it'll get better over time in terms of how to use it in that a lot of the times the apps break or they don't work. Um, my, like, bold case vision for this would be it's valuable for you as a consumer to have a source of memory and context on yourself. Similar to kind of like a login with Google, Sam has said they're gonna launch login with ChatGPT. And so then that means that Maybe you're not ordering an Uber through ChatGPT, but any other product you can authenticate through, and it can borrow your tokens, it can borrow your memory, it can borrow everything that it knows about you from ChatGPT. I think that is probably more of where we're headed versus, uh, solely using every app in the ChatGPT interface. Um, I love the idea that like in two, three, five years, Onboarding to software should not be a thing. Like, you should be able to log in with a ChatGPT or a Claude, and that new software product should know everything about you and, like, set up perfectly to cater to you, and that's really exciting.

AI assessment note: “I think the usage has been pretty minimal so far”

Q Are you seeing founders come in from like areas you geographies, backgrounds you never would have imagined? Like, is there like, just to get like some, you know, a six year old guy who's been at a company his whole life and never met that developer and now is just clawing his way through.

A Yes, definitely. I think we've definitely seen some geographies pop up where we haven't seen huge tech hubs before. I would say Paris is one. Um, Stockholm is another with lovable and now a whole kind of other wave of companies. Some of those founders do move to SF just because of the, the talent density. I think in general though, like pre AI for if you were building, especially an enterprise business, say software for HVAC, like the people you would want to back in that market is the guy who has done HVAC You know, knows the market inside out, built a company there before, and actually what we're seeing now are the better bets are like the very scrappy high hustle teams that will be able to keep up with the pace of model development and continue to productize the models in the most compelling ways and ship them to customers. Like, so it's a different, ah, maybe archetype of founder that's kind of winning now.

AI assessment note: “Yes, definitely. I think we've definitely seen some geographies pop up”

Q again. You had a really funny, funny, uh, piece about that. You said it's almost like Apple's having some major issues with their AI implementation and strategy. They should probably look into that. But it just keeps happening, right? That, that this keeps getting delayed and, you know, you start to lose faith over time, even with the Google partnership, that they're going to be able to figure this out.

A Yeah. I was always like a little bit skeptical. I mean, I've obviously been super skeptical of Siri over the past, having used it over the past, but like when they announced the Google partnership, I was always a little bit skeptical of the initial rollout. Cause it's like, how are they going to, it's sort of what we're talking about with the government. Like, right. Like you can't just swap these things in. It may seem like it's, it's that simple, but like, there's a lot of like underlying things that need to be connected. Look at Amazon for an example of that, right? Like, look how long it took them to, to rework Alexa, to be able to To work with things like anthropics models and, and all the models that they're using behind the scenes to sort of upgrade Alexa. It took over a year and then they promised something and they couldn't deliver on the timing of it. And now we're seeing the same thing. We've seen the same thing play out with Apple. It just takes a long time to get like all of the little pieces, uh, in place because the last thing Apple can afford to do right now is put something out there, even in beta. I think even in some sort of, you know, like thing where it's any, any forward facing user facing, um, Uh, service, um, and just have it flop again. That would be just a death now. I think to, they would have to change the Siri name at that point. You would have like…

AI assessment note: “I was always a little bit skeptical of the initial rollout.”

Q again. You had a really funny, funny, uh, piece about that. You said it's almost like Apple's having some major issues with their AI implementation and strategy. They should probably look into that. But it just keeps happening, right? That, that this keeps getting delayed and, you know, you start to lose faith over time, even with the Google partnership, that they're going to be able to figure this out.

A Yeah. I was always like a little bit skeptical. I mean, I've obviously been super skeptical of Siri over the past, having used it over the past, but like when they announced the Google partnership, I was always a little bit skeptical of the initial rollout. Cause it's like, how are they going to, it's sort of what we're talking about with the government. Like, right. Like you can't just swap these things in. It may seem like it's, it's that simple, but like, there's a lot of like underlying things that need to be connected. Look at Amazon for an example of that, right? Like, look how long it took them to, to rework Alexa, to be able to To work with things like anthropics models and, and all the models that they're using behind the scenes to sort of upgrade Alexa. It took over a year and then they promised something and they couldn't deliver on the timing of it. And now we're seeing the same thing. We've seen the same thing play out with Apple. It just takes a long time to get like all of the little pieces, uh, in place because the last thing Apple can afford to do right now is put something out there, even in beta. I think even in some sort of, you know, like thing where it's any, any forward facing user facing, um, Uh, service, um, and just have it flop again. That would be just a death now. I think to, they would have to change the Siri name at that point. You would have like…

AI assessment note: “It just takes a long time to get like all of the little pieces”

Q public sector, you saw a jump from 77 to 88% in accuracy for complex tasks. Healthcare saw a jump from 60 to 78%, and legal saw A jump from 57 to 69%, uh, uh, accuracy on complex tasks. Uh, that's pretty, pretty big. It seems like this model has, has almost been under hyped. Uh, can you talk a little bit about these, these jumps and what the significance is?

A I think, I think probably the, the main takeaway should be that, that the progress of these meaningful jumps that we've been seeing in AI coding Over the past couple of years where, you know, the model at best could do a couple lines of code You know, in a, in a kind of type ahead type format two, two and a half years ago in, in coding space. And now obviously people are giving the model a task of, you know, write me tens of thousands of lines of code for a full project. And, and we've just seen this incredible rate of progress and this March, uh, up toward, you know, more and more capability over time, uh, with, uh, within coding. I think that same trend is going to come to other Other now fields of knowledge work. And so, so this jump in sonnets model from four or five before six, I think represents an example of what happens when these models just get trained across more areas of knowledge work. What happens when they are getting better and better at reasoning capabilities that go beyond coding? What happens when they get better at using tools and deciding when to use tools? And that's what our complex work eval You know, is, is meant to represent is, is sort of how does it think through a problem? How does it decide it's got the right answer? How does it check its work? Um, and these models are getting much better at, at being able to deliver on that. So I think that'll be …

AI assessment note: “I think that same trend is going to come to other Other now fields of knowledge work.”

Q teams. The mission alignment team was created in twenty-twenty-four to promote the company's stated mission to ensure that artificial general intelligence benefits all of humanity. So of course, yeah, of course it makes sense to, you know, disband that one. Uh, they had also had, like, super alignment, which was also disbanded. Steven, you were close to this, uh, To this stuff. Uh, what is the, the implications on that?

A Seems pretty bad. Wish I were more surprised. Like, um, you know, at the end of 20, 24, which was when this team existed, open AI had announced plans to convert from a nonprofit to a for-profit in what seemed to me to be like pretty egregiously in violation of their commitments to the public. Um, and you know, they ended up Having to do a softer version of that because the attorneys general of California and Delaware got involved. So they didn't ultimately do something quite, quite so bad, but it's like, there's huge pressure on them. You know, they're planning to go public. Um, Josh who leads the team is a longtime friend of mine or who led the, the team, the mission alignment team is a longtime friend. I think really highly of him. I think that he sees the issues with AI very clearly. Um, You know, it does not surprise me that this is not quite so welcome at open AI any longer.

AI assessment note: “Seems pretty bad. Wish I were more surprised.”

Q Say, say more about it. Talk about what do you think is not being done that could be done?

A I see it every day. Uh, I mean, and I'll, I'll, I'll open up a little window. Like one of the reasons I was super excited about joining Cohere is because it's one of the few places that, you know, that we have a team that does research. So I get to see, you know, day to day what's happening in research. We have a team that does modeling. So I get to see the models that were built and look at the evaluations, a full spread of evaluations, and we have a product. That's product is an agentic platform that is going to real clients. So you get to see the whole thing, and I see something that our models can do, and I see some things that we've built into the products, and then we go, and there's a lot of customers that are not using the full functionality for all sorts of reasons. Um, so I think like that, that between like what we have in terms of capacity versus what's being deployed right now, there's a big gap between that. Sometimes the reasons are, um, are, uh, capacity questions. Like a lot of actual, we talk a lot about super intelligence, big models. In reality, paying customers want like a good trade-off in terms of performance for efficiency. So, you know, we'll train bigger models, but we'll deploy smaller models because it gives us that trade-off. It's like good enough intelligence to get the job done. And I'm like, well, we could give you so much more. No, no, no, it's …

AI assessment note: “between like what we have in terms of capacity versus what's being deployed right now”

Q It's your words. Um, so there's so much attention to quantum, like we talked about earlier, and there's a lot of hype, there's a lot of stuff that, you know, maybe we, we talked about earlier, people are putting out there, not so credible. How do we sort the hype from the truth?

A That's a hard question to answer. Um, at the end of the day, I think you need to, uh, ask questions and, um, kind of apply judgment. If you hear ridiculous statements, like numbers like, I don't know, you know, A million cubits in three years. I mean, it doesn't quite even pass the red face test. But I think it boils down to asking questions. If, if you've got somebody, um, kind of giving a presentation and throwing out what you think are outlandish numbers, and they won't let you ask questions, That would raise a red flag in my mind, right? You need to be able to ask questions. You need to be able to drill down. You need to be able to come up with an informed opinion. Get academics engaged in the discussion to help evaluate what you're hearing. Not enough of that is going on right now. I think there's this tendency to just let people say whatever they want to say, and, you know, maybe it'll all be good for the industry. I don't think that's the right answer. I think, you know, empty promises are, create problems, right? And we must, we must hold everybody accountable for, you know, what they're saying and what they're doing, and there has to be proof points. When, when we talk about something like quantum supremacy, we publish the paper. We publish the data. Anybody can go out and recreate the results. Right? Um, you know, you, you have to just make sure that there's data to s…

AI assessment note: “you have to just make sure that there's data to support the claims”

Q mean, how, thinking through the exponential, do you think that they could do that again? I mean, I think they got close to ten billion last year. They didn't quite 10 X, but they got close. Do you think they could get in the range of a hundred billion? Because as these numbers get bigger, the task to 10 X becomes harder. It's not like an easy thing to do.

A They're certainly on the right track to get there. You know, they, they've got a lot of interest. They've embedded themselves in both very big corporations, but they also have like a, you know, dedicated fans in like, you know, the developer community and like individuals. So if you think about the revenues of some of the big tech companies now, you know, Google's not far off generating that much net profit in, you know, half of that in a quarter, right? So if you believe that OpenAI and Anthropic are the next, you know, you know, the next members of the Magnificent Seven or the Magnificent Nine, there's no reason why they can't get to a hundred billion in annual revenue. Now, of course, what has to happen for them to be viable companies long-term is that their costs have to come down. Because remember, these startups, Don't actually make any money at the moment. Uh, and if they are going to go for IPOs this year or next, uh, you know, and they're hiring advisors, bankers, and lawyers to do so, eventually, you know, the whole of their balance sheet will be laid bare and people will be able to take, take a better, take a better look, uh, under the hood and determine whether they're viable long-term businesses. But certainly Anthropic is, is going about things in the right way to secure its funding, like lock in, lock in more strategic partners long-term. So yeah, I mean, I think…

AI assessment note: “there's no reason why they can't get to a hundred billion in annual revenue.”

Q So one, one more question about this. Do you think this is AI related as in, do you think AI is like automating Amazon employees jobs, or do you think this is a financial decision that might make room for some of the AI investment?

A I think it's a little bit of both. I mean, it always is. They, they talked about thinning out layers of bureaucracy as every company does when they make cuts. Um, but I also think that they're seeing some of the, you know, Employees that have been able to integrate AI into their daily work life and become more productive. I think they're kind of looking ahead at that. Um, we don't have the exact demographic data of who they've let go, but I think it'd probably be more heavily weighted into the older employees that, you know, are maybe less able or less willing to adapt to this new AI powered way of working. Um, and certainly if you look, well, we get Amazon results next week, but there's no slowdown in momentum of the company's growth. So without, even though they've been taking all these employees out of the company, you see a short term hit because you have to pay all of them severance, but longer term, your cost base comes down dramatically.

AI assessment note: “I think it's a little bit of both. I mean, it always is.”

Q trying to figure out whether all these stories of people building their own custom software, uh, actually will, uh, Actually will lead to the change that many of them are promising or, uh, or whether there's actual meat behind this, you know, is Dave Clark's CRM going to fall apart in a week? And it was just a nice post. For LinkedIn engagement or is something real actually happening here?

A I'm quite optimistic about this trend. Um, I actually think the term vibe coding will be like information superhighway where it's a term we don't use in the future because the idea that your software is something that you can change yourself will be something we expect rather than a novel concept. I have two things that sound contradictory, but I don't think they are. So first is Most of the, the cost of software is in maintaining it, not building it. Uh, and that's why most people would prefer to buy a solution off the shelf because you want to amortize the cost of maintaining software among thousands of clients and not have everyone bear it. Just think your ERP system and a new accounting standard comes out. If every company in the world has to go vibe code that new accounting standard, someone's going to get it wrong.

AI assessment note: “I'm quite optimistic about this trend.”

Q don't think they've confirmed that number is a good bet that we will make the money. Although the money, of course, you know, it would be funny if they also had another chart, what they paid for that compute, uh, because the money is certainly still a drop in the bucket compared to the compute they're buying. So what do you think about this? Obviously it's time to the fundraise.

A No, no, I, I agree, because it was, I think it was an incredibly well-crafted corporate communication, because it laid out this really clear story, like, one-to-one ratio of growth in compute to revenue in terms of, like, nine xing, and, and on face, I agree, it sounds nice, it sounds really good, it's saying, like, we're actually scaling compute in parallel with revenue, but you're right, it's, We know they're losing ungodly amounts of money. Even this week we saw Anthropic, I think it was 5.8 billion. They're on pace to lose 5.6 billion last last year is now confirmed is what was lost. So like, and we, everyone knows this and has been talking about it, that these companies have been burning ungodly amounts of cash. Um, so when you dig in one level deeper, I think it doesn't really answer the question, but It's still, I thought, was a pretty, kind of, craftily put together argument of why they actually are ok, but, yeah, it doesn't, it still doesn't add up overall.

AI assessment note: “I thought, was a pretty, kind of, craftily put together argument”

Q That there have been a lot of tricks that have been put on top of LLMs. Um, I hear often about scaffolding and orchestration and AI that can use a tool to search the web, but it won't remember what it learns. As soon as you close that session, it forgets. Is that just a limitation of the large language model paradigm?

A Well, look, I think there is, and I'm definitely a subscriber to the idea that maybe we need one or two more big breakthroughs before we'll get to AGI, and I think they're along the lines of things like continual learning, better memory, longer context windows, or Or perhaps more efficient context windows would be the right way to say it. So don't store everything, just store the important things. That would be a lot more efficient. That's what the brain does. Um, and better long-term reasoning and planning. Now, it remains to be seen whether just sort of scaling up existing ideas and technologies will be enough to do that. Uh, or we need one or two more, uh, uh, really big insightful innovations. I'm probably, if you were to push me, I would, I would be in the latter camp. Um, but I think, um, no matter what camp you're in, we're gonna need Large foundation models as the key component of the final AGI systems. Of that, I'm sure. So, I don't, I'm not subscriber to someone like Jan LeCun who thinks, you know, that there's sort of some kind of dead end. I think the only debate in my mind is, are they a key component or the only component? So, I think it's between those two, two options. And, and for me, we, this is one advantage we have of having such a deep and rich research bench. We can go after both of those things At maximum, with maximum, uh, force, both, you know, scaling …

AI assessment note: “we need one or two more big breakthroughs... along the lines of things like continual learning”

Q All right. Now help us dream a bit before we leave here. Uh, when you talk a little bit about the potential for this technology, personalized medicine, uh, new chemistry, what, what about this technology enables those type of things?

A Yeah. So, um, We today don't have enough computational power in classical computers to basically, uh, simulate and determine properties of molecular structures. That's just beyond the reach of classical. So the only way we can develop new drugs is by, um, kind of making them. And testing them, right? We can't do it digitally. We can't do it computationally. Quantum, uh, classical computers just don't have the computational power to be able to do that. Quantum computers do. What this means is that we'll be able to explore new molecular structures, um, much faster, much more efficiently than is possible today, allowing us to find These amazing new drugs or, you know, amazing new, um, um, materials, uh, that can create all kinds of interesting things to benefit society.

AI assessment note: “Quantum computers do. What this means is that we'll be able to explore new molecular structures”

Q their heads a long time ago. Um, Now it seems like the technology is actually getting there to the point where Maybe it will be useful. Maybe it can make sense of our context. So Cristiano, when you think about, all right, I'm going to put chips in glasses and maybe some other different formats, uh, and people will use them and have X experience. What is that experience? Yes.

A Let's talk about the experience and I'm going to, uh, break this conversation. I'll talk about the experience and I'm going to talk about the technology that goes, uh, uh, you know, behind it. So think about how glasses are performing today. You know, you have, for example, the Meta Ray-Ban glasses. I think there's going to be other glasses coming within the Google ecosystem this year. And what are the glasses doing today? Like you have cameras, so you see what you see, you can understand the image, it can annotate the image, and you have a microphone, it has a speaker, may or may not have a display. You know, you have use cases even without the display, like you have the Meta Ray-Ban glasses. What'd the experience look like? You're going to be First of all, for those things to, to be, uh, to get scale, they have to have very low friction, and the experience has to be useful, otherwise it's like a gimmick, you're not gonna use it. So, the experience can be like this. Uh, I am, I'm talking to you, and then, uh, let's say I, I see somebody, uh, in the audience, and I, and I just said, uh, who is this person? And the glass will tell me. Uh, I don't know. Let me check. Uh, I, I check on the web. It's this person, uh, here is, this is this person's name. I said, oh, ok. Yeah, you know, you met her before. There was an email that was sent to you, uh, from the, from this person. It ha…

AI assessment note: “It has to be something like you have, like you have your friend with you”

Q be paying them. That's one business line, but like that they, no one has figured out AI search, LLM search based advertising. Actually, who do you think's gonna, who would you bet does it first? Fiji Simo over at OpenAI or the masters, the old timers, Google, and they figure out how to actually charge people to show up in LLM search. If you had to take, take a bet.

A Oh, it's not even a question. I mean, it is the one company that's built a multi-trillion dollar business out of search advertising, right? I, I think that OpenAI may do it, but remember, even with the code red that they had recently trying to fight against the product, the business model, uh, was put on hold. They put on the, the ad rollout on hold, even though we're seeing some of it. I, I think there's, there's no doubt, you know, and we're gonna talk a little bit about the fact that Google's gonna be able to share data across different products. Um, but I think there's no doubt that Google will take some of its targeting, some of its expertise from the other sides of its business, so it's not just starting at zero, um, with this LLM. But actually, now I'm curious to hear your perspective, Ranjan. Why, so we're talking, obviously, about the product side of things. Um, why have you zeroed in on the business side of it? So something about the business side and these, this deal that's making you, you know, making something light up in your head.

AI assessment note: “it is the one company that's built a multi-trillion dollar business out of search advertising”

Q than I thought it would be. I thought that there was going to be a race where some companies would leap out further ahead and would take others some time to catch up. But it looks like right now you have lots of model builders with their frontier models, uh, exhibiting performance that's so similar is difficult to tell which is the best. So what do you make of that?

A I would say that, uh, inherently this is a technology that is going to get commoditized. Uh, the reason for that is that it's actually not hard to build. Uh, you have around 10 labs in the world. That know how to build that technology, that get access to similar data, uh, that follows the same recipes and algorithms, which are very, uh, it's very short actually, like the knowledge you need to actually train a model is fairly short. So because it's short, it actually circulates. Uh, so there's no IP differentiation gap that you can create. So it's very hard to actually leapfrog and to be way ahead of the competition because there's some diffusion of knowledge that is just making everybody do the same things. And so The question there is therefore, where is the value accruing? Uh, and what kind of business model should you pursue to actually make sure that in the end you're turning profitable? Uh, and then the challenge that we see with some of our competitors is that they're investing billions or hundreds of billions into creating assets that are deprecating very fast because those are commodities. And so for us, it has always been at Mistral, it has always been question Like one of the biggest question of the industry, uh, is that you need to invest enough to actually bring value to enterprises, but you also need to invest, uh, reasonably so that you can build unity economics t…

AI assessment note: “inherently this is a technology that is going to get commoditized.”

Q to be a moat forever. And the other way is, you know, the model is actually not the value. It's the know-how of, and, and the implementation side of things. So you can make the model open source, but then provide a service. To businesses to be able to figure out how to take that model and put it into action and actually get results. Are those the two choices?

A Uh, yeah, that's, uh, it's kind of the fork that we see in the industry. Uh, and, uh, our view there has been to be at on the second one, uh, to really the open source, the open source implementation side, which brings customization, but it also brings decentralization in that, uh, If you assume that the entire economy is going to run on AI systems, well, enterprises will just want to make sure that nobody can turn off their systems. So it's the same way if you have a factory, you connect it to, to the grid, you want to make sure that nobody's going to turn off the grid because they don't like you. If AI effectively becomes a community, which is what's happening, and if you treat intelligence as electricity, then you just want to make sure that your, your access to intelligence cannot be throttled. And so that's also one of the things that Open source technology can bring.

AI assessment note: “yeah, that's, uh, it's kind of the fork that we see in the industry”

Q Uh, you mentioned that there's a saturation effect. So, uh, without getting too technical, are, are the models sort of done with getting better? Like are, are, let me put it this way. Are AI models going to continue to get better given the fact that they all seem to be hitting saturation?

A They will get better in more and more specific domains. Uh, in that, uh, I think we've really collectively made them very clever and able to reinvent about long context and able to Call multiple tools, etc. But if you go and want to effectively put them into production in a bank or in a manufacturing company, well, the models need to learn about the, all of the knowledge that is contained into the companies themselves. And so what it effectively means is that for very precise directions, let's say I want to make my model extremely good at discovering materials or extremely good at Designing, uh, plane, uh, designing planes. I will need to go and sweat it a little bit and, and get the right reward signal and get the right experts and ask them to make my model specifically good in that very precise direction. And so we are definitely not done doing that because what we are all racing for is the right environment and the right signal provider for specific capabilities. Uh, but the broad horizontal reasoning capabilities Still going to improve them, but nobody's going to improve them in a way that is creating strong, that is creating a strong gap versus its competitors. So the strong gap is actually in the, in the, in working with vertical experts that know exactly how they design a plane and that actually explain to the model how to do it. And you have like a wealth of directions …

AI assessment note: “They will get better in more and more specific domains.”

Q what a lot of the infrastructure has been used for build, scaling these models, throwing more compute at them, throwing more data, making the models bigger, and then the idea is that the models get better. So are you seeing most of your demand in the training side of things? Or has it gone to inference where like companies are actually using the models, uh, and deploying them into production?

A It's a great question. And I think it, it talks to the split or this kind of delineation of where the market's been for the last three years and where it's going. Um, you know, our customer base for the last three years has primarily been the largest AI labs and enterprises that are building the capabilities of AI. Right. It's now shifted from the people building those capabilities to the people that want to use those capabilities to change business outcomes. And this is where all the enterprise adoptions coming from. Um, you know, it's, uh, one of my favorite services out there is lovable, right? You go to lovable. You can build any app you want. There's a chat bot that helps you go through it. Um, you know, we're finally starting to see people chain together these capabilities to build real products that solve problems. And our business for the last three years has really been around the creation of those capabilities and has very quickly shifted to include not just the creation of them, but the deployment of them and use in business practices. All right. So, um, one of the things that I didn't expect was that, uh, what looked like training two years ago is how inference was going to look today. Right. Is that You're still dependent upon, uh, highly connected storage. Um, you know, your backend networks become critical to this because the models are so large. So there's reall…

AI assessment note: “shifted to include not just the creation of them, but the deployment of them”

Q But can I ask, how have you set up the company to make sure that you're not the distressed asset when the contract, if the contract happens?

A Is everybody last year talked about how customer concentration and exposure to Microsoft was a bad thing, but they have a better balance sheet than the U S government. Right? Like I'm not worried about them performing in their longterm obligations to us. Like, that's basically the best possible position we can be in, and we've been super thoughtful about the way that we choose which customers to work with and how we manage the credit exposure so that we're, like, we're certain that the investments we make will be paid back, and if you look at the people that are providing us the, the debt to do those projects, like Blackstone, right, they're the, some of the most sophisticated people in the world, and for their underwriting committee, uh, committees to come in and say, yes, I wanna do this, and I wanna scale it up as aggressively as possible, like, You're telling me you're gonna pitch some financial analyst against John Gray? I'm gonna go with John Gray.

AI assessment note: “we've been super thoughtful about the way that we choose which customers to work with”

Q do Amazon. We'll do Apple, and we'll see what else we can get to. Netflix? I don't know. Let's, let's do that right after this. And we're back here on Big Technology Podcast with Rita Albregatti. He's the technology editor at Simaphore. A headline from the future about Amazon. I have it here. Amazon partners with OpenAI and Anthropic to bring shopping into chatbots. Are we going to see it?

A Yeah. I mean, that would make total sense to me. I think that would make total sense. Cause right now, I mean, that's what people want, right? That's what everyone is trying to basically do that with Amazon now. And they would love to like disintermediate Amazon, like, like all these, all these chatbot companies perplexity to like, they'd love to just have you be able to shop on Amazon and never have to go to Amazon. And of course, like that's a bad deal for Amazon. So the natural, I think the natural solution is they've got to work together and they've got to figure out how to, you know, with whatever it is, MP, MCP servers or something else, um, to just kind of make this, make this shopping experience work.

AI assessment note: “Yeah. I mean, that would make total sense to me.”

Q was when the application was starting to materialize, like you really had a sense that this stuff might work, and now twenty-twenty-five has been like, um, everybody's taking all of their money and putting all of it into this project in the hopes that, uh, you know, it really will continue growing the way that it has. Is that, is that a good sort of representation of where we are?

A I like that. I like that. Yeah. I do think that's right. And I did, I did write in the predictions last year about, you know, infrastructure and the fact that it was probably an obvious prediction, but that like we were just going to see a huge, like multiples more compute this year than we, than we did last year. Um, and also talked about how these data centers were going to get so big that they're going to be, these large frontier models are going to be trained across multiple locations, which we started to see in like the last quarter. So yeah, I mean, I, I totally agree with that. And I think we'll see, I think we'll see that continue next year.

AI assessment note: “I do think that's right. And I did, I did write in the predictions”

Q and reinforce on what works. You know, when someone pushes something, that's pretty good, uh, indication that you did a good job and you should do more of that. Uh, all these other, uh, types of disciplines are more open-ended. So is it just gonna take a little longer? And maybe that's why code was first, but is it gonna take a little longer? How do you see that evolving?

A That's a very valid point, right? One of the ways we can make, let's say, models and agents, uh, Improve, ah, how they perform tasks in a certain domain is by obviously having a very clear reward function, let's say, on, on which, and enough cases on which we train the model or the agents. And, ah, ah, with code, let's say we have created the ability to have a very clear signal, ah, when somebody uses the product. But also, in my opinion, the other thing that happened with code, in addition to actually, let's say, training the models on code, because there's a lot of code available, ah, we built products. We built IDs that essentially users started using, even if the answer was not perfect. And that started generating a lot more data and how users actually respond to these things, right? When do they accept the change and when they do not accept the change? And that itself is very valuable data and, uh, you know, creates a data flywheel, but then we can go back and improve the way we generate code. I do think that paradigm is applicable to other domains. So you have to probably start with something that is useful, that let's say humans are on the driver's seat, engineers in our case, and, uh, As long as it's, it's a better way of working, it's, it's, it's faster than not without that tool, then you can also create this data flywheel where we get essentially, um, uh, uh, some so…

AI assessment note: “I do think that paradigm is applicable to other domains.”

Q doing. Uh, but I imagine that if one agent breaks, uh, the whole thing goes down. Um, so can you talk a little bit about like how you get them? I mean, trying to get a chap out to do one thing you want is sometimes a challenge. So how are you getting multi-agent, uh, processes and workflows to work? Because it seems to me like a very difficult problem.

A Yes. So first of all, I mean, maybe to use an example, right? Let's say you have two agents that they have to collaborate. Maybe one agent goes and understands documents, and maybe there's another agent that goes and takes actions into a code base, right? To be able to perform a task. So if you have a situation with these two agents, and oftentimes you have more than two, maybe you have five or 10, you have to figure out the plan you want to execute, right? How do you coordinate the work across them? And then they have to essentially do the work and communicate back to each other What they learned, right? So to do that reliably, I think you have to have many layers of, let's say, uh, guardrails, if you wish, right? Not guardrails in the sense of not letting the, the, the models of the agents do something, something wrong, but oftentimes what we do is we have an agent do some work, another agent review that work, and provide feedback, and have the first agent iterate, right? And then now when it expands across multiple agents, maybe They all do the work. They produce an outcome. In our case, let's say an agent goes and does, uh, investigates an incident and produces, like, an analysis of what happened and how to fix it. Then we have another agent go review that work and force the first agent to go back and, you know, redo the work if it finds, like, some, some, uh, hole in its r…

AI assessment note: “we have an agent do some work, another agent review that work, and provide feedback”

Q auditing and the fixing, uh, with technology itself. Um, so then what does that, I mean, what do you think that, that, uh, if this works the way you anticipate, where do you think that leaves the end state of engineering skills? Um, you know, do you think that, that engineers will risk, uh, you know, having some of those skills atrophy if the AI does a good enough job?

A Yeah. Like with every technological evolution, obviously we get a lot of leverage as humans to produce, you know, outcomes a lot faster, right? We're not resisting using machines for other domains. I think the same applies to, to, to software and, you know, software over the last 50 years moved, uh, you know, from very low level, let's say coding for the machine itself to like different layers of abstraction with operating systems and high level languages. I think AI is just another abstraction. And I don't think the answer to this is, ah, or the concern is, are humans, are engineers gonna atrophy the sort of skills to produce code or run code? I think the real answer is that we should produce agents that do both parts very well. They should produce code very quickly, but they should also be able to run, maintain, improve, troubleshoot code the same way. And engineers should be now, should be, will be very quickly, in my opinion, operating at the high level of abstraction, Where they won't have to worry. Also, a lot about these low-level, kind of, specific bespoke things of the tool that you have at your disposal, like query languages, and, you know, how exactly should I call this API or this CLI to get to an answer. Simply, this is going to be done, all this heavy lifting and stressful work is going to be done by AI, and we're going to be operating at the level above. And I th…

AI assessment note: “I think AI is just another abstraction... I don't think it is a risk”

Q eight hundred million. Reports say approaching nine hundred million. Um, but then on the other side, you have distribution advantages at places like Google. And so, I'm curious to hear your perspective. If the models, do you think the models are gonna commoditize? And if they do, what matters most? Is it distribution? Is it how well you build your applications? Is it something else that I'm Not thinking of.

A I don't think commoditization is quite the right framework to think about the models. There will be areas where different models excel at different things. For the kind of normal use cases of chatting with a model, maybe there will be a lot of great options. For scientific discovery, you will want the thing that's right at the edge that is optimized for science, perhaps. Um, so models will have different strengths, and the most economic value, I think, Will be created by models at the frontier and we plan to be ahead there. Um, and we're like very proud that five two is the best reasoning model in the world and the one that scientists are having the most progress with, but also, um, we're very proud that it's what enterprises are saying is the best at all of the tasks that a business needs to, to, you know, do its work. Um, so there will be, you know, times that we're ahead in some areas and behind in others, but the overall most intelligent model I expect to have Uh, significant value, even in a world where free models can do a lot of the stuff that people, that people need. The, the products will really matter. Distribution and brand, as you said, will really matter. Um, in ChatGPT, for example, personalization is extremely sticky. People love the fact that the model gets to know them over time, and you'll see us push on that, uh, much, much more. Um, people have experiences …

AI assessment note: “I don't think commoditization is quite the right framework to think about the models.”

Q planning elements for weeks now, and I can just come in in a new window and be like, alright, let's pick up on this trip, and it, it has the context, and it knows, it knows the guide I'm going with, it knows what I'm doing, uh, the fact that I've been, like, planning fitness for it, and can really synthesize all of those things. How good can memory get?

A I think we have no conception because the human limit, like even if you have the world's best personal assistant, They don't, they can't remember every word you've ever said in your life. They can't have read every email. They can't have read every document you've ever written. They can't be, you know, looking at all your work every day and remembering every little detail. They can't be a participant in your life to that degree, and no human has like infinite perfect memory. Um, And AI is definitely gonna be able to do that. And we actually talk a lot about this. Like right now, memory is still very crude, very early. We're in like the, you know, the GPT-II era of memory, but what it's gonna be like when it really does remember every detail of your entire life and personalized across all of that, and not just the facts, but like the little small preferences that you had that you maybe like didn't even think to indicate, but the AI can pick up on, uh, I think that's gonna be super powerful. That's one of the features that still, maybe not a twenty-twenty-six thing, but that's one of the parts of this I'm most excited for.

AI assessment note: “what it's gonna be like when it really does remember every detail”

Q You've talked about building a cloud. Um, here's an email we got from a listener. At my company, we're moving off Azure and directly integrating with OpenAI to power our AI experiences in the product. The focus is to insert a stream of trillions of tokens powering AI experiences through the stack. Is that the plan to build a big, big cloud business in that, in that way?

A First of all, choice of tokens, a lot of tokens. And if, you know, you asked about the need for compute and our enterprise strategy, like enterprises have been clear with us about how many tokens they'd like to buy from us. And we are going to again fail in 2026 to meet demand. But the strategy is companies, most companies seem to want to come to A company like us, and say, I'd like to enable my company with AI. I need an API customized for my company. I need ChatGPT Enterprise customized for my company. I need a platform that can like run all these agents that I can trust my data on. I need the ability to get trillions of tokens into my product. I need the ability to have all my internal processes be more efficient, and we don't currently have like a great all-in-one offering for them, and we'd like to make that.

AI assessment note: “we don't currently have like a great all-in-one offering for them, and we'd like to make that.”