The Exchanges

Q Um, and... Can I ask you, we, we, we, we saw Benioff say that he spends three hundred million a year on Anthropic, which equates to about 3.8% of developer salaries on Anthropic. To make it justify the valuations that we're seeing for these companies, it needs to be 20%. Do you have any concern in that movement from 3.8% to 20%?

A No. I mean, I, I think if you look at I've never done this in any detail, but if you look at what we pay hardware engineers, and you look at what the tools, which we, the EDA tools they use, I bet you're much closer to 15 or 20% than two or three percent. What's happened is historically software engineers use very low-cost tools, and hardware engineers used extremely expensive EDA tools. And so, that's interesting, isn't it? I mean, I, I, I think we, the cost of bugs And hardware is so high that we became accustomed to using many expensive tools. And in software, we threw people at the problem rather than tools. And as AI becomes more productive, I certainly don't see a problem where software engineers using 50 or a 100,000 a year each in tokens. There are forty-seven million software engineers in the world. I mean, that's five trillion dollars just in software engineering token use.

AI assessment note: “No. I mean, I, I think if you look at what we pay hardware engineers”

Q Again, going back to my idea of what the future looks like, what should one expect from that? Does it ease? Does it ease over time? What happens to the cost?

A Well, I, I think the, the challenge here is that, that these are extremely, uh, lumpy. Items, right? You can't just add a little bit of manufacturing capacity at a fab. You have to build a fab for forty billion dollars, and it takes five years to build. So if you see demand explode, you cannot respond quickly. All you can do is fill your factory. Once your factory is filled, you've got to build another factory, right? It's a step function in your ability to meet that demand. And the step is huge and takes years, and so if demand stays high, uh, they are, we're going to continue to see memory shortages for at least the next several years.

AI assessment note: “we're going to continue to see memory shortages for at least the next several years”

Q terms of the different models there. Again, sorry to cite it, but it's, it's kind of handy having just done it. Jonathan said that you would definitely have OpenAI and Anthropic build out their own ships, because then they would have control of their own destiny. Do you think OpenAI and Anthropic build their own ships so they don't have self-reliance on NVIDIA in the way that they do today?

A I think that there is a long history of software companies failing to build chips. The list is, is very large. I think, uh, whether, uh, OpenAI can do it, uh, whether they can do it through partnership with other vendors, with Broadcom, with smaller, more innovative companies is an open question. Um, but I, I think that, uh, you know, companies at the size of Microsoft have been unable to deliver, uh, chips, right? I think, uh, uh, there are plenty of examples as you look across the Fang, uh, group where chips were tried. I mean, probably the most successful is Google and they're 10 years in, right? Maybe longer. You know, soft, modern software does not fit well In a chip making framework. I mean, weekly sprints don't work well on two year long projects. Um, you know, move fast, break things often is not the way you think in the chip world. The way you think in the chip world is measure twice before you cut once because your bugs cost you six months and tens of millions of dollars. And so it's a very different mentality. And where there's been success, it has frequently been acquired. Apple got into the chip business through buying PA Semi. Amazon got into the chip business through acquiring Annapurna. Um, uh, Google acquired the talent from a collection of companies, uh, and then set it in a BU that was a side and under somebody who, who had sort of enormous respect in the org…

AI assessment note: “there is a long history of software companies failing to build chips.”

Q are putting a lot of money into compute and hardware and compute and hardware startups. And I wanted to ask you, should I be following them? What should I be looking for in them? I'm seeing some incredibly young founders. I was with a nineteen-year-old founder trying to take on NVIDIA this morning, um, raising twenty million dollars for a pre-seed. How, how should I be thinking about this, Andrew?

A Well, I, if you don't know a lot about hardware, I wouldn't invest in hardware. I think, uh, Harry, it's probably the same in many things, is that, um, I think hardware is not an easy place to make money. It's a place that has historically rewarded experience, both from investors and from entrepreneurs. I think the number of different technologies involved in designing a chip, Uh, is extraordinary. Not just the, the logic, which is what most people think about. When, when you think about chip design, that's just the front end part that that that's writing in very low level software. Um, but the, the selection of tools, right? We pay millions of dollars a year in tools, selection of geometry, right? Which fab and having a relationship with a fab, you're going to pay 20 or thirty million in NRE. And if you have a bug in your, in your chip, you got paid again.

AI assessment note: “if you don't know a lot about hardware, I wouldn't invest in hardware.”

Q We've spoken before and off record about, kind of, personal lives. I'm intrigued. When you are a public company CEO and you're going public, the world wants a piece of you. You're public, now you're public. Any advice on how to sustain an amazing marriage And an amazing relationship while also being a public company CEO and going through that process?

A I, I would say that pick a wife with patience. Pick a partner, a husband or a wife, partner, um, who understands w w what it is to, to be an entrepreneur. I, I don't think, and I look at my co-founders and, and our leaders, it, it is every day when you're a leader of a, of a startup, a pressure test on your soul. Every single day. And if you're a real leader, when you are 30 people, a little, a little company picnic, you look out, and what you see are mortgage payments and braces that need to be done that you're responsible for. And that doesn't change. And, uh, I, I think that if you really believe that, and you, you hold that in your heart every day, you carry real weight with you. And I, I think, uh, you have to share that with your partner so they understand. It's really hard if they don't. I think almost everybody, and maybe your, your, your, your partner has felt this, and I think every CEO I know has told the story of their partner telling them that They're more lonely when you're sitting next to them thinking about work and your mind is just ripping on work than they were when you weren't in the house. And I, I, I think that what we do is a family thing. There's a price to be paid and how often you see your wife. I mean, I'm on the road three weeks a month. I mean, put it this way, uh, Emirates airline sends me a Christmas basket. This is an Arab airline sending a Jewis…

AI assessment note: “pick a wife with patience. Pick a partner, a husband or a wife”

Q Do you think we should be selling chips to China as a result?

A No. No. I, I think, um, I think, uh, let, let's remove all of us that are self-interested, and even though I'm arguing against my self-interest, right, um, if you remove me, and you remove Jensen, you remove Lisa, and you remove everybody in the chip industry, and, and you say, if we sell to, to somebody in the security business, and you ask this question, if we sell leading edge technology to China, will their military use it? Everybody says yes. There is no debate on that point. Their military will use it. You ask a second question, which is, if you sell our leading edge technology, will they, will their government use it through their industry to compete with us, all right, in an advantaged way? The answer is also yes. In that, and so that's where I stop. There's complete agreement that those two things are true by everybody in the security business and outside of the chip business. And now, you can say that keeping them in our ecosystem is the best way to manage that problem. That's one argument, and there's some merit to that. There is keeping them from building their own Their own ecosystem is something that's in our interest. There's real merit in that. I don't agree with either of those arguments, but, but they're real arguments and they have real merit. They are at least today our industrial adversary. And, uh, as you travel the world and you see sort of the results of…

AI assessment note: “No. No. I, I think, um, I think, uh, let, let's remove”

Q If I said that you have one policy change that you could usher through with no resistance, what would it be?

A Um, I, I would allow TSMC and, uh, Samsung, uh, both to, uh, a 20 year period free from all local and legal, local ordinances, all of them, to build fabs in their desired location in the US. If that's Arizona, that's great. If that's Texas, that's great. 20 years, no local rules, no, allow them to build fabs And I, I would say that we use the same rules we use in Taiwan. Don't, don't build garbage. Use exactly the same construction techniques and rules, et cetera, that you, that you've built fabs successfully elsewhere in the world. But local ordinances are disastrous. And not intended to cover pyramids, right? Fabs are modern pyramids, Harry. I mean, they are the greatest things humans make in manufacturing, in the manufacturing world, by far. Nothing's closed.

AI assessment note: “allow TSMC and, uh, Samsung, uh, both to, uh, a 20 year period free”

Q Dude, you are too kind. Uh, listen, I want to start with the billion dollar raise that you just announced yesterday. Um, can you just talk to me about the billion dollar raise, why it's important, why now, and what it means for the company?

A Well, look, it was the largest raise ever done in, in our category. Uh, it was done at the highest valuation and with the, the premier investors. So, uh, At, at late stage investing, you're looking for, uh, the likes of Fidelity. They are the, uh, What would the English call it? The sort of Oxford or Cambridge of investing, right? I mean, they, they are the, uh, the premier, uh, public market investors, and when they choose to lead a round, uh, it, it brings the Wall Street a great deal of confidence, and so, uh, we were really happy to partner with them and with the treaties to lead the round, and then we, uh, were able to get enormous participation from Tiger Global, From Valor, from 17, uh, 89. So that's .1. I think .2 is that, um, We, we now have sort of the dry powder to, uh, to really push and to take the opportunities in front of us, uh, to build out our manufacturing to the scale and scope we want, to add new data centers. We added five this year in the US to add more data centers, and we have more big ideas. I, I think incremental improvements, Uh, uh, make believe gains achieved by dropping from, from, you know, eight bit to four bit. Uh, those aren't gonna get us to, uh, to the promised land in AI. We, we need, we've got real work to do as a community. And I, I think this, this funding puts us in the catbird seat for that.

AI assessment note: “we now have sort of the dry powder to, uh, to really push”

Q but SRAM is obviously memory, like, on chip versus off chip. Seemingly great, but he said it's completely unable to handle scale, and so although it may be quicker, for anyone who wants to do large scale, It is incapable at present of doing that, and that's a fundamental need and requirement of any of the large providers. Do you think that's fair, and how do you think about that?

A Well, not only is it fair, it's the reason we went to wafer scale. So let me explain. What your friend said is strictly true, in that SRAM is blazing fast and low capacity. HBM is a flavor of DRAM. It has high capacity, and it's very slow. Now, NVIDIA and all GPUs, including, uh, uh, AMDs chose a big capacity memory that is slow because it's perfect for graphics. You don't have to go to memory very often. You can hold a lot, you don't go very often. SRAM Is blazing fast, but it can't hold very much. So the problem on traditional chips is if you put memory on the chip, you are using space that could be otherwise used for compute. You have a fixed amount of real estate. And so if you put half memory, then you have half your real estate's available for compute. And so, our idea was that if we, we sort of, if we built a chip that was the size of a dinner plate, we could stuff it to the gills with fast SRAM, overcoming the limitation of SRAM, which is it doesn't store very much, by putting a huge amount down, by using more silicon area. Now, if you're an SRAM solution today in a normal size chip, And you're trying to do a trillion parameter model, use four or 5000 chips. What a mess. You know how many cables that is? Do you know the, the, the impact to the AI? It's a horrible mess, right? And it limits you from doing things you want to do with the AI, uh, like speculative decode. It…

AI assessment note: “Well, not only is it fair, it's the reason we went to wafer scale.”

Q but SRAM is obviously memory, like, on chip versus off chip. Seemingly great, but he said it's completely unable to handle scale, and so although it may be quicker, for anyone who wants to do large scale, It is incapable at present of doing that, and that's a fundamental need and requirement of any of the large providers. Do you think that's fair, and how do you think about that?

A Well, not only is it fair, it's the reason we went to wafer scale. So let me explain. What your friend said is strictly true, in that SRAM is blazing fast and low capacity. HBM is a flavor of DRAM. It has high capacity, and it's very slow. Now, NVIDIA and all GPUs, including, uh, uh, AMDs chose a big capacity memory that is slow because it's perfect for graphics. You don't have to go to memory very often. You can hold a lot, you don't go very often. SRAM Is blazing fast, but it can't hold very much. So the problem on traditional chips is if you put memory on the chip, you are using space that could be otherwise used for compute. You have a fixed amount of real estate. And so if you put half memory, then you have half your real estate's available for compute. And so, our idea was that if we, we sort of, if we built a chip that was the size of a dinner plate, we could stuff it to the gills with fast SRAM, overcoming the limitation of SRAM, which is it doesn't store very much, by putting a huge amount down, by using more silicon area. Now, if you're an SRAM solution today in a normal size chip, And you're trying to do a trillion parameter model, use four or 5000 chips. What a mess. You know how many cables that is? Do you know the, the, the impact to the AI? It's a horrible mess, right? And it limits you from doing things you want to do with the AI, uh, like speculative decode. It…

AI assessment note: “Well, not only is it fair, it's the reason we went to wafer scale.”

Q How do you think about that? Is that just pricing power, which they are taking advantage of?

A Absolutely. I mean, the, the, the short answer is, why does it make sense for, uh, AWS to, to, to build a training part? Well, because they want to get rid of the 78% gross margin that NVIDIA is charging. That's why it makes sense. Just like that. It might be, on the high-end chips, 85%. And I, I think people don't like that historically. Historically, people sort of put that in the back of the mind and they remember it. And, you know, when Intel stumbled, the, the number of people who came out of the, the woodwork to kick them when they were down was extraordinary. And it was sort of years of pent up Sort of frustration, uh, came out when, when the giant stumbles. And I, I think, uh, we've seen that again and again.

AI assessment note: “because they want to get rid of the 78% gross margin that NVIDIA is charging.”

Q are putting a lot of money into compute and hardware and compute and hardware startups. And I wanted to ask you, should I be following them? What should I be looking for in them? I'm seeing some incredibly young founders. I was with a nineteen-year-old founder trying to take on NVIDIA this morning, um, raising twenty million dollars for a pre-seed. How, how should I be thinking about this, Andrew?

A Well, I, if you don't know a lot about hardware, I wouldn't invest in hardware. I think, uh, Harry, it's probably the same in many things, is that, um, I think hardware is not an easy place to make money. It's a place that has historically rewarded experience, both from investors and from entrepreneurs. I think the number of different technologies involved in designing a chip, Uh, is extraordinary. Not just the, the logic, which is what most people think about. When, when you think about chip design, that's just the front end part that that that's writing in very low level software. Um, but the, the selection of tools, right? We pay millions of dollars a year in tools, selection of geometry, right? Which fab and having a relationship with a fab, you're going to pay 20 or thirty million in NRE. And if you have a bug in your, in your chip, you got paid again.

AI assessment note: “if you don't know a lot about hardware, I wouldn't invest in hardware.”

Q Can I just interrupt and ask what is yield and why is it impossible to solve?

A Ok, ah, a wafer become, begins, it's a, ah, ah, a 12 inch, ah, diameter circle, slice of, of silicon. And your, your chip is punched out of this the way your mother might take a cookie cutter and cut out cookie dough. And, ah, during the process at some point, just like your mom might have done, she lifts up the edges, and all the little bits are removed, and what's left are just the cookies. Those are your chips. Um, now what happens is there are a set of naturally occurring flaws, and that's like your mother closing her eyes and throwing up a handful of, of M&Ms. Now, the bigger the cookie, right, the higher probability you hit an M&M. The bigger the chip, the higher the possibility that you have a flaw. And traditionally what you did when you had a flaw was you threw away the chip, or you sold it as a less valuable part. You shut down part of the chip and sold it as a less valuable part, something called binning. So, every wafer is going to have flaws. The bigger your chip, the higher probability you hit a flaw, and the more way, the more part of silicon is wasted when you throw it away. This is what everybody thought was known truth. And one of the, the things our, our team realized was that there are other ways to handle flaws. Like, what, what if instead You built your computer, you built your processor out of hundreds of thousands of identical tiles, and say there was a …

AI assessment note: “The bigger the chip, the higher the possibility that you have a flaw.”

Q Are export controls being implemented properly? Do you think that is a good idea? You know, everyone was going with Deepsea, wow, how did this happen? They must have stolen chips. How could this be? What do you think about export control?

A It turns out that they probably did use chips in Singapore. Um, I think the following, I, I think managing Software and managing hardware compliance are, are extremely different things. Because their, their vector of diffusion is different. There's different weights. Right? If you sell a, a server that weighs five or 600 pounds, arrives on a pallet, you can go visit it. Right? You, you want to deploy it in Kazakhstan. You can put a data center, and you can have somebody from the embassy visit it. Take photos of it once a month. It's not going anywhere. Right? You, you can keep track of who uses it and provide logs, and that's much, much harder with software. And open source is a, a whole nother level. Right? And so, um, that's the first observation. The second is that We had, I got to know the, the, the leadership in commerce in the previous administration. I didn't always agree with their policies, but it is a world of unintended consequences. You, ah, sought to limit Chinese access to EDA tools to delay the growth of a Chinese chip market. And so U.S. venture capitalists backed tons of Chinese companies in Shenzhen to build EDA tools, right? I mean, right? Right. This is a unbelievably slippery, dynamic, challenging problem, and I don't know if it's a tractable problem to delay another nation's progress on a technical trajectory Is an enormously challenging thing. And, ah, I,…

AI assessment note: “This is a unbelievably slippery, dynamic, challenging problem, and I don't know if it's a tractable problem”

Q It's been an extraordinary nine years. Can you just help me understand how does the movement into an age of AI change the requirements from a chip perspective of what is needed for a provider and how that then resulted in how you built Cerebris?

A The way to think about, uh, A chip is, it does two things. It does calculations, and it moves data, right? This is what, what, what a chip does. And, ah, sometimes along the way, it stores data. And so, ah, what AI presented was a very unusual combination of challenges. First, the underlying calculation is trivial. It's a matrix multiplication, and an FMAC can be developed by any second year electrical engineering student. So you say to yourself, holy cow, this has a huge number of very, very simple calculations. The hard part with AI work is results and intermediate results have to be moved a lot. And therein is the most complicated part. They have to be moved to memory and from memory. And they have to be broken up and moved among GPUs. And what we saw was that this was going to be the hard problem. And that if we could solve for that problem, we would build an AI computer that was faster and use less power.

AI assessment note: “The hard part with AI work is results and intermediate results have to be moved”

Q When you speak about kind of being the fastest and across all benchmarks being the fastest, what matters the most? Is it being the fastest? Is it being the most efficient? Is it being the least costly? How do you think about the stack of prioritization for your customers?

A I think it varies. I, I think, uh, um, look, if, if, if you go to, to, to, to get a cancer diagnosis on, God forbid, your mother or, uh, your wife, I think, uh, uh, 93% accuracy is just plain not as good as 94% accuracy. And you pay a lot and wait another week to understand what the accuracy is, right? You pay a lot. Right. Now, on the other hand, uh, if you want Llama- four or five B to generate data to help you tune Llama- seven DB, uh, maybe you can wait a few days, three days, a week, more. You don't, there's no urgency there. On the other hand, if you want an answer from perplexity, Right? You don't want to wait 45 seconds for a search answer. Right? You don't want to wait in a chat. You don't want to wait three minutes for R-one on GPUs to give you an answer. What we know is that in interactive mode, milliseconds matter. In interactive mode, what Urs Holtz over at Google years ago showed was that you can destroy your user's attention With milliseconds of delay. So being the fastest matters everything in that domain. So I, I think what you have to do is sort of be thoughtful and say in some cases being the fastest doesn't matter. Uh, we'll call those batch. Uh, lots of maybe the cheapest matters there. In other domains, there is no search if you gotta wait eight minutes to get an answer. Right? That, that, that's not a product. That, when you go fast, a whole set of, of ne…

AI assessment note: “in some cases being the fastest doesn't matter... In interactive mode, milliseconds matter.”

Q How do you think about how the cost of inference goes down with the surge of demand that we mentioned, you know, over a hundred X, does the price reduce a hundred X? Does it follow Moore's law continuously? How do we think about the ever reducing price of inference?

A Look, there, there are, ah, the cost of inference is built up of, of several pieces, right? There's the power and space that is consumed to generate the response, right? That, that, that's a data center cost. That's an OpEx item, number one. Number two, there's the, ah, cost of, of the, the computer. We can drive down the cost of the computers with each generation by driving up their performance, et cetera. The other thing we can do is we can develop more efficient algorithms. Our AI algorithms today are not particularly efficient. There's a tremendous amount of room. Uh, in a GPU, most of the time it's doing inference, it's five or seven percent utilized. That means it's 95 or 93% wasted. So, over time, I think as an industry, we get better at things. We can drive the cost of compute down, we can build more efficient data centers with lower PUEs, and our algorithms will get more efficient so that our utilizations on our now cheaper computers are higher, so you get a higher percentage of the maximum number of flops. You get more tokens per unit time for the same power.

AI assessment note: “the cost of inference is built up of, of several pieces”

Q Do you think we will see a peaking of demand? You've seen so many different...

A Not if AI continues to improve in usefulness. I mean, what's happened here is, and this is something that I haven't heard others sort of talk about, is that somewhere in twenty-twenty-five, the models got smart enough to be really useful. Before that, Harry, these were sort of, sort of a novelty. AI was like, cool, and then nobody used it. Remember, we, we make AI with training, and we use it with inference. And so once the, the AI we made, 20, twenty-five-ish, first half, got smart, we began using it. And this explosion in demand that Jensen described, alright, and that we very much agree with is happening, that's because people are using it every day. And they're using it on more and more problems. They're using it on harder problems, and it is sweeping through different demographic groups. It's not just twenty-eight-year-olds in Silicon Valley. It's my eighty-five-year-old father. It's, right, it's eleven-year-old, my eleven-year-old niece. It is, right, it is sweeping through demographic groups, and they're using it all the time. And that is what's driving this demand. And so, um, If we continue to find ways to make the AI, the frontier models, smarter and more useful, we'll keep using it. The demand will continue to, to con, on this sort of exponential curve.

AI assessment note: “Not if AI continues to improve in usefulness.”

Q I was chatting to a friend who is a phenomenal mind, and he said that Google will become the lowest cost producer of tokens because they own the full stack from TPUs, data centers, networking, power procurement. Do you think that's right, that that full stack ownership will lead to their highest margin, lowest cost stability?

A There are, uh, pros and cons of that strategy, right? Uh, the pro is you have everything from the ground, right, land, all the way up to tokens. The downside is you can only sell your TPU to yourself. And historically volume mattered a lot. And so your market is constrained by your own demand. Whereas if you were able to sell to the whole market, you might have more, more demand and be able to drive down the cost. It's an open question. Google is threatening that argument. I think your friend's argument is reasonable, but there has historically been a challenge if you only have one customer yourself for your hardware. And, uh, that has historically limited the size of the opportunity landscape for you.

AI assessment note: “I think your friend's argument is reasonable, but there has historically been a challenge”

Q Can I ask, Andrew, I sit here in London. Should I be worried? And you have the best frontier labs in the US. You have amazing open source and amazing manufacturing capabilities in China. What does Europe really have? We kind of failed on the model front. Mistral is the leader, but sadly nowhere near others. Should I be worried?

A You should be worried at the pattern. The pattern of sort of lack of success across a range of technologies. It's not just that the leading AI companies are, most of them are in the US. But the leading chip companies, but the leading software companies, right? Of course, there's some examples, SAP and some others, but there has emerged in Europe a, uh, a sort of be afraid of it, then regulate it, tax it, or sort of mentality that works against entrepreneurship. And I think Europe, uh, and this isn't true across the board, and clearly there are pockets outside of Cambridge, and in London, and in, in, in Stockholm, where the guys at Lovable are doing really interesting stuff, and there are all sorts of counter examples. But on the whole, given its population, the opportunity to do vastly better on the innovation front across industries is sitting there, unexercised. And, uh, that, that I think is, is a worry.

AI assessment note: “You should be worried at the pattern. The pattern of sort of lack of success”

Q We've spoken before and off record about, kind of, personal lives. I'm intrigued. When you are a public company CEO and you're going public, the world wants a piece of you. You're public, now you're public. Any advice on how to sustain an amazing marriage And an amazing relationship while also being a public company CEO and going through that process?

A I, I would say that pick a wife with patience. Pick a partner, a husband or a wife, partner, um, who understands w w what it is to, to be an entrepreneur. I, I don't think, and I look at my co-founders and, and our leaders, it, it is every day when you're a leader of a, of a startup, a pressure test on your soul. Every single day. And if you're a real leader, when you are 30 people, a little, a little company picnic, you look out, and what you see are mortgage payments and braces that need to be done that you're responsible for. And that doesn't change. And, uh, I, I think that if you really believe that, and you, you hold that in your heart every day, you carry real weight with you. And I, I think, uh, you have to share that with your partner so they understand. It's really hard if they don't. I think almost everybody, and maybe your, your, your, your partner has felt this, and I think every CEO I know has told the story of their partner telling them that They're more lonely when you're sitting next to them thinking about work and your mind is just ripping on work than they were when you weren't in the house. And I, I, I think that what we do is a family thing. There's a price to be paid and how often you see your wife. I mean, I'm on the road three weeks a month. I mean, put it this way, uh, Emirates airline sends me a Christmas basket. This is an Arab airline sending a Jewis…

AI assessment note: “pick a wife with patience. Pick a partner... who understands what it is”

Q And I, I think that, uh, but did you have in your mind the other IPOs and when liquidity would be best, excitement would be highest?

A Did we know, uh, when we set the date that chips would be on a run? And that it was impossible for, for, for XAI and OpenAI and et cetera to get public before. We didn't know any of that when we set the date. Um, but what we did know, uh, was that we had a chance to be the first and only AI peer play in the entire market. There's only one, and that's us. And we had a chance to bring an extraordinary growth story To public market investors who had been shut out. We tried again and again, and that's how you get lucky. Harry is smart, hardworking people, relentless work. They get lucky and occasionally they, they, they find the perfect time.

AI assessment note: “We didn't know any of that when we set the date.”

Q I agree with that. How important was that for you? And is there a stage where actually it doesn't matter being that increment more important? Like, 6.7 times. This is so much more important. It's not 20% more important.

A That's right. You know, I, I think for hard problems, there is no upper bound how much faster you want to be, nor the value of speed. If in three minutes we can solve problems that, that take others 20 minutes, then think of all the extra problems we get solved. And think of, if I'm your competitor and I'm solving your hard problems in three minutes and you're taking 20, imagine over a day or a week, you get, you get smoked. You will be smoked in this, in this example. And that is the way this is going. Um, speed is of the essence, and it's true in coding. It's true in agentic flows. It's true in every part of the, uh, of the AI landscape. I mean, let me just ask you this question. How big is the market for a slow search? Really? It's zero. How big is the market for dial-up? For slow internet, right? How much would I have to pay you? Let's try, turn it around and say, there's a negative market here. If I gave you a thousand dollars a month to have slow internet in your home, right, you wouldn't take it. A thousand dollars a month, you wouldn't take it. That's how impossible it is to engage with an important technology slowly. Why do we believe that inference will be any different? There'll be zero marking for slowing them.

AI assessment note: “for hard problems, there is no upper bound how much faster you want to be”

Q And so to my question on COGS, do we see like a plateauing of COGS in terms of it can't get cheaper and this is the stable state? Do we see a meaningful reduction?

A How do, I think what happens over time, Harry, is, is that we, all of us, we, we improve our designs. The designs deliver more tokens per unit time. They deliver faster tokens. Now, we, we are 15 x faster because of architectural reasons. We will continue to, to improve over time. Nvidia, they will continue to improve over time. I believe the gap will widen between our performance and their performance. But all of us, the whole industry, us, Nvidia, uh, AMD, Qualcomm, ARM, everybody's chips will be better in three or four years than they are today. They will produce more per unit power, and they will produce more per per dollar cost. So over time, the history of our industry is a massive reduction in the cost per unit compute.

AI assessment note: “the history of our industry is a massive reduction in the cost per unit compute”

Q You said about the real work to be done. I think it's quite difficult for everyone who's not really in the market to understand what the hell's going on, given all the news that we see. Can you help us just with the lay of the land in the last three months of where are we at now? What's changed? Let's start there.

A The first thing, Harry, is we, we, we are in a, a stage of the market where the, the claims are enormous, right? Where tens of billions of dollars are being done here and there, and nobody's reading the fine print that it's over five years and it's up to, this is the, the, the great sort of CYA word in, in, in marketing history is it will be up to a hundred billion over five years. Right, well, up to means it could be 30, could be 12, could be 40, right? It won't be bigger, right? You could pick a lot of big numbers and it won't be bigger than. And so, I, I think as you read these deals, I, I think, you know, you, you have to really think about the time frame over which they're being done. You have to think about whether, uh, anybody is actually counting, right? Lots of people are saying they're gonna, you know, gonna bring hundreds of billions of dollars of jobs to the US and this and that. I mean, in eight months, has anybody got a little, little spreadsheet, like nine jobs plus one factory? I mean, who, who, who holds anybody to account? And the answer is nobody. And so I, I think that's number one. I think number two, What this signals more than anything is that there is unbelievable demand and nobody knows where it will go in the future. That it's so big and happening so quickly that, that they don't know. I mean, we, we have customers coming to us and saying, uh, we would…

AI assessment note: “we are in a, a stage of the market where the, the claims are enormous”

Q anticipate, what are the bottlenecks today in your mind? You know, again, we had Jonathan at Grok on and he was like, actually, you know, I had someone come and demand five times the supply that I have in total, and I only, that was from one customer, um, supply is mine. How do you think about the bottlenecks that we have to reach the insatiable demand that you mentioned?

A I think if you go back to planning, if you've got customers demanding five X that your capacity, I mean, you probably didn't get your planning right. Right. Uh, you probably should have, you should have planned better. I think there are bottlenecks, uh, at every level that are meaning real and meaningful. I think, uh, I think that the first one is expertise. I, I think, uh, we have, uh, fundamental limitations in AI expertise. We're not making enough AI practitioners. We're not making enough data scientists who understand, uh, data pipelines. We are not, we're not, we're just, our universities aren't minting enough. And our, our challenges in the US with immigration, uh, don't help that. Um, we, uh, have historically sucked the best and the brightest first on J ones to come to our schools and H ones to stay. If that is not our policy, we need to make them, right? If the government decides that that is not the way they want to build a workforce, instead they want to build it, uh, out of people who live here, we need to do a better job of training those people. We need to do a better job of teaching them in K through 12. We need to do a better job of educating them in our universities in order to, to, to make the number of, uh, of, uh, of engineers we need to, to meet this demand. That's a bottleneck. And it's why the, the best and the brightest are, are getting such extraordinar…

AI assessment note: “I think that the first one is expertise. I think we have fundamental limitations”

Q Specifically, but speaking of a hundred billion into opening, how did you analyze that? For me reading that, I didn't really know how to analyze it. It's so unprecedented.

A Well, I, I, I think it was designed for nobody to understand it. If one wants to make something very clear in an investment, we've invested this amount of this valuation, the deal's done now. If you want to make something more difficult, it's up to this amount over an unbalanced specified amount of time at no valuation given or a valuation specified, but it can change. Right? It wasn't designed for you or, or other analysts to, to anchor on different things, and that's a very reasonable thing for, for, for both of them, but it's just, it's not an analyzable, uh, thing. What, what, what, beyond the fact that, that Nvidia has chosen to try and lock up a portion of Uh, OpenAI's demand. Uh, by investing in them. That, that, that's about as much as you can say.

AI assessment note: “it's not an analyzable, uh, thing. What, what, what, beyond the fact”

Q Did you have signs that it would work?

A Yeah, we did. And we weren't running around like chickens without our heads. We were going through engineering process. Each failure was, you know, we did a full FA, a failure analysis. Each time we fixed the cause, we did another one, didn't work, did another one, didn't work. And each time we got a little better. And we got better and better and better. Uh, and then we solved it. And when the founders, the first one that worked, the founders were in a tiny little lab that was a converted conference room that for cooling, we had the windows open and we'd blown a hole in the wall so we could get external, a chiller outside and poured in. And when we had it running, the founders stood there together and stared at the box running, which is about as interesting as watching paint dry. And we stood there, and we couldn't believe it. It was like, we have just solved a problem that, for 75 years, the smartest people in our industry have been unable to solve. And we have done it. And we stood there for like half an hour. And it was, it was one of the highlights of, of my career.

AI assessment note: “Yeah, we did. And we weren't running around like chickens without our heads.”

Q I mean, you are a sucker for punishment, aren't you? I mean, really like five times, like Christ, Andrew, did you not get beaten alive enough? My question to you though is like, I believe in the value of serial entrepreneurship. I've spoken to many who don't. Respectfully, how do you think about the inherent benefits that you have having done it four times before?

A Okay, I think if you are in a business in which running a business is a benefit, then experience matters a great deal. Uh, I think if you are in a business in which you look like Your customer. There was a reason why social, social networks were started by people right out of college or in college, because dating is top of their mind, right? I mean, that, that, that's, they look like their customers. And that was more important than knowing anything about running a business. And so in that environment, it will certainly select for people who are of the demographic that their customers are. They know that backwards and forwards. But if you want to have a business that has manufacturing in it, that has a supply chain, That, uh, has you managing hundreds or thousands of engineers to a timeline, to a schedule. I, I don't think anybody would turn around your statement and with a straight face say, you know, what I'm looking for is an engineering leader with no experience, right? Well, no, and I don't want somebody who's led a team of four or 500 who has experienced the challenges of growth. What I'm looking for is somebody with no experience.

AI assessment note: “if you are in a business in which running a business is a benefit, then experience matters”

Q I totally agree with you. On that, sorry, are NVIDIA going to have a clusterfuck of unhappy customers who bluntly have waited so long for chips, by the time they get them, the chips are outdated and they're going, what?

A All of that's an opportunity for, uh, for us and others. I think that's, that's opportunity. I think being, uh, a market share leader isn't easy either. Um, but when you're late, Right? Uh, when the bully falls, everybody wants to give him a kick. Right? I mean, that's, a lot of that happened at Intel. Um, they'd been the dominant player, and when they fell, everybody was happy to jump in and kick them when they were down. And so, uh, I think there is a, uh, a real opportunity in the potential for NVIDIA Customer Unhappiness, for sure. For those of us who, who are competing with them. I mean, if you can't get your gear, you may as well test somebody else's. And that, that's a huge opening.

AI assessment note: “I think there is a, uh, a real opportunity in the potential for NVIDIA Customer Unhappiness”