The Exchanges

Q I've definitely experienced that with our portfolio of companies that are building large trading clusters. Uh, it, uh, CoreWeave has a reputation for reliability that not everyone has reached. Can you just help characterize, if you fast forward, like, two and a half, three years now, like, what is the scale of the problem today?

A Yeah, so if you look at, um, kind of CapEx, right, let's starting with that. So CapEx for AI compute and infrastructure in twenty-twenty-six, you know, at least from the hyperscalers is projected to be between 666 190. Uh, billion dollars. And over the next several years, um, you know, that scales to trillions of dollars, right? And so the, the scale of the problem is how do you build, um, you know, that size of CapEx efficiently? And I think a lot of that has to do with not only, you know, your ability to have access to, you know, those core elements, um, energy, power, you know, uh, and, and your ability to have data center space, et cetera, But I think one of the things that's not talked about as much is capital, and access to capital, and how is capital structured. Um, and what I mean by that is, this is, you know, billions to trillions of dollars of CapEx, and just using equity dollars alone is not an efficient way to scale this. That's obviously massive dilution, you know, there's, there's, it's not an easy problem to solve.

AI assessment note: “CapEx for AI compute and infrastructure in twenty-twenty-six... scales to trillions of dollars”

Q Couple topics to hit before we lose you. Um, uh, new players. How do you think about the sovereigns and what they're doing in their build outs? Yeah, I think, um, they seem to be able to fund themselves to some degree. Exactly, right.

A Um, you know, you saw the news from India last week, uh, obviously a lot of the news in the Mideast, Southeast Asia. I think, you know, we're continuing to see that sovereigns view, compute, and AI, you know, as, uh, and even we do here in the, in the United States as, as, as a matter of national security. Um, and obviously the funding of those clusters is, is very different than funding like a private cluster, and so you've got, you know, Government capital that can be used for that. I, so I think there's two things that, you know, I find interesting in that space. I think one is who are the partners, um, that are going to build those, that capacity and what are the cybersecurity kind of implications and environments for that? And so those are, those are the two nuances I think with Sovereigns is they need to find players that can rapidly scale compute, um, In the, in their countries, and oftentimes they don't necessarily have these players that know how to build and scale GPU compute. And I think that's a great place for the United States to lean in and help build, you know, sovereign ecosystems around the world. And then there's a matter of cybersecurity and how do you make it into a, a truly, um, you know, safe ecosystem for, for those sovereigns. And so I think there's a lot of work to do still on the cyber side, um, especially as you look at, you know, scaling sovereign A…

AI assessment note: “I think there's two things that, you know, I find interesting in that space.”

Q I have been a heavy AI user for several years, but reasoning advances the ability to scale up inference, especially around code, means I'm up against my max limit all the time in a way that was not true, uh, uh, uh, initially. How does the inference workloads actually growing? I mean, it's a, it's a good demand signal that there is value, but how does that change your business?

A Yeah. So I think one thing that's interesting that we're seeing is obviously there's been the shift from training to inference, you know, over the last few years that that split continues to grow on the inference side as usable, uh, and ROI positive applications get developed. I think the two things I see on the inference side now is, um, inference has, is a lot more complex than I think initially thought. And what I mean by that is it's not as simple as, Um, you know, you train a model and then you, it's easy to inference it. In some certain cases you can do that on similar infrastructure, but there are issues around latency, um, fungibility of that, uh, and, and really optimizing the cost of your compute on the inference side. Um, how do you manage, uh, you know, peaks of inference demand? And, and obviously it's not linear like training and your GPUs are on all the time, you know, a hundred percent of the time. And so with inference, you have a lot more variability. Um, and so there's a lot more nuances, uh, in, in optimizing inference. I think the second thing that's observed, um, that I've seen is, uh, inference is definitely a memory problem, a memory throughput problem. Um, you know, on the inference side, you know, you have these kind of phases called pre-fill and decode, right? And how you optimize that across a fleet of GPUs is actually a unique technical problem. Um,…

AI assessment note: “shift from training to inference, you know, over the last few years that that split continues”

Q I am too. And I think one of the things that, uh, is going to make a big difference in this ecosystem is like, can, The inference clouds like base 10, can they deliver reliability that you would expect from a cloud, like a traditional cloud? Um, because the, like, Uh, distributed data center operations that, you know, they consume today do not offer that reliability. Right.

A And the other thing that's interesting is, um, you know, this was additional reporting from last week. Um, if you're familiar with Silicon data, they, they put together a lot of, you know, data on spot pricing and price per token performance. This is Carmen Lee's company. And one thing that, that I think it was really interesting in some, some, in an article she, uh, published last week, uh, had to do with How two pieces of compute that look identical on paper have wildly different performances, everything from reliability to cost to speed. And I think as you distribute, um, you know, have distributed inference, how do you, um, uh, you know, mash together very different types of compute and try to optimize reliability, I think is super interesting. Um, and that gets to kind of one thing I, I find really interesting that NVIDIA is doing is, is this concept of AI factories. And building AI factories, um, you know, behind corporates and AI companies. And maybe the way I unpack that is you've got kind of more large monolithic cloud players, the hyperscalers and the Neo clouds that are building large scale, um, you know, cloud environments. Uh, and a lot of where I think NVIDIA and others see this going is yes, those are going to be important components and those are going to be huge markets. But corporates fortune, you know, 500 AI companies that use a ton of compute will want dedi…

AI assessment note: “mash together very different types of compute and try to optimize reliability”