The Exchanges

Q to give a little bit of a history lesson to start here, which is, um, can you just talk through pre-AI wave, you know, before the last couple of years, we had been on an interesting trajectory with regard to the growth of compute as it pertains to Energy consumption overall. So you just walk us through the history there, what had been happening for the past couple of decades?

A Yeah, so let's, uh, go back to the dot-com boom. So the late-nineteen-nineties, there was a lot of focus on computing in various forms, and people were very excited about different things, including fiber-optic networks and e-commerce and other stuff, and there was a lot of speculation round about the year 2000 that Uh, computers and related technologies were gonna end up using a lot of electricity. There were projections widely cited by, you know, people of many different, uh, persuasions that, you know, the internet was gonna use half of all electricity in the next 10 years, this kind of thing. There were a couple of guys running around, you know, pushing that whole narrative. And that led to Pressure on people who are at the national labs and other places to try to figure out, well, what were the numbers? Well, you know, what was actually going on? I was a staff scientist at Lawrence Berkeley National Lab at that time, and I had done work on computing electricity before that in the early to mid-nineties, so people came to me and said, well, you know, do these numbers make any sense? And we ended up doing careful work. We found that those projections were off Massively. Like, these guys were claiming that all computing used 13% of all electricity use, and when we actually tallied the numbers, it was three percent. And we, you know, figured...

AI assessment note: “let's, uh, go back to the dot-com boom. So the late-nineteen-nineties”

Q we weren't yet seeing, we were seeing a boom in compute. We weren't yet seeing these dramatic energy efficiency improvements, but partially in response to that, in the next big growth cycle for compute, we did see all these energy efficiency improvements. And so the next time compute really took off, We didn't see the same spike in electricity consumption. Is that, like, the right way to think about it?

A That's right. Yeah, I think that the growth in the use of computing continued, right? It wasn't that it stopped, you know, it was actually, it was continuing, but the industry focused on efficiency, right? And they figured out a bunch of different ways to improve the efficiency of computing. Some of that involved redesigning the silicon, right? So you're shrinking transistors and changing hardware architectures and this kind of thing, but also it involves moving from very inefficient corporate data centers to much more efficient hyperscale data centers. And we can talk about why that's important, but basically the hyperscalers have economies of scale. And they figured out design tricks to make the, uh, efficiency of delivering computing and, sorry, electricity and, uh, cooling to the computers much more efficient. And they have much higher utilization of their equipment, and so when you think about it in terms of cost and energy use per compute, these hyperscale facilities are much, much better than the kind of traditional corporate facilities. And there was a big movement towards those hyperscalers in that That later period, towards 2018.

AI assessment note: “That's right. Yeah, I think that the growth in the use of computing continued”

Q percent growth in electricity consumption. So that, that's like the operative question right now, and I guess before we get into answering it, What are you seeing out there in terms of the variability of projections? This is one of the interesting things. Nobody really knows, right, what's gonna happen, and so I think part of the results is that I've seen forecasts that are, like, all across the board.

A Yeah, so there's huge variability in the forecast. There's even variability in what we think of as history. So for the year, 20, 22, the International Energy Agency did these projections. Uh, they did two of them in, in, in, 20, 24, and they started by trying to do history, and they looked at twenty-twenty-two, and the first study they released in January had a number that was 50% greater than the second study that they released in October, and so even the same institution looking at a historical year Had this uncertainty. They, they identified an uncertainty range of 220 to 340 terawatt hours per year for global data center electricity. So that's a pretty big Uncertainty range for a historical year. So there's a lot of things we don't understand.

AI assessment note: “there's huge variability in the forecast. There's even variability in what we think of”

Q data centers is going to look something like what it did from 2000 to 2020, which is a, a spike for a period of time where it is Growing faster, followed by a longer period of time where it is growing much slower, if at all, because of some combination of energy efficiency improvements and possibly hitting a ceiling on service demand. Is that the way to think about it?

A Yeah, I think, I think that's a reasonable way to think about it. I, I don't think anyone can know for sure, because it does depend on the growth in service demand, but assuming the industry does what it usually does, which is it sees a problem and it fixes it, uh, and they, the, the reason why I'm optimistic about that is because the industry's incentives are to reduce the cost of delivering their product. Right, and that's been historically true for data centers is that energy and capital turn out to be, you know, huge parts of their total cost, and if they can do things a lot more efficiently, they will because it will save them money and make them money. And so, it's a little different than some other sorts of energy using applications where you have, you know, you don't have these strong incentives. These, these players have pretty strong incentives to do things in the most cost-effective way. That isn't always true in a boom time. Right? When it's the two or three years of building things out super fast, they're basically just trying to find enough people and enough equipment to, to put in enough data centers to meet the demand that they perceive. But, uh, but once they, you know, have a little slack in their, uh, in their building, then they can, they can start to think, oh, how can we do this better? Or they see the constraint, right? There's all these stories, you know…

AI assessment note: “Yeah, I think, I think that's a reasonable way to think about it.”

Q able to do ever so much more, um, AI, and as a result, from an energy perspective, nothing changed. Um, that, that's my sense of kind of, like, the conventional wisdom. I know that is, you, you have a slightly different view there, but, but do you think that that is what the world generally thinks? Like, do you agree with me that that's the conventional wisdom at the moment?

A I think that The people who are delivering AI, who are suppliers of AI, they honestly believe that the demand for their product is infinite, as you described. And the Jevons paradox is a complicated thing that we don't even need to get into, but we can because it's fun and interesting. Uh, because that was used, uh, by the head of Microsoft to explain why he thought that demand would not be affected. The electricity demand would not, would continue to grow massively because essentially there's infinite demand for their product. And that assumption is something that people should examine. I think there's a lot of people who believe that, and of course, you know, in economics, if something gets cheaper, people use more of it, but the question of how much more they use is a question of, you know, what is the value being delivered by this technology, and if the value of the technology is very, very high, people will be willing to pay more to have it, right? And so, costs may come down, and maybe that means they'll use more of it, uh, but the question is, at some point, is there a kind of saturation? You know, kind of a, a leveling off of this. And so, you know, the Jevons paradox is also called the rebound effect, and it's typically used to describe efficiency. Like, the, the idea is that the efficiency of something, like a car, goes up, so it's cheaper to operate the car, so peopl…

AI assessment note: “I think that The people who are delivering AI... honestly believe that the demand”

Q Is that because of China or something? Like, why is it so hard to estimate historically?

A Well, because the, the data are often held as proprietary information, and so we as analysts, first, we, we aren't, we don't have access to the proprietary information, but when there is such information, So these data collection companies, IDC and Gartner and so on, they collect the information, but there is a time lag. For releasing the information. So there's like a lag because it has to be collected and processed and so on. And so by the time, you know, we're getting, you know, 20, 22 data, it's 20, 25, right? Or 20, 24. And so there's this time lag and it takes time to analyze and so on. So it's both the inaccessibility of certain data and then the time lags associated with real data And real analysis that leads to this uncertainty in the history. And this is distinct from the uncertainty in the projections, right, that you were, you know, that you started the question with, but I think it's important for people to understand that we, we don't even know with high precision the historical numbers for twenty-twenty-two and twenty-twenty-three. And when something grows super fast, like AI, which has been growing gangbusters, It's very hard. Like, things happen in six months or a year, and, you know, it's, it's two or three years later that we actually get the real data that people have confidence in. So, so that's a big uncertainty just in the historical data, and then we can…

AI assessment note: “because the, the data are often held as proprietary information”

Q Does that argument hold water for you? Or, like, where would you poke holes in it?

A To me, that it's, you know, it's effectively the same thing. They're basically saying that constraints are, uh, related to how fast we can supply the AI. So, effectively, even if it's not infinite, it's, it's high enough that they will You know, build as much as they can, given these other constraints. And so the question is, is that actually true? I think most people seem to believe it in the industry. I think that's pretty common. I think most people think that's true. But is it true? That's the conversation that I think people need to have, because You know, there are issues with these models. We'll talk about the efficiency side of it, because I have a lot to say about things you can do to do this more efficiently. But on the demand side, there are real questions about accuracy, right? Traditionally, what has happened, and this is another set of assumptions that the whole industry seems to make, historically, they have gotten increases in accuracy by scaling, throwing more computers at the challenge. And they've had, you know, whatever it is, 10 or 20 or sometimes 30% increases for a, uh, you know, a tenfold increase in compute capacity, and when you have, you know, computers improving very rapidly, you can do that over time, but now we're reaching the point where the scale Of the construction challenge for increasing capacity, you know, computing capacity 10 or a hundredfo…

AI assessment note: “on the demand side, there are real questions about accuracy”

Q Yeah, let's, okay, so let's talk about the projections then. What's the, what's the range of projections that you've seen?

A Well, well, let's, let's actually not start with those quantities. Let's start with conceptually what we think is happening. There's two drivers of electricity use in data centers, or AI data centers, or any data centers. There's the growth in the service demand, the demand for compute, the demand, you know, the AI queries, or search queries, or something else, some measure of service, right? And if those, uh, measures of service demand are growing, that of course puts upward pressure on electricity use. Then the other big uncertainty is on the efficiency of delivering the service. And if you can make the serve, you know, deliver the service much more efficiently, then you'll be able to meet that service demand, but not increase electricity use in, in the aggregate. So those are the two forces. And when people make a projection, um, Of what future electricity use will be, they are implicitly or explicitly making assumptions about service demand growth and efficiency of delivering that service demand. And so that's the kind of the first core level understanding of the, of what's happening here when, when people are making a projection. And they're making assumptions. Usually they're implicit assumptions that are unstated. And, uh, some are better at explaining what those assumptions are, like the, the LBNL report to Congress that came out in December of 24. That was estimating d…

AI assessment note: “let's actually not start with those quantities. Let's start with conceptually”