Belady: Chip efficiency gains increase total power demand via Jevons Paradox
Christian Belady · Can chip efficiency slow AI's energy demand? · Jul 18, 2024 · at 17:39
Data center pioneer Christian Belady discusses why hardware efficiency breakthroughs, such as Nvidia's Blackwell GPU claims, will not reduce aggregate data center power draw.
0:00 / 0:37exact quote · 37.7s
720p mp4 · rendered on demand · StarZero watermark
“In fact, getting it very efficient may actually drive more use of it to where it'll increase and not decrease. And so I think there's a much broader view you have to take when someone says something's more efficient, it's, and they say it's going to solve all your problems and we're not going to have this power increase because everything's going to be solved with this, in this case, as you're saying, the 25 X. I think what happens is the cost of compute goes down. And when the cost of compute goes down, applications that weren't viable as a business before, now all of a sudden become viable, and that's what drives everything.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Christian Belady
Belady: AI's biggest benefit is hardware engineering efficiency, not consumer chatbots
“The reality is the real benefit is engineering the products that we have to drive efficiency. And I see. I see that happening relatively slowly.”
Belady: Moore's Law made the hardware industry complacent for six decades
“And frankly, it's actually made the industry complacent. Not much has changed over the years in the past six decades. If you look at it, cause Moore's law pretty much gave performance improvements for free.”
Belady: Multi-Chip Modules Drive Power Requirements Up Every Generation
“Now what you're seeing is As you're integrating more and more and getting more and more into the multi chip module, the power keeps going up generation after generation. And so that's a kind of a new dynamic that's really driving a faster need for power to go …”
Belady: AI power demand is purely additive, unlike cloud efficiency migrations
“So now what you see is AI coming along and that's like a completely new application and it's a massive application. So now that's just stacking on top of this other transition of, as things move from enterprise Into the cloud. And now your stack on top of that…”
Belady: Speed of light latency forces AI into massive, centralized data centers
“I think, and again, this is probably beyond my scope, but I think it really has to do with the size of the models and the latency, right? It takes too much time for light to travel larger distances, as hard as that is to believe. And so they tend to concentrat…”
Belady: Dispatching data center backup generators could solve short-term grid shortages
“That's something that we could probably use everywhere in the short term. That's a way to solve that, the energy shortage problem. I don't think it's magic. Every data center has generation, backup generation. Why, why can't it be used?”