Belady: Modern GPU modules consume 1,000 watts compared to 200W CPUs
Christian Belady · Can chip efficiency slow AI's energy demand? · Jul 18, 2024 · at 8:16
Christian Belady discusses the power consumption differential between legacy CPU data center workloads and modern AI GPU clusters with host Shayle Kann.
0:00 / 0:31exact quote · 31.1s
720p mp4 · rendered on demand · StarZero watermark
“Yeah, the traditionally CPUs run in the realm of about a hundred to 200 watts is typically what they would consume in terms of power. GPUs are substantially more, and generation after generation, it's increased And they're 200. And 50 watts to the earlier versions, to now they're going up to about a thousand watts for a GPU module. So it's significantly more power than what you have seen in, in CPUs in the past.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Christian Belady
Belady: Chip efficiency gains increase total power demand via Jevons Paradox
“In fact, getting it very efficient may actually drive more use of it to where it'll increase and not decrease. And so I think there's a much broader view you have to take when someone says something's more efficient, it's, and they say it's going to solve all …”
Belady: AI's biggest benefit is hardware engineering efficiency, not consumer chatbots
“The reality is the real benefit is engineering the products that we have to drive efficiency. And I see. I see that happening relatively slowly.”
Belady: Moore's Law made the hardware industry complacent for six decades
“And frankly, it's actually made the industry complacent. Not much has changed over the years in the past six decades. If you look at it, cause Moore's law pretty much gave performance improvements for free.”
Belady: Multi-Chip Modules Drive Power Requirements Up Every Generation
“Now what you're seeing is As you're integrating more and more and getting more and more into the multi chip module, the power keeps going up generation after generation. And so that's a kind of a new dynamic that's really driving a faster need for power to go …”
Belady: AI power demand is purely additive, unlike cloud efficiency migrations
“So now what you see is AI coming along and that's like a completely new application and it's a massive application. So now that's just stacking on top of this other transition of, as things move from enterprise Into the cloud. And now your stack on top of that…”
Belady: Speed of light latency forces AI into massive, centralized data centers
“I think, and again, this is probably beyond my scope, but I think it really has to do with the size of the models and the latency, right? It takes too much time for light to travel larger distances, as hard as that is to believe. And so they tend to concentrat…”