Janus: AI training latency constraints require single-site gigawatt data centers
Brian Janous · Under the hood of data center power demand · Jun 13, 2024 · at 11:19
Former Microsoft energy lead Brian Janus discusses how AI computing architecture changes physical data center siting and electrical grid requirements compared to traditional distributed cloud computing.
0:00 / 0:38exact quote · 38.6s
720p mp4 · rendered on demand · StarZero watermark
“Because those training models are in and of themselves a big machine, and so you can't, and this is not my forte and my area of expertise around the actual architecture inside of a training model, but my understanding of sort of the constraints there is that you can't have a training model that exists across multiple regions because there's a very low latency requirement for that model, which means it has to largely be on a single building or single campus or very close together. So that started forcing the need to find individual points on the grid that could support by themselves gigawatt scale.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Brian Janous
Behind-the-meter nuclear data centers have no credible case for decades
“I don't think there's a credible argument for behind the meter nuclear at a data center in the near future. And by near future, I mean, The next couple of decades.”
Solar and batteries have killed Texas's $9,000 per megawatt-hour price spikes
“The only argument, you know, you ever had for building something like a baseload generator in Texas is that sometimes the price would go to 5000 or 9000 dollars a megawatt hour, but with the massive amounts of solar and storage coming out of the grid, which yo…”
Janous: Traditional colocation data centers lack the energy talent for AI.
“And those companies that, that filled the gap in the last era are really not particularly well equipped, equipped to fill it in this era because the depth of energy talent at those companies is actually quite thin.”
Janous: Compute demand will exceed available power supply through 2030.
“It's possible, though I suspect that given the, my fundamental belief is that the demand for compute, let's just say through twenty-thirty, will exceed the available power in the market. And therefore, the marginal value of the next watt or gigawatt, frankly, …”
Janus: AEP will not connect 90 gigawatts in next decade
“There's no reason to believe that AEP is going to connect 90 gigawatts in the next 10 years.”
Janus: AI Training Loads Will Not Run as Intermittent Renewable-Following Workloads
“So I think it's a little bit overblown to say, and I think this is also true of Something like the conversation around crypto, but that these are highly curtailable loads that you could just, you know, attach a training model to a wind farm and only run it, yo…”