Appenzeller: Naive AI training runs at under 10% GPU utilization
Guido Appenzeller · The True Cost of Compute · Sep 1, 2023 · at 5:39
Guido Appenzeller, Andreessen Horowitz (a16z) special advisor, discusses real-world GPU hardware utilization challenges during AI model training.
0:00 / 0:07exact quote · 7.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“If you naively implement it, you can probably go below 10% utilization, but, you know, you can probably get into the tens of percent with a little bit of with a little bit of work.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Guido Appenzeller
Appenzeller: Intel-Nvidia partnership means AMD is 'fucked'
“I think AMD is fucked, right? I mean, they're, you're just, If your two arch nemesis suddenly team up, that's the worst possible news you can have, right? They were already struggling, right? Their cards are good, their software stack is not, right? They were …”
Appenzeller: Capital moats in AI are shallow speed bumps
“If that assumption is true, I think this means that the moat that's created by these large capital investments is actually not particularly deep, right? It's more of a speed bump than Then, you know, something that prevents new entrants.”
Appenzeller: Intel lacks competitive AI chips and its Gaudi effort is done
“They can't, they don't have anything competitive, right? There was the Gaudi effort that's more or less done, right? There was the internal graphics chips, which never competed really at the high end, right?”
Appenzeller: Intel's customer concentration allowed hyperscalers to push down prices
“One of the biggest problems we had was that our customer base sucked, right? I mean, we were selling to, most of the chips went to the large hyperscalers, you know, which they're way too concentrated, and they build their own chips, and so you can push down yo…”
Appenzeller: High compute costs will force AI into usage-based pricing
“It'll push more and more to, I think, just usage based pricing, right? I think if you have a, if you have an underlying commodity that you're reselling to some degree that has, that is that large a part of your cost of goods, right? You need to go to usage bas…”
Appenzeller: Novel coding lacking AI training data is only 0.01% of development
“I think the good news is that is 0.01% of all software development, right, for the, I don't know, you know, 100,000 ERP system implementation or so, right, that we have tons of training data, and I think these tools can be very, very powerful.”