Appenzeller: AI compute will bifurcate between local and cloud tasks
Guido Appenzeller · Chasing Silicon: The Race for GPUs · Aug 25, 2023 · at 19:42
Guido Appenzeller evaluates the performance and infrastructure trade-offs between local device compute and cloud-based server compute for AI workloads.
0:00 / 0:33exact quote · 33.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“There's probably going to be a little bit bifurcation there on quality and parameters, right? You can run things locally but you can probably run them a lot better in the cloud, right? Because you have a much bigger server there. So it probably depends on what you want to do, right? If I just want to have a better spell checker you know, that, that checks my email or maybe, you know, just some simple completion. That's perfectly fine. I can run that on my phone. On the other hand, if I want something, you know, that is more, You know, write a good speech or, you know, summarize a complex text. They might be like, all that are going to run the cloud because it takes so many more operations.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Guido Appenzeller
Appenzeller: Intel-Nvidia partnership means AMD is 'fucked'
“I think AMD is fucked, right? I mean, they're, you're just, If your two arch nemesis suddenly team up, that's the worst possible news you can have, right? They were already struggling, right? Their cards are good, their software stack is not, right? They were …”
Appenzeller: Capital moats in AI are shallow speed bumps
“If that assumption is true, I think this means that the moat that's created by these large capital investments is actually not particularly deep, right? It's more of a speed bump than Then, you know, something that prevents new entrants.”
Appenzeller: Intel lacks competitive AI chips and its Gaudi effort is done
“They can't, they don't have anything competitive, right? There was the Gaudi effort that's more or less done, right? There was the internal graphics chips, which never competed really at the high end, right?”
Appenzeller: Intel's customer concentration allowed hyperscalers to push down prices
“One of the biggest problems we had was that our customer base sucked, right? I mean, we were selling to, most of the chips went to the large hyperscalers, you know, which they're way too concentrated, and they build their own chips, and so you can push down yo…”
Appenzeller: High compute costs will force AI into usage-based pricing
“It'll push more and more to, I think, just usage based pricing, right? I think if you have a, if you have an underlying commodity that you're reselling to some degree that has, that is that large a part of your cost of goods, right? You need to go to usage bas…”
Appenzeller: Novel coding lacking AI training data is only 0.01% of development
“I think the good news is that is 0.01% of all software development, right, for the, I don't know, you know, 100,000 ERP system implementation or so, right, that we have tons of training data, and I think these tools can be very, very powerful.”