Papermaster: AI architectures should split workloads between cloud efficiency and edge latency
Mark Papermaster · No Priors Ep. 53 | With AMD CTO Mark Papermaster · Feb 29, 2024 · at 34:59
AMD CTO Mark Papermaster explains the architectural division between cloud and edge compute for AI applications.
0:00 / 0:30exact quote · 30.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Writing applications that where you don't have that latency, that, ah, you know, that, ah, dependency on, on a lag in computing, run it on the cloud. It's going to be the most, ah, it's going to be the most efficient because you're optimizing this massive data center, ah, with the most efficient computing. But write the algorithm such that where you do have that need for super low latency, you just need that instance response, have those aspects of the algorithms be at the edge, or in fact on your end user device.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Mark Papermaster
Papermaster: AMD's MI300 AI accelerator beats rivals on compute and power
“It's got a tremendous performance advantage, and we did that very purposely. We created very efficient engines for the math processing that you need for that training or inference processing, but we also brought the memory that you need to have more efficient …”
Papermaster: Hugging Face tests new models equally on AMD and NVIDIA GPUs
“We've partnered with Clem and his team. They test as they release any of those language models they're testing on AMD with our Instinct GPUs equally as they're testing on NVIDIA.”
Papermaster: AMD rejects proprietary walled gardens in favor of open source
“Sarah, the point is, we're not about locking in someone with a proprietary wall garden software stack. What we want is we want to win with the best solution, and we want, or we're committed to open source and we're committed to giving our customers choice.”
Papermaster: Electrical power availability will become the ultimate AI data center constraint
“This is, I think, ultimately going to be certainly a key constraint and you see you know, all the major operators looking for sources of power”