Srivastava: Customer compute demands continuously escalate to A100s and H100s
Tuhin Srivastava · No Priors Ep 56 | With Baseten CEO and Co-Founder Tuhin Srivastava · Mar 21, 2024 · at 8:28
Baseten CEO Tuhin Srivastava discusses how AI inference infrastructure requirements have shifted toward higher-end NVIDIA GPUs since late 2022.
0:00 / 0:13exact quote · 13.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“At the end of 2022, most of our customers were using T-folds and AT&Gs. You know, after that it changed to A-one hundreds. Now it's going to H-one hundreds. Like the computer needs aren't necessarily going down. They're only going up, especially as these services scale up.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Tuhin Srivastava
Srivastava: The ease of running CUDA workloads on AMD is significantly overstated
“I've, I personally think that it's pretty overstated how easy it is to run something that looks like CUDA or CUDA in some form on an AMD chip seems, seems like a challenge to me.”
Srivastava: AI Defensibility Comes from Workflow Integration, Not Models Alone
“To the extent that that is encoded in a model, I think a lot of their business will be at risk, but to the extent that it is encoded in workflows that is where they will be able to develop mode.”
Srivastava: Frontier Labs Lack the User Signal to Displace Vertical Apps
“My argument would be here is that actually, you know, it's very, very hard for a frontier model company to go to either way at that, because they just don't have access to that user signal, and what will happen over time is the folks who have access to that us…”
Srivastava: AI-Native Startups Represent 99% of Total Inference Call Volume
“I think if you look by inference count, it'd be 99% the full.”
Srivastava: Startups should not do post-training before achieving product-market fit
“Hey, go find, go prove to yourself with the best in class model that you have something worth optimizing. And I think, you know, A lot of, you know, if a customer comes to us, was that meme, which was like, it was like two years ago, it feels like there's no G…”