A100
product on 5 shows · 11 statements across 6 episodes
Acquired 
Latent Space 
No Priors 
Invest Like the Best 
20VC
11 statements about A100, every show
Patel: GPU useful life is reaching 7 to 8 years, not under 5
“There's people who have argued GPUs full lives are less than five years. Complete nonsense. There are clusters now re-signing three or four year old hopper clusters re-signing for three or four more years. There's a 100 clusters that are re-signing for another…”
Feldman: Claims of a two-year AI chip depreciation cycle are wrong
“I think people are clearly still getting value from H-one hundreds, and that's more than two years, right? So you know If you say it's a two-year depreciation, you're empirically wrong, right? I mean, they are and I think people are still getting value from A-…”
Srivastava: Securing H100s still requires weeks of provider negotiations and escalations
“I think customers are still struggling with availability for the most premium chips. And I think, you know, whether that's eight, 108, 100, I think even when there is availability, you're oftentimes looking for like three to six weeks of negotiating with cloud…”
Patel: Hugging Face libraries achieve only 15% MBU for inference
“Hugging Face's libraries are actually very inefficient, like incredibly inefficient for inference. You get like, 15% MBU on, on, on, on some configurations, like eight, eight, eight, eight, eight, eight, 100, and LLAMA-seventy-beat, you get like, 15%, which is…”
Polosukhin: Repurposed crypto mining GPUs cannot meet frontier AI training needs
“The challenges, the GPUs there are like, not the ones that AI folks want to use, right? Like kind of all the AI is really zeroed in on like, how do we get a 100 or H 100 and the GPUs that like folks used for Ethereum mining and like similar is like older ones …”
Nvidia split consumer and data center GPU architectures in September 2022
“Up until NVIDIA's current GPU generation, the hopper generation of GPUs for the data center, there was only one GPU architecture at NVIDIA, and that same architecture and those same chips from the same wafers made at TSMC, some of them went to consumer gaming …”
Nvidia launched the H100 GPU in September 2022 retailing at $40,000
“So they launched it in September, twenty-twenty-two. It's the successor to the A-One hundred. One GPU, one H-One hundred, cost 40,000 dollars.”
Nvidia's H100 GPU is nine times faster for AI training than A100
“So, the reason you want an H-one hundred is they're 30 times faster than an A-one hundred, which mind you is only like two and a half years older. It is nine times faster for AI training.”
Cloud instances cost $30 hourly for A100s and $100 hourly for H100s
“You can get access to a DGX server. That's eight A 100 for about 30 bucks an hour, or you can go over to AWS and get a P five dot 48 X large instance, which is eight H 100, which I believe is an HGX server for about a hundred dollars an hour.”
Nvidia DGX Cloud starting prices are $37,000 monthly for an A100 system
“So starting price for DGX Cloud is 37,000 dollars a month, which will get you an A-one hundred based system, not an H-one hundred based system.”
DGX Cloud rental pricing yields a three-month capex payback on A100 hardware
“A listener helped us out and estimated that the cost to actually build an equivalent A-one hundred DGX system Would be today something like a 120 K. Remember, this is the previous generation. This is not H-one hundreds. And you can rent it for 37 K a month. So…”