nvidia-h100, every mention
31 scenes · ← back to nvidia-h100
tap a year for its mentions
every year anyone Ben Gilbert 18David Rosenthal 13
Verbatim, from the transcripts: the passages where nvidia-h100 comes up
Google Part III: The AI Company. Google is amazingly well-positioned... will they win in AI? (Audio) · Acquired
- ▶ 3:38:43 Ben Gilbert I mean, these were the H 100.
NVIDIA CEO Jensen Huang · Acquired
- ▶ 43:58 David Rosenthal Their data centers are filled with A-Hundreds and H-Hundreds, and as you probably know, with the rising demand for AI, there's been a huge surge in the need for high-performing GPUs, leading to a noticeable scarcity of
- ▶ 44:14 David Rosenthal Crusoe has been ahead of the curve and is among the first cloud providers to offer NVIDIA's H-one hundreds at scale.
- ▶ 44:34 Ben Gilbert To illustrate, they already have several customers already running large-scale generative AI workloads on clusters of NVIDIA H-H-H-GPUs, which are interconnected with 3200 gigabit InfiniBand and leveraging Crusoe's network-attached block…
Nvidia Part III: The Dawn of the AI Era (2022-2023) (Audio) · Acquired
- ▶ 23:45 David Rosenthal Nvidia is one of their major partners, and literally Crusoe's data centers are nothing but racks and racks of A-one hundreds and H-one hundreds.
- ▶ 59:36 Ben Gilbert So while memory has scaled up, I mean, we're gonna get flashing all the way forward, the H-one hundreds on-chip RAM is like, 80 gigabytes.
- ▶ 1:06:51 David Rosenthal Grace, CPU, Hopper, GPU, Grace Hopper, the H-one hundreds.
- ▶ 1:07:07 David Rosenthal So you buy a, you know, top-of-the-line RTX-X-X, what-have-you gaming card right now, that is no longer the same architecture as the H-One-Hundreds that are powering ChatGPT.
- ▶ 1:08:04 David Rosenthal And by NVIDIA bifurcating their chip architectures into a gaming segment that does not have this latest COWAS technology, this allows them to monopolize, like, a huge amount of TSMC's capacity to make the COWAS chips, specifically for…
- ▶ 1:11:32 Ben Gilbert Like, believe me, for all sorts of reasons, NVIDIA wants everyone who wants H-one hundreds to have H-one hundreds.
- ▶ 1:11:56 David Rosenthal So, like you were saying, Ben, of course you can, and lots of people do, just go buy H-One hundred.
- ▶ 1:12:55 David Rosenthal So they're selling H- one-hundreds and A- one-hundreds before that to all these cloud providers.
- ▶ 1:15:14 Ben Gilbert Well, let's talk about the H-One hundred.
- ▶ 1:15:24 Ben Gilbert One GPU, one H-One hundred, cost 40,000 dollars.
- ▶ 1:15:36 Ben Gilbert The 500,000 dollars is the eight 40,000 dollar H-one hundreds in a box with the gray CPU and, you know, the nice bow around it.
- ▶ 1:16:03 Ben Gilbert And these 40,000 dollar H-one hundreds have margin of their own.
- ▶ 1:16:21 Ben Gilbert So, the reason you want an H-one hundred is they're 30 times faster than an A-one hundred, which mind you is only like two and a half years older.
- ▶ 1:16:32 Ben Gilbert The H-one hundred is literally purpose-built for training LLMs, like the full self-driving video stuff.
- ▶ 1:16:54 Ben Gilbert This one H one-hundred, which they're calling A-GPU, has 18 and a half thousand cores that are capable of running CUDA software.
- ▶ 1:17:41 Ben Gilbert This is one H 100.
- ▶ 1:21:53 David Rosenthal So today, like we've been saying, if you want to use H-one hundreds and A-one hundreds, say you're an AI startup, the way you're probably going to do that is you're going to go to a cloud, either a hyperscaler or a dedicated GPU cloud like…
- ▶ 1:22:24 Ben Gilbert That's eight A 100 for about 30 bucks an hour, or you can go over to AWS and get a P five dot 48 X large instance, which is eight H 100, which I believe is an HGX server for about a hundred dollars an hour.
- ▶ 1:25:19 David Rosenthal So starting price for DGX Cloud is 37,000 dollars a month, which will get you an A-one hundred based system, not an H-one hundred based system.
- ▶ 1:25:49 David Rosenthal This is not H-one hundreds.
- ▶ 1:49:31 David Rosenthal As part of that, NVIDIA can no longer sell their top-of-the-line H-one hundreds or A-one hundreds to anybody in China.
- ▶ 2:15:56 Ben Gilbert Then the process that the H 102.5 D is using to make sure that memory is extremely close to compute.
Costco (Audio) · Acquired
- ▶ 1:40:17 Ben Gilbert Literally, their data centers are nothing but rows and rows of the latest NVIDIA A-Hundreds and H-Hundreds, so they have the benefit of focus and specialization.
Nike (Audio) · Acquired
- ▶ 1:08:22 Ben Gilbert A 100 and H 100.
- ▶ 1:08:36 Ben Gilbert A single NVIDIA H 100.
Nvidia: The Machine Learning Company (2006-2022) · Acquired
- ▶ 1:07:03 Ben Gilbert It's not the A-one hundred, it's the H-one hundred.
- ▶ 1:07:05 David Rosenthal Uh, so the A-one hundred, they just announced the H-one hundred.