Nvidia H100, every mention

mentionsepisodesper episode tap a year for its mentions

Verbatim, from the transcripts: the passages where Nvidia H100 comes up

loading…

▶ 1:12:18 David George Yeah, that's, this is how you know that, like, very old, whatever, the price is sold up of H 100 is very high.

▶ 33:57 David George Um, and we very closely monitor the price of chips in the secondary market, and the price to rent A-one hundreds and H-one hundreds, um, has actually held up very, very well.

▶ 1:22:21 unnamed speaker Um, but one of the findings that I thought was really interesting was TCO was sort of 1.66 X, uh, H 100 for GB 200. 2 times in the scene

▶ 10:32 Dwarkesh Patel Like, the marginal cost of keeping an H 100 running is much lower than the cost of keeping a human alive for a year.
▶ 14:31 Dwarkesh Patel Um, an H 100 costs 40,000 dollars today. 3 times in the scene
▶ 35:50 Dwarkesh Patel The reason I find that implausible is that I, I think that is, that will be true in the short term because, um, right now there's ten million H 100 equivalents in the world. 8 times in the scene

▶ 51:56 unnamed speaker How could it be an answer other than Nvidia other than H 100?

▶ 2:22 Guido Appenzeller If you look through the steps, right, pre-training, that's typically done on very large computer infrastructure, that's where, you know, you need the large 10,000 H-one hundreds or, or, or more, and you pretty much want to train on all the…

▶ 20:24 Jeff Schmidt And there is a real reason, which is that we don't have 20,000 H 100. 3 times in the scene
▶ 24:59 unnamed speaker H 100.
▶ 41:21 Anjney Midha People buying 10,020 thousand, soon, you mentioned Elon buying a 100,000 H-one hundreds, you know, co-located in a single place.
▶ 44:20 Anjney Midha The current distro, uh, experiment still uses H 100. 8 times in the scene
▶ 49:14 Jeff Schmidt What we didn't have was the huge stack of H 100. 3 times in the scene
▶ 54:29 unnamed speaker Uh, it could be possible to train seven B to like four trillion tokens with maybe 1000 H 100, like just rented all over run pod, right?
▶ 1:09:05 Anjney Midha Of people who run data centers who might, which might have, you know, two K plus H 100. 3 times in the scene
▶ 1:12:02 unnamed speaker And the problem with that is that right now on specialized, unspecialized hardware, like general hardware, like Nvidia H 100, the inference time and the backdrop time is almost the same.

▶ 1:12:47 Chris Dixon So instead of having to go buy, you know, Facebook just bought, I think it was 10,000 H 100, ten billion dollars in capex.

▶ 33:35 Andreas Blattman But now we have extremely nice accelerators and like with the newest H-H-E-H-E-H-E-H-E-H-E-H-E-S, it's insane how fast these GPUs are actually running, how fast you can train models on those.

▶ 4:17 Noam Shazeer Well, I, I think we just need a sort of global pause of like six months, no, about four months until we get enough H-one hundreds online to train our next model.
▶ 13:25 Noam Shazeer Like if, I mean, if you just look at it, um, I, I think I saw an article yesterday, like NVIDIA is going to build like another one and a half million H 100, like next year. 2 times in the scene

▶ 7:18 Dario Amodei Um, and so I think that, that factor of 100 plus the compute inherently getting faster with the H-one hundreds, uh, that's been a particularly big jump because of the move to lower precision.

▶ 7:35 Guido Appenzeller They're coming up with a new one called the H-one-hundred, you know, which is of the next generation.

Turn any episode into a week of clips.