NVIDIA H100

“If I was to buy an H 100 for, let's say, 30,000 dollars, that's how much the, kind of, a single card would cost. If I wanted to rent an H 100 for one hour spot instance from GCP, it could cost me five dollars. If I rented it from, like, You know, Azure or AWS,…”

“Like, the H 100 used to be eight bucks, or something like that, per hour, like two years ago, and now you can get it for two bucks.”

“They got H 100. They didn't get the B 200. They didn't get the most current. They are a generation and a half, maybe two generations behind. So the, this was not a great deal. It was a good deal for Elon. He had them sitting around, but they were forced to tak…”

“Computer is not fungible today. So forget fungibility of compute across different manufacturers, like NVIDIA and AMD. Within a manufacturer, NVIDIA chips, for example. The H 100, the GB 200, the GB 300, these are all completely different chip types. So if you …”

“Just a couple of weeks ago, we rolled out the world's first real-time demoable real-time video generation on a single H 100 GPU.”

“I think people are clearly still getting value from H-one hundreds, and that's more than two years, right? So you know If you say it's a two-year depreciation, you're empirically wrong, right? I mean, they are and I think people are still getting value from A-…”

“They're getting close to five years old. And they're still operating well, they're still earning more than their operating costs by quite a bit. You would never deploy an H 100 today, but they're still profitable to run, right?”

“If you're running DeepSeq inference, the performance difference per GPU is like north of like six, seven X, and it continues to optimize you know, for DeepSeq inference. And so the, you know, then it's like, well, I'm only paying 60% more for six X, and it's l…”

“The marginal cost of keeping an H 100 running is much lower than the cost of keeping a human alive for a year.”

“An H 100 costs 40,000 dollars today. The yearly cost of running it is, like, thousands of dollars.”

“H 100 has the same amount of flops as a human brain.”

“Reality is whether you have an older stack or if you have even maybe the newest NVIDIA's newest H 100 in your environment, regardless of what they are, they're being choked out a little bit by these old world or these legacy orchestration and scheduling layers…”

“There's going to be a need for inference. Very hard to say whether it will be worth, you know, everybody's money to do it on H 100. That is a bubble that I think will blow some time.”

“H 100 comes along and inference is it's worth five times the price. And it may be runs twice in terms of performance on inference. That is on training. It's a lot better, but on inference, it's like maybe twice as fast when it actually, when it came out, it ra…”

“The thing is these chips are on the market. They're here. I can, you know, out tab on Chrome and get one. That is something that, you know, I don't take lightly. Availability that is right. So I think Nvidia is used to stay at least if not for the H-one hundre…”

“The supply of H 100 was actually a smooth out over the year so that they decided so that they didn't have like a big, you know, spike in deliveries and then a quarter less, right?”

“I think people don't actually realize that like a forty-ninety and like an H 100 are in a lot of ways the same card. For the non-gamers in the room, explain the forty-ninety. The chip that's inside of them is almost identical. The chip, the actual compute chip…”