GPU

“It's why these companies need, you know, millions of GPUs to train on. Not for, oh, I'm going to make a million GPU model. But because I need to try a bajillion different things because I don't know what will work.”

“They have ByteDance, who's, I think, the third largest user of GPUs in the world after OpenAI, and You know, probably Meta, although ByteDance may be bigger than Meta, the third largest user of GPUs in the world, or second maybe, even ByteDance is.”

“500 megawatts is twenty five billion dollars of CapEx, like, come on, like, once you put in the GPUs and everything, right, it's like, that's a ton of money”

“It actually turns out that as these models get bigger, you can no longer run an experiment on a thousand GPU cluster and replicate it on a 100,000 GPUs. You need to run that experiment On 50,000 GPUs, and maybe it takes, you know, days, and so there's a very h…”

“Moving the big recommender systems that power the advertising and the recommendation Systems from CPUs to GPUs, and you've had massive efficiency gains, and that's why all the revenue growth at these companies has accelerated.”

“And the ROI on AI has Empirically, factually, unambiguously been positive. Like, I just always find it strange that there's any debate about this, because the largest expenditures on GPUs are public companies. They report something called audited quarterly fin…”

“Now, if you have the data and you have enough GPUs, you can solve damn near anything.”

“The last Biden administration executive order said that you could not sell a GPU without federal government approval. Like, that was a real executive order, and that got reversed, but like, we were that close. To being basically out of the global chip game.”

“There's people who have argued GPUs full lives are less than five years. Complete nonsense. There are clusters now re-signing three or four year old hopper clusters re-signing for three or four more years. There's a 100 clusters that are re-signing for another…”

“Google and Anthropic are not heavy, heavy users of GPUs on the training side, but OpenAI, they'll, they'll start having their new class of models.”

“So the environments can get more and more complex, and those environments run on CPUs. They don't run on GPUs. They don't run on ASICs. The ASICs run the model,”

“And that was really strange because by Deep Seek Monday, it was super clear That this was going to be the most positive thing that had ever happened to compute demand. Prices in the AWS availability zones in Asia had already, like, doubled. You were seeing GPU…”

“The companies that are doing the best today, that are seeing kind of their values increase the most, that are creating economic value, Are the companies with the highest ratio, highest effective ratio of utilized GPUs per human.”

“And you have TPU, Tranium, and AMD, Which are all you know, essentially trying to be a better GPU, and today I think probably Tranium is doing the best.”

“And the disaggregation of inference means that I think these GPUs are going to have 10 or 15 year lives.”

“Every GPU, every TPU, every AI chip that was serving these models were just, like, fundamentally built before this and are retrofit to serve these modern models.”

“Bitcoin miners run at under a quarter of the voltage of GPUs.”

“And so we were trying to think about like a lot of things such that if the revenue growth took longer, To materialize than we thought it might, we could have, you know, consumer apps and media and all these other things that could help us monetize the GPUs tha…”

“You can, in a lot of cases, get slightly better outcomes at half the cost. But again, that half the cost, I think a lot of people hear that, they're like, that's bad for AI demand. It's actually not at all because the cost the user pays has, you know, is just …”

“So you have, like, on the calls, you have both Andy Jassy and Cyanadella are out there saying, look, we're just building data centers. Like, these are the shells. We might not use them now. Maybe we'll use them in the future. And we only buy GPUs when we know …”

“Cerebris, Grok and a couple others that are coming out of stealth now, I think have made a very interesting bet on not just building another GPU, but actually building a different kind of accelerator that focuses on a different memory hierarchy. They want to m…”

“That operation runs at, you know, 70, 80% of peak utilization, and it's limited not by software, but by power. The way NVIDIA quotes peak flops is a little optimistic. You never hit that because of power throttling”

“There's a fundamental trade off on the GPU between being throughput oriented or latency optimized. And everyone has chosen latency optimization because the shape of usage was chatbot oriented.”

“Cerebris and Grok and maybe a couple others, you should think of them as accelerators. What they are really good at is being used in conjunction with a more traditional GPU-like device that critically has this off-chip memory built in.”

“The GPU is fundamentally a throughput machine. The GPU is happiest when you give it a lot of work to do and let it chew through that work at peak utilization of its compute units.”