GPU

“We can get significant speedups when we port these algorithms To GPU, when we leverage GPUs and FPGAs to build future servers, and what that means is we can bring that scalability gap down, and so we can get about 10 X back if we move by taking these, ah, by t…”

“What we are doing now with 10,000 GPUs, people will probably do in the future with maybe a small GPU cluster.”

“What's driving the whole machine learning revolution, if we can call it that is I mean, there are three things, and one is Moore's Law, so the availability of a huge amount of computation, and in particular the development of GPUs, or the application of GPUs t…”

“GPUs are wonderful for training the deep learning algorithms, but I think there is still a lot of space in rapid testing or inference time Chips where it can be used in recognition, you know, in devices, in embedded devices.”

“And we went through this again, when GPUs came out, we actually were some of the first applications on GPUs Even before programming languages existed on GPUs.”

“There is room to create things that are 1000 times better or faster or more performant from a power perspective by creating custom ASICs for ML.”

“Games have driven GPUs. So like, just like the, that market is one where the gamers have been endlessly hungry for more polygons, and that created this kind of, you know, Nvidia and this whole industry around it. Which then had these interesting, you know, the…”

“It's you know, hours in the day that go by where someone's not asking for GPUs is almost non-existent.”

“When it comes to training models, the current paradigm for training models requires that all of the GPUs that train the model, these, you know, these computers that do the training, they all have to be like in the same room.”

“Right, which is like, and then once they get the chips, we're not gonna have enough power, and once we have the power, we're not gonna have enough cooling, and so, so, so, so there are many, many steps.”

“And that starts with them placing orders, 12 to 36 months in advance to take delivery of GPUs, because if you don't get in front of that line, it's over. You're getting it after everybody else, right?”

“GPU failure rates at best are the same and likely worse, right? Gen on gen, because everything's getting hotter, faster, et cetera.”

“How you buy GPUs is like buying cocaine. You call up a couple people, you text a couple people, you ask, yo, how much you got? What's the price?”

“HBM is more than half the cost of the GPU.”

“Most companies in the valley spend, what, 75% of their round on GPUs, right?”

“This, you know, this is like Intel with the GPU. Like they missed the GPU. In 2005. Right. And they missed the opportunity to buy the company, to do the work or whatever, and they kind of missed the data center too. It just took a longer time to figure out tha…”

“GPUs still internally use all the logic of showing things on a screen. You know, you buy like a GPU for a data center. It still has little ports on the back to connect a monitor to it. They'll never be used, right?”

“When there's a constraint, we almost like, which happens all the time we almost always prioritize giving the GPUs to research over supporting the product. Part of the reason we want to build this capacity so we don't have to make such painful decisions. There …”

“At the peak of the bubble, 97% of the fiber that had been laid In America was dark. Contrast that with today. There are no dark GPUs.”

“We were the first to really build a large-scale compute cluster. A thousand now where we have plans to move to the 10,000 range.”

“The GPUs are somewhat zero-sum, right? So the data isn't.”

“Transformers are really they're a big innovation because they made the constructs of the GPU work extremely well.”

“If you were designing AI chips from scratch today, you wouldn't build a full GPU. You would build dedicated AI chips that were much more straight, much more specifically adapted to AI and would have, I think, would just be much more economically efficient”

“With AI, if you have proprietary data and you have enough GPUs, you can solve, like, almost any problem.”

“With AI, if you have data, you know, particularly proprietary data, and you have enough GPUs, you can solve, like, almost any problem. It is magic. But it means that you can throw money at the problem. And we've never had that in tech.”

“If you put a GPU in the system and a data center, it gets fully utilized immediately.”

“If you have enough money and some good data, you can buy enough GPUs and solve basically anything in software.”

“Serving large language model is a fundamentally different problem. Because serving it requires to run it on accelerators like GPUs or TPUs, and it is a computationally intensive process that will require a lot of engineering and ensuring that for each request,…”

“AlexNet, First, you know, kind of, like, neural network to run on, on GPUs that we care about ran on two GPUs. And that's not like there are no missing decimal points or commas in there. Literally two.”

“Probably BERT. And before that, it was like ResNet for computation, like images, computer vision classification. So ResNet already need to run on NVIDIA K-eighty, which is kind of one of the first SEU on AWS and other places. And, but way over, but even at thi…”

“This is like, we're flat out, and people are reselling GPUs for [514] Ben Horowitz: Four times what they bought them for”

“By the way, it is so bad that right now if we have new companies going for GPUs, it's often in Mexico or Australia or in other country just because it is so difficult in the United States.”

“They, I, for sure, I don't think they will generate free cash flow anytime soon. I think they're gonna generate a lot of operating cash flow, and then they'll use that to buy a lot of, you know, GPUs XPUs, whatever, whatever I'm gonna call them.”