GPU

“We're at an inflection point where the growth in data is outpacing the growth in compute, and I believe, I strongly believe, I'm a little biased, but, ah, that GPUs are set to kind of become a dominant force in compute over the coming decade. Ah, they already …”

“And even nowadays, like, you know, it's increasingly hard to get those GPUs in academic positions. And so it's making increasing amounts of sense for, I think the flight that we're seeing from academia to industry.”

“And ideally somebody else is, it's not venture capital dollars that are being used to buy and manage those GPUs. So it's a huge benefit.”

“I'm not an expert in the field, but I believe that this is gonna we need Another giant leap of innovation to get there. I'm not sure just throwing compute and increasing the number of GPUs without some fundamental discovery, it's gonna create this.”

“By the time you get that out and it's actually scalable and it can really train it and all the software is ready for it. And I can now go on AWS and spawn up your new hardware, like, Nvidia will also have been a hundred X faster with their latest and greatest …”

“And then if you one day need a thousand GPUs, we can get you, we can typically get you a thousand GPUs like pretty quickly, like talking minutes.”

“Modal makes it easy to build, scale and deploy applications in the data, AI, machine learning realm. So, so we basically, you can think of as like, we take, you write a little bit of Python code, and we take that code, we stick it in a container, we execute th…”

“Just on GPUs, but yeah.”

“As soon as you get to a point where there's some convergence on an architecture that's looks like it's stable and is revenue generating and developers are coming to sort of work on it and confirm that it is like the thing, then you can flip towards doing a cus…”

“It is not that I can linearly add more GPUs and train increasingly a more larger model. If I do so, the time it takes becomes exponentially longer.”

“Currently, pre-training just uses the most GPUs of all the parts, so it needs the most GPUs, right?”

“And so when you look at who rents the most GPUs in the world, it's three companies, right? So one of them is obviously OpenAI. Second one, actually they were bigger than OpenAI. They are bigger than OpenAI today, or no, they were bigger than OpenAI than OpenAI…”

“But now the workload is so large that there is room for specialization that will give you 10 X increases in certain domains, right?”

“When you run inference on a few GPUs, or when you run small scale trainings on hundreds of GPUs, margin for error is a lot larger than when you run trainings on thousands of GPUs at the same time.”

“I don't know it goes to the extreme to where GPUs are because that is very, very, very extreme. But it is quite highly, high probability that there will be shortages of CPUs going forward.”

“Now it can't run a sandbox with a GPU. It just doesn't work so that you can't have a firecracker.”

“Lambda's designed a piece of software that allows us to give you anywhere from 16 up to You know, 4000 GPUs in a web interface.”

“I think that the, that market is starting to mature that, that, that may be an eventuality is having more complex securities that surround GPUs. But I think for right now, people are starting to realize that it's a great credit investment and that's what's cha…”

“Most of the other NeoClouds either don't have the ability to launch a cluster from their website or max out, I'll say, 32 GPUs.”

“I believe that in the future, everybody in the United States will need the computational power of one GPU or more to just do their daily work You know, enjoy life, whether it's getting access, whether it's getting entertained, whether it's being productive, wh…”

“The usable life is longer than the accounting depreciation schedule. And what really matters is the economic usable life. And so what we're starting to see is that like the people who are the naysayers, oh, this is going to be, you're going to throw these GPUs…”

“DLSS is our real-time AI for graphics, and it makes a small GPU run like a big GPU. It's about 10 times more efficient because rather than computing the color of every pixel for every frame, we use AI to infer the color.”

“The JPs have a huge failure rate, so I'm sure you guys have spoken about this. Infant mortality is enormous, and they fail all the time.”

“And so the speed of moving waits to compute is about two and a half thousand times faster here than on a Wilben GP.”

“In inference in AI, it's the exact opposite. You move a huge amount of data, all the weights, from memory to compute, and you need one calculation to generate the next word. And then you have to do it again. So all the time is dominated by the movement of data…”

“Say, if you run Gemma four, On your GP, you might get like a hundred tokens per second if you have a fast card. If you run it in their cloud, you get anywhere from 800 to 1500 tokens per second. So call it 10 X faster.”