AI Training

“The failure rates of GPUs used so intensively for training purposes are much higher than inference specific usage.”

“Trading has a percentage of semiconductor demand to compute is going to asymptote to something not approaching zero, but very small.”

“Because at scale, the number of chips deployed for inferencing will be orders of magnitude greater than whatever you're doing for training.”

“This is not a time, right, where that traditional kind of, ah, weekly training sessions, right, is working anymore. And also, also when you deliver training today, right, you might have 80% sitting there saying, I already know all this, and 20%, wow, this is w…”

“There's training that mostly still all happens here in the United States. For USG reasons, for making sure that a national asset in effect is happening on US soil.”

“Inference, I think, is very sensible for orbital compute. Training will be done on Earth for a long time.”

“So before it was about accruing world knowledge and the web helps a lot with this. Now it's more and more about acquiring know-how.”

“So there's really no difference between training infrastructure we deployed to build those capabilities and what our customers are ultimately using to serve them.”

“Six months ago, I would have said it was two-thirds training and one-third inference. It's probably close to fifty-fifty now.”

“I actually think AI training will be used next in banking and healthcare, and then after that in, in many other different enterprise contexts.”

“For a multi-stage reasoning test that requires a PhD in multi-different languages, and, like, human feedback is going to be important in that for the next decade.”

“I think in that, but for that particular problem, my guess is that problem ultimately has to be solved through legislation. It's ultimately a legislative question.”

“The counter argument to that, which, you know, which we believe is, well, it's not copying, right? There's a distinction between training and copying, just like in the real world, there's a distinction between reading a book and copying the book, you know, as …”

“For maybe five more years, you could have, you could keep increasing the share of energy that we're spending on training data centers or the fraction of TSMC's leading edge nodes. Wafers that we dedicate to making AI chips, or even the fraction of GDP that we …”

“We found that there's actually something very, very, very precise and surprising underlying AI training. This really blew us away that there are these nice trends that are as precise as anything that you see in physics or astronomy.”

“You can see scaling laws in the reinforcement learning phase of AI training.”

“And this is an ongoing effort where OpenAI, Anthropic, Google's teams are paying a lot of money for experts to give the AI their own personal knowledge.”

“Training scales the size of your research team. Inference scales the size of your customer base.”

“It only works if you really know which chip you're going to do. If you don't, then it's a little harder. So it makes, in my head, it makes more sense for inference where you've already established it, but for training there's so much, like, experimentation.”

“In terms of infra, probably the number one thing that is the number one difference between these two is the need for interconnect. So if you do, you know, production, you, if you can avoid to have interconnect between, you know, let's say a cluster of GPUs, of…”

“When we started, the first misconception, which people don't hold anymore, is that training was more expensive than inference.”

“In fact, there's more inference in training than there is updating the model weights, because you have to generate hundreds of possibilities And then, oh, you only train on a couple of them, right?”

“Look, I think it's, it would be an easy investment if there is a predictable outcome, but I don't, do you really think that just adding GPUs and with the existing algorithm to train, they will suddenly become, become godlike, intelligent? I don't understand. A…”

“I certainly understand the urge for people to see the numbers being spent on training and be concerned that it's not going to recoup in value, but I think that Those numbers are actually small relative to the long-term value that the technology will deliver.”

“Because those training models are in and of themselves a big machine, and so you can't, and this is not my forte and my area of expertise around the actual architecture inside of a training model, but my understanding of sort of the constraints there is that y…”

“So I think it's a little bit overblown to say, and I think this is also true of Something like the conversation around crypto, but that these are highly curtailable loads that you could just, you know, attach a training model to a wind farm and only run it, yo…”

“The key constraint with energy is not necessarily, is there enough energy in the world, but more so for training, is there enough energy in one place?”

“It will make training more expensive because instead of just doing one backward pass, you now potentially have to do many forward passes because at each forward pass, you're going to come up with some output. Then the model has to decide which of those outputs…”

“In training, I think, you know, there's less software differentiation. So in training, I think there's certainly, like, better economics of, like, buying big clusters.”