compute

“If you don't have enough compute, you figure out how to make do with smaller models, but like, no one as of today, I think, would feel like they have enough compute. I don't think, like, I've heard any company within the AI space be like, oh yeah, like, we fee…”

“Being able to invest more or less compute into getting more or less accurate answers is, I think, one of the core things we care about, and that I think is currently undervalued in the AI space.”

“Fundamentally, I don't think, like, the, like, the stuff has actually, like, the underlying principles of research hasn't really changed that much, except for, like, compute.”

“One thing that they said in the paper, it's not here, but that that it, they, it takes about two X to compute to train the This consistency model from as a as a distillation of whatever they distilled from. So approximately twice the compute.”

“We're back in the hardware era where people are acquiring and managing and optimizing compute. And I think that will really matter in terms of capability and companies.”

“What you basically have to do is you have to put more compute into the query, into the search, until you get the full comprehensiveness. And I think there's an interesting point here, which is that not all queries are made equal. Some queries, just like this b…”

“The thing I do feel reasonably confident about saying is that the test time inference is probably going to quite significantly expand the amount of compute that was used for inference.”

“And so the hack of a VC or some capital partner offering equity for compute is always some arbitrage on the credit risk.”

“We're going to move from a world where most of the compute is training the model as we've deployed these models more, you know, more of the compute goes to inferencing them and actually using them.”

“And so I think that the question of exactly how, you know, if you don't do work, do you survive? I think the answer will be yes. You'll have plenty of material, your material needs met. But I think the question of Can you do more? Can you have not just generat…”

“Making the data better can be a massive compute multiplier. It can change the performance per dollar by orders of magnitude.”

“I think human feedback is kind of like a bit of like a side branch, because you can't really pour that much compute Into it, right? It's like, you take the model, and you, like, elicit it to be a little bit better in terms of personality”

“You know, in some ways, I think the story of Time Horizon is that progress has been remarkably continuous over, over so many years, so many orders of magnitude of compute and effective compute.”

“The suggestion in this paper is that if you think that algorithmic progress, you know, that, that is coming up with the transformer, coming up with RLHF, you know, MOEs, all of this stuff, better learning rate schedules is, is is itself a function of compute b…”

“Compute might become a bottleneck again, because previously, like if you want to train a new model, say you want to generate new synthetic data and then, or write a new algorithm, it might take a few weeks. And during that period of time, you don't, you might …”

“This, you know, this crude way of saying it is intelligence is log of compute kind of works.”

“We should be thinking about spending as much on the environments as on the compute and the data, because the environments literally shape the, what intelligence can emerge.”