The Ledger, every show

“We're not gonna have too many more cycles, possibly zero more cycles, of a model that's almost an order of magnitude bigger in terms of the number of parameters than what came before, and thereby more powerful.”

“Are we going to see a GPT-V that's as big a leap over GPT-V as GPT-V was over GPT-V? I'm frankly skeptical.”

“And the other way to look at synthetic data is, okay, you take one trillion tokens, you train a model on it, and then you output 10 trillion tokens, so you get to the next bigger model, and then you use that to output a hundred trillion tokens. You know, I'll …”

“So people have been worried, for instance, about bots creating misinformation with AI and influence in elections and that sort of thing. We're very, very skeptical that that's going to be a real danger.”

“Even if one country decides that models should be closed, the odds of getting every country to enact that kind of, ah, ah, rule are, you know, just vanishingly small. So if our approach to safety with AI is going to be premised on ensuring that quote unquote b…”

“And what we've seen in the nearly year and a half since GPT-IV came out is that we haven't really had models That have surpassed it in a meaningful way.”

“When we look at the way that AI is architected today, that kind of fear has no basis in reality. Maybe one day in the future, you know, people are going to build AI systems where that becomes at least somewhat possible. And we should, you know, we should have …”

“These models are already trained on essentially all of the data that companies can get their hands on.”

“A hundred and fifty billion hours of video sounds, you know, really impressive. But when you put that video through a speech recognizer and actually extracts the text tokens out of it and deduplicated and so forth, it's actually not that much. It's an order of…”

“It's got, you have to actually deploy AI to be able to get to certain types of learning, and I think that's gonna be very slow, and I think the a good analogy is self-driving cars, of which we had prototypes, you know, two or three decades ago, but for these t…”

“And I predict that we're going to see the same thing with models when models get cheaper. They're put into a lot more things, and so the total amount that companies are spending on inference is actually going to increase.”

“So I think that's going to happen both with models as well as with these hardware cycles. You know, I can't predict how long that's going to take, but we are, I think, going to get to a world where models do get commoditized.”

“Folks focused on superintelligence didn't feel very welcome at the company, and there has been an exodus of very prominent people, and Anthropic has picked up a lot of them. So it seems like we're seeing a split emerging where OpenAI is more focused on product…”

“There's no way really to catch AI-generated text or homework answers.”

“When there is so much pressure to do well on these benchmarks, developers are intentionally or unintentionally optimizing these models In ways that look good on the benchmarks, but don't look good in real world evaluation.”

“I'm very skeptical that these new kinds of learning are going to get to a point anytime soon where they're going to become the default way in which people learn.”

“I do find it interesting that NVIDIA itself Has been trying to migrate really, really hard out of hardware into becoming a services company.”

“Over the lifetime of a model, when you have billions of people using it, the inference cost actually adds up, and for many of the popular models, that's the cost that dominates.”

“That's right. So that, that actually is our prediction. People we predict are going to be forced to rely much more on getting their news from trusted sources.”

“We have you know, close to state of the art AI models that can already run on people's personal devices, and I think that trend is only going to accelerate.”