The Ledger, every show

“That's my expectation. To me, like what's interesting about AI from a business side, right, is like it forces the question of like, what Companies or offerings are going to be bundled or integrated and which ones are going to get unbundled. And I actually thin…”

“I think I think the move to go pull up the ladder behind them is already beginning. And I think that lawmakers don't really understand this technology at all. And so their default instinct is sort of listened to the most credible source. And usually those cred…”

“The second way of improving model performance is just starting to be tapped now, and that's also going to absorb a boatload of compute. So because of that, I actually am not worried about diminishing returns to the compute over time.”

“So put another way for just scaling up a base language model you need to double the amount of compute for that language model for it to be predictably consistently smarter.”

“But I think pure model scaling does not deliver solutions to reasoning.”

“I do think there will not be that many LLM players. I think that there will probably be, my guess is somewhere between five to seven long-term steady state LLM providers at maximum scale, just because of the costs involved.”

“And so I think as a result, I think Apple is just going to completely crush at everything that looks like something that's really private, something that's fine-tuned on your own particular data, but doesn't require massive reasoning capability. And that will …”

“Companies that sell models to developers will either need to effectively be the like first party effort of one of these big clouds, or they have a short window between now and commoditization to build such a big economic flywheel that they can afford to stay i…”

“I just think in five to 10 years, people are gonna use their computers by giving them high-level goals, right?”

“Like, that's not, I don't think this is how this is going to play out. I think the way this is going to play out is, Is that what we're going to have as humans fundamentally be the drivers of these agentic systems that like that, like basically give everybody …”

“What's going to prevent this for what I think has a hope of preventing this from just being a hype cycle that falls flat like AB is that those things are, those shoes are yet to drop. And as they do, the capabilities of these models are going to continue to im…”

“And I think in the next five years, open will always lag closed. And because open will always lag closed, because open just has fewer resources behind them and fewer incentives for people to go to go make things to be open as these things become more and more …”

“Agents in five years time, I mean it's kind of gonna be, like, a non-invasive, like, brain-computer interface, basically. I think that's what an agent will be. Like, all of us are gonna be up-leveled. We're going to, you know, it's gonna feel like the same tra…”

“Multimodal models are becoming more of a thing, we're behavioral cloning the visual world, but really what we're just going to have is this like universal byte model, right? Where like tokens of data that have high signal come in, and then all of those pattern…”

“In a world where foundation models are looking more and more commodity. And if, and I think a huge amount of gain is going to happen from how do you use foundation models as like the, like well learned behavioral cloner to go solve agents.”

“I actually think like it's really clear today. Multimodal models are the default foundation model, right? It's just going to supplant LLMs. Like why did you just train a giant multimodal model?”

“I think pure play foundation model companies are just gonna be pinched by how good The next couple of llamas are going to be, and the next, like, what next good open source thing, and then seeing the really big players put ridiculous amounts of compute behind …”

“One, I'm so excited for someone to train a foundation model of robots. Like, it's just, I think it's just gonna work. Like, I will die on this hill. I mean, like, again, this whole time, like, we've been on this podcast, just continually saying, you know, like…”

“I think it's like 10 x larger than the disinformation problem, if not a hundred x larger. In a world where these models are much more easily used as a mechanism for attack than a mechanism for defense, there's gonna be this period of time over the next couple …”

“ChatGPT was really just GPT-III with instruction. It was basically like more chat tuning, but GPT-III API came out, I think, over a year before ChatGPT did, but only developers could play with it.”

“When we were training GPT-II, we trained GPT-II in various different sizes. And at the smallest size, the model was just, like, unable to do three-digit arithmetic. But as the models got bigger and bigger and bigger, we didn't change anything else. We just had…”

“I think the general capability of reasoning will need to be solved at the model provider level. And that's because What you're actually doing is you're not just using the model to reason. You are trying to improve the model's ability to reason, which means the…”

“Every one of the major clouds is working on And every major LLM provider is working on a strategy to have their in-house chips because that way they have better margins.”

“I think the TPU team when I was at Google was, like, sub-five hundred people and their budget was a shoestring budget, and yet somehow every generation they taped out quite good chips that were then used to train Gemini and Palm and are used by third parties n…”

“I think in a couple years everyone's going to have access to, like, an AI teammate that they can delegate Arbitrary tasks do at work, and then also be able to, you know, use it to the sounding board and like just be way, way, way more productive”

“We were the first company to figure out how to get this level of analysis of what's happening in videos as a service.”

“What OpenAI realized before basically everybody but DeepMind was that the next phase of AI after Transformer was not going to be about research paper writing. It was going to be about let's choose a major unsolved scientific problem and just try to solve it.”

“So I think what it's gonna do is it's gonna make humans at work much more like generalists, and it's gonna have, like, causes to create larger and larger, oh, sorry, like or giving people sort of, like, larger and larger scope over various different, like, are…”

“We're going to be on this adoption curve for enterprise AI for a very, very long time.”

“The number one driver of AI progress over the next couple of years is going to be the deep co-design and co-evolution of like product and users for feedback and actual technology.”

“The last meeting we did with Microsoft, Before Microsoft invested in OpenAI, Sam Altman, myself, and our CFO flew up to Seattle to do the final pitch meeting. And I'd been a founder before, so I always had like a tremendous amount of anxiety about partner meet…”

“In the process of solving a particular task, if a human gives the language model one piece of feedback, like you're writing code and you forgot to import the Python OS library, then the solve rate for these problems would jump up like 20, 30%.”

“In terms of where we're going to see it In industry, it's likely to be still in a kind of vertical by vertical system for a little while before we start seeing a little more convergence.”

“I think second or second or third day of my time at OpenAI Greg and Ilya pulled me in a room and were like, hey you know, the you should take over our directs and we'll go mostly do IC work.”

“Krzyzewski's work on ILS VRC, which is the ImageNet Large Scale Visual Recognition Challenge, where in one year, essentially they blew away the previous state of the art with a deep convolutional neural network by about, like, half of the final error rate on t…”

“Around 300 hours of video, it's probably even more now actually, are uploaded to YouTube every minute right now.”