The Ledger, every show

“That muscle did not exist during my time at Google. And I think had they had it, what they would have done would be say, hey, Noam Shazir, you're a brilliant guy. You know how to scale these things up? Like, here's half of all of our TPUs. And then I think the…”

“That's my expectation. To me, like what's interesting about AI from a business side, right, is like it forces the question of like, what Companies or offerings are going to be bundled or integrated and which ones are going to get unbundled. And I actually thin…”

“I think I think the move to go pull up the ladder behind them is already beginning. And I think that lawmakers don't really understand this technology at all. And so their default instinct is sort of listened to the most credible source. And usually those cred…”

“I now think that the correct organizational structure for building an AI organization is actually to eliminate the concept of having a separate safety team. Because by creating a safety team, you're now defining this is the unsafe team, this is the safe team. …”

“The second way of improving model performance is just starting to be tapped now, and that's also going to absorb a boatload of compute. So because of that, I actually am not worried about diminishing returns to the compute over time.”

“So put another way for just scaling up a base language model you need to double the amount of compute for that language model for it to be predictably consistently smarter.”

“Now the critical path for model improvement is, is is shifting over to this to this sort of broader sort of simulation slash synthetic data slash like RL loop sort of path. I think it's just a natural consequence of the fact that it's so expensive to just keep…”

“A model trained that way is only as good as the smartest data in the training set. Like, it cannot discover new knowledge, because its job, the way the models are trained, is to do what a human would do in that situation.”

“I kind of think that chat bots like chat GPT and stuff and agents are kind of becoming different species of technology a little bit, right? Like I think they'll be useful in very different ways and what they need to be used for super different, right?”

“Having hallucinations in chatbots and in, like, image generators is, like, a really good thing, right? Because it gives you like, a starter tool for, like, getting to, like, solve the blank page problem, right? Like, and, like, gives you, like, little bits of …”

“But I think pure model scaling does not deliver solutions to reasoning.”

“I do think there will not be that many LLM players. I think that there will probably be, my guess is somewhere between five to seven long-term steady state LLM providers at maximum scale, just because of the costs involved.”

“The underlying thing that everyone's realizing now is that LLMs themselves are not a product. Like, an actual product is this entire software system that uses LLMs in it.”

“In the future, when more and more software is just like the logic of software is actually just handled by a, by an LLM, nobody cares anymore about what the base computing primitive is. All you need to do is access these models and compose these models to go so…”

“And so I think as a result, I think Apple is just going to completely crush at everything that looks like something that's really private, something that's fine-tuned on your own particular data, but doesn't require massive reasoning capability. And that will …”

“I think the degree to which GPT four O was like, I think relatively under hyped relative to what I think the true scientific improvements have been in that model is, is, is this pretty big gap?”

“Companies that sell models to developers will either need to effectively be the like first party effort of one of these big clouds, or they have a short window between now and commoditization to build such a big economic flywheel that they can afford to stay i…”

“I just think in five to 10 years, people are gonna use their computers by giving them high-level goals, right?”

“Like, that's not, I don't think this is how this is going to play out. I think the way this is going to play out is, Is that what we're going to have as humans fundamentally be the drivers of these agentic systems that like that, like basically give everybody …”

“What's going to prevent this for what I think has a hope of preventing this from just being a hype cycle that falls flat like AB is that those things are, those shoes are yet to drop. And as they do, the capabilities of these models are going to continue to im…”

“And I think in the next five years, open will always lag closed. And because open will always lag closed, because open just has fewer resources behind them and fewer incentives for people to go to go make things to be open as these things become more and more …”

“You know, when you go work with a coworker, right, you don't just you don't just talk back and forth. You, like, actually, like, share the same canvas. You'll go use a whiteboard. You'll go, like, maybe look at the same thing on the computer together, and then…”

“Agents in five years time, I mean it's kind of gonna be, like, a non-invasive, like, brain-computer interface, basically. I think that's what an agent will be. Like, all of us are gonna be up-leveled. We're going to, you know, it's gonna feel like the same tra…”

“What's the percentage of work done today that's addressable by agents? It's like, A thousand X that maybe? 10,000 X that? I don't know. Something in that order of magnitude.”

“I think agents are just absolutely the correct long-term direction, right? You just go to find what AGI is, right? You're like, hey, like, Well, first off, actually, I don't love AGI definitions that involve human replacement because I don't think that's actua…”

“Like de novo RL is like a pretty terrible way to get there quickly. Why are we rediscovering all the knowledge about the world? Like years ago, I had a debate with a Berkeley professor as to like what will it actually take to build HCI? And his view is basical…”

“Multimodal models are becoming more of a thing, we're behavioral cloning the visual world, but really what we're just going to have is this like universal byte model, right? Where like tokens of data that have high signal come in, and then all of those pattern…”

“I actually think that being an augmentation company Forces you to go develop your core AI capabilities faster than someone who's saying, ah, ok, my job is to deliver you a lights off solution for X.”

“In a world where foundation models are looking more and more commodity. And if, and I think a huge amount of gain is going to happen from how do you use foundation models as like the, like well learned behavioral cloner to go solve agents.”

“I actually think like it's really clear today. Multimodal models are the default foundation model, right? It's just going to supplant LLMs. Like why did you just train a giant multimodal model?”

“If you go itemize out the number of things you want to do on your computer for which every step has an API those numbers of workflows add up pretty close to zero.”

“I think pure play foundation model companies are just gonna be pinched by how good The next couple of llamas are going to be, and the next, like, what next good open source thing, and then seeing the really big players put ridiculous amounts of compute behind …”

“One, I'm so excited for someone to train a foundation model of robots. Like, it's just, I think it's just gonna work. Like, I will die on this hill. I mean, like, again, this whole time, like, we've been on this podcast, just continually saying, you know, like…”

“I think it's like 10 x larger than the disinformation problem, if not a hundred x larger. In a world where these models are much more easily used as a mechanism for attack than a mechanism for defense, there's gonna be this period of time over the next couple …”

“During that twenty-twelve, twenty-eighteen era, the way people made progress what I mean by bottom-up basic research is you hire the most brilliant scientists, they come to work every day with, like, No near-term objective they're being held accountable to. An…”

“ChatGPT was really just GPT-III with instruction. It was basically like more chat tuning, but GPT-III API came out, I think, over a year before ChatGPT did, but only developers could play with it.”

“When we were training GPT-II, we trained GPT-II in various different sizes. And at the smallest size, the model was just, like, unable to do three-digit arithmetic. But as the models got bigger and bigger and bigger, we didn't change anything else. We just had…”

“I think the general capability of reasoning will need to be solved at the model provider level. And that's because What you're actually doing is you're not just using the model to reason. You are trying to improve the model's ability to reason, which means the…”

“Every one of the major clouds is working on And every major LLM provider is working on a strategy to have their in-house chips because that way they have better margins.”

“I think the TPU team when I was at Google was, like, sub-five hundred people and their budget was a shoestring budget, and yet somehow every generation they taped out quite good chips that were then used to train Gemini and Palm and are used by third parties n…”

“I do think that in the agent space, it's extremely important that you own the entire stack from what is the end user interface. I think like we were talking about the Apple example earlier, owning the interface gives you tremendous leverage in this era of AI t…”

“One of the things we do, for example, is we really try to not sign deals that are coming out of experiment budget because we want quality revenue basically.”

“And so that's why the HCI problem, like people just aren't spending enough time thinking about like chat is obviously not it.”

“There's a collection of really hard-wanted knowledge that you get only by being at the frontiers of scale. And that hard-wanted knowledge, a lot of it's not published. A lot of it is, like, stuff that, like, it's actually not even easily reducible to what look…”

“I think in a couple years everyone's going to have access to, like, an AI teammate that they can delegate Arbitrary tasks do at work, and then also be able to, you know, use it to the sounding board and like just be way, way, way more productive”

“Everyone under values the importance of really good sensors and actuators. And actually a lot of what's helped us get a lot of reliability is like a really strong focus on like, actually, why does the model not do this thing? And the non-trivial amount of time…”

“Having an advertiser be the financial party responsible for paying for these services means that ultimately these AI services are beholden to that advertiser and not to the end user.”

“We were the first company to figure out how to get this level of analysis of what's happening in videos as a service.”