The Ledger, every show

“I don't have a lot of patience as a scientist for people who are predicting the extremist scenarios, the catastrophic risks of AI, you know, AI becomes our overlord kind of scenario.”

“Now, you know, where we're maybe getting a little bit ahead is thinking that just RL out of the box is gonna give us AGI. That part, a lot less so. You know, if you look at the curve of progress, RL is terribly inefficient, and so the amount of signal you need…”

“My worry about closing the doors down now is that the models are only getting better. And so if we don't release them now, we really miss an opportunity to develop the muscle we need to make these models safer. And I don't think today's model are the ones that…”

“The transformer moment for reasoning and choosing action and being able to plan at different levels of granularity. We're still far away from doing that.”

“That's the part that the reasoning models don't do. They do really well at like one level of granularity... But the going back and forth between different levels of sort of resolution of action, it's really hard. So on the technical terms, we call it hierarchi…”

“I tend to actually place my bet not on the fact that we're going to reach like a single super intelligent agent, but on the fact that we are much more likely to live in a future where there's going to be many agents for many things. And so some agents will abs…”

“Oh, I'm still super bullish on RL in that, like, the concept itself is so fundamental. You know, this idea of training through a system of rewards, of indicating what's valuable and what's not valuable through numerical values, like, that is so fundamental. It…”

“RL, to shape the behavior of models, to get them to be social creatures, that we have no idea how to do. I mean, I don't know if you have children, but like shaping their behaviors, you know, the number of times you can repeat the same thing, and still they do…”

“Compute and data have a more linear effect on progress. You build more compute. You run bigger models. You can typically get better performance. You feed in more data. It's not just quantity. You need to worry about quality and diversity as well, but roughly i…”

“They've been, the scaling laws have been remarkably robust. They don't play exactly as we expect, but still, they've been remarkably robust. Lots of people have bet against scaling laws in the past. And I would say overall, you know, we've seen a pretty, prett…”

“Oh, I don't think that's unrealistic at all.”

“Time will tell. I don't think it's necessarily needed to go at that scale, but time will tell.”

“I don't think it's a phase in the sense that I do think this partnership, we'll call it, between humans and machines, where humans provide guidance to machine, like, we are in this for a long time. What will change is the nature of the information that the AI …”

“We've done a ton of work on AI on the platform over the last few years. That was what I would call more narrow specialized models. We can continue to do that, but the bigger step change are gonna come through the more general model, building foundation models,…”

“It gives us a path. It may not be the path, but it gives us at least a path to move towards general intelligence. And it's an exciting one. It's one that, that we're exploring. It doesn't mean that we've stopped exploring other paths towards general intelligen…”

“I do believe they can generalize. I think they generalize relatively narrowly, or at least, you know, as long as you stay close, you get a good manifold of information. When you start to go really far afield from your data, because the dimensions are so large,…”

“Often this notion of general intelligence is, is articulated in, in the context of like a single agent, a single uber intelligent agent. And I don't think that's really where we will move towards either.”

“A lot of the ingredients of how to do reasoning have been explored in AI for 40 years. And are published and well known to anyone who's taken even an undergrad level course in AI. So I'm not saying they, that there's not any innovation in the work that, that o…”

“World models are absolutely essential when you want to build agents, because these agents are going to take actions, which is going to change the world. You want to be able to predict these effects.”

“I confess I have a little bit of trouble with continual learning as a concept because I feel the community has never been able to nail, like, how do we articulate the problem in a way that we all agree on it? And so everyone who does work on continual learning…”

“And that's why, honestly, for many years I've been so much an advocate for open science. I just don't believe that you can keep these ideas boxed in unless you're willing to keep people boxed in, which we are not willing to do. And so I don't think we have a w…”

“If these entry level employees are able to use AI properly, they're skipping ahead to the level where they can actually be fully functioning analysts and they can essentially do 10 X the job with the tools.”

“We're at a stage where AI is really starting to be useful, maybe not as useful as people think it is, but we are there.”

“So to just like flat out replace a portion of your workforce is actually Pretty unrealistic.”

“With respect to AI security, I think there's a new front that's opening up with the development of agents, and frankly, there's a lot we don't know yet in terms of the vulnerability of these systems.”

“I don't think it becomes that productive to put a bunch of AI superstars all together in a room without the execution machine, without the social glue.”

“You do need a few of these, like, uber talents in the team. There's a relatively, you know, short number of people who just Understand this technology very deeply. You do need some of this talent, and if you can afford it, you should get some of that talent. B…”

“So you think of code generation, like right now we're in the phase we were for image 10 years ago. Yes, there's a lot of bad code that's getting generated. There's a lot of code that will get thrown away, but wait another 10 years and I think the quality of th…”

“What matters now is sort of, you know, picking the quality out of the volume, and so if I fast forward 10 years on code generation, when we have the ability, To generate a ton of code to do a ton of different things. We're going to need some selection mechanis…”

“I think about it as a bubble with bigger variants. It's like, you know, the upswing is going to be bigger, and you know, there's going to be big downswings as well, and so there's a lot of variants into the system right now.”

“Think of evaluations as, like, unit test for the performance of your system. I mean, software engineers will know what that is, right? Like, you run through that evaluation, and that gives you, like, a signal of how the system is doing in a particular dimensio…”

“I think the, we have to be careful about taking shortcuts and saying, you know, because suddenly, you know, we have certain platforms, we have AI, and so on, that is causing that mental illness.”

“That's a deep mistake. I mean, I will continue to believe that, especially for research, the ideas need to circulate, and this thought that you can just, like, close us down is, is absolutely false. I mean, people are circulated.”

“The truth is right now, AI is moving so fast that it's really useful to be close to products that are in the hands of billions of people to be able to have that quick product innovation, that quick signal back to the research.”

“We will definitely be exploring some of that. Yes. Yes. I mean, we built a lot of hardware for reality labs. We have some specific needs and, you know, as much as we, you know, look at that for the AR VR devices, there's also a great group doing some of that i…”

“Right now the progress in the research community that's working on continual learning isn't necessarily connecting to the work that's going on on scaling.”

“Extending the context length is kind of the easiest way to go about it, but there's quite a bit of progress that is being made on this.”

“There's some hope that by training enough on code, the machine essentially like infers these kinds of structural cues.”

“In reality, paying customers want like a good trade-off in terms of performance for efficiency. So, you know, we'll train bigger models, but we'll deploy smaller models because it gives us that trade-off. It's like good enough intelligence to get the job done.”

“It's also the people who are in leadership position, which instead of like writing out a memo suddenly can go out and like produce a full fledged prototype. They don't need, you know, 10 people, 10 staffers to help them produce their prototype. They have an id…”

“Those who are able to prototype in this way, it doesn't mean that whatever your vibe coded into a weekend suddenly turns into a hundred million dollar business, right? But it's a way to communicate with your teams, your intention. So as long as you have good i…”

“The other way to think about sovereignty that we're hearing a lot is that companies want a robust plan for AI. And so, you know, they want options. They may be using one model, but they actually want to have another model to be able to compare, to benchmark if…”

“I used to be quite skeptical that neural networks were necessarily the ultimate A solution to machine learning. I seem to be quite wrong on this one.”

“It's coming down, especially in domains where we have good reward functions.”

“One of the things that Cohere is doing is actually to develop AI models that run on premise. So that means enterprise bring it in, they run it locally, so the company has to worry about the training Of the models. Obviously we want world-class models for the n…”

“I think one of the biggest challenges, the fact that it's very hard to have predictability, right? Everyone wants to know when are we going to hit the breakthrough? Everyone wants to know how many GPUs do I actually need? Everyone wants to know, like, what's t…”

“So, in that sense, I think it's the most interesting one, it's the most frustrating one, and it's the most difficult one, certainly from investors' point of view, to know where to put your chips.”

“When you need to sell AI to a business, you get a real signal of what works, what doesn't work. And you know, we've been using these academic benchmarks for many years. You get some signal, but it's not the same as getting this to do productive work.”