The Ledger, every show

“I actually think architectures don't matter as much as everyone else thinks they do.”

“I think if we hadn't invented the transformer, we would have gotten there with whatever LSTM you know, state space model, whatever, anything else people were developing, we would have gotten there.”

“Oftentimes I know people say, oh, well, AI is, it's just predicting words. That's all it's doing. Therefore it can't be intelligent. It can't be. And I think that's just. Demonstrably wrong. What I think is amazing, though, is the scientific fact that when you…”

“The domain I use models for most probably is coding, and also doing things like transcribing lectures and stuff like this. On those tasks, I am absolutely not seeing plateauing gains. The latest models, they are notably better than the previous iteration, and …”

“This is sort of like these models have a buffer overflow in all of them that we know about, and most importantly, that we don't know how to patch and fix. We don't know how to fix this yet with models”

“The first is that I think the vast majority of AI safety should not be about these topics. The vast majority should be about quite practical concerns we have on making systems safer, like the kind that I've talked with you about so far.”

“I, for a lot of my career, was thinking that model architectures really mattered, and by having clever, complex architectures and sub-modules inside architectures, you would have, that was the route to sort of better AI systems. For the most part, I don't beli…”

“Traditionally this has been an area where both in terms of safety models don't get better by just being bigger, unlike most other areas where models do get better by being bigger. Safety has not been like that traditionally. You know, you have to train them ex…”

“However, one thing that we are finding, and this is actually, I think we're kind of crossing this point too. Is that in a lot of the latest experiments, we can do much better than people, than human red teamers now at breaking these models. When I say we, I me…”

“It is clearly a different form of intelligence than people. It's some alien intelligence that is vastly different, and that difference is actually often brought out to a large degree by things like adversarial attacks and red teaming, because there are certain…”

“It's like we could kind of run experiments on the brain, observe every neuron in it, reset its state to prior states, and run counterfactuals, none of which we can do with humans, and yet we still understand neither very well. Even with that, all that ability,…”

“One of the most effective ways of doing capability elicitation is actually through some amount of what you would call red teaming, right? So if a model refuses a task because it thinks it's being evaluated, but it knows how to complete that task, getting it to…”

“So I think this is really sort of an underappreciated point that we're reaching this point, this sort of phase where a lot of security, a lot of science has this potential to kind of explode. Not because we're going to get better at it, but because agents can …”

“You can't just sort of trust models to get safer by getting bigger. You have to put in the work to actually make them safer. And this is, I think what a lot of AI companies are investing in. This is why we in fact do have models that are improving on these dim…”

“Certain fields. I think things like robotics is still one. I don't think we're quite at the, let's just scale it up level with robotics yet. Some companies might argue we are. I don't think we are. I think we're still in the let's explore methods to find the r…”

“Once we had done that, we found that when you had these weird terms that you sort of flipped around to optimize one, to optimize the response for one model, you could just take those same exact strings you would optimize, paste them into a commercial model, an…”

“Yes, I think so, actually. I think if you run with proper guardrails, you know, we release guardrails for coding agents, for example. If you're on proper guardrails with proper sandboxing, and right now, yes, you probably also take some care to be a little bit…”

“I don't write code anymore. I do all my work now, and I do lots of, you know, I still do some research, right? It's entirely telling Codex what to do.”

“I don't think people have properly internalized the fact that the vast majority of intelligence comes from self training effectively.”

“I think the current trajectory we're on is going to get us, even if there were no more breakthroughs, I think, you know, with the minor additions that we are doing right now, we will get to incredibly capable systems, even if we were to freeze things right now…”

“The discovery That when you train big enough models on lots of text, and then turn, and then a little bit of additional sort of, you know, fine-tuning text, and then turn them loose to generate, that this generates long-form coherent thought. That was probably…”

“You can train word predictors, and they produce intelligent, coherent, long-form responses. This is one of the most notable, if not the most notable, scientific discovery of the past 1020 years. Maybe much longer than that, right?”

“We are nowhere close to hitting the limits of available data in these models. Arguably, we're unable to process it because we don't have enough compute and things like this, but we're nowhere close to data limits in other senses.”

“There are massive amounts of data available, And I think we have not yet figured out how to properly leverage those due to either limitations of compute. I mean, you have to process all that data and it does take, we don't have current models to do this very w…”

“But it is also true that if you just take a fixed data set and run over it multiple times, if you use a bigger model, it will often work better, right? So I think that we have not really reached the plateau there.”

“So I think this perception has more to do with people's limited imagination of what they can do with these models, and less to do with the models themselves. But that will evolve over time. People will start figuring out you can use them for better and better …”

“Companies that are right now thinking about training their own models and things like this, and it's just sort of the default that of course you would do this, that this must, won't be an economically viable thing to do. In the future, and so it won't happen a…”

“I'm not really sure what the rationale is for saying that we've, we plateaued in the compute sense. Most scaling laws that I've seen certainly suggest it can keep going. It's more expensive. You could argue that it's by far just scaling is, may not be the most…”

“And I am massively uncertain as to when this will happen, but a massive shift that I've undergone is I think this will probably happen in my lifetime. I think the answer to AGI has always been in academia, not in my lifetime. And the timeframe I give this righ…”

“The winners in this new world Will the companies that say survive and thrive and become dominant in this new world? Not talking about the AI companies for now, talking about sort of the rest of the companies that the ones that people worry that they're going t…”

“Video is a short blip where we sort of think that there's some objective evidence for a hundred years of our history, and that's going to be now, that's no longer true pretty soon.”

“I think this is essentially already solved in many cases by these models that can already solve and analyze code to find vulnerabilities.”

“Personally, I think cyber attacks are a much more clear and present threat than, for example, bio threats and things like this.”

“If you look at the current best models that there are right now, so things like GPT-IV, Claude, 3.5 Gemini, things like this, I would not currently be all that nervous about having an open source model that was as capable as these in terms of the catastrophic …”

“I think there will come a time when a certain capability, a certain ability of these models Reaches the point that should give us pause when it comes to just turning these things over to whoever and what, however, they want to use them.”

“I think killing jobs is much more immediate of a concern than killing humans.”

“We do not understand how these things work internally, the possible correlated failures, the possible attack factors, all sorts of things. We don't understand it.”

“And especially when there's the possibility of correlated failures, right? So it's not just that there's a lot of AI systems out there, it's that there's actually a few models that everyone is using. And if you find vulnerabilities in the agents that everyone …”

“So generally speaking, the issue with this is that frontier models are extremely bad at automated red teaming because they have a lot of safeguards built into them. So if you try to use them to jailbreak another model, they will actually refuse their safety tr…”

“The problem with McInterp is it's a lot, it's been about sort of testing small hypotheses. Hypothesis. And you know, you have a hypothesis, you'll find some small thing, you'll test that in isolation. But I don't think it's really become a science yet.”

“So, so I think SOC II is not a great model. We'll just say, but it is a model.”

“The name gray swan is sort of in reference to black swan events, which are things no one could see coming. A gray swan is an unlikely event that you can kind of see coming. And that's kind of where we are with all of this, right? This is going to happen. We kn…”

“In the case essentially where we have more questions, we can delay model release if we feel that we need to understand that, that better.”

“Models, definitely, objectively, I would say, in a lot of scenarios we can measure, are safer than they were a year ago.”

“I have never expressed a P. Doom and things like this. I just think it's a very weird concept as if the world is some stochastic set of dice that you can roll multiple times and that we don't have direct influence over this.”

“I think the way you solve things is through, through ongoing exploration of what's happening and through, through interaction with the frontier.”

“Reasoning models were much more effective because you can't really do the same trick of optimizing for a probability with a reasoning model that has a whole trace of reasoning that happens in the middle and kind of reflect a bit more. So it's much harder to br…”

“Coding agents are extremely good Mechinterp researchers.”