Opinion
Kolter: Neural network architectures matter less than commonly believed
“I actually think architectures don't matter as much as everyone else thinks they do.”
What-if
AI capabilities would have been achieved even without inventing transformers
“I think if we hadn't invented the transformer, we would have gotten there with whatever LSTM you know, state space model, whatever, anything else people were developing, we would have gotten there.”
Insight
Kolter: AI models do not get safer automatically by scaling up
“You can't just sort of trust models to get safer by getting bigger. You have to put in the work to actually make them safer. And this is, I think what a lot of AI companies are investing in. This is why we in fact do have models that are improving on these dim…”
Opinion
Kolter: Robotics AI is not yet ready for pure compute scaling
“Certain fields. I think things like robotics is still one. I don't think we're quite at the, let's just scale it up level with robotics yet. Some companies might argue we are. I don't think we are. I think we're still in the let's explore methods to find the r…”
Assertion Supported
Kolter: Adversarial prompts optimized on open-source LLMs break commercial models
“Once we had done that, we found that when you had these weird terms that you sort of flipped around to optimize one, to optimize the response for one model, you could just take those same exact strings you would optimize, paste them into a commercial model, an…”
Opinion
Kolter: AI agent benefits outweigh security risks if deployed with proper guardrails
“Yes, I think so, actually. I think if you run with proper guardrails, you know, we release guardrails for coding agents, for example. If you're on proper guardrails with proper sandboxing, and right now, yes, you probably also take some care to be a little bit…”
Disclosure
Kolter no longer writes code manually, relying entirely on AI agents
“I don't write code anymore. I do all my work now, and I do lots of, you know, I still do some research, right? It's entirely telling Codex what to do.”
Insight
The vast majority of modern AI intelligence comes from self-training
“I don't think people have properly internalized the fact that the vast majority of intelligence comes from self training effectively.”
Prediction Not checkable as stated
Zico Kolter: Current AI trajectory will yield capable systems without breakthroughs
“I think the current trajectory we're on is going to get us, even if there were no more breakthroughs, I think, you know, with the minor additions that we are doing right now, we will get to incredibly capable systems, even if we were to freeze things right now…”
Opinion
Text model scaling is one of humanity's top scientific discoveries
“The discovery That when you train big enough models on lots of text, and then turn, and then a little bit of additional sort of, you know, fine-tuning text, and then turn them loose to generate, that this generates long-form coherent thought. That was probably…”
Disclosure
OpenAI's Safety Committee has authority to delay model releases
“In the case essentially where we have more questions, we can delay model release if we feel that we need to understand that, that better.”
Assertion Not checkable as stated
Zico Kolter: AI models are objectively safer than a year ago
“Models, definitely, objectively, I would say, in a lot of scenarios we can measure, are safer than they were a year ago.”
Disclosure
Kolter: Calculating a P(doom) for AI is a flawed concept
“I have never expressed a P. Doom and things like this. I just think it's a very weird concept as if the world is some stochastic set of dice that you can roll multiple times and that we don't have direct influence over this.”
Insight
Kolter: AI safety is solved through frontier interaction, not pauses
“I think the way you solve things is through, through ongoing exploration of what's happening and through, through interaction with the frontier.”
Insight
Kolter: Reasoning models are much harder to jailbreak via probability optimization
“Reasoning models were much more effective because you can't really do the same trick of optimizing for a probability with a reasoning model that has a whole trace of reasoning that happens in the middle and kind of reflect a bit more. So it's much harder to br…”
Insight
Kolter: AI coding agents are extremely good mechanistic interpretability researchers
“Coding agents are extremely good Mechinterp researchers.”
Prediction Not checkable as stated
Kolter: AI agents might turn mechanistic interpretability into a science
“I think that we actually might finally be able to make more what I would consider a science of this through essentially leveraging mass research by agents deployed for this problem.”
Insight
Kolter: Scientific progress occurs when young researchers ignore old guard beliefs
“Basically progress happens when The current crop of young researchers ignores the things they've been taught that the old guard believes.”
Assertion Not checkable as stated
A complete modern LLM takes only 200 to 300 lines of Python
“You have this code this code to build a complete large language model that can train on a large data set and learn to speak, runs on GPUs yes, eventually is trained with RL and tool calls. That entire set of code, probably two to 300 lines of Python code.”
Opinion
Kolter: AI companies should establish board-level safety and security committees
“I think it's actually very important that AI companies start to establish similar governance policies, because this is something that requires that level of just oversight and of assurance. It is a becoming You know, a massive industry, and just like there are…”
Insight
Kolter: AI safety focus is shifting from single models to ecosystems
“I think actually one of the big trends we're seeing is a lot of safety is moving from the model level to the ecosystem level and talking about, you know, what's not one model capable of, but what's AI broadly capable of.”
Insight
Kolter: Renaming AI Safety Summit to AI Action Summit shows political shift
“The, you know, AI safety conference was, or AI safety summit was renamed the AI action summit or something is, has some significance actually in terms of the sort of taking temperature of where the world is politically.”
Assertion Not checkable as stated
OpenAI differentiated early on by prioritizing model scale over new methods
“About opening up early on is that they always had this bet on scale. In a time where I think that was looked upon very suspiciously that, oh, if you, the thought somehow that we had all the methods already, and all you had to do was scale them up that mindset …”
Prediction Not checkable as stated
Kolter: Universities will drive AI breakthroughs in math and science
“There's going to be a whole lot of breakthroughs happening with AI enablement in math and basic science, those kinds of things. Universities, I think will play a foundational role in shaping that future.”
