The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 11 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Kolter: Adversarial prompts optimized on open-source LLMs break commercial models
“Once we had done that, we found that when you had these weird terms that you sort of flipped around to optimize one, to optimize the response for one model, you could just take those same exact strings you would optimize, paste them into a commercial model, an…”
Zico Kolter: Current AI trajectory will yield capable systems without breakthroughs
“I think the current trajectory we're on is going to get us, even if there were no more breakthroughs, I think, you know, with the minor additions that we are doing right now, we will get to incredibly capable systems, even if we were to freeze things right now…”
Zico Kolter: AI models are objectively safer than a year ago
“Models, definitely, objectively, I would say, in a lot of scenarios we can measure, are safer than they were a year ago.”
Kolter: AI agents might turn mechanistic interpretability into a science
“I think that we actually might finally be able to make more what I would consider a science of this through essentially leveraging mass research by agents deployed for this problem.”
A complete modern LLM takes only 200 to 300 lines of Python
“You have this code this code to build a complete large language model that can train on a large data set and learn to speak, runs on GPUs yes, eventually is trained with RL and tool calls. That entire set of code, probably two to 300 lines of Python code.”
OpenAI differentiated early on by prioritizing model scale over new methods
“About opening up early on is that they always had this bet on scale. In a time where I think that was looked upon very suspiciously that, oh, if you, the thought somehow that we had all the methods already, and all you had to do was scale them up that mindset …”
Kolter: Universities will drive AI breakthroughs in math and science
“There's going to be a whole lot of breakthroughs happening with AI enablement in math and basic science, those kinds of things. Universities, I think will play a foundational role in shaping that future.”
Zico Kolter: Reinforcement learning is now the foundation of all AI post-training
“RL is now the foundation of really all post training. It's all done by RL.”
Zico Kolter: Agentic AI systems have far more autonomy than last year
“The amount of autonomy granted to agentic systems now is far greater than a year ago.”
Kolter: State-of-the-art AI security combines classifiers, safety training, and opsec
“What they look like is basically classifiers on input. So you'll read what the, what a user types in classifiers on things like tool responses to classifiers. And when I say classifier, I just mean things that will read texts and kind of classify whether or no…”
CMU undergrad AI course has students build an LLM from scratch
“You build a LLM completely from scratch. You use PyTorch, but you build one from scratch that, you know, can be a chatbot. You train it on data. You RL it to solve math problems with tool calls. You do all of this. And this is a undergrad level course.”