The Ledger, every show

“I think that we actually might finally be able to make more what I would consider a science of this through essentially leveraging mass research by agents deployed for this problem.”

“Basically progress happens when The current crop of young researchers ignores the things they've been taught that the old guard believes.”

“You have this code this code to build a complete large language model that can train on a large data set and learn to speak, runs on GPUs yes, eventually is trained with RL and tool calls. That entire set of code, probably two to 300 lines of Python code.”

“The real negative outcome is that people are not going to believe anything that they see. It didn't even need AI to get there, but AI is absolutely an accelerant for this process.”

“What that means is our current algorithms, we are not yet maximally extracting the information from data we have. And there are way more deductions and inferences and other processes that we can apply to our current data to provide more value.”

“I define AGI as a system that acts functionally equivalent to a close collaborator of yours over the course of about a year long project.”

“The thing that frustrates me most, honestly speaking, is the degree of certainty that some people have about whether we will definitely get there very, very soon or even more on the flip side, that there's absolutely no way that we will ever achieve AGI with t…”

“And the people that have been sort of ringing this bell for a while saying, look, this is coming, this is this, you know, they've, in many cases, in my view, been proven right. And I've updated my sort of posterior beliefs based upon the evidence I've seen.”

“And so what irks me the most about a lot of people's sort of philosophy of AGI is that, to a certain extent, how little It seems like observable evidence has changed their beliefs one iota. You know, they had certain beliefs about what it would take to get to …”

“A lot of the details about how those regulations sometimes evolve can be a bit misguided or miss the point or somehow Just when I read them, basically they're going to become dated in, in a matter of months, because they're dealing with things and they're appr…”

“We have a much better handle on regulating the downstream uses of AI. Like, when it comes to misinformation, we already have laws that deal with sort of libel and things like this. In many cases, because AI is acting as an accelerator, there are situations in …”

“Right now, the mod, the AI models that we have, for lack of a better phrasing, are not able to reliably follow specifications.”

“The nuclear weapon analogy is actually not a great one, because nuclear weapons have one purpose, which is to destroy things. Maybe a better analogy is sort of nuclear technology, period, because it has the ability to create nuclear weapons, but it also has th…”

“At least for now, there's a constant stream of closed models that are released sometime before an equivalently capable open, open weight model, right? And I think this is actually a very good thing because my hope would be that, and we've sort of found ourselv…”

“Kind of on the contrary, I thought that data was sort of, you know, data had to be highly curated to be valuable, and the value in data came essentially from very manual labeling of this data and human intensive curation. The big amazing insight Of current AI …”

“Most fascinating things about coding agents actually is they can do a lot of experimentation in an automated fashion. Yeah. They will give new hope. They'll breathe new life into mechanter research.”

“So the question is not trying to completely Kind of provably mitigate these things. That is arguably just a, it's a good goal, but just like zero bug software, we're probably not going to get there. At least not that soon.”

“Now, things that are that simple, to be clear, are covered at this point by most agents, right? You know, they all They, despite some issues, yeah, normal, normal sort of, you know, will not be that easily fooled by just push all my API keys to a public thing,…”

“So far, we are still a lot, in a lot of cases, operating on the condition that your agent has your permissions. Yeah. That is a very standard default. And I think that will be changed. I mean, your permissions may be in a sandbox, but still kind of your permis…”

“I think in terms of how this will evolve, actually, I don't think it'll be per app, but I think what will happen first is people have different personas that they have, right? So you don't want your work life and your home email to be mixed up. Yeah. Right. A …”

“I think it's actually very important that AI companies start to establish similar governance policies, because this is something that requires that level of just oversight and of assurance. It is a becoming You know, a massive industry, and just like there are…”

“I think actually one of the big trends we're seeing is a lot of safety is moving from the model level to the ecosystem level and talking about, you know, what's not one model capable of, but what's AI broadly capable of.”

“The, you know, AI safety conference was, or AI safety summit was renamed the AI action summit or something is, has some significance actually in terms of the sort of taking temperature of where the world is politically.”

“About opening up early on is that they always had this bet on scale. In a time where I think that was looked upon very suspiciously that, oh, if you, the thought somehow that we had all the methods already, and all you had to do was scale them up that mindset …”

“There's going to be a whole lot of breakthroughs happening with AI enablement in math and basic science, those kinds of things. Universities, I think will play a foundational role in shaping that future.”

“Most evaluations are done kind of in a, They measure expected value, basically. They measure sort of how well does it work on average, and security measures how well does it work in the worst case.”

“But they are, they require that degree of complexity to really jailbreak modern systems in a, for information that has this sensitivity to it.”

“Things like prompt injection are really a new security vulnerability for AI agents, and they mean that your risk is not just that you could have some, the model says something mean to you or something like that. Or even they could just write bad code. It could…”

“AI security of agents is this interaction between what can the agent be manipulated into doing? What might it do accidentally? And what credentials or access does it have to really affect change?”

“RL is now the foundation of really all post training. It's all done by RL.”

“Reasoning models were the next big breakthrough. Those are rare. They do take kind of a, you know, both, both a massive scale and kind of a bit of luck to get there.”

“The entire complexity of an AI system evolves from the data they're trained on.”

“Public models are trained on the order of, you know, 30 terabytes of data or something like this, right? So 30 terabytes of text data.”

“I work almost exclusively with the largest models that are available to me because it just works better, and when I don't have a given task that I'm doing over and over, when I want to have that generality, I want to work with the larger models that are availa…”

“Once you have a task, a rote task that you're repeating again and again enough times, And you know a small model can do it. It probably does become valuable to specialize a small model for that task only.”

“And I think there will be most likely consolidation, but I'm not quite sure how it will play out.”

“Do we have AI products that are able to be maximally used by workforces? And the answer to this is right now is no. Clearly there is a gap between what People could use these things for and what they're using them for right now.”

“RAG based systems are so Are so common here, and so, and probably will remain, even with the advent of fine-tuning availability, they're going to remain a useful paradigm.”

“This is not in certain use cases, any riskier than just having your data in the cloud to begin with, which all of them typically do. They've all moved that way. So I think this will just happen naturally with progression of time.”

“The concern is that not that they can do this sort of autonomously, maybe initially, but that they can lower the bar so far in the skill level required to create these things that effectively it puts them in the hands of a huge number of bad actors.”

“I want to develop and improve safety of these tools Because I want to use them. It's not that we have some moral imperative that we have to develop these tools. I mean, maybe there are, or we have to develop AI and AGI. Maybe that's true. But that's not what m…”

“I think there are certain things like, for example, AI safety, where we very much need to work as a world to help set standards and help better the future of everyone here. Because yes, certain things can be done by countries. Capabilities can maybe advance mo…”

“The amount of autonomy granted to agentic systems now is far greater than a year ago.”

“I was trying to get both John Schulman and Andre Karpathy to apply for faculty jobs at CMU, and I was trying to understand where they were, if they were going to apply, what they're going to be like, and they said, no, I think I'm going to be doing this startu…”

“What they look like is basically classifiers on input. So you'll read what the, what a user types in classifiers on things like tool responses to classifiers. And when I say classifier, I just mean things that will read texts and kind of classify whether or no…”

“You build a LLM completely from scratch. You use PyTorch, but you build one from scratch that, you know, can be a chatbot. You train it on data. You RL it to solve math problems with tool calls. You do all of this. And this is a undergrad level course.”

“Also, I am recently on the board of OpenAI which I joined at this point a couple weeks ago, and it's been extremely exciting as well.”

“I got, actually the day before I started as department head, I got an email from Brett, the chair of the board, just saying, hey, do you want to talk about maybe joining the OpenAI board?”