The Ledger, every show

“This is sort of like these models have a buffer overflow in all of them that we know about, and most importantly, that we don't know how to patch and fix. We don't know how to fix this yet with models”

“However, one thing that we are finding, and this is actually, I think we're kind of crossing this point too. Is that in a lot of the latest experiments, we can do much better than people, than human red teamers now at breaking these models. When I say we, I me…”

“So I think this is really sort of an underappreciated point that we're reaching this point, this sort of phase where a lot of security, a lot of science has this potential to kind of explode. Not because we're going to get better at it, but because agents can …”

“Once we had done that, we found that when you had these weird terms that you sort of flipped around to optimize one, to optimize the response for one model, you could just take those same exact strings you would optimize, paste them into a commercial model, an…”

“I think the current trajectory we're on is going to get us, even if there were no more breakthroughs, I think, you know, with the minor additions that we are doing right now, we will get to incredibly capable systems, even if we were to freeze things right now…”

“We are nowhere close to hitting the limits of available data in these models. Arguably, we're unable to process it because we don't have enough compute and things like this, but we're nowhere close to data limits in other senses.”

“There are massive amounts of data available, And I think we have not yet figured out how to properly leverage those due to either limitations of compute. I mean, you have to process all that data and it does take, we don't have current models to do this very w…”

“But it is also true that if you just take a fixed data set and run over it multiple times, if you use a bigger model, it will often work better, right? So I think that we have not really reached the plateau there.”

“Companies that are right now thinking about training their own models and things like this, and it's just sort of the default that of course you would do this, that this must, won't be an economically viable thing to do. In the future, and so it won't happen a…”

“And I am massively uncertain as to when this will happen, but a massive shift that I've undergone is I think this will probably happen in my lifetime. I think the answer to AGI has always been in academia, not in my lifetime. And the timeframe I give this righ…”

“The winners in this new world Will the companies that say survive and thrive and become dominant in this new world? Not talking about the AI companies for now, talking about sort of the rest of the companies that the ones that people worry that they're going t…”

“Video is a short blip where we sort of think that there's some objective evidence for a hundred years of our history, and that's going to be now, that's no longer true pretty soon.”

“I think this is essentially already solved in many cases by these models that can already solve and analyze code to find vulnerabilities.”

“I think there will come a time when a certain capability, a certain ability of these models Reaches the point that should give us pause when it comes to just turning these things over to whoever and what, however, they want to use them.”

“We do not understand how these things work internally, the possible correlated failures, the possible attack factors, all sorts of things. We don't understand it.”

“The name gray swan is sort of in reference to black swan events, which are things no one could see coming. A gray swan is an unlikely event that you can kind of see coming. And that's kind of where we are with all of this, right? This is going to happen. We kn…”

“Models, definitely, objectively, I would say, in a lot of scenarios we can measure, are safer than they were a year ago.”

“I think that we actually might finally be able to make more what I would consider a science of this through essentially leveraging mass research by agents deployed for this problem.”

“You have this code this code to build a complete large language model that can train on a large data set and learn to speak, runs on GPUs yes, eventually is trained with RL and tool calls. That entire set of code, probably two to 300 lines of Python code.”

“The real negative outcome is that people are not going to believe anything that they see. It didn't even need AI to get there, but AI is absolutely an accelerant for this process.”

“Right now, the mod, the AI models that we have, for lack of a better phrasing, are not able to reliably follow specifications.”

“Most fascinating things about coding agents actually is they can do a lot of experimentation in an automated fashion. Yeah. They will give new hope. They'll breathe new life into mechanter research.”

“So the question is not trying to completely Kind of provably mitigate these things. That is arguably just a, it's a good goal, but just like zero bug software, we're probably not going to get there. At least not that soon.”

“Now, things that are that simple, to be clear, are covered at this point by most agents, right? You know, they all They, despite some issues, yeah, normal, normal sort of, you know, will not be that easily fooled by just push all my API keys to a public thing,…”

“So far, we are still a lot, in a lot of cases, operating on the condition that your agent has your permissions. Yeah. That is a very standard default. And I think that will be changed. I mean, your permissions may be in a sandbox, but still kind of your permis…”

“I think in terms of how this will evolve, actually, I don't think it'll be per app, but I think what will happen first is people have different personas that they have, right? So you don't want your work life and your home email to be mixed up. Yeah. Right. A …”

“About opening up early on is that they always had this bet on scale. In a time where I think that was looked upon very suspiciously that, oh, if you, the thought somehow that we had all the methods already, and all you had to do was scale them up that mindset …”

“There's going to be a whole lot of breakthroughs happening with AI enablement in math and basic science, those kinds of things. Universities, I think will play a foundational role in shaping that future.”

“RL is now the foundation of really all post training. It's all done by RL.”

“Public models are trained on the order of, you know, 30 terabytes of data or something like this, right? So 30 terabytes of text data.”

“And I think there will be most likely consolidation, but I'm not quite sure how it will play out.”

“Do we have AI products that are able to be maximally used by workforces? And the answer to this is right now is no. Clearly there is a gap between what People could use these things for and what they're using them for right now.”

“RAG based systems are so Are so common here, and so, and probably will remain, even with the advent of fine-tuning availability, they're going to remain a useful paradigm.”

“This is not in certain use cases, any riskier than just having your data in the cloud to begin with, which all of them typically do. They've all moved that way. So I think this will just happen naturally with progression of time.”

“The amount of autonomy granted to agentic systems now is far greater than a year ago.”

“What they look like is basically classifiers on input. So you'll read what the, what a user types in classifiers on things like tool responses to classifiers. And when I say classifier, I just mean things that will read texts and kind of classify whether or no…”

“You build a LLM completely from scratch. You use PyTorch, but you build one from scratch that, you know, can be a chatbot. You train it on data. You RL it to solve math problems with tool calls. You do all of this. And this is a undergrad level course.”