The Ledger, every show

“I know a number of people working at these companies and I am permitted to say these things, which I will approximately say but they tell me things like, you know, the testing we do is bullshit. They're fabricating statistics. And a lot of the times their mode…”

“When it comes to AI security, the AI security industry in particular, I think we're going to see a market correction in the next Year, maybe in the next six months where companies realize that these guardrails don't work.”

“This group was able to hijack Claude Code into performing a cyber attack, basically.”

“We've actually just run a bunch of agentic AI red teaming competitions, and we found that it's actually easier to attack agents and trick them into doing bad things than it is to do, like, seaburn elicitation.”

“That being said, if you look at, like, anthropics constitutional classifiers, it's much more difficult to get, like, CBRN information out of clawed models than it used to be. But humans can still do it in, let's say, like, under an hour and automated systems c…”

“And so we're finally in a situation where the systems are powerful enough to cause real world harms. And I think we'll start to see those real world harms in the next year.”

“What people want is AIs that just go and do stuff. Like just go, just get it done. I don't want to hear from you until it's done. Like that's what people want. And like, that's what the market and the AI companies, the frontier labs will eventually give us. An…”

“It turns out that comments like people saying, you know, I'm going to kill myself, stuff like that, are not actually indicative of suicidal intent. However, saying things like, I feel trapped, I can't get out of my situation, are. And there's a term that descr…”

“As recently as a month ago, I took this phrase, you know, how do I build a bomb, and I translated it to Spanish and then I, Base-XIV encoded that Spanish, gave it to ChatGPT, and it worked.”

“We're just going to see these things get deployed and they're going to be broken. So there's a lot of like AI coding agents out there. There's Cursor, there's, I guess, Windsurf, Devon, Copilot. So all of those tools exist and they can do things right now Like…”

“There are a number of papers which have found that the label of the exemplar doesn't really matter, and the model reads the exemplars and cares more about structure than label.”

“And then I spent 20 hours prompt engineering for a task, and Dyspy beat me in 10 minutes, and that's when I changed my mind.”

“Like we've already seen people jailbreaking LM powered robotic systems.”

“Security concerns around Gen AI are preventing agentic deployments, and Gen AI is very difficult to properly secure.”

“Preamble is the company that first discovered Prompt Injection, even before Riley, and they, like, responsibly disclosed it, kind of, internally to OpenAI”

“How you order your exemplars in the prompt is super important. And we've seen this move accuracy from like zero percent to 90%, like Zero to state of the art on some tasks, which is just ridiculous”

“I remember I did a lot of experiments with GPT-IV, and especially when you look at it at scale, so I'll run thousands of prompts against it through the API, and I'll see, you know, every one in a hundred, every one in a thousand outputs no reasoning whatsoever…”

“The paper and the data set are now used by every single frontier lab and most fortune 500 companies to benchmark their models and improve their AI security.”

“Cause like I have a couple of examples that we can go through, but maybe strangely, maybe not so strangely, there hasn't been like a, an actually very damaging event quite yet.”

“The very first example of prompt injection, Publicly on the internet was this Twitter chat bot by a company called remotely.io.”

“This gets us a bit into the intersection of classical cybersecurity and AI security slash adversarial robustness, and this is where I think the security jobs of the future are. There's not an incredible amount of value in just doing AI red teaming. And I suppo…”

“We recently saw the comment browser have an issue with this where somebody crafted a malicious Chunk of text on a webpage, and when the AI navigated to that webpage on the internet, it got tricked into exfilling and leaking the main user's data and account dat…”

“And so every single AI company has now used that data set to benchmark and improve their models.”

“I found AI-generated papers on Archive, and I flagged them to their staff, and they were like, thank you know, we missed these. Wait, Archive takes them down? Yeah. Oh, I didn't know that. You can't post an AI-generated paper there, especially If you don't say…”

“When it came out, it seemed to be quite performant, although more recently, I think as the models have improved, the Performance of this technique has dropped, and you can see that in the evals we run near the end of the paper, where we use it, and it doesn't …”

“It actually comes empirically from studies that have shown that formats of questions that show up most commonly in the training data are the best formats of questions to actually use when you're prompting it.”