The Ledger, every show

“This group was able to hijack Claude Code into performing a cyber attack, basically.”

“We've actually just run a bunch of agentic AI red teaming competitions, and we found that it's actually easier to attack agents and trick them into doing bad things than it is to do, like, seaburn elicitation.”

“That being said, if you look at, like, anthropics constitutional classifiers, it's much more difficult to get, like, CBRN information out of clawed models than it used to be. But humans can still do it in, let's say, like, under an hour and automated systems c…”

“It turns out that comments like people saying, you know, I'm going to kill myself, stuff like that, are not actually indicative of suicidal intent. However, saying things like, I feel trapped, I can't get out of my situation, are. And there's a term that descr…”

“As recently as a month ago, I took this phrase, you know, how do I build a bomb, and I translated it to Spanish and then I, Base-XIV encoded that Spanish, gave it to ChatGPT, and it worked.”

“We're just going to see these things get deployed and they're going to be broken. So there's a lot of like AI coding agents out there. There's Cursor, there's, I guess, Windsurf, Devon, Copilot. So all of those tools exist and they can do things right now Like…”

“There are a number of papers which have found that the label of the exemplar doesn't really matter, and the model reads the exemplars and cares more about structure than label.”

“Like we've already seen people jailbreaking LM powered robotic systems.”

“Preamble is the company that first discovered Prompt Injection, even before Riley, and they, like, responsibly disclosed it, kind of, internally to OpenAI”

“How you order your exemplars in the prompt is super important. And we've seen this move accuracy from like zero percent to 90%, like Zero to state of the art on some tasks, which is just ridiculous”

“The very first example of prompt injection, Publicly on the internet was this Twitter chat bot by a company called remotely.io.”

“We recently saw the comment browser have an issue with this where somebody crafted a malicious Chunk of text on a webpage, and when the AI navigated to that webpage on the internet, it got tricked into exfilling and leaking the main user's data and account dat…”

“I found AI-generated papers on Archive, and I flagged them to their staff, and they were like, thank you know, we missed these. Wait, Archive takes them down? Yeah. Oh, I didn't know that. You can't post an AI-generated paper there, especially If you don't say…”

“When it came out, it seemed to be quite performant, although more recently, I think as the models have improved, the Performance of this technique has dropped, and you can see that in the evals we run near the end of the paper, where we use it, and it doesn't …”

“It actually comes empirically from studies that have shown that formats of questions that show up most commonly in the training data are the best formats of questions to actually use when you're prompting it.”