Willison: AI 'lethal trifecta' risk is solvable only by eliminating one core capability
Simon Willison · An AI state of the union: We’ve passed the inflection point & dark factories are coming · Apr 2, 2026 · at 1:21:00
Simon Willison breaks down his security framework for AI agents handling private data, untrusted inputs, and external communication.
0:00 / 0:40exact quote · 41.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“You have a lethal trifecta. Anytime your agent has three things, it's got Access to private information. There's information that you've exposed to it, like your private inbox that, that is, is private in some way. It's exposed to malicious instructions. So there's no way somebody attacking you can get their text into your system, like sending you an email. And the third leg is exfiltration or some mechanism that the agent can send data back to that attacker, like forwarding an email. So if you've got a system where you've got private emails, anyone can email you instructions and it can email them back. That's a, That's the classic lethal trifecta. That's a huge security problem. The only way to fix it is to cut off one of those three legs.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Simon Willison
Willison: GPT-5.1 and Claude Opus 4.5 marked an inflection point for coding agents
“In November, we had what I call the inflection point where GPT, 5.1 and Claude Opus 4.5 came along and they were both just, they were incrementally better than the previous models, but in a way that crossed a threshold where previously, if you had these coding…”
Willison: Code is easier for AI agents because it is verifiably right or wrong
“Like code is easier than almost every other problem that you pose these agents because code is obviously right or wrong. Like it produces code. You run the code. Either it works or it doesn't work. There might be a few subtle hidden hidden bugs, but generally …”
Willison: Large-scale agentic engineering will always require deep developer expertise
“The art of having them help you build software, you could deploy to a million people. That's not, that's never going to be easy. That's never going to be trivial. That's always going to require a great deal of depth of experience in what software and how softw…”
Willison: StrongDM implemented a policy where engineers never read code
“The next rule though is nobody reads the code and this is the thing which StrongDM started doing back in, I think it was August last year. They said, okay, we're not going to read the code.”
Willison: AI agents have become credible security researchers in recent months
“At the same time, the agents are getting really good at security penetration testing now. And this is a new thing. I think in the past, again, in the past sort of three to six months, they've started being credible as security researchers, which is sending sho…”
Willison: Mid-career software engineers face the greatest AI displacement risk
“The problem is the people in the middle. Like if you're mid career, if you haven't made it to sort of super senior engineer yet, but you're not sort of new either. That's the group, which ThoughtWorks resolved work probably in the most trouble right now.”