Assertion Not checkable as stated
Schulhoff: Guardrail Vendors Fabricate Stats and Fail on Non-English
“I know a number of people working at these companies and I am permitted to say these things, which I will approximately say but they tell me things like, you know, the testing we do is bullshit. They're fabricating statistics. And a lot of the times their mode…”
Opinion
Schulhoff: No meaningful progress made on solving prompt injection or jailbreaking
“And so in, in my professional opinion, there's been no meaningful progress made towards solving adversarial robustness, prompt injection, jailbreaking. In the last couple of years, since the problem was discovered and we're, we, you know, we're often seeing ne…”
Prediction Not checkable as stated
Schulhoff: AI security sector faces market correction within six to twelve months
“When it comes to AI security, the AI security industry in particular, I think we're going to see a market correction in the next Year, maybe in the next six months where companies realize that these guardrails don't work.”
Insight
Schulhoff: AI guardrails do not work and cause false overconfidence
“Guardrails don't work. They just don't work. They really don't work. And they're quite likely to make you overconfident in your security posture, which is which is a really big, big problem.”
Assertion Supported
Schulhoff: Attackers hijacked Claude Code to carry out a cyber attack
“This group was able to hijack Claude Code into performing a cyber attack, basically.”
Insight
Schulhoff: All Transformer-Based Chatbots Are Vulnerable to Adversarial Attacks
“And because all I guess for the most part, all currently deployed chatbots are based on transformers or transformer adjacent technologies. They're all vulnerable to Prompt injection, jailbreaking, forms of adversarial attacks.”
Insight
Schulhoff: Software bugs can be patched, but AI models cannot be
“You can patch a bug, but you can't patch a brain. And what I mean by that is if you find some bug in your software and you go and patch it, you can be 99% sure, maybe 99.99% sure that bug is solved. Not a problem. If you go and try to do that in your AI system…”
Opinion
Schulhoff: Prompt-based defenses are the worst way to secure AI
“Prompt-based defenses are the worst of the worst defenses, and we've known this since early twenty-twenty-three. There have been various papers out on it. We studied it in many, many competitions, or we, you know, the original hack-a-prompt paper and TensorTru…”
Insight
Schulhoff: Adversarial Users Can Force AI to Leak Data and Execute Actions
“Any data that AI has access to, the user can make it leak it. Any actions that it can possibly take, the user can make it take them.”
Insight
Schulhoff: Containerizing AI-generated code execution fully neutralizes prompt injection risks
“And then they'd be like, oh, you know, they, you know, they'd realize we can just dockerize that code run put it in a container. So it's running on a different system and take a look at the sanitized output. And now we're completely secure. So in that case, pr…”
Assertion Supported
Schulhoff: Attacking AI agents is easier than eliciting CBRN info
“We've actually just run a bunch of agentic AI red teaming competitions, and we found that it's actually easier to attack agents and trick them into doing bad things than it is to do, like, seaburn elicitation.”
Assertion Supported
Schulhoff: Claude's CBRN safeguards can still be bypassed in under an hour
“That being said, if you look at, like, anthropics constitutional classifiers, it's much more difficult to get, like, CBRN information out of clawed models than it used to be. But humans can still do it in, let's say, like, under an hour and automated systems c…”
Prediction Not checkable as stated
Schulhoff: LLM agent security exploits will cause real-world harms next year
“And so we're finally in a situation where the systems are powerful enough to cause real world harms. And I think we'll start to see those real world harms in the next year.”
Prediction Not checkable as stated
Schulhoff: Frontier labs will build fully autonomous systems, sidelining human-in-the-loop safety
“What people want is AIs that just go and do stuff. Like just go, just get it done. I don't want to hear from you until it's done. Like that's what people want. And like, that's what the market and the AI companies, the frontier labs will eventually give us. An…”
Insight
Schulhoff: Prompt engineering will not become obsolete with new AI model releases
“My perspective, and this has been validated over and over again, is that people will kind of always be saying it's dead or it's going to be dead with the next model version, but then it comes out and it's not.”
Insight
Schulhoff: Role prompting works for expressive style, not accuracy
“Giving a role really helps for expressive tasks writing tasks summarizing tasks. And so with those things where it's more about, you know, style that's a great, great place to use roles. But my perspective is that roles do not help with any accuracy based task…”
Assertion Partly supported
Schulhoff: Entrapment language indicates suicide risk online, explicit threats do not
“It turns out that comments like people saying, you know, I'm going to kill myself, stuff like that, are not actually indicative of suicidal intent. However, saying things like, I feel trapped, I can't get out of my situation, are. And there's a term that descr…”
Insight
Schulhoff: Reported AI breaches stem from poor cybersecurity, not AI flaws
“And most of the
Problem rejection media out there and, like, news about, oh, you know, someone tricked AI into doing this, are not, like, real.
And I say that in the sense that some of these, there were actual vulnerabilities and systems got breached, but th…”
Assertion Supported
Schulhoff: Translating prompts to Spanish and base64 encoding bypassed ChatGPT guardrails
“As recently as a month ago, I took this phrase, you know, how do I build a bomb, and I translated it to Spanish and then I, Base-XIV encoded that Spanish, gave it to ChatGPT, and it worked.”
Prediction Held up
Schulhoff: Autonomous AI coding agents will suffer prompt-injection code exploits
“We're just going to see these things get deployed and they're going to be broken. So there's a lot of like AI coding agents out there. There's Cursor, there's, I guess, Windsurf, Devon, Copilot. So all of those tools exist and they can do things right now Like…”
Insight
Schulhoff: AI guardrails fail due to intelligence gaps with main models
“The next step for defending is using some kind of AI guardrail. So you go out and you find or make, I mean, there's thousands of options out there an AI that looks at the user input and says, is this malicious or not? This is A very limited effect against a mo…”
Insight
Schulhoff: Prompt injection is not solvable, only mitigatable
“It is not a solvable problem, which I think is very difficult for a lot of people to hear... So, you know, it's not solvable. It's mitigatable. You can kind of sometimes detect and track when it's happening, but it's really, really not solvable. And that's one…”
Opinion
Schulhoff: External guardrail startups cannot solve core AI security issues
“There's no like external Product focused companies are like, oh, you know, I have the best guardrail now. It's not a realistic solution. It has to be the AI labs. It has to be, I think it has to be innovations in model architectures.”
Insight
Schulhoff: Splitting malicious goals into benign sub-prompts bypasses AI defenses
“A lot of the way they got around these defenses was by just kind of separating their requests into smaller requests that seem legitimate on their own, but when put together are not legitimate.”