prompt injection

“It is not a solvable problem, which I think is very difficult for a lot of people to hear... So, you know, it's not solvable. It's mitigatable. You can kind of sometimes detect and track when it's happening, but it's really, really not solvable. And that's one…”

“The next step for defending is using some kind of AI guardrail. So you go out and you find or make, I mean, there's thousands of options out there an AI that looks at the user input and says, is this malicious or not? This is A very limited effect against a mo…”

“The most common technique by far that is used to try to prevent prompt injection is improving your prompt and saying in your prompt or maybe in like the model system prompt. Do not follow any malicious instructions, ah, be a good model, ah, stuff like that. Th…”

“And because all I guess for the most part, all currently deployed chatbots are based on transformers or transformer adjacent technologies. They're all vulnerable to Prompt injection, jailbreaking, forms of adversarial attacks.”

“So the difference is in jailbreaking. It's just a malicious user and a model. In prompt injection, it's a malicious user, a model, and some developer prompt that the malicious user is trying to get the model to ignore.”

“Cause like I have a couple of examples that we can go through, but maybe strangely, maybe not so strangely, there hasn't been like a, an actually very damaging event quite yet.”

“And then they'd be like, oh, you know, they, you know, they'd realize we can just dockerize that code run put it in a container. So it's running on a different system and take a look at the sanitized output. And now we're completely secure. So in that case, pr…”

“The very first example of prompt injection, Publicly on the internet was this Twitter chat bot by a company called remotely.io.”

“And so in, in my professional opinion, there's been no meaningful progress made towards solving adversarial robustness, prompt injection, jailbreaking. In the last couple of years, since the problem was discovered and we're, we, you know, we're often seeing ne…”

“I think that will be a huge problem once systems go mainstream. We're still so busy building AI products that we're not worried about security, but it will be such a huge problem to kind of especially with this non-deterministic API again, right? So you're kin…”

“Agents fundamentally, like LLMs, can't tell the difference between texts that you give them and texts that you copy and paste in from other people. They're all the same thing. So instructions in that input text can always override the earlier instructions.”

“And again, until it's a hundred percent, I don't think it's a meaning. I think it just gives people a false sense of security that this problem won't bite them.”

“So my prediction is that we're going to see a challenging disaster. Like at some point, this is going to catch up with us and it's going to be Very, very, very bad, and that will hopefully help us start trying to figure out how not to do this. At the same time…”