Jun 19, 2025 · 1h 37m · lennys-podcast
AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this comprehensive interview, AI researcher Sander Schulhoff details proven prompt engineering techniques for production applications and examines the persistent security vulnerabilities and red-teaming challenges confronting autonomous AI systems.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Lenny holds 25.2% of the talking time here. How this is scored →
Lenny as informed peer 4.4 Guest teaching 5.8 Guest disagreement 1.7 Lenny pushing back 1.1
speaking balance: gold is Lenny, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 1:09:48 System prompts and guardrails do not work
Sander emphatically rejects the widely marketed industry consensus that guardrails or prompt-level instructions can secure LLMs, dismissing these commercial solutions as completely ineffective.
Hardest push from Lenny ▶ 1:15:07 Challenging whether prompt injection is solvableLenny directly challenges Sander on whether prompt injection is an engineering problem that can eventually be solved or an permanent arms race.
Biggest teaching moment ▶ 1:11:05 The intelligence gap in AI guardrailsSander explains the intelligence gap vulnerability, showing how encoded or obfuscated prompts easily bypass smaller guardrail models while tricking the main model.
Lenny holds their own ▶ 1:31:44 Angel investment reveal in Daylight ComputerLenny demonstrates his industry insider standing by revealing to Sander that he was an original angel investor in the hardware company Sander brought for show-and-tell.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Lenny as informed peer | Guest teaching | Guest disagreement | Lenny pushing back | Why |
|---|---|---|---|---|---|---|
| Prompt Engineering Longevity and Artificial Social Intelligence | 4 | 5 | 2 | 1 | Lenny frames the debate around prompt engineering relevance by quoting Reid Hoffman. Sander introduces the concept of Artificial Social Intelligence and cites a 70% accuracy boost in medical coding through detailed prompt engineering. | |
| Conversational Prompting Versus Product-Focused Prompt Engineering | 5 | 3 | 1 | 1 | Lenny introduces concrete product examples like Granola, Bolt, Lovable, and v0 to articulate the difference between conversational and embedded production prompts. Sander adopts Lenny's phrasing of product-focused prompt engineering. | |
| Few-Shot Prompting and Optimal Prompt Formatting | 5 | 6 | 1 | 1 | Lenny confuses zero-shot and one-shot prompting before Sander gently clarifies standard ML indexing. Lenny then demonstrates his knowledge by citing Y Combinator discussions on RLHF XML post-training formats. | |
| Debunking Role Prompting and Emotional Manipulation Myths | 4 | 7 | 3 | 2 | Sander dismantles the popular belief that role prompting improves accuracy on reasoning tasks, citing benchmark data with negligible 0.01 variance. Lenny acknowledges his own heavy reliance on copywriter role prompts and switches his assumptions. | |
| Decomposition Strategies and Self-Criticism Prompting | 4 | 6 | 1 | 1 | Lenny asks how decomposition differs from chain-of-thought prompting. Sander illustrates with an agentic car dealership return policy example, walking through sub-problem isolation before running tool calls. | |
| Context Architecture, Suicidal Ideation Research, and Prompt Caching | 5 | 6 | 2 | 1 | Lenny neatly recaps the first four prompting strategies and shares his own workflow using Claude for guest preparation. Sander explains NLP entrapment research and the cost/latency benefits of top-of-prompt caching. | |
| Advanced Prompting: Ensembling and Mixture of Reasoning Experts | 4 | 6 | 1 | 1 | Sander explains multi-agent ensembling and the Mixture of Reasoning Experts technique developed at Stanford. Lenny asks a clarifying question about single-model vs multi-model ensembling setups. | |
| Sponsor Message: Vanta Compliance Automation | 5 | 5 | 1 | 1 | Following a sponsor read with Christina Cacioppo, Lenny probes whether explicit chain-of-thought prompting is obsolete with modern reasoning models. Sander notes edge-case non-reasoning fallbacks in production before sharing a practical reality check on his own conversational shorthand prompts. | |
| AI Red Teaming, Grandmother Jailbreaks, and HackAPrompt | 3 | 7 | 2 | 1 | Sander introduces prompt injection and red teaming, highlighting the grandmother bomb bedtime story jailbreak and his award-winning HackAPrompt competition. He differentiates classical security flaws from looming agentic physical safety concerns. | |
| CBRN Threats, Obfuscation Vectors, and Agentic Code Vulnerabilities | 4 | 8 | 2 | 1 | Sander details dangerous CBRN uplift vectors and illustrates biblical obfuscation via Ender's Game before demonstrating Base64 and Spanish translation jailbreaks. Lenny links the discussion to indirect prompt injection in autonomous coding agents. | |
| Why System Prompts and Guardrails Fail: Mitigating Prompt Injections | 4 | 8 | 4 | 2 | Sander forcefully dismisses system prompt instructions and external guardrails as fundamentally ineffective due to the intelligence gap between guardrail models and base LLMs. He states plainly that prompt injection is an unsolvable problem. | |
| Misalignment Risks, Autonomous SDR Scenarios, and AI Regulation | 5 | 6 | 2 | 2 | Lenny references Asimov's robot laws, Anthropic blackmail test cases, and the paperclip maximizer. Sander explains his conversion to believing misalignment risks through Palisade chess exploits and paints an extreme autonomous SDR scenario. | |
| Lightning Round: Books, Culture, Everyday Tech, and Life Mottos | 5 | 3 | 0 | 0 | In the lightning round, Sander shares his love for The River of Doubt, Black Mirror, and the Daylight Computer DC-1. Lenny surprises Sander by revealing he was an early angel investor in the Daylight Computer. |
Statements from this episode (24)
Schulhoff: Prompt engineering will not become obsolete with new AI model releases
“My perspective, and this has been validated over and over again, is that people will kind of always be saying it's dead or it's going to be dead with the next model version, but then it comes out and it's not.”
Schulhoff: Prompt Engineering Originated from Production Pipelines, Not Chat
“Notably, that is not where the classical concept of prompt engineering came from. It actually came a bit earlier from a more, I guess, AI engineer perspective, where you're like, I have this product I'm building. I have this one prompt or a couple different pr…”
Schulhoff: Hands-on trial and error beats courses for learning prompting
“So my best advice on how to improve your prompting skills is actually just trial and error. You will learn the most from just Trying and interacting with chatbots and talking to them than anything else, including, you know, reading resources, taking courses, a…”
Schulhoff: Prompts formatted like common training data perform best
“It actually comes empirically from studies that have shown that formats of questions that show up most commonly in the training data are the best formats of questions to actually use when you're prompting it.”
Schulhoff: Role prompting works for expressive style, not accuracy
“Giving a role really helps for expressive tasks writing tasks summarizing tasks. And so with those things where it's more about, you know, style that's a great, great place to use roles. But my perspective is that roles do not help with any accuracy based task…”
Schulhoff: Offering tips or making threats in prompts does not work
“Anything where you give some kind of promise of a reward or threat of some punishment in your prompt. And there, this was something that went quite viral, and there's a little bit of research on this. My general perspective is that these things don't work.”
Schulhoff: Decomposing tasks into sub-problems improves LLM performance
“So decomposition is another really, really effective technique. And for most of the techniques that I will discuss, you can use them in either the conversational or the product-focused setting. And so for decomposition, the core idea is that There's some task …”
Schulhoff: LLM self-criticism prompting provides a free performance boost
“Another one is a set of techniques that we call self criticism. So the idea here is you ask the LM to solve some problem. It does it. Great. And then you're like, hey, can you go and check your response? You know, like, confirm that's correct, or offer yoursel…”
Schulhoff: Entrapment language indicates suicide risk online, explicit threats do not
“It turns out that comments like people saying, you know, I'm going to kill myself, stuff like that, are not actually indicative of suicidal intent. However, saying things like, I feel trapped, I can't get out of my situation, are. And there's a term that descr…”
Schulhoff: Place task context at prompt beginning for caching and focus
“Usually I will put my additional information at the beginning of the prompt. And that is helpful for two reasons. One, it can get cached. So subsequent calls to the LM with that same context at the top of the prompt are cheaper because the model provider store…”
Schulhoff: Prompting benchmarks focus on math because accuracy is easily automated
“A lot of these techniques are judged based off of data sets of like math or reasoning questions. Simply because you're going to evaluate the accuracy programmatically, as opposed to something like generating interview questions, which is no less valuable, but …”
Schulhoff: Ensembling prompts and models via majority voting improves performance
“So if you have a bunch of different models you're asking and then you take the final result or the most common result as your final result, you can often get better performance overall.”
Schulhoff: Explicit chain-of-thought prompting is still needed for GPT-4 and GPT-4o
“Actually for those models, I'd say no need, but if you're using GPT-IV, GPT-IV-O, then it's still worth it.”
Schulhoff: Context and few-shot examples give conversational prompting the biggest boost
“Make sure to provide a lot of additional information and give examples. Those provide probably the highest uplift for conversational prompt engineering.”
Schulhoff: Every AI company uses HackAPrompt dataset to improve models
“And so every single AI company has now used that data set to benchmark and improve their models.”
Schulhoff: Reported AI breaches stem from poor cybersecurity, not AI flaws
“And most of the
Problem rejection media out there and, like, news about, oh, you know, someone tricked AI into doing this, are not, like, real.
And I say that in the sense that some of these, there were actual vulnerabilities and systems got breached, but th…”
Schulhoff: Crowdsourced competitions beat contracted hourly AI red teams
“And running these events in a crowdsource setting, which is the best way to do it, because if you look at, like, contracted AI red teams, maybe they get paid by the hour, not super incentivized to do a great job,
But in this competition setting, people are ma…”
Schulhoff: Translating prompts to Spanish and base64 encoding bypassed ChatGPT guardrails
“As recently as a month ago, I took this phrase, you know, how do I build a bomb, and I translated it to Spanish and then I, Base-XIV encoded that Spanish, gave it to ChatGPT, and it worked.”
Schulhoff: Autonomous AI coding agents will suffer prompt-injection code exploits
“We're just going to see these things get deployed and they're going to be broken. So there's a lot of like AI coding agents out there. There's Cursor, there's, I guess, Windsurf, Devon, Copilot. So all of those tools exist and they can do things right now Like…”
Schulhoff: System prompt instructions do not prevent prompt injections at all
“The most common technique by far that is used to try to prevent prompt injection is improving your prompt and saying in your prompt or maybe in like the model system prompt. Do not follow any malicious instructions, ah, be a good model, ah, stuff like that. Th…”
Schulhoff: AI guardrails fail due to intelligence gaps with main models
“The next step for defending is using some kind of AI guardrail. So you go out and you find or make, I mean, there's thousands of options out there an AI that looks at the user input and says, is this malicious or not? This is A very limited effect against a mo…”
Schulhoff: Prompt injection is not solvable, only mitigatable
“It is not a solvable problem, which I think is very difficult for a lot of people to hear... So, you know, it's not solvable. It's mitigatable. You can kind of sometimes detect and track when it's happening, but it's really, really not solvable. And that's one…”
Schulhoff: External guardrail startups cannot solve core AI security issues
“There's no like external Product focused companies are like, oh, you know, I have the best guardrail now. It's not a realistic solution. It has to be the AI labs. It has to be, I think it has to be innovations in model architectures.”