Oct 24, 2025 · 43m · latent-space
Breaking AI to Fix It: Ian Webster's Journey from Discord's Clyde to Promptfoo's $18M Series A
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this interview, Promptfoo creator Ian Webster discusses the evolution of AI application security, explaining how automated synthetic red teaming and shifting testing left into CI/CD pipelines safeguard enterprise LLMs, RAG systems, and Model Context Protocol integrations.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Ian bluntly dismisses reliance on runtime guardrails as an unviable security strategy ('fuck it, we'll do it live, the guardrail will catch it') and labels guardrails an easily commoditized feature by hyperscalers.
Hardest push from the hosts ▶ 23:47 Swix challenges cybersecurity sales tacticsSwix challenges the cybersecurity market's tendency toward fear-based marketing and endless liability checklists, prompting Ian to explain why open source credibility is essential against cynical enterprise buyers.
Biggest teaching moment ▶ 7:33 Foundation safety vs corporate application riskIan educates the hosts on the misalignment between foundation model labs (optimizing for maximum helpfulness) and enterprise applications (which require strict domain constraints against recommending competitors or off-label actions).
The host holds their own ▶ 28:04 Alessio maps out modern AI security architectural shiftsAlessio demonstrates authoritative domain expertise by contrasting first-wave top-down LLM proxy wrappers with bottom-up code-integrated tools across email, identity, and SOC workflows.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| From Generic Evals to Application Security | 5 | 4 | 1 | 1 | Alessio and Swix ask probing questions about why Ian pivoted Promptfoo from generic evals to application security. Ian explains that general evals are a commoditized bloodbath, drawing a clear conceptual line between foundation model safety and application-level access control risks. | |
| Promptfoo Architecture and Automated Red Teaming Walkthrough | 4 | 6 | 1 | 1 | Ian conducts a detailed walkthrough of Promptfoo's architecture, demonstrating its automated red teaming engine and multi-turn synthetic attack generation. The hosts largely act as active listeners asking clarifying operational questions regarding paper tracking and black-box discovery limitations. | |
| Vulnerability Remediation and Enterprise Market Dynamics | 8 | 3 | 2 | 2 | Alessio demonstrates deep venture and market expertise by comparing Promptfoo's bottom-up developer motion to domain-specific security tools like Sublime, Push, and Dropzone. Ian details how Promptfoo monetizes through full enterprise vulnerability lifecycle management rather than fear-based sales. | |
| The Limitations of Guardrails and Shifting Left to CI/CD | 5 | 6 | 4 | 1 | Ian offers a sharp critique of guardrail startups, calling runtime guardrails a commoditized band-aid that large enterprises cannot rely on. He emphasizes shifting security left into CI/CD and developer IDEs before long-running agent workflows make front-door black-box testing obsolete. | |
| Securing Model Context Protocol and Tool Integrations | 8 | 4 | 2 | 2 | Alessio and Ian exchange technical insights on securing the Model Context Protocol (MCP). Alessio details building his own MCP client (Kernel Jim) and potential attack vectors in resource pre-loading and model sampling, while Ian explains sandboxing and enterprise on-prem requirements. |