Feb 6, 2025 · 1h 2m · latent-space
Agent Engineering with Pydantic + Graphs — with Samuel Colvin, CEO of Pydantic Logfire
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode, Pydantic founder Samuel Colvin joins Alessio Fanelli and Swix to explore Pydantic's expansion into artificial intelligence, detailing Pydantic AI's type-safe agent graph architecture, the performance optimizations behind Pydantic V2 and V3, and Logfire's high-throughput observability platform.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 30.6% of the talking time here. How this is scored →
The hosts as informed peer 5.4 Guest teaching 4.8 Guest disagreement 2.7 The hosts pushing back 2.7
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 26:32 Samuel dismisses exponential AI speed claims and explains the role of agent frameworks
Samuel forcefully refutes optimistic claims about exponential model improvements, arguing that agents and graphs are merely temporary guardrails compensating for models lacking sufficient intelligence.
Hardest push from the hosts ▶ 29:01 Swyx challenges Pydantic AI for re-implementing model adaptersSwyx delivers an extended critique pressing Samuel on why Pydantic AI reimplements model adaptation rather than relying on dedicated gateways like LiteLLM or Portkey.
Biggest teaching moment ▶ 44:51 Samuel breaks down why ClickHouse failed for Logfire's span query engineSamuel details specific interval comparison bugs in ClickHouse and demonstrates how building custom kernels on Apache DataFusion in Rust resolved their analytical storage needs.
The host holds their own ▶ 16:52 Swyx leverages Temporal experience to evaluate Pydantic AI orchestration boundariesSwyx utilizes his background as a former Temporal team member and investor to dissect the boundary tradeoffs between code-level control flow, process isolation, and infrastructure orchestration.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| JSON Schema Adoption and Pydantic V2/V3 Performance Roadmap | 5 | 5 | 2 | 2 | Alessio and Swyx inquire about JSON Schema origin stories and compare Pydantic's lifecycle to Javascript validation libraries like Zod. Samuel details the internal Rust architecture planned for V3 and how JSON schemas became integrated via FastAPI. | |
| Transitioning to a Startup and Model Latency Impact | 4 | 6 | 1 | 1 | Alessio asks how Pydantic transitioned focus to GenAI tooling. Samuel explains how the core V2 Rust rewrite resulted in a 20% reduction in time-to-first-token for major LLM provider SDKs. | |
| Launching Pydantic AI and Elevating Framework Standards | 4 | 6 | 4 | 1 | Samuel sharply critiques recent opportunism and poor software engineering standards across the existing agent framework ecosystem, detailing how Pydantic AI enforces type safety, generics, and runnable docstring tests. | |
| Pydantic AI Architecture and Type-Safe Workflow Graphs | 5 | 5 | 3 | 2 | Alessio and Swyx drill into graph representations and type safety. Samuel explains how he shifted from being cynical about graphs to using node dataclass return type introspection for compile-time type safety. | |
| Workflow Orchestration, Distributed State, and Cloudflare Workers | 7 | 4 | 2 | 5 | Swyx brings his deep domain experience from Temporal to challenge Pydantic AI entering workflow orchestration territory. Samuel clarifies the library scope versus infrastructure, highlighting upcoming distributed state and Cloudflare Pyodide execution. | |
| Multi-Agent Patterns, Swarms, and the Bitter Lesson | 6 | 4 | 4 | 4 | Swyx presents the Bitter Lesson and compound AI systems argument where larger base models wipe out bespoke agent scaffolding. Samuel agrees and dismisses hype about exponential model speedups, arguing agent frameworks simply compensate for model unintelligence. | |
| Model Adapters, API Standardization, and Continuous Integration Testing | 7 | 4 | 4 | 6 | Swyx delivers an informed rant questioning why Pydantic AI builds its own model adapter layer instead of delegating to LiteLLM or Portkey. Samuel pushes back citing type safety requirements, proxying security risks, and industry standardization on OpenAI-compatible schemas. | |
| Mock Testing, LLM Evals, and OpenTelemetry Observability | 5 | 5 | 2 | 2 | Alessio and Samuel discuss mock testing and evals methodology. Samuel highlights the OpenTelemetry semantic conventions for GenAI and points out how Pydantic AI avoids proprietary vendor lock-in unlike LangSmith. | |
| OpenTelemetry Span Latency and Migrating to Apache DataFusion | 6 | 6 | 3 | 3 | Swyx questions why Logfire switched from ClickHouse to Timescale and then to DataFusion. Samuel offers an in-depth breakdown of ClickHouse interval bugs, JSON limitations, and why custom Rust-based DataFusion kernels provide superior query control. | |
| Logfire Market Positioning, Licensing, and Pydantic Run Sandbox | 5 | 4 | 2 | 2 | Swyx and Alessio ask how Logfire differentiates against Datadog and Sentry, as well as licensing choices. Samuel explains the closed-source model for Logfire alongside MIT-licensed Pydantic AI and showcases the Pydantic Run browser sandbox. | |
| Reinventing Notebooks with Marimo and Pydantic AI Roadmap | 5 | 4 | 3 | 2 | The conversation covers plain-text notebooks with Marimo, building in London versus San Francisco, and startup hiring discipline. Samuel defends his strict focus on coding and profitability runway over rapid hiring. |
Statements from this episode (21)
Swix: Pydantic reached nearly 300 million downloads in December
“It's, you know, it's almost three hundred million downloads in December.”
Colvin: Pydantic and FastAPI normalized Python type-hint schemas
“Back in 2017, when I first started it, the like different thing it was doing was using type int to define your schema. That was controversial at the time. It was like genuinely disapproved of by some people. I think the success of Pydantic and libraries like f…”
Colvin: Sebastián Ramírez originally added JSON Schema to Pydantic for OpenAPI
“So I originally didn't implement JSON Schema inside Pydantic and then Sebastian, Sebastian Ramirez, FastAPI came along and like the first I ever heard of him was over a weekend. I got like 50 emails from him or 50 like emails as he was committing to Pydantic a…”
Colvin: Rust-native data storage could yield 3x to 5x Pydantic speedup
“We have plans to move some of the, basically store the data in rust types after validation. Not convert to Python types. So then if you were doing like validation and then serialization, you would never have to go via a Python type. We reckon that can give us …”
Colvin: Major AI lab cut time-to-first-token 20% upgrading to Pydantic v2
“I can't say which one, but one of the big foundational model companies, when they upgraded from Pydantic v one to v two, their number one internal metric of performance is time to first token That went down by 20%.”
Colvin: Building better GenAI tools is easy compared to replacing Postgres
“To go and implement a database that's better than Postgres is a, like, Sisyphean task, whereas building tools that are better for Gen.ai than some of the stuff that's about now is not very difficult. Putting the actual models themselves to one side.”
Colvin: Agent framework engineering quality lags far behind Python ecosystem
“Looking in general at the ecosystem of agent frameworks, the engineering quality is far below that of the rest of the Python ecosystem.”
Colvin: Pydantic AI reimplemented its agent architecture as graphs under the hood
“We actually merged the PR this morning. That means our agent implementation without changing its API at all is now actually a graph under the hood, as in it is built using our graph library.”
Colvin: Pydantic AI will soon support state storage across graph nodes
“We will add in soon support for, well, basically storage so that you can store the state between each node that's run. And then you'd be, the idea is you can then distribute a graph and run it across compute. And also, I mean, the other weird, the other bit th…”
Colvin: Cloudflare Workers struggles with binary Python dependencies like Pydantic
“Pyodide, which is Python running inside the browser in Scripton, is supported now by Cloudflare. They just, they basically, they're having some struggles working out how to manage, ironically, dependencies that have binaries, in particular Pydantic”
Colvin: OpenAI Said Pydantic AI Resembles Production-Ready Swarms
“OpenAI have got in touch with me, and basically, maybe I'm not supposed to say this, but basically said that Pydantic AI looks like what Swarms would become if it was production-ready.”
Colvin: Agent Frameworks Exist Because AI Models Are Not Clever Enough
“Agents, agent frameworks, graphs, all of this stuff is basically making up for the fact that right now the models are not that clever.”
Colvin: AI providers are centralizing around OpenAI's API standard
“I think the truth is that everyone is centralizing around OpenAI's SD API as the one to do. So DeepSeek support that. Grok with a K support that. Olama also does it. Well, I mean, if there is that library right now, it's more or less the OpenAI SDK.”
Colvin: Pydantic AI will be first framework implementing OpenTelemetry GenAI attributes
“I suspect Pedantic AI will be the first agent framework that implements those semantic attributes properly, because again, we control Pedantic AI, and we can say this is important for observability, whereas most of the other agent frameworks are not maintained…”
Colvin: GenAI telemetry carries vastly more sensitive PII than traditional metrics
“With Gen AI, that, that distinction doesn't exist because it's all just like messed up in the text. If you have that same patient asking an LLM how to, what drug they should take or how to stop smoking, you can't extract the PII and not send it to the Observab…”
Colvin: OpenTelemetry repos will eventually host AI SDK semantic instrumentation
“I suspect eventually most of those semantic, like that instrumentation of the big, of the SDKs will live, like I say, inside the main open telemetry repos.”
Colvin: Logfire migrated from ClickHouse to Timescale before choosing DataFusion
“We don't use ClickHouse. We started building our database with ClickHouse, moved off ClickHouse onto Timescale, which is a Postgres extension to do analytical databases. And then moved off Timescale onto Data Fusion, and we're basically now building, it's Data…”
Colvin: Gen AI observability will merge into general-purpose observability platforms
“Web observability stopped being a thing, not because the web stopped being a thing, but because all observability had to do web. If you were talking to people in 2010 or 2012, they would have talked about cloud observability. Now that's not a term because all …”
Colvin: Jupyter Notebooks Are Worse Than or Similarly Bad to Excel
“I have very strong opinions about, you know, proper, like Jupyter notebooks, this idea that like you have to run the cells in the right order. I mean, a whole bunch of things. It's basically like worse than Excel or similarly bad to Excel.”
Colvin: Every Region Outside US and China Is Way Behind on AI
“I think, look, like, everywhere that isn't the US and China knows that we're, like, way behind on AI.”
Colvin: Pydantic Freezes Hiring to Maintain Multi-Year Runway
“We are not hiring right now. I want, I would love Bloodleaf and Logfire to have a bit more commercial traction and a bit more revenue before I hire some more people. It's quite nice having a few years of runway, not a few months of runway.”