Colvin: Major AI lab cut time-to-first-token 20% upgrading to Pydantic v2
Samuel Colvin · Agent Engineering with Pydantic + Graphs — with Samuel Colvin, CEO of Pydantic Logfire · Feb 6, 2025 · at 6:07
Samuel Colvin is the creator of Pydantic and CEO of Pydantic Logfire. He is discussing the performance impact of rewriting Pydantic in Rust.
0:00 / 0:12exact quote · 12.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I can't say which one, but one of the big foundational model companies, when they upgraded from Pydantic v one to v two, their number one internal metric of performance is time to first token That went down by 20%.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Samuel Colvin
Colvin: Agent framework engineering quality lags far behind Python ecosystem
“Looking in general at the ecosystem of agent frameworks, the engineering quality is far below that of the rest of the Python ecosystem.”
Colvin: LLMs are 100x faster under four specific engineering conditions
“My take is that there are four, four things where if you can cover all four of these things, LLMs are not like three X faster or five X faster. They're like a hundred X faster.”
Pydantic built a VIP issue scorer after closing an OpenAI founder's ticket
“Basically this started off because one of the OpenAI co-founders created an issue on Pydantic. And we just closed it and said it was wrong. And so we have this that, like, injects itself and tries to summarize someone and it gives them a, like, brutal score of…”
Colvin: Agent Frameworks Exist Because AI Models Are Not Clever Enough
“Agents, agent frameworks, graphs, all of this stuff is basically making up for the fact that right now the models are not that clever.”
Colvin: Logfire migrated from ClickHouse to Timescale before choosing DataFusion
“We don't use ClickHouse. We started building our database with ClickHouse, moved off ClickHouse onto Timescale, which is a Postgres extension to do analytical databases. And then moved off Timescale onto Data Fusion, and we're basically now building, it's Data…”
Colvin: Gen AI observability will merge into general-purpose observability platforms
“Web observability stopped being a thing, not because the web stopped being a thing, but because all observability had to do web. If you were talking to people in 2010 or 2012, they would have talked about cloud observability. Now that's not a term because all …”