Mar 12, 2026 · 1h 0m · latent-space
Retrieval After RAG: Hybrid Search, Agents, and Database Design — Simon Eskildsen of Turbopuffer
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Turbopuffer founder Simon Eskildsen joins the Latent Space podcast to discuss building a high-performance, cost-effective search engine for unstructured AI data by decoupling storage and compute on cloud object storage and NVMe SSDs. He shares architectural breakthroughs, enterprise customer case studies like Cursor and Notion, and insights into engineering culture and investor relations.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 16.3% of the talking time here. How this is scored →
The hosts as informed peer 4.2 Guest teaching 4.5 Guest disagreement 1.1 The hosts pushing back 1.2
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 47:45 Playful challenge on candidate hiring filters and maps
Simon playfully confronts and disqualifies the hosts when they fail to relate to obsessively scrolling on maps as a core engineer trait.
Hardest push from the hosts ▶ 42:56 Alessio presses Simon on why not merge into CursorAlessio directly challenges Simon on why he did not simply join Cursor internally given their massive growth rather than taking on startup risk.
Biggest teaching moment ▶ 18:18 Masterclass on compare-and-swap and S3 limitationsAfter Swyx admits not knowing what compare-and-swap is, Simon breaks down the exact mechanics of CAS metadata management and cloud consistency.
The host holds their own ▶ 49:05 Swyx formulates DevRel through geographic mappingSwyx counters Simon's map obsession by demonstrating how developer relations is fundamentally about charting boundaries and user journeys across technical trade-offs.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Introductions and the Danish Programming Mafia | 5 | 3 | 1 | 1 | Swyx introduces the Danish programming mafia lineage and Alessio asks Simon to define Turbopuffer's identity between search engine and vector database. Simon articulates Turbopuffer as an unstructured data search engine for AI. | |
| Three Conditions for Building a Generational Database | 4 | 6 | 1 | 1 | Alessio sets up the comparison against Elasticsearch and traditional search architectures. Simon delivers an authoritative breakdown of the three generational conditions needed to build a massive database company. | |
| Origin Story: Shopify Scaling and Readwise Cost Bottlenecks | 4 | 4 | 1 | 1 | Swyx prompts Simon to recount his background scaling databases at Shopify and angel engineering at Readwise. Simon details how a $30,000 monthly vector infra cost estimate inspired the napkin math behind Turbopuffer. | |
| Napkin Math Architecture: Object Storage and NVMe | 5 | 5 | 2 | 2 | Alessio and Swyx ask about read vs. write workloads and compare Turbopuffer's approach to Neon. Simon explains designing the database around object storage and NVMe bandwidth limits to minimize round trips. | |
| Cloud Storage Innovations and Compare-and-Swap Primitives | 4 | 7 | 1 | 1 | Swyx asks about S3 strong consistency and admits not knowing compare-and-swap (CAS). Simon educates the hosts on how CAS enabled them to avoid running a consensus cluster like Zookeeper. | |
| Notion Onboarding: Extreme Latency Optimization and Dark Fiber | 4 | 6 | 1 | 2 | Swyx and Alessio question why Turbopuffer bought dark fiber between AWS and GCP rather than running standard coordination layers. Simon explains how reducing network hops allowed Notion's aggressive latency SLAs to be met. | |
| Customer Deep Dive: Slashing Cursor's Infrastructure Costs by 95% | 4 | 5 | 1 | 1 | Swyx asks Simon to share his perspective on the Cursor customer journey. Simon recounts fixing Cursor's early database issues, onboarding them in Tmux, and slashing their infrastructure costs by 95%. | |
| Codebase Search Architecture and Security Posture | 5 | 4 | 1 | 1 | Swyx inquires about codebase search workloads and brings up the grep vs. RAG debate. Simon explains Cursor's custom embedding models and high security posture using bucket-level encryption. | |
| The Shift from RAG to Agentic Concurrency | 5 | 5 | 1 | 1 | Alessio and Swyx explore how search queries evolved from single context-stuffing calls to massive parallel agent queries. Simon discloses a 5x price cut to accommodate agentic concurrency. | |
| Hardware Economics, Pricing Iterations, and Early Profitability | 4 | 5 | 1 | 1 | Alessio asks how Turbopuffer prices against value and storage. Simon recounts accidentally achieving early profitability because he ran the early infrastructure on his personal credit card. | |
| Partnering with Investor Lockie: Radical Honesty Over Database Expertise | 4 | 4 | 1 | 1 | Swyx asks whether picking a generalist investor like Lockie was better than selecting a database specialist. Simon shares his philosophy of total transparency and offering money back if PMF failed. | |
| Founder Commitment and the Gravity of Capital | 5 | 4 | 1 | 3 | Alessio presses Simon on whether he considered simply joining Cursor rather than raising capital and founding a standalone company. Simon details the intense founder commitment required once taking outside capital. | |
| Defining the 'P9' Engineer and Engineering Excellence | 5 | 5 | 2 | 2 | Swyx asks about Simon's P9 engineer concept. Simon humorously tests the hosts on their love for maps and trains, and explains how top engineers bend software to their will toward first-principles limits. | |
| Turbopuffer Roadmap: Full-Text Search and Scaling Milestones | 4 | 5 | 1 | 1 | Swyx asks about the future roadmap. Simon describes expanding from vector search into full-text search, beating Lucene on long queries, and preparing ANN v4 and v5 for web-scale datasets. | |
| Future Database Acts and the Discipline of Startup Focus | 5 | 4 | 1 | 1 | Swyx asks whether Turbopuffer plans to natively support graph queries. Simon explains that underlying key-value primitives enable graph and OLAP workloads, but startup focus prevents premature overextension. | |
| Green Tea Connoisseurship and Precision Rituals | 3 | 5 | 1 | 1 | Swyx and Simon discuss Simon's passion for specialty green tea. Simon describes his Airtable tracking 200 teas, harvest seasons, and traveling with a digital thermometer to hit exact water temperatures. | |
| Podcast Conclusion and Farewell | 1 | 0 | 0 | 0 | Swyx and Alessio wrap up the podcast episode and thank Simon for joining. |
Statements from this episode (31)
Eskildsen: Model weights compress reasoning, not all world knowledge
“We can take all of the world's knowledge, all of the exabytes and exabytes of data that there is, and we can use those tokens to train a model, but we can't compress all of that into a few terabytes of weights, right? We can compress into a few terabytes of we…”
Eskildsen: Every company will connect all data to AI within years
“I don't think you're going to find a company over the next few years that doesn't directly or indirectly have all their data available for search and connected to AI.”
Eskildsen: Turbopuffer loses no data if all servers shut down
“In fact, you could turn off all the servers that TurboPuffer has, and we would not lose any data because we have All completely all in on our big storage.”
Eskildsen: Kardashian Sales on Shopify Peaked at 1 Million Requests per Second
“And a lot of that was really just the you know the Kardashians would drive very, very large amounts of data to Shopify as they were rotating through all the merge and building out their businesses. And we just needed to make sure we could handle that, right? A…”
Eskildsen: Elasticsearch Was the Most Difficult and Aggravating Database at Shopify
“The database that was the most difficult for me to scale during that time, and that was the most aggravating to be on call for, was Elasticsearch. It was very, very difficult to deal with, and I saw a lot of projects that were just being held back in their amb…”
Eskildsen: Vector Search for Readwise Would Have Cost Six Times Its Total Infra Bill
“But this was a company that was spending maybe five grand a month in total on all of their infrastructure. And when I did the napkin math on running the embeddings of all the articles, putting them into a vector index, putting it in prod, it's going to be like…”
Eskildsen: Object-storage-first databases trade 200ms write latency for pure upside
“The only real downside to that is that if you go all in on object storage, every write will take a couple hundred milliseconds of latency, but from there, it's really all upside, right? You do the first query, it takes half a second”
Eskildsen: Modern hardware requires databases designed around concurrent requests, not sequential roundtrips
“You really have to build a database where you have as few round trips as possible, right? This is how CPUs work today. It's how NVMe SSDs work. It's how S three works that you want to have a very large amount of outstanding requests, right? Like basically go t…”
Eskildsen: Neon retrofitted Postgres for S3, while Turbopuffer built pure object-storage consensus
“I think neon neon was first to, and they're trying to retrofit it onto Postgres. And then they built this whole architecture where you have it in memory, and then you sort of like, you know, mmap back to S-III, and I think that was very novel at the time to do…”
Eskildsen: AWS S3 Only Achieved Strong Consistency in December 2020
“S III only became consistent in December of twenty-twenty.”
Turbopuffer Bought Oregon Dark Fiber to Serve Notion Across Clouds
“It started getting really painful in like mid-twenty-twenty-four, because we were closing deals with Notion actually, that was running in AWS, and we're like, trust us, you really want us to run this in GCP?
And they were like, no, I don't know about that, lik…”
Eskildsen: AWS S3 Did Not Support Compare-and-Swap Until Late 2024
“Compare and swap to do metadata, which wasn't in S three until late, 20, 24.”
Eskildsen: Oregon GCP-to-AWS latency is roughly 14ms due to Seattle routing
“Like U.S. East, the Google, like the GCP and AWS data centers are like within a millisecond on each other on the public exchanges. But in Oregon, uniquely the GCP data center sits like a couple hundred kilometers, like east of Portland and the AWS region sits …”
Eskildsen: The worst outages stem from multi-system unsynchronized state
“And the worst outages are the ones where you have state in multiple places that's not syncing up. So it really came from a, like, just a very pure source of pain of just imagining what we would be okay being woken up at three a.m. About, and having something i…”
Eskildsen: Turbopuffer's Notion workload still runs reliably on GCP
“This workload still runs on GCP for what it's worth, right? Because it's so, it was just, it was so reliable. So it was never about moving off GCP.”
Eskildsen: AI shifts build-versus-buy from technical ability to speed
“And I think AI has also changed the buy versus build equation in terms of, it's not really about, can we build it? It's about, do we have time to build it?”
Eskildsen: Turbopuffer reduced Cursor's infrastructure costs by 95%
“They migrated everything over the next like week or two, and we reduced her cost by 95%, which I think like kind of fixed their per user economics.”
Eskildsen: Macro AI predictions are a waste of time because nobody can forecast AI
“I don't like just doing like thought pieces on this is where it's going and like trying to be all macroeconomic about AI. That's as turned out to be a giant waste of time because no one can really predict any of this. So I just, Collect case studies.”
Eskildsen: Cursor obfuscates file paths and encrypts code stored in Turbopuffer
“Cursors like security posture into Turbo Puffer is exceptional, right? They have their own embedding model, which makes it very difficult to reverse engineer. They obfuscate the file paths. They are like, You, it's very difficult to learn anything about a code…”
Swix: All retrieval workloads are hybrid, combining semantic, text, regex, and SQL
“All workloads are hybrid. Like, you know like you want the semantic, you want the text, you want the regex, you want SQL. I don't know. But like, it's silly to like be all in on like one particular query pattern.”
Eskildsen: Notion and Cursor agents run unprecedented search query concurrency
“Notion does a ridiculous amount of queries in every round trip, just because they can. And I'm also now, when I use the cursor agent, I also see them doing more concurrency than I've ever seen before.”
Eskildsen: Turbopuffer cuts query pricing 5x for agentic workloads
“We've reduced query pricing. This is probably the first time actually I'm saying that, but the query pricing is being reduced. Like, Five X. And we'll probably try to reduce it even more to accommodate some of these workloads of just doing very large amounts o…”
Eskildsen: Turbopuffer is profitable
“It now means that we're profitable because we had so much pricing pressure in the beginning.”
Eskildsen: Anthropic, Notion, and Cursor use Turbopuffer across three deployment models
“You can run Turbo Puffer either in SAS, right? That's what cursor does. You can run it in a single tenant cluster. So it's just you. That's what Notion does. And then you can run it in, in, in BYOC where everything is inside the customer's VPC. That's what, fo…”
Eskildsen: Offered to return capital if Turbopuffer lacked PMF by year-end
“I don't think I've said this publicly before, but I just called Locky and was like, well, Locky, like, if this doesn't have PMF by the end of the year, like, we'll just like return all the money to you. But it's just like, I don't really, Justine and I don't w…”
Eskildsen: Turbopuffer benefited from having an investor without database expertise
“The other people were talking at the time where database experts, like they, you know, knew a lot about databases and Lockheed didn't. This turned out to be a phenomenal asset, right? I like Justine and I know a lot about databases. The people that we hired kn…”
Eskildsen: Turbopuffer interview debriefs default to rejecting candidates unless championed
“And so I have a document called Traits of the P-Nine engineer, and it's a bullet point list, and I look at that list after every single interview that I do, and in every single recap that we do, and every recap we end with, I end with some version of, I'm gonn…”
Eskildsen: Turbopuffer ANN v3 searches 100B vectors with 40ms p50 latency
“ANN v. Three can search a hundred billion vectors with a p-fifty of around 40 milliseconds and a p-ninety-nine of 200 milliseconds. Maybe other people have done this. I'm sure Google and others have done this, but we haven't seen anyone at least not in like a …”
Eskildsen: Turbopuffer outperforms Lucene on long LLM search queries
“Turbo Puffer today has a fairly start of the state of the art full text search engine.
We beat Lucene on some queries, in particular, very long queries that we've optimized for, because those are the text search queries we see today.”
Eskildsen: Scaling a database company requires supporting almost every query plan
“If you want to build a big database company, The database over time has to implement more or less every query plan, because when you have your data in a database, you expect it to over time, not just search, but also, Hey, I want to aggregate this column. I wa…”