May 31, 2023 · 30m · mad
Long Term Memory for AI with Pinecone Founder & CEO, Edo Liberty
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of Data Driven NYC on The MAD Podcast, host Matt Turck interviews Pinecone Founder and CEO Edo Liberty about vector databases serving as long-term memory for generative AI. Liberty explains vector embeddings, Retrieval-Augmented Generation (RAG), Pinecone's developer-first growth model, and the future of autonomous AI agents.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 21.8% of the talking time here. How this is scored →
Matt as informed peer 3.2 Guest teaching 5.6 Guest disagreement 1.2 Matt pushing back 0.2
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 29:01 Rejecting prompt engineering as a sustainable profession
Edo directly rejects the popular premise that prompt engineering is a vital long-term role, arguing it is merely a temporary limitation of early crude AI models.
Hardest push from Matt ▶ 24:16 Host playfully encouraging competitor trash-talkMatt nudges Edo to criticize vector database competitors by reminding him the podcast is recorded and granting permission to speak freely.
Biggest teaching moment ▶ 4:07 Reframing model ingestion into external memory accessEdo reframes Matt's question on data transformation mechanics, explaining that modern AI architecture gives models access to external memory during inference rather than forcing data into models.
Matt holds his own ▶ 7:00 Connecting MLOps feature stores to vector databasesMatt demonstrates his tech stack knowledge by bringing up feature stores from MLOps history to probe whether vector databases belong to the same architectural category.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Defining Vector Embeddings and High-Dimensional Space | 2 | 6 | 1 | 0 | Matt opens by citing Pinecone's $138M funding round and asking for foundational definitions. Edo provides a comprehensive explanation of how neural networks process vector embeddings as brain-like activations. | |
| Multi-Modal Data Transformation and External AI Memory | 1 | 7 | 3 | 0 | When Matt asks about technical mechanics for converting video into numbers, Edo gently dismisses the premise as less interesting. He reframes the topic around external memory retrieval using a medical school analogy. | |
| What Is a Vector Database? | 4 | 6 | 1 | 1 | Matt shows MLOps domain knowledge by asking whether feature stores overlap with vector databases. Edo clarifies that feature stores serve real-time state changes while vector databases provide long-term semantic memory. | |
| Semantic Search vs. Traditional Keyword Search | 1 | 6 | 0 | 0 | Matt asks Edo to explain semantic search. Edo provides historical context on keyword indexing dating back to ancient print, contrasting it with searching by conceptual meaning. | |
| Pinecone's Founding Story and Edo Liberty's Background | 5 | 4 | 0 | 0 | Matt demonstrates deep background knowledge on Edo's career, interjecting details about his Yale PhD and the current scale of AWS SageMaker. Edo details his journey from academia through Yahoo and AWS to founding Pinecone in 2019. | |
| The Generative AI Tech Stack and Autonomous Agents | 3 | 6 | 1 | 0 | Matt lists key components of the generative AI stack, prompting Edo to elaborate. Edo explains how autonomous agents act as recursive software layers planning multi-step tasks. | |
| Enterprise Pinecone Use Cases and Reducing Hallucinations | 4 | 5 | 0 | 0 | Matt asks about enterprise use cases and correctly suggests vector databases act as long-term memory to prevent hallucinations. Edo confirms this hypothesis with internal measurement metrics. | |
| Implementing Retrieval-Augmented Generation (RAG) | 5 | 5 | 0 | 0 | Matt outlines a realistic architecture scenario for querying GPT-4 with enterprise data and asks about go-to-market strategies. Edo outlines the end-to-end prompt embedding pipeline and Pinecone's product-led growth model. | |
| Architectural Differentiation and System Scale | 4 | 5 | 2 | 1 | Matt asks about market competition and playfully encourages Edo to badmouth competitors. Edo politely declines, focusing instead on Pinecone's custom storage architecture and customer obsession. | |
| Audience Q&A: Hyperscalers, Recommendation Systems, and Prompt Engineers | 3 | 6 | 4 | 0 | During audience Q&A, an attendee asks about the longevity of prompt engineering roles. Edo strongly rejects the premise that prompt engineering is a permanent profession, calling it a temporary workaround for crude early technology. |
Statements from this episode (10)
Liberty: AI architecture is shifting toward external memory during inference
“I think there is a significant shift in the way that people think about these building these smart AI systems. And They don't try to fit the data into the model as much as give the model access to the right data when it's doing the inference, right?”
Edo Liberty: Vector databases serve as long-term memory for AI
“A vector database is the infrastructure that, that supports that long-term memory, right?”
Liberty: Keyword search dominated due to scaling efficiency, not search quality
“Traditional search is, is, is keyword based. Not because keyword search is that great, but just because we have great mechanism to run it at scale, and it's very efficient.”
Edo Liberty: Semantic search mirrors human memory better than keyword lookup
“With semantic search, the whole idea is that You know what it means. You can, you should be able to search by the meaning in free text, and the match would not be because, oh, like three words matched, and they're next to each other, but rather the meaning is …”
Liberty: Pinecone did not foresee the massive ChatGPT-driven AI surge
“We didn't, by the way, foresee any of this ChatGPT thing happening. We knew it was, it would keep growing, but that, I think, completely took everybody by surprise, including us.”
Liberty: A developer used Pinecone to build facial recognition for cows
“One of my favorite applications is somebody built a face detection for cows”
Liberty: Pinecone measures hallucination reduction as a core product metric
“In fact, I was looking at experiments today from one of our teams, and we literally measure reduction in hallucination as one of the core metrics that, that, that we try to drive.”
Liberty: Even naive RAG implementation significantly reduces AI hallucinations
“Like, you can play with it in a million different ways, but even if you do it relatively naively, that already gives you a huge bump in, in inaccuracy or reduction in hallucination, depending on how you want to measure it.”
Liberty: Cloud hyperscalers are actively building competing vector database capabilities
“I know for a fact they're looking at it. I know for a fact they will have something.”
Liberty: Prompt engineering is a fad, not a long-term profession
“A lot of people believe that prompt engineering is incredibly crucial. I don't necessarily buy into that. I don't know if you remember, but people, like, thought that, like you know query building for Google would be a profession too, right? And that didn't qu…”