Feb 22, 2024 · 31m · no-priors
No Priors Ep. 52 | With Pinecone CEO Edo Liberty
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Pinecone founder and CEO Edo Liberty joins hosts Sarah Guo and Elad Gil on No Priors to discuss vector database architecture, enterprise retrieval-augmented generation (RAG), and why specialized infrastructure outperforms legacy databases and massive context windows. Liberty details the launch of Pinecone Serverless and shares his architectural vision of decoupling AI reasoning from external knowledge storage.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 20.4% of the talking time here. How this is scored →
The hosts as informed peer 5.6 Guest teaching 5.9 Guest disagreement 1.9 The hosts pushing back 1.2
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 20:42 Edo calls out infinite context claims as token sales marketing
Edo rejects the premise of infinite context windows, pointing out that vendors sell by the token and that stuffing entire corpora into context is practically absurd.
Hardest push from the hosts ▶ 15:29 Sarah challenges Edo on whether hybrid search is merely temporarySarah directly questions Edo's prediction on hybrid search, asking him to clarify if combining keywords with embeddings is just a transient stopgap.
Biggest teaching moment ▶ 20:57 Edo dismantles the long-context window replacement narrativeEdo explains why context stuffing degrades accuracy and scales poorly, using the analogy that sending an entire query with the internet is as impractical as replacing Google with full-corpus context.
The host holds their own ▶ 19:20 Elad shares Databricks insight on open-source business modelsElad demonstrates industry depth by citing Databricks founder Ali Ghodsi's sports metaphor regarding the difficulty of building a viable enterprise business on top of open-source software.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Origins of Pinecone and Vector Database Category | 5 | 5 | 1 | 1 | Sarah and Elad ask foundational questions about vector databases and the timing of Pinecone's founding in 2019 before the generative AI explosion. Edo explains how models represent data as numeric embeddings rather than raw text/pixels. | |
| Implementing RAG Architecture and Benchmarking Accuracy | 6 | 6 | 1 | 1 | Sarah frames the trade-offs in context length and reliability that lead developers to RAG. Edo explains Pinecone's Common Crawl experiment showing RAG cuts hallucinations across major models by up to 50%. | |
| Enterprise Production Use Cases with Notion and Gong | 5 | 6 | 2 | 1 | Edo gently corrects Elad's question by clarifying that Canopy is an open-source framework whereas Serverless is Pinecone's core database architecture. He then explains how production customers like Notion and Gong scale to billions of vectors. | |
| Simplifying RAG Pipelines with Canopy Framework | 6 | 6 | 2 | 1 | Sarah probes developer pain points and asks why traditional solutions like Postgres/PGVector or Elastic cannot suffice. Edo details why retrofitted vector search on relational engines breaks down under scale and cost constraints. | |
| Mechanics of Hybrid Search and Vector Representations | 6 | 6 | 2 | 2 | Sarah presses Edo on whether hybrid search combining keywords and embeddings is merely a temporary bridge. Edo explains the mathematical equivalence of keywords to sparse vectors and predicts explicit keyword matching will fade. | |
| Pinecone's Managed SaaS Architecture Versus Open Source | 7 | 5 | 2 | 1 | Sarah questions Pinecone's proprietary closed-source model relative to open-source database norms. Elad reinforces this with an analogy from Databricks CEO Ali Ghodsi comparing open-source commercialization to hitting a grand slam with a baseball bat after a golf hole-in-one. | |
| Long Context Windows Versus Vector Database Retrieval | 4 | 7 | 4 | 2 | Elad asks about the rise of massive context windows and infinite context. Edo pushes back forcefully against marketing hype, noting that model providers profit off token billing and that context stuffing degrades performance while proving economically unfeasible. | |
| Enterprise Data Privacy Personalization and GDPR Compliance | 6 | 6 | 1 | 1 | Elad raises concerns about data leakage across multi-tenant enterprise models. Edo explains how decoupling model inference from data storage in a vector DB preserves GDPR compliance and instantaneous data deletion without complex retraining. | |
| Choosing Between Prompt Engineering Fine-Tuning and RAG | 6 | 6 | 1 | 1 | Elad breaks down the three architectural options developers weigh: prompt engineering, fine-tuning, and RAG. Edo contrasts the scientific promise of fine-tuning with the commercial reality that poor execution frequently degrades model accuracy. | |
| Pinecone Infrastructure Roadmap and Classical IR Challenges | 5 | 6 | 3 | 1 | Elad asks for Pinecone's roadmap and broader AI predictions. Edo passionately critiques the inefficiency of current monolithic architectures that cram the internet into GPU memory, calling for clear separation between reasoning and knowledge engines. |
Statements from this episode (17)
Liberty: Mainstream Engineers Were Already Adopting BERT by 2019
“In 2019, the earthquake had already happened. Deep learning models and so on have already been grappled with. Large language models and transformer models like BERT and others started being used by the more mainstream engineering cohorts.”
Liberty: RAG Over Internet Data Reduces LLM Hallucinations by 50%
“And you could see that if you augment all of them with RAG on, even on the internet, which is data that they were trained on, you can reduce hallucinations significantly up to 50% sometimes.”
Liberty: RAG Levels the Playing Field Across Different Foundation Models
“Interestingly enough, many of them actually start behaving quite similarly in terms of level of accuracy, even though without RAG, they actually have quite different behaviors. So it's sort of both like a uniform improvement and a little bit of leveling the pl…”
Liberty: Notion Q&A runs AI question answering on Pinecone
“Notion Q&A now runs on, on Pinecone, and they serve essentially question answering with AI to tens of thousands and probably hundreds of thousands of their own customers.”
Liberty: Gong uses Pinecone for all customer sales call search
“Gong does the same thing with sales calls. Again, serves all of their use cases for all of their customers, and so on.”
Liberty: Pinecone Serverless Tested with Tens of Billions of Vectors
“We've tested it with tens and tens of billions with live customers and live traffic.”
Liberty: Retrofitted Vector Indexes Like pgvector Fail at Production Scale
“Those other products don't work. They don't work either because they don't scale in terms of the efficiency scale, cost, the trade-offs that they can offer, because they're not designed to do this. They're designed to do something else. They kind of thought ab…”
Liberty: Keyword search is a deeply flawed retrieval method for AI
“With other search technologies, this is again, this is the wrong search mode. If you're searching with keywords and just not finding The relevant information, because the embeddings, the contextual space in which these pieces of text, documents, or images live…”
Liberty: Proper embedding retrieval rarely requires keywords alongside embeddings
“Our research actually shows that when you do this well, we, you very rarely need keywords alongside embeddings, but getting embeddings to perform perfectly is, is actually, it could be quite intricate.”
Liberty: High 90s Percentage of OSS Code Comes From Vendor Employees
“And in fact, if you look at statistics, even companies that are open source, 99% of the contributions are actually from the company itself. Not 99, but high nineties.”
Liberty: OSS Vector Database Competitors Are Already Struggling With Commercialization
“And in fact, we already see, even though new players in the vector database space that, that, that basically started to try to take us down, all took the open source angle. We already see them, even young as they might be, they are already struggling, struggli…”
Liberty: Pinecone Serverless is the fourth near-complete rewrite of its database
“Serverless is the fourth complete, almost complete rewrite of the entire database at Pinecon.”
Liberty: Context Window Stuffing Increases LLM Costs Without Improving Results
“There's plenty of evidence that increasing the context size doesn't actually improve results unless, you know, you do this very carefully, right? So just what's called constant stuffing is not helping. You just pay more and don't actually get much for it.”
Liberty: RAG Architectures Pair Small Models With Trillion-Parameter Vector Databases
“Already today, we have users who use not even very large models, you know, maybe a few billion parameters, and the vector database next to the model contains trillions of parameters. And they get, you know, much better performance that way.”
Liberty: Vector databases enable GDPR compliance without complex model unlearning
“And the added benefit to that is, by the way, that you can be GDPR compliant. You can actually delete data. So if, you know, so, you know, if you're a company like a legal company and somebody deletes a document, you can just delete it from the vector database…”
Liberty: Fine-tuning without expert teams often worsens model performance
“Unless you have the research team and the AI experts that know how to fine tune, you might actually make things significantly worse. Okay. So there is, there's nothing that says that more data is going to make your model do better. In fact, it oftentimes gets …”
Liberty: Foundation Models Fundamentally Flawed by Combining Reasoning With Knowledge
“Foundational models get it fundamentally wrong. When we learn how to Build the subsystems of AI correctly, and for each one of them to do their roles optimally. Either we're going to do, be able to do the, to achieve the same tasks much cheaper, faster, better…”