Feb 15, 2024 · 33m · mad
Vector databases and the $8 trillion open source market | Bob van Luijt, CEO of Weaviate
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of 'The MAD Podcast with Matt Turck', Weaviate Co-Founder and CEO Bob van Luijt discusses the central role of vector databases and Retrieval Augmented Generation (RAG) in modern AI infrastructure. He explores high-dimensional embeddings, open-source monetization, competitive landscape dynamics, and the economic impact of AI-native platforms.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 14.7% of the talking time here. How this is scored →
Matt as informed peer 4.1 Guest teaching 4.0 Guest disagreement 1.0 Matt pushing back 1.1
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 22:48 Bob throwing shade at MongoDB
Bob mockingly dismisses general-purpose database incumbents adding vector search by stating it's 'a bit late though', prompting Matt to call out 'Shots fired'.
Hardest push from Matt ▶ 20:10 Host pressing for competitor evaluationMatt directly names competitors Pinecone and Chroma and challenges Bob to evaluate criteria contrasting Weaviate with them, encouraging him to 'name names'.
Biggest teaching moment ▶ 13:05 Vector search limitations on exact codesBob clearly illustrates where pure vector embeddings fall short by demonstrating how models fail on specific identifiers like 'ABC123', explaining the necessity of hybrid BM25 search.
Matt holds his own ▶ 6:03 Host RAG architecture summaryMatt succinctly synthesizes the complex relationship between LLMs, RAG, vector databases, and embedding models, prompting Bob to confirm 'That's fair to say'.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Explaining Vector Embeddings with the Supermarket Analogy | 2 | 4 | 0 | 0 | Matt asks a broad foundational question requesting a definition of vector embeddings for the audience. Bob uses a clear supermarket aisle analogy to explain vector similarity and geometric representations of data. | |
| Recapping RAG and the Evolution of Embedding Models | 5 | 3 | 1 | 1 | Matt demonstrates good understanding by playing back the RAG stack architecture and observing the recent shift in embedding models toward price competition. Bob confirms Matt's summary and outlines production memory and latency bottlenecks. | |
| Evaluating RAG Today and Generative Feedback Loops | 4 | 5 | 1 | 2 | Matt probes whether RAG is actually working as advertised or just industry hype. Bob framing RAG as a primitive first step, introducing the concept of Generative Feedback Loops where models write updated vectors back into the database. | |
| Hybrid Search and Native Model Integrations | 4 | 5 | 0 | 0 | Matt brings up Weaviate's specific focus on hybrid search. Bob educates on why vector search fails for specific alphanumeric IDs (like support ticket numbers) and why combining BM25 keyword search with vectors natively is essential. | |
| Weaviate Platform Differentiators and Enterprise Security | 3 | 2 | 1 | 1 | Matt prompts Bob for platform differentiators and explicitly reminds him to address enterprise security when Bob misses it. Bob jokes about hating security before explaining tenant isolation features. | |
| Real-Time Streaming Integrations with Confluent and Spark | 6 | 4 | 2 | 2 | Matt shows tech industry memory by teasing the perennial 'year of real-time' claim and explicitly names competitors Pinecone and Chroma to compare vector databases. Bob banters about not knowing the competitors and explains real-time Spark/Confluent pipelines. | |
| General-Purpose Databases vs. AI-Native Vector Platforms | 5 | 4 | 3 | 3 | Matt challenges Bob on the threat of incumbent databases like MongoDB adding vector features. Bob dismisses MongoDB as 'a bit late though', prompting Matt to banter 'Shots fired', while Bob explains why purpose-built AI-native platforms win. | |
| Open Source Business Model and the $8 Trillion Economic Value | 4 | 5 | 0 | 0 | Matt asks a sharp strategic question regarding open-source vs proprietary product boundaries. Bob articulates the open-source monetization strategy and cites a Harvard study valuing open source at $8 trillion to the global economy. |
Statements from this episode (16)
Van Luijt: RAG was the first unique vector database use case
“And reg was the first unique use case, if you will, that emerged around the ecosystem of vector databases.”
Van Luijt: RAG carries less hallucination risk than model fine-tuning
“That is something that is, works better than fine-tuning, for example, because if you fine-tune, then you're still dealing with potential hallucination Fair enough, with RAC that's possible too, but it's like, it's less it's less risky.”
Van Luijt: Current industry RAG implementations are primitive
“The way we, and I, with we, I mean like everybody in this room working on this stuff, are doing RAC right now, is actually pretty primitive, right?”
Van Luijt: Language is the most common use case for vector embeddings
“Language is the use case we see the most right now”
Van Luijt: Pure vector search alone is often not enough
“Vector search alone is often not enough, right?”
Van Luijt predicts 2024 will be the year of generative feedback loops
“I would not be surprised if this year, twenty-twenty-four, is the year of generative feedback loops.”
Van Luijt: Vector models perform poorly on specific alphanumeric IDs
“Matching on A, B, C, one, two, three is horrible. That's like, the model is super bad at that. Why? Because it was never trained on your on your knowledge base.”
Van Luijt: AI models are now the bottleneck for real-time serving
“The bottleneck is now these models.”
Van Luijt: More companies stream data with pre-calculated vector embeddings
“More and more companies start to literally stream the data with pre-calculated vector embeddings.”
Van Luijt: Vector database value comes from indexing, not basic storage
“The vector embedding is just a data type. So it's just an array of floating point numbers. I mean, you can store that in an old Oracle database, right? So, but the thing where it becomes interesting is the index.”
Van Luijt: Virtually every database now supports vector embeddings
“What we try to start to see now is that basically every database under the sun supports vector embeddings.”
Van Luijt: MongoDB is late to the vector database market
“It's great that they've seen the light too. It's a bit late though, but it's like a, you know, good for them.”
Van Luijt: Most billion-scale vector deployments are just similarity search
“So the most deployments, like the really, the billion scale deployments are very much just similarity search, right?”
Van Luijt: Commercial value sits around open source, not within core technology
“The job to be done, the value to be captured does not per se sit in the open source technology itself. It sits in the layers around that.”
Van Luijt: Harvard study values open-source software at $8 trillion
“They estimated that the value of open source to companies right now is eight trillion dollars, right? And if companies would have to rebuild it themselves, that would three and a half times be that number.”
Van Luijt: OpenAI recommended Weaviate when deprecating its search endpoint
“OpenAI had a they had a search endpoint that they deprecated. This is pre-ChatGBT, so, but they, so they deprecated that search endpoint, And say, well, you know, you just can buy our embeddings. There are three ways that you can index them and search for them…”