Sep 6, 2023 · 44m · mad
From NLP Start-Up to Generative-AI Platform: Milos Rusic (deepset) Unpacks Product-Market Fit
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of The MAD Podcast, host Matt Turck interviews Milos Rusic, Co-Founder and CEO of Deepset, about the company's evolution from an NLP consulting firm into the creator of the open-source Haystack framework and Deepset Cloud platform. Rusic discusses enterprise Retrieval-Augmented Generation (RAG), mitigating model hallucinations, open-source monetization, and real-world deployment cases across Global 2000 companies.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 18% of the talking time here. How this is scored →
Matt as informed peer 3.8 Guest teaching 4.8 Guest disagreement 1.2 Matt pushing back 1.8
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 28:03 Rejection of open-core framing
Milos explicitly pushes back against Matt's implicit assumption that Deepset follows an open-core business model, emphasizing that all core technical components remain fully open source.
Hardest push from Matt ▶ 20:09 Host probes core infra vs Lego blocksMatt directly challenges Milos on whether Deepset is actually building original low-level cloud infrastructure or merely stitching together existing third-party Lego blocks.
Biggest teaching moment ▶ 11:13 Detailed breakdown of RAG pipeline stepsMilos walks step-by-step through document chunking, vector filtering, retrieval, and generator models to educate the host on why multi-step orchestration is required for LLMs.
Matt holds his own ▶ 5:30 Host defines Haystack conciselyMatt demonstrates clear technical understanding by jumping in to summarize Haystack as an orchestration framework that connects models and databases, receiving full agreement from the guest.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| FirstMark Opening Logo Animation | 3 | 3 | 1 | 1 | Matt opens with detailed background stats on Deepset's financing and co-founders before asking about their bootstrap origin. Milos warmly agrees and outlines their journey from 2018 consulting into venture-backed growth. | |
| Creation of Haystack and Pivot to Deepset Cloud | 5 | 4 | 1 | 2 | Matt interjects cleanly to define Haystack as an orchestration framework for NLP deployment, which Milos enthusiastically confirms. Milos details their transition from custom consulting work to open-source Haystack and Deepset Cloud. | |
| Navigating the Transition from Consulting to SaaS | 4 | 6 | 1 | 2 | Matt brings up the well-known venture challenge of shifting company DNA from services to product. Milos acknowledges the point and educates on the technical workflow of Haystack using a podcast Q&A example. | |
| Supported Vector Databases and Model Integrations | 4 | 5 | 1 | 2 | Matt probes into vector database compatibility and integration overhead. Milos explains how maintenance is co-owned with DB vendors as vector products differentiate. | |
| Deepset Cloud Commercial SaaS Platform Overview | 3 | 5 | 1 | 1 | Matt prompts an overview of Deepset Cloud. Milos contrasts GenAI evaluation complexities with classic time-series ML, explaining the workflow tooling required for production LLM apps. | |
| Infrastructure Scaling and Heavy Lifting in the Cloud | 5 | 4 | 2 | 3 | Matt pushes on whether Deepset is building low-level infrastructure or assembling existing cloud blocks. Milos clarifies that cloud infra is mature and Deepset's value lies in removing undifferentiated heavy lifting. | |
| Mitigating Model Hallucinations with RAG and Guardrails | 3 | 5 | 1 | 1 | Matt highlights the hallucination feature set. Milos breaks down RAG architecture and guardrail nodes that block erroneous outputs in regulated industries. | |
| Deepset Cloud Commercial Pricing Strategy | 4 | 5 | 2 | 3 | Matt asks about consumption pricing and the split between open source and paid tiers. Milos politely rejects the open-core label, clarifying that core tech remains open source while SaaS handles enterprise workflow. | |
| Enterprise Go-To-Market Strategy and Buyer Personas | 4 | 5 | 1 | 2 | Matt asks how Deepset sells across ML engineers, developers, and product owners. Milos explains that developers build technical trust while Product Owners hold the budget for business use cases. | |
| Enterprise Case Studies: Manz Legal and Airbus | 3 | 6 | 1 | 1 | Matt invites specific customer stories. Milos details legal intelligence at Manz and single-pilot cockpit data access at Airbus. | |
| Global 2000 Enterprise vs. Startup AI Adoption | 4 | 5 | 1 | 2 | Matt notes that enterprise clients differ from typical startup customers and asks about Global 2000 appetite. Milos explains the contrast between enterprise control requirements and startup speed needs. | |
| Emerging AI Trends and LLM Observability | 3 | 5 | 1 | 1 | Matt asks about broader AI trends and the European tech ecosystem. Milos highlights LLM observability and discusses European AI talent alongside conservative enterprise adoption trends. |
Statements from this episode (15)
Deepset announced a Balderton-led Series B, reaching $45 million total funding
“And you just announced last week a Series B round of financing led by my friend James Wise at Balderton, making it a total of forty-five million dollars raised to date across the A and the B.”
Deepset's early consulting clients included Airbus and European federal authorities
“We worked for companies like Airbus many federal authorities in Europe software providers in Germany.”
The only meaningful way to adopt NLP and LLMs is cloud-based
“The only meaningful way to adopt to adopt NLP and LLMs is in the cloud”
Haystack open-source framework reaches 10,400 GitHub stars and 186 contributors
“Which is a very popular open source project with 10 .4000 stars for people who care about the things and a 186 contributors.”
RAG systems need dedicated hallucination detectors to verify LLM outputs
“If you want to make sure that there's not a, that the model doesn't hallucinate, you probably want a hallucination detector on top, right? Something that classifies an answer and confirms, is this answer really part of my database?”
Model transparency and research drive enterprise AI adoption
“More transparency we can create also around, you know, how models have been trained, You know, the more research we have around the model, of course, the more trust it creates and easier it is for companies to adopt.”
Maturing database products differentiate, making integrations harder to maintain
“All these products are maturing and also differentiating, right? So it is becoming more and more of an effort, to be honest, to really manage this.”
Consumption is the best metric to reflect LLM system value
“Is probably the best metric to somehow reflect the value that comes out of these LLM systems, right? Because in the end, the more requests a model gets, or application gets, probably the more valuable it is. The more data you load into it, probably the more va…”
AI startups prioritize rapid shipping over accuracy and factfulness
“If you care so much around about the factfulness of your applications, you're probably not a startup, right? Startups, you know, you want to ship fast. You do whatever works best. You assemble something, it's shipped.”
Deepset's core LLM tech stack will always remain open source
“Everything that you need to really have an LL application, like technically the full tech stack, that will always be open source.”
Enterprise GTM requires selling directly to use-case product owners
“To sell into the enterprise, what you need is you really need a use case, right? And you really need to have clarity about the use case. And the best person to sell to is the person that is responsible for serving a use case. And this is a persona that we refe…”
RAG will be the first LLM architecture to see mainstream enterprise adoption
“We see that this whole RAC architecture is probably the first one that will really see a broad, massive adoption, right? Simply because it's all about accessing and, you know, somehow working with information that is already around. And this is where we see, l…”
Airbus single-pilot AI search application will take a few more years
“For Airbus, the case is it's something they are working on. It's probably something that will take a few more years because it's all embedded in this overall vision of single pilot operations.”
Rusic highlights London-based LLM observability startup Context
“One company I like is, for example, Context from London.”
Europe's conservative enterprise tech adoption limits local startup growth ceilings
“Europe is that simply, you know, a bit more conservative in the way they adopt new technologies, right? And this is this is usually then a bit harder for these companies to, you know, to grow in their home markets, right?”