StarZero All shows Try the podcast clipper →

Nov 21, 2024 · 35m · mad

Building the Easy Button for Generative AI | May Habib, CEO, Writer

May Habib · 28m spoken Matt Turck · 4m spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

In this episode of The MAD Podcast, host Matt Turck interviews May Habib, CEO and Co-Founder of Writer, exploring how the company grew into a $1.9 billion full-stack generative AI platform. They discuss Writer's technical innovations—including proprietary Palmyra LLMs, graph-based RAG, and AI Studio—alongside enterprise security, go-to-market strategies, and the future of autonomous AI workflows.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 13.6% of the talking time here. How this is scored →

Matt as informed peer 3.3 Guest teaching 4.0 Guest disagreement 0.6 Matt pushing back 0.9
05100:0010:0020:0030:001:34–6:54 · Matt as informed peer 5/10 What Writer Does & Enterprise Value Proposition Matt demonstrates clear preparation by citing Writer's rumored $200M round at a $1.9B valuation, their previous Series B, and May's former company Cordoba. May provides a history of how they evolved from translation ML to transformer-based writing assistants.6:54–10:44 · Matt as informed peer 4/10 Building the Full-Stack AI Platform Architecture Matt directly raises the common criticism that early AI apps were thin wrappers on third-party LLMs. May clarifies that except for a brief three-month test with GPT-3, Writer has always built its own models down to the base LLM layer.10:44–18:05 · Matt as informed peer 4/10 Deep Dive: Graph-Based RAG vs. Vector Databases Matt asks May to compare graph-based RAG against traditional vector database approaches. May delivers a detailed technical breakdown explaining why vector chunking fails on nested tables and numerical data compared to flat JSON triples in Postgres with fusion-in-decoder techniques.18:05–20:22 · Matt as informed peer 2/10 AI Guardrails and Enterprise Governance Matt asks how Writer handles enterprise guardrails and compliance. May explains that simple regex filters fail in complex compliance environments, requiring fine-tuned LLM rewrites for PII and brand tone.20:22–23:16 · Matt as informed peer 1/10 Writer AI Studio: Custom Enterprise Application Development Matt checks whether AI Studio functions as the primary developer environment for custom apps. May details how complex enterprise logic requires extensive scaffolding beyond simple prompting.23:16–27:18 · Matt as informed peer 5/10 In-House Models: Palmyra X and Synthetic Data Efficiency Matt notes that Palmyra X 004 reportedly cost only $700,000 in GPU compute to train, contrasting it with typical mega-model training budgets. May explains how synthetic data designed specifically for LLM consumption drastically improves training efficiency.27:18–31:26 · Matt as informed peer 3/10 Navigating Enterprise AI Adoption & Go-To-Market Strategy Matt asks about the adoption curve and enterprise sales strategy. May describes navigating fortune 50 decision-makers and competing against hyperscalers with dedicated vertical solutions.31:26–35:25 · Matt as informed peer 2/10 The Future of Generative AI: Autonomous Workflows and Super Apps Matt prompts May on upcoming industry trends. May rejects the incremental 'productivity' framing, arguing that autonomous AI workflows will deliver 300% to 500% capacity expansions.1:34–6:54 · Guest teaching 2/10 What Writer Does & Enterprise Value Proposition Matt demonstrates clear preparation by citing Writer's rumored $200M round at a $1.9B valuation, their previous Series B, and May's former company Cordoba. May provides a history of how they evolved from translation ML to transformer-based writing assistants.6:54–10:44 · Guest teaching 3/10 Building the Full-Stack AI Platform Architecture Matt directly raises the common criticism that early AI apps were thin wrappers on third-party LLMs. May clarifies that except for a brief three-month test with GPT-3, Writer has always built its own models down to the base LLM layer.10:44–18:05 · Guest teaching 7/10 Deep Dive: Graph-Based RAG vs. Vector Databases Matt asks May to compare graph-based RAG against traditional vector database approaches. May delivers a detailed technical breakdown explaining why vector chunking fails on nested tables and numerical data compared to flat JSON triples in Postgres with fusion-in-decoder techniques.18:05–20:22 · Guest teaching 4/10 AI Guardrails and Enterprise Governance Matt asks how Writer handles enterprise guardrails and compliance. May explains that simple regex filters fail in complex compliance environments, requiring fine-tuned LLM rewrites for PII and brand tone.20:22–23:16 · Guest teaching 3/10 Writer AI Studio: Custom Enterprise Application Development Matt checks whether AI Studio functions as the primary developer environment for custom apps. May details how complex enterprise logic requires extensive scaffolding beyond simple prompting.23:16–27:18 · Guest teaching 5/10 In-House Models: Palmyra X and Synthetic Data Efficiency Matt notes that Palmyra X 004 reportedly cost only $700,000 in GPU compute to train, contrasting it with typical mega-model training budgets. May explains how synthetic data designed specifically for LLM consumption drastically improves training efficiency.27:18–31:26 · Guest teaching 4/10 Navigating Enterprise AI Adoption & Go-To-Market Strategy Matt asks about the adoption curve and enterprise sales strategy. May describes navigating fortune 50 decision-makers and competing against hyperscalers with dedicated vertical solutions.31:26–35:25 · Guest teaching 4/10 The Future of Generative AI: Autonomous Workflows and Super Apps Matt prompts May on upcoming industry trends. May rejects the incremental 'productivity' framing, arguing that autonomous AI workflows will deliver 300% to 500% capacity expansions.1:34–6:54 · Guest disagreement 0/10 What Writer Does & Enterprise Value Proposition Matt demonstrates clear preparation by citing Writer's rumored $200M round at a $1.9B valuation, their previous Series B, and May's former company Cordoba. May provides a history of how they evolved from translation ML to transformer-based writing assistants.6:54–10:44 · Guest disagreement 1/10 Building the Full-Stack AI Platform Architecture Matt directly raises the common criticism that early AI apps were thin wrappers on third-party LLMs. May clarifies that except for a brief three-month test with GPT-3, Writer has always built its own models down to the base LLM layer.10:44–18:05 · Guest disagreement 1/10 Deep Dive: Graph-Based RAG vs. Vector Databases Matt asks May to compare graph-based RAG against traditional vector database approaches. May delivers a detailed technical breakdown explaining why vector chunking fails on nested tables and numerical data compared to flat JSON triples in Postgres with fusion-in-decoder techniques.18:05–20:22 · Guest disagreement 0/10 AI Guardrails and Enterprise Governance Matt asks how Writer handles enterprise guardrails and compliance. May explains that simple regex filters fail in complex compliance environments, requiring fine-tuned LLM rewrites for PII and brand tone.20:22–23:16 · Guest disagreement 0/10 Writer AI Studio: Custom Enterprise Application Development Matt checks whether AI Studio functions as the primary developer environment for custom apps. May details how complex enterprise logic requires extensive scaffolding beyond simple prompting.23:16–27:18 · Guest disagreement 1/10 In-House Models: Palmyra X and Synthetic Data Efficiency Matt notes that Palmyra X 004 reportedly cost only $700,000 in GPU compute to train, contrasting it with typical mega-model training budgets. May explains how synthetic data designed specifically for LLM consumption drastically improves training efficiency.27:18–31:26 · Guest disagreement 1/10 Navigating Enterprise AI Adoption & Go-To-Market Strategy Matt asks about the adoption curve and enterprise sales strategy. May describes navigating fortune 50 decision-makers and competing against hyperscalers with dedicated vertical solutions.31:26–35:25 · Guest disagreement 1/10 The Future of Generative AI: Autonomous Workflows and Super Apps Matt prompts May on upcoming industry trends. May rejects the incremental 'productivity' framing, arguing that autonomous AI workflows will deliver 300% to 500% capacity expansions.1:34–6:54 · Matt pushing back 0/10 What Writer Does & Enterprise Value Proposition Matt demonstrates clear preparation by citing Writer's rumored $200M round at a $1.9B valuation, their previous Series B, and May's former company Cordoba. May provides a history of how they evolved from translation ML to transformer-based writing assistants.6:54–10:44 · Matt pushing back 3/10 Building the Full-Stack AI Platform Architecture Matt directly raises the common criticism that early AI apps were thin wrappers on third-party LLMs. May clarifies that except for a brief three-month test with GPT-3, Writer has always built its own models down to the base LLM layer.10:44–18:05 · Matt pushing back 1/10 Deep Dive: Graph-Based RAG vs. Vector Databases Matt asks May to compare graph-based RAG against traditional vector database approaches. May delivers a detailed technical breakdown explaining why vector chunking fails on nested tables and numerical data compared to flat JSON triples in Postgres with fusion-in-decoder techniques.18:05–20:22 · Matt pushing back 0/10 AI Guardrails and Enterprise Governance Matt asks how Writer handles enterprise guardrails and compliance. May explains that simple regex filters fail in complex compliance environments, requiring fine-tuned LLM rewrites for PII and brand tone.20:22–23:16 · Matt pushing back 0/10 Writer AI Studio: Custom Enterprise Application Development Matt checks whether AI Studio functions as the primary developer environment for custom apps. May details how complex enterprise logic requires extensive scaffolding beyond simple prompting.23:16–27:18 · Matt pushing back 2/10 In-House Models: Palmyra X and Synthetic Data Efficiency Matt notes that Palmyra X 004 reportedly cost only $700,000 in GPU compute to train, contrasting it with typical mega-model training budgets. May explains how synthetic data designed specifically for LLM consumption drastically improves training efficiency.27:18–31:26 · Matt pushing back 1/10 Navigating Enterprise AI Adoption & Go-To-Market Strategy Matt asks about the adoption curve and enterprise sales strategy. May describes navigating fortune 50 decision-makers and competing against hyperscalers with dedicated vertical solutions.31:26–35:25 · Matt pushing back 0/10 The Future of Generative AI: Autonomous Workflows and Super Apps Matt prompts May on upcoming industry trends. May rejects the incremental 'productivity' framing, arguing that autonomous AI workflows will deliver 300% to 500% capacity expansions.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 59% · guest 41%0:00 · Matt 59% · guest 41%3:00 · Matt 3.7% · guest 96.3%3:00 · Matt 3.7% · guest 96.3%6:00 · Matt 20.6% · guest 79.4%6:00 · Matt 20.6% · guest 79.4%9:00 · Matt 11.6% · guest 88.4%9:00 · Matt 11.6% · guest 88.4%12:00 · Matt 6% · guest 94%12:00 · Matt 6% · guest 94%15:00 · Matt 0.2% · guest 99.8%15:00 · Matt 0.2% · guest 99.8%18:00 · Matt 6.4% · guest 93.6%18:00 · Matt 6.4% · guest 93.6%21:00 · Matt 15.7% · guest 84.3%21:00 · Matt 15.7% · guest 84.3%24:00 · Matt 0% · guest 100%24:00 · Matt 0% · guest 100%27:00 · Matt 15.7% · guest 84.3%27:00 · Matt 15.7% · guest 84.3%30:00 · Matt 7.3% · guest 92.7%30:00 · Matt 7.3% · guest 92.7%33:00 · Matt 17.7% · guest 82.3%33:00 · Matt 17.7% · guest 82.3%
Sharpest disagreement ▶ 31:39 Pushing back on industry terminology

May explicitly rejects the popular hype term 'agentic', pointing out that millions of humans hold agent job titles and reframing the capability as autonomous action.

 Hardest push from Matt ▶ 6:54 Questioning thin wrapper origins

Matt directly challenges Writer's technical differentiation by asking whether they originated as a thin wrapper on top of third-party models.

 Biggest teaching moment ▶ 14:39 Masterclass on graph vs vector RAG

May provides a deep architectural critique showing how vector chunking destroys context in enterprise documents containing nested tables and numerical data.

 Matt holds his own ▶ 23:16 Citing specific Palmyra X training figures

Matt demonstrates high domain familiarity by bringing up the exact Palmyra X 004 model name and questioning its low $700k training figure against mainstream foundation model costs.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
What Writer Does & Enterprise Value Proposition 5200 Matt demonstrates clear preparation by citing Writer's rumored $200M round at a $1.9B valuation, their previous Series B, and May's former company Cordoba. May provides a history of how they evolved from translation ML to transformer-based writing assistants.
Building the Full-Stack AI Platform Architecture 4313 Matt directly raises the common criticism that early AI apps were thin wrappers on third-party LLMs. May clarifies that except for a brief three-month test with GPT-3, Writer has always built its own models down to the base LLM layer.
Deep Dive: Graph-Based RAG vs. Vector Databases 4711 Matt asks May to compare graph-based RAG against traditional vector database approaches. May delivers a detailed technical breakdown explaining why vector chunking fails on nested tables and numerical data compared to flat JSON triples in Postgres with fusion-in-decoder techniques.
AI Guardrails and Enterprise Governance 2400 Matt asks how Writer handles enterprise guardrails and compliance. May explains that simple regex filters fail in complex compliance environments, requiring fine-tuned LLM rewrites for PII and brand tone.
Writer AI Studio: Custom Enterprise Application Development 1300 Matt checks whether AI Studio functions as the primary developer environment for custom apps. May details how complex enterprise logic requires extensive scaffolding beyond simple prompting.
In-House Models: Palmyra X and Synthetic Data Efficiency 5512 Matt notes that Palmyra X 004 reportedly cost only $700,000 in GPU compute to train, contrasting it with typical mega-model training budgets. May explains how synthetic data designed specifically for LLM consumption drastically improves training efficiency.
Navigating Enterprise AI Adoption & Go-To-Market Strategy 3411 Matt asks about the adoption curve and enterprise sales strategy. May describes navigating fortune 50 decision-makers and competing against hyperscalers with dedicated vertical solutions.
The Future of Generative AI: Autonomous Workflows and Super Apps 2410 Matt prompts May on upcoming industry trends. May rejects the incremental 'productivity' framing, arguing that autonomous AI workflows will deliver 300% to 500% capacity expansions.

Statements from this episode (12)

Assertion Not checkable as stated
A customer rebuilt Habib's previous startup Qordoba over a single weekend
“We had a customer basically in a weekend rebuild Cordoba using Writer.”
May Habib Nov 21, 2024 ▶ 6:20
Disclosure
Writer relied on GPT-3 for only a brief three-month period
“So we, we've always had our own technology down to the LLM. There was a very brief period, literally like three months, where we, for some of our apps, used GPT-III because it was just so much better.”
May Habib Nov 21, 2024 ▶ 7:22
Disclosure
Writer stores its knowledge graph as flat JSON in Postgres
“I mean, it is a flat JSON file. This is stored in a Postgres database. It's not even, A graph database.”
May Habib Nov 21, 2024 ▶ 13:12
Insight
Vector RAG struggles with nested tables and overlapping context, Habib explains
“When you are doing a vector-based approach, you're going through, and you are, you've chunked Right, that policy, and, you know, against the prompt, you are trying to find the most relevant chunks, right, in that policy, which means it does really badly, right…”
May Habib Nov 21, 2024 ▶ 15:28
Assertion Contradicted
Vector RAG requires full re-indexing for single-file updates, Habib claims
“And when that policy, even one policy gets updated, you're throwing away the entire embedding store and starting over.”
May Habib Nov 21, 2024 ▶ 16:12
Assertion Not checkable as stated
L'Oréal had built around 180 applications using Writer by late 2023
“This was L'Oreal about a year ago, you know, literally like a 180 apps, and we needed help, right?”
May Habib Nov 21, 2024 ▶ 21:47
Disclosure
Writer trained its Palmyra X 004 model using just $700k in compute
“Just on GPUs, but yeah.”
May Habib Nov 21, 2024 ▶ 23:31
Disclosure
Writer provides 1B randomized tokens for customers to audit copyright compliance
“We give folks, this is another reason we build our own models, a billion randomized tokens that they can query themselves to validate that we use copyright free information and that, you know, our bias and toxicity distribution is what we say it is”
May Habib Nov 21, 2024 ▶ 25:58
Prediction Not checkable as stated
Habib predicts AI will be able to run companies within five years
“This is AI that in three to five years is going to be able to run companies.”
May Habib Nov 21, 2024 ▶ 26:52
Assertion Not checkable as stated
A Fortune 50 CIO personally demoed an eight-person AI startup
“I mean, literally, I was talking to a CIO yesterday. Fortune 50 had just taken personally a demo from an eight-person startup, because people are so curious.”
May Habib Nov 21, 2024 ▶ 28:05
Assertion Not checkable as stated
Hyperscalers spend more on single enterprises than AI startups raise total
“The consultancies and the strategic advisory firms and every hyperscaler can literally throw more innovation dollars at the enterprise than you will raise in three years from venture capital, right?”
May Habib Nov 21, 2024 ▶ 28:28
Insight
Enterprise AI drives 500% capacity shifts, not just 10% productivity gains
“None of this is about productivity, right? You being able to go through Three decades of consumer research on Listerine to be able to develop a new flavor, right? You being able to develop all of the go to market around that new product in literally a third of…”
May Habib Nov 21, 2024 ▶ 33:56
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more