Dec 29, 2024 · 1h 21m · lennys-podcast
Why great AI products are all about the data | Shaun Clowes (CPO at Confluent)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Confluent Chief Product Officer Shaun Clowes joins Lenny Rachitsky to discuss how product managers can generate 100x leverage, why enterprise SaaS moats endure in the age of AI, and how high-performing product organizations combine product-led growth, data fluency, and disciplined strategic decision-making.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Lenny holds 22.8% of the talking time here. How this is scored →
Lenny as informed peer 3.6 Guest teaching 3.4 Guest disagreement 0.9 Lenny pushing back 0.4
speaking balance: gold is Lenny, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 25:03 Shaun rejects SaaS vulnerability claims as a fantasy
Shaun forcefully dismisses the popular industry premise that generative AI will easily commoditize vertical SaaS incumbents, calling the entire assumption a complete misunderstanding of enterprise business rules.
Hardest push from Lenny ▶ 31:18 Lenny pushes the primacy of distribution advantagesLenny steers Shaun away from pure product mechanics to push the argument that distribution moats become the ultimate decisive factor when product cloning becomes trivial.
Biggest teaching moment ▶ 19:40 Shaun breaks down information decay rate in AIShaun educates listeners and breaks down the core technical reality that foundation models are dumb synthesis engines whose output quality is entirely dictated by real-time context and data decay rates.
Lenny holds their own ▶ 23:51 Lenny validates AI data primacy with Anthropic exampleLenny brings high-level industry intelligence by citing Mike Krieger's insights from Anthropic to independently prove Shaun's thesis on model research versus UX optimization.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Lenny as informed peer | Guest teaching | Guest disagreement | Lenny pushing back | Why |
|---|---|---|---|---|---|---|
| Episode Preview and Core Themes | 0 | 0 | 0 | 0 | Lenny provides an introductory preview of the episode themes and introduces Shaun Clowes' background. Because this is a scripted intro monologue with guest teaser clips, no live interaction scores apply. | |
| Sponsor Messages: Interpret and BuildBetter.ai | 0 | 0 | 0 | 0 | Host sponsor read and podcast introduction. There is no conversational interaction between host and guest. | |
| The Problem with Product Management and 10x PM Leverage | 4 | 3 | 1 | 0 | Shaun outlines why product management remains relatively undeveloped and explains how high-leverage PMs create outsized returns. Lenny validates Shaun's perspective with shared agreement about anti-PM sentiment. | |
| Escaping the Delivery Trap by Looking Outside the Building | 4 | 4 | 2 | 1 | Shaun explains why PMs spend too much time inside the building on execution rather than external customer reality. Lenny gently challenges whether PMs might mistakenly think they already talk to customers enough, prompting Shaun to explain confirmation bias. | |
| Using LLMs to Analyze Customer Research and Strategy | 3 | 5 | 1 | 0 | Shaun details tactical methods for using LLMs to interrogate user interview transcripts and identify blind spots in strategy. Lenny asks clarifying follow-ups regarding tooling choices and internal setups. | |
| Why AI Product Success Hinges on Data Management | 6 | 5 | 2 | 0 | Shaun explains why data recency and context pipelines matter far more than underlying LLM foundation models. Lenny strongly demonstrates his own expertise by drawing a direct comparison to Mike Krieger's insights from Anthropic. | |
| Defending SaaS Moats: Why AI Won't Easily Clone Incumbents | 3 | 6 | 3 | 1 | Shaun dismisses the popular narrative that AI will easily commoditize SaaS giants, explaining that deep custom business logic and workflows are the real moats. Lenny sets up the counter-narrative of AI software cloning for Shaun to dismantle. | |
| Distribution Challenges and Modern Data-Integrated Applications | 5 | 4 | 1 | 2 | Lenny pushes the conversation toward distribution advantages, arguing that product copying forces companies to win on go-to-market. Shaun agrees and explores how modern tools like Ashby integrate live data directly into user workflows. | |
| Rethinking Data-Driven PMing: Compass vs. GPS | 6 | 4 | 1 | 1 | Shaun describes data as a compass rather than a GPS and cautions against blind data-driven dogma. Lenny adds a strong domain insight citing Shopify's long-term holdout experiment results, which Shaun compares to his Atlassian experience. | |
| Sponsor Message: Wix Studio | 5 | 4 | 1 | 0 | Following an ad read, Shaun breaks down the evolution of B2B growth and product-led growth at Atlassian. Lenny contributes relevant context on Charlie Munger's incentive principles and corrects Atlassian's current customer scale. | |
| Managing Your Career Like a Bingo Card | 4 | 4 | 0 | 0 | Shaun shares his philosophy of treating career roles like a bingo card to build broad versatility across domains. Lenny shares his own advisory framework for PM career exploration and resume building. | |
| Failure Corner: Learning When to Kill a Zombie Product | 4 | 2 | 0 | 0 | Shaun reflects on a failure where his team maintained a zombie environmental tracking product for two years before finally killing it. Lenny links the takeaway to a relevant guest story about Wiz identifying strategic clarity early. | |
| Essential PM Principles: Calendar Control and Decision Velocity | 3 | 3 | 0 | 0 | Shaun shares final parting PM heuristics around calendar defense and avoiding analysis paralysis through the 30/70 decision rule before moving toward the wrap-up. |
Statements from this episode (22)
Clowes: 10x product managers produce 100x return via leverage
“It must be true that if there's 10 times engineers, and I would argue there definitely are, there must be 10 times product managers too. But at the same time, those 10 times product managers, because product management is ultimately about leverage, so it's abo…”
Clowes: A product manager's job is rejecting 90% of incoming requests
“Realistically, when you're a product manager, your job is to say no to 90% of things that get brought your way. And so that kind of makes you the bad person pretty much from the start. And so you're saying no to 90%, so you can say yes to 10%.”
Clowes: Most PMs talk to familiar users and ignore disconfirming evidence
“Most of the time people go talk to the people they always talk to and they learn nothing particularly new. They don't synthesize the results that they got from the conversation. They don't seek out the counterfactual. They don't seek out the proof that they're…”
Clowes: User research reaches diminishing returns between 7 and 14 interviews
“The idea is that once you interview between seven and 14 people, you stop learning new things. Less than seven, you don't learn enough. More than 14, you stop learning anything new.”
Clowes: Off-the-shelf LLMs are sufficient for user research analysis
“Mostly I find that like the straight up LLMs themselves are good enough.”
Confluent uses LLMs to cluster and summarize inbound customer requests
“At Conform, we get a ton of inbound customer requests, as you can imagine. Coming from the field or directly from customers, we use LLMs to take in those asks to summarize what they're about, to find other asks that are like that one, like really in a compelli…”
Clowes: AI product quality depends on data context, not replaceable models
“What will make my AI experience really great? It's definitely not going to be the models because like these models are mostly going to be somewhat replaceable and you could say, okay, well, is it going to be the prompts? Maybe, but you know, so many good promp…”
Clowes: 90% of AI development effort goes into data management
“It's a data management problem. Like, it's getting access to good data, getting access to high quality data, getting access to timely data, and getting it to the LLM to get the LLM to make a smart decision. That's where 90% of the calories go.”
Clowes: Business rules, not UIs, drive enterprise SaaS lock-in
“The real thing, which is staring everybody in the face, is it's all about the business rules. Like, that is what drives the lock-in.”
Clowes: AI will entrench dominant enterprise SaaS incumbents over startups
“The most likely outcome is that the currently dominant companies are going to get more dominant. But I don't think this idea that it would just cause a spring up of a whole bunch of new, new apps that will more easily challenge the incumbents makes any particu…”
Clowes: AI makes startup distribution harder by flooding channels with spam
“In the world of AI, it seems like distribution is more likely to get hard than easy. So, so like you know, if you think about, for example, diminishing returns on cold email, because cold email is getting easier and easier to send even worse spam. Like it soun…”
Clowes: Ashby succeeds by embedding recruiting outcome data into workflows
“Yeah, it's Ashby. It's the one all the cool kids are talking about now. And it's funny because like people literally talk about it in comparison to all the, even the, you know, the last generation of modern SAS ATSs or whatever And they talk about it in glowin…”
Clowes: Data acts as a compass rather than a GPS for decisions
“Data is more like a compass than a GPS. Right? Like if you look at data as a way of like giving you the answer, you're always wrong. You're always wrong or you're slow. Wrong or slow or sometimes both. Because mostly data doesn't give you the answer. It just t…”
Clowes: When data clashes with intuition, the data is usually wrong
“If you're looking at a piece of data and the result tells you something that your intuition tells you is like insanely wrong, like that you're probably not right. First believe your intuition and go and prove yourself right. Like don't just take it at first gl…”
Clowes: Atlassian kept 10% of users in a permanent global experiment holdout
“At Adassian, we did something similar. We had a global holdout group, actually, that was held out of all experiments. There was an interview, experiment platform couldn't target that group at all. So, 10% of all people never saw anything ever.”
Clowes: Great distribution and customer success can sell almost any software
“Almost all ills can be, you know, filled in by really great distribution. Like, if you have a really good ground game, really good marketing, a really good ground game, and kind of jamming your product into the channel, like you're jamming your product in fron…”
Clowes: Companies will ignore end-user experience without dedicated incentives
“If there is nobody in the organization whose true incentive Is to measure that success, the end user success, their enjoyment, their happiness, their retention, their engagement early on, it will not happen. Or at best it will be a hobby”
Clowes: Atlassian reached 80,000 customers without any direct sales force
“When I joined Atlassian, and like I shared with you, we had no Salesforce at all, actually at all. Literally nobody to sell the software, it sold itself or it didn't get sold at all. And we grew to have 80,000 customers.”
Clowes: Top professionals are scribble-shaped rather than T-shaped
“It's interesting because people used to talk about people who are T-shaped or whatever. I've never really loved the analogy, because it's more like people are scribble shaped, like, I mean, like, there's the really best people you've worked with, they're more …”
Clowes: Team killed failing product after customer offered multimillion-dollar contract
“The final straw was actually when a customer finally wanted to pay for it. Like it had been a market for two years and we found ourselves with a customer who wanted to pay millions of dollars for it. They were ready to sign on the dotted line. And that was act…”
Clowes: PMs cannot succeed if they let calendars dictate their time
“If you let your calendar rule you, then nothing good will happen. Like, you know, I know people talk about that a lot, but it's surprisingly common in product management in particular, that people end up ruled by their calendar. And so it's related to that who…”