Sep 5, 2024 · 1h 44m · lennys-podcast
The original growth hacker reveals his secrets | Sean Ellis (author of “Hacking Growth”)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Growth pioneer Sean Ellis joins Lenny Rachitsky to unpack the mechanics of measuring product-market fit, establishing the famous 40% benchmark, and building sustainable, full-funnel growth engines. Through detailed case studies from Dropbox, LogMeIn, and Lookout, Ellis outlines practical frameworks for customer activation, onboarding optimization, and cross-functional experimentation.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Lenny holds 24.7% of the talking time here. How this is scored →
Lenny as informed peer 4.7 Guest teaching 5.2 Guest disagreement 1.4 Lenny pushing back 0.8
speaking balance: gold is Lenny, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 46:10 Rejecting growth hacking as quick hacks
Sean firmly interjects to reject Lenny's characterization of growth hacking as a collection of short-term hacks, defending his original holistic definition.
Hardest push from Lenny ▶ 1:29:05 Lenny defends detailed prioritization over simple ICELenny pushes back against Sean's dismissal of expanded prioritization frameworks by highlighting scenarios where detailed Reach estimation prevents severe misallocation of engineering effort.
Biggest teaching moment ▶ 6:40 Leading indicator vs lagging retention cohortsSean educates Lenny on why the 40% survey is strictly a leading directional proxy for early PMF rather than a definitive replacement for lagging cohort retention data.
Lenny holds their own ▶ 1:22:56 Lenny shares Airbnb's North Star metric methodologyLenny demonstrates deep growth domain expertise by contextualizing Airbnb's nights-booked metric against marketplace dynamics and infrequent transaction cadences.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Lenny as informed peer | Guest teaching | Guest disagreement | Lenny pushing back | Why |
|---|---|---|---|---|---|---|
| Unpacking the Sean Ellis PMF Test | 4 | 5 | 1 | 0 | Lenny opens by framing the importance of the PMF survey and invites Sean to break down the core question. Sean explains the mechanics of asking how disappointed users would be if the product disappeared in a purely collaborative, instructional tone. | |
| The 40% PMF Benchmark and Lookout Case Study | 4 | 6 | 2 | 1 | Sean reframes Lenny's summary by clarifying that the 40% threshold is merely a leading indicator while retention cohorts are the ultimate lagging test. He then delivers a detailed breakdown of turning Lookout from 7% to 40% by refocusing onboarding and messaging around antivirus protection. | |
| Extracting Customer Insights and the Xobni Case Study | 5 | 5 | 1 | 1 | Lenny synthesizes Sean's turnaround strategy and prompts for actionable steps once a startup hits 40%. Sean explains qualitative drilling using open-ended questions followed by multiple-choice tests, citing his Xobni email discovery. | |
| Survey Timing, Switching Costs, and Webs.com Case Study | 4 | 5 | 1 | 1 | Lenny asks about false positives, prompting Sean to detail the Webs.com case study where high scores reflected high switching costs and customer investment rather than superior utility. Sean outlines specific criteria for sampling active, activated users. | |
| Limitations, Cultural Nuances, and Origin of the Survey | 5 | 5 | 1 | 1 | Lenny brings in audience questions from Shreyas Doshi and Nubank's cultural nuances to probe limitations. Sean candidly explains the survey's origin story as an attempt to get honest answers from notoriously hard-to-please senior managers at Xobni. | |
| Navigating Early Adopters, Sample Sizes, and Target Niches | 4 | 5 | 1 | 1 | Lenny explores score durability and sample sizing. Sean breaks down how early adopter motivation shifts from curiosity to utility over time, and shares his heuristic of preferring passionate niche user bases over broad but lukewarm audiences. | |
| Survey Tooling and Superhuman's Methodological Twist | 5 | 6 | 1 | 1 | Sean discusses tooling and explains Superhuman's twist on his original methodology. While Sean historically advised ignoring 'somewhat disappointed' users to avoid diluting the core product, Superhuman analyzed which features could convert those users who shared the core benefit without compromising must-have users. | |
| Defining Growth Hacking and the LogMeIn Activation Turnaround | 4 | 6 | 3 | 1 | Sean immediately interjects to correct Lenny's comment implying growth hacking was originally conceived as one-off hacks, clarifying it was always about holistic sustainable growth. Sean then illustrates the immense power of activation through LogMeIn's 10x funnel fix. | |
| Sponsor Break: Merge | 4 | 5 | 1 | 0 | Following the sponsor break, Sean explains the download-rate dropoff at LogMeIn and how addressing user skepticism about freemium pricing unlocked a 300% conversion boost. Sean emphasizes designing activation around tangible value delivery. | |
| Growth Channels and the Dropbox Referral Story | 6 | 5 | 2 | 1 | Lenny outlines a four-engine growth taxonomy and compares it to historical examples like Yelp and DoorDash. Sean reframes growth into demand generation versus harvesting and recounts architecting Dropbox's two-sided referral engine with Albert Ni. | |
| Freemium Strategy, Engagement Cadence, and North Star Metrics | 6 | 5 | 1 | 1 | Sean and Lenny analyze freemium mechanics and North Star metrics, comparing Facebook's shift to DAU with Amazon's monthly purchases and Airbnb's nights booked. Lenny contributes his direct operator knowledge from Airbnb while Sean explains why revenue should never be a North Star. | |
| Evolution of Growth, ICE vs. RICE, and AI in Experimentation | 6 | 5 | 2 | 2 | Andrew Chen's question prompts a discussion on cross-functional growth friction. When discussing ICE versus RICE, Sean defends his model by arguing that Reach is inherently captured in Impact, while Lenny offers a balanced counter-perspective on when detailed reach estimation (D-RICE) prevents costly mistakes. | |
| The Power of Asking the Right Questions | 4 | 4 | 1 | 0 | Sean closes the main discussion by reflecting on the importance of framing the right qualitative questions before jumping into technical solutions. Lenny validates the insight before transitioning to the lightning round. |
Statements from this episode (26)
Product-market fit is defined by whether a product is a must-have
“It's a simple question that helps you figure out, you know, does anyone consider your product a must have or, you know, ideally who and how many people consider it, but ultimately it's about trying to figure out, you know, do, is your product a must have, whic…”
PMF survey score is a leading indicator, whereas retention is lagging
“I would say it's a leading indicator of product market fit. The lagging indicator is, do they actually keep using it? So probably retention cohorts are more, more accurate.”
Lookout raised its PMF score from 7% to 40% in two weeks
“And so the next cohort of people that we surveyed were at 40% saying they'd be very disappointed without the product. So that was really took two weeks to make those changes. Six months later, it was 60% on, on the Score. And then I think they hit the billion …”
Ellis: Improving retention depends far more on onboarding than tactical fixes
“Moving retention is really hard, but it's usually much more a function of onboarding to the right user experience than it is about the kind of the tactical things that people try to do to improve retention.”
Ellis: The 40% PMF benchmark is an alignment tool, not a rigid rule
“I don't think it's that firm. You know, to me, I think the real power is having some kind of target for the team to be shooting for that basically says, we're not going to aggressively start to grow until we hit this target.”
Webs.com scored around 90% on Sean Ellis's PMF survey
“And so I didn't have high hopes when I ran the survey, but it came back with one of the highest scores I'd ever seen. And it was like, 90% of the people saying they'd be very disappointed and they could no longer use the product.”
Ellis: PMF survey scores reflect both product utility and switching costs
“So sort of switching costs, I think can factor in there. So it's a function of both switching costs and utility of the product.”
Survey active users who used the product at least twice recently
“What I recommend is a random sample of people who've really use your product. So they, they've gone in, they didn't just sign up, but they went in and hopefully like hit that activation moment. They've used it twice at, you know, two plus times and they've ide…”
Ellis: Product-market fit survey does not work for one-off products
“I think one-off products are probably not, not good products to run the question on.”
Ellis: Customer acquisition should be the final step in growth
“Like customer acquisition is so hard that if you, if you're not really efficient at converting and retaining and monetizing people, you're gonna really struggle on the customer acquisition side.”
Ellis: Dropbox beta users flipped from 90% tech enthusiasts to utility seekers
“Through the six months I was there, I would ask, I'd ask one question, like multiple times a month to, I broke kind of the early beta users into a bunch of different lists, and I'd ask, which best describes you? I like to be among the first to try cool new tec…”
Ellis: Starting with a deeply passionate niche improves long-term startup survival
“I prefer kind of a more passionate Customer base that's, and work from there just because I think your biggest competition when you're really innovating is, is just like being irrelevant. And so if you're like deeply relevant to anyone, I think that gives you …”
Nubank requires a 50% PMF threshold before launching new products
“Every new product at new bank they build before they launch it, they wait for a 50% threshold for people to say, 50% of people would be disappointed if this product did not exist as they're developing it. And only then do they launch it publicly.”
Ellis: Modifying products for lukewarm users risks diluting core user value
“If you start paying attention to what your somewhat disappointed users are telling you, and then you start tweaking onboarding and product based on their feedback, maybe you're going to dilute it for your must have users. And that ultimately it becomes kind of…”
LogMeIn scaled marketing from $10k to $1M monthly after fixing activation
“And so in three months, we improved the signup to usage rate by a thousand percent. So we went from only five percent of people using the product to 50%. I went back, tried the exact same channels that previously only scaled to 10,000 dollars a month. Now they…”
LogMeIn lifted downloads 300% by offering paid trials alongside free tiers
“Our next test, once we articulated what the problem was, our next test gave us a 300% improvement in the download rate, which was a, we gave them a choice, download a trial of the paid version or download the free version, put the graphical check mark next to …”
Ellis: LogMeIn grew almost entirely from paid search
“LogMeIn, we grew
Almost entirely off of paid search.”
Ellis: 80% of new LogMeIn users came through word-of-mouth
“80% of our new users were coming in through word of mouth and
You know, I had a hundred million devices connected in on our system.”
Referral programs only accelerate existing word-of-mouth; they cannot fix unshared products
“To me, I think it's a great accelerant when it's already working, but it can't fix it if people don't want to talk about your product.”
Ellis: Successful freemium requires two distinct, great products
“Freemium towards having a free and a premium version of your product to really work in any business, it needs to be that your free product is so good that people naturally have word of mouth around that product. And then to be economically viable, you have to …”
Ellis: North Star metrics should correlate with revenue, not be revenue
“It should correlate to revenue growth, but not necessarily like rev revenue. Shouldn't be the North star metric, but you know, as you grow value across your customer base, you should be able to grow revenue at the at the same rate.”
Airbnb used nights booked, not revenue, as its North Star metric
“At Airbnb was, our North star metric was Knights book. And so it's similar to Amazon. It's not like the money Airbnb made from bookings, but it's like Knights booked. And it was really, and basically every experiment we ran is like, is this increasing nights b…”
Sean Ellis argues Reach in the RICE framework is an unnecessary addition
“I think it's an unnecessary addition, but maybe it's, I'm just being protective of my original idea. That, that the I in ICE is impact. And it's essentially saying, best case scenario, how much impact could we get from this? And reach is a super important part…”
Sean Ellis: AI will model experiment outcomes and could replace ICE
“What I think is going to be really interesting is that over time, I think AI is going to actually change our ability to model out potential outcomes on experiments and start to Whether, whether it's a more informed way of doing ice or replaces ice that, that u…”
Sean Ellis uses ChatGPT to draft answers to advice questions as himself
“Almost every question that I get, I go to chat GPT and say, how would Sean Ellis answer this? And it gives me an initial draft to like make a couple of tweaks and definitely that allows me to answer a lot more.”
TikTok's founding growth team based their early tactics on Sean Ellis's writing
“I met with the original founding growth team at TikTok. They're based in Singapore and they had, I can't remember what the previous product was called, but they started with the previous product. And then when TikTok came, they were in place to be the initial …”