Duboe: Quantifiable success criteria must be set upfront to avoid revisionist history
Mike Duboe · Mike Duboe: Top 5 Lessons Scaling Stitch Fix to IPO; Why CAC/LTV is a BS Metric | 20VC #978 · 20VC with Harry Stebbings · Feb 16, 2023 · at 0:00
Mike Duboe, Partner at Greylock and former Head of Growth at Stitch Fix, outlines the framework used to structure growth experiments and maintain analytical honesty.
0:00 / 0:26exact quote · 26.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Anyone could submit an experiment idea, even if you're not on the growth team, but we built this very simple kind of one pager framework that included an objective hypothesis you're testing design or resources required to execute the experiment timeline. So how long you need the experiment to run and then what success quantifiably looks like. So you're not actually going and having revisionist history. I'm like, Hey, it actually did work even though, you know, the data showed this. And then finally, the next steps that one takes, if the experiment fails or succeeds.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Mike Duboe
Duboe: Direct response attribution models offer false precision compared to holdout testing
“Many growth marketers aim for this, like almost a false sense of precision to give credit to a conversion on any one channel or any fractional set of channels. Oftentimes that's just false precision. And so even though we were highly analytical at stitch fix w…”
Duboe: Startups must never hire a growth lead before finding product-market fit
“One principle that for sure is true is like never hire a growth person, pre-product market fit.”
Duboe: Turning off paid acquisition is the ultimate product-market fit test
“One measure of product market fit health is what happens when you turn off all non-organic acquisition activities. And so ideally you should see some ongoing user base that's retained and ideally generating new users, even if at a much slower clip, right? If y…”
Duboe: Paid marketing channel performance always degrades as customer acquisition scales
“The thing that many people miss early on is that paid degrades with scale. Yeah. Everyone says like, you know, you'll get better at it. So you'll fight kind of the laws of gravity that might happen in a short term, but on a long-term time horizon, performance …”
Duboe: Subscription businesses need one-year CAC payback; e-commerce needs first-transaction payback
“Most, most consumer companies, if you're not paying back, like if you're a subscription business, you're not paying back within a year, that's probably problematic. Ideally if you're e-commerce, you're paying it back on first transaction”