Aug 4, 2024 · 1h 14m · lennys-podcast

Lessons from scaling Uber and Opendoor | Brian Tolkin (Head of Product at Opendoor, ex-Uber)

Brian Tolkin · 48m spoken Lenny Rachitsky · 19m spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

In this episode, Brian Tolkin (Head of Product and Design at Opendoor, ex-Uber) shares operational frameworks, product discovery strategies, and leadership lessons for scaling technology businesses that bridge software and real-world operations.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Lenny holds 28.8% of the talking time here. How this is scored →

Lenny as informed peer 3.8 Guest teaching 5.1 Guest disagreement 0.4 Lenny pushing back 0.4
05100:0015:0030:0045:001:00:001:57–6:25 · Lenny as informed peer 3/10 Transitioning from Operations to Product Leadership Lenny opens with friendly rapport and introduces Brian's unusual career path from operations to Head of Product. Brian details how grounding product instincts in direct customer support and onboarding creates operational leverage.6:27–10:00 · Lenny as informed peer 4/10 Bridging Product and Ops: The Genesis of Product Operations Lenny asks about product teams undervaluing ops, and Brian articulates the twin-turbine jet engine analogy. Brian also clarifies that while he formalized product operations at Uber, predecessors at Google had already explored similar functions.10:01–13:35 · Lenny as informed peer 4/10 Early Uber Mechanics: Manual Surge Pricing and Naming UberX Lenny inquires about early Uber pricing myths, and Brian explains how surge pricing was initially a manual, human-in-the-loop system controlled by local GMs rather than an automated algorithm.13:36–16:50 · Lenny as informed peer 3/10 High-Stakes Launches: UberPool in China and Opendoor's COVID Pivot Brian shares the high-stress launch story of UberPool in Chengdu and Opendoor's rapid pivot to virtualizing home inspections during the initial COVID lockdowns.16:51–20:51 · Lenny as informed peer 3/10 Opendoor Overview and Core Product-Ops Lessons Brian breaks down the mechanics of Opendoor's core value proposition and shares architectural lessons on building software that accounts for real-world entropy.20:52–25:35 · Lenny as informed peer 5/10 Scaling from Manual Operations to Software Automation Lenny brings up Casey Winters' hot take that operations is a sign of inefficiency. Brian agrees in principle but reframes ops as the leading edge for experimentation before software automation takes over.25:36–30:57 · Lenny as informed peer 4/10 Frameworks for High-Impact Product Reviews Brian shares best practices for conducting constructive product reviews, emphasizing tight attendee lists, psychological safety, and clear separation between mandate and suggestion.30:59–40:01 · Lenny as informed peer 5/10 Sponsor Break: Attio CRM After an ad break, Lenny explores Jobs to be Done methodology. Brian explains how Opendoor avoids dogmatic template-filling and instead uses JTBD to understand rare, high-stakes home transactions.40:02–47:04 · Lenny as informed peer 4/10 Experimentation and Intuition in Low-Volume Environments Brian lays out statistical strategies for low-volume experimentation when traditional AB testing lacks sample power, emphasizing alternative statistical techniques and disciplined intuition.47:07–52:53 · Lenny as informed peer 4/10 Navigating Competition: Zillow's iBuying Venture and Eventual Partnership Brian explains the operational complexity of vertically integrated iBuying that ultimately led Zillow to shutter its buying arm and partner with Opendoor instead.52:55–1:00:19 · Lenny as informed peer 3/10 Maintaining Composure and Discerning the 'Kernel of Truth' Brian discusses maintaining emotional composure under pressure and finding the true signal or kernel of truth across noisy feedback channels.1:00:20–1:04:01 · Lenny as informed peer 4/10 Failure Corner: UberPool's Launch Mistakes and Decentralized Experiments Brian recounts early missteps with UberPool's commuter cohort rollout in San Francisco and how heavy promotion helped discover that pure geographic liquidity was the key metric.1:04:02–1:06:10 · Lenny as informed peer 4/10 Matching Product Manager Archetypes to Problem Types Brian shares his perspective on PM archetypes, noting that hiring should focus on matching specific PM problem-solving profiles to the domain's unique constraints rather than defaulting to generic hiring.1:06:11–1:13:29 · Lenny as informed peer 3/10 Lightning Round and Brian's 5-Hour July 4th Uber Interview Brian wraps up with rapid-fire recommendations and tells the story of his intense five-hour July 4th interview at Uber, which included an unexpected working session with Travis Kalanick.1:57–6:25 · Guest teaching 4/10 Transitioning from Operations to Product Leadership Lenny opens with friendly rapport and introduces Brian's unusual career path from operations to Head of Product. Brian details how grounding product instincts in direct customer support and onboarding creates operational leverage.6:27–10:00 · Guest teaching 6/10 Bridging Product and Ops: The Genesis of Product Operations Lenny asks about product teams undervaluing ops, and Brian articulates the twin-turbine jet engine analogy. Brian also clarifies that while he formalized product operations at Uber, predecessors at Google had already explored similar functions.10:01–13:35 · Guest teaching 6/10 Early Uber Mechanics: Manual Surge Pricing and Naming UberX Lenny inquires about early Uber pricing myths, and Brian explains how surge pricing was initially a manual, human-in-the-loop system controlled by local GMs rather than an automated algorithm.13:36–16:50 · Guest teaching 5/10 High-Stakes Launches: UberPool in China and Opendoor's COVID Pivot Brian shares the high-stress launch story of UberPool in Chengdu and Opendoor's rapid pivot to virtualizing home inspections during the initial COVID lockdowns.16:51–20:51 · Guest teaching 5/10 Opendoor Overview and Core Product-Ops Lessons Brian breaks down the mechanics of Opendoor's core value proposition and shares architectural lessons on building software that accounts for real-world entropy.20:52–25:35 · Guest teaching 5/10 Scaling from Manual Operations to Software Automation Lenny brings up Casey Winters' hot take that operations is a sign of inefficiency. Brian agrees in principle but reframes ops as the leading edge for experimentation before software automation takes over.25:36–30:57 · Guest teaching 5/10 Frameworks for High-Impact Product Reviews Brian shares best practices for conducting constructive product reviews, emphasizing tight attendee lists, psychological safety, and clear separation between mandate and suggestion.30:59–40:01 · Guest teaching 5/10 Sponsor Break: Attio CRM After an ad break, Lenny explores Jobs to be Done methodology. Brian explains how Opendoor avoids dogmatic template-filling and instead uses JTBD to understand rare, high-stakes home transactions.40:02–47:04 · Guest teaching 6/10 Experimentation and Intuition in Low-Volume Environments Brian lays out statistical strategies for low-volume experimentation when traditional AB testing lacks sample power, emphasizing alternative statistical techniques and disciplined intuition.47:07–52:53 · Guest teaching 5/10 Navigating Competition: Zillow's iBuying Venture and Eventual Partnership Brian explains the operational complexity of vertically integrated iBuying that ultimately led Zillow to shutter its buying arm and partner with Opendoor instead.52:55–1:00:19 · Guest teaching 5/10 Maintaining Composure and Discerning the 'Kernel of Truth' Brian discusses maintaining emotional composure under pressure and finding the true signal or kernel of truth across noisy feedback channels.1:00:20–1:04:01 · Guest teaching 5/10 Failure Corner: UberPool's Launch Mistakes and Decentralized Experiments Brian recounts early missteps with UberPool's commuter cohort rollout in San Francisco and how heavy promotion helped discover that pure geographic liquidity was the key metric.1:04:02–1:06:10 · Guest teaching 4/10 Matching Product Manager Archetypes to Problem Types Brian shares his perspective on PM archetypes, noting that hiring should focus on matching specific PM problem-solving profiles to the domain's unique constraints rather than defaulting to generic hiring.1:06:11–1:13:29 · Guest teaching 5/10 Lightning Round and Brian's 5-Hour July 4th Uber Interview Brian wraps up with rapid-fire recommendations and tells the story of his intense five-hour July 4th interview at Uber, which included an unexpected working session with Travis Kalanick.1:57–6:25 · Guest disagreement 0/10 Transitioning from Operations to Product Leadership Lenny opens with friendly rapport and introduces Brian's unusual career path from operations to Head of Product. Brian details how grounding product instincts in direct customer support and onboarding creates operational leverage.6:27–10:00 · Guest disagreement 1/10 Bridging Product and Ops: The Genesis of Product Operations Lenny asks about product teams undervaluing ops, and Brian articulates the twin-turbine jet engine analogy. Brian also clarifies that while he formalized product operations at Uber, predecessors at Google had already explored similar functions.10:01–13:35 · Guest disagreement 1/10 Early Uber Mechanics: Manual Surge Pricing and Naming UberX Lenny inquires about early Uber pricing myths, and Brian explains how surge pricing was initially a manual, human-in-the-loop system controlled by local GMs rather than an automated algorithm.13:36–16:50 · Guest disagreement 0/10 High-Stakes Launches: UberPool in China and Opendoor's COVID Pivot Brian shares the high-stress launch story of UberPool in Chengdu and Opendoor's rapid pivot to virtualizing home inspections during the initial COVID lockdowns.16:51–20:51 · Guest disagreement 0/10 Opendoor Overview and Core Product-Ops Lessons Brian breaks down the mechanics of Opendoor's core value proposition and shares architectural lessons on building software that accounts for real-world entropy.20:52–25:35 · Guest disagreement 1/10 Scaling from Manual Operations to Software Automation Lenny brings up Casey Winters' hot take that operations is a sign of inefficiency. Brian agrees in principle but reframes ops as the leading edge for experimentation before software automation takes over.25:36–30:57 · Guest disagreement 0/10 Frameworks for High-Impact Product Reviews Brian shares best practices for conducting constructive product reviews, emphasizing tight attendee lists, psychological safety, and clear separation between mandate and suggestion.30:59–40:01 · Guest disagreement 1/10 Sponsor Break: Attio CRM After an ad break, Lenny explores Jobs to be Done methodology. Brian explains how Opendoor avoids dogmatic template-filling and instead uses JTBD to understand rare, high-stakes home transactions.40:02–47:04 · Guest disagreement 1/10 Experimentation and Intuition in Low-Volume Environments Brian lays out statistical strategies for low-volume experimentation when traditional AB testing lacks sample power, emphasizing alternative statistical techniques and disciplined intuition.47:07–52:53 · Guest disagreement 1/10 Navigating Competition: Zillow's iBuying Venture and Eventual Partnership Brian explains the operational complexity of vertically integrated iBuying that ultimately led Zillow to shutter its buying arm and partner with Opendoor instead.52:55–1:00:19 · Guest disagreement 0/10 Maintaining Composure and Discerning the 'Kernel of Truth' Brian discusses maintaining emotional composure under pressure and finding the true signal or kernel of truth across noisy feedback channels.1:00:20–1:04:01 · Guest disagreement 0/10 Failure Corner: UberPool's Launch Mistakes and Decentralized Experiments Brian recounts early missteps with UberPool's commuter cohort rollout in San Francisco and how heavy promotion helped discover that pure geographic liquidity was the key metric.1:04:02–1:06:10 · Guest disagreement 0/10 Matching Product Manager Archetypes to Problem Types Brian shares his perspective on PM archetypes, noting that hiring should focus on matching specific PM problem-solving profiles to the domain's unique constraints rather than defaulting to generic hiring.1:06:11–1:13:29 · Guest disagreement 0/10 Lightning Round and Brian's 5-Hour July 4th Uber Interview Brian wraps up with rapid-fire recommendations and tells the story of his intense five-hour July 4th interview at Uber, which included an unexpected working session with Travis Kalanick.1:57–6:25 · Lenny pushing back 0/10 Transitioning from Operations to Product Leadership Lenny opens with friendly rapport and introduces Brian's unusual career path from operations to Head of Product. Brian details how grounding product instincts in direct customer support and onboarding creates operational leverage.6:27–10:00 · Lenny pushing back 1/10 Bridging Product and Ops: The Genesis of Product Operations Lenny asks about product teams undervaluing ops, and Brian articulates the twin-turbine jet engine analogy. Brian also clarifies that while he formalized product operations at Uber, predecessors at Google had already explored similar functions.10:01–13:35 · Lenny pushing back 1/10 Early Uber Mechanics: Manual Surge Pricing and Naming UberX Lenny inquires about early Uber pricing myths, and Brian explains how surge pricing was initially a manual, human-in-the-loop system controlled by local GMs rather than an automated algorithm.13:36–16:50 · Lenny pushing back 0/10 High-Stakes Launches: UberPool in China and Opendoor's COVID Pivot Brian shares the high-stress launch story of UberPool in Chengdu and Opendoor's rapid pivot to virtualizing home inspections during the initial COVID lockdowns.16:51–20:51 · Lenny pushing back 0/10 Opendoor Overview and Core Product-Ops Lessons Brian breaks down the mechanics of Opendoor's core value proposition and shares architectural lessons on building software that accounts for real-world entropy.20:52–25:35 · Lenny pushing back 2/10 Scaling from Manual Operations to Software Automation Lenny brings up Casey Winters' hot take that operations is a sign of inefficiency. Brian agrees in principle but reframes ops as the leading edge for experimentation before software automation takes over.25:36–30:57 · Lenny pushing back 0/10 Frameworks for High-Impact Product Reviews Brian shares best practices for conducting constructive product reviews, emphasizing tight attendee lists, psychological safety, and clear separation between mandate and suggestion.30:59–40:01 · Lenny pushing back 1/10 Sponsor Break: Attio CRM After an ad break, Lenny explores Jobs to be Done methodology. Brian explains how Opendoor avoids dogmatic template-filling and instead uses JTBD to understand rare, high-stakes home transactions.40:02–47:04 · Lenny pushing back 1/10 Experimentation and Intuition in Low-Volume Environments Brian lays out statistical strategies for low-volume experimentation when traditional AB testing lacks sample power, emphasizing alternative statistical techniques and disciplined intuition.47:07–52:53 · Lenny pushing back 0/10 Navigating Competition: Zillow's iBuying Venture and Eventual Partnership Brian explains the operational complexity of vertically integrated iBuying that ultimately led Zillow to shutter its buying arm and partner with Opendoor instead.52:55–1:00:19 · Lenny pushing back 0/10 Maintaining Composure and Discerning the 'Kernel of Truth' Brian discusses maintaining emotional composure under pressure and finding the true signal or kernel of truth across noisy feedback channels.1:00:20–1:04:01 · Lenny pushing back 0/10 Failure Corner: UberPool's Launch Mistakes and Decentralized Experiments Brian recounts early missteps with UberPool's commuter cohort rollout in San Francisco and how heavy promotion helped discover that pure geographic liquidity was the key metric.1:04:02–1:06:10 · Lenny pushing back 0/10 Matching Product Manager Archetypes to Problem Types Brian shares his perspective on PM archetypes, noting that hiring should focus on matching specific PM problem-solving profiles to the domain's unique constraints rather than defaulting to generic hiring.1:06:11–1:13:29 · Lenny pushing back 0/10 Lightning Round and Brian's 5-Hour July 4th Uber Interview Brian wraps up with rapid-fire recommendations and tells the story of his intense five-hour July 4th interview at Uber, which included an unexpected working session with Travis Kalanick.

speaking balance: gold is Lenny, purple is the guest (3 minute bins)

0:00 · Lenny 77.8% · guest 22.2%0:00 · Lenny 77.8% · guest 22.2%3:00 · Lenny 71.7% · guest 28.3%3:00 · Lenny 71.7% · guest 28.3%6:00 · Lenny 31.5% · guest 68.5%6:00 · Lenny 31.5% · guest 68.5%9:00 · Lenny 18.8% · guest 81.2%9:00 · Lenny 18.8% · guest 81.2%12:00 · Lenny 13.6% · guest 86.4%12:00 · Lenny 13.6% · guest 86.4%15:00 · Lenny 21.3% · guest 78.7%15:00 · Lenny 21.3% · guest 78.7%18:00 · Lenny 12.5% · guest 87.5%18:00 · Lenny 12.5% · guest 87.5%21:00 · Lenny 18% · guest 82%21:00 · Lenny 18% · guest 82%24:00 · Lenny 28.4% · guest 71.6%24:00 · Lenny 28.4% · guest 71.6%27:00 · Lenny 30% · guest 70%27:00 · Lenny 30% · guest 70%30:00 · Lenny 50.9% · guest 49.1%30:00 · Lenny 50.9% · guest 49.1%33:00 · Lenny 11.6% · guest 88.4%33:00 · Lenny 11.6% · guest 88.4%36:00 · Lenny 26.1% · guest 73.9%36:00 · Lenny 26.1% · guest 73.9%39:00 · Lenny 36.8% · guest 63.2%39:00 · Lenny 36.8% · guest 63.2%42:00 · Lenny 24.9% · guest 75.1%42:00 · Lenny 24.9% · guest 75.1%45:00 · Lenny 21.6% · guest 78.4%45:00 · Lenny 21.6% · guest 78.4%48:00 · Lenny 36.1% · guest 63.9%48:00 · Lenny 36.1% · guest 63.9%51:00 · Lenny 37.1% · guest 62.9%51:00 · Lenny 37.1% · guest 62.9%54:00 · Lenny 21% · guest 79%54:00 · Lenny 21% · guest 79%57:00 · Lenny 8.2% · guest 91.8%57:00 · Lenny 8.2% · guest 91.8%1:00:00 · Lenny 14.3% · guest 85.7%1:00:00 · Lenny 14.3% · guest 85.7%1:03:00 · Lenny 23% · guest 77%1:03:00 · Lenny 23% · guest 77%1:06:00 · Lenny 55.4% · guest 44.6%1:06:00 · Lenny 55.4% · guest 44.6%1:09:00 · Lenny 4% · guest 96%1:09:00 · Lenny 4% · guest 96%1:12:00 · Lenny 28.9% · guest 71.1%1:12:00 · Lenny 28.9% · guest 71.1%
Sharpest disagreement ▶ 37:20 Pushing back against dogmatic framework adherence

Brian gently rejects dogmatic use of product frameworks, stressing that forcing content into standard templates without cultural internalization yields poor results.

Hardest push from Lenny ▶ 24:00 Challenging ops necessity with Casey Winters hot take

Lenny pushes on whether operations is fundamentally just a temporary sign of software inefficiency that should be engineered away.

Biggest teaching moment ▶ 10:15 Educating on the reality of manual surge controls

Brian reveals that early Uber surge pricing was manually managed and geofenced by city GMs rather than driven purely by algorithms.

Lenny holds their own ▶ 52:10 Synthesizing competitive positioning and Stripe Atlas parallel

Lenny draws on insights from a Stripe Atlas interview to illustrate why category leaders benefit when adjacent platforms realize full vertical integration is too hard and pivot to partnership.

the scores for every segment, with the reasoning behind each
ChapterTopicLenny as informed peerGuest teachingGuest disagreementLenny pushing backWhy
Transitioning from Operations to Product Leadership 3400 Lenny opens with friendly rapport and introduces Brian's unusual career path from operations to Head of Product. Brian details how grounding product instincts in direct customer support and onboarding creates operational leverage.
Bridging Product and Ops: The Genesis of Product Operations 4611 Lenny asks about product teams undervaluing ops, and Brian articulates the twin-turbine jet engine analogy. Brian also clarifies that while he formalized product operations at Uber, predecessors at Google had already explored similar functions.
Early Uber Mechanics: Manual Surge Pricing and Naming UberX 4611 Lenny inquires about early Uber pricing myths, and Brian explains how surge pricing was initially a manual, human-in-the-loop system controlled by local GMs rather than an automated algorithm.
High-Stakes Launches: UberPool in China and Opendoor's COVID Pivot 3500 Brian shares the high-stress launch story of UberPool in Chengdu and Opendoor's rapid pivot to virtualizing home inspections during the initial COVID lockdowns.
Opendoor Overview and Core Product-Ops Lessons 3500 Brian breaks down the mechanics of Opendoor's core value proposition and shares architectural lessons on building software that accounts for real-world entropy.
Scaling from Manual Operations to Software Automation 5512 Lenny brings up Casey Winters' hot take that operations is a sign of inefficiency. Brian agrees in principle but reframes ops as the leading edge for experimentation before software automation takes over.
Frameworks for High-Impact Product Reviews 4500 Brian shares best practices for conducting constructive product reviews, emphasizing tight attendee lists, psychological safety, and clear separation between mandate and suggestion.
Sponsor Break: Attio CRM 5511 After an ad break, Lenny explores Jobs to be Done methodology. Brian explains how Opendoor avoids dogmatic template-filling and instead uses JTBD to understand rare, high-stakes home transactions.
Experimentation and Intuition in Low-Volume Environments 4611 Brian lays out statistical strategies for low-volume experimentation when traditional AB testing lacks sample power, emphasizing alternative statistical techniques and disciplined intuition.
Navigating Competition: Zillow's iBuying Venture and Eventual Partnership 4510 Brian explains the operational complexity of vertically integrated iBuying that ultimately led Zillow to shutter its buying arm and partner with Opendoor instead.
Maintaining Composure and Discerning the 'Kernel of Truth' 3500 Brian discusses maintaining emotional composure under pressure and finding the true signal or kernel of truth across noisy feedback channels.
Failure Corner: UberPool's Launch Mistakes and Decentralized Experiments 4500 Brian recounts early missteps with UberPool's commuter cohort rollout in San Francisco and how heavy promotion helped discover that pure geographic liquidity was the key metric.
Matching Product Manager Archetypes to Problem Types 4400 Brian shares his perspective on PM archetypes, noting that hiring should focus on matching specific PM problem-solving profiles to the domain's unique constraints rather than defaulting to generic hiring.
Lightning Round and Brian's 5-Hour July 4th Uber Interview 3500 Brian wraps up with rapid-fire recommendations and tells the story of his intense five-hour July 4th interview at Uber, which included an unexpected working session with Travis Kalanick.

Statements from this episode (15)

Insight
Tolkin: Starting in operations is a strong foundation for product leaders
“Starting on the operation side gave a really deep understanding of like how the business actually works, right? You are truly operating it day in and day out. And the success of the city is, you know, in large part driven by The inputs that you are putting int…”
Brian Tolkin Aug 4, 2024 ▶ 2:59
Insight
Tolkin: Local operations teams iterate faster and gather better qualitative insights than PMs
“The reality is operations teams, local teams can iterate faster, can scale talking to customers really much more efficiently have great qualitative insights.”
Brian Tolkin Aug 4, 2024 ▶ 7:18
Assertion Supported
Uber's surge pricing was manually managed by city GMs until 2013
“Surge pricing for actually quite some times, probably all of 2012, certainly, 2013, probably, I don't know when we necessarily switched was very much a human in the loop system or a very manual system where GMs in every city would control basically the paramet…”
Brian Tolkin Aug 4, 2024 ▶ 10:25
Assertion Supported
UberX was named after a temporary placeholder in early financial models
“Uber, Uber X this product in the early days was going to be all hybrid and had a bunch of different potential names. I was not personally driving this. This was someone else on the operations team, but they dealt the model for What this product could be, and t…”
Brian Tolkin Aug 4, 2024 ▶ 12:32
Assertion Supported
Tolkin: Opendoor halted home purchases for months in 2020 to virtualize operations
“We actually turned off the core business and we stopped buying homes for a few months. Hey, we can't go in and we don't know if anyone's going to be buying any homes. And so, you know, what do we do? And we took those few months and then came out the other sid…”
Brian Tolkin Aug 4, 2024 ▶ 16:16
Assertion Not checkable as stated
Tolkin: Early Uber concentrated engineering on dispatch and pricing
“Which is why most of the engineering effort for Uber was on the dispatching system and the pricing system. That's just where the leverage was at the time, given, given the scarcity of resources.”
Brian Tolkin Aug 4, 2024 ▶ 19:01
Assertion Not checkable as stated
Early UberBlack drivers had two-hour, individual, in-person onboarding sessions
“In the early days is at small scale actually back when it was the Uber black drivers, every driver was individually onboarded in like a 90 minute to a two hour in person in the office onboarding with deep setting of expectations.”
Brian Tolkin Aug 4, 2024 ▶ 21:26
Insight
Tolkin: Keep Product Reviews Under 10 People but Distribute Docs Broadly
“Big subscribers of the best conversations happen when they're relatively small. So try and keep it under, under 10. Could be wide distribution of the document, right? The artifacts created are actually really powerful and they're powerful for the whole team to…”
Brian Tolkin Aug 4, 2024 ▶ 29:43
Insight
Infrequent user behaviors require rigid product frameworks to build empathy
“This on average in the U S is, is something people do once in seven years. You know, I'm sure the average had opened doors, something similar. And so it's a little bit harder to like be a customer. I took Uber every day, probably used Airbnb a number of times …”
Brian Tolkin Aug 4, 2024 ▶ 33:06
Insight
Tolkin: Run power analyses before committing to A/B tests
“Don't Just force yourself into A B testing without running the power analysis and say like, Hey, are we going to get results? What is the size that will detect? And what is the runtime of that experiment? And is that, and be honest, like, is that acceptable?”
Brian Tolkin Aug 4, 2024 ▶ 41:19
Insight
Dropping A/B test confidence to 80% is acceptable in low-volume environments
“You may be able to reduce your power and say, Hey, we're going to run at 80% confidence for all of our experiments instead of this traditional 95%, because that's a worthy trade-off. And if we're wrong one more time out of 10, that's okay.”
Brian Tolkin Aug 4, 2024 ▶ 43:15
Insight
Tolkin: Ship on intuition instead of pursuing false statistical precision
“If you're not going to get significance, if there's no other techniques at your disposal, then sometimes you just gotta trust your intuition and ship it. And if that's where, if that's what you believe, then that's what you believe. And you shouldn't spend tim…”
Brian Tolkin Aug 4, 2024 ▶ 43:43
Assertion Not checkable as stated
Tolkin: Early Uber ignored support efficiency to focus entirely on core matching
“If you think about the early Uber network from the simplest form, you've got A rider and a driver, and you need to connect them, price the transaction, and issue some receipts, probably, you know, and collect payment. So it's like, okay, do we do that really w…”
Brian Tolkin Aug 4, 2024 ▶ 58:24
Insight
UberPool's company corridor matching failed because marketplace liquidity is everything
“We very quickly realized that back to sort of like what the kernel truth is here is like liquidity is the only thing that matters. And there just wasn't enough. There was never going to be enough to sort of do this, like company-based thing. That wasn't the st…”
Brian Tolkin Aug 4, 2024 ▶ 1:01:23
Assertion Supported
Uber's viral stunts like puppies on-demand originated from decentralized city teams
“That was, by the way a example of like fully distributed. The benefit of having those early petri dishes someone a local marketing manager is like, Hey, this would be fun. It would be really fun and the platform can support it. And those promotions were fantas…”
Brian Tolkin Aug 4, 2024 ▶ 1:03:15
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 300 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.