Brian Tolkin, an early Uber employee and Head of Product at Opendoor, discusses how local market autonomy produced Uber's early marketing campaigns.
Insight
Dropping A/B test confidence to 80% is acceptable in low-volume environments
“You may be able to reduce your power and say, Hey, we're going to run at 80% confidence for all of our experiments instead of this traditional 95%, because that's a worthy trade-off. And if we're wrong one more time out of 10, that's okay.”
Insight
Tolkin: Ship on intuition instead of pursuing false statistical precision
“If you're not going to get significance, if there's no other techniques at your disposal, then sometimes you just gotta trust your intuition and ship it. And if that's where, if that's what you believe, then that's what you believe. And you shouldn't spend tim…”
Insight
Tolkin: Local operations teams iterate faster and gather better qualitative insights than PMs
“The reality is operations teams, local teams can iterate faster, can
scale talking to customers really much more efficiently have great qualitative insights.”
Assertion Supported
Uber's surge pricing was manually managed by city GMs until 2013
“Surge pricing for actually quite some times, probably all of 2012, certainly, 2013, probably, I don't know when we necessarily switched was very much a human in the loop system or a very manual system where GMs in every city would control basically the paramet…”
Insight
Tolkin: Keep Product Reviews Under 10 People but Distribute Docs Broadly
“Big subscribers of the best conversations happen when they're relatively small. So try and keep it under, under 10. Could be wide distribution of the document, right? The artifacts created are actually really powerful and they're powerful for the whole team to…”
Assertion Not checkable as stated
Tolkin: Early Uber ignored support efficiency to focus entirely on core matching
“If you think about the early Uber network from the simplest form, you've got A rider and a driver, and you need to connect them, price the transaction, and issue some receipts, probably, you know, and collect payment. So it's like, okay, do we do that really w…”