Insight
Schaffer: Growth teams should accept lower confidence intervals to run more experiments
“And so there are definitely times where I will advocate for and I will push for and I will myself use. Lower, lower confidence intervals than 95%, especially if that, like, doubles the amount of experiments that you can run in a year, right? End of the day, th…”
Assertion Not checkable as stated
Schaffer: Adding onboarding questions boosted Twilio signups by five percent
“I'm not kidding. It improved conversion. Like there's no personalization, nothing past it, just the questions. It improved conversion by like five percent, like just improved signups.”
Insight
Schaffer: Evaluating Growth Teams on Short Timelines Encourages Vanity Metrics
“Any growth team that's beholden to short timeline wins and improvement is always going to be dangerous. That's a, that's an environment that's conducive to vanity metric usage and fabric, you know, massaging the data to fit.”
Insight
Schaffer: Moving to PLG requires redefining customer problems, not unbundling enterprise products
“The thing that I think sticks across all companies, not just Amplitude making the shift is just that, that when you're doing this, do not think this is a copy paste, but like chop it for parts thing. Don't start with your product when you're building out your …”
Insight
Schaffer: Executive disconnect from customers creates career leverage for frontline employees
“Your executive team and executive teams at companies are often very sharp, but the nature of their day to day just does not link them with customers. Right. And that means that over time, especially as a company grows, they often lose access to some of the bes…”
Insight
Schaffer: Market expansion and product additions increase developer onboarding friction
“As we were penetrating new markets, adding more products, we were adding complexity and we were pulling in folks who were a little bit less motivated. And those things contributed to people saying, this is difficult.”
Insight
Schaffer: Customer-facing staff hold strong leverage to break into product
“Customer facing teams, such as those who want to crack into product. Your insights are extremely valuable. You're talking to customers every day. You know more about their problems and their pain than a lot of other people do.”
Insight
Schaffer: Peer leadership exposure makes promotion calibrations much easier for managers
“One of the things that, you know, It comes into play a lot when managers are figuring out promotions and doing those things. It's like they'll sit in a room off on calibrations and, you know, with a bunch of people, and it makes it a lot easier when those peop…”
Insight
Schaffer: Poor conversion in logical funnels signals psychological friction
“And if something like very logical isn't converting well, sometimes it means that you're battling against the psyche of the user and you want to take a step back and You know, think about and learn about where someone is psychologically in your space.”
Assertion Supported
Schaffer: Microsoft and Netflix experimentation shows 80% to 90% of hypotheses fail
“Companies like Netflix and Microsoft, there's over and over again. 80 plus percent. Some companies say 90% of things fail.”
Insight
Schaffer: A/B testing is one of the most expensive validation methods
“A-B testing is one of the most expensive kinds of ways to validate an experiment, right? It you know, often requires design and engineering and the PM or growth person or marketing person who's crafting it, right? All these things Or investments that take a lo…”
Insight
Schaffer: Growth teams should set annual low-medium-high target ranges
“I think it's good, especially for newer teams, but even teams in general commit, commit to something that you can kind of do over the course of a year and, you know, low, medium, high, right.”
Insight
Schaffer: Developers bypass marketing websites to go straight to product signup
“Developers, like almost two one, do not look at your marketing website at all. They go straight to your signup flow. So what that means is all that beautiful context that you're setting and, you know, the product and pricing and all that stuff, like very, you …”
Insight
Schaffer: Ask candidates about recent releases rather than cherry-picked successes
“The thing is everyone has like a big success story. Everyone does. So it really doesn't actually tell you very much to ask someone like, what's a great thing you released? Cause everyone can tell that. Instead, take, take that away. What's most recent ship is …”
Insight
Schaffer: Integrating analytics with session replays explains user anomalies without interviews
“You know, talking to customers is very powerful, but in this case, just that simple use of kind of connecting and threading those technologies together could really get a good picture of that without needing to engage customers.”
Assertion Not checkable as stated
Schaffer: A FAANG company deployed Twilio without ever speaking to sales
“There was a Fang company who signed up for Twilio. Built a POC, launched to production, all this, and operate in that space for months without engaging once with sales.”
