Growth Experiment
topic on 2 shows · 5 statements across 5 episodes
5 statements about Growth Experiment, every show
Cheng: Typical experiment win rate is 30% to 50%
“I mean, in my experience, the typical Win rate, and I hate to use that term for experiments, is, is often something like 30 to 50%.”
Batchu: Growth experiments typically have only a 30% success rate
“Growth experiments in my history are typically like, 30 percent-ish success rate. So the vast majority of things that you try don't work.”
Gotthilf: Statistically significant failed experiments reveal high-leverage user moments
“If you run an experiment with statistical significance and it harmed your product, that means that you found a moment that really matters to users. And if you found a moment that really matters to users, you can manipulate it in multiple ways for positive and …”
Failing to document growth experiments leads to repeated mistakes by new hires
“The third thing is choose a system of records, document learnings, document results is such a big part. Otherwise it get lost. And after two years and new hires begin to do the same experiments you already did like two years ago and didn't work.”