Experiments
topic on 2 shows · 2 statements across 2 episodes
Startups For the Rest of Us 
Lenny's Podcast
2 statements about Experiments, every show
Vollset: Founders need experiment volume more than overanalyzing results
“The main problem isn't trying to figure out what's working. It's doing enough experiments so that something is working. And like, look, if they make a bunch of, if they do. 25 things and then two or three of them are working, but they're not totally sure which…”
Dropping A/B test confidence to 80% is acceptable in low-volume environments
“You may be able to reduce your power and say, Hey, we're going to run at 80% confidence for all of our experiments instead of this traditional 95%, because that's a worthy trade-off. And if we're wrong one more time out of 10, that's okay.”