StarZero All shows Try the podcast clipper →

sample ratio mismatch

2 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Jul 27, 2023 by Ronny Kohavi · across every show →

Everything said about sample ratio mismatch, oldest first

Jul 27, 2023 negative
Assertion Partly supported
Roughly 8% of Microsoft's online experiments had sample ratio mismatches
“And where we share that at Microsoft, even though we'd be running experiments for a while is around eight percent of experiments that suffered from the sample ratio of dispatch.”
Ronny Kohavi Jul 27, 2023 ▶ 57:19 The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon)
Jul 27, 2023 neutral
Opinion
Bots are the most common cause of sample ratio mismatches in experiments
“So when you say most common, I think the most common is bots.”
Ronny Kohavi Jul 27, 2023 ▶ 58:40 The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon)
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 300 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more