Duolingo purposely killed a winning in-lesson XP experiment to preserve UI simplicity
Jackson Shuttleworth · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · Dec 15, 2024 · at 40:18
Jackson Shuttleworth, Group PM leading retention at Duolingo, explains why the company killed a statistically winning experiment that added UI clutter to core lessons.
0:00 / 0:21exact quote · 21.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“It was a nice win, but we had added that UI element and You know, at least at the time, it was less clear what we would do with it, and we realized that, like, long-term, it was just gonna get in the way, and we'd rather, for simplicity's sake, pause that, you know, shut it down, and keep the lesson, you know, to be the, a little bit of a learning sanctuary it was.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jackson Shuttleworth
Adding an explicit opt-out button to streak goals increased user commitment
“We captured, you know, just what happens if we add an opt out button and adding an opt out button when you would think like as a PM, oh, I'm like, you know, now users might not engage with my feature. That's a bad thing, but it was a huge win to let them do th…”
Strategic friction via haptics and animations significantly boosts user retention
“Your phone, you know, like there being a really cool haptic pattern as you extend your streak all of this stuff wins and it's cool because I think it wins There's a few reasons. One is it just makes you feel good, right? You know, you get some cool moment in y…”
Switching Duolingo streaks to one daily lesson drove massive DAU growth
“One of the most impactful experiments we ran was about, this was actually as I was joining, we had, or just after I joined, we had run this experiment was to move it from a XP based streak to just do one lesson a day and you'd extend your streak. And, you know…”
Duolingo's user retention jumps exponentially between days one and seven
“And what we found is that once you get to seven days, loss aversion kicks in and you retain. So going from a one to a two day streak, huge jump in retention, two to three day streak, slightly less, but still huge. And it's up until day seven. Once you hit day …”
Pre-selecting streak goals for users significantly reduced Duolingo engagement
“We thought once we built you know, a goal picker screen where you could pick between different streak lengths, we were like, oh, well, let's like recommend that users do a hard, you know, a harder goal and thinking that, okay, well, a harder goal is going to b…”