The Exchanges

Q and teams should be focused on like the iterating on backlinks to optimize SEO for Quizlet to ensure that you're one of the top ranked sites in the country? Or should they be focused on like, One or two really big bats per year that will change the trajectory of a company. How do you think about that little iterations which lead to changes or big bats that are bold?

A Yeah, I mean, maybe this is fairly obvious, but I think the degree to which you should be taking big swings that maybe have lower confidence, but much higher impact if they hit versus smaller optimizations is largely a function of the maturity of your company, or maybe set a different way. You often hear within the growth community, this idea of an S curve, right? So like you launch a product, you're at the early part of the S curve. It's just starting to get adoption. You find your early adopter group. Then you hit a tipping point. Growth takes off. You're in the steep part of the S curve where you're getting hyper growth. And then at some point you start to saturate that market and you hit the upper end of the S curve where it flattens out and growth stalls. So if you're in the early part of that S curve, which by definition means any seed or series a startup, You should not be focusing on small optimizations. It's a waste of time for a couple reasons. One, because it's not going to move the needle enough to make a difference. Two, you should still have lots of low hanging fruit. If you're still that early, you're either working on the wrong product. You don't have product market fit. You're working on the wrong solution. Or you should have lots of low hanging fruit left to optimize, which means you should be taking bigger swings. And then the third reason is oftentimes, even…

AI assessment note: “largely a function of the maturity of your company”

Q How do you think about when's the right time to hire a growth team? I actually interviewed someone and they said you should hire one pre-product market fit, because product market fit is about getting the data to determine what works and what doesn't work, and growth people will be able to increase the supply side of data.

A Yeah, I generally disagree with that position, and I, I don't want to say that it can't be the right answer in certain cases, but in my opinion, Before you have product market fit, you don't even really know if you're building the right product. And then even once you have product market fit, you then need to figure out how do I get this product to scale within the right channel so that my unit economics remain healthy as I push beyond my highest intent early adopters, and it gets harder and harder to cost efficiently scale my business. And so I think typically it's like right around the time where you first Demonstrate you have strong product market fit that you want to bring in your first growth leader, because you'll have a much better understanding of who that person needs to be and what sort of team they need to build. Having said that, I will say the one caveat to this is oftentimes the best growth leaders come out of functions like, uh, engineering analytics or product. And so what that means is I've seen cases where you have pre-product market fit companies where there's, it's often somebody who's a little bit earlier in their They're, they're just really bright. They're really hungry. And they sort of have the natural skill set of like the intellectual curiosity and the drive to just understand, you know, why, why are we seeing growth in this area? Why is this target c…

AI assessment note: “right around the time where you first Demonstrate you have strong product market fit”

Q How do you think about hard decisions on paywalls? There's quite a few products. I think photo room is one of them, which has like a very hard product decision where you don't get anything for free. It's like you hit it and you got to pay versus three free. A freemium layer. How do you think about that hard versus softer paywall decision?

A Yeah, well, this, I mean, this is one of those that sounds like such a simple question, and it's actually much more complex because when you think about the optimal paywall positioning, it depends on what your product is, um, who your target customer is, what their willingness to pay is, how many Cheap and easy substitutes there are for your product in the market. And there's another piece around like what your price point is. Cause obviously the higher your price point is the longer it's, it's a more considered purchase, which just generally means it's going to take consumers. They need more proof before they're going to put their credit card down. And so all of these things matter. And, um, this is oversimplifying it, but I would say on one end of the spectrum, you have a freemium product like Quizlet that is targeting students. And teachers, but the vast majority of Quizlet's subscribers are students. So these are people who have low willingness or even ability to pay for your subscription. They have plenty of other EdTech products out there on the market. You've got Chegg Hero, Chegg, Course Hero, um, Brainly. Uh, you've got more and more AI products that are cropping up. I mean, even ChatGPT, right? And so Quizlet's a product where a more friendly and less intrusive paywall makes a lot of sense because they get a ton of value from word of mouth and from SEO that is driven …

AI assessment note: “optimal paywall positioning, it depends on what your product is”

Q really well is kind of the gamification of it in, in a way, though, actually, um, And that takes me to, like, games love to give easy levels to start with. Oh, you know, yay, I passed level one, super easy. And it makes me think of, like, time to value. How important do we think, or how do we think about time to value today in consumer subs apps?

A Yeah, well, I think time to value is, it's becoming more and more important for any business because human attention spans are getting shorter, right? Like, we're constantly bombarded with, Advertisements with notifications, as we talked about earlier, right? And so it's just so easy as a consumer to get distracted, which means that the window of time that a, that any product, but certainly consumer subscription product has to get a user's attention is getting shorter and shorter. Um, and so what that means is new user onboarding has become one of the most important parts of any consumer subscription app experience. Now the old conventional wisdom and I'll put myself in this category was shorter onboarding flows are better, right? Like the moment the user installs the app, they should understand your core value promise within the first 30 to 60 seconds of using it. And as quickly as possible, you want to get them to the aha moment and get them to enter a trial, um, so that you're able to continue to send them lifecycle marketing emails and ideally convert them into a subscriber. I think over the last few years, that's gotten a little bit more nuanced though. And, uh, Perhaps the canonical example of this is new, right? Which has more than a hundred screens in its onboarding flow, or at least the version of the onboarding flow I've gone through, right? It's, it's got so much per…

AI assessment note: “time to value is, it's becoming more and more important for any business”

Q What's the most dangerous myth around startup growth?

A I think some of the more recent ones are, um, if you build it, they will come. This idea that if I just build a really great product, then it will grow itself. I think that hasn't been true for a while, but it's getting harder and harder because an AI is going to make this even worse, right? It's never been easier to create a product. You have more and more stories of non-technical people using LLMs to create an app In a weekend and then finding a niche audience and getting it out there. And there's only going to be more and more of that. And so, you know, if you build it, they will come is, is, is a myth that has been debunked and will continue to be debunked because you really need to solve for distribution. And then I think the second one we talked about a little bit earlier is I can just pay my way to success. I can, I can essentially buy growth. That's, that can be true in the short term. And there are certain cases where paid acquisition can be a really effective Tool for, uh, accelerating growth in nascent markets or introducing a new product and getting the flywheel going. Uh, but more often than not, especially as a consumer subscription app, if you don't have a high enough percentage of organic acquisition, then at some point it's going to catch up with you.

AI assessment note: “if you build it, they will come. This idea that if I just build”

Q I would love to see the abandonment rate and how it changes In the different durations of the flow, because no one's going to go through 15 minutes and be like, ah, fuck it. Nah, don't feel like it. You're like, I'm nearly there. I'm nearly there, you know?

A Yeah, I think that's right. Well, I would love to see that data too. My hypothesis, and this may be completely wrong, but my hypothesis is it's a bit bimodal, where you're going to have a good percentage of users drop off within the first 10 to 20 screens Of Noom's onboarding experience. But once they've gotten far enough, they're probably sticking around for a reason. And in Noom's case, like Noom's whole value promise is we are going to deeply understand you and your unique psychology to understand how to help you lose weight where many other solutions have failed. And I think a lot of Noom users are users who have probably tried alternative solutions and they just haven't worked. And so when you think about it that way, Ok, once you've gotten to whatever the screen is, screen 20, screen 30, you're probably not going to stop, because you're pot invested to borrow a poker term, and you want to see it through, because if this works, it is solving a really acute pain point in your life.

AI assessment note: “my hypothesis is it's a bit bimodal”

Q Phil, can we just spend our way to dominance? Can we spend our way to growth?

A There are some, some rare exceptions where I think that has happened. Typically in marketplace businesses where the network effects are so strong once you get to the tipping point that you can have upside down unit economics for months or even years. And then, you know, once you Flip that tipping point. You pay it all back. And I think there was a period over the last decade where the funding environment and, um, VC preferences for, you know, massive user growth rates over profitability were such that a model like that could work and with the right business model and with the right network effects, it might be able to justify itself. But more often than not, certainly in the case of consumer subscription, that's, that's just not the case.

AI assessment note: “There are some, some rare exceptions where I think that has happened.”

Q What are the biggest mistakes that you think startups make when they hire the first one or two people in growth?

A Yeah, well, one of them I can speak to, um, because I made this mistake personally Early and early in my career when I was first starting to hire people. So I led the core product team and I bought it and then I moved over to Quizlet and help them build and scale their growth team, their product growth team. And when I first started to hire product managers, and this was not just true of growth. It was true of core product as well. Um, I had this tendency to, uh, I guess, focus too much on specialist skill sets. Like I want to hire somebody To work on the core product experience who has a design background and who's just really, really good at understanding user psychology and building the perfect aesthetics for an end user experience. Or, you know, I want to hire somebody to work on growth who comes from an extremely analytical background, ideally, like, you know, has, has already had three to four years of growth PM experience or comes from a quantitative background, like investment banking or, or whatever the case may be. Well, it turned out that was just misguided because I think product managers need to be strong generalists. They need to be able to flex to whatever the demands of the business dictate in any given quarter. Um, and then also, I just think with growth PMs in particular, the, I, I think that the traits that are most correlated with success Are deep intellectu…

AI assessment note: “I had this tendency to, uh, I guess, focus too much on specialist skill sets.”

Q How do you solve for that dude? You mentioned or there's whoop as well. I see so many people and bluntly. Everyone says that, hey, I kind of get what works and what doesn't after a certain time period. What would you do if you were within whoop or to solve for that diminishing value over time?

A So I think there are two things that these companies are doing really, really well to make sure that people continue to use their products over time. One is they're selling a hardware device, which means two things. One, they're getting a lot of revenue from their user upfront, as opposed to relying on SAS subscription revenue. And two, it sort of escalates commitment. So I'm somebody who's really been into mountain climbing for much of my life. There's this factor of like, you go out and you buy all the equipment and it almost like makes you more excited to go do the mountain climb, right? Cause now I have all this fancy equipment. I gotta go use it. It's the same thing with these products. Like, if you drop 304 hundred dollars on an Aura ring or a Whoop band, you're gonna be more invested in, in using it and getting maximum value out of it. So that's the first piece, is they've been clever about monetizing their product off of the hardware device as much or more than the subscription. The second component of it, and I think Aura is truly best in class at this, is the velocity with which they are launching new features, new insights, and Explaining those new features to users either through the app itself or through emails or other channels is pretty remarkable. And so I think all of that contributes to longer subscriber retention rate.

AI assessment note: “there are two things that these companies are doing really, really well”

Q What's the most dangerous myth around startup growth?

A I think some of the more recent ones are, um, if you build it, they will come. This idea that if I just build a really great product, then it will grow itself. I think that hasn't been true for a while, but it's getting harder and harder because an AI is going to make this even worse, right? It's never been easier to create a product. You have more and more stories of non-technical people using LLMs to create an app In a weekend and then finding a niche audience and getting it out there. And there's only going to be more and more of that. And so, you know, if you build it, they will come is, is, is a myth that has been debunked and will continue to be debunked because you really need to solve for distribution. And then I think the second one we talked about a little bit earlier is I can just pay my way to success. I can, I can essentially buy growth. That's, that can be true in the short term. And there are certain cases where paid acquisition can be a really effective Tool for, uh, accelerating growth in nascent markets or introducing a new product and getting the flywheel going. Uh, but more often than not, especially as a consumer subscription app, if you don't have a high enough percentage of organic acquisition, then at some point it's going to catch up with you.

AI assessment note: “if you build it, they will come. This idea that if I just build”

Q Okay. You mentioned that the stats, the streaks element, there's also like badges and leaderboards and notifications. If we take that as a kind of tactic that people use or a lever that people use for, for engagement, for retention, what are the biggest lessons that you have in how to do stats, streaks, badges well?

A I'll step back for a moment on this question because I think stats, streaks, badges, achievements, Coins, currencies, like, they are all tactics rooted in this idea of gamification, which is like, how do I take my core value promise and the natural usage frequency of my product and bolt onto that, this additional set of rewards that makes the user feel delighted. Underlying that is like, what is, what is the underlying motivation Of your user. And so there's this great framework called the Octalysis framework that I teach about in my Reforge course. Um, this guy, Yukai Cho, uh, invented the framework after spending many years in the gaming industry. And it talks about these eight core drivers of human motivation. And so for products like Duolingo, some of the most important drivers for Duolingo's product are things like, um, achievement. I want to feel like I've learned something new on Duolingo. Uh, accomplishment. Like I, I want to feel like I'm learning the language and I'm learning it faster and better than, than my peers. Um, there's a degree of ownership. Like I'm somebody who wants to become a good Spanish language speaker. And so I take ownership over that and I want to improve my ability over time. Um, there's a little bit of social status, like social influence. So I, this is where leaderboards come in, comes in, right? Like I want to be viewed as somebody who Is one …

AI assessment note: “fundamentally grounding your understanding first in the core motivational drives”

Q We were talking before about the definition of growth. If I were to ask you, how do you define growth and growth team? What comes to mind when I say that?

A Growth has become such an overloaded term in tech. Right? Does growth mean product? Does it mean marketing? Does it mean zero to one? Does it mean scaling product market fit expansion? You know, B to B versus B to C. There, there are a lot of different dimensions to it, but to me, what's fundamentally changed over the last 15 years or so since Facebook created what I would call the first modern growth team is that the product itself has become the most valuable asset powering the growth of many tech companies. And I think that's particularly true for a lot of consumer products because a lot of them don't have sales teams. And so when we talk about growth, I mean, fundamentally we're talking about how do you accelerate your ability to get your product into the hands of more users faster and have those users understand why that value matters to them and why it will make their lives better. That's what's ultimately going to lead to higher conversion rates, better retention, better monetization over time. But, but the way that looks is very different across different companies and products.

AI assessment note: “fundamentally we're talking about how do you accelerate your ability to get your product”

Q Okay. You mentioned that the stats, the streaks element, there's also like badges and leaderboards and notifications. If we take that as a kind of tactic that people use or a lever that people use for, for engagement, for retention, what are the biggest lessons that you have in how to do stats, streaks, badges well?

A I'll step back for a moment on this question because I think stats, streaks, badges, achievements, Coins, currencies, like, they are all tactics rooted in this idea of gamification, which is like, how do I take my core value promise and the natural usage frequency of my product and bolt onto that, this additional set of rewards that makes the user feel delighted. Underlying that is like, what is, what is the underlying motivation Of your user. And so there's this great framework called the Octalysis framework that I teach about in my Reforge course. Um, this guy, Yukai Cho, uh, invented the framework after spending many years in the gaming industry. And it talks about these eight core drivers of human motivation. And so for products like Duolingo, some of the most important drivers for Duolingo's product are things like, um, achievement. I want to feel like I've learned something new on Duolingo. Uh, accomplishment. Like I, I want to feel like I'm learning the language and I'm learning it faster and better than, than my peers. Um, there's a degree of ownership. Like I'm somebody who wants to become a good Spanish language speaker. And so I take ownership over that and I want to improve my ability over time. Um, there's a little bit of social status, like social influence. So I, this is where leaderboards come in, comes in, right? Like I want to be viewed as somebody who Is one …

AI assessment note: “grounding your understanding first in the core motivational drives”

Q So you're an angel in my company and I'm hiring this First few growth people, and I'm like, ah, the interview process. Ok, I, I want those traits. What questions should I ask that will lead to revealing answers that will let me know if they're potentially good growth hires?

A Yeah, there are a few things I like to do. So one is I, I do often like to assign a homework, uh, assignment. And I'm, I'm very careful about this because, for a couple of reasons. One, I think if you assign, I guess to step back, there are lots of ways to introduce bias. Into a homework assignment like this, right? Like number one, if you give an assignment that is specifically about your company, then especially if you're an early stage company that not everybody has heard of or not everybody deeply understands, you're going to get a lot of bias just based on who already knows your product and who doesn't. Um, number two, and this is particularly true as you start to be looking at people who have families or they have other demands on their time, right? If, if you, if you give an assignment and And oftentimes it'll be like, don't spend more than a couple hours on this. Well, of course, you know, the best candidates are going to want to roll up their sleeves and, and do everything it takes to deliver the best possible, uh, output. But different people have very different limitations on their time, especially if they already have a full time role. They've got a family at home. And so what I found works best is, um, pick an assignment that truly shouldn't take more than an hour or two before you hit diminishing returns. On, you know, spending more cycles on it. And that number t…

AI assessment note: “one is I do often like to assign a homework assignment”

Q Okay, so the next step post definition is great. Should we have a growth team that is separate, or should we have it baked into product, into marketing? How do you answer that to founders that ask?

A I mean, I think this really comes down to understanding from a first principles perspective, what is driving the growth of your business? And I think the mistake that a lot of founding teams make, especially now that growth has become such a buzzword and a lot of VCs will tell their companies, Understandably so. Like, hey, you need to hire a growth leader because growth is important and teams with great growth teams have a lot more success, which isn't wrong. But before you do that, you need to understand, okay, for my company and my product and my target customer, what is going to maximize the growth of the business? And I can speak to Quizlet, where I spent almost four years, uh, leading user growth and ultimately subscriptions as well. You know, at Quizlet, this was an education product That for years grew through a combination of word of mouth between teachers and their students and between, uh, students and their classmates. And then at some point it kind of hit this tipping point where all of the content that Quizlet had created originally as a digital flashcards app meant that they were appearing in more and more of these long tail search queries on Google where students were trying to get help, uh, getting unstuck on homework assignments or they were preparing for exams. And so, uh, over time, because it became one of the most traffic websites, uh, in the U S especially…

AI assessment note: “this really comes down to understanding from a first principles perspective”

Q So you're an angel in my company and I'm hiring this First few growth people, and I'm like, ah, the interview process. Ok, I, I want those traits. What questions should I ask that will lead to revealing answers that will let me know if they're potentially good growth hires?

A Yeah, there are a few things I like to do. So one is I, I do often like to assign a homework, uh, assignment. And I'm, I'm very careful about this because, for a couple of reasons. One, I think if you assign, I guess to step back, there are lots of ways to introduce bias. Into a homework assignment like this, right? Like number one, if you give an assignment that is specifically about your company, then especially if you're an early stage company that not everybody has heard of or not everybody deeply understands, you're going to get a lot of bias just based on who already knows your product and who doesn't. Um, number two, and this is particularly true as you start to be looking at people who have families or they have other demands on their time, right? If, if you, if you give an assignment and And oftentimes it'll be like, don't spend more than a couple hours on this. Well, of course, you know, the best candidates are going to want to roll up their sleeves and, and do everything it takes to deliver the best possible, uh, output. But different people have very different limitations on their time, especially if they already have a full time role. They've got a family at home. And so what I found works best is, um, pick an assignment that truly shouldn't take more than an hour or two before you hit diminishing returns. On, you know, spending more cycles on it. And that number t…

AI assessment note: “So one is I, I do often like to assign a homework, uh, assignment.”