The Exchanges

Q But like when you see, like, the ability to message drivers, the ability to know timing, the ability to add notes to orders, all those little things which actually people can love. What is that?

A For sure, for sure. Well, the key is that you ship all of those things in carefully designed and controlled experiments where you have a set of metrics that you're looking to improve. So you can imagine, for example, the driver chat is designed to reduce What's called rider experience time, the time that it takes a rider to get from the restaurant to handing you the, the order. And there's often, you know, a couple minutes at the end of R.E.T. where they're looking for, you know, looking for your apartment, you know, fumbling around or whatever, and you're sitting there being like, God, if the guy just hits to be, you know, I'll have my food now. Do I really have to go outside, whatever? You know, and so you can design an experiment which very clearly shows whether or not R.E.T., like rider experience time, Decreases by the amount that you expect it to in the population that has messaging. What you do is you give messaging to, you know, I don't know, somewhere between 20 and 50% of the users, and then you just look at the delta in RET between those two, and you have to, you can do, you know, power analysis so that you know how big the experiment needs to be, how long it has to run, and then at the end of that, as long as you, as long as there were no execution problems with the feature, and as long as you designed the experiment correctly, you're going to get an answer. You mig…

AI assessment note: “the key is that you ship all of those things in carefully designed and controlled experiments”

Q What are some big lessons to you in the right way to structure those teams? Could be size, could be roles, could be mentality. Any big lessons on the right way to structure those teams?

A Yeah, well, I think every product team should be probably between, you know, six to eight people. Um, most of those people should be engineers. If possible, one of those person, one of those people should be a data scientist. One should be a designer. Um, and then I think a PM is optional, you know. Um, that team should work together against, like, clear, coherent, outcome-based goals, which are not SHIP goals, but are like, we are going to increase the, you know, the revenue that we generate for this thing by 10% is like a decent goal, but a better goal is we're going to increase, you know, people's satisfaction with the product by 10%, or we're going to increase people's sales by 10%, which we believe is going to lead to a 10% increase in revenue. You'll find that that often leads to a 20% increase in revenue. You know, and so that attitude is unbelievably important that they are building a thing to serve a person or a group of people on the other side. Um, and I think that they have to feel truly empowered to reach that goal and feel supported by somebody who, you know, checks in and says, oh, do you have everything that you need? How are you doing? Are you on time? Are you on schedule? That kind of thing.

AI assessment note: “every product team should be probably between, you know, six to eight people.”

Q When you review product decisions, I think we learn a lot from successes and failures. When you think about the single best product decision you made at Facebook, what do you think it was with hindsight, and what did you learn?

A You know, there were two, I'm gonna give you two examples. Um, so I, I joined the ads org I, I don't know, like a couple months after the IPO. And, uh, you know, I, a lot of people don't remember this, but the Facebook IPO wasn't great, and ads revenue wasn't really performing the way that we wanted it to. And to support the kind of valuation that we, we had or that we wanted, we needed to accelerate revenue growth really significantly. And I think part of the challenge for the company was that, um, historically, Mark had said, you know, I, there was a perception that he didn't really care about ads, and I think the, The ask that I got was to just go over there and, and kind of, I don't know, make it okay for good consumer PMs to work on ads. There was this view that, that that was important. I think the organization felt like it hadn't shipped anything of quality in a long time. And I kind of looked around and tried to find something unloved without a team on it, where we could just very rapidly ship something where the only thing that was necessary, it didn't need to Even be successful was that everybody in the organization would say, oh, we're capable of building something great. You know, we're capable of producing quality at speed. That thing ended up being this page insight. So if you go to Facebook pages and you're a page admin, you can go in and you can see how your pos…

AI assessment note: “That thing ended up being this page insight... Rebuilt that product in a few months”

Q But like when you see, like, the ability to message drivers, the ability to know timing, the ability to add notes to orders, all those little things which actually people can love. What is that?

A For sure, for sure. Well, the key is that you ship all of those things in carefully designed and controlled experiments where you have a set of metrics that you're looking to improve. So you can imagine, for example, the driver chat is designed to reduce What's called rider experience time, the time that it takes a rider to get from the restaurant to handing you the, the order. And there's often, you know, a couple minutes at the end of R.E.T. where they're looking for, you know, looking for your apartment, you know, fumbling around or whatever, and you're sitting there being like, God, if the guy just hits to be, you know, I'll have my food now. Do I really have to go outside, whatever? You know, and so you can design an experiment which very clearly shows whether or not R.E.T., like rider experience time, Decreases by the amount that you expect it to in the population that has messaging. What you do is you give messaging to, you know, I don't know, somewhere between 20 and 50% of the users, and then you just look at the delta in RET between those two, and you have to, you can do, you know, power analysis so that you know how big the experiment needs to be, how long it has to run, and then at the end of that, as long as you, as long as there were no execution problems with the feature, and as long as you designed the experiment correctly, you're going to get an answer. You mig…

AI assessment note: “you ship all of those things in carefully designed and controlled experiments”

Q What are some big lessons to you in the right way to structure those teams? Could be size, could be roles, could be mentality. Any big lessons on the right way to structure those teams?

A Yeah, well, I think every product team should be probably between, you know, six to eight people. Um, most of those people should be engineers. If possible, one of those person, one of those people should be a data scientist. One should be a designer. Um, and then I think a PM is optional, you know. Um, that team should work together against, like, clear, coherent, outcome-based goals, which are not SHIP goals, but are like, we are going to increase the, you know, the revenue that we generate for this thing by 10% is like a decent goal, but a better goal is we're going to increase, you know, people's satisfaction with the product by 10%, or we're going to increase people's sales by 10%, which we believe is going to lead to a 10% increase in revenue. You'll find that that often leads to a 20% increase in revenue. You know, and so that attitude is unbelievably important that they are building a thing to serve a person or a group of people on the other side. Um, and I think that they have to feel truly empowered to reach that goal and feel supported by somebody who, you know, checks in and says, oh, do you have everything that you need? How are you doing? Are you on time? Are you on schedule? That kind of thing.

AI assessment note: “every product team should be probably between, you know, six to eight people”

Q Truly delightful. Do people not share that? Do people not tell their friends?

A Yeah, I think they do. So I, I think that maybe what you're getting at is there are different strategies for distribution. So we had this conversation at the office this morning. We had a marketing meeting this morning. We're, we're about to remove wait lists. We're about to really launch sling. And, you know, we had a conversation about whether we should do paid marketing or not. And, you know, we have a, we have some people who are already using the product and returning and using it a lot and, and giving us great feedback. And we had a long conversation about whether or not Our strategy in the short term should be, you know, a bunch of paid marketing or whether or not it should be, uh, you know, viral growth and just encouraging people to tell their friends. And, you know, I think we're going to do viral growth to start with, because to your point, if it is really great, people will tell their friends and they will tell each other. We may have to encourage them to do that. We may have to help them to do that in various ways.

AI assessment note: “Yeah, I think they do. So I, I think that maybe what you're getting at”

Q What are the biggest mistakes that founders make when building out their product team?

A This comes back to founder mode a little bit. I, I think that people often, when their company starts growing, start thinking that they are out of their depths as a founder and that they don't understand what is needed from the product organization. They don't understand what's needed from the finance organization. And they kind of go against their gut because somebody's telling them, oh, well, this person is great. They did a great job at this company or whatever. I think you really have to believe your gut, um, partially because you're probably correct, at least about your company, like, you know, a director or a VP from Google is not necessarily gonna do well at your, like, hundred person company, but also because you need to trust the people in that role so much that if you have that kind of, like, doubt at the time that you're hiring them about whether or not they're the right person, you just shouldn't do it. Like, that relationship is just so deeply important. Um, you have to, you have to listen to yourself on these things.

AI assessment note: “a director or a VP from Google is not necessarily gonna do well at your”

Q What was the worst product mistake you made at Facebook, and what did you learn from that?

A Oh man, you know, there are a few, and I, I don't, I don't know how much, it's always difficult to talk about things when, you know, people are still there and all of that kind of stuff. I, I think that we spent a lot of time building this thing called Audience Insights, which was, like, the logical, it was an extension of a lot of the theories that we had where we thought, we thought, oh, well, we have all this amazing data, like, population-level data where we can help marketers kind of drill down and find their audience, you know. It's probably targeted sort of at brand marketers, and we said, oh, you can go in and you can explore the entire population of people in the world and what they're interested in. You can see That, like, Yankees fans also tend to like, you know, 20 VC or whatever, right? Um, and the idea was that it would, like, help you develop creative strategies and mark, you know. It turned out to be incredibly difficult to build. It required a huge amount of novel technology. And, you know, it, like, it was a nice to have. Like, it was just a nice to have. So I think that discreet thing, um, you know, we, I wouldn't, I wouldn't prioritize it the same way now.

AI assessment note: “we spent a lot of time building this thing called Audience Insights”

Q Speaking of, like, serving communities incredibly well, Deliveroo serves incredible communities with immense products. I obviously consume Deliveroo every day, hence my complete inability to cook. But you were CPO there for two and a half years. It's a fascinating role change, really, from Facebook to Deliveroo CPO in that respect. How did your time at Deliveroo impact your product thinking?

A It's an unbelievably different environment, and, you know, I've now done, well, social media, advertising, online video, food delivery, a regulated bank, and, and now stablecoins, and they're all dramatically different, and I, I think that the most striking difference with Deliveroo, uh, compared to Facebook, both are fast-paced, but Deliveroo Uh, plays a game every day from breakfast to dinner, which doesn't stop. It's real time. Every order is being delivered in real time. It's either on time or not. You either have enough riders to deliver on time or not. It's either raining or it's not. It's Sunday night and everybody's starving. Uh, the restaurant is overrun by riders standing outside waiting for the food to be cooked. Um, and then at the end of the day, you look at how many orders you did and whether or not you grew from the day before. It is a real time, unbelievable exercise in logistics, but also move, just physical movement, which leads to a dramatically different culture and way of working and way of building software than you get when you're building a-

AI assessment note: “leads to a dramatically different culture and way of working and way of building software”

Q Does that not go against every concept of positive visualization? I see the success, I make it happen.

A Well, I mean, I don't know how much I, I think that you have to have these kind of competing ideas in your head at the same time, which are like on one hand, we've built a thing that we really believe in, and we, we're putting our best foot forward, and we believe that it is going to work. And at the same time, we intellectually understand that when you first ship something, it's probably going to have something wrong with it, or some misunderstanding of the Customer need or your solution that needs to be addressed. Sometimes that's so drastic as, well guys, we just wasted our last six months and we never should have built this in the first place. And sometimes it's something as simple as, ah, you know, the copy on the second street screen of the new user experience is wrong and people don't get it, you need to change it. You know, you need to be open to like both of those things and it could be anywhere on that spectrum. I think you just have to be, You know, emotionally prepared for having to do that work, and if you're not emotionally prepared for doing that work, uh, then, you know, when you don't hit your goals with a launch, what happens is the team and everybody around the team who makes decisions around this stuff, um, becomes deflated, mood goes down, and you kill the project too early.

AI assessment note: “you have to have these kind of competing ideas in your head at the same time”

Q What two to three trays or one to two trays are less obvious but crucial in great PMs?

A You know, I, I think great PMs combine, um, deep intuition about people in markets, uh, with, Great taste, uh, like great product taste, great design taste. They don't need to be great designers, but they need to under, they need to understand it and be able to tell you, oh, that's good, that's bad. Together with, um, a deep understanding of engineering, um, or at least the ability to operate with engineers and, and, and have respect. Together with a, you know, an understanding of the company's context and what it needs to accomplish. Does it need to grow? Does it need to be profitable? Does it need profitable growth? You know, all of these things. And if you don't have all of those things, I think it's very hard to be a good PM. And I think the thing that is not always obvious, uh, is just deep intellectual honesty. You know, you need somebody who is willing to say, from the first instance, I, I don't know the answer to that. I'm gonna go figure it out.

AI assessment note: “the thing that is not always obvious, uh, is just deep intellectual honesty.”

Q What was the best product decision you made at Delivery?

A I think that that broad focus was, was the most important one, I, I think that my favorite was, ah, I went on a trip to Spain at some point with, with a bunch of people from the team, and we, um, went there and we really learned the importance of selection. We went to Madrid, and Madrid is a very long city, and there's one, there's one five guys on one end. Um, and it turns out that people who live on the other end, the way that we designed our, our system originally It would dispatch on kind of a neighborhood basis, and so, you know, you probably experienced this living in London ordering delivery. You're like, well, why can't I order from this restaurant, which is like three blocks away, because it's in another zone. And, you know, that was kind of optimal for London, which is a city, or we thought it was optimal for London, which is a city of, um, residential neighborhoods arranged around high streets, and the restaurant is on the high street. Um, you know, Madrid is very different than that. It kind of has like a central business district, and there's like a, You know, restaurants and then, you know, residential that goes out much further. It turns out that the people in the outskirts of the residential area still want five guys. You know, and they actually will tolerate a longer trip and understand that it'll be a little bit soggy and a little bit cold when it arrives beca…

AI assessment note: “we really learned the importance of selection”

Q How you approach products. How did being a regulated bank impact your approach to product?

A I mean, massively, and I had to learn how to do that. I think at various times I really chafed at that or found that very difficult. Um, and I remember having this amazing conversation with this guy, Ian, who's the chief risk officer at, at Monzo, who, who also became a really good friend and he's, he's brilliant. And he just said at some point, he was like, you know, you have to understand that most of, or many of the crises that nations have faced in history are the result of financial crises or banking failures. And society has a, uh, you know, has a right, has an obligation to protect itself and people from those kind of, uh, situations, from those kind of outcomes. You know, I, I think this is like a really important Thing. It's a really important lesson. You, you have to understand where people are coming from, and once you understand that, and you think about that critically, and you change your perspective from somebody who is trying to make a graph go up and to the right, and trying to produce value for people as quickly as possible, and experiment, and, you know, you're like, why can't I test pricing? I've tested pricing my entire career. Like, what are you talking about? I can't test pricing. And you change your perspective to, like, so much Of society is based on the stability of the banking system.

AI assessment note: “I mean, massively, and I had to learn how to do that.”

Q Do you think the US was the right move? I mean, both Revolut and Monzo did it, both with limited success.

A I don't think it was a wrong decision at all. Like, I think that the US is the very big prize. I think that you have to, when you're developing a bank, and it's very interesting, we've kind of taken the opposite perspective with Sling in a way, which is, you know, weird in a way, but banking is very regionally specific. The way that mortgages work in the UK and the way that mortgages work in the US are completely different. You know, credit cards in the US versus credit cards in the UK, completely different. The way that people think about debt, the way that people think about saving, the way that people think about investing, the pace at which people get paid, when people pay their bills and how they pay their bills, all these things are different. And I think it's very easy to think that you can take a highly localized product and just lift it and shift it somewhere else with a different license and it's going to work. I suspect that the only way to truly do that successfully is to have people in the local market who take the bones of the thing, who deeply understand what has and has not worked about it, and then shape it around, like, local insights. And I think that this is probably more particular to banking than almost anything else.

AI assessment note: “I don't think it was a wrong decision at all. Like, I think that the US”

Q And what did you do to turn it around?

A Well, I think there are two things that are very important. Uh, you need to give people a win. You know, so Page Insights that we talked about earlier is an example of a win like that. You know, well, we, we can be proud about a thing that we shipped. Uh, and then I think you also have to give people a reason to work hard or a thing to believe. So, like, when I, uh, I worked on sharing, uh, at Facebook, which is like all the photos, text, videos, links, everything that ends up in newsfeed from regular people. So for a while I did ads, which was like, Everything from companies and celebrities, and then I moved over to, to, to people. You know, in both cases, there's like a weird thing. You can imagine people saying, I don't want to work on ads, because I don't want to work on the thing that, like, makes money, and, ah, you know, detracts from the experience, right? Um, you know, ads are annoying, and all this kind of stuff. You can reframe that, and you can say, actually, the thing that we're doing is we're helping individuals Discover the products that they, that, that are going to improve their lives, and we're helping businesses find customers, and the more efficient we can make that process, the better we can make that process, the more economic growth is going to happen in this world, and actually the thing that you're doing when you have a billion people looking at these a…

AI assessment note: “you need to give people a win... give people a reason to work hard”

Q When you review product decisions, I think we learn a lot from successes and failures. When you think about the single best product decision you made at Facebook, what do you think it was with hindsight, and what did you learn?

A You know, there were two, I'm gonna give you two examples. Um, so I, I joined the ads org I, I don't know, like a couple months after the IPO. And, uh, you know, I, a lot of people don't remember this, but the Facebook IPO wasn't great, and ads revenue wasn't really performing the way that we wanted it to. And to support the kind of valuation that we, we had or that we wanted, we needed to accelerate revenue growth really significantly. And I think part of the challenge for the company was that, um, historically, Mark had said, you know, I, there was a perception that he didn't really care about ads, and I think the, The ask that I got was to just go over there and, and kind of, I don't know, make it okay for good consumer PMs to work on ads. There was this view that, that that was important. I think the organization felt like it hadn't shipped anything of quality in a long time. And I kind of looked around and tried to find something unloved without a team on it, where we could just very rapidly ship something where the only thing that was necessary, it didn't need to Even be successful was that everybody in the organization would say, oh, we're capable of building something great. You know, we're capable of producing quality at speed. That thing ended up being this page insight. So if you go to Facebook pages and you're a page admin, you can go in and you can see how your pos…

AI assessment note: “That thing ended up being this page insight.”

Q What was the best product decision you made at Delivery?

A I think that that broad focus was, was the most important one, I, I think that my favorite was, ah, I went on a trip to Spain at some point with, with a bunch of people from the team, and we, um, went there and we really learned the importance of selection. We went to Madrid, and Madrid is a very long city, and there's one, there's one five guys on one end. Um, and it turns out that people who live on the other end, the way that we designed our, our system originally It would dispatch on kind of a neighborhood basis, and so, you know, you probably experienced this living in London ordering delivery. You're like, well, why can't I order from this restaurant, which is like three blocks away, because it's in another zone. And, you know, that was kind of optimal for London, which is a city, or we thought it was optimal for London, which is a city of, um, residential neighborhoods arranged around high streets, and the restaurant is on the high street. Um, you know, Madrid is very different than that. It kind of has like a central business district, and there's like a, You know, restaurants and then, you know, residential that goes out much further. It turns out that the people in the outskirts of the residential area still want five guys. You know, and they actually will tolerate a longer trip and understand that it'll be a little bit soggy and a little bit cold when it arrives beca…

AI assessment note: “we really learned the importance of selection”

Q When did you look at data, come to a conclusion, and that conclusion was really wrong?

A Maybe a different way of answering this, slightly different from the question that you're asking, I think. It's very hard to accurately interpret data, and it's very difficult to not lie to yourself with data. And, you know, you can cut data any way that you want. Um, I once shipped something into the, it's called the dive bar at the time at Facebook. It was like a social product, a social sharing surface, and the dive bar was this thing, if you swiped the Facebook app to the left, it opened up on the right. It was originally used, I think, for, uh, messenger contacts. I think something like 75% of Facebook users opened the dive bar, and so, There was an argument to be made that it was a great surface to ship into. The thing is that like, 99% of those people opened it by accident. And didn't want anything to do with it. And then immediately closed it. You know, so you can run all sorts of different analyses. You need to consider the entire, you know, set of things around you in order, in order to understand it. And, you know, I've made those kind of mistakes thousands of times.

AI assessment note: “I think something like 75% of Facebook users opened the dive bar”

Q This is so great. Now, I think there is a moment when someone falls in love with product and the design and the simplicity of it. When did you fall in love with product, Mike?

A Man, I've basically wanted to build things on the internet for almost the entire time that I can remember. Like I, even before the internet, my brother had an Amiga Back in the day, and he used CompuServe on it, and he was all in the forums. He was like a sysop on the film forum on CompuServe. And I just thought it was the coolest thing ever. I think from that point on, I just wanted to build things, like, I wanted to make things, like, digital things, and I, like, dropped out of high school when I was 15 and went to work at a startup, and just always wanted to build things on The internet. Like, it's just what I always wanted to do.

AI assessment note: “I think from that point on, I just wanted to build things”

Q Do you think the US was the right move? I mean, both Revolut and Monzo did it, both with limited success.

A I don't think it was a wrong decision at all. Like, I think that the US is the very big prize. I think that you have to, when you're developing a bank, and it's very interesting, we've kind of taken the opposite perspective with Sling in a way, which is, you know, weird in a way, but banking is very regionally specific. The way that mortgages work in the UK and the way that mortgages work in the US are completely different. You know, credit cards in the US versus credit cards in the UK, completely different. The way that people think about debt, the way that people think about saving, the way that people think about investing, the pace at which people get paid, when people pay their bills and how they pay their bills, all these things are different. And I think it's very easy to think that you can take a highly localized product and just lift it and shift it somewhere else with a different license and it's going to work. I suspect that the only way to truly do that successfully is to have people in the local market who take the bones of the thing, who deeply understand what has and has not worked about it, and then shape it around, like, local insights. And I think that this is probably more particular to banking than almost anything else.

AI assessment note: “I don't think it was a wrong decision at all.”

Q Speaking of, like, serving communities incredibly well, Deliveroo serves incredible communities with immense products. I obviously consume Deliveroo every day, hence my complete inability to cook. But you were CPO there for two and a half years. It's a fascinating role change, really, from Facebook to Deliveroo CPO in that respect. How did your time at Deliveroo impact your product thinking?

A It's an unbelievably different environment, and, you know, I've now done, well, social media, advertising, online video, food delivery, a regulated bank, and, and now stablecoins, and they're all dramatically different, and I, I think that the most striking difference with Deliveroo, uh, compared to Facebook, both are fast-paced, but Deliveroo Uh, plays a game every day from breakfast to dinner, which doesn't stop. It's real time. Every order is being delivered in real time. It's either on time or not. You either have enough riders to deliver on time or not. It's either raining or it's not. It's Sunday night and everybody's starving. Uh, the restaurant is overrun by riders standing outside waiting for the food to be cooked. Um, and then at the end of the day, you look at how many orders you did and whether or not you grew from the day before. It is a real time, unbelievable exercise in logistics, but also move, just physical movement, which leads to a dramatically different culture and way of working and way of building software than you get when you're building a-

AI assessment note: “dramatically different culture and way of working and way of building software”

Q What was the worst product mistake you made at Facebook, and what did you learn from that?

A Oh man, you know, there are a few, and I, I don't, I don't know how much, it's always difficult to talk about things when, you know, people are still there and all of that kind of stuff. I, I think that we spent a lot of time building this thing called Audience Insights, which was, like, the logical, it was an extension of a lot of the theories that we had where we thought, we thought, oh, well, we have all this amazing data, like, population-level data where we can help marketers kind of drill down and find their audience, you know. It's probably targeted sort of at brand marketers, and we said, oh, you can go in and you can explore the entire population of people in the world and what they're interested in. You can see That, like, Yankees fans also tend to like, you know, 20 VC or whatever, right? Um, and the idea was that it would, like, help you develop creative strategies and mark, you know. It turned out to be incredibly difficult to build. It required a huge amount of novel technology. And, you know, it, like, it was a nice to have. Like, it was just a nice to have. So I think that discreet thing, um, you know, we, I wouldn't, I wouldn't prioritize it the same way now.

AI assessment note: “we spent a lot of time building this thing called Audience Insights”

Q you, is kind of a great product line, but also an angel investor. And he likes a long road to the start line in terms of product, whereby it's actually quite difficult to get V-one out the door because there is so much to do. I'm just intrigued for you. How do you think about that long road to the start line versus get a quick V-one out, test, see?

A Well, so, you know, implicit, I guess, in some of what I was talking about earlier is this idea that I really believe that before you build something, you should have a theory of the world. You should, you know, understand who your user is, what they want to accomplish. You should understand your strengths and weaknesses in doing that and have a, you know, a theory of like the thing that you're going to build and how it is going to accomplish your goals and the user's goals. Um, and that might take you anywhere from 10 minutes to develop to a year to develop. You know, it's very, that's very variable. It really depends on what you're doing. I don't really think you should start writing code until you have that. Now, the problem is that most people's theory of the world, uh, you know, prior to shipping a thing and having contact with real live human customers is wrong, and it might be five percent wrong, or it might be 50% wrong, and you don't really know until you ship. You can do user research along the way, but, you know, the reality is that, that users lie when you talk to them, and they, they will always tell you what they think you want to hear. You know whether or not it's good because they'll come back or they won't come back. You know, if they churn, you were wrong. Then the thing that people often do, one of the big mistakes I think people make, is they then give up to…

AI assessment note: “I don't really think you should start writing code until you have that.”

Q How do you think about discussion within teams? Is talk good, or is dictatorial product vision better?

A I don't really feel like those are the choices as I would frame them. First of all, I think it's very important to have quantitative You know, objective measures of success. So when you're having a debate about something, about what you should do, you need to ground that somehow, you know, and you need to have goals. I think goals are just unbelievably important. So if somebody is saying, and this is where nice to haves become difficult, right? So if Spotify's goal is to increase listening hours by 10% this quarter or whatever, I'm totally making up what that is, and you say, we're going to build this thing which is nice to have, the correct question is, Well, how much is it going to increase listening hours this quarter? And if the answer is five percent, and the team that is arguing for it is, you know, vociferous, like, really believes that, is really making the case, and they have credibility, then I think that as a product leader, depending on how large the organization is, if you're at a really big company, you might say, okay, great. Set that goal. You're gonna hit half of our time spent listening goal for the quarter with this project. Let's go build it. Let's see if it works, you know, and, and sometimes people will back down and say, oh, it's actually, you know, one percent or half a point, and sometimes they'll say, great, thank you. You know, sometimes they're right…

AI assessment note: “I don't really feel like those are the choices as I would frame them.”

Q When did you look at data, come to a conclusion, and that conclusion was really wrong?

A Maybe a different way of answering this, slightly different from the question that you're asking, I think. It's very hard to accurately interpret data, and it's very difficult to not lie to yourself with data. And, you know, you can cut data any way that you want. Um, I once shipped something into the, it's called the dive bar at the time at Facebook. It was like a social product, a social sharing surface, and the dive bar was this thing, if you swiped the Facebook app to the left, it opened up on the right. It was originally used, I think, for, uh, messenger contacts. I think something like 75% of Facebook users opened the dive bar, and so, There was an argument to be made that it was a great surface to ship into. The thing is that like, 99% of those people opened it by accident. And didn't want anything to do with it. And then immediately closed it. You know, so you can run all sorts of different analyses. You need to consider the entire, you know, set of things around you in order, in order to understand it. And, you know, I've made those kind of mistakes thousands of times.

AI assessment note: “75% of Facebook users opened the dive bar... 99% of those people opened it by accident.”

Q you, is kind of a great product line, but also an angel investor. And he likes a long road to the start line in terms of product, whereby it's actually quite difficult to get V-one out the door because there is so much to do. I'm just intrigued for you. How do you think about that long road to the start line versus get a quick V-one out, test, see?

A Well, so, you know, implicit, I guess, in some of what I was talking about earlier is this idea that I really believe that before you build something, you should have a theory of the world. You should, you know, understand who your user is, what they want to accomplish. You should understand your strengths and weaknesses in doing that and have a, you know, a theory of like the thing that you're going to build and how it is going to accomplish your goals and the user's goals. Um, and that might take you anywhere from 10 minutes to develop to a year to develop. You know, it's very, that's very variable. It really depends on what you're doing. I don't really think you should start writing code until you have that. Now, the problem is that most people's theory of the world, uh, you know, prior to shipping a thing and having contact with real live human customers is wrong, and it might be five percent wrong, or it might be 50% wrong, and you don't really know until you ship. You can do user research along the way, but, you know, the reality is that, that users lie when you talk to them, and they, they will always tell you what they think you want to hear. You know whether or not it's good because they'll come back or they won't come back. You know, if they churn, you were wrong. Then the thing that people often do, one of the big mistakes I think people make, is they then give up to…

AI assessment note: “I don't really think you should start writing code until you have that.”

Q How do you think about discussion within teams? Is talk good, or is dictatorial product vision better?

A I don't really feel like those are the choices as I would frame them. First of all, I think it's very important to have quantitative You know, objective measures of success. So when you're having a debate about something, about what you should do, you need to ground that somehow, you know, and you need to have goals. I think goals are just unbelievably important. So if somebody is saying, and this is where nice to haves become difficult, right? So if Spotify's goal is to increase listening hours by 10% this quarter or whatever, I'm totally making up what that is, and you say, we're going to build this thing which is nice to have, the correct question is, Well, how much is it going to increase listening hours this quarter? And if the answer is five percent, and the team that is arguing for it is, you know, vociferous, like, really believes that, is really making the case, and they have credibility, then I think that as a product leader, depending on how large the organization is, if you're at a really big company, you might say, okay, great. Set that goal. You're gonna hit half of our time spent listening goal for the quarter with this project. Let's go build it. Let's see if it works, you know, and, and sometimes people will back down and say, oh, it's actually, you know, one percent or half a point, and sometimes they'll say, great, thank you. You know, sometimes they're right…

AI assessment note: “I don't really feel like those are the choices as I would frame them.”

Q And what did you do to turn it around?

A Well, I think there are two things that are very important. Uh, you need to give people a win. You know, so Page Insights that we talked about earlier is an example of a win like that. You know, well, we, we can be proud about a thing that we shipped. Uh, and then I think you also have to give people a reason to work hard or a thing to believe. So, like, when I, uh, I worked on sharing, uh, at Facebook, which is like all the photos, text, videos, links, everything that ends up in newsfeed from regular people. So for a while I did ads, which was like, Everything from companies and celebrities, and then I moved over to, to, to people. You know, in both cases, there's like a weird thing. You can imagine people saying, I don't want to work on ads, because I don't want to work on the thing that, like, makes money, and, ah, you know, detracts from the experience, right? Um, you know, ads are annoying, and all this kind of stuff. You can reframe that, and you can say, actually, the thing that we're doing is we're helping individuals Discover the products that they, that, that are going to improve their lives, and we're helping businesses find customers, and the more efficient we can make that process, the better we can make that process, the more economic growth is going to happen in this world, and actually the thing that you're doing when you have a billion people looking at these a…

AI assessment note: “you need to give people a win... give people a reason to work hard”

Q Truly delightful. Do people not share that? Do people not tell their friends?

A Yeah, I think they do. So I, I think that maybe what you're getting at is there are different strategies for distribution. So we had this conversation at the office this morning. We had a marketing meeting this morning. We're, we're about to remove wait lists. We're about to really launch sling. And, you know, we had a conversation about whether we should do paid marketing or not. And, you know, we have a, we have some people who are already using the product and returning and using it a lot and, and giving us great feedback. And we had a long conversation about whether or not Our strategy in the short term should be, you know, a bunch of paid marketing or whether or not it should be, uh, you know, viral growth and just encouraging people to tell their friends. And, you know, I think we're going to do viral growth to start with, because to your point, if it is really great, people will tell their friends and they will tell each other. We may have to encourage them to do that. We may have to help them to do that in various ways.

AI assessment note: “Yeah, I think they do.”

Q When was your life most out of whack?

A Probably my first startup. Um, I don't know. I think you get into these phases where, you know, you become very lopsided, you know, and I've certainly worked on things. I have kind of an obsessive personality, you know, Sling's growing now, and, uh, we have a channel, we have a Slack channel called Pulse, where we see every sign up, and we see every transaction, and then, you know, we have a dashboard that updates regularly with You know, transaction volumes and all this stuff. Man, I, like, look at that Slack channel every four minutes, you know, every three minutes, and I have to stop myself from doing that, and that's the kind of stuff that I think is dangerous. That, that's what people mistake for grinding or working hard or sleeping on the factory floor. You know, when you're just doing the same thing over and over again, you're looking at, you know, you're looking at it and you're obsessing about the numbers or something. That's very different from, You know, having a detached view or a semi-attached view to how quickly you're growing, interrogating it, you know, understanding what you need to build or what you need to do differently in order to grow faster, and I think that people often mistake those two types of work for each other, if that makes sense, and that first kind of, like, obsessive refreshing of the thing or that obsession with, like, hours or the, or the, th…

AI assessment note: “Probably my first startup. Um, I don't know. I think you get into these phases”