The Exchanges

Q I, I really like one of the suggestions, which is, like, are there any learnings that could be generalized to other aspects of the business? Is there an example of one in the past that you could share Just to illustrate that a little bit.

A I'll give you a really tactical example. Um, a really tactical example was we were doing some very basic AB testing on our homepage, and we saw that a red button by far outperformed anything else. Now red is like, as a button is generally like a bad idea. Uh, it has a negative connotation. It's like something is wrong and don't hit the red button, but we couldn't ever find anything that outperformed on a pure AB testing basis, like that red button. Now it turned out when you actually went and looked at the data by segment, although it outperformed in general, it's severely underperformed for like the enterprise segment. Um, it's like a great example, like Simpson's paradox. When we took that information and like removed that from the web team and applied it to the content team and apply it to everyone else that was using the red button as a best practice. That was like incredibly valuable because most of the content we were generating was for enterprise. Most of the webinars we were generating were for enterprise. Like most of the direct mailers we were sending were for enterprise. Uh, so what we were actually doing by saying, Hey, red is the best button or, or sorry, the best practice was actually like decreasing the performance of all of these things we were doing that were specific for the enterprise segment.

AI assessment note: “we took that information and like removed that from the web team and applied it to the content team”

Q I, I really like one of the suggestions, which is, like, are there any learnings that could be generalized to other aspects of the business? Is there an example of one in the past that you could share Just to illustrate that a little bit.

A I'll give you a really tactical example. Um, a really tactical example was we were doing some very basic AB testing on our homepage, and we saw that a red button by far outperformed anything else. Now red is like, as a button is generally like a bad idea. Uh, it has a negative connotation. It's like something is wrong and don't hit the red button, but we couldn't ever find anything that outperformed on a pure AB testing basis, like that red button. Now it turned out when you actually went and looked at the data by segment, although it outperformed in general, it's severely underperformed for like the enterprise segment. Um, it's like a great example, like Simpson's paradox. When we took that information and like removed that from the web team and applied it to the content team and apply it to everyone else that was using the red button as a best practice. That was like incredibly valuable because most of the content we were generating was for enterprise. Most of the webinars we were generating were for enterprise. Like most of the direct mailers we were sending were for enterprise. Uh, so what we were actually doing by saying, Hey, red is the best button or, or sorry, the best practice was actually like decreasing the performance of all of these things we were doing that were specific for the enterprise segment.

AI assessment note: “When we took that information and like removed that from the web team and applied it”

Q My question to you on the number one, academically, is that not just learning playbooks? And do you worry that then you'll speak to your friend, George, who will tell you what worked at Samsara, but ramps are totally different business. And so respectfully, to your point on playbooks earlier, they're not applicable often. Does academic learning really work?

A I think it does, but it depends how you look at it. So yes, there are playbooks that you can just take and copy and paste. There are also playbooks that you can look at and think critically and adapt. And then there are playbooks to kind of come up with from scratch. I think a lot of academic learning is taking a past playbook and adapting it. And like the example that always comes to mind for me is in, in the world of attribution and marketing. So like media mix marketing, oh, sorry, media mix modeling or marketing mix modeling, depending upon who you talk to MMM. Like everyone's talking about it now is the best way to do attribution. Blah, blah, blah. Uh, that's a really old concept. That's like from the old, like mad men style advertising days in like the 19 fifties and sixties. So I think that's a good example where, Hey, you could actually go and look at how did they measure the impact of ads in literal newspapers before anything was digital, uh, back in like the 19 fifties or sixties. And how do you actually adapt that playbook or that methodology to modern day, like digital advertising or modern day digital marketing as a whole? Um, there's probably a million examples like that. Actually, direct mail is probably another example of that. There are so much.

AI assessment note: “I think it does, but it depends how you look at it.”

Q Um, how do you, how do you do it then? Help me.

A I, I think Samsara actually had like a great model for this. The CEO and founder, like he was very big on learning and very big on reading. And I remember like one time, uh, he was hanging out in like the cafeteria and we're like, what do you like to do in your free time? He's like, I like to read books. Uh, and so as a result, you saw how that permeated throughout the culture. So I remember they launched something when I was there called leadership principles and everyone that was a leader within the company, Got a big box shipped to their home and it had literally like 15 books in it and they're all business books. And the expectation was that you read one of these books every single month and then rejoin a conversation with other other peers. And this is all organized to discuss like one of the principles in these books that Samsara wanted you to embody. Uh, and then there was like an element where you actually then had to go put it into practice and demonstrate you were putting into practice. Like to me, that is a great example of being very intentional about Hey, we value learning and development here, and we're going to add a structure and we're going to add accountability to it to make sure that you are not just learning things, but putting it into practice. That took time. That took a lot of investment. Uh, that took really it coming from the founders. Um, and I think l…

AI assessment note: “I think Samsara actually had like a great model for this.”

Q What do I do is, what do I do as a leader to give you those best 30 days? Do I just say, just shadow the shit out of me? Do I say, go sit in support? Do, where do you go?

A Great question. I think this again goes back to, to the investing and learning. I think the leader needs to be incredibly detailed. In what those first 30, 60, 90 day plans look like. Who you're meeting, what you're doing with your time. Like, actually, this is something I learned from, from Samsara as well. Like, I remember my first 30 days were written out in excruciating detail. I think the first two weeks were completely scheduled out, like, to the minute for me. Uh, and as a result, it was also very clear to understand if someone was learning and picking things up, and it was very easy to compare folks if they were given the same, like, structured onboarding. Um, so I firmly believe, like, the first 30 days is, like, you learn the business and you learn the job. Now, beyond those 30 days, like, you should be able to start showing, like, some sort of step change impact, start having some ideas, start making things your own, and then by 90 days you should be starting to actually, like, show the fruits of those new ideas and the fruits of, like, that new experience that you're bringing or new perspective that you're bringing.

AI assessment note: “I think the leader needs to be incredibly detailed. In what those first 30, 60, 90 day plans look like.”

Q Now, listen, as we just said, I know a show is going to be great when I actually have to do very little work because you give me such great suggestions. You said to me before, growth is just science, and most marketers are bad at science. I was always terrible at science, so, you know, it doesn't bode well for me. What did you mean by that statement?

A The goal of growth is to figure out how to grow the business, and usually, like, early on, that's very top of funnel focus. How do you figure out how to get more leads? How do you figure out a channel that works and is repeatable and, Has, uh, predictable outputs, given some inputs. And the honest answer is like, no one really knows every business is different. So even if you understand from past experience, something that's worked, usually just taking like that one playbook or that one tactic and copy and pasting it to a new company, like generally doesn't work. And I think that's the tendency of, of a lot of marketers. So when I say that like growth is mostly just science, I mean that you kind of have to come in with a blank slate, form a hypothesis, and then run a bunch of experiments and. You'll be surprised about what works. You'll be surprised at what doesn't work. Um, but ultimately if you, if you run enough of those experiments, you'll, you'll find something. Uh, and I think that's usually lost on a lot of marketers because they don't think in terms of experiments. They think in terms of like, what do I know and how can I apply it here?

AI assessment note: “growth is mostly just science, I mean that you kind of have to come in”

Q If you just burn a shitload of cash on a channel super quickly, say you're spending, um, 10 K and you're like, shit, it works. Let's put 200 K on it. The 200 K will not be nearly as efficient as that 10 K. Would it not have been better to do 30, 50, 70, and gradually get up there than just whack it as hard as possible? I'm naive.

A So, so yes, but it depends. So if you're able to graph that response curve going from 10 K to 200 K, obviously you're probably not going to be able to go to 10 K to 200 K overnight. Um, if you were, that actually would probably be a good problem to have. Uh, but yes, if you're able to like graph that response curve and see when something goes from linear to start to decay in terms of response, that tells you, it's like, okay, we're no longer getting an expected output given the input. And then we have a conversation on if the returns are worth it. So figuring out the asymptote where things start to actually plateau is, is the most important thing. And I think most companies or most startups, at least they get to that, ask them to asymptote too slowly. And they should really be scaling much, much faster if they find something that works. Um, having said that, I think like most things saturate probably more slowly than people expect. Uh, and so going from 10 K to 200 K might not be as insane as it sounds. Uh, but it depends. You've got to watch it.

AI assessment note: “yes, but it depends. So if you're able to graph that response curve”

Q Is there a stage of company where it becomes interesting?

A I think if you, if you're seeing all of your core direct response channels start to saturate, then it's something that you have to start doing. And the assumption is that like investing in brand is going to do one of a few things. It's either going to make folks that have a problem and are aware they have a problem aware of you. And I think that's how probably most companies start thinking about it. But the more interesting aspect is like true demand generation. It's like, Hey, there are people out there that don't realize they have a problem and brand marketing, like, instead of being like, Hey, try XYZ product or try XYZ company. It's like this problem exists. Like there's actually a better way. And of course, like we're the better way. But that I think is like a much more interesting type of brand marketing to open up like demand for a new part of the market that you aren't able to reach with traditional channels.

AI assessment note: “if you're seeing all of your core direct response channels start to saturate”

Q Can I ask you, does growth sit in its own team or there? Are you in the product team? Are you in the marketing team? Where do you sit?

A I think my personal opinion is I think growth should be independent. I think like the growth team's mandate should be to figure out how to grow the business. And that should be more than just marketing. It should be more than just product. It should mean that like you have the mandate to do whatever is the highest leverage. And to me, that means you probably should report to I honestly, that's one of the reasons why I love ramp is the growth team reports to one of the co-founders. Um, I think some companies have chief growth officers. I think other folks have growth organizations that are more like SWAT teams and kind of roam between different parts of the business. Ultimately, it does depend on, on the business, but I think they should be as independent as possible.

AI assessment note: “I think my personal opinion is I think growth should be independent.”

Q What did you do that was sloppy that you wish wasn't sloppy?

A Um, I'm reminded of this one time. I won't say which company, uh, but we had a webpage and the webpage's conversion rate was trending down and it was trending down for a long time. It was our number one channel. And it was like, Hey, what are we, What are we going to do to solve this? And there were kind of two paths. There was one where we could run a bunch of individual well-controlled experiments and understand, Hey, changing this button or changing this H one is like, what's going to improve the page. And it worked or it didn't, or we could just run everything at once, kind of use our gut, kind of use our past experience and like hope that it worked. And we ended up going that, that latter route, uh, which was definitely the sloppy route, the less rigorous experiment. Um, it had ended up work. It did end up working. Uh, so we did end up like three X-ing the webpage conversion rate, like over the course of a couple of weeks and helped us hit our number that quarter. I say that that was like a mistake because we never actually knew what did or didn't work. And we had to go back and undo a lot of the, um, changes that we made once we did end up AB testing them. But I think that goes back to like the portfolio of bets. Like we were operating on a very short time horizon. So we had to like optimize for velocity versus like rigor. And then once we were no longer under the gun, we…

AI assessment note: “we ended up going that, that latter route, uh, which was definitely the sloppy route”

Q You said about running experiments there and running a number of them. Can I ask, in your mind, is growth about Increasing performance by one or two percent in many different areas, or is it about needle moving chapters of a company and being much more pivotal in that respect?

A The short answer is it has to be both, and what I mean by that is if you're doing a good job, and it depends on the stage of the company, but in general, if you're doing a good job, you're thinking in terms of different time horizons, so you have to have some, like, bucket of bets or experiments that are going to be those, like, big swing Huge step changes in impact, but those are generally high risk, high rewards. You can't just do that. Otherwise you're going to fail and miss your number this quarter. Um, at the same time, like you need to have some bets where you have high confidence, but probably not going to move the needle a ton, but it'll help you get the two, three, four, five percent improvement like this quarter. And then you have everything in between. So like ideally you're being intentional with how you're allocating your resources across like everything from like the very long term to the very short term. And how you make those bets and how you allocate those resources is actually a conversation you should probably have with finance, with leadership, and align it to the goals of the company. But that's my way of thinking about it.

AI assessment note: “The short answer is it has to be both”

Q Do you think early stage founders should look at LTV? Often CAC to LTV is kind of the hailed metric. George, you know, as well as I do, it's so difficult to calculate LTV in any product, let alone early stage products. How do you think about LTV and its utility value to early stage founders?

A I think it's a reasonable framework. I think you have to have some threshold that you agree on as a business. Like this is what we're willing to spend, and this is what we think a customer is worth. But the reality is it's like false precision. Like you're not going to know where your LTV is if you've been in business for a year, um, or six months or whatever it might be. So I wouldn't over index on that false precision. I would instead just acknowledge that there's some threshold we're going to be okay with, uh, in terms of spending to acquire a customer and what that customer is worth. And that should change over time as you learn things and run experiments.

AI assessment note: “I wouldn't over index on that false precision.”

Q Can we, can we actually? Because it's, it's like, it totally makes sense. Like, do things that people don't know, do things that people don't believe. How do you find them?

A Um, I think this actually is, is a good segue into, like, How do you just learn in general, uh, as a member of the growth team or someone in a company in general? I would say that there's three areas. Like you can learn academically. You can go read a book and learn what's worked for other companies or other growth folks or, uh, companies in the past, even our great area, like aspects of, of being able to learn. So learning academically is one aspect. Um, I think you can also learn from your peers so you can go and learn from folks at other companies, growth people at other companies. I think that's a great like area of, of, uh, of finding alpha. Um, granted, probably not the, the best, uh, unfair advantage since someone else knows about it, but I actually think the most interesting is to go and learn from other niches. So like other verticals, other geographies, especially if they're tangential to what you're doing. Uh, a great example is like WhatsApp. Like WhatsApp as a marketing channel is not huge for most brands, uh, or most companies in the United States, but it's massive, uh, in a lot of like international regions. And I think like, great. Could we go try like WhatsApp instead of. Sending emails. Um, it'll probably fail, but again, if you're doing enough of these experiments, you'll find something that works that no one else is doing.

AI assessment note: “most interesting is to go and learn from other niches. So like other verticals”

Q What did you see in direct mail that no one else saw? Because so many of your friends were like, George crushed it on direct mail, and no one thought this was a good idea. So what did you see that no one else saw?

A No one else was doing it, and it was incredibly scalable. Like, that was pretty much it. Like, the fact that no one else was doing it, it's like, okay, that's interesting, we should try it. But the fact that if it works, it'd be incredibly scalable, and the fact that you can run it with very large sample sizes meant that we could run a lot of experiments in parallel and learn really, really quickly. Like, there's very few channels where you can go, like, Hey, tomorrow let's go reach like 200,000 people. Uh, direct mail, email, those are like kind of some of the only channels that, that would allow you to do something at that scale, and that's run a lot of experiments.

AI assessment note: “No one else was doing it, and it was incredibly scalable.”

Q And so you do a, sorry, just so I understand, because so many founders get really granular, and I get notebooks out. For, like, pre-mortems, this is right before we're about to start, we plot out the three main things that could likely kill this project?

A That's one way to do it. Uh, I would actually get more specific than that. When you're planning the experiments, like, why would this fail? It's like, oh, we're not going to have a large enough sample size as we predicted, or like, there's a million things. You should probably write out those million things. And then I think what's more interesting is when you do a post-mortem, if the experiment failed for something that you didn't actually anticipate, something that you didn't like factor into your, your experimental design, that I think is an interesting conversation. But if it's something that you would kind of anticipated and maybe the probability was wrong or, Whatever it might be. That is less interesting, and I probably wouldn't even do a post-mortem for that.

AI assessment note: “That's one way to do it. Uh, I would actually get more specific than that.”

Q Is there any profile before we actually dig into skills and skill detection? Is there any profile where you're like, I don't love that background. It's just tough to be good at growth with that background.

A Yeah. If you've been at a company, especially a company that's an order of magnitude or larger, bigger for more than a few years, it's, it's just really hard to then change your mindset and go to a smaller startup and think about like how hard you have to like get your hand or how much you have to get your hands dirty and how differently you need to think about things. Like, I think if you've been at a much, much larger company, you're going to tend to think more in terms of playbooks and, and rely more on your past experience. Then on what you can learn from others or what you can learn from, from maybe like other geographies, other companies or other peers. Uh, so going from playbook to first principle thinking is generally difficult to do. So I would, I would, I personally like stayed away from that type of profile.

AI assessment note: “If you've been at a company, especially a company that's an order of magnitude or larger”

Q Howard Schultz's book on Starbucks and leadership lessons from Starbucks. I run a fucking media company in a world of TikTok and Deep Seek. I mean, it's completely different to the point on playbooks. The leadership principles probably don't align. Most leadership books written 20 years ago do not take account for a post-COVID millennial generation. How do we think about applicability and actually the lessons we teach being wrong?

A I don't think business has actually changed that much tactically. Sure. Maybe it has the channels TikTok didn't exist 30 years ago, but I don't actually think business and being a good manager in particular has changed that much, or the skills necessary to be a good manager have changed that much. A great example is like the book, the goal. Like I I'm a big fan of that book. I have some recency bias because we reread it recently, but one of the key concepts in that is like the theory of constraints. And now this, this doesn't have anything to do with management, but the concept of the theory of constraints existed 40 years ago, and it still exists today. Like every team is operating with some bottleneck in their process and being able to identify that bottleneck and remove it is a very valuable skill for anyone, but it's especially valuable for a manager. Uh, that's how you get the most out of people. That's one of the core jobs of being a manager. So that's just one example. Like I would argue that most business books, as long as you are vetting them well, and as long as like you are being intentional on what is the principle you want to pull out of this and have your team put into practice. There's probably something that can be learned from, from almost any book, regardless of how old it is.

AI assessment note: “I don't actually think business and being a good manager in particular has changed that much”

Q Okay. Let's assess take home assignment. What do you like to give as a take home assignment and what advice would you give for me?

A Yes. Uh, make it as real as possible and make it quantitative. Uh, and actually maybe the third thing I'd say is understand what good looks like before you send the test out. So a great example is I like to just take a Salesforce dump or a dump of data and be like, Hey, what's the best campaign? What worked best here? Um, what were the best leads, whatever it might be. And if it's real world data, it's going to be really messy. Like there's going to be a bunch of tricks they have to catch. They're going to be a bunch of mistakes. They have to figure out duplicate leads or, um, the dates don't align or missing data or whatever it might be. So I think like that itself is like, can they work with real world data is I think a really interesting question to answer. And then the other is like, how much of this did they think through? What was their thought process? Like how quickly were they able to adapt? How quickly were they able even to like do the tests? Did they ask questions about it? All of those I think are signals before you even see the output. Uh, to understand, like, are they at least thinking about things in the right way? But the real world is messy. They should, they should work with real world data.

AI assessment note: “I like to just take a Salesforce dump or a dump of data”

Q What are the most common reasons growth hires don't work?

A I think like the number one reason is it was, it was on the manager. It was a mishire. Uh, they didn't scope the role effectively or the role changed. The needs of the business changed and they hired the wrong person, uh, the wrong profile. I think generally managers try to come up with like a laundry list of skills that they want to hire for. And they try to find someone that checks most of those boxes, but That's almost the easy way to do things. I think the much harder way is to get really, really crystal clear on what is the one thing that we need, like what is the one skill or trait or piece of experience, one problem we need to solve and hiring someone that you have high confidence can do that or fill that gap versus check a bunch of boxes. So not getting clear on that, I think is the number one reason why you would miss hire. And that's mostly on the manager, if I'm being honest.

AI assessment note: “they didn't scope the role effectively or the role changed.”

Q What do I do is, what do I do as a leader to give you those best 30 days? Do I just say, just shadow the shit out of me? Do I say, go sit in support? Do, where do you go?

A Great question. I think this again goes back to, to the investing and learning. I think the leader needs to be incredibly detailed. In what those first 30, 60, 90 day plans look like. Who you're meeting, what you're doing with your time. Like, actually, this is something I learned from, from Samsara as well. Like, I remember my first 30 days were written out in excruciating detail. I think the first two weeks were completely scheduled out, like, to the minute for me. Uh, and as a result, it was also very clear to understand if someone was learning and picking things up, and it was very easy to compare folks if they were given the same, like, structured onboarding. Um, so I firmly believe, like, the first 30 days is, like, you learn the business and you learn the job. Now, beyond those 30 days, like, you should be able to start showing, like, some sort of step change impact, start having some ideas, start making things your own, and then by 90 days you should be starting to actually, like, show the fruits of those new ideas and the fruits of, like, that new experience that you're bringing or new perspective that you're bringing.

AI assessment note: “leader needs to be incredibly detailed. In what those first 30, 60, 90 day plans look like.”

Q You said about running experiments there and running a number of them. Can I ask, in your mind, is growth about Increasing performance by one or two percent in many different areas, or is it about needle moving chapters of a company and being much more pivotal in that respect?

A The short answer is it has to be both, and what I mean by that is if you're doing a good job, and it depends on the stage of the company, but in general, if you're doing a good job, you're thinking in terms of different time horizons, so you have to have some, like, bucket of bets or experiments that are going to be those, like, big swing Huge step changes in impact, but those are generally high risk, high rewards. You can't just do that. Otherwise you're going to fail and miss your number this quarter. Um, at the same time, like you need to have some bets where you have high confidence, but probably not going to move the needle a ton, but it'll help you get the two, three, four, five percent improvement like this quarter. And then you have everything in between. So like ideally you're being intentional with how you're allocating your resources across like everything from like the very long term to the very short term. And how you make those bets and how you allocate those resources is actually a conversation you should probably have with finance, with leadership, and align it to the goals of the company. But that's my way of thinking about it.

AI assessment note: “The short answer is it has to be both”

Q Do CACs get cheaper over time as brand becomes better known, you become more branded in an ecosystem? Or do they get more expensive as you saturate the core target market and you have to expand into maybe less directly relevant ICPs?

A The only right answer is that like, yes, CACs become more expensive. Like as you get more and more market share, it makes sense that it's every incremental acquisition is going to cost more than previous. Um, having said that, I think the reality is that's usually not true. Usually you figure out new products to sell that actually make the LTVs improve. You figure out new geographies to break into. You figure out new channels that work. You figure out maybe that, hey, actually combining different channels has a halo effect and you're not fully capturing that and what the customer acquisition cost is. So the reality is that, no, there's, it takes a really long time to get to the point where like the macro effect of saturation is hitting your CAC. Um, but that's how I think about it.

AI assessment note: “the reality is that, no... it takes a really long time”

Q Can I ask you, does growth sit in its own team or there? Are you in the product team? Are you in the marketing team? Where do you sit?

A I think my personal opinion is I think growth should be independent. I think like the growth team's mandate should be to figure out how to grow the business. And that should be more than just marketing. It should be more than just product. It should mean that like you have the mandate to do whatever is the highest leverage. And to me, that means you probably should report to I honestly, that's one of the reasons why I love ramp is the growth team reports to one of the co-founders. Um, I think some companies have chief growth officers. I think other folks have growth organizations that are more like SWAT teams and kind of roam between different parts of the business. Ultimately, it does depend on, on the business, but I think they should be as independent as possible.

AI assessment note: “I think growth should be independent. I think like the growth team's mandate should be”

Q Is there a stage of company where it becomes interesting?

A I think if you, if you're seeing all of your core direct response channels start to saturate, then it's something that you have to start doing. And the assumption is that like investing in brand is going to do one of a few things. It's either going to make folks that have a problem and are aware they have a problem aware of you. And I think that's how probably most companies start thinking about it. But the more interesting aspect is like true demand generation. It's like, Hey, there are people out there that don't realize they have a problem and brand marketing, like, instead of being like, Hey, try XYZ product or try XYZ company. It's like this problem exists. Like there's actually a better way. And of course, like we're the better way. But that I think is like a much more interesting type of brand marketing to open up like demand for a new part of the market that you aren't able to reach with traditional channels.

AI assessment note: “if you're seeing all of your core direct response channels start to saturate”

Q What are the most common reasons growth hires don't work?

A I think like the number one reason is it was, it was on the manager. It was a mishire. Uh, they didn't scope the role effectively or the role changed. The needs of the business changed and they hired the wrong person, uh, the wrong profile. I think generally managers try to come up with like a laundry list of skills that they want to hire for. And they try to find someone that checks most of those boxes, but That's almost the easy way to do things. I think the much harder way is to get really, really crystal clear on what is the one thing that we need, like what is the one skill or trait or piece of experience, one problem we need to solve and hiring someone that you have high confidence can do that or fill that gap versus check a bunch of boxes. So not getting clear on that, I think is the number one reason why you would miss hire. And that's mostly on the manager, if I'm being honest.

AI assessment note: “I think like the number one reason is it was, it was on the manager.”

Q You said there about bets, and I often think about growth very much like venture, which is you place a number of investments slash bets, and you observe, and then you wait to see what works and double down. Do you agree with that analogy? How do you think about what is enough bets and failure rate associated?

A Ooh, yes. So it's absolutely a portfolio, but I think like an investment portfolio, it depends what you're optimizing for, and it depends on the stage of the company. So your risk tolerance is going to depend on the stage of the company. It's going to depend on if you're optimizing for growth or profitability or, or whatever it might be. So it is a portfolio. Um, I think that you should assume if you're doing things right, you should assume the majority of your bets are going to fail, which is why I always believe that velocity is probably more Important than getting things perfect. Uh, but it is a spectrum. Like, you could do some really well-controlled, rigorous experiments, and it's gonna be incredibly academic, and you're gonna learn something, but it might take you, like, a year to, like, say something conclusive. Um, or you can just run a bunch of experiments at an incredibly high velocity, and it's kind of sloppy, and similarly, you might actually, you might not learn anything, uh, because you made a bunch of mistakes, you didn't fully think through how you'd measure something. So you kind of have to balance it, but in general, I'd probably be biased to Run thing, run more things faster than run something perfectly.

AI assessment note: “So it's absolutely a portfolio, but I think like an investment portfolio”

Q How do you think about enough time, but not being slow?

A It goes back to the portfolio. Like if you're going to allocate 20, 30% of your time to like longer term bets, that's fine. Be okay with the fact that you're not going to get results. Anytime soon. Um, having said that, it would be great if you could figure out what some leading indicators are or scope down the experiment. So you could get some signal that like, Hey, we have confidence this will or won't work. I think in general, I've always tried to prioritize experiments based on impact and effort. I think those are obvious ones, but also confidence in time to results. Like if you're really confident, something's going to work, you should just do it. Um, if you're really unconfident that something's going to work, but the time to results is really fast. You should do that as well. And I think usually those two dimensions are lost when people are prioritizing. They tend to just go for impact and effort and not think about confidence or time to results.

AI assessment note: “if you're going to allocate 20, 30% of your time to like longer term bets”

Q Do you think early stage founders should look at LTV? Often CAC to LTV is kind of the hailed metric. George, you know, as well as I do, it's so difficult to calculate LTV in any product, let alone early stage products. How do you think about LTV and its utility value to early stage founders?

A I think it's a reasonable framework. I think you have to have some threshold that you agree on as a business. Like this is what we're willing to spend, and this is what we think a customer is worth. But the reality is it's like false precision. Like you're not going to know where your LTV is if you've been in business for a year, um, or six months or whatever it might be. So I wouldn't over index on that false precision. I would instead just acknowledge that there's some threshold we're going to be okay with, uh, in terms of spending to acquire a customer and what that customer is worth. And that should change over time as you learn things and run experiments.

AI assessment note: “I think it's a reasonable framework. But the reality is it's like false precision.”

Q And so you do a, sorry, just so I understand, because so many founders get really granular, and I get notebooks out. For, like, pre-mortems, this is right before we're about to start, we plot out the three main things that could likely kill this project?

A That's one way to do it. Uh, I would actually get more specific than that. When you're planning the experiments, like, why would this fail? It's like, oh, we're not going to have a large enough sample size as we predicted, or like, there's a million things. You should probably write out those million things. And then I think what's more interesting is when you do a post-mortem, if the experiment failed for something that you didn't actually anticipate, something that you didn't like factor into your, your experimental design, that I think is an interesting conversation. But if it's something that you would kind of anticipated and maybe the probability was wrong or, Whatever it might be. That is less interesting, and I probably wouldn't even do a post-mortem for that.

AI assessment note: “That's one way to do it. Uh, I would actually get more specific than that.”

Q Can we, can we actually? Because it's, it's like, it totally makes sense. Like, do things that people don't know, do things that people don't believe. How do you find them?

A Um, I think this actually is, is a good segue into, like, How do you just learn in general, uh, as a member of the growth team or someone in a company in general? I would say that there's three areas. Like you can learn academically. You can go read a book and learn what's worked for other companies or other growth folks or, uh, companies in the past, even our great area, like aspects of, of being able to learn. So learning academically is one aspect. Um, I think you can also learn from your peers so you can go and learn from folks at other companies, growth people at other companies. I think that's a great like area of, of, uh, of finding alpha. Um, granted, probably not the, the best, uh, unfair advantage since someone else knows about it, but I actually think the most interesting is to go and learn from other niches. So like other verticals, other geographies, especially if they're tangential to what you're doing. Uh, a great example is like WhatsApp. Like WhatsApp as a marketing channel is not huge for most brands, uh, or most companies in the United States, but it's massive, uh, in a lot of like international regions. And I think like, great. Could we go try like WhatsApp instead of. Sending emails. Um, it'll probably fail, but again, if you're doing enough of these experiments, you'll find something that works that no one else is doing.

AI assessment note: “I actually think the most interesting is to go and learn from other niches.”