The Exchanges

Q be a little bit different now, Brett. Like, how does that work? Do you know what I mean? It's like, so just help me understand. You decided you're going to do Sierra, and you're like, ah, what? Yeah, I mean, I guess it's kind of a question of, like, why fundraise? But then there's also a question of, like, how did you approach that? Now you could raise from anyone.

A Well, first, why did I fundraise? Um, I really believe in the importance of boards and having stakeholders and the accountability. Of, you know, having a board and investors and employees, and I want the employees coming to Sierra to know that Clay and I aren't doing this as a side hustle. You know, we want to build a generational company. And then similarly, I really value the advice. I've been a board member as well as an executive, and I really value the strategic advice I got. So when we started the company, I just called Peter Fenton, who I've worked with twice before. He's the only person I talked to. And, you know, that, that was our first board member. And with our subsequent round, similarly.

AI assessment note: “why did I fundraise? Um, I really believe in the importance of boards”

Q kind of the value that could come from that. Absolutely. There is kind of a step before that though, which is, you know, the models themselves are actually so good and so advanced that they bundle all verticalized or unbundled software products really and subsume them, so to speak. To what extent do you think that is a threat that everything will really just be subsumed by very sophisticated models?

A I don't believe that will happen personally. Um, I, Going back to, uh, analogies are dangerous, but I, I think they might be illustrative in this case. I actually think the AI market commercially will play out like the cloud market did over the past 20 years. So if you look at the cloud market, I would say there's really big, three big categories of cloud software. The first is infrastructure as a service. So Amazon web services, Azure, Google cloud services like that there's tool makers. So, you know, snowflake, Databricks, Datadog, you know, basically what is the software that you need confluent? What is the software that you need when your company is moving to the cloud? And then there's software as a service. So Salesforce service now Adobe, um, and the extremely long tail of solutions there. And I would say, you know, we were talking about the companies, the public companies in the stock market and that kind of two billion to twenty billion dollar range. There's a huge number of, Really interesting and really valuable software as a service solutions. Why did that play out that way? Um, you know, one could argue and, and, you know, certainly I heard isn't, isn't Salesforce just a database in the cloud. I'm like, come on, you know, like it's a solution, you know, it's a solution for sales service and marketing teams, and it has a ton of value and the same, you know, um, redu…

AI assessment note: “I don't believe that will happen personally.”

Q I know this one's a little bit off back because it's not even on the schedule, so you're like, you're breaking the rules from round one, but when I go through the different achievements you have, it really is incredible. When you were young, did you know that you were going to be successful? Did you have that innate feeling?

A I don't think so. You know, when I was young, first I wanted to be Indiana Jones, which I know is not a job, but to me, he was by far the coolest example of an adult that I'd ever seen. By the time that I, uh, you know, was in school and started thinking about a job, I wanted, I thought I wanted to be an attorney, um, in high school. And, uh, I'm happy to tell the story. It's actually kind of an interesting story, but I ended up getting a job at a gas station and then, uh, Sort of hustling my way into making a website for a mechanic that was nearby. Um, I was getting paid four dollars and 25 cents at the gas station an hour, which was minimum wage at the time. And, uh, ended up getting paid 400 dollars for the website. So I quit the gas station job the next day and ended up making websites for a lot of local businesses in my, my hometown. Most of those websites endured for decades, you know, because it turns out if you're a florist, it's not like you Actively SEO in your website. So, you know, my, my fingerprints on the internet in 1996 and 97 lasted for longer than you'd expect. And even when I went to Stanford, I, um, I think if you'd met me that summer before, I probably would have said, I probably want to be a lawyer, but then the combination of my accidental entrepreneurial, um, experience, plus going to Stanford in the dot-com bubble, I, my first quarter at Stanford, I to…

AI assessment note: “I don't think so. You know, when I was young, first I wanted to be”

Q Um, I'm here, Elon. I'm ready to write the check. Um, my question to you is, Are we in peak AI and is this the ultimate sign of a bubble?

A I think we are in a bubble, but I think, um, bubbles have different shapes. Um, and there's a Mark Twain quote that history doesn't repeat itself, but it rhymes. And I think the AI bubble will rhyme with the dot com bubble. And I believe with the benefit of hindsight, Most of the excess of the.com bubble might have been justified. Um, if you look at the top market cap companies in the world, they include Amazon. They include Google. Um, you know, if you look at, uh, across segments, it's, uh, PayPal, eBay. If you look at the, you know, enterprise software companies like Salesforce started in the, uh, if I'm remembering correctly, All of these companies were started in the.com bubble. And I think people associate mentally and emotionally, the.com bubble is associated with web van and pets.com. But actually, if you look at the most frothy statements about the.com bubble and the transformation of the economy and the fast forward almost 30 years from that point, maybe it was true. Uh, you know, when you look at how much Amazon disrupted commerce, How much, uh, you know, uh, consumer payments have, have been transformed by digital technology. It took a few waves of, of technology like smartphones and NFC to really, you know, fully, um, realize that vision and, you know, a huge percentage of, you know, the gains in the stock market over the past 30 years have more or less been these …

AI assessment note: “I think we are in a bubble, but I think, um, bubbles have different shapes.”

Q kind of the value that could come from that. Absolutely. There is kind of a step before that though, which is, you know, the models themselves are actually so good and so advanced that they bundle all verticalized or unbundled software products really and subsume them, so to speak. To what extent do you think that is a threat that everything will really just be subsumed by very sophisticated models?

A I don't believe that will happen personally. Um, I, Going back to, uh, analogies are dangerous, but I, I think they might be illustrative in this case. I actually think the AI market commercially will play out like the cloud market did over the past 20 years. So if you look at the cloud market, I would say there's really big, three big categories of cloud software. The first is infrastructure as a service. So Amazon web services, Azure, Google cloud services like that there's tool makers. So, you know, snowflake, Databricks, Datadog, you know, basically what is the software that you need confluent? What is the software that you need when your company is moving to the cloud? And then there's software as a service. So Salesforce service now Adobe, um, and the extremely long tail of solutions there. And I would say, you know, we were talking about the companies, the public companies in the stock market and that kind of two billion to twenty billion dollar range. There's a huge number of, Really interesting and really valuable software as a service solutions. Why did that play out that way? Um, you know, one could argue and, and, you know, certainly I heard isn't, isn't Salesforce just a database in the cloud. I'm like, come on, you know, like it's a solution, you know, it's a solution for sales service and marketing teams, and it has a ton of value and the same, you know, um, redu…

AI assessment note: “I don't believe that will happen personally.”

Q it next week. Gemini kills it next week. Open AI has crushed it. And I'm sitting here like, fuck, I'm getting dizzy. Like, which one should I use? Oh my God. And then Claude comes and it's like five things you can do with Claude that you can't do with anything else. And I'm like, Christ, I've got no idea what's going on. Are they the fastest technology to commoditize?

A I will start with the high level. I really like Reid Hoffman's framing of this market as foundation models and frontier models. So foundation models are any of these large language models that aren't necessarily the best of the best or the higher per highest parameter count, but particularly now, uh, where you have relatively low parameter count models that are meet or exceed the quality of say GPT, 3.5. Uh, that market of foundation models is quite important, um, and quite commoditized. You know, I think that in that market, uh, probably if you need a model like that, you should download llama. That's, that's the answer. It's like, you don't need much of a cheat sheet on that, you know, and, um, or maybe Mr. All, but pick one of the open source models that are adequate and fine tune it. The frontier model market is a little different when you talk about this, um, you know, the, the experience you've had being dizzy using these tools. My perspective is that, uh, we've seen real leaps there. So when chat GPT came out, that was a meaningful step function change that lasted for a while. Um, and the insight around instruction tuning and the quality of sort of the GPT models after GPT three was pretty remarkably different. Similarly, when GPT four came out, I, I haven't done the math on it, but it certainly had a meaningful lead for quite a while. Um, and now you're seeing a lot of,…

AI assessment note: “that market of foundation models is quite important, um, and quite commoditized.”

Q exact same. And also like, I think everyone took my entrepreneurship away from it. And I was like VC. I literally don't think anyone has the, the view of leaders that you've had working alongside Zuck. Benny off, uh, board of Shopify with Toby, uh, board of OpenAI with Sam. This is the greatest leaders of a generation. What do they have that is non-obvious that makes them great leaders?

A One of the things that I've admired the most about the leaders you mentioned, whether it's Larry and Sergey, Mark Benioff, Mark Zuckerberg, uh, Marissa, whom I worked for at, at Google, um, is this sort of relentless drive. Every time you might, uh, get comfortable with a situation, they're all looking out towards the horizon. Uh, I always found Mark Zuckerberg particularly remarkable at this. Uh, Every time I thought I was thinking long-term, whatever Mark was thinking was about two X farther in the future than I was thinking. And, you know, it was so, uh, disconcerting and motivating for me, uh, when I was there, uh, I think when I became chief technology officer of Facebook, after they had acquired my social network, I was 29. If I'm remembering correctly, I think it was, uh, 2009. Um, and see how his brain worked to definitely change my perspective on what Bold leadership meant and taking bets that could have been unpopular or complex in the short term to achieve a long-term goal. And I think you really see with some of the great entrepreneurs, this ability to think extremely long-term and make decisions, uh, that, you know, especially if you're nowadays, if you're a public company, it's such a challenging, uh, you know, cadence to, to, uh, Parade yourself out in front of investors every three months. And, you know, while investors claim to be long-term, very few have the p…

AI assessment note: “this ability to think extremely long-term and make decisions”

Q Um, I'm here, Elon. I'm ready to write the check. Um, my question to you is, Are we in peak AI and is this the ultimate sign of a bubble?

A I think we are in a bubble, but I think, um, bubbles have different shapes. Um, and there's a Mark Twain quote that history doesn't repeat itself, but it rhymes. And I think the AI bubble will rhyme with the dot com bubble. And I believe with the benefit of hindsight, Most of the excess of the.com bubble might have been justified. Um, if you look at the top market cap companies in the world, they include Amazon. They include Google. Um, you know, if you look at, uh, across segments, it's, uh, PayPal, eBay. If you look at the, you know, enterprise software companies like Salesforce started in the, uh, if I'm remembering correctly, All of these companies were started in the.com bubble. And I think people associate mentally and emotionally, the.com bubble is associated with web van and pets.com. But actually, if you look at the most frothy statements about the.com bubble and the transformation of the economy and the fast forward almost 30 years from that point, maybe it was true. Uh, you know, when you look at how much Amazon disrupted commerce, How much, uh, you know, uh, consumer payments have, have been transformed by digital technology. It took a few waves of, of technology like smartphones and NFC to really, you know, fully, um, realize that vision and, you know, a huge percentage of, you know, the gains in the stock market over the past 30 years have more or less been these …

AI assessment note: “I think we are in a bubble, but I think, um, bubbles have different shapes.”

Q can be learned? So like, for me, people think, oh, I could never be an interviewer or do content. And I just say, listen, I was shit. I've just done 3000 over 10 years. I learned to be a little bit better, hopefully, so I'm now tolerable. But it's just like going to the gym. What do you think everyone thinks is an innate skill that actually can be learned?

A Leadership. I actually have made so many, I mean, you've probably heard the phrase, oh, this person's such a natural leader. And I clearly, if you're a sociopath, you probably won't be a great leader, you know, but if you have decent amount of EQ, uh, the innate skills of becoming a leader at different scales of organizations is absolutely something I believe can be learned. It's interesting because if you meet, uh, people who have served in the armed Forces, uh, in the Western world. Most of the military treats leadership as a craft that you learn. Um, and it's in part because of your, you know, uh, growing through the ranks of say the army, you know, at each step, you're managing larger and larger, uh, groups of, of soldiers. And, you know, they, they formalized a lot of like principles of leadership. In contrast, I, I think the, you know, if you go into most large companies and you go into You know, a promotion discussion, you know, corporate promotion discussion, like that person is just not a natural leader, not that person needs to, you know, uh, train or learn these skills. And I think we've, um, well, it's not true of all of corporate America. Um, I do think it's one of those things that I think, you know, companies should invest in more, uh, which is like formal training of characteristics of leadership, uh, how to motivate, you know, uh, people who are different than …

AI assessment note: “Leadership. I actually have made so many, I mean, you've probably heard the phrase”

Q the next three to five years will actually be the biggest winners in AI. And you've seen a lot of these consulting firms post Billions in profit. There was one that actually had more revenue than open AI. Do you agree that AI services companies will be a dominant strain of this community and that they will be needed though for the implementation of this next generation of application layer?

A In the early days of technology adoption, you tend to have Very low level platform building blocks available and quite a bit of professional services spend because there is no option other than building it yourself. So you tend to get a short term spike in professional services spend along with some low level building blocks. And my guess is at least some of that revenue you're describing is companies not having an out of the box software as a service solution available. They see these amazing models like GPT for available. And if they want to apply them to their business, you know, a year and a half ago, two years ago, their only option was essentially to pay, um, one of these firms to, to do the last mile themselves over time. I do think that that will diminish as solutions become available that, um, have shorter and simpler implementations. I think that's what companies like mine are doing is essentially You know, reducing the, the last mile to actually configure the software. However, the reason I think, you know, this is nuanced and, and you may be right. And, and actually I think it can be a, a lot of value that professional services firms provide is around change management. So if you manage, uh, imagine you have a contact center, uh, in the Philippines, uh, managed, you know, as a, as a BPO with one of these customers, And you're migrating, you know, huge percentage of …

AI assessment note: “I do think that that will diminish as solutions become available”

Q We mentioned the commoditizations of foundation models as a technology. We've also seen price dumping and a race to the bottom in terms of price as well in a lot of cases. How do you think about AI business models that are sustainable given incredible training and inference costs?

A So when I made the comment earlier about skeptical of companies doing pre-training, it was really based on the premise that most companies should be Applying AI to build solutions and most companies should have relatively modest training costs and most of their costs should be correlated with inference, which should be correlated with revenue and usage of your product. Uh, and, and I think that that's an essentially because if you end up pre training a very large model, you know, you end up with some such upfront capital requirements. You have to have a really valuable business model on the other side of that to justify that investment. So first I think companies should really focus on how to find product market fit, you know, prior to taking on meaningful training costs that are fine tuning might be fine. Uh, you know, but you have certainly sort of pre-training models on the inference side. Um, I actually think, uh, the costs of AI are going down really, really rapidly. Um, I've seen a lot of people tracking sort of The cost of the GPT models over time. And what's remarkable about the cost going down is the quality is also going up, you know, so it reminds me, you joked about when you were born, but, uh, but well, around the time you were born, every time I got a new computer in my house, it was twice as cheap and twice as good. So I think, you know, on the inference side, I …

AI assessment note: “most companies should have relatively modest training costs and most of their costs should be correlated with inference”

Q Why do you, why do you need the phone at all? If I can just talk to myself, which would look kind of weird, but normal because I do often as the venture ambassador with too much free time, clearly, uh, like, you know, I could ask myself, Hey, like get an Uber. Uh, I I'm here. Do we see the removal of the phone?

A It certainly seems feasible, but it's, I temper that with the, if you look at the past 15 years of consumer electronics innovation, how many companies, including the ones that make smartphones, have tried to make devices that, you know, replaced or augmented the phone unsuccessfully. You know, this, this device here, this phone, it's so good at so many things, and everyone already has one. It's essentially, Completely removed the market for almost every other type of consumer device. Um, so in the short term, my intuition is that the combination of a smartphone with Ray-Ban glasses or AirPods or the like, um, probably, you know, meaning you might need to look at your screen less, uh, than you do today. But my intuition is because of the prevalence of smartphones around the world, it will still end up being, you know, the, the primary Computer that mediates those conversations, but to the point that you made, you know, as conversational experiences start working more, the, I always get the big phone, you know, just cause I like the big screen. You know, I, I think a lot changes and, you know, I always go back to the early app store days and the early apps being such skeuomorphic apps like flashlights. And then you have the mobile native experiences like WhatsApp door dash, you know, Uber, Um, Instacart. It took one generation for, for those things to, to really, um, exist. I hav…

AI assessment note: “it will still end up being, you know, the, the primary Computer”

Q Do you think people are born entrepreneurs, or do you think it can be learned?

A I think most things can be learned. Uh, I've Through certainly through my career, I have most of the time when I've thought of something as an innate skill, um, I've later recalibrated and felt that with like enough focus, one can, um, improve at most things. I, I could never become like an Olympic track star, nor could I ever, you know, win the fields medal. So I, I, obviously I don't mean to trivialize true innate ability, but on skills like public speaking or leadership, uh, or, Um, even things that aren't quite fields metal, like becoming, you know, good at finance. I think most people can with, with focus. The thing that is unusual about being an entrepreneur is, um, how intense it is. And I do think there's a certain personality type that is conducive to that. You know, I think, uh, it's hard to be an entrepreneur if you're prone to anxiety because everything's on fire all the time. You know, that's just the nature of the business. Um, and, uh, As a, as a consequence of probably there's certainly some nature, not just nurture there, but I have met folks who, you know, might not have identified as an entrepreneur earlier in their career, who develop the confidence in their own, you know, resolve, uh, through early parts of their career and end up great. And it's really interesting too. If you look at the enterprise software industry versus, uh, consumer, you know, a lot of…

AI assessment note: “I think most things can be learned.”

Q the next three to five years will actually be the biggest winners in AI. And you've seen a lot of these consulting firms post Billions in profit. There was one that actually had more revenue than open AI. Do you agree that AI services companies will be a dominant strain of this community and that they will be needed though for the implementation of this next generation of application layer?

A In the early days of technology adoption, you tend to have Very low level platform building blocks available and quite a bit of professional services spend because there is no option other than building it yourself. So you tend to get a short term spike in professional services spend along with some low level building blocks. And my guess is at least some of that revenue you're describing is companies not having an out of the box software as a service solution available. They see these amazing models like GPT for available. And if they want to apply them to their business, you know, a year and a half ago, two years ago, their only option was essentially to pay, um, one of these firms to, to do the last mile themselves over time. I do think that that will diminish as solutions become available that, um, have shorter and simpler implementations. I think that's what companies like mine are doing is essentially You know, reducing the, the last mile to actually configure the software. However, the reason I think, you know, this is nuanced and, and you may be right. And, and actually I think it can be a, a lot of value that professional services firms provide is around change management. So if you manage, uh, imagine you have a contact center, uh, in the Philippines, uh, managed, you know, as a, as a BPO with one of these customers, And you're migrating, you know, huge percentage of …

AI assessment note: “professional services firms provide is around change management.”

Q Brett, what was the hardest thing with Sierra that you did not anticipate being so hard?

A I'll describe a technology problem and then I'll describe the human problem that was harder than I expected around it. So generative AI is very creative, but inherently non-deterministic. Uh, you know, there, it's very hard to create determinism, the same inputs creating the same outputs in particular, because if you think about the breadth of human language, you know, and not, it's just inherently less precise than, than most. And then similarly, if you afford Um, AI, the ability to reason, um, you know, sort of by definition, you can't enumerate all the possible outcomes from there. So when you're building industrial grade agents, you know, for businesses that have real business rules they need to follow, um, we like to say software's going from the age of rules to the age of goals and guardrails. And the hard challenge there is how do you enable businesses to express their goals and guardrails?

AI assessment note: “I'll describe a technology problem and then I'll describe the human problem that was harder”

Q Can I ask your advice? You've sat on some of the best boards. I sit on boards now. I am a young board member. I want to be the best board member that I can be. Is there any advice that you'd give me having seen many different types of boards and types of entrepreneurs?

A You know, I think the, the art form as a board member is how to be involved enough without jumping into the operations of a company and knowing how to give advice in a way that the CEO and the management team actually hears. Um, so, you know, I think the, finding that balance of creating the cadence with the companies you work with to get the information you need so that you know where you're gonna add value when you know when to like, you know, uh, Call the proverbial bat phone because something's wrong, um, is the biggest art form. So I would say, you know, board members who treat every engagement the same are probably not doing it right because different executive teams, different CEOs have, will hear device advice in different ways, and the businesses are very different. So I think really treating it very uniquely and finding an operating cadence, you can get the information you need to actually provide good advice.

AI assessment note: “the art form as a board member is how to be involved enough without jumping”

Q can be learned? So like, for me, people think, oh, I could never be an interviewer or do content. And I just say, listen, I was shit. I've just done 3000 over 10 years. I learned to be a little bit better, hopefully, so I'm now tolerable. But it's just like going to the gym. What do you think everyone thinks is an innate skill that actually can be learned?

A Leadership. I actually have made so many, I mean, you've probably heard the phrase, oh, this person's such a natural leader. And I clearly, if you're a sociopath, you probably won't be a great leader, you know, but if you have decent amount of EQ, uh, the innate skills of becoming a leader at different scales of organizations is absolutely something I believe can be learned. It's interesting because if you meet, uh, people who have served in the armed Forces, uh, in the Western world. Most of the military treats leadership as a craft that you learn. Um, and it's in part because of your, you know, uh, growing through the ranks of say the army, you know, at each step, you're managing larger and larger, uh, groups of, of soldiers. And, you know, they, they formalized a lot of like principles of leadership. In contrast, I, I think the, you know, if you go into most large companies and you go into You know, a promotion discussion, you know, corporate promotion discussion, like that person is just not a natural leader, not that person needs to, you know, uh, train or learn these skills. And I think we've, um, well, it's not true of all of corporate America. Um, I do think it's one of those things that I think, you know, companies should invest in more, uh, which is like formal training of characteristics of leadership, uh, how to motivate, you know, uh, people who are different than …

AI assessment note: “Leadership. I actually have made so many, I mean, you've probably heard the phrase”

Q We mentioned the commoditizations of foundation models as a technology. We've also seen price dumping and a race to the bottom in terms of price as well in a lot of cases. How do you think about AI business models that are sustainable given incredible training and inference costs?

A So when I made the comment earlier about skeptical of companies doing pre-training, it was really based on the premise that most companies should be Applying AI to build solutions and most companies should have relatively modest training costs and most of their costs should be correlated with inference, which should be correlated with revenue and usage of your product. Uh, and, and I think that that's an essentially because if you end up pre training a very large model, you know, you end up with some such upfront capital requirements. You have to have a really valuable business model on the other side of that to justify that investment. So first I think companies should really focus on how to find product market fit, you know, prior to taking on meaningful training costs that are fine tuning might be fine. Uh, you know, but you have certainly sort of pre-training models on the inference side. Um, I actually think, uh, the costs of AI are going down really, really rapidly. Um, I've seen a lot of people tracking sort of The cost of the GPT models over time. And what's remarkable about the cost going down is the quality is also going up, you know, so it reminds me, you joked about when you were born, but, uh, but well, around the time you were born, every time I got a new computer in my house, it was twice as cheap and twice as good. So I think, you know, on the inference side, I …

AI assessment note: “most companies should have relatively modest training costs and most of their costs should be correlated”

Q Why do you, why do you need the phone at all? If I can just talk to myself, which would look kind of weird, but normal because I do often as the venture ambassador with too much free time, clearly, uh, like, you know, I could ask myself, Hey, like get an Uber. Uh, I I'm here. Do we see the removal of the phone?

A It certainly seems feasible, but it's, I temper that with the, if you look at the past 15 years of consumer electronics innovation, how many companies, including the ones that make smartphones, have tried to make devices that, you know, replaced or augmented the phone unsuccessfully. You know, this, this device here, this phone, it's so good at so many things, and everyone already has one. It's essentially, Completely removed the market for almost every other type of consumer device. Um, so in the short term, my intuition is that the combination of a smartphone with Ray-Ban glasses or AirPods or the like, um, probably, you know, meaning you might need to look at your screen less, uh, than you do today. But my intuition is because of the prevalence of smartphones around the world, it will still end up being, you know, the, the primary Computer that mediates those conversations, but to the point that you made, you know, as conversational experiences start working more, the, I always get the big phone, you know, just cause I like the big screen. You know, I, I think a lot changes and, you know, I always go back to the early app store days and the early apps being such skeuomorphic apps like flashlights. And then you have the mobile native experiences like WhatsApp door dash, you know, Uber, Um, Instacart. It took one generation for, for those things to, to really, um, exist. I hav…

AI assessment note: “It certainly seems feasible, but it's, I temper that with”

Q Are WhatsApp not best placed in terms of instilling an app store for every big brand in the world to implement their own channel, and then you have existing distribution to a billion, however many users it is, integrated already into functionality and apps that they use already?

A I think WhatsApp is very well situated, um, and it's in particular, if you look at the usage of WhatsApp in places like Brazil and India, um, you know, it is, uh, Uh, approximating this already, but I think, you know, large language models and agents like the ones we build at Sierra open the door to sort of much more full featured experiences. Um, but I also think the true same is true as most mobile platforms. You know, I think that, you know, when you install an app on this, uh, it's probably gonna be an app and an agent in the future. Like when we work with our customers, you know, we wanna enable them take their AI agent and whatever form factor becomes a dominant consumer experience. You should be Be able to install your agent in that, in that experience.

AI assessment note: “I think WhatsApp is very well situated, um, and it's in particular”

Q Uh, but I, I, I, you know, remember reading about it. Uh, but, but we can't continuously have step changes. Can we? Are we at a stage where you start to see slightly diminishing returns?

A Those questions are distinct to me. So starting with the step step function, Maybe, maybe not. You know, I, I don't think it's a foregone conclusion that we'll have step function changes. I do think that, you know, I believe the most responsible way to develop AGI is responsible iterative deployment. And the reason for that is I believe that as you're thinking about things like the societal impact access to this technology and the safety, uh, side of AGI as well, that the, Best way we can learn about how to, you know, ensure that these models benefit humanity is to consistently release them, learn, uh, from those experiences on the safety side, learn about the harm, learn about really specific vulnerabilities like jailbreaking and improve it at every turn. We could end up with a plateau of progress, or as you said, diminishing returns, the three inputs to, you know, progress and AI are Number one data, number two, compute number three algorithms and methodology. Um, so if you look at the history, short history of sort of this current wave of modern AI, you know, it started, I think with the transformers model, which was a, um, paper from Google called attention is all you need, which changed the scale, uh, uh, with which you could, um, Build these models, which led to, you know, many of the sort of GBT breakthroughs that, that came next. Um, you ended up with instruction tuning…

AI assessment note: “I don't think it's a foregone conclusion that we'll have step function changes.”

Q Do you think people are born entrepreneurs, or do you think it can be learned?

A I think most things can be learned. Uh, I've Through certainly through my career, I have most of the time when I've thought of something as an innate skill, um, I've later recalibrated and felt that with like enough focus, one can, um, improve at most things. I, I could never become like an Olympic track star, nor could I ever, you know, win the fields medal. So I, I, obviously I don't mean to trivialize true innate ability, but on skills like public speaking or leadership, uh, or, Um, even things that aren't quite fields metal, like becoming, you know, good at finance. I think most people can with, with focus. The thing that is unusual about being an entrepreneur is, um, how intense it is. And I do think there's a certain personality type that is conducive to that. You know, I think, uh, it's hard to be an entrepreneur if you're prone to anxiety because everything's on fire all the time. You know, that's just the nature of the business. Um, and, uh, As a, as a consequence of probably there's certainly some nature, not just nurture there, but I have met folks who, you know, might not have identified as an entrepreneur earlier in their career, who develop the confidence in their own, you know, resolve, uh, through early parts of their career and end up great. And it's really interesting too. If you look at the enterprise software industry versus, uh, consumer, you know, a lot of…

AI assessment note: “I think most things can be learned.”

Q it next week. Gemini kills it next week. Open AI has crushed it. And I'm sitting here like, fuck, I'm getting dizzy. Like, which one should I use? Oh my God. And then Claude comes and it's like five things you can do with Claude that you can't do with anything else. And I'm like, Christ, I've got no idea what's going on. Are they the fastest technology to commoditize?

A I will start with the high level. I really like Reid Hoffman's framing of this market as foundation models and frontier models. So foundation models are any of these large language models that aren't necessarily the best of the best or the higher per highest parameter count, but particularly now, uh, where you have relatively low parameter count models that are meet or exceed the quality of say GPT, 3.5. Uh, that market of foundation models is quite important, um, and quite commoditized. You know, I think that in that market, uh, probably if you need a model like that, you should download llama. That's, that's the answer. It's like, you don't need much of a cheat sheet on that, you know, and, um, or maybe Mr. All, but pick one of the open source models that are adequate and fine tune it. The frontier model market is a little different when you talk about this, um, you know, the, the experience you've had being dizzy using these tools. My perspective is that, uh, we've seen real leaps there. So when chat GPT came out, that was a meaningful step function change that lasted for a while. Um, and the insight around instruction tuning and the quality of sort of the GPT models after GPT three was pretty remarkably different. Similarly, when GPT four came out, I, I haven't done the math on it, but it certainly had a meaningful lead for quite a while. Um, and now you're seeing a lot of,…

AI assessment note: “that market of foundation models is quite important, um, and quite commoditized.”

Q Brett, what was the hardest thing with Sierra that you did not anticipate being so hard?

A I'll describe a technology problem and then I'll describe the human problem that was harder than I expected around it. So generative AI is very creative, but inherently non-deterministic. Uh, you know, there, it's very hard to create determinism, the same inputs creating the same outputs in particular, because if you think about the breadth of human language, you know, and not, it's just inherently less precise than, than most. And then similarly, if you afford Um, AI, the ability to reason, um, you know, sort of by definition, you can't enumerate all the possible outcomes from there. So when you're building industrial grade agents, you know, for businesses that have real business rules they need to follow, um, we like to say software's going from the age of rules to the age of goals and guardrails. And the hard challenge there is how do you enable businesses to express their goals and guardrails?

AI assessment note: “I'll describe a technology problem and then I'll describe the human problem”

Q On the flip side, how much agency do you give a human who's been trained for a week and sits in your Detroit customer service department and could get high and then abuse a customer? Like, do you know, I always think we forget this when we talk about AI hallucinations, we're like, yeah, and humans hallucinate a shitload too.

A This is the interesting thing about Modern large language models and what I think the industry has come to call generative AI is I think it violates most of the rules we have in our head about computers. You know, computers are designed to be reliable. You click this button, the same thing happens every time you click it. Um, they're designed to be databases. They're not designed to be creative, right? They're designed to like give you facts, uh, follow the rules that we have really, really fast. And, you know, just think about, uh, Software engineering, the craft of software engineering, you know, there's entire methodologies now about how to get increasingly reliable software, which involves using source control like GitHub and using immutable binaries so that, you know, you can roll back and have the same behavior you had yesterday. If something goes wrong, we've essentially spent decades trying to make things deterministic, repeatable, reliable. And now you make this new piece of software that is Slow, somewhat expensive, extremely creative and fairly non-deterministic. It like blows people's minds. I think that as a consequence, people are modeling like AI through the lens of how do we make it as deterministic as software was two years ago. I'm not sure that's the right model. I actually think, you know, the, the thought exercise you did is okay. Let's assume that You know…

AI assessment note: “Why don't you just use the same mechanisms to deal with the AI as well?”

Q I do want it. You mentioned agents there. I do want to move into kind of agents in the future of agents. First off with Sierra, why did you can literally do anything? Bri, if we're honest, why did you decide to do Sierra?

A So let me, uh, just describe what Sierra does, and then I'll, I'll tell you why that's, I think it's very exciting. So at Sierra, we help primarily consumer brands build branded customer facing AI agents. So if you buy a new Sonos speaker or If you're having a problem with your speaker, you'll chat with the Sonos AI powered by our platform. Um, if you get a new car and it's got Sirius XM, you'll chat with Harmony, which is their AI agent. If you go to retail sites like Olokai or Chubbies, you'll chat with, um, I think the Chubbies agents named Duncan's mothers or something. It's a really great personality agent, um, that will help you everything from finding your order to order returns and exchanges. Um, so We're essentially helping companies build their branded AI agent, um, for all parts of their customer experience. The reason why I think this is a really exciting area for our customers and for me personally, is that I think we're in the era of conversational software. So I, I remember when, uh, in 2007, when, when Steve jobs announced this and now I'm guessing you were 11 then based on our previous conversation. So you may not remember it as vividly as I do,

AI assessment note: “I think we're in the era of conversational software.”

Q On the flip side, how much agency do you give a human who's been trained for a week and sits in your Detroit customer service department and could get high and then abuse a customer? Like, do you know, I always think we forget this when we talk about AI hallucinations, we're like, yeah, and humans hallucinate a shitload too.

A This is the interesting thing about Modern large language models and what I think the industry has come to call generative AI is I think it violates most of the rules we have in our head about computers. You know, computers are designed to be reliable. You click this button, the same thing happens every time you click it. Um, they're designed to be databases. They're not designed to be creative, right? They're designed to like give you facts, uh, follow the rules that we have really, really fast. And, you know, just think about, uh, Software engineering, the craft of software engineering, you know, there's entire methodologies now about how to get increasingly reliable software, which involves using source control like GitHub and using immutable binaries so that, you know, you can roll back and have the same behavior you had yesterday. If something goes wrong, we've essentially spent decades trying to make things deterministic, repeatable, reliable. And now you make this new piece of software that is Slow, somewhat expensive, extremely creative and fairly non-deterministic. It like blows people's minds. I think that as a consequence, people are modeling like AI through the lens of how do we make it as deterministic as software was two years ago. I'm not sure that's the right model. I actually think, you know, the, the thought exercise you did is okay. Let's assume that You know…

AI assessment note: “Why don't you just use the same mechanisms to deal with the AI as well?”

Q Uh, but I, I, I, you know, remember reading about it. Uh, but, but we can't continuously have step changes. Can we? Are we at a stage where you start to see slightly diminishing returns?

A Those questions are distinct to me. So starting with the step step function, Maybe, maybe not. You know, I, I don't think it's a foregone conclusion that we'll have step function changes. I do think that, you know, I believe the most responsible way to develop AGI is responsible iterative deployment. And the reason for that is I believe that as you're thinking about things like the societal impact access to this technology and the safety, uh, side of AGI as well, that the, Best way we can learn about how to, you know, ensure that these models benefit humanity is to consistently release them, learn, uh, from those experiences on the safety side, learn about the harm, learn about really specific vulnerabilities like jailbreaking and improve it at every turn. We could end up with a plateau of progress, or as you said, diminishing returns, the three inputs to, you know, progress and AI are Number one data, number two, compute number three algorithms and methodology. Um, so if you look at the history, short history of sort of this current wave of modern AI, you know, it started, I think with the transformers model, which was a, um, paper from Google called attention is all you need, which changed the scale, uh, uh, with which you could, um, Build these models, which led to, you know, many of the sort of GBT breakthroughs that, that came next. Um, you ended up with instruction tuning…

AI assessment note: “We could end up with a plateau of progress, or as you said, diminishing returns”

Q ask you, when you look at Google and Amazon, their cash count to fund this is cloud. Zuck and Meta do not have a cloud business being their cash cow to fund this. What does that enable or mean that Zuck can do differently with the cash cow not being cloud? Say that after 10 tequilas. Is there anything he can do differently? Is there any freedoms that he has?

A Yeah, I thought it was, you know, one of the things that, uh, I, Have changed my mind on over the past year is how quickly open source foundation models would be impactful. So I had a thesis when we started Sierra, uh, in, in March of last year, that eventually we'd end up with a few, a frontier model, uh, frontier models, essentially, uh, Built and financed by some of the hyperscalers, um, in partnership with the research labs, and that we would eventually have a meaningful open source model or two, the equivalent of Postgres and MySQL in the database market that would come out eventually be adopted by one of the, uh, larger tech companies that, that wasn't one of the hyperscalers just in the same way Google adopted Linux Or Facebook adopted my sequel and memcache, uh, you know, and contributed a lot of, uh, patches upstream to, to those projects. Um, and I would say Mark Zuckerberg sort of accelerated that by a meaningful amount, not only the timing of when that happened, but the quality, you know, Lama 3.1 is a really high quality model. Um, so I think it comes from what you said, you know, without a cloud business to finance it, his incentives are different than You know, the, the cloud providers. Uh, and I think he wrote, uh, no need for me to say it. I mean, if you just read his post on why he believes that this is the right strategy, I thought it was a really well articu…

AI assessment note: “without a cloud business to finance it, his incentives are different than”

Q If we think about that reducing cost, uh, over time and Moore's law proving out, we're also just seeing, you know, matter. We're seeing, uh, Amazon. We're seeing Google. I say they are going to invest ungodly amounts in the next three to five years. Does that go against Moore's law and the reducing costs for them? And how do you think about those two seemingly kind of paradoxical things?

A I think the large hyperscalers, uh, you know, are in a challenging position where there's a big difference between, you know, owning and operating one of the best frontier models and not. Um, so as a consequence, I think that, you know, I probably make very similar decisions to all of those firms because there's so much value in, you know, for consumer products, for, uh, infrastructure as a service providers, um, to have the, the, a differentiated frontier model available to their customers, um, that, you know, the betting on the future and then similarly betting on breakthroughs and AGI, I think is, Is really rational. Um, but the reason I was talking about the sort of Moore's law part of it is I think that sort of like in the infrastructure as a service market, it really consolidated around a very small handful of, of companies and much like AI, you know, building data centers like scale helps. So, you know, the more data centers you operate, the more you can afford the capex to expand your data center footprint. Um, it's just one of those things, which I think You know, it should be financed and built by the large hyperscalers because of the capex requirements to do so. And I think as the training market sort of consolidates and people start, you know, I think it will probably help because the revenue will sort of consolidate, you know, around those providers as well. Um, so…

AI assessment note: “I think these companies have an imperative because of the potential impact of AI to spend”