The Exchanges

Q but I'm seeing all the money that they're raising and you have to make a lot of revenue to justify that and eventually profit. So, so, so does that factor into your decision at all in terms of like how much to rely on these tools? Because one day maybe they could go away or maybe they could be 10 X more expensive and then it might change the calculation.

A Yeah. It's a funny question because right now I feel like we and everybody, all of us are beneficiaries of, you know, venture capitalists that are helping subsidize our AI usage really. Right. I mean, you, you, I think you're referring to the fact that it often costs more to service customers than you're bringing in revenue. I mean, you know, the big AI houses are not profitable now. But we're certainly the beneficiary of it now. So we'll make, hey, well, the sun shines. And I think for us, it's about making sure that we're so efficient and judicious in how we use AI that it's not ridiculous and wasteful. And so like we're doing, we, we monitor, we see people who have like, uh, some of our best developers might be using, you know, a thousand dollars in anthropic credits a month. And then we find somebody who's using like 4000 a month. Like, well, what are you doing over there? And so we're, we're working to corral and make sure that our usage is responsible, that we protect our customer data, that we use it at the right pace, that sort of thing.

AI assessment note: “it's about making sure that we're so efficient and judicious in how we use AI”

Q Is the voice AI more effective at getting employees to fall for these scams than text?

A Yeah, it is until the employees know to be looking for it. And it's like, uh, it's very much along generational lines. You won't be surprised to hear this. I mean, it's, um, you know, my parents, uh, kind of their baby boomer generation and You know, I've talked to my mom and like, they're a little bit like in awe of what's going on. Like, did you see that thing on Facebook? And I'll be like, mom, that was fake. You know, um, she's very discerning in other things, but, uh, sometimes the AI stuff gets her. Meanwhile, um, I was playing something for my kids the other day, uh, and they immediately were like, that's fake AI voice. It was a famous person saying something like, that's not real. I'm like, how did you know? You know, but they're 1618 years old. Like they're in tune with this stuff.

AI assessment note: “Yeah, it is until the employees know to be looking for it.”

Q So have you seen the, um, the sophistication of attacks increase and the volume of attacks increase thanks to generative AI?

A For sure. You know, it's, um, at our company now, I'm not directly dealing with threats, but, uh, I've worked at companies where we are and I look back at the things that we had to deal with just a few years ago when we were doing incident response and the attacks that I see even directed at us now. And it used to be You know, take a phishing email. It's an example that everybody can relate with, right? We've all seen, gotten phishing emails. They used to be sort of broken English, clunky. Look at the URL. You'd be like, that's not real. Um, they've gotten so convincing now that someone who's been doing this for a long time, I still have to like do a triple take and say, is this thing real or not? I find myself now, especially, you know, you have to be really careful in my position. So I'll, I'll call the CFO or the general counsel and just say, Hey, does this thing look real? Like, let me bounce it off you. So I don't know. We're all have to be, we have to be a lot more careful, a little more cynical of what we receive.

AI assessment note: “For sure... They used to be sort of broken English... gotten so convincing now”

Q start to see this crazy build out that open AI, Oracle and others are driving. That the numbers really only make sense if you get to AGI, if you get AI that cure cancer, and so much of that is speculative, and that is where we might be getting into a danger zone. So Ranjan, I'm just going to turn it over to you. What do you think about that?

A Yeah, I'm really glad that he's making this distinction between those who are kind of just pushing the idea that it's time to monetize these investments and drive a world-changing technology evolution versus this debt-fueled second push. I've been looking at this a lot over the last couple of weeks around the Oracle deal, the AMD deal, all these deals are focused on CapEx and not product. Again, I mean, there's been some incredible Moments in product over the last couple of weeks, SOAR included, which we're going to talk about, and I've been waiting to talk to you again. But I think overall, this idea that, you know, it has to be AGI, it has to be something that justifies all of this capex, you know, investment, is what Oracle, Meta, CoreWeave, all these companies are betting. And, and there's been absolutely nothing That shows us that we're actually headed in this direction, so it starts to feel more uncertain and irrational. Nothing's stopping it right now, but I think it's a good thing that we're talking about it.

AI assessment note: “there's been absolutely nothing That shows us that we're actually headed in this direction”

Q Okay. I want to ask, ask you a question about something you said at the beginning of that answer, which is that you don't believe that we're going to end up having software that, uh, basically condenses everything into one platform. Uh, why is that?

A So I do believe that we'll have software that convinces into everything in one platform. That, that is absolutely what we're focused on. What I don't believe is this notion that you're going to use just AI chat experience for all of your software needs and that you won't actually need UI, um, or application layers to, to back that up. And, you know, we, we, I think as, as humans, you always, I mean, take chat for example, right? Chatting team chat. You are always going to want A consistent experience where I can go see the history of all my messages, regardless of how I send a message or receive it. You're going to want a DM that you can go visualize that, that history of your communications with a given, a given person. And that same thing is true for all of those, those core primitives. Um, so I certainly don't believe, you know, that, that everybody's going to want just to have a single AI chat all day long, and they're not going to want to see any application layers or experiences. Um, but I do believe that, you know, these separate software categories will all converge. I think AI will ultimately also converge with, with software.

AI assessment note: “So I do believe that we'll have software that convinces into everything in one platform.”

Q the marketing database just live in the same product? Um, and, and then is this something that is going to be built by the incumbents? Like, is this something you already mentioned, OpenAI and Grok, uh, embedding Uh, spreadsheets. So is this something that's going to be built by these big foundational research houses that need to turn profits? Or do you think it will be something that others build?

A I tend to think that the giants will build offerings, uh, but it will take them a long time. I think also that the AI labs, um, will struggle cracking into B to B, especially with collaboration. Um, that is like a, you know, a core block in AI right now of AI chat. It's, it's your own chat, right? You're chatting with your own stuff and that's, that's it. There is no like element of collaboration. Um, and the context is also missing and, and, and, and the AI labs today, but they know that, right? And, and so that is, is certainly part of, of their strategy in the longterm. Um, but yeah, I do think to answer your question, yeah, I think that you will have one software provider that has ERP system of record. It has all of your customer system of record. It has your marketing, um, certainly has HR and project management and work management along with team communication.

AI assessment note: “I tend to think that the giants will build offerings, uh, but it will take them a long time.”

Q Figma's doing it too? Why is Figma also? It's one, one that you've listed as well.

A Yeah, you know, they are, and I not, look, I, I think if I was them and, and their case, it's, it's hard to say, you know, is it the right decision? Is it, is it the wrong decision? Uh, because in, in Figma's case, you know, they do have that engagement layer, like where people are actually working. They also have the, Platform, the data platform layer, like the source of truth of designs and the very detailed specs along with that. It's not just a screenshot. They have all of, all of the actual components within the wireframes and yeah, they, they started locking it down to where, where, you know, we were. Uh, synchronizing all of your Figma prototypes and you could search across them and, you know, you could do AI across the prototypes. Um, and recently they, uh, notified us that. Uh, you know, they've, they are deprecating that across the board, not just, not just for us, for, for everyone. Um, and so that will, you know, that will, will be a bit of, I actually think that it does, it will create problems where consumers are pissed off because they're using their favorite vibe coding tool right now, and they're trying to plug it into Figma. And when that access gets cut off, you have to go use Figma's product, right? Which is imagine, I imagine that is, is kind of the intention here. Is to go use their Figma make product for, for vibe coding rather than using something else o…

AI assessment note: “when that access gets cut off, you have to go use Figma's product”

Q for Sam, um, but I will say that When you get this stuff, starting to understand physics, you're just a step closer to, I think, uh, the end goals. And ChatGPT, of course, like, seemed like a little toy at first, but obviously has emerged into something more. And, uh, I get this, the criticism, but I tend to side with Sam on this one. What do you think, Max?

A Yeah, I think it's, I think it's a tough call. I think I agree with Sam in some ways that he's right that ChatGPT, it wasn't really understood at first how this really was going to be a vehicle for a lot of people to use AI and a lot of people to actually get the benefits of AI. And I think we've completely seen that play out in the last few years. I think it's a little less clear to me how Sora gives, gives people the benefits of AI world models in the same way. I totally agree that The underlying technology is really impressive. I mean, when the first Sora came out, I think last year, it blew people away just with how realistic the videos were, with how accurate it could get certain details. Um, and this one is objectively better. And I, I think it's impossible to argue that there hasn't been a great leap in performance of the AI video models in the last year or two. Um, But ChatGPT at its core is about helping people. That is, I do actually take OpenAI at its face when it says that it optimizes ChatGPT to be helpful and not to just keep users in, you know, these engagement loops. Um, I do think that I go to ChatGPT all the time to get an answer, I get it, and then I get off the app. I think with Sora, it does feel to me like it is, I mean, he said it pretty plainly. It's, it's about And entertaining people. It's about, you know, it's supposed to be fun. And I think that thes…

AI assessment note: “I think it's a little less clear to me how Sora gives”

Q are scary, said OpenAI pre-training researcher John Holloman in a post on X. We're going to do our best to make sure AI helps. And does not hurt humanity. I mean, where have we heard that before? You had actually multiple, uh, current OpenAI employees being like, uh, I don't know about this. That's, that doesn't happen too often in a major launch. So talk about the significance of that.

A Yeah, I think the level of kind of concern expressed by a lot of OpenAI researchers at the launch of this product felt very unique to me, uh, in a way that you don't usually see. Usually when an AI company launches a product, everyone is very gung-ho about it. Everyone is really excited. And I think this time just felt a little different. Um, I think all, I think a lot of researchers at OpenAI were very aware that their company was not just launching a new product. It's not a new feature in ChatGPT. This is Social media app. This is uncharted territory for open AI. And I think that raised a lot of concerns for people. I mean, something, you know, I've heard over and over from AI executives, and I'm sure you've heard the same, Alex, is just, you know, they all talk about how we don't want to repeat the mistakes of the social media era. And I believe a lot of the researchers working out these labs don't want to do that. I think a lot of them grew up In an era where people were addicted to social media. Uh, and I think a lot of them don't want to create the same kind of dynamic with the products that they built. Um, but I, I do think that just the incentives of running a social media platform are sometimes going to be very, uh, intoxicating and strong. And, you know, I don't think Meta exactly set out to build, uh, you know, addictive Uh, platform, but that's what you do when you …

AI assessment note: “the level of kind of concern expressed by a lot of OpenAI researchers... felt very unique”

Q they're one of the first things that could be totally annihilated by AI. I think it's worth taking a minute to talk about this because you have this new technology and new app. It's obviously very engaging. Number one in the app store. But instead of the painstaking process of creating content online, you could just prompt it. So what do you think this means for the creator economy, Max?

A Yeah, it's really a interesting question. I mean, I think that people said some version of this for writers when ChatGPT came out that, you know, why would I hire a freelance journalist to write something if I could just ask ChatGPT to write it? And I mean, certainly there are countless websites out there with articles, uh, written by ChatGPT now. Uh, so to some extent that's proven to be true. Uh, and yet, uh, Alex, we, we still can make a living doing what we do. So, I think what have we learned from the text version of this story that's already played out is that people value the human touch and the human taste and judgment, um, and they value kind of a professional's opinion, uh, you know, on certain topics that has become actually, I think, more valuable in the AI era. And I think that was something that a lot of people missed when ChatGPT first came out, which is that, uh, it actually made really high quality human content more valuable. Um, I wouldn't be surprised if a similar thing happens with the creator economy. Like, I think that we will see just more content in general as a result of these AI video models and Sora. Um, and I think that it probably does make it harder for content creators to stand out. It probably, you know, advertisers are looking at these AI video models and going, huh, that's pretty interesting. Um, should I, you know, maybe Put some of my revenu…

AI assessment note: “I wouldn't be surprised if a similar thing happens with the creator economy.”

Q The economics of this AI boom, do they make sense if they just boil down to serving us better ads, or does it have to be something more?

A I think that ads have been the biggest business on the internet for a long time now, and I have no, I don't think that's going to change. Like, I, like, I think that that is going to continue being the case. I think it could probably be a bigger business than it is today. Um, I think you could probably get more specific kinds of ads and AI. I mean, I, I share so I and other people, everyone shares so much more information with an AI chatbot than you do With your Instagram feed, with Google, and you have to think that, you know, these companies that already have this great profile on you, if they could fine tune it even more to show you just the exact product that you want in the moment that you want it, uh, even better than they can today. I mean, there's clearly a lot of value there. Um, so yeah, I, I do think that even if it just boils down to ads, I mean, that's, that's gotten Google and meta pretty far. So I wouldn't, you know, doubt that.

AI assessment note: “I do think that even if it just boils down to ads”

Q had been planned for twenty-twenty-eight, but the company is now looking to accelerate development. This is, this is a massive story. Thanks to Mark Gurman from, from Bloomberg to, for reporting it out, but it's fascinating to me that Apple looking over at Meta now sees glasses as so important that it's shelving a big redevelopment of the Vision Pro, uh, reportedly to go after this. What's the significance, Max?

A Yeah, I mean, I think that, honestly, Meta's Ray-Ban display glasses were so impressive to a lot of people. I mean, if you try to book an appointment in the Bay Area to go demo these, uh, it's basically booked out for the next two or three months. Um, I mean, you basically just can't even, yeah, you can't even get an appointment right now. Um, so, uh, and you have to demo them to buy them, which is a weird detail about them. But But the, the demand for them I think has been incredible. I think people were really fascinated by this new computing form factor. I think it's a lot more compelling than putting a pair of big goggles like an Apple Vision Pro or a MetaQuest on your face. Um, I don't know about you. I can't stand, uh, the headsets to be completely frank. I think they get hot. I think they're kind of uncomfortable, but the glasses are very comfortable to slide on. And Uh, I, for one, am really excited about this development. I think this is like a much better direction for Apple to go in than making a better Vision Pro. Um, but of course, Apple is in a much better position to win here. I mean, they can connect it to iMessage. They can connect it to all the apps on your phone. So I'm not sure what claim Meta really has besides being first to market here.

AI assessment note: “I think this is like a much better direction for Apple to go in”

Q You think, so you think that this level of build out is healthy?

A I, I think we definitely are not nearly at the point at which there is too much AI infrastructure, given I think the number of AI workloads that are coming for the world. Um, and, um, you know, I think we're seeing on a, over the last couple of years as people use AI, they get value, they use it more, the models get better and people then use it even more for new use cases. And, and I think at this point across the industry, With AI, we're still more supply constrained than we are demand constrained. And I think, um, I, you know, I expect that to continue over the next couple of years as the technology continues to evolve and as people start to integrate AI into more and more workflows.

AI assessment note: “definitely are not nearly at the point at which there is too much AI infrastructure”

Q the AI world? I mean, like we talked about previously, GPUs were designed for gaming. They happened to do parallel processing. Actually ended up being really good for, uh, you know, large language models, the training and the, and the inference. Um, what's your perspective on, on whether this industry is going to continue to run on that type of chip, uh, and what the potential is for custom silicon?

A I think, um, a couple of things. I think one is, um, I think the increasing the, um, The, the number of tokens you can get per watt per dollar is going to be the game over the next couple years and, and maximizing, um, the ability of our cloud to deliver the best, uh, Volume of tokens for every watt of power, for every dollar that's spent, where the dollar is spent on energy, it's spent on the GPUs, it's spent on the data center infrastructure, it's spent on the network, and it's spent on everything else, is, is the thing that, again, we're laser focused on, and, uh, it is, you know, there's a bunch of steps as part of that, GPUs being a critical component of it. Um, And, you know, one of the things that our scale gives us the ability to do is to invest for, uh, kind of nonlinear improvements in that type of productivity and that type of yield. You know, if you've got, you know, A million dollars of revenue on a couple hundred GPUs. You're not going to be investing in custom silicon. Um, when you're at our scale, you will be. Um, and you're not just investing in custom skill again for GPUs for pre-training or for inferencing. You're, you're looking at what can we be doing for synthetic data generation with silicon? What can we be doing from a compression perspective with custom silicon? What can we be doing from a security perspective? And we have bets across all of those, many…

AI assessment note: “every GPU server that we're running in the fleet right now, um, is using custom silicon”

Q So, I mean, talk a little bit about how, what is an AI governance, uh, process or, or document and then how does that, How does not having one sort of hold a company back?

A Yeah. So I think, I think different companies approach this differently. There is no one way to solve this. Obviously there are a few frameworks, uh, uh, companies that are deploying, but essentially you want to have, it's, it's not even AI governance, it's really data governance. So you want to make sure that all of your systems that are housing, you know, like sensitive data or, or critical data, That there is a system to make sure that only people who should have access to it have access to it. Now there are different levels of sensitivity, right? Like if somebody just accessed some meeting notes, uh, for a team that may not be as consequential, but imagine, uh, if I was in a meeting and I was talking about a customer and the customer information was accessed by somebody that could be. Problematic. My other example was, you know, my salary information was disclosed. That could be problematic. Or if a team is working on a confidential M&A, right, they're trying to buy a company, and that project name or the company name got kind of exposed internally. That would be problematic. And so, I think a lot of companies, you know, a good practice has been classifying the different types of sensitive information you have in the company. And then really making sure that all access to that information is really locked down, and you have this centralized way of managing permissions, so t…

AI assessment note: “it's not even AI governance, it's really data governance”

Q And what about outside of the coding realm? Is there a potential for this in use cases outside of coding?

A Yeah, so there, there are other categories. I know in healthcare, healthcare has a lot of, Uh, well, it's highly regular, but it has a lot of manual processes and entry, manual entry. So, so I've seen, I can mention, uh, customer names. We have customers who are using agents to essentially process medical information for their, for their patients and customers. So, uh, a lot of benefits there. And you can imagine other scenarios. I think, uh, retail, uh, and pretty soon retail and e-commerce is another area where you will see Uh, these agents play a, a pretty big role. Um, I, I can see travel being another category. You know, I, I wanna travel, uh, to Japan in November. I can easily, uh, instruct an agent to say, you know, go find me the right airfare, build me an itinerary, and if the agent knows kind of my preference of what, what kind of, um, me, you know, if I have any meal preferences, any hotel preferences. So, Those things are really valuable because I'm really busy, I don't have the time to figure this out, and an agent can go do all this work, um, for me, and I think it's extremely helpful, uh, to me as a consumer. Now, of course, hopefully it's done with all the security controls, and, you know, it's not signing me up and buying airfare that I haven't approved, but, but, so there are many scenarios where I think agents will make our lives easier, and, and today you're…

AI assessment note: “Yeah, so there, there are other categories. I know in healthcare”

Q sometimes thought partner and companion are the same thing, depending on how much you trust your AI companion to handle your thoughts. Uh, why do you believe in the, the agent use case? Because, uh, it, it has become, we've talked about this on the show. It has become a bit of a buzzword, uh, in the business world right now. Um, so, so why is it worth the hype?

A I think there are three characteristics about agents that are super interesting. One, they're asynchronous. It can, it can go do, you don't have to be in front of, uh, your laptop or, or phone and, you know, the input is not limited to like your mouse and how fast you can type. So it's asynchronous. Second is I think now, especially now with agents, they can do long running tasks. Now, most of the experience we have today, when we ask questions, we're doing research, I think you'll see agents responding fairly quickly, but You will, but there are tasks that require quite a bit of research that may take a long while, and so I think agents are perfect for these long-running tasks. So whether it's minutes, hours, I think there'll be some tasks that will run for days or weeks.

AI assessment note: “there are three characteristics about agents that are super interesting. One, they're asynchronous.”

Q Wait, hold on. I'm going to push back on that. How do you see practical guidance and seeking information is distinct from traditional search? Like, wouldn't that be the main, or at least a very large component of overall traditional search?

A Okay, good point. Seeking information definitely is search, right? That is without a doubt search, but the fact that practical guidance is number one, like I think there was a, um, a misconception that AI was going to be, um, just like search redone, a refactoring of search, and it's probably led by publishers who are like, where's our traffic going? But the fact that practical guidance is number one, where you can go to, and we're gonna talk about it more, but you can go to ChatGPT, Say, I'm facing this situation. What should I do? Whether that's a work situation, a health situation, a relationship situation, uh, getting fit, which we're going to talk about. That to me is a brand new use case. You just have not had that online at all.

AI assessment note: “Seeking information definitely is search... That to me is a brand new use case.”

Q Can you define what this, what this is?

A Um, so, um, outcome based firm fixed prices is, you know, to put like, uh, you know, you're going to have a house built. You, you have a price that you pay for the house upfront. Uh, uh, time and material is, you know, you, you, you have a house built and you just pay as you go based on the changes and all those other things. Right. Um, and so, um, both have advantages and both have disadvantages. I think, uh, early days in the government, mo many things were firm fixed price. Um, and outcome based. In other words, you want an outcome at the end. I want to, I want to land a, you know, a person on the moon or I want to do whatever happens to the, you could be an outcome based type contract. Um, sometimes time and material makes a lot of sense when you're asking, uh, the government to do something extraordinary they've never done before. Uh, or no one's ever done like three D printing and organ transplant, right? We don't know that we can do that. No, one's going to sign up for an outcome based contract like that. But, uh, migrating, uh, from on-prem to the cloud should be an outcome-based contract that we know how to do that. So, so when you know how to do something, um, uh, outcome-based makes a lot of sense, uh, and firm fixed price makes a lot of sense. When it's something that the government's really pushing the edge of technology on, that's when you sort of have more of a t…

AI assessment note: “outcome based firm fixed prices is... you have a price that you pay for the house upfront”

Q Okay. I mean, did you see there was that story of, I think it was veterans records being held in a cave? Uh, how does that happen?

A Um, so, so, so that's not really accurate exactly, right? There, there are, there, there is the need, uh, for long-term storage at NARA and other places like that of, of data, uh, that is underground, um, and that data is also stored purposefully, um, uh, on, uh, non-electronic formats, and the reason for that is, Um, we have legal requirements, but the government to keep that data forever. Um, now you could change those legal requirements. There's reasons for those legal requirements. We have legal requirements to keep that data forever. And if you stored it on some type of technology, you'd be constantly having to upgrade that technology. You know, you would have started storing it on, you know, a 1600 DPI tapes. Then you would have had to migrate that to 2800. And then you would have had to migrate that to 62 50. Then you migrate that to 37 K. Then you would have migrate that To disks and then you go and on and on and on and on. Right. Um, so there, there's a certain amount of logic to that. Um, it is stored in OCR character, so it can be, uh, automated at any time. So there, there are, you know, there, there are certain things that are true. Other, other things are, are, are, in my opinion are misrepresented.

AI assessment note: “there is the need, uh, for long-term storage at NARA and other places”

Q And there's not really a, what does that greater, uh, number of signals give you?

A Well, um, it really, the important, it, it, it provides us, uh, um, a more detailed picture, uh, and a more complete picture of what the brain is doing at any given time, because I think one of the important things to, to realize about the brain, um, at least the parts of the brain that we think of as being most relevant, um, is that almost all of our conscious experience, whether that is movement or sensation or vision or, um, sort of, Decision making or memory all happens, um, basically at the surface of the brain. We call that the cortex. Uh, I think that's not an intuition that everybody has because we think of the brain as a three dimensional structure, which it is living inside the head. And so people think of it as, um, as kind of all of the functions of the brain. People have this intuition that are, they're kind of uniformly distributed in this, uh, in this, you know, 1500 grams of tissue, but actually they're, That all that processing is not uniformly distributed within that volume. It's almost all very, very close to the surface.

AI assessment note: “it provides us, uh, um, a more detailed picture”

Q And when you apply it to stroke patients, uh, is it that, You are able to decode what they want to say and then help them say it? Or is it if they lost some movement Actually using electrical signals to help them move again.

A So, uh, we're still talking, we're still talking about a function that is decoding intention and expressing that through digital means. So for example, somebody who, you know, who has a stroke on the, uh, you know, on one side of their brain and can't move a hand, a hand, for example, or can't move it well enough to type, uh, you know, that, that kind of deficit, which is debilitating for people who are trying to return to work. Uh, especially if it's in the dominant hand, for example, that kind of deficit could be augmented by a direct thought to digital world, uh, communication. Does that make sense? We're not talking about yet stimulating the brain in a way that restores the hand back to normal or that, um, you know, provides an arthosis over the hand that moves the hand again. I do think that will come and we're already talking to partner companies, uh, to do that kind of a thing. But the therapy from brain computer interfaces is primarily something that kind of reads brain activity in real time and establishes, you know, um, intuitive, smooth communication with the digital world.

AI assessment note: “we're still talking about a function that is decoding intention and expressing that through digital means”

Q to par, we would get some company that's, but that would be able to do this. Um, so again, like when you think about, you have to prioritize what you work on, you have to decide Um, whether or not to go full steam ahead with this agentic move. Um, how do you go from taking the research and development and deciding when it's time to put it into production?

A I think we're convinced that we're convinced that's the future, and so that leads to the decision to invest and the confidence to invest. Um, you, and then you roll it out to the extent that it's, to the extent it's ready to be rolled out. So some things you've seen us do with search are we start in, in, in labs. Even with AI overviews, which has become a huge success, started as search generative experience in labs, and so that enables us to get started, get our feet wet, and then we build confidence through that to actually know When it's ready for everyone. And so we have a, we also run experiments along the way, et cetera. So we don't have to make these as sort of yes or no, black or white sort of decisions. We're able to invest with the conviction because we know it's the future and then build it and then assess where we are in terms of how, how broadly we roll it out.

AI assessment note: “we start in, in, in labs... enables us to get started... build confidence”

Q urgency to get things in labs, get it tested, this boldness to say, all right, we have a format of search. That might disrupt our current form of search, but we're just gonna roll it out right next to regular search and see what happens. Uh, how did Google go from the company that was, it seemed like, to the outside afraid of generative AI to one that embraces it?

A I think the technology needed a lot of maturation, right? Very early on, if you sort of rewind, uh, rewind to those early days, The technology hallucinated quite a bit. Um, uh, certainly within the context of search, factuality matters a lot. We don't, we don't, you know, people trust that when they, when they search for something on Google, the results they get are, um, are highly accurate, um, they're high quality, et cetera. So, um, that was a, that, that was a big area of concern for us, and the technology was, uh, was, was truly nascent, um, and, and, and had a lot of risks around it. Uh, so we invested, um, aggressively Um, and, and Liz can talk more about this in, um, in grounding, in factuality, in, in getting to that point. And so that was really, I would say that was the real unlock for us is, is, is, was developing the technology and developing, um, all sort of everything around it, um, that we could feel confident that was a thing that made sense to put in front of our users.

AI assessment note: “that was the real unlock for us is, is, is, was developing the technology”

Q terms of the way that it sends traffic to publishers, still nascent the way that you monetize. Uh, but every time I'm like looking at the numbers for the quarter or seeing them come in, I'm just like, huh? You know, it doesn't seem like it should be possible to grow. I think it's like double digits search revenue somewhere in that area. So in this moment, what's going on?

A So there's a few interesting things that are all kind of happening at the same time. Um, the first is, uh, queries continue to grow, um, and we've, and we've shared that on a year-on-year basis, queries are growing. Um, a lot of that is, is propelled by generative AI, right? And so generative AI has been, um, in a lot of ways, a fuel for growth, um, as, as, as users realize they can ask more questions, uh, they can bring more questions, uh, to search. So that's, that's, that's a big part of it. Uh, the second part of it is AIO reviews fit into the existing, um, uh, search results page. The search, the existing search results page has, um, advertising on as well. Um, and so as we drive more queries, um, you know, those queries monetize and, and, and we, and we have an existing, um, advertising model for that, which also drives traffic, um, um, to advertisers who are selling things or enable people to book hotels and things like that. Um, so that, that's, that's another, um, uh, major piece of it. The third piece is, at the same time as all of that's happening on the, on the, on the, on the results page itself, we're applying generative AI to our advertising products, too. Um, and so, you know, the, in a lot of ways, the holy grail of advertising for the longest time has been an advertising come to us with, here's my site, here's how much I'm willing to spend, here's the ROI goal…

AI assessment note: “The first is, uh, queries continue to grow... propelled by generative AI”

Q speak. Uh, it takes the search from typical keyword searches to something somewhat more conversational. Uh, do you think that the future of search is going to be AI mode style, or is it gonna be both? Because When I've thought about generative, uh, generative AI, ChatGPT, you know, in the early days, I've always thought it's different use cases than traditional search. So how does Google think about it?

A I think there's a large spectrum of, um, questions that people ask, right? Um, if all you wanted to do is get to big technology, you don't have to go have a conversation about it. You're just going to continue to type in the keyword, navigate to the site, right? Um, other times you have harder tasks. Those are the ones that invite more of that conversational style. Um, but sometimes it's not black and white. Somebody asks a question that they think is a quick question and then they get inspired and then they continue the conversation. Um, we were coming up here, uh, And we were wondering how fast the elevator is up to the top. It's pretty fast is the, is the short answer, right. Um, so we gave a response. Then, then I ran and went and read an article about how they said they, they changed the speed of it, um, to actually allow more people to get up to the, um, uh, exploratory deck, um, on One World Trade Center. So I learned about it. Then I had another question to ask. And so I think it came in with a quick fact But then it evolved into a longer conversation. Um, and so I think the thing about search is it's a very broad space, and so there's all different types of questions, and so I don't think use cases are all one or the other. Um, I think we'll see people interact fluidly. I also don't think it's just text, right? You said, is it quick or not? Um, we've rolled out search …

AI assessment note: “I don't think use cases are all one or the other.”

Q Well, then why are we gonna have two different experiences? Is there a point where Google's technology gets smart enough that if I type in, let's say, big technology, it's like, oh, you want the link? Here's the link. Or if I type in a longer query, it should maybe have some understanding of the intent there and answer longer.

A Yeah, I think we, um, absolutely attempt, you know, continue to evolve to try and adapt it. Um, to some degree, that is what the main search page has historically done. Um, we've had image search, which says, okay, if you want a few images, you get them on the main search page, but if you want to dive in deeper, you can go to image search. Right now, that's the way AI mode is really is, okay, if you don't, if you just have a question, you shouldn't have to think about it. Just go to search, ask your question. If you want to dive in deeper, then you can engage with AI mode, um, and we'll see how they both evolve. Does AI mode continue to grow? Um, does the main search take in more of AI mode capabilities and evolve? We'll see where they end up.

AI assessment note: “Right now, that's the way AI mode is really... we'll see how they both evolve.”

Q The thing I've heard is that it's driving, uh, marketers nuts because they used to run by a certain playbook. Now they're not quite sure what to do. Maybe that's good. Liz, what do you think? Is that good that they're not able to game the system just yet?

A I mean, I think, um, what we've always, ah, tried to do with our guidelines is get publishers to focus less on how do they change the ranking, and more on, are you producing content that people want to read? Ok, um, and so, that's what I would tell the marketers. Could you stop working on trying to figure out how you focus on the system? And really just, is this content that you as a user would want to go read and spend five minutes on? Um, and I think the more that we can Surface that content, um, the more it becomes the case that people aren't focused on answering the first part of the question, but really thinking what do they bring to the content, um, what's the, the, the thought in it, the perspective, the experience that they bring, and that's what they're talking about, um, then I think the better everyone will be, and so I think if marketers have to just focus on building content for people, that, that's awesome.

AI assessment note: “if marketers have to just focus on building content for people, that, that's awesome.”

Q character that I speak with and I say, Hey, can you help me figure it out? And instead of being that guy on the sidewalk who everybody hates, like typing and walking, now I'm just having a conversation with the internet. So are we going to end up like Liz, are you already brainstorming this idea of having like a Google search avatar that we speak with or a personality?

A Um, I don't think we've quite thought of it as an avatar, but we have actually, um, rolled out Search Live, um, in labs that allows you to have more of a conversational, um, experience, ask the questions, get a response. Um, we also bring up links so that if you get intrigued by something, you can continue to dig in. Um, I don't think it's necessarily at this point that we're thinking of it as pick your favorite personality. I think it's important that Google has more of a, a neutral, um, and trustworthy one of its It doesn't have a particular angle on things, but brings multiple perspectives, but the idea that it should be fluid in an audio form and doesn't have to be by text is absolutely something we see, and this is how we started to roll out Search Live.

AI assessment note: “I don't think we've quite thought of it as an avatar, but we have”

Q Could there be some collateral damage, uh, with the solution like you're implementing? For instance, I'm looking at the names of these publications, Condé Nast Time, the AP, the Atlantic. I imagine they get a lot of traffic from search as it is today. So if you put this blocker up, does that impact their SEO for instance?

A Yeah. So we've been very, very careful to say that the traditional search today is not blocked. Um, and, and even AI driven search today isn't blocked, but you're going to see us give publishers the tools to differentiate between search indexing, uh, and derivative content. So the way I would think about this is the Google experience today. It may be that publishes. I still want to appear in the 10 blue links, but I don't want to be in the AI overview or the answer box and the granularity of being able to say, okay, Google, I understand you use one bot, but we need that to be treated similarly. And, and again, I am hopeful and in my conversations with Google, um, I, I am increasingly hopeful that they understand the importance of this and giving that granularity. But if for some reason they don't, I am also a hundred percent certain that regulators are paying a ton of attention to this. And that around the world, you will see them force Google to split their crawler out into, into announcing exactly what it is doing. Again, I, I, I think that that's kind of the, hopefully we get to an agreement with Google way before that has to happen. Um, but, but that's, uh, inevitably I think Google is going to have to say, you know, if you don't want us to use your content for derivatives, you have a way of, of, of controlling that. While still appearing in search.

AI assessment note: “we've been very, very careful to say that the traditional search today is not blocked”