The Exchanges

Q Okay. I'm going to email you after this. And then when it comes to cybersecurity, obviously you talked about how you're dealing with all these governments that would like to hack into sites across the web. Have they been able to use generative AI tools or automated coding to get more, to become more effective at what they do?

A Yeah. I mean, I think that, that anytime a new technology comes out, bad guys are going to use it as well as, as good guys. And so we have seen, and we will continue to see some horror stories around, you know, the family that, uh, was tricked by some gang in order to wiring their life savings because something, someone that sounded like their, Their daughter called and said, I've been arrested in Mexico. You know, I need, I need to pay to get out, um, or, or, or other things. I think we were seeing a real rise in, especially out of North Korea, North Koreans posing as if they were, um, applicants, uh, to various, uh, various jobs. And then that is, um, you know, allowing them access, which they can then use to, um, to do, to do any number of nefarious things. All of that again, assisted by AI. So I think that's, um, that's been, that's been sort of on the, on the bad guy side. The, the good news though, is that the good guys, you know, folks like Cloudflare, we have been using AI as well in order to not only detect these things, but get smarter at detecting attacks earlier in the process.

AI assessment note: “All of that again, assisted by AI.”

Q very happy, very clearly is not happy with the New York Times. Uh, pursuing OpenAI and especially the actions that the Times are taking in their lawsuit, like forcing OpenAI to preserve their chat logs, which I think is wrong. Uh, but, but it is interesting. So what do you think about is there, are we going to see an evolution from these one-off deals to this marketplace style world?

A Well, I think that, I mean, we've seen this story many times before. I mean, Napster was along, it was a wild west. There were a bunch of lawsuits from, you know, the publishing, the music industry. Uh, targeting Napster and the like, and then along comes iTunes, which starts out as 99 cents a song, but eventually evolves into what is much more, much closer to a Spotify model of a subscription and a pool of funds that then get distributed out to all the creators. So, so I think we've seen this story before. Um, and I think that one of the things that's really important is that open AI and others are willing to pay for content. They do the deals that are there. And I don't think it's right to just say, we'll do a deal to avoid The void lawsuits. Again, I think that, that when you talk to leading AI companies, they understand that people are doing the work to get, create content. They need to get compensated for that content. And if it's not gonna be through subscriptions or ads or ego, it's gotta be through something else. And so exactly how that happens, we'll figure out. But what I know won't work is if open AI is paying for your content, but you're giving it away for free to everyone else.

AI assessment note: “eventually evolves into what is much more, much closer to a Spotify model”

Q five comes out yesterday or as the release happens, Sam says, I kind of hate the term AGI. Because everyone at this point uses it to mean a slightly different thing, but this is clearly a model that is generally intelligent. Help me understand what's going on, uh, because it seems like maybe, maybe he wants to call it AGI, but you're not yet, so why is this not AGI?

A Well, it is, it is a hard thing to define. Um, you know, you ask the, the joke here is you ask five people what AGI is, you'll get seven answers. Um, and I think the way we kind of look at it is it's a cumulative process, right? It's a system. Um, and I think you have to define kind of what is it that that system is and what do you expect it to be able to do? And for me, at least that's a system that is reliably able to learn new things that are kind of out of distribution by virtue of its ability to reason, to Think to solve problems, to use tools, to come up with new ideas. And so I do, I think we're at a system that I would call AGI. No. Um, but I think we see, we start to see the traces and the, um, the pieces of that overall system for, for generalized learning start to come together, uh, in models like GPT-V and I suspect suspect in its successors. Um, I don't know if we'll have a point where we are like, okay, we've crossed from a non-AGI world into an AGI world. Um, and even if there were, I'm not sure we'd actually realize it necessarily until after the fact, because one of the things we've learned working with the models that we have is the capability overhang is significant. Um, I think when Sam refers to the intelligence of the models and having a PhD in your pocket, we haven't yet really exploited that as, uh, as a thing. Um, you know, in some sense, like, I think …

AI assessment note: “And so I do, I think we're at a system that I would call AGI. No.”

Q know, I think Sam mentioned this on the media call, but GPT-III, he said, was high school level intelligence, GPT-IV, maybe the level of a college student, and GPT-V, an expert. So I guess, I wonder for OpenAI, is the quest to add more intelligence to the mix, Or is it to focus on capabilities other than smarts? Some of the things that you mentioned, like memory and continual learning.

A It's gonna be, I think, all of those things. Um, certainly there are some unsolved problems. Uh, you mentioned a few here, and I would agree with those, um, that, you know, you'd expect a really smart person to, you know, kind of comes by default that our models still struggle with. Um, and so there's open research there that we still have to do, I think, to be able to kind of close the loop on what I would call the full spectrum of intelligence. Um, but, you know, there's intelligence like we were talking about earlier in, in the podcast expresses in a lot of different ways. Um, and part of it is just your, you know, pure IQ. It's your knowledge of how things work and your ability to recall information, but then it's also your ability to reason about how to use other tools to solve problems. Uh, it's your ability to be reflective and to look back on your own chain of thought, your own line of thinking, and actually course correct when you feel like, you know, I actually went down the wrong path and maybe I didn't come up with the right strategy to solve this problem. And so, uh, that's one of the cool things we see is GPT-V on those vectors. Um, we can actually reliably measure as better than the previous systems we had. And for us, I think one of the real world things that we really want to understand is how do they actually perform, uh, in, you know, in, in the real world? H…

AI assessment note: “It's gonna be, I think, all of those things.”

Q level biology, um, the average chatbot user may not feel that even though it's, um, even though it's gotten much smarter. So I guess, I'm curious how, how you think this will be reflected, the increased smarts will be reflected. And the average users chat GPT experience and the plus users experience who've been using these reasoning models for a while, is it going to feel any different for them?

A Yeah. Um, I saw something on, on X that was akin to what you're describing, which someone basically kind of said, I think for the, you know, upper echelon of, of ChatGPT users who are probably in the paid tiers, who are very, you know, active on a daily basis and are really kind of expert level using these systems, it, it, it's gonna feel like an improvement, but maybe a, uh, you know, a more subtle improvement, but for the average user, for the free user, um, and we're, we're bringing GPT-V to our free tier, It will feel like a dramatic increase. Um, if you actually look at kind of the way free users have used ChatGPT, most of them have actually not experienced the power of the reasoning models. Um, they mostly are using GPT-IV-O, um, and, you know, they, they mostly are kind of using it for this very kind of, um, you know, turn-based kind of like very quick, uh, you know, back and forth, almost search-like, uh, that ways that I think don't actually kind of express the full capability of the model. And so for a lot of people, this will be the first time using a model that has reasoning capability. And not only will it be, you know, the first time using it, uh, with reasoning, but it'll be the first time that they're experiencing a model making a decision about how long to think about a problem and how good of an answer to give relative to how hard the question is. And so we ex…

AI assessment note: “for the average user, for the free user... It will feel like a dramatic increase.”

Q And do you think it's gonna work in this model?

A Yes, but I will make a prediction here. And that is that this is, people are going to say this is going to displace Hollywood. And I think this is just an entirely new category in and of itself, because I think one of the things, one of the great enjoyments of watching good entertainment is not being too involved in the creation process, sitting back and being surprised by the brilliance of the people that are creating The content that you're watching. Uh, same with video games. Like part of the enjoyment of video games is, you know, not necessarily like creating your own path is just like following the, you know, brilliant output of thousands of games designers. If you're playing something like Assassin's Creed and not having to really worry about doing that yourself. Like, I think our brains have two different modes. We have like kind of doing and consuming, so to speak. And, um, This type of AI creation actually puts, ah, the consuming bucket in a more of a creation place, if that makes sense. So I think that if we're gonna basically transpose the entertainment side of our lives into the creation part of our lives, it doesn't displace entertainment, it just creates a new category that sits maybe right between those two.

AI assessment note: “Yes, but I will make a prediction here.”

Q model, um, has been confounding to some. You can spend 200 dollars a month and get the equivalent, I spoke to one developer, they got the equivalent of 6000 dollars a month, ah, from your API. Um, Ed Zitron has pointed out, the more popular that your models get, the more money you're going to lose if people are super users. Of this technology. So how does that make sense?

A So, um, uh, so actually pricing schemes and rate limits are surprisingly complicated. Um, so, so some of this is basically the result of when we released our, um, uh, when we, when we released Claude Code in the max tier, which we eventually tied together, actually not fully understanding the implications of, you know, the ways in which people could use the models and how much they were actually able to get. Over the last few days, as of the time of this, uh, as of the time of this interview, we've adjusted that particularly on the larger models like Opus. I think it's no longer possible to spend that much, um, uh, uh, with, uh, with a, with a 200 dollar, uh, uh, subscription. And, you know, it's possible more changes will, more changes will come in the future, but we're always going to have a distribution of users who use a lot and, and, and users who lose, who use some amount. And it, it doesn't necessarily mean we're losing money that there are some, Some users who get more, um, uh, you know, who, if you were to measure via API credits, spend, you know, get, get a better deal on the consumer, on the, on the consumer subscription than they would on the API products, right? There's a, there's a lot of assumptions there. Um, uh, and I can tell you that, that some, that some of them, that some of them are wrong. Um, uh, we are not in fact, uh, losing money.

AI assessment note: “we've adjusted that particularly on the larger models... we are not in fact, losing money.”

Q developer customers. But as we've reported, this is still the information. OpenAI is more than aware of this issue and has been working in recent months to improve the coding capabilities of its models. I mean, what are the implications that if OpenAI is, let's say, able to equal or pull ahead with Anthropic, which we know is the state of the art in coding, uh, with this new model?

A Well, they've pursued very different, you know, you know, subscription revenue models so far. OpenAI has You know, it's almost the verb, like to Google, you know, you chat or you GPT the question, especially if you're a young student, whereas Anthropica's claw doesn't just, just doesn't have the same brand recognition, but it has relentlessly gone after what they call enterprise customers, like big businesses, offers them access to their technology through APIs and has longer, bigger, and more visible contracts. OpenAI has long been jealous of this. It wants in on the game. Uh, it's also competing with Microsoft. Yeah. It's its own partner in offering these services through, you know, the Azure platform, but increasingly Google and Gemini, which, which trumps its, its coding chops. So if open AI is able to prove that its models are at least as good. If not much better, then it can start to take back some of this. And it really does change the, the competitive landscape. Cause I think GPT is the, you know, undisputed winner of like the consumer chatbot wars so far. What it hasn't proved is that it can make the transition to the business and governmental world in the same way that some of its competitors have. And maybe they were forced to go down that route because open AI was just sucking all of the oxygen out of the room on the app store.

AI assessment note: “then it can start to take back some of this. And it really does change”

Q Now I think a lot of people will take issue with my first question. They'll be like, the U.S. is not in the lead, or who cares? And it's not important. The U.S. can develop technology. China can develop technology. So why do you think it's important the U.S. stays in the lead, and what are the consequences if it falls behind?

A Well, we're in an arms race in technology because, ah, there are many things about the U.S.-China relationship that are not adversarial. They are the two largest economies. We are going to have to find a way to trade together. But, ah, in security policy, we are adversaries. And, ah, I would say that's largely a decision that Beijing made. So as a result, the technology race, ah, is also adversarial at the high end. You know, I think one of the really kind of silliest, ah, Statements that I made, or maybe I, I would say kind of a dumb speech, if you will, is when Xi Jinping said that they were going to surpass the United States in frontier technologies like AI and quantum, and he gave a date within essentially 10 years. So what did he think was going to happen? We were going to get our backs up, we were going to start to think of it as an adversarial race, and, um, I am one who believes that if somebody's going to win the race on these frontier technologies, it had better be a democracy. Because if something goes wrong in AI, and it's quite possible that something will, as a matter of fact, it's probable that something will. Maybe it's even predictable that something will. We will have, um, investigative reporting. You'll probably be doing it on your show. We'll have, uh, congressional hearings. The Chinese will do what they did with COVID. They'll hide it. They'll lie about it…

AI assessment note: “if somebody's going to win the race on these frontier technologies, it had better be a democracy.”

Q international students in a moment, because it's, it's not pretty, and we should discuss it in more depth. But let me ask you, how do you, why do you think this is happening? Um, I'm gonna give the Trump administration's rationale in a moment. Um, but just from a philosophical level, how do you get to the point where you start to see university funding as something you can pause?

A I've been a university professor for more than 40 years, right? They hired me when I was 11. I just want that to be understood by your audience. But, but, um, I think universities have become detached from society and from reality as well. And, uh, so it's not just, uh, it goes a little bit both ways. What do I mean by that? Uh, clearly we haven't made the case very well for what we do. Maybe it's that, uh, people take for granted Some of the, ah, innovations that have come out of universities, but if you walked and asked even a very highly educated, ah, member of the attentive public about how the research system that we just described worked, they probably wouldn't know. So maybe we shouldn't take that for granted anymore. Maybe we should make it clear why this is happening. Ah, secondly, I do think that, ah, universities and elites sometimes have looked down On people who, quote, weren't their own kind. I do think that the stories that come out about the running down of American values, American institutions, America is too racist, America's, you know, people get tired of that, and they don't like the attack on their country, and they don't like the attack on their culture, and unfortunately, it's become a little bit associated, it's become a lot associated with elite universities. So, I would say to us, let's look in the mirror a little bit, too.

AI assessment note: “I think universities have become detached from society and from reality as well.”

Q off. We've discussed it on the show that it seems like it's harder to get an entry level job, uh, today than it ever has been. Uh, maybe outside of recessions, uh, like in a good economy, it seems like it's difficult. So do you ascribe any of that to the AI boom, or is that just something else that's happening and we're sort of conflating the correlation and causation?

A Um, I think, uh, we don't know. So, um, the answer is that we don't know what the hell's going on. You know, I've seen data on like entry-level jobs, you know, there, there has been a slowdown entry-level hiring, but there's a number of reasons this could be. So one reason is the macroeconomy. One reason is, um, two, there was over hiring of entry-level jobs during the, like, twenty-twenty-one during the bubble, and now people, like, glutted on entry-level jobs and are now cutting back for a while. Um, there's, you know, there's other stuff too, and there, there's, there's the glut of people who just all stampeded into the same few majors and career paths. And so, um, yeah, so, so there's all these things going on. Uh, and it's not, it's very much unclear as to what's going on. So there was a good economist article by someone who really doesn't believe this thesis, you know, claiming to debunk it all. And I, I don't think they completely debunked it, but they raised lots of other possible explanations that were pretty sufficient, I think, to explain it. We don't know which is which. It could still be AI. This could still be what's happening. Uh, but I think that we don't have enough evidence to conclude one way or another what's going on there.

AI assessment note: “the answer is that we don't know what the hell's going on”

Q ports, and others are like, well, you know, it's, uh, everything is back to normal, and, and Trump will, you know, everybody's anticipating that Trump will chicken out again, and you have the S&P, which, you know, the day we're recording is up nearly four percent on the year. It was down 15% on the year April eighth, so an unbelievable rebound, uh, from the S&P. So what's going on?

A So I do think that the reason why the financial markets are doing well is because people have decided that Trump's going to chicken out and that these pauses will be permanent. And then they'll announce quote unquote deals that allow us to cut all these tariffs. And that basically Trump's a bag of wind. I think traders are betting on that. It's pretty obvious. Um, and the reason why financial markets abruptly crashed when Trump announced the tariffs was because they thought this is for real. When they decided it wasn't for real, they bought and finite financial market prices, asset prices went back up. Um, however, do I think that that's really true? I think that there's some, there's some issues. There are some negative effects. That these tariffs are going to have that even if Trump does chicken out, there's uncertainty about what if Trump doesn't chicken out on this or that. Also, it's not like tariffs haven't gone up. They have gone up just like more, much more modestly than Trump was threatening. That will go up a bunch of things. How big an impact it's going to have is hard to say because currency movements adjust and other things adjust to like balance it out. But like, it's going to be, there's going to be some pain from that, especially in certain sectors. And, uh, Um, the threat of tariff-driven inflation will lead the Fed to keep interest rates higher a little longer…

AI assessment note: “the reason why the financial markets are doing well is because people have decided”

Q Okay. And then, so what is a CEO looking at inside the platform when they're making choices?

A Yeah, there's three ways to consume information in our platform. The first one is in what we call the research AI, and that's where you Typically see most people comfortable. It's an interrogation center, a prompting center. You can ask any question, but it also has an entire library of all of the metrics available. So as you send a new data source, every 24 hours the AI calculates that and adds all of that metric library into the entire experience. We call that the large metric model. We haven't heard anybody say that yet. That one's ours. So this large metric model is where you start from. And then you start to favorite and star and tune and then provide feedback loops where the AI picks up on these preferences and starts to build around that. Then it moves up into heat maps and you have business units. So let's say you have finance, sales, marketing, ops, customer success. All of these different business units are automatically heat scored as well. And then the final layer is the very top, which is that there are so many things to action on. How do we prioritize those? And the AI starts to sort and filter and give you what's most important to decide today. Alex, I think our goal here is that you get your coffee in the morning and you get your snow fire and you program your day.

AI assessment note: “there's three ways to consume information in our platform. The first one is”

Q Yeah, well, anyway, I used to not be afraid of these AIs, and now more and more, I'm like, uh, there's some fear here. So by the way, what do you think about a CEO's job? Because it does change the CEO's job as well.

A I think they're the biggest benefactors in this new age. Yeah. Being a CEO is, uh, it's exhausting. You wake up every morning and your best part of you is trying to solve a problem. The best CEOs that I know, they don't go to work and they're like, oh, I'm gonna build this today, or I'm gonna, uh, I'm gonna dream this up today. No, the best CEOs that I know that are running the best companies, they just fix problems. And so, what I think is gonna be cool about what we're doing is, and maybe this is a little selfishness talking here, is I wanna have something tell me what problems I should be focusing on every morning.

AI assessment note: “I think they're the biggest benefactors in this new age.”

Q 1500 pages crawled, One visit. Uh, Anthropic, it's, was, it's now gone up from 6000 to 60,000 pages crawled. One visit. Prince says, people aren't following the footnotes. Is this time to panic? I mean, I sort of, I'm gonna title this, uh, this conversation or the, the beginning of this podcast episode, The Web's Existential AI Threat. Am I getting over my skis, Reid? Am I, is this hyperbole?

A I think it's always a great time to panic in the media business, to be honest. Um, and I'm, and that's not just a joke. I mean, I think it's, it's like, I remember, you know, being in, uh, you know, studying journalism in college and talking about, you know, panicking, uh, in the media business. And at that time it was, you know, just, just the web was, uh, was completely destroying it. I think what's interesting and Matthew Prince has been on this, you know, he's been talking about this now for a while. I've talked to him about it. I think it's, I think it's admirable what he's doing, but I also think that, you know, traffic was never a good metric to judge whether, you know, a story is valuable and media articles valuable. So if that metric goes away, then yeah, I mean, it's going to hurt the, it's going to hurt the current industry, but I think it actually is, is maybe in the end, a positive thing. Um, and I think there's, there's two things going on. I mean, one is the traffic and the other is like, You know, that they're scraping all these websites and pulling in the information and that part, if you, if you get rid of traffic as a metric, you don't need, uh, you don't need to allow these things to scrape your websites, right? I mean, it's possible to, to put a hard paywall in place. And to essentially stop that scraping. So, but then you don't get, then you don't get the …

AI assessment note: “I think it's always a great time to panic in the media business”

Q it was definitely, it was definitely a mistake. It was definitely a screw up, but nothing that you wouldn't see, I think from, I mean, even these Waymo's sometimes will do screwy things and, but ultimately like, you know, they didn't hit any pedestrians. They didn't get in any accidents. I think, you know, In the end, I, I think it was a pretty successful launch. What do you think?

A Yeah. I mean, I, I watched the video and I mean, look, the thing is, this is the thing with all tech. If it works 95% of the way, that's not good enough. If it works 99.5% of the way, that's not good enough in self-driving cars. So, um, things have been fine so far, uh, but it definitely has, we need to see a lot more. To decide whether or not this was, this is going to really work for them. If it works, obviously it's a major, major boon, uh, for the company. And certainly from the videos, it looked a lot like the Waymo experience. You get in the car's driving itself. Um, They have some predetermined routes. There are tele operators. So there are some shortcuts being taken so far. And there is, there are people that are part of this, you know, people list driving experience, but I think, yeah, it's important there. They are off to the races. They're gonna get, they're gonna get moving. They're gonna get data. And, um, I don't know why I'm, my fingers are across that they pull it off and pull it off safely. Uh, but I think it's too early to tell.

AI assessment note: “we need to see a lot more... but I think it's too early to tell.”

Q if this was a thing that could persist. We had the Fed raising rates, making the economy You know, with the intent of slowing the economy down And then in comes ChatGPT. And what happened since then? I think the S&P 500 is up, what, 50% since then? So, with that context, I want to ask you again, how much is AI responsible for where the stock market is today?

A Yeah. And again, I'm gonna point out, it's the narrative. Like, that's really what people talk about, but it's not even what's not necessarily driving share price gains. I mean, a few stocks have, of course, Done very, very well in terms of share price gains. But the reason I, I'd say it's not necessarily the thing driving the stock market is that there's other narratives that do drive multiple expansion. So for instance, if we had this AI story, but we thought the Fed was going to continue to tighten and actually reduce monetary Liquidity , then we wouldn't actually have a rising stock market. I don't know if AI could power through a Fed that would be trying to kill the economy, or if oil went to 300 because of some geopolitical event and was at a sustained level, and so we had a recession. I don't know if we could have, um, a market doing well, and I don't, as strange as it sounds, I think it would be tough for the AI trade To work just because cost of money would be so high, or, you know, you'd have a lot of companies getting super cautious, and then AI would be forking and developing somewhere else outside the US. So, the answer is narratives drive prices, but AI isn't the only sort of story in the stock market.

AI assessment note: “narratives drive prices, but AI isn't the only sort of story in the stock market.”

Q rate to get proof of concepts out the door is pretty small. One out of every five actually gets shipped into production, and often it's a scaled down version of that. So what you're saying is interesting. You're saying it's not their fault. It's that these models are not reliable enough to do what they need to do because they don't learn on the job. Am I getting that right?

A Yeah, and you're talking about reliability. It's just, um, they just can't do it. So if you think about what makes humans valuable, It's not their raw intelligence, right? Any person who goes onto their job the first day, even their first couple of months, maybe they're just not going to be that useful because they don't have a lot of context. What makes human employees useful is their ability to build up this context, to interrogate their failures, to build up these small improvements and efficiencies as they practice a task. And these models just can't do that, right? You're stuck with the abilities that you get out of the box and they are quite smart. So you will get five out of 10 on a lot of different tasks that they'll Often they'll, on any random task, they'll probably might be better than an average human. It's just that they won't get any better. Um, I, for my own podcast, I have a bunch of little scripts that I've tried to write with LLMs where I'll get them to rewrite parts of ION scripts to make them more, uh, turn auto-generated transcripts and do like human written like transcripts or to help me identify clips that I can tweet out. So these are things which are just like short horizon language in language out tasks, right? This is the kind of thing that the LLM should be Just amazing guy because it's a debt center in their, uh, of what should be in their repertoir…

AI assessment note: “Yeah, and you're talking about reliability. It's just, um, they just can't do it.”

Q dollars. There is a pressure to deliver to investors. There are reports that safety is becoming less of a priority as market pressure makes them go and ship without the typical reviews. So is this kind of a risk for the world here that these companies are developing this stuff? Many started with the focus on safety and now it seems like safety is taking a backseat to financial returns.

A Yeah, I think it's definitely a concern. Like, We might be facing a tragedy of the common situation where obviously all of us want our society and civilization to survive. Um, but maybe the immediate incentive for any lab CEO is to Look, if there is an intelligence explosion, it has a really tough dynamic because if you're a month ahead, you will kick off this loop much faster than anybody else. And what that means is that you will, uh, you will be a month ahead to super intelligence, but nobody else will have it right. Like you will get, you'll get the 1000 X multiplier on research much faster than anybody else. And so it could be a sort of winner take all kind of dynamic there. Um, and Therefore they might be incentivized. Like I think to keep this system, keep this process in check might require slowing down, um, using these alignment techniques to like that, which might be sort of a tax on the speed of the system. And so, yeah, I do worry about the, the pressures here.

AI assessment note: “Yeah, I think it's definitely a concern. Like, We might be facing a tragedy”

Q Yeah. Can, so can you talk about what precisely it does different than traditional reasoning models?

A Like the current, um, reasoning thinking models, most of the time, at least I can talk from, from our research point of view, builds a single chain of thought, right? And then as you build a single chain of thought, and as the model continues to attend to its chain of thought, it builds a better understanding of what response it wants to give you. It can alternate between different hypotheses, reflect on what it has done before. Now, Of course, like one, if you think about it just also in a visual kind of space, one kind of scalability that you can bring onto the table is, can you have multiple parallel chains of thoughts so that you can, you can actually, um, analyze different hypotheses in parallel, and then you will have more capacity exploring different kinds of hypotheses, and then you can look at, you can compare those, and then you can eliminate the ones, Or you can, you can, you can continue pursuing, and you can sort of expand on particular ones. It's a very intuitive process in a way, but of course it is more involved.

AI assessment note: “can you have multiple parallel chains of thoughts so that you can”

Q Sergey, this is a good one for you. Do you think that AGI is gonna be reached by one company and it's game over? Or could you see Google having AGI, OpenAI having AGI, Anthropic having AGI? China having AGI?

A Wow, um, that's a great question. I mean, I guess I would suppose that one company or country or entity will reach AGI first. Now it is a little bit of a, you know, kind of a spectrum. It's not like a completely precise thing, so it's conceivable that there will be more than one roughly in that range at the same time. After that, what happens, I mean, I think it's very hard to foresee, but you could certainly imagine there's going to be multiple entities that come through and In our AI space, you know, we've seen, ah, whatever, when we make a certain kind of advance, like other companies are quick to follow, and vice versa, when other companies make certain advances, it's, you know, it's a kind of a constant leapfrog, so I do think there's an inspiration element that you see, ah, and that would probably encourage more and more entities to cross that threshold.

AI assessment note: “you could certainly imagine there's going to be multiple entities that come through”

Q So what you're saying is basically that a human customer service agent is the quality of, is the equivalent of, uh, artisanally produced good?

A Um, I think partially, maybe not so much that, because what I wanted to get to is that once that happened, we started suddenly appreciate crafted items, right? So today, if you buy a piece of furniture that is done by artisan or, uh, an artist or somebody, you know, is done like, uh, humanly crafted, we actually pay a higher price for that than we pay for something that comes out of a standard factory in some, you know, uh, and it's just like manufactured by a machine. And so, our conclusion reflecting on these things was that, like, the human connection matters. People Appreciate talking to a human. They feel a human connection. There's an emotional connection. And we believe that like this means that there will be a higher appreciation. And if a company wants to be competitive, it, it will actually be a competitive edge to offer a human connection. And so, but obviously that's kind of a different type of human connection than maybe some of the Customer service we offered historically, because this will have to be, you know, high quality, ah, ah, Skilled people that are very familiar with Clona and understand Clona, and that was not always the case, right? We relied a lot on, like, outsourced agents. They maybe would come in. They barely knew our product. They were just, like, asked according to some very strict template to answer, like, have you paid or not paid? And that was…

AI assessment note: “I think partially, maybe not so much that, because what I wanted to get to”

Q fact that it's able to go and comb the internet for these events and then take into context some of the, um, the context that I've given it with these documents, I think is very impressive. And that's just one use case. So I was, I'm not asking it to be a lawyer. I'm kind of asking it to be what you said, an itinerary planner. What's wrong with that?

A Uh, so, I mean, first of all, you, you have these lovely documents from your friends, and I guess what you're saying is missing is whatever current events are. So they've given you some sort of like, these are general things to look for, but they haven't looked into what's going on right now. Um, what's wrong with that? You know, on several levels, um, what, what would we do in a prior age, like even pre-internet, right? The local newspapers would list current events. Here's what's going on. If you landed in a city, you would go find The local, probably local indie newspaper and, and look up the events page. And that system was based on a series of relationships within the community between the people putting on festivals and the newspaper writers, and it helps support probably the local news information ecosystem, which was a good thing. Um, but on top of that, uh, if something wasn't listed, you could think about why is this not listed? What's the relationship that's missing? Um, Your ChatGPT output is going to give you some nonsense, and you're right, this is a use case where you can verify whether this is real or not. Um, it is also likely going to miss some things, and the things that are not surfaced for you are not surfaced because of the complex set of biases that got rolled into the system, plus whatever the roll of the die was this time. Um, and anytime someone says, …

AI assessment note: “What's wrong with that? You know, on several levels”

Q But isn't the black box the thing in the plane that tells you what actually happened?

A Well, that's a different thing, right? So a black box in a plane is actually a flight recorder that records a lot of data. But what we mean in machine learning by black box is you have a model where you have the inputs and you have the outputs. You know how you calculate them, but you don't really understand how the system gets there. So in this case, you're doing all this matrix multiplication. Nobody really understands it. And so nobody can actually give you a straightforward answer for why O three hallucinates more than GPT four. We can just observe it. That's what happens with black boxes is you, you empirically observe things, And you say, well, it does that, but you don't really know why, and you don't really know how to fix it either. Another example, just in the last couple of days is apparently Sam Altman reported, I forget the new model is, is stubborn or what was it?

AI assessment note: “Well, that's a different thing, right? So a black box in a plane”

Q Because the idea was people would just, you know, work from home, shop from home. Um, the idea was that that was a little bit too much, uh, because people went back to old habits, but it would show up. As a benefit because the company would just get much better logistically with all of that investment. Is that, is that true? And is that playing out for you guys?

A Yeah. So the, I think the summary I would say is, uh, just tweaking a little bit what you said, we built as aggressively as we did in the pandemic in order to keep up with demand. We wanted to, um, ensure, uh, superlative customer experience. The first, uh, weeks, months of the pandemic were Unprecedented for us and everybody else, given the supply chain disruptions. And so we, we, you know, uh, prime became both on the delivery and shopping side and also on the entertainment side sort of be became indispensable, uh, for all of us around the world. Um, and so we built up our, um, warehouse, uh, capacity really, really rapidly. Um, like you said, of course, then, you know, there's a lot of businesses. There was a slowdown as the, You know, the, the world opened up and so on and so forth. The reality is while we've right-sized some parts of that supply chain footprint on the margin, we've largely grown our business into that capacity, right? And we've continued to invest in new capacity. We've continued to invest in robotics, um, uh, and, you know, keep on making our supply chain, um, more and more performant. The biggest change we've done is also flipping the supply chain to a regionalized supply chain, which has put More inventory closer to the customer. Um, and that has sped up our speeds and also reduced our cost to serve. And so we've seen a lot of goodness, um, in terms of …

AI assessment note: “we've seen a lot of goodness, um, in terms of the investments we've made”

Q Right, so how does the model then learn how to, you know, figure this out, and in what context is it accurate to say blue and red? Right.

A So, so I mean, um, first of all, the model doesn't just output one token, right? It outputs a distribution. Um, it turns out the most, the way most people take it is they take the top, uh, top K, i.e. the most high probability. So yes, blue is obviously the right answer if you give it to anyone on this planet. Um, but there are situations and contexts where the sky is red is the appropriate sentence, but that's not just an isolation, right? It's like if the prior passage is all about Mars and all this, and then all of a sudden it's like, And that's like a quote from a Martian settler, and it's like the sky is, and then the correct token is actually red, right? The correct word. Um, and so it has to know this through the attention mechanism, right? Um, if it was just the sky is blue, always you're going to output blue because blue is, let's say, 80%, 90%, 99% likely to be the right option. But as you, as you start to add context about Mars or any other planet, right? Other planets have different colored, colored atmospheres, I presume. Um, you end up with this, um, distribution starts to shift. Right? If, if I add, we're on Mars, the sky is, you know, then, then all of a sudden blue goes from 99%, you know, in the prior context window, right? The, uh, the text that you sent to the model, the attention of it, all of a sudden it realizes the sky is blue, uh, preceded by that, the,…

AI assessment note: “and so it has to know this through the attention mechanism, right?”

Q Okay. And so why is it called pre-training?

A So, so pre-training is, is, is sort of called that because it is what happens, you know, before the actual, uh, training of the model, right? Uh, the objective function in pre-training is to just predict the next token. Uh, but predicting the next token is not what humans want to use AIs for, right? I want it to ask a question and answer it. Uh, but in, in most cases, asking a question does not necessarily mean that the next most likely token is, is the answer, right? Oftentimes it is another question, right? Uh, for example, if I ingested the entire SAT, um, you know, and I asked a question, the next, like, five answers, then all the next tokens would be like, A is this, B is this, C is this, D is this, like, no, I just want the answer. Right? Um, and so pre-training is, the reason it's called pre-training is because you're ingesting humongous volumes of text no matter the use case. Right? Um, and you're learning the general patterns across all of language. Right? I don't actually know that king and queen relate to each other in this way. And I don't know that king and queen are opposites in these ways. Right? Um, and so this is why it's called pre-training is because you must get a broad general understanding of the entire sort of world of text. Before you're able to then do post training or fine tuning, which is let me train it on more specific data that is specifically usef…

AI assessment note: “called that because it is what happens, you know, before the actual, uh, training”

Q Yeah, so what are these new attacks like?

A Yeah, so, I mean, we see some of these at Vanta too. There's a lot of impersonation. Um, it is just easier to, you know, I can think of the old way was the joke at Vanta was, if you joined the company, you would get three text messages from me asking me to buy you gift cards, but things were probably like somewhat misspelled, and you're probably wondering, you know, does a CEO really need me to go out and buy gift cards? Again, it was sort of this, you know, attack that was easy to laugh about. Um, that one still happens, but there's versions of that that are much more compelling. So CrowdStrike has recently talked about How they had one of their customers, um, I think it was the CEO, but an executive at one of their customers actually impersonated in this credible way and, you know, go ask employees that I think over video to go do something for them. And, um, so again, it's kind of, you can see there's some of these same attacks, but just way more compelling.

AI assessment note: “There's a lot of impersonation... versions of that that are much more compelling.”

Q And now, are you using artificial intelligence tools to be able to, you know, put together these reports, or to be able to address some of the concerns that we were discussing earlier on?

A We are in a, in a bunch of different ways. Um, we started building AI and from the ground up across our product. So everything from helping a company decide, uh, which security controls are best for them, given their maturity, given what their customers are asking for. Um, when there's a, you know, error or vulnerability, how do you fix that? Uh, AI is helpful for giving a, again, first pass or a suggestion for the team to take action on. And then, um, a lot of the process of these audits or requirements is just a lot of documentation. Back to the security questionnaires I talked about, um, at the beginning. Uh, there's, you know, you have to do the work, and then you have to tell people about how you did the work. Uh, when I talk to security professionals, they usually really like doing the work. That's why they got into the job. They generally don't like having to tell people about the work in a bunch of different forms. So, you know, an audit, a screenshot, a security questionnaire, and a conversation. And so one of the places we've seen AI Be really effective is again to take all that work that was done, summarize it, and put it in the right format, um, so the teams can focus on actually doing the work.

AI assessment note: “We are in a, in a bunch of different ways.”

Q Yeah, and can you talk about how that happens? So is it basically a generative AI model that's monitoring, uh, the people as they work, and then being able to sort of translate that activity into natural language? Talk a little bit about how that, how they operate that way.

A So it is definitely generative AI models, um, kind of the foundation models, plus some post-training, um, and a ton of kind of quality improvements to make sure, uh, there, there's high accuracy from security and compliance of the place where I think there is less tolerance for creativity and true generation and, you know, more, more desire for accuracy. But basically take these models and not observe, uh, what people do, but observe the outputs. Um, so what have they done? Like what was there before? What did they do? Look at the output rather than kind of the work itself. But then take those outputs and say, okay, maybe you went and, I don't know, changed a bunch of configurations in your cloud infrastructure, um, in a way that's more secure. Okay, we can take that new configuration, uh, can turn that into a policy document. So there's a standard kind of document written in sort of legalese or compliance that describes what there is. Take that configuration, um, translate it to answers for the security questionnaire. So the next time you're asked about Your cloud infrastructure configuration , you got the answer there, direct from, from, you know, the reality, um, of what's in the system live, uh, and then finally take some of, again, take that same information, map it to a bunch of compliance frameworks, and so if you're going through an audit or going for a particular, um, …

AI assessment note: “not observe, uh, what people do, but observe the outputs”