The Exchanges

Q back and talk about what that news is, but also more broadly, like, why are bots in the conversation? And also just this, I mean, do you guys even agree with this notion that there's like, it's like, it was web, and now it's apps, and now it's bots. And by the way, we should probably start by talking about what the hell a bot is in the first place.

A So yeah, there's, there's different ways you can come at this question. So what Facebook has announced is Is a platform that lets a brand talk to you inside Facebook Messenger. Um, it sends a message, you reply, it can see the message and produce a response. And the messages from the brand can, are a bit more sophisticated than just a bit of text in that they can be a card, they can have a couple of calls to action, they can put three or four cards up that you can swipe back and forth. Um, which means that you don't have to know what you can ask in the way that you would otherwise. You don't, you're not faced with just kind of a blank command line the way you would with like an SMS service that you might have been using 15 or 20 years ago. Um, you get a little bit more of kind of an interactive model. Um, and so they've launched, there's a payment more plan in there. There's a few other bits and pieces. Very oddly, there's no sort of social or viral mechanic in it, which we can talk about a bit later. Um, and this fits into, like, two or three big industry preoccupations. Like, there is one preoccupation, which is AI and natural language processing, and the sense that you can sort of So you can talk to the bot and it understands what you said and send something back. There's a second preoccupation, which is, um, finding like another runtime after the web and mobile apps, prefer…

AI assessment note: “So what Facebook has announced is Is a platform that lets a brand talk to you”

Q Okay, so that raises the question then, and this is always the question you have to ask with free services, what's in it for Google? And I'm talking about Google Photo right now.

A Um, I think there's, there's several answers to that. One of them is they get all the photos, and they can analyze them, and that's more data for them to understand, the same as Google Books say. It just gets data in, and then think about it, and you can work stuff out from that. The second is, the more that you're logged into Google, the more that you're using Google services, the more that they have a sense of who you are, and your identity, and where you've been, and what you might be interested in. And that translates through to better Google Now recommendations, for example, and Better map directions and all sorts of things and better search results customized for you. It also, of course, results in more relevant advertising. Um, I mean, I think the, the, the fundamental way to understand Google is as a vast machine learning engine and everything that they do is about reach. And they don't really care what the reach is as long as they've got more and more of it. Um, you know, and that's reach both getting data in and reach getting data out. Um, and which device you use or what kind of data it is, is less important than the fact of the reach. Which is why this is on iOS.

AI assessment note: “One of them is they get all the photos, and they can analyze them”

Q Last question. Is there anything from the presentation that you want to make sure, uh, listeners leave with?

A The thing that I, I used last year, and I, I used again, is an IBM ad I found from the early fifties, which has got a picture of a sea of, of engineers all holding up slide rules. And there's an IBM ad, and it says, you know, an IBM electronic calculator gives you a 150 extra engineers. And that's like, how many pictures have you seen at A-sixteen Z where that was the pitch? Um, and we kind of remember, like, we go through these waves of these, these fundamental technology changes every 10 or 15 or 20 years. And they're all amazing, and change everything, and are completely unlike anything that's happened before. And so, AI is amazing and transformative and completely unlike anything that's happened before. Mobile was quite a big deal too, and so was the internet, and so were PCs, and so was computing. Those were all also very big deals where it was hard to tell what was going to happen, and so we should sort of presume as a base case, ok, well we're going to go through that again, and you know, that will produce a bunch of things that ruin people's lives, and it will put a bunch of people out of work, um, and You know, there'll be a bunch of stuff that we're not very happy about, um, and there'll be a bunch of stuff that we all think is great, and then in 20 years time we'll kind of forget that there was a world when computers couldn't do that. Um, I mean, here we are, we've b…

AI assessment note: “we go through these waves of these, these fundamental technology changes”

Q Yeah. And just on, on the, on the coding, how could, could we have figured, could we have foreseen that that would have been the, the, the use case that really would have taken off or what sort of reflection on that?

A Well, um, every, deterministically, you could have said, well, look, who's messing about with this stuff? Software developers. What are software developers going to try and make work? Software development. Um, so, you know, at a very kind of simplistic, naive level, well, yeah, the stuff that should work is software development. First is software development, just as like, kind of, I often compare this moment to like the internet in like, 97, 98, but it's also like the PCs in the early eighties or the late seventies. It's incredibly exciting, but it's not quite quite clear what it's for, and it doesn't quite work yet, and clearly the first thing that people did with PCs was make computers, um, and the first thing that people are doing with LLMs, and in a sense LLMs are computers, is to make more compute, um, and so that's not terribly surprising. I think this shift has been at the beginning of this year, clearly, that our agentic coding went from being kind of useful to really changing everything. And I'm not sure you could have, you, clearly there were people who would say, well, this is going to be able to do absolutely anything. And so they will say, well, yes, look, I told you. Um, but I don't think anyone kind of, kind of, could have determinously predicted exactly when that was going to happen and that it was going to be coding at Woodwork first.

AI assessment note: “I don't think anyone kind of, kind of, could have determinously predicted exactly when”

Q new and, and, and different questions or what, what is sort of your, you know, if I woke up on a, in a coma, uh, after reading your, you know, your original presentation, let's say, you know, the one after GPT three launch, uh, came out, um, and then seeing this one now. What were the sort of most surprising things or things that we learned that updated those questions?

A So I think we have a lot of new questions this year. So I feel like, you know, you could make a list of, as it might be, half a dozen questions in spring of 23. Like, open source, China, Nvidia, does scaling continue? Um, what happens to images? Um, does, how long does OpenAI's lead remain? And those questions didn't really change in 23 and 24, and most of those questions are kind of still there. Like, the Nvidia question hasn't really changed, you know, the answer on China. The answer on, you know, well, how many models will there be? The answer is, okay, there's going to be anybody who can spend a couple of hundred, you know, can spend a couple of billion dollars can have a frontier model. That was pretty obvious, you know, in early 23. It took a while for everyone to understand that. And big models and small models, will we have small models running on devices? No, because the small models, the capabilities are moving too fast for the small models to shrink the small model onto the device. But those questions kind of didn't change for two, two and a half years. I think we now have, I think, A bunch of more product strategy questions as you see real consumer adoption and open AI and Google building stuff in different directions, Amazon going in different directions, Apple trying and obviously failing and then then trying again to do stuff. There's some sense of like, there is…

AI assessment note: “I think we now have, I think, A bunch of more product strategy questions”

Q And again, as far as the ecosystem, Chris Dixon here at the firm, he talks about it as a new kind of medium. Um, yes. Does this push that, that access to it much quicker?

A Yes. It makes it much easier. Um, you know, it's the box brownie story all over again. I think the, um, you know, that doesn't mean there aren't a whole bunch of challenges of what you actually do. Um, I mean, there's a sort of, you know, I think three D video is really at the kind of pre Eisenstein stage. You know, somebody has actually got to work out, Hey, you can cut, um, And you can move the camera and then start filming again. Right. Oh, wow, wow, what does that mean? You can do montage, you know, you can have a car crane on a camera, and this is all the stuff that had to be invented because people started out just filming a theatre, um, and it takes a while to work out you can actually move the camera and you can cut the film, and it's exactly the same thing for VR, um, you know, what does it mean for a director to be shooting a scene where you are Focusing on where you want people to look, and you're close, in your choice of lens, and you close up, and you're, you know, how you frame the shot, when the person who's watching this can kind of turn their head and look out the window.

AI assessment note: “Yes. It makes it much easier. Um, you know, it's the box brownie”

Q And various people, including, you know, uh, CEO of Google have said that the risk of under-investing is riskier than over-investing. Is there any level of capex where that stops being true, and are we getting there now?

A Well, there's a financial gravity problem in that, um, Microsoft, Meta, and Google are all on, in line to spend over 50% of revenue on CapEx this year. And, you know, we think of telecoms as being capital intensive. Telecom spends sort of 15 to 20% of revenue on CapEx. Um, and so, You know, seven hundred billion dollars is the guidance from the big four companies this year. Well, you know, telecoms is 300, mobile is 200, total telecoms is 300. Oil and gas, depending on which Definition of which bits of it you're counting is anything from seven hundred billion to a trillion dollars, I think, from memory. Depends which, exactly who you ask. Um, so seven hundred billion dollars a year is not an impossibly large amount of money. It's what big global infrastructure costs. It's just a lot of money. Clearly, like, those companies could not spend one and a half trillion next year. Or if they did, they'd have to borrow it, and they certainly couldn't sustain that level of spending, um, for any length of time.

AI assessment note: “Clearly, like, those companies could not spend one and a half trillion next year.”

Q And, and what have we learned about sort of, uh, you know, say more about what this means for engineers, junior engineers, senior engineers, sort of the, the jobs discussion, how, how teams are organized, uh, et cetera. What have we learned so far?

A I don't think we've learned anything. I mean, you know, this, this didn't, this didn't, this didn't work six months ago. And everyone is scrambling around trying to work out what it means. And, you know, you can get very, very into the noise and the detail and what did somebody say at a party yesterday? Say, oh my God, that's how it's all going to work. Um, no, it's going to take a couple of years for this all to settle down, you know, if nothing else because of the pricing, you know, you've got this enormous crunch between the demand and supply and hence the pricing. Um, so we don't know what, You know, what a team's going to look like. I think people are asking new questions around, you know, the sort of the obvious one of, you know, do you hire junior people? And if so, what are they doing? And why were you hiring junior people in the past? And were you actually hiring to do the thing that they did? Or were you hiring them to do something else? And so if you automate away a class of stuff that used to get done by people, then what will happen? And that sort of becomes much more real now in software development, because you actually are A bunch of stuff that used to be done by people. So those questions are kind of now rather than theoretical, but I don't think anybody can possibly say they kind of know what the market structure is going to look like or what the career of a s…

AI assessment note: “I don't think we've learned anything.”

Q Going back to your, it's a good segue to your point you made earlier of like, hey, you know, we know, you know, Apple's, Apple won. Next question. As a segue, what are some of the next questions that you're most focused on or that, you know, we should be paying most attention to?

A So, I think one way to, um, so there's some of the questions that we've already talked about, like, well, how far does the stack of the models go? Can the models differentiate? And so, I think another is obviously is, is at what point are there, do we see more and more classes of use case where The models are good enough, and we don't need the most expensive, fastest, biggest, heaviest model in the cloud, and you can use an older model, you can use an open source model, you can have a model running on device. Obviously, this is what Apple's going to be talking about in a couple of weeks. Um, you know, how much can you push onto the device where the compute is free, or free to you? Anyway, it doesn't have marginal cost for the developer. Another classic question is, it's almost like the questions move out of technology. So, if you're looking at a law firm, or A consultancy, um, or an investment bank, or basically anyone in professional services, where you traditionally have this pyramid structure, and you can automate a great chunk of what the people at the bottom of the pyramid were doing, what happens? And the only thing you can say there is, if you have never worked at a law firm, or never worked at Bain, BCG, McKinsey, probably not going to have a good idea of how this works. Because you probably don't really know what it is that all those associates are doing, and you also …

AI assessment note: “how far does the stack of the models go? Can the models differentiate?”

Q Yeah. And just on, on the, on the coding, how could, could we have figured, could we have foreseen that that would have been the, the, the use case that really would have taken off or what sort of reflection on that?

A Well, um, every, deterministically, you could have said, well, look, who's messing about with this stuff? Software developers. What are software developers going to try and make work? Software development. Um, so, you know, at a very kind of simplistic, naive level, well, yeah, the stuff that should work is software development. First is software development, just as like, kind of, I often compare this moment to like the internet in like, 97, 98, but it's also like the PCs in the early eighties or the late seventies. It's incredibly exciting, but it's not quite quite clear what it's for, and it doesn't quite work yet, and clearly the first thing that people did with PCs was make computers, um, and the first thing that people are doing with LLMs, and in a sense LLMs are computers, is to make more compute, um, and so that's not terribly surprising. I think this shift has been at the beginning of this year, clearly, that our agentic coding went from being kind of useful to really changing everything. And I'm not sure you could have, you, clearly there were people who would say, well, this is going to be able to do absolutely anything. And so they will say, well, yes, look, I told you. Um, but I don't think anyone kind of, kind of, could have determinously predicted exactly when that was going to happen and that it was going to be coding at Woodwork first.

AI assessment note: “deterministically, you could have said, well, look, who's messing about with this stuff?”

Q Going back to your, it's a good segue to your point you made earlier of like, hey, you know, we know, you know, Apple's, Apple won. Next question. As a segue, what are some of the next questions that you're most focused on or that, you know, we should be paying most attention to?

A So, I think one way to, um, so there's some of the questions that we've already talked about, like, well, how far does the stack of the models go? Can the models differentiate? And so, I think another is obviously is, is at what point are there, do we see more and more classes of use case where The models are good enough, and we don't need the most expensive, fastest, biggest, heaviest model in the cloud, and you can use an older model, you can use an open source model, you can have a model running on device. Obviously, this is what Apple's going to be talking about in a couple of weeks. Um, you know, how much can you push onto the device where the compute is free, or free to you? Anyway, it doesn't have marginal cost for the developer. Another classic question is, it's almost like the questions move out of technology. So, if you're looking at a law firm, or A consultancy, um, or an investment bank, or basically anyone in professional services, where you traditionally have this pyramid structure, and you can automate a great chunk of what the people at the bottom of the pyramid were doing, what happens? And the only thing you can say there is, if you have never worked at a law firm, or never worked at Bain, BCG, McKinsey, probably not going to have a good idea of how this works. Because you probably don't really know what it is that all those associates are doing, and you also …

AI assessment note: “Another classic question is, it's almost like the questions move out of technology.”

Q net new behaviors? We're, we're starting to see some in terms of, you know, people in engaging and talking with, you know, chatbots instead of humans or, or, um, or in addition, um, and then there's a question of, Hey, are these done by the, Um, model providers that currently exist, or are these done by, you know, net new companies, both on, you know, sort of enterprise and consumer?

A Well, this is always the question is how far up the stack does the new thing go? Um, and, you know, I was, I was talking about this with another former, former A-sixteen person who pointed out that like in the, the, the mid nineties, um, people kind of argued that, well, you know, the operating system does all of it. And the app, Windows apps are basically just kind of thin Win-thirty-two wrappers. And you know, Office is basically just, you know, a thin Win-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X Again, it's like frameworks are useful, but that's not made, maybe not a useful way of thinking about what's going on. And the same thing now, like, how much does this need single dedicated understanding of how that market it works or what that market is and what you would do with that? Um, I mean, I remember when we were at A-sixteen Z, there was an investment in a company called Everlaw, which is cloud, um, cloud, legal discovery in the cloud. And so machine learning happens, and so now they can do translation. Are they worried that lawyers are going to say, well, we don't need you guys anymore. We're just going to go out and get a translate app and a sentiment analysis app from AWS. Like, no, that's not how law firms work. Law firms want to buy a thing that solves, they want to buy legal discovery, software management. You know, they don't want to …

AI assessment note: “People buy solutions. They don't buy technologies.”

Q Yeah. The talk about, uh, open AI, uh, talk about what's most, uh, surprised you or w w how have you kind of made sense of their sort of strategy development and, and the questions that they have going forward?

A Well, you know, it's always been such a, such a, a tranquil, drama-free environment, so, you know, it's, you know, obviously they've had the, the issue with, with, with Fiji Sumayu having to take a medical leave, um, which kind of shuffled things up a bit. Look, clearly the, uh, second half, last quarter of last year, the, the question was, right, well, the models are the models, but what else, and how do we get people to do other stuff with this? You know, ask ChatGPT for 15 ideas for what we could do to build value on top of infrastructure, and then we'll do all of them. It's almost literally what it looked like. And then, um, um, Anthropic, with having less capital raised, said, no, we're going to focus on coding, and they got coding working. Um, whether that was, like, a deliberate strategy or, kind of, they stumbled into it is, you know, for other people to say, but, like, clearly that worked. But the question kind of still remains. It's like, the stuff that's working right now Is software development and some things in some other fields. And then there's a lot of people who are kind of excited about using this around the edges and using this for some things. And there's clearly this kind of very widespread between people in the valley who bought, you know, a cluster of Mac studios and are running OpenClaw all day versus, um, you know, those other 40% of people who say, ye…

AI assessment note: “ask ChatGPT for 15 ideas for what we could do to build value”

Q like Nvidia and, and going, you know, up, it, it seems like, uh, they have better margins and are occurring a lot of the value, but it's unclear if that will, You know, remain the same, or if, um, there will be sort of applications, uh, you know, if it will look more like the internet. How, how would you even begin to predict, you know, the answer to such?

A Well, so, so two answers to this. I mean, there's all these sorts of quotes about how history works. And, you know, my favorite one is, history teaches us nothing except that something will happen. And, you know, you can always expose factors, say, well, of course it worked out like that. But it was generally wasn't obvious at the time. And, you know, in particular, I remember, you know, by sort of 15 years ago, a lot of really, really clever people in tech looked at the iPhone and Android and said, you know, this is open versus closed again. And, and we're just going to crush the iPhone, which of course isn't what happened. And then I can go and explain why. But, you know, all of these, all of these comparisons are useful. None of them are predictive. Um, and you know, it's always obvious in hindsight, I, you know, it's funny I've,, I, I, I've done a couple of podcasts recently, and I've I published this presentation, and there's like a, there's like a, a class of comment on this stuff, which is to say, you know, Benedict, you're not doing your job, you're supposed to tell us what's going to happen, you're supposed to make predictions, and all you seem to do is say, well, we don't know. And there's kind of two problems with that. One of them is, there's a class podcast of places where I actually do say, like, I don't think this is going to work, I think it's going to work like…

AI assessment note: “all of these comparisons are useful. None of them are predictive.”

Q Do you think there's going to be a reckoning around token maxing? Uh, is it possible that companies have been overshooting AI usage and when they do proper ROI studies, they'll pull back?

A Well, obviously, you know, you've had people like Using the most expensive model to dick around on the internet. Um, which is kind of what happened with mobile, you know, in 2010. Like, you know, you, you got a 10,000 dollar bill and you said, wait, wait, I thought this was a flat rate bundle. Like what happened? Um, so there's like, you've got, you've got, obviously you've got a bunch of, of like silly slash meaningful stories. Um, I think there's also a point of like, I think maybe what's slightly more interesting as a question is, um, And clearly there's going to be a point in which, as I've said several times, we're at a moment of kind of massive disequilibrium, and the pricing has got to get back into alignment with the cost, and the usage has got to get it into alignment with the pricing and the ROI. The challenge is it's a bit tricky. This early stage is quite hard to know what the ROI is. It's, it's rather like giving everybody the internet in the late nineties and saying, okay, go off, be more productive. And if you look at, like, there's a survey from Deloitte, there's also a survey from the Fed that's in my presentation, Where if you go and ask CFOs, where have you seen the benefits? And most of the benefits so far have been stuff that's pretty hard to measure. So like better analytics, better customer support, um, more productivity. You could make more slides more q…

AI assessment note: “the usage has got to get it into alignment with the pricing and the ROI”

Q Yeah. The talk about, uh, open AI, uh, talk about what's most, uh, surprised you or w w how have you kind of made sense of their sort of strategy development and, and the questions that they have going forward?

A Well, you know, it's always been such a, such a, a tranquil, drama-free environment, so, you know, it's, you know, obviously they've had the, the issue with, with, with Fiji Sumayu having to take a medical leave, um, which kind of shuffled things up a bit. Look, clearly the, uh, second half, last quarter of last year, the, the question was, right, well, the models are the models, but what else, and how do we get people to do other stuff with this? You know, ask ChatGPT for 15 ideas for what we could do to build value on top of infrastructure, and then we'll do all of them. It's almost literally what it looked like. And then, um, um, Anthropic, with having less capital raised, said, no, we're going to focus on coding, and they got coding working. Um, whether that was, like, a deliberate strategy or, kind of, they stumbled into it is, you know, for other people to say, but, like, clearly that worked. But the question kind of still remains. It's like, the stuff that's working right now Is software development and some things in some other fields. And then there's a lot of people who are kind of excited about using this around the edges and using this for some things. And there's clearly this kind of very widespread between people in the valley who bought, you know, a cluster of Mac studios and are running OpenClaw all day versus, um, you know, those other 40% of people who say, ye…

AI assessment note: “the question was, right, well, the models are the models, but what else”

Q Now, can you share how, uh, how did this compare with mobile, um, or other sort of platform shit in terms of, you know, user, early user adoption on, on sort of the, you know, weekly or daily user?

A So, I think there's, there's, there's, there's a bunch of different ways to answer this. One of them is, like, we're always standing on the shoulders of giants, and the grocery's always compounding. So mobile didn't need to wait for, um, the internet or cellular networks. Like, mobile data, mobile internet didn't need to wait for cell, we kind of needed to wait for cellular data, but we didn't need to wait for, like, the internet to happen. And the internet didn't need to wait for PCs, and PCs didn't need to wait for consumer electronics and semiconductors and so on. So you've always got this accelerating adoption, and you know, when, when, when your boss, my old boss Marc Andreessen was working on Netscape, there were like double digit millions of PCs on the entire planet. So like, no, you couldn't have nine hundred million weekly active users because there weren't nine hundred million PCs. So there's always that acceleration. So that's one point. I think the second point is like at the early stage of any of these shifts, it's not really clear how it's going to work and nothing works. You know, like, I'm just about old enough to remember this. I'm not sure how, how, how old you are, but like, you know, anyone in their thirties doesn't really remember a time when it was completely normal that you'd be working, and then everything on the screen would just freeze, and you'd just …

AI assessment note: “you couldn't have nine hundred million weekly active users because there weren't nine hundred million PCs.”

Q like Nvidia and, and going, you know, up, it, it seems like, uh, they have better margins and are occurring a lot of the value, but it's unclear if that will, You know, remain the same, or if, um, there will be sort of applications, uh, you know, if it will look more like the internet. How, how would you even begin to predict, you know, the answer to such?

A Well, so, so two answers to this. I mean, there's all these sorts of quotes about how history works. And, you know, my favorite one is, history teaches us nothing except that something will happen. And, you know, you can always expose factors, say, well, of course it worked out like that. But it was generally wasn't obvious at the time. And, you know, in particular, I remember, you know, by sort of 15 years ago, a lot of really, really clever people in tech looked at the iPhone and Android and said, you know, this is open versus closed again. And, and we're just going to crush the iPhone, which of course isn't what happened. And then I can go and explain why. But, you know, all of these, all of these comparisons are useful. None of them are predictive. Um, and you know, it's always obvious in hindsight, I, you know, it's funny I've,, I, I, I've done a couple of podcasts recently, and I've I published this presentation, and there's like a, there's like a, a class of comment on this stuff, which is to say, you know, Benedict, you're not doing your job, you're supposed to tell us what's going to happen, you're supposed to make predictions, and all you seem to do is say, well, we don't know. And there's kind of two problems with that. One of them is, there's a class podcast of places where I actually do say, like, I don't think this is going to work, I think it's going to work like…

AI assessment note: “all of these comparisons are useful. None of them are predictive.”

Q net new behaviors? We're, we're starting to see some in terms of, you know, people in engaging and talking with, you know, chatbots instead of humans or, or, um, or in addition, um, and then there's a question of, Hey, are these done by the, Um, model providers that currently exist, or are these done by, you know, net new companies, both on, you know, sort of enterprise and consumer?

A Well, this is always the question is how far up the stack does the new thing go? Um, and, you know, I was, I was talking about this with another former, former A-sixteen person who pointed out that like in the, the, the mid nineties, um, people kind of argued that, well, you know, the operating system does all of it. And the app, Windows apps are basically just kind of thin Win-thirty-two wrappers. And you know, Office is basically just, you know, a thin Win-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X-X Again, it's like frameworks are useful, but that's not made, maybe not a useful way of thinking about what's going on. And the same thing now, like, how much does this need single dedicated understanding of how that market it works or what that market is and what you would do with that? Um, I mean, I remember when we were at A-sixteen Z, there was an investment in a company called Everlaw, which is cloud, um, cloud, legal discovery in the cloud. And so machine learning happens, and so now they can do translation. Are they worried that lawyers are going to say, well, we don't need you guys anymore. We're just going to go out and get a translate app and a sentiment analysis app from AWS. Like, no, that's not how law firms work. Law firms want to buy a thing that solves, they want to buy legal discovery, software management. You know, they don't want to …

AI assessment note: “People buy solutions. They don't buy technologies.”

Q new and, and, and different questions or what, what is sort of your, you know, if I woke up on a, in a coma, uh, after reading your, you know, your original presentation, let's say, you know, the one after GPT three launch, uh, came out, um, and then seeing this one now. What were the sort of most surprising things or things that we learned that updated those questions?

A So I think we have a lot of new questions this year. So I feel like, you know, you could make a list of, as it might be, half a dozen questions in spring of 23. Like, open source, China, Nvidia, does scaling continue? Um, what happens to images? Um, does, how long does OpenAI's lead remain? And those questions didn't really change in 23 and 24, and most of those questions are kind of still there. Like, the Nvidia question hasn't really changed, you know, the answer on China. The answer on, you know, well, how many models will there be? The answer is, okay, there's going to be anybody who can spend a couple of hundred, you know, can spend a couple of billion dollars can have a frontier model. That was pretty obvious, you know, in early 23. It took a while for everyone to understand that. And big models and small models, will we have small models running on devices? No, because the small models, the capabilities are moving too fast for the small models to shrink the small model onto the device. But those questions kind of didn't change for two, two and a half years. I think we now have, I think, A bunch of more product strategy questions as you see real consumer adoption and open AI and Google building stuff in different directions, Amazon going in different directions, Apple trying and obviously failing and then then trying again to do stuff. There's some sense of like, there is…

AI assessment note: “I think we now have, I think, A bunch of more product strategy questions”

Q that software is someone sat down and designed a workflow and said, this is the right way of doing this from now on. But you also said that a process grows out of the way just a business runs. Does that just take time? Or do you think we need more experimentation, iteration from these vertical AI startups to get Is this the right shape of software for the future?

A Well, in a sense, I mean, maybe kind of a, an interesting turn on this is this is both what, what strategy consultants do and software companies do is they kind of look at what's going on inside a company and say, well, this is a crap way of doing it. This would be a better way of doing it. It would achieve your objectives better. And a software company kind of encodes that in software and, you know, a strategy consultant to see kind of encodes that in, you know, workflows and org charts and processes and training and, you know, objectives and, you know, maybe tells them to buy some software to do that thing. Or maybe now increasingly maybe builds them that software as well. Um, I think, um, another thing to talk about here is how much of what's done inside an organization is implicit, and not documented, and not in the training data, and not something that anybody in that company could actually kind of sit down and draw you a new flowchart of and explain to you. Um, that's what, that's a big chunk of the value of Bain, BCG, McKinsey. Is that they have license to come into a company and talk to everyone and talk to the people they're not allowed to talk to that are in a different org and not get fired. And to go and work out how this actually works as opposed to how it's supposed to work and why it is that people aren't doing the strategy because actually guess what their bonus…

AI assessment note: “how much of what's done inside an organization is implicit, and not documented”

Q I'm not yet at the conviction that it's going to be any bigger. I'm curious what, what sort of inspires that sort of, uh, you know, sort of state statement, and then also what might change your mind either way? You know, that it might not be as big as the internet, because of course the internet was obviously very big, but also that, hey, perhaps it might be bigger.

A Well, so I think, you know, I don't want to, I made a diagram of kind of S-curves kind of going up the slide and someone said, well, what's the axis on this diagram? I, you know, I don't want to kind of get into, you know, is this, is this five percent bigger than, than internet or is it 20% bigger? I think the question is more like, is it another of these industry cycles or is it a much more fundamental change in, in what technology can be? Is it more like computing or electricity as a sort of structural change rather than Here's a whole bunch more stuff we can do with computers. I think that's sort of the, the, the, the question. And there's a funny sort of disconnect, I think, in, in looking at debates about this within tech, because, you know, I watched this, this, this, this, one of the, um, OpenAI live streams a couple of weeks ago, and they spend the first 20 minutes talking about how they're going to have, like, human-level, PhD-level AI researchers, like, next year. And then the second half of the stream is, oh, and here's our API stack that's going to enable 100,000 of new software developers just like Windows. And in fact, literally quote Bill Gates. And you think, well, those can't kind of both be true. Like either I've got a thing, which is a PhD level AI researcher, which by implication is like a PhD level CPA. Or I've got a new piece of software that does much ac…

AI assessment note: “I don't want to kind of get into... I think the question is more like”

Q I'm not yet at the conviction that it's going to be any bigger. I'm curious what, what sort of inspires that sort of, uh, you know, sort of state statement, and then also what might change your mind either way? You know, that it might not be as big as the internet, because of course the internet was obviously very big, but also that, hey, perhaps it might be bigger.

A Well, so I think, you know, I don't want to, I made a diagram of kind of S-curves kind of going up the slide and someone said, well, what's the axis on this diagram? I, you know, I don't want to kind of get into, you know, is this, is this five percent bigger than, than internet or is it 20% bigger? I think the question is more like, is it another of these industry cycles or is it a much more fundamental change in, in what technology can be? Is it more like computing or electricity as a sort of structural change rather than Here's a whole bunch more stuff we can do with computers. I think that's sort of the, the, the, the question. And there's a funny sort of disconnect, I think, in, in looking at debates about this within tech, because, you know, I watched this, this, this, this, one of the, um, OpenAI live streams a couple of weeks ago, and they spend the first 20 minutes talking about how they're going to have, like, human-level, PhD-level AI researchers, like, next year. And then the second half of the stream is, oh, and here's our API stack that's going to enable 100,000 of new software developers just like Windows. And in fact, literally quote Bill Gates. And you think, well, those can't kind of both be true. Like either I've got a thing, which is a PhD level AI researcher, which by implication is like a PhD level CPA. Or I've got a new piece of software that does much ac…

AI assessment note: “Is it more like computing or electricity as a sort of structural change”

Q that software is someone sat down and designed a workflow and said, this is the right way of doing this from now on. But you also said that a process grows out of the way just a business runs. Does that just take time? Or do you think we need more experimentation, iteration from these vertical AI startups to get Is this the right shape of software for the future?

A Well, in a sense, I mean, maybe kind of a, an interesting turn on this is this is both what, what strategy consultants do and software companies do is they kind of look at what's going on inside a company and say, well, this is a crap way of doing it. This would be a better way of doing it. It would achieve your objectives better. And a software company kind of encodes that in software and, you know, a strategy consultant to see kind of encodes that in, you know, workflows and org charts and processes and training and, you know, objectives and, you know, maybe tells them to buy some software to do that thing. Or maybe now increasingly maybe builds them that software as well. Um, I think, um, another thing to talk about here is how much of what's done inside an organization is implicit, and not documented, and not in the training data, and not something that anybody in that company could actually kind of sit down and draw you a new flowchart of and explain to you. Um, that's what, that's a big chunk of the value of Bain, BCG, McKinsey. Is that they have license to come into a company and talk to everyone and talk to the people they're not allowed to talk to that are in a different org and not get fired. And to go and work out how this actually works as opposed to how it's supposed to work and why it is that people aren't doing the strategy because actually guess what their bonus…

AI assessment note: “an interesting turn on this is this is both what, what strategy consultants do”

Q your thesis is this idea that models are gonna end up as commodities, and yet the, you know, the layer that's raising the most money, uh, you know, in the, in the fastest time in history is these foundation model companies. Um, so given that, what advice might you have for them, either, either collectively or we can pick on someone individually, um, in order to, in order to adapt?

A It's not that I know that they're gonna be Become commodities. My position is more, want more now. Well, like, hey, here is a, here is a chain of argument that says that deterministically it looks like these things will be commodities and explain to me why they won't be. Um, and that's as far as I would commit to that. Um, I think the, you know, the, the raising all this money, I kind of go back to my point about mobile, which again has no predictive value, but it's a worthwhile observation is that the mobile industry is very big and spends a lot of money on infrastructure and isn't very profitable and all the cool stuff is done by somebody else. And then you can do, you know, well, what's the return on capital? And the answer is, well, it depends which mark, whether you're in America or Europe or India or China. Um, But meanwhile, like, that was a worthwhile thing to do, and it produced a return for somebody, but then it didn't, ended up not controlling the whole thing, and other people ended up getting more value from that, um, than they did. Um, you know, I don't have the number in my head. What was Google's net income last year was, what, fifty billion dollars or something? What's net income for, you know, the total telecoms industry? I should really subscribe to Bloomberg, then I could just answer these questions instantly. Um, but, like, you have a pretty safe bet that Go…

AI assessment note: “It's not that I know that they're gonna be Become commodities.”

Q Does this imply a less consolidated SaaS environment than, than before AI? Maybe less bundling or single behemoths like the Microsoft Enterprise?

A Gosh, way to bring me back down to earth. This is a SaaS industry going to be less consolidated than it. That's all great, but tell us about the stocks. Um, what are the kind of building blocks that we can put down here? So obviously it's going to be way cheaper and quicker to build software. Obviously there's going to be a bunch of stuff you could do with software that you just couldn't do before at all. Um, and so there will be more competition. There will, and of course this comes with a new margin structure, but as per our compensation early, we don't really know what that margin structure is going to look like. Um, are you going to go to, you know, outcome-based pricing? It's really hard to, like, tie each button press in a piece of enterprise software to P&L. Like, sometimes you can buy in Salesforce or something. It was an awful lot of software. It would be really hard to say, well, you know, the, the, the work I did today did this to the EPS. Therefore, this is what we should pay for it. Um, this is what we should pay for that piece of software. I don't think that makes sense. I mean, long time. Anyway, what does the pricing structure look like over time versus now? Um, there will be more competition. It will be easier to build stuff and quicker to build stuff. The way that I sort of thought about, I suppose this is, there's maybe kind of two framings to think about thi…

AI assessment note: “it's going to be way cheaper and quicker to build software... there will be more competition.”

Q Does this imply a less consolidated SaaS environment than, than before AI? Maybe less bundling or single behemoths like the Microsoft Enterprise?

A Gosh, way to bring me back down to earth. This is a SaaS industry going to be less consolidated than it. That's all great, but tell us about the stocks. Um, what are the kind of building blocks that we can put down here? So obviously it's going to be way cheaper and quicker to build software. Obviously there's going to be a bunch of stuff you could do with software that you just couldn't do before at all. Um, and so there will be more competition. There will, and of course this comes with a new margin structure, but as per our compensation early, we don't really know what that margin structure is going to look like. Um, are you going to go to, you know, outcome-based pricing? It's really hard to, like, tie each button press in a piece of enterprise software to P&L. Like, sometimes you can buy in Salesforce or something. It was an awful lot of software. It would be really hard to say, well, you know, the, the, the work I did today did this to the EPS. Therefore, this is what we should pay for it. Um, this is what we should pay for that piece of software. I don't think that makes sense. I mean, long time. Anyway, what does the pricing structure look like over time versus now? Um, there will be more competition. It will be easier to build stuff and quicker to build stuff. The way that I sort of thought about, I suppose this is, there's maybe kind of two framings to think about thi…

AI assessment note: “and so there will be more competition. There will, and of course this comes”

Q Do you think there's some kind of co-evolution between AI native software and new types of interfaces? For example, new customer service AI platforms that might not have had as much human facing UI or system of record software being built without a front end at all because its primary user will be AI agents querying it directly?

A So, I think these are kind of interesting ideas. They're things I struggle to have a strong opinion on, because, you know, they're not kind of, not deep, deep into the weeds of, of how enterprise infrastructure gets bought. I, I wonder how new some of these questions are. Um, I mean, remember, um, Chris Dixon saying, like, 1015 years ago that, you know, APIs is a new BD, and software wouldn't need, software companies could just, you know, open up your APIs, and like, well, like, what's old is new. Um, you know, you don't need an API anymore, you just have an MCP server, and like, people will just plug in the agent, we'll just plug into that. Um, I don't know, I think the challenge with a lot of this stuff is that all the, the decisions are really exception handling. Like, the question is always what can you not automate? What requires someone to make a decision, um, and some judgment, and have an opinion about it, because maybe that hasn't been written down, or that didn't happen before, or it doesn't look quite the way it happened before. Um, I think there's a sort of, you know, there are various ways of kind of think about separating out what gets automated and what doesn't.

AI assessment note: “They're things I struggle to have a strong opinion on”

Q If you had to guess, what are the use cases outside of coding that could potentially yield, you know, daily activity?

A So, I should say the sort of presentation that I published a couple of weeks ago, there were sort of three sections, and one of them is talking about capital and capex and infrastructure and foundation models and differentiation, which is the stuff we talked about. And the second is, well, how would you build software with this, and what does this do for the software industry, and what would software look like if you, and what, what is the margins, what happens to the margins of the companies and everything else. And the third section I called change, which is kind of getting to this point, um, and I opened it with, again, what it appears to upset Certain category of person where I said, you know, the Yogi Berra quote that, you know, predictions are hard, especially about the future. Um, and I think there's a sort of a back test point, which is imagine asking these kind of questions about the internet in 1997. What would you have got? What would you have not got? But I think one way you can look at this is to say, well, this is automation. That this makes a class of thing that people used to do. That couldn't be automated, now you can automate that. And so then, well, what does that mean? And I proposed, like, three or four ways of, sort of, buttons to press. First one is, is this just price elasticity? Which is really what the Jevons paradox is. Like, if you make it cheaper to…

AI assessment note: “predictions are hard, especially about the future.”

Q If you had to guess, what are the use cases outside of coding that could potentially yield, you know, daily activity?

A So, I should say the sort of presentation that I published a couple of weeks ago, there were sort of three sections, and one of them is talking about capital and capex and infrastructure and foundation models and differentiation, which is the stuff we talked about. And the second is, well, how would you build software with this, and what does this do for the software industry, and what would software look like if you, and what, what is the margins, what happens to the margins of the companies and everything else. And the third section I called change, which is kind of getting to this point, um, and I opened it with, again, what it appears to upset Certain category of person where I said, you know, the Yogi Berra quote that, you know, predictions are hard, especially about the future. Um, and I think there's a sort of a back test point, which is imagine asking these kind of questions about the internet in 1997. What would you have got? What would you have not got? But I think one way you can look at this is to say, well, this is automation. That this makes a class of thing that people used to do. That couldn't be automated, now you can automate that. And so then, well, what does that mean? And I proposed, like, three or four ways of, sort of, buttons to press. First one is, is this just price elasticity? Which is really what the Jevons paradox is. Like, if you make it cheaper to…

AI assessment note: “predictions are hard, especially about the future. Um, and I think there's a sort of”