The Exchanges

Every argument clarity score on this site is built from rows on this page. Each question and answer was assessed with names hidden, the host's own answers included, on four things from 1 to 5: directness (does it answer the question asked), coherence (do the ideas follow), precision (concrete details and clear references), compression (says a lot per word). The weighted mix (30/30/25/15) is the exchange score. A person's published score averages their exchange scores on raw tape only, at least 8 of them, shrunk toward the cohort mean. Full method →

Benedict Evans no published score: only 5 usable exchanges on raw tape, and a fair score needs 8+ · coarse estimate ≈4.0/5 from 11 raw and produced exchanges record → ← everyone

Every exchange below was scored with names hidden, four dimensions each from 1 to 5. An exchange's score is 0.30·directness + 0.30·coherence + 0.25·precision + 0.15·compression. The published score averages the raw tape exchange scores and shrinks small samples toward the cohort mean, so five great answers can't beat twenty good ones. Produced feed rows count only toward coarse estimates, never toward a full score.

clear all ✕
11exchanges match
5on raw tape
3redirected or not addressed
Answered produced feed D 5 · C 5 · P 4 · Cm 4 4.60

Q Great. Going back to big tech, perhaps with Oracle that you just Mentioned. Isn't that kind of both a gamble, but also kind of rationale for a company like Oracle to be leveraging an old business to turn it into something new?

A Well, you know, everyone in, in, generally in a bubble, everybody's a rational actor. Almost everyone's a rational actor given their situation. If you're Sam Altman, you've got a commodity technology. You've got, you're competing with people who have giant legacy cash flows. You don't have your own infrastructure. Don't really have any differentiation, but you've got massive mindshare. So what do you do? Well, you try and swap that for hard assets, and you try and talk your way into a self-fulfilling prophecy. You try and swap that for hard assets, and you try and turn those lightly engaged nine hundred million weekly active users into something, something more tangible. If you are Larry Ellison, like, you've got this very cash generative legacy business that's been in structural decline for 25 years. Most people going through YC have never heard of Oracle, like, almost literally. Um, no one has invited you to a party since like, 1998. Um, so what do you do when here is this thing? You grab onto it with both hands and you burn your way through. The same with NVIDIA. I mean, I haven't looked at, I haven't updated my number here, but like, I think Q three last year, I think NVIDIA had something over seventy billion dollars of trading 12 months free cash flow. So they can't give the money to TSMC fast enough. TSMC won't take it fast enough, and TSMC, as it says, like, dude, this i…

AI assessment note: “you've got this very cash generative legacy business... You grab onto it with both hands”

Answered produced feed D 5 · C 4 · P 4 · Cm 4 4.30

Q At some point there was a hope That memory would be adding some level of defensibility and moat to all those, uh, chatbot interfaces, including very much chat GPT. That doesn't seem to have happened. Why is that?

A Well, so two or three things. Firstly, I think very obviously it only works if you're using it a lot already. And so if you haven't worked out a bunch of stuff to do with this, and most people have not, then you don't see the memory. OpenAI did this, this data, the end marketing at the end of last year, where they gave everyone a cute little graphic of how many messages they'd done and told everyone what What decile they were in. And so I went on Reddit and grabbed like 300 screenshots of people posting these, and it turns out that if you did a thousand posts, if you did a thousand prompts last year, you're in the top 20%. So basically 80% of people hit return less than a thousand times last year, so average of less than three times a year. Obviously charts growing up in the year, so you can't really average it across the year, but like if you typed return a thousand times in the whole year, you're not really using this every day. And so there's no memory there. Me, then the sort of a subsidiary point would be, okay, is that a network effect or is that stickiness? It's probably more stickiness. And how sticky, and what happens if you just ask ChatGPT to tell you everything it knows about you, and then paste that into Claude, or vice versa. So it was kind of unclear whether that was a thing. I mean, the whole thing does feel very sort of, in a lot of ways, in that it's clear thi…

AI assessment note: “it only works if you're using it a lot already”

Answered produced feed D 5 · C 4 · P 4 · Cm 4 4.30

Q But do you think that, similarly to a lot of what happened with the internet, we need to go through a phase of destruction? First, before we figure that out?

A Well, will a lot of these companies go bust? Yes. You know, there's, I think I may have mentioned last time, there's this classic book on the history of bubbles and the title is this time it's different, which has a double meaning because a in a bubble people say it's no, it's different. It's not a bubble and they're wrong, but also there, when they say it's different, they're kind of right. Like the dot-com bubble was different from every previous bubble and what's going on now clearly is different from the dot-com bubble. Like we don't have loads of IPOs of consumer companies with no profits. Like we don't have any IPOs. Um, it's not being driven by retail. Speculation in private, in public market stocks is not being funded by venture capital apart from anything else. Um, but it can still be a bubble. Um, just, you know, it's an uninteresting observation. I think the, the, you know, are, do we have a lot of overinvestment? Yes, of course. Well, a bunch of this end up not producing a return. Like, yes, of course, that's like how this stuff works. Where were the smoking holes in the ground? A bit more difficult to tell. I mean, you know, there are people who, you know, obviously you can kind of, you know, Look at the new clouds or look at Oracle and say, you know, leverage doesn't tend to end well, but you know, Hey, I haven't been an equity analyst for 20 years. I don't know. …

AI assessment note: “Well, will a lot of these companies go bust? Yes.”

Answered produced feed D 4 · C 4 · P 4 · Cm 3 3.85

Q Do you think there could be a related concept of, uh, ephemeral software?

A Well, this is, this is going to be the other half of it, is now you've got, you expand this category of, like, improvised software, and so where something, it might have been that you do it, get the CSV and do it in Excel, and, or, Maybe a Perl script if you're that kind of person. Now maybe you'll ask ChatTPT to do it for you. Analyze this for me. Do this for me. And often it's like stuff that you couldn't have done in Excel before. You know, it's like, look at all those PDFs, look at all those PowerPoints, and then look at this PowerPoint. Is there something in all of those PowerPoints that we haven't said in this one? Which is the kind of thing that's got these sort of fuzzy answer that it's kind of right. That works very well with an LLM, except it wouldn't work if there were PDFs, because it can't read PDFs, so you have to tell people it can't read PDFs. So that sense of, like, this sort of middle ground, and maybe my taxonomy is wrong, but I think that's more interesting as a way of looking than just saying, oh, everyone will just vibe code their own, their own thing. No, no one will vibe code their own ERP or their own frame.io, but they may ask Anthropic or Gemini or, or, or ChatGPT, can you do this thing for me? In a way that's sort of analogous to they might have done it in Excel.

AI assessment note: “this is going to be the other half of it, is now you've got”

Answered produced feed D 4 · C 4 · P 4 · Cm 3 3.85

Q Yeah, and you talk to a lot of big corporations as part of your consulting and your public speaking. What's your sense of the overall sentiment? Are people more bewildered than they used to be? Is some of this starting to sink in? What do you see them do?

A So I certainly wouldn't say bewildered. I mean, at least not any more bewildered than anyone in tech. And if I think, you know, what's the end center line, you know, if you're in control, you're not going fast enough. Like if you understand any of this, you're not paying attention. I was like, I came across the other day that the physics, famous physics professor says, you know, I'm going to teach you an eight week class on quantum today. I don't understand quantum at the end of this class. You won't understand quantum either. Um, It certainly applies to ad tech, but it's like the Schleswig-Holstein question. You know the joke about that? Oh, the Schleswig-Holstein question is this famously complicated anti-century politics question, and I think Palmerston said only three people have ever understood that the German professor has gone mad, somebody at Bismarck who is dead, and myself who has forgotten. Anyway, you can cut that out. The point is that no one really knows the answers to does it go to AI and AGI, and what will it do next year? But everyone's got a bunch of stuff deployed. Everyone deployed co-pilot and went, oh, okay, that wasn't very successful. Everyone is kind of like giving everybody the internet in 1997 and saying, there you are, be more productive. Doesn't, didn't really work. Everyone now has a bunch of pilots. Some of those pilots have made their way into pr…

AI assessment note: “I certainly wouldn't say bewildered.”

Answered raw tape D 4 · C 4 · P 4 · Cm 3 3.85

Q If generative AI is going to be around us, um, everywhere, obviously a key question is bias. And you've been thinking about bias in AI for, for a long time. What, what's your, 20, 24 view of it?

A So, it's interesting. I read about this in, in my newsletter this week or last week, this week, I think, because this was this, Bloomberg did a story where they liked it. They looked at people using ChatGPT to screen resumes. And it finds bias. Um, and there was a thing in 2018 where Amazon had an internal project to use machine learning to filter resumes, CVs, because obviously they're hiring a huge scale, and what they find is that historically they mostly hired men, and so the pattern of a successful candidate is a man. And meanwhile, it's not that it was looking at gender equals male in a database. It was looking at like what sports people played and even more subtle things like, you know, what language people would be using to describe their accomplishments. And it doesn't have a model of male and female. It's just has a model of all of the people we hired played football and none of them paid played lacrosse. Sorry, I'm stereotyping in Britain. Lacrosse is a girls sport in America. Maybe it's a boys sport. I don't know.

AI assessment note: “Bloomberg did a story where they liked it. They looked at people using ChatGPT to screen resumes. And it finds bias.”

Answered raw tape D 4 · C 4 · P 3 · Cm 3 3.60

Q So a little bit to the, it's hard to predict and the data question. Is there an argument to be made that, um, actually generative AI may be grossly overhyped?

A Well, so this is, it's back to the, like, um, how excited about this should we be? Like every, you know, platform shift, there's a kind of, you know, it's a classic on a hype cycle thing. Um, And then there are the people who say, no, it's not, it's just going to keep growing. We've got this exponential growth, and this one isn't going to do that. It's going to go straight through to AGO and kill us all. Clearly, if we're not in a bubble now, we're going to have a bubble. It's because that's just like the nature of the light, the cycle of life. There will be a bubble around each new technology. Um, there was a bubble around iPhone apps, you know, the bubble around cloud and a bubble around any new thing. There is a bubble of some kind. It's kind of interesting though, because they're kind of, you would know more about this than me, but like clearly venture was kind of venture fund investment has kind of gone down radically since two years ago. Investments have gone down radically. So you've kind of got a crash in a bubble kind of happening at the same time.

AI assessment note: “Clearly, if we're not in a bubble now, we're going to have a bubble.”

Answered raw tape D 3 · C 4 · P 4 · Cm 3 3.55

Q When you talk to large corporations, um, what do you tell them? What kind of conversations do you have with them in terms of their AI strategy and what it should be?

A So, I had this conversation with the head of a big industrial company last summer, who said, uh, remember when everybody needed a five G strategy? And I remember it was, I wrote something about it. It was kind of at the time. It was kind of hilarious because it was like every big company had read about five G on the plane in the economist or business week or something, and they land and they send everyone an email saying, what's our strategy for this? And the answer for most people was, you don't need one. I mean, the high five G hype in hindsight was very weird because it was just, there was, there was, it was, it was just a faster pipe. It wasn't like a transformative new technology, the way people talked about it. Um, clearly AI or rather let's get specific generative AI, generative machine learning clearly is a, Fundamentally different thing in a way that five G wasn't. And a lot of people need to have, think about it or think about it, what it might mean for their company. Um, after that you go off and you think, okay, so do we pay Bain, BCG, McKinsey? Do we pay WPP or Publicis? Do we pay Cognizant or Infosys or Accenture? Accenture would be really upset that I put them in that category, but you know what I mean? What kind of a question is this? Um, Do we pay Adobe? And again, this comes back to this kind of platform shift conversation. So, you know, imagine you are, pick …

AI assessment note: “imagine you're caterpillar. What's your generative AI strategy? Well, some parts of that will be”

Redirected raw tape D 2 · C 4 · P 4 · Cm 3 3.25

Q You have a very, uh, nuanced and interesting view on AGI. Can you, can you talk to that? How in twenty-twenty-four should we think about that?

A Well, all my opinions are nuanced and interesting. Come on. Um, it's funny. So, my grandfather was a science fiction writer in the sort of twenties, thirties, forties, fifties. Um, and he wrote a story called A Logic Named Joe in, I think, 1946. And premise is everybody has a home computer, which is called a logic. Um, computer at the time is a job title. So everyone has a logic and they're all connected to a global network and they're connected to these global databases in the cloud called tanks, I think. And so you can sit at this thing and do your banking or bought your flights or do online dating or, you know, look up any piece of information and answer any argument. And so it's basically describing the internet. Um, and one of these things has some kind of manufacturing defect, which means it starts Just being helpful and answering any question that anybody asks. And for some reason, because of the way the network works, like, it sees all the questions. I don't think my grandfather would quite sort through the network architecture. So this thing sees any question that's asked anywhere on the network, which doesn't seem very realistic. But anyway, just start answering them. Like, any question. Like, how do I murder my wife? Someone types this in as a joke, and it pauses and says, what colour is her hair? And then it suggests an undetectable poison that only kills blondes. A…

AI assessment note: “my grandfather was a science fiction writer in the sort of twenties, thirties, forties”

Redirected raw tape D 2 · C 4 · P 4 · Cm 2 3.10

Q Precisely that question of, um, use cases seems to be the, the 20, 24 question or 20 22 was, okay, this exists. 2023 was, okay, let's experiment. 2024 seems to be the beginning of use cases. You've, you've talked about bundling versus unbundling to build on, on, on what you just said about, uh, is it going to be one product or multiple products? How do you think about it?

A Well, so the analogy that I was thinking about the other day, and my last answer was basically analogies, but the analogy that kind of occurred to me recently was to think about spreadsheets. So you get PCs emerge in the late seventies and early, really take off in the early eighties. And if you read Dan Bricklin talking about creating Visicalc, you know, he shows, and the young people of today won't know this, but before either of us were born spreadsheets was paper. You can still buy them on, on Amazon. You can buy a pad of spreadsheet paper, pre-printed paper, like, like a full size paper. Um, and so you've got all these accountants whose job is basically making spreadsheets by hand, maybe with a mechanical electronic calculator, but you, you know, you're filling in by hand each cell in the spreadsheet and over and over again. And he shows them PhysiCalc and they like blows their mind because suddenly something that would have taken a week takes like 10 minutes. You know, iterate, recalculate, recalculate, recalculate, because it's not building the model, it's changing the assumptions. That's so, so, so transformative. And so there are all these stories of accountants who are saying, you know, I did a week's work in an afternoon. And at the time, I remember PCs at the time were like five, 10,000 dollars, which is an interesting comparison, incidentally, with the Vision Pro, …

AI assessment note: “the analogy that kind of occurred to me recently was to think about spreadsheets.”

Redirected produced feed D 1 · C 4 · P 4 · Cm 3 2.95

Q Still in the same vein of comparison between the dot-com bubble and the dot-com crash and today. Do you take some comfort in the concept of a piece dividends from all of this that ends up working out for the economy?

A One of the really basic ways this is different from every other platform shift. A lot of, I spent a lot of time saying, well, this is kind of how platform shifts work, and this is what happened the last five times. The one way this is unquestionably different is that with all the other platform shifts, we knew what the physical limits of the science were. We knew how it worked. We knew what could happen, what could happen like next month. So you didn't know what, like, the iPhone three or four would be, but you knew it wouldn't fly, and it wouldn't have a one-year battery life. You didn't really know how the internet would evolve in 1997, but you knew that telcos wouldn't give everybody on the world fibre internet next week. Whereas with LLMs, we kind of don't know the physical limits of how this could evolve. And so it may be that, you know, next week we have a paper that means that you can get more or less the same results for one percent of compute. You know, maybe that's a silly statement, but, and, but it might be 10% of the compute. And we don't know that. Whereas you did know, absolutely no, no one was going to publish a paper that said you could get like the same compute with one percent of the transistors on a chip, but we don't know those physical limits. And so we don't know the parameters of what could and couldn't happen to cause a pricing collapse. What's happened…

AI assessment note: “One of the really basic ways this is different from every other platform shift.”

page 1
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.