May 22, 2025 · 1h 15m · mad
AI Eats the World: Benedict Evans on What Really Matters Now
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Technology analyst Benedict Evans sits down with Matt Turck on The MAD Podcast to evaluate the real-world state of artificial intelligence, separating commercial realities from hype. He provides a pragmatic breakdown of foundational model commoditization, enterprise adoption, Big Tech distribution strategies, historical platform shifts, and the technical limits of probabilistic computing.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 15.7% of the talking time here. How this is scored →
Matt as informed peer 4.8 Guest teaching 3.7 Guest disagreement 2.9 Matt pushing back 2.2
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 1:07:04 Dismissing agent and robot demos as fake hype
Benedict rejects the premise of recent AI agent and humanoid robot demonstrations, calling them fake, scripted, and 'bullshit' comparable to staged autonomy demos.
Hardest push from Matt ▶ 15:14 Host pushes back on Deep Research dismissalMatt refuses Benedict's broad write-off of Deep Research, detailing his own effective qualitative workflow and arguing that users can adapt to the tool's true strengths.
Biggest teaching moment ▶ 12:23 Dissecting errors in OpenAI's mobile market benchmarkBenedict educates the host on how OpenAI failed basic data analysis in their marketing demo by confusing web traffic with device adoption and transcribing the percentages backward.
Matt holds his own ▶ 29:41 Host challenges AGI narrative with Windsurf acquisitionMatt demonstrates sharp market knowledge by citing OpenAI's move to buy Windsurf, challenging why an alleged near-AGI lab would need to buy a specific point-solution developer tool.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Podcast Hook: Benedict Evans on AI Capabilities and Limitations | 3 | 3 | 2 | 1 | Matt sets up the discussion by asking if AI represents a platform shift or paradigm shift. Benedict takes the floor with a broad analysis comparing AI model commoditization to Moore's Law and mid-nineties PC magazines. | |
| The AI Hype Cycle, Internet Analogies, and the Metaverse Comparison | 4 | 4 | 3 | 2 | Matt prompts Benedict on practical AI use cases based on his previous writings. Benedict clarifies the difference between error rate reduction and deterministic reliability. | |
| Critiquing OpenAI's Deep Research and Flawed Benchmarks | 4 | 6 | 4 | 1 | Matt briefly corrects a name slip between DeepSeek and Deep Research. Benedict dissects OpenAI's Deep Research promotional examples, showing how it pulled flawed StatCounter data and inverted basic statistics. | |
| Adapting Workflows to New Technology and Historical Shifts | 6 | 2 | 2 | 4 | Matt counters Benedict's skepticism toward Deep Research by highlighting his own success using it for qualitative tasks. Benedict agrees that OpenAI showcased the exact wrong quantitative use case. | |
| Real-World Utility vs. Crypto Comparisons and Generational Adoption | 5 | 3 | 3 | 2 | Matt highlights Google Trends data showing seasonal drops in ChatGPT usage tied to school holidays. Benedict contrasts AI's real enterprise adoption against crypto's search for actual use cases. | |
| Platform Shift Historical Analogies and Distribution Wars | 4 | 4 | 2 | 2 | Matt brings up model commoditization and moats. Benedict explains how consumer mindshare differs from raw benchmark performance, contrasting Perplexity and Claude with ChatGPT. | |
| OpenAI's Pivot to Applications and Corporate Strategy | 6 | 3 | 2 | 3 | Matt highlights OpenAI hiring Fiji Simo as CEO of Applications as a telling move. Benedict notes the irony of AGI claims existing alongside the need to build traditional application layers. | |
| Enterprise SaaS Traction, Coding Tools, and Legal Adoption | 6 | 3 | 2 | 3 | Matt cites industry reports that OpenAI is acquiring Windsurf to question why an alleged near-AGI company needs niche coding tools. Benedict compares developer tool acceleration to early AWS dynamics. | |
| Big Tech AI Strategies: Meta, Microsoft, Nvidia, and Apple | 5 | 5 | 3 | 2 | Matt guides the discussion to big tech strategies, focusing on Apple. Benedict details Apple's WWDC Siri demonstration, explaining why multi-step agentic systems remain unbuilt across the industry. | |
| Apple, Google, and the Challenge of AI Search | 5 | 4 | 4 | 3 | Matt shares observations of non-technical users defaulting to ChatGPT as a search engine. Benedict forcefully clarifies that LLMs are not search engines. | |
| Enterprise API Utility and Hyperscaler Infrastructure CapEx | 5 | 3 | 2 | 2 | Benedict outlines hyperscaler CapEx commitments exceeding $300B. Matt points out that AWS is positioned to benefit regardless of model breakthroughs because of compute demand. | |
| Meta's Distribution Strategy and Standalone AI App Challenges | 5 | 3 | 2 | 2 | Matt points to Meta AI's newly launched standalone app and its unique social feed feature. Benedict analyzes Meta's distribution choices between unbundling separate apps and adding tabs. | |
| Graphical User Interfaces (GUI) and Consumer Generative Media | 6 | 3 | 2 | 3 | Matt asks if the social feed serves as the new GUI for chatbots. Benedict expands on how traditional GUIs guide user workflows unlike blank prompt windows. | |
| Monetization, Advertising Models, and Interest Graphs | 5 | 4 | 4 | 2 | Matt brings up OpenAI exploring ad models. Benedict delivers a sharp commentary on EU regulatory actions against Meta and discusses personal interest graphs. | |
| Enterprise AI Deployment, Consulting Boom, and Platform Shifts | 4 | 5 | 2 | 2 | Benedict outlines his three-step model for platform shifts. Matt asks about transitioning between internal efficiency gains and market-redefining companies. | |
| AI Agents, Overpromised Demos, and Failure Modes | 5 | 4 | 5 | 3 | Matt probes whether constrained domain-specific agents like Figma tools work today. Benedict sharply dismisses flashy agent and humanoid robot demos as staged hype. | |
| The Fall of AI Doomerism and Real-World Security Realities | 4 | 4 | 5 | 1 | Matt asks why AI doomerism faded so quickly. Benedict mockingly characterizes doomers as insular autodidacts reliant on circular philosophical arguments. |
Statements from this episode (34)
Evans: AI frontier models are commodities spread across half a dozen organizations
“The thing that's become very clear, if it wasn't clear a year ago, is that the models themselves are sort of commodities in that, you know, there's half a dozen people who have a state-of-the-art model.”
Evans: Building AI models is getting costlier while usage costs fall
“Building a model gets more expensive, but the cost of using the model gets cheaper.”
Evans: Current AI models still cannot replace existing software like Excel
“But it still can't actually replace any of the software you use. It can't replace Excel, it can't.”
Evans: Lower AI error rates cannot replace required absolute accuracy
“And if you cannot depend on this to be right all the time, as opposed to slightly more of the time, then you either can't use it, or you have to use it in very different ways to the ways you could use it if it was always right.”
Evans: You cannot trust AI research reports on unfamiliar topics
“But if you go to it and say, give me a 40 page report on something I don't know much about, you can't trust any line of that report, because most of it will be right, probably, or it will be roughly right, but if there's anything, but you won't be able to depe…”
Evans: OpenAI's Deep Research inverted numbers in its own marketing example
“Then the interns typed the number in wrong. Like, it was literally the wrong percentage. It was like, 65, 35, instead of 35, 65.”
Evans: Users are forcing non-deterministic AI into deterministic roles
“We're still at that beginning of, like, forcing it to do deterministic, forcing it to be a deterministic system, which of course it isn't.”
Evans: Generative AI is deployed in thousands of companies, unlike crypto
“There are hundreds and hundreds of companies who've already got this in production, doing stuff that's really useful where it works, where you understand what it is. So this is just objectively wrong to say that it's useless. It's already not, not in the way t…”
Evans: AI is not currently on a path to 100% factual accuracy
“Are you telling me this is going up to the point that I'm going to be able to use deep research and the numbers will all be right, and I'll know that they're all right? Because I don't think we're on a path to that. Or at least I don't think we know that we're…”
Evans: ChatGPT usage on Google Trends drops sharply in summer and Christmas
“I mean, it's funny if you look at Google Trends. There's a big sag in the summer and then a big sag in the Christmas week.”
Evans: Sam Altman's role is mostly fundraising, politics, and promotion
“Like a lot of Sam Altman's role at the moment, it's like you could split his role into capital raising, politics, like internal tech politics and promotion.”
Evans: Consumer chat interfaces are thin wrappers, not vertical SaaS
“The only thin, thin GPT wrappers are what you get when you go to chatgpt.com and claude.com and grok and all these others. That's a thin wrapper on a model. Whereas you know, name your vertical enterprise SaaS company. That's not a thin wrapper.”
Evans: OpenAI Product Chief Kevin Weil is building a thin GPT wrapper
“Kevin Wheel is building a thin GPT wrapper. I mean, I love Kevin, but like, that's his job, is to build a thin GPG wrapper.”
Evans: Legal AI adoption is slowed by plausibility versus accuracy
“There's a huge difference between a legal brief that looks right and a legal brief that is right.”
Evans: GPTs could lower software creation costs by an order of magnitude
“The analogy that's been floating around, I think, is, is to compare this with AWS, in the sense that AWS was a sort of an order of magnitude change in how easy you could get a startup out of the door. You didn't need to write all this stuff yourself and buy in…”
Evans: Nvidia sells integrated computer systems, not just GPU chips
“People still think of NVIDIA as making GPUs in the sense that they make chips and sell chips. That's not what they do. They sell computers. They sell custom computers, kind of like Sun Microsystems did with a whole networking stack and a software stack on top …”
Evans: AI scientists possess no special expertise on global geopolitical threats
“One of the sort of fallacies here is a sort of an appeal to authority, which is, you know, well, that person is an AI scientist, so they must know if this is a threat to world peace. No, they don't. They're an AI scientist. They don't know anything more about,…”
Evans: Apple's AI writing tools are just better spellcheck
“They have the writing tools, so you can select a bunch of text and hit proofread. And it, it's like spellcheck. Or you can summarize it, or you can select some text and turn it into a table. It's useful. It's a feature. It's just a feature. It's like spellchec…”
Evans: Google and Meta delayed LLMs in 2022 due to high error rates
“This is why Google and Meta didn't launch their own LLMs in twenty-twenty-two when they had them as well, because they looked at them and said, well, they're wrong too much.”
Evans: Big Tech data center spending will exceed $300B this year
“Google, Meta AWS, not Amazon overall, AWS only, and Microsoft spent about two hundred twenty billion dollars building data centers last year, and will spend about 300, maybe over 300 this year, depending on where their numbers come out.”
Evans: Meta and AWS both benefit from AI becoming cheap commodity infrastructure
“In a sense, like, AWS and Meta are on the same page, and then Meta wants this to be cheap, generic commodity infrastructure that's sold at marginal cost, and they will differentiate on cool Facebooky stuff on top. Amazon want this to be cheap, generic commodit…”
Evans: Foundational LLMs are similar, but OpenAI holds consumer mindshare
“The models are all sort of the same, but OpenAI is the only one that anyone uses, that, that, that has consumer mindshare.”
Evans: Consumer AI apps currently lack viral loops and network effects
“There's no viral loop. There's no network effect. There's no reason why you should use the one your friends use. There's no reason this one gets better because everyone else uses it, at least not yet.”
Evans: No standalone breakout consumer apps exist wrapping ChatGPT API
“We don't have a breakout, there's no standalone breakout consumer app. There's no one, there's no, there's all these, there's, there are all these enterprise SaaS stuff. There is not really a consumer equivalent. There aren't hundreds of consumer apps using th…”
Evans: Memory features in AI models create user stickiness, not network effects
“I think that stickiness, I don't think it's a network effect.”
Evans: Apple has user interest graph data but refuses to use it
“Apple also in principle has an interest graph. It just refuses to use it.”
Evans: OpenAI lacks visibility into user search, purchases, and social media activity
“Back to open AI, they've got a partial view on you, but like, they don't know what you've bought. They don't know what you've searched for. They don't know where you go. They don't know what Instagram you look at and what TikTok you look at and what YouTube yo…”
Evans: Accenture recorded $1.4B in generative AI bookings in one quarter
“Accenture built 1.4 billion, booked 1.4 billion of new generative AI bookings last quarter.”
Evans: 20% to 30% of large companies have deployed AI projects
“Every big company is now, like, 20 to 30% of big companies have got stuff in deployment, but every big company's got pilots.”
Evans: Elon Musk's autonomy and humanoid demos are fake
“It's bullshit. Yeah, but it's not a real demo. It's not working.”
Evans: Siri and Alexa were decision trees despite natural language capabilities
“There was like a trap with Siri and Alexa, which was that natural language processing worked, so you thought it was AI, and it wasn't. It was actually still just an IVR, it was still just a tree.”
Evans: Humanoid robots represent mechanical mobility, not AGI
“And there's a trap with these humanoid robots, which is some people look at them and think it's AGI, and it's not, it's just a robot that's got legs instead of wheels, but it's still a robot.”