Apr 10, 2024 · 51m · mad
Is AI a platform shift or a paradigm shift? With Benedict Evans
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of The MAD Podcast, tech analyst Benedict Evans joins host Matt Turck to analyze generative AI's impact on software architecture, enterprise workflows, tech hype cycles, and societal risk. Evans contextualizes current AI developments within historical computing shifts, examining practical enterprise integration, algorithmic bias, and the limits of modern large language models.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 6.9% of the talking time here. How this is scored →
Matt as informed peer 3.1 Guest teaching 5.5 Guest disagreement 2.6 Matt pushing back 0.8
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 43:40 Benedict's explosive rejection of AI ethics regulations
Benedict forcefully rejects naive proposals for AI regulation and ethics codes, bluntly asking 'What the fuck are you talking about?' when people suggest regulating against software bugs.
Hardest push from Matt ▶ 13:24 Matt's synthesis of AGI vs vertical AI stackMatt directly challenges and reframes Benedict's narrative by forcing a clear distinction between an all-encompassing AGI model and vertical AI-powered enterprise software solutions.
Biggest teaching moment ▶ 38:15 The skin cancer ruler flaw in machine learning datasetsBenedict educates the host on how AI bias actually operates under the hood, demonstrating that models match statistical artifacts like rulers in cancer photos rather than obvious demographic variables.
Matt holds his own ▶ 9:05 Matt validating enterprise software unbundling patternsMatt demonstrates deep enterprise tech domain knowledge by connecting Benedict's historical spreadsheet narrative to the historical unbundling of databases.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Generative AI as a Platform Shift vs Previous Machine Learning | 3 | 5 | 1 | 1 | Matt asks a structured opening question comparing generative AI as a platform shift to previous machine learning waves. Benedict monologues at length, tracing the decade-long evolution from 2013 image recognition pattern matching to current generative capabilities. | |
| Lessons from Spreadsheets, Enterprise Tooling, and Unbundling | 4 | 4 | 1 | 1 | Matt interjects to validate Benedict's spreadsheet analogy, connecting it directly to how enterprise software unbundled core databases. Benedict agrees and expands on how enterprise SaaS unbundles Excel and Oracle. | |
| Where AI Sits in the Software Stack and the Command Line Analogy | 6 | 4 | 2 | 2 | Matt demonstrates sharp expertise by distilling Benedict's thesis into a clear synthesis regarding model capabilities versus vertical software layers. Benedict agrees and elaborates on how far up the stack models will go. | |
| Enterprise AI Strategy: Infrastructure, Vendors, and Business Impact | 2 | 6 | 1 | 0 | Matt asks a high-level prompt about enterprise AI strategy. Benedict delivers an uninterrupted masterclass explaining vendor roadmaps, incumbent features, and how AI impacts business models differently across industries. | |
| Platform Shift vs. Paradigm Shift and Practical Limits of Automation | 3 | 5 | 2 | 1 | Matt prompts Benedict on platform shift vs paradigm shift. Benedict grounds the discussion by sharing his personal workflow where ChatGPT fails to automate practical multi-file data extractions. | |
| Historical Sci-Fi, Lack of AI Theory, and Managing Unknown Risks | 2 | 7 | 4 | 1 | Benedict offers an intellectual critique of AGI predictions, comparing historical physics theories like Apollo or Newton to the total lack of a scientific theory of artificial intelligence. He forcefully rejects attempts to assign numerical probabilities to existential risk. | |
| Philosophical Proofs, Silicon Valley Scenes, and the Hype Cycle | 4 | 5 | 3 | 2 | Matt asks whether generative AI might be grossly overhyped. Benedict breaks down Silicon Valley subcultures and hype cycles, dismissing both existential doomers and total cynics who equate AI to scams like NFTs. | |
| Machine Learning Waves and Image Recognition | 2 | 6 | 2 | 0 | Matt introduces the topic of AI bias. Benedict educates listeners on how machine learning pattern matching identifies hidden artifacts in training data, using the classic ruler in skin cancer photography as a prime example. | |
| Software Vulnerabilities, AI Regulation, and the Post Office Scandal | 2 | 7 | 7 | 0 | Benedict references the UK Post Office scandal to argue that software errors are institutional failures, aggressively ridiculing naive calls for 'AI ethics codes' or government regulation to prevent software bugs. | |
| The Spatial Computing Landscape and Apple Vision Pro Utility | 3 | 6 | 3 | 0 | Matt brings up spatial computing and the Apple Vision Pro. Benedict analyzes the spatial computing landscape, arguing that headsets cannot replace smartphones unless optical technology advances to unobtrusive glasses. |
Statements from this episode (20)
Evans: Framing machine learning as pattern recognition unlocked enterprise adoption
“And it took a while to work out that the right level of abstraction was to think that this is pattern recognition.”
Evans: 2023's immediate generative AI use cases were coding and brainstorming
“But we're still, and we had, I think in 20, 23, that wave of the initial, oh my God, you can use it for that right now things, which is basically coding and brainstorming.”
Evans: The typical enterprise today runs 400 to 500 SaaS apps
“Why is it that the typical enterprise today has four to 500 SaaS apps?”
Evans: Every enterprise software company is unbundling Oracle, Gmail, or Excel
“Basically every enterprise software company for the sake of argument is unbundling Oracle, Gmail, or Excel.”
Evans: No-code apps have a natural ceiling in the enterprise
“I feel like no code apps have kind of a natural ceiling”
Evans: Fully autonomous end-to-end AI workflows require AGI
“But you kind of know at the same time that in practice, what I've just described would kind of require AGI. In practice, there's like 10 ways that that's going to break And never mind the hallucination problem, which is a whole separate conversation. There's j…”
Evans: Predictions that only tech giants had enough AI data were wrong
“And this is clearly what happened with the last wave of machine learning. There was a brief moment where people said it needs all this data. Only Google's got all the data. There's going to be, like, three people who've got enough data to do AI, and that turne…”
Evans: Incumbents Always Try to Turn Platform Shifts Into Features
“The classic pattern of the platform shift is the incumbents always try and make it a feature.”
Benedict Evans still hasn't found a daily use case for ChatGPT
“In my actual job, I can't work out something that I would actually use ChatGPT for.”
Benedict Evans: Science lacks an underlying theory for how LLMs work
“We don't have any equivalent set of theories for intelligence or artificial intelligence. We have a lot of theories of how some bits of it might work. But we do not have a theory of what we have and what, and in what sense is what we have is different and the …”
Benedict Evans: We cannot predict what happens when doubling LLM training data
“We can't do that with LLMs either. We don't know what will happen if you put double the data in or why, or we don't know why it works with this much data or not.”
Benedict Evans: The tech industry lacks enough human data to 100x LLMs
“And so that means you kind of can't do like a prediction. There's no Moore's law here where you can say, well, it'll get to that power of compute level at this, but at this point set aside the fact that we actually don't have enough data to give it 10 or a hun…”
Evans: Intense tech subcultures drove his departure from Silicon Valley
“One of the reasons I left Silicon Valley is I couldn't deal with this kind of thing.”
Benedict Evans: Generative AI will inevitably experience a market bubble
“Clearly, if we're not in a bubble now, we're going to have a bubble. It's because that's just like the nature of the light, the cycle of life. There will be a bubble around each new technology.”
Evans: Image recognition from the prior ML wave was not overhyped
“But that doesn't mean that image recognition was overhyped.”
DeepMind AI discovered unknown biological differences between male and female retinas
“DeepMind did a project with Moorfields, which is iHospital in the UK. And they were looking at retinas. And their system discovered a difference between male and female retinas, and apparently medical science didn't actually know there was a difference between…”
Evans: Machine learning functions as an infinitely fast intern
“One of the ways I used to talk about machine learning is that it gives you infinite interns. Like you would like someone to listen to every call coming into the call center and tell me if the customer is angry. Like you've got a million calls a day. You don't …”
Evans: Major software failures stem from institutional rot, not missing regulation
“You don't look at this and say, well obviously the solution is that we needed to have a database regulator that makes sure that databases don't have bugs. Rather you look at it and say, well, this is an institutional failure. A in Fujitsu in the post office an…”
Evans: Meta Quest 3 lacks traction with roughly 10M active users
“Meta has the Quest III, which is a perfectly good, credible consumer device. It does not have traction. Does not have a, it's probably got, I don't know, maybe ten million active users. Huge abandonment rate in the past. It's not really good for anything other…”
Evans: VR headsets will not replace smartphones as primary computing platforms
“You're not going to wear a headset, no matter how light and cheap it is, all day outdoors. I would not have worn it walking here, even if it weighed a hundred grams and had completely perfect pass-through. Therefore, it can't replace your phone. Therefore, we …”