Jul 31, 2024 · 1h 9m · news
Ethan Mollick: Why OpenAl Abandons Products, The Biggest Opportunities They Have Not Taken | E1184 · 20VC with Harry Stebbings
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Wharton professor Ethan Mollick joins Harry Stebbings to dissect the profound business, educational, and societal implications of the AI revolution. He argues that navigating this transition requires understanding the deep disconnect between AI labs and corporate needs, restructuring education around active learning, and adapting investment strategies for an AGI-driven future.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Harry holds 20% of the talking time here. How this is scored →
Harry as informed peer 5.3 Guest teaching 6.2 Guest disagreement 3.3 Harry pushing back 4.2
speaking balance: gold is Harry, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 43:50 Rejecting Sam Altman's 100x heuristic
Ethan forcefully rejects Sam Altman's 100x model improvement heuristic, calling it baffling, unhelpful, and operational nonsense for decision-makers.
Hardest push from Harry ▶ 52:06 Challenging AI tutor superiority over top video lecturesHarry directly pushes back on Ethan's education optimism, questioning whether interactive AI tutors offer any genuine order of magnitude improvement over world-class video lectures.
Biggest teaching moment ▶ 48:39 Turkey RCT showing raw GPT-4 math tutoring degraded test performanceEthan educates Harry on the dangers of naive AI deployment by citing a Wharton study in Turkey where students using raw GPT-4 scored worse on exams due to passive cheating.
Harry holds his own ▶ 9:56 Pitting Kevin Scott's compute thesis against Alexandr Wang's data bottleneck thesisHarry demonstrates strong technical domain knowledge by directly contrasting Microsoft CTO Kevin Scott's compute view against Scale AI founder Alexandr Wang's data bottleneck thesis.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Harry as informed peer | Guest teaching | Guest disagreement | Harry pushing back | Why |
|---|---|---|---|---|---|---|
| Analyzing Llama 3.1 and Open Source AI | 3 | 4 | 2 | 2 | Harry asks Ethan about the newly released Llama 3.1 model and how it impacts open versus closed source dynamics. Ethan explains that social media buzz exaggerates weekly leader shifts while steady enterprise adoption moves much slower. Harry listens collaboratively without deep pushback. | |
| The Four Potential Outcomes of AI Development | 5 | 5 | 2 | 4 | Harry introduces an apt analogy comparing current AI model development to incremental iPhone releases that plateaued into minor camera and calculator updates. Ethan acknowledges the cleverness of the comparison but clarifies that AI capabilities expand via jagged intelligence rather than smooth hardware iterations. | |
| Identifying the True Bottlenecks of AI Performance | 6 | 6 | 3 | 4 | Harry demonstrates strong preparation by citing opposing technical theses from Kevin Scott on compute vs Alexandr Wang on data bottlenecks. Ethan explicitly takes a contrarian position, arguing end users do not care about technical bottlenecks and introduces the historical concept of 'reverse salients'. | |
| The Steam Engine Analogy and 'Skilled Artisans' of AI | 5 | 5 | 2 | 3 | Harry references Ethan's writing regarding steam engines vs picks and shovels. Ethan explains why VCs misunderstand technology diffusion, noting that steam engine value required skilled artisans rather than raw engines. Harry questions why labs fail to provide documentation, and Ethan attributes it to Silicon Valley's obsession with AGI scaling. | |
| The Debate Over Open Source AI and Its Security Risks | 6 | 6 | 3 | 4 | Harry frames the open source safety debate using Vinod Khosla and Marc Andreessen's opposing positions. Ethan rejects simplistic binaries, detailing tactical risks like automated spearfishing alongside strategic corporate maneuvers like Meta spoiling competitors' margins. | |
| Designing Agile Regulatory Models for AI | 6 | 6 | 3 | 6 | Harry pushes back on regulatory optimism by highlighting the EU AI Act's stringent constraints and potential to stifle European tech adoption. Ethan agrees that over-regulation is dangerous, backing up his point with Wharton data comparing US venture capital concentration against France and Germany. | |
| The Geographic Imperative of Silicon Valley | 5 | 7 | 3 | 4 | Ethan educates Harry on geographic necessity, citing empirical data showing that VC deals correlate directly with 40-mile physical radii and direct flight routes. Harry presses on product abandonment, and Ethan details OpenAI's neglect of Code Interpreter due to internal AGI compute focus. | |
| The Rise of 'Secret Cyborgs' and Enterprise Adaptation | 4 | 6 | 2 | 2 | Ethan outlines the 'secret cyborg' phenomenon inside enterprises, where workers secretly use AI tools to finish tasks but conceal usage out of fear of increased workloads or job loss. He shares a striking anecdote about a bank executive who used ChatGPT on her phone to draft a ban on ChatGPT. | |
| The Realities of Job Displacement and Industrial Revolutions | 6 | 6 | 4 | 6 | Harry pushes back against optimistic talent redistribution claims by citing Klarna's 70% efficiency jump and widespread customer service layoffs. Ethan supports the pushback, warning against market complacency and referencing historical disruptions like 1930s telephone operators and Luddite riots. | |
| AI's Democratic Potential vs. the Tech Elite Gap | 5 | 6 | 3 | 5 | Harry voices concern over a widening economic divide where 1% of Silicon Valley elites leverage 10x AI productivity while everyday UK citizens fall behind. Ethan counters by explaining non-coders' prompt engineering advantages, citing his wife's prompts being adopted as benchmark standards by Google. | |
| The Evolution of Consumer AI Interfaces | 4 | 5 | 2 | 3 | Harry asks why university students achieve 70%+ AI adoption while enterprise adoption lags. Ethan explains that academic assignments have clear, low-friction solution paths, whereas corporate workflows require domain nuance and organizational context. | |
| The Limits of the Lean Startup Method in Radical Tech Eras | 5 | 6 | 4 | 4 | Ethan criticizes the classic Lean Startup methodology, arguing that iterative product-market fit testing fails during radical technological shifts. Harry acknowledges his own VC training in incrementalism and asks what alternative funding model functions in radical eras. | |
| How Venture Capital Must Adapt to AGI Timelines | 5 | 7 | 5 | 3 | Ethan points out a glaring strategic contradiction in venture capital: VCs publicly proclaim AGI is 5 years away while funding wrapper startups that would immediately be rendered obsolete by AGI. Harry asks Ethan to clarify why startups cannot survive an AGI world. | |
| The Hyped Timelines and Practical Gaps of AGI | 7 | 7 | 8 | 6 | Harry cites Sam Altman's rule of thumb regarding whether startups will get steamrolled by 100x model improvements. Ethan forcefully rejects the premise, labeling Altman's 100x heuristic baffling and unhelpful for actual operational decisions. Harry defends the heuristic with humor. | |
| Ethan's Perspective on AI and Education Reform | 5 | 8 | 3 | 4 | Ethan explains why AI tutors cannot replace physical schools due to complex social and motivational structures. When Harry asks about managing public school class sizes, Ethan cites a Wharton randomized controlled trial in Turkey showing students using raw GPT-4 math tutoring performed worse on exams due to passive cheating. | |
| Why Learning Must Remain Difficult in the AI Era | 6 | 7 | 4 | 7 | Harry directly challenges Ethan's optimistic vision of AI education, questioning whether interactive AI tutors offer any genuine order of magnitude improvement over watching top-quality video lectures. Ethan reframes the debate around pedagogical science, active learning, and flipped classrooms. | |
| The Two-Sigma Tutoring Effect and Pedagogical Design | 5 | 7 | 3 | 4 | Ethan breaks down Bloom's Two-Sigma effect in educational psychology and explains specific prompt engineering constraints required for educational bots. Harry asks if the lack of a human emotional bond limits AI tutoring, and Ethan explains that forcing self-reflection drives the learning gains. | |
| Why Subject Matter Expertise Matters in the AI Age | 5 | 7 | 3 | 4 | Ethan highlights why domain expertise is required to spot subtle errors in AI prompts, drawing a parallel to research showing expert VCs are immune to founder charisma because of domain knowledge. Harry asks if traditional homework is obsolete, and Ethan presents alternative interactive assignment designs. | |
| Compute, Energy, and the Infinite Demand for Intelligence | 6 | 6 | 3 | 4 | Harry quotes Sam Altman's statement that compute is the currency of the future and asks about long-term energy constraints. Ethan agrees under an AGI assumption, noting that intelligence demand is infinite, while providing grid context that data centers currently consume 1% of US power. | |
| AI in Democracy, Politics, and Human Behavior | 6 | 7 | 4 | 5 | Harry cites VC Jeff Lewis's prediction that citizens will vote for algorithms instead of political candidates. Ethan rejects the premise as techno-dystopian, but educates Harry on research showing AI is hyper-persuasive, making users 81.7% more likely to change their opinion in conversations. | |
| Infinite Content Creation and Bestseller List Manipulation | 6 | 7 | 3 | 5 | Harry discusses content devaluation due to infinite AI supply and asks if someone can buy their way onto the New York Times bestseller list for $75,000. Ethan reveals how the NYT uses manual dagger annotations to filter out bulk corporate purchases. | |
| Quick Fire Round: AI Future, Chips, and Meaning of Work | 5 | 6 | 3 | 3 | In a rapid-fire round, Ethan highlights the upcoming existential crisis surrounding meaning at work for middle managers whose tasks are automated. He concludes with a humorous request not to tell Sam Altman he disagreed with him. |
Statements from this episode (56)
Mollick claims his 1990s startup invented the paywall
“So the startup company I helped co-found invented the paywall.”
Mollick: Public underestimates upcoming closed-source AI model releases
“I think people are probably over underestimating how much ammunition the Closed source labs have and are going to release in the near future.”
Mollick: Open-Source GPT-4 Level AI Models Will Become Ubiquitous
“We now have an open source GPT four capable model and it's going to be everywhere.”
Mollick: AI researchers lack understanding of their models' societal implications
“The people train the models are all computer scientists, basically. I mean doing computer science, And they don't have a huge idea of the implications of the systems.”
Mollick: AI progress will continue as workplace integration has barely begun
“Option one is this is it like the, you know, the models don't get much better or, you know, and it's sort of this whole thing sort of fizzles out. I think this is unlikely because I think not only will models get better, but also we haven't even started integr…”
Mollick: AI's 'jagged' capabilities prevent it from fully replacing human labor
“Like right now that, you know, as we talk about in our research, AI is jagged. So it's really good at some stuff, really bad at other things. So, you know, and as a result, it can't sub in for all of human work because on one hand, it'll do a great job and som…”
Mollick: Tech's biggest financial returns come from solving 'reverse salients'
“The money is all to be made in the reverse salient, right? So like if you can make a billion dollars as a data company and, but you know, because that's the area everyone's stuck on, you become a data company, right? This is, I mean, it's kind of capitalism an…”
Mollick: Steam engine progress stalled until James Watt's patents expired
“Things didn't really take off until Watts patents expired and it can be openly adapted”
Mollick: AI value accrues to skilled integrators, not compute providers
“The real value of the steam engine came from having skilled artisans in your factory who said, I've got this thing that can make power go back and forth. How do I create the gearing to connect that to my, you know, my spinning Jenny, my ammunition manufacturin…”
Mollick: ChatGPT instantly obsoleted a bank's expensive custom GPT-3 tool
“I spoke to a very large financial institution, spent a huge amount of money building a GPT three powered sales assistant tool that as soon as chat GVD came out was instantly obsolete.”
Mollick: Lack of AI manuals leads to 'documentation by rumor'
“There is no manual out there for this stuff. There's not even a dissent, like there's not even a set of points about what the AI is good at and what it's bad at. And as a result, like it's, I would call it documentation by rumor.”
Mollick: AI advice boosts performance for already-successful Kenyan founders
“We know that people who get advice from AI do better as founders in Kenya, if they were already doing well”
Mollick: Open-source AI safety guardrails will be breached immediately
“And I do think that open models will immediately have their guardrails breached.”
Mollick: Meta open-sources AI models to spoil competitors' businesses
“So Meta doesn't really want to make money from models. So they're going to spoil, you know, their rivals, right?”
Mollick: Governments should adopt fast follow-up AI regulation, not pre-regulate
“It has a really nice model for AI regulation that I think is probably right, which is when you have a new technology, you don't know what the problems and issues are going to be. You do fast follow-up regulation. So you don't try and pre-regulate because you d…”
Mollick: Penn grads raised more VC in 2022 than France and Germany
“More money went to graduates from Penn, from the school I teach at, and last time I checked in twenty-twenty-two than everybody in France and Germany put together.”
Mollick: Empirical research shows founders must move to Silicon Valley
“That's an empirical result from a bunch of studies. Companies that, you know, there's been a study of Israeli companies in the Valley, New York companies. Like, it's just, the issue is, is that that's where the connections are, and it turns out Zoom only gets …”
Mollick: Average distance between a VC and portfolio company is 40 miles
“I mean, the average distance, at least pre-pandemic, I would, I'd be surprised if it actually changed. The average distance between a VC and a company to invest in is about 40 miles.”
Mollick: Direct SFO flights boost local VC investment in destination cities
“In fact, when a direct flight is added between SFO and another city, the VC investment in that city goes up. Because it's just getting, it's easier to fly there and help do, ah, and do monitoring there.”
Mollick: There are 16 managers for every software coder
“For every coder, there are 16 managers, right?”
Mollick: OpenAI partially abandoned its world-changing Code Interpreter product
“Code Interpreter is, is a huge world-changing product for data analysts that got partially abandoned by OpenAI.”
Mollick: OpenAI's $3B revenue run rate is largely an accident
“I mean, they're incidentally making three billion dollar run rate this year, I think by like just accident, but there isn't like a, there isn't really a product there right now.”
Mollick: Only 5-10% of professionals have used advanced AI models
“Five to 10% of people in any room, whether, by the way, Silicon Valley, actual people, right, who aren't at a lab, whether that's at a large bank, whether that's at a conference of innovation professionals, maybe five to 10% have used those models, and maybe t…”
Mollick: ChatGPT saves Danish knowledge workers 50% time on 30% of tasks
“A new study just came out of Denmark of people who are using chat GPT and, you know, in knowledge intensive work environments. And, you know, they're estimating that in, you know, over 30% of their tasks, they're saving 50% of their time.”
Mollick: Using AI to cut headcount forces employees to hide AI usage
“And we're used to IT solutions being a cost saving measure, right? If I get a 30% of productivity boost, I fire 30% of people. Your people are never going to show you how to use AI at that rate.”
Stebbings: AI is replacing 90% of customer service teams in most cases
“Which is replacing, in most cases that I see, 90% now of customer service teams.”
Mollick: 1930s switchboard automation permanently displaced older operators
“When the telephone switchboards went from sort of manual to digital in the starting, or not digital at that point, but mechanical in the 19, starting in the 19 thirties, At that point, I think one out of every 16 women had spent time as a telephone operator. I…”
Mollick: Good human managers make better AI power users than coders
“So coders are often not the best users. Often the best users are people who are actually really good at working with humans.”
Mollick: Google used his wife's prompt as a fine-tuning benchmark
“Google used her prompt as the gold standard to measure their fine tuned models against.”
Mollick: Multimodal interaction is the primary consumer interface for AI
“I think multimodal is really the answer here.”
Mollick: ChatGPT adoption exceeds 70% in universities versus single digits elsewhere
“There's a reason why adoption rates are over 70% in universities for chat GPT. And while they're like at a few percent elsewhere in the world, we figure stuff out like this.”
Mollick: AI chatbots already write essays better than most humans
“Chatbots right now, like, if you want to see the future where it's better than human, it's actually in homework, where in most cases it kind of solves the issue, writes an essay better than most people.”
Mollick: Lean startup methodology fails during breakthrough innovation eras
“I think the problems of the lean method are coming home to roost. What every VC wants to see is, you know, and especially in like app facing stuff is they want to see product market fit. There's a method we have, right? You come up with like a, you know, rough…”
Mollick: Silicon Valley founders build narrow wrappers despite predicting AGI
“It is very strange from one hand for all of these people in Silicon Valley to be like, yeah, you know, AGI is coming. And then the applications they're building are like these very narrow, like, hey, I slapped something on top of llama. And you know, it's like…”
Mollick: No current startup will survive in an AGI world
“The real problem right now is every startup in the world is betting against AGI. Which I find really funny because all the funders are like, yeah, AGI is coming in the next five years. If it is, why are you funding these startup companies? Like none of them ar…”
Mollick: Claude can design and iterate playable app prototypes in one prompt
“You could tell Claude come up with 30 ideas for a product to serve, you know, market X, then rate them all on quality and feasibility level. Then create, this is one prompt, by the way, then create a playable prototype of the interface for the application. The…”
Mollick: Crypto made all technology feel like hype and short-term speculation
“I also think crypto did us dirty in this kind of front, which is like, it made all technology feel like hype. And it emphasized again, short buck return.”
Mollick: Major AI labs lack educators and fail to understand teaching
“And I don't think a lot of the AI firms know that I know this cause we're deep working with all of them on things like education and like, they don't really understand education. There's no educators there. So they don't really understand what teachers do. And…”
Mollick: AI tutors cannot replace human teachers due to motivation needs
“People need extrinsic motivation to learn. It turns out that there's value in having an instructor guiding the direction of a class, that there's value in putting things into practice. So even an incredible AI tutor that knows you and loves you really well, do…”
Mollick: Unprompted GPT-4 math tutoring led to lower test scores
“The first randomized control trial we have, I have some of my colleagues at Wharton was giving GPT-IV people for math tutoring in Turkey. Now, they didn't do a huge amount of, like, you know, it was an assigned class, and they used the system, but it turns out…”
Mollick: AI tutors will enable flipped classrooms with larger class sizes
“I think in the long term we'll have flipped classrooms where that 20 per, where that giant classroom is actually fine because a lot of your learning is done outside of class with, you know, AI tutor help. And then inside of class will be activities, exercises,…”
Mollick: Wharton Generative AI Lab will open-source an AI tutoring system
“And by the way, we've actually built a version of this already at the General of AI Lab at Wharton. We'll be open sourcing all of that, like, that does this kind of stuff.”
Mollick: AI tutors asking students if they understand are poorly designed
“Our rule of thumb is that if it asks you, if you understand a topic or you're ready to move on, it's a bad tutor because humans don't know when they're ready to move on or not.”
Mollick: Subject matter expertise is critical to making AI work effectively
“Subject matter expertise is going to be absolutely critical in making AI work.”
Mollick: Research shows VCs are unaffected by pitch presentation skills
“There's this really interesting research that shows that venture capitalists Are not swayed at all by the quality of the speaker. Their ability to be a good speaker or not is absolutely irrelevant. Amateur and angel investors are swayed by that.”
Mollick: Homework effectiveness fell from 80% to 20% due to cheating
“There's this great study at a repeating university that found that homework It improved, when you did the homework, it improved something like 80% of people's test scores in 2008. And by 2020, it only helped 20% of people. And that's not because homework stopp…”
Mollick: His Wharton classes are now 100% AI-based with custom simulators
“My classes are a hundred percent AI based at this point. Students have AI mentors and tutors they talk to. They have AI based assignments. When they learn how to do hiring, I built, we built a simulator that actually makes them have to fake hire somebody and t…”
Mollick: Nuclear power plants will be built rapidly for AI compute
“And we're going to build a lot of nuclear power plants, I guess, and, you know, in relatively short order.”
Mollick: AI queries use 100x more energy than Google searches
“And the truth is that AI uses a lot more energy per query. We don't know exactly, probably two orders of magnitude than a Google search, but a lot less orders of magnitude energy than a human doing the same amount of work, right, with a laptop.”
Mollick: AI accounts for at most 0.1% of US power consumption
“Right now, one percent of US power goes to data centers, and maybe 10% of that goes to AI at most.”
Mollick: AI is 81.7% more persuasive than humans in controlled experiments
“Like now on the other hand, we do find that AI is hyper persuasive already, right? In a controlled experiment where you do, where you're apps to be, you talk to a normal person versus the AI, you're 81.7% more likely to change your views, the AI's view than to…”
Mollick: Selling 6,000 hardcover copies weekly yields NYT Bestseller status
“If you're, like, you're selling, like, 6000 hardcover copies in a week. Like, that's getting on the New York Times bestseller list.”
Mollick received NYT bulk-sale dagger after 500-copy corporate order
“I actually got the little dagger on mine because a company bought 500 copies, which wasn't the main reason for the list, but they would have found that suspicious.”
Mollick: Exponential progress in AI models will continue for a while
“I have gone back and forth on how much juice the technology has left, and now I'm back to the, it has lots of juice left, like the exponential continues for a while, and I think I was not clear on that for a long time.”
Mollick: Major AI companies will inevitably enter custom chipmaking
“Yeah, I mean, the, your requirement in any supply chain pipeline is to eat the value. If you're, you know, like, that's the whole idea of, you know, how those things work. So, like, if the, if you're spending a lot of money on chips, you go into chip making. J…”
Mollick: AI automating middle management will cause a workplace meaning crisis
“What's going to happen that I'm very worried about is when you realize as a middle manager that AI does your work and nobody cares like, what does that mean for the nature of work? Like, how does that matter if people don't care? Like, if AI subs in and does s…”
Shorts cut from this episode
▶ How to become a bestselling author ✍️ · 20VC with Harry Steb (@1:03:09)
▶ Are all startups doomed? ☠️ · 20VC with Harry Stebbings (@0:22)
▶ Do you lie about using AI? 🤫 · 20VC with Harry Stebbings (@27:02)
▶ Is AI the next steam engine? 🚂 · 20VC with Harry Stebbings (@13:22)
▶ Why OpenAI scraps too many products 🤖🗑️ · 20VC with Harry (@0:02)