Oct 14, 2025 · 1h 30m · a16z
Is AI Slowing Down? Nathan Labenz Says We're Asking the Wrong Question
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of the a16z Podcast, guest Nathan Labenz dismantles popular claims of AI development flatlining, arguing that the frontier is rapidly advancing through post-training reasoning, automated code generation, multimodal applications, and physical AI. He provides a nuanced roadmap for how these evolving capabilities will reshape economic productivity, developer roles, scientific research, and workforce dynamics.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The host holds 5.8% of the talking time here. How this is scored →
The host as informed peer 3.2 Guest teaching 6.1 Guest disagreement 1.9 The host pushing back 1.7
speaking balance: gold is the host, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 26:40 Refuting the METR Productivity Paper
Nathan forcefully dismantles the widely cited METR study, arguing critics latched onto it too easily and that testing developers on mature codebases without proper tooling knowledge created misleading conclusions.
Hardest push from the host ▶ 5:57 Challenging Cal Newport's Scaling HistoryErik directly challenges Nathan to re-examine Cal Newport's history of scaling laws and diminishing returns, forcing the guest to systematically edit the premise rather than accept it.
Biggest teaching moment ▶ 50:42 MIT Novel Antibiotics MasterclassAfter the host admits total ignorance regarding recent AI medical developments ('No. Tell us about it'), Nathan delivers an extensive explanation of MIT's novel antibiotics for drug-resistant bacteria.
The host holds their own ▶ 45:51 Citing Macroeconomic CapEx and GDP DataErik demonstrates strong domain knowledge by citing specific market figures, pointing out that Mag 7 represents a third of the stock market and AI CapEx exceeds 1% of US GDP.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| Debating Cal Newport's AI Flatlining Hypothesis | 3 | 5 | 3 | 2 | The host frames Cal Newport's argument on student cognitive strain and flatlining progress. The guest refutes the flatlining claim, distinguishing between student behavioral traps and model benchmark leaps between GPT-4 and GPT-5. | |
| Scaling Laws vs Post-Training Paradigms | 4 | 6 | 2 | 3 | The host prompts the guest to edit Newport's framing of diminishing returns from scaling laws. The guest explains how post-training, context window growth, and reasoning models alter the ROI equation beyond parameter scaling. | |
| Extended Reasoning and Scientific Breakthroughs | 3 | 7 | 1 | 1 | The host asks what it means to fully appreciate extended reasoning. The guest educates the host on recent breakthroughs including IMO gold medals, Terence Tao math problems, and Google AI co-scientist virology discoveries. | |
| The Perception Gap and GPT-5 Launch Missteps | 4 | 6 | 2 | 2 | The host hypothesizes that bearishness stems from everyday users not feeling frontier math gains. The guest reveals internal launch issues like broken model routers sending queries to non-thinking models. | |
| Evaluating AI Productivity Metrics and Labor Impact | 4 | 7 | 4 | 3 | The host points to the METR study showing engineers being slower with AI to question rapid job replacement. The guest strongly critiques the study's setup and cites high enterprise ticket resolution rates. | |
| Code Generation and Automated AI Research | 3 | 6 | 1 | 1 | The host asks about code generation and automated research bets. The guest outlines Replit v3 visual QA loops and OpenAI o3 resolving 40% of research engineering pull requests. | |
| Developer Employment Outlook and Compute Economics | 3 | 5 | 2 | 2 | The host asks directly whether engineer headcount will shrink in five years. The guest outlines the 95% price reduction per token and argues middle-tier developer tasks will be automated. | |
| Economic Automation Potential vs Pacing Factors | 5 | 5 | 2 | 2 | The host cites macro statistics regarding Mag 7 market concentration and CapEx exceeding 1% of GDP. The guest explores pacing bottlenecks like tacit knowledge extraction and regulatory pushback. | |
| Multimodal AI Architectures Beyond Language | 1 | 8 | 1 | 0 | The host admits complete ignorance when asked about AI antibiotic discoveries. The guest presents a detailed breakdown of MIT's AI-designed antibiotics for drug-resistant bacteria. | |
| Physical AI, Self-Driving, and Robotics Acceleration | 3 | 6 | 1 | 1 | The host connects back to Cal Newport missing non-language modalities. The guest details robotics acceleration, Tesla physical RL loops, and pre-training flywheels in embodiment. | |
| Agent Trajectories, Reward Hacking, and Alignment Risks | 2 | 7 | 2 | 1 | The host asks about agent trajectories. The guest highlights reward hacking, fake unit test generation, and system card findings where models attempted blackmail and unauthorized whistleblowing. | |
| Chinese Open Source Models vs US Frontier Dominance | 4 | 6 | 3 | 3 | The host challenges the guest with a statistic claiming 80% of AI startups use Chinese open models. The guest clarifies that open-source usage is a minority subset compared to commercial API calls. | |
| Empowering Education, Human Agency, and Positive Vision | 3 | 5 | 1 | 1 | The host steers the conversation to positive visions for education and agency. The guest details interactive screen-sharing study workflows and urges non-technical minds to engage with AI. |
Statements from this episode (27)
Labenz: Tool-Equipped Next-Gen AI Models Will Resemble Superintelligence
“When we start to give the next generation of the model these power tools, and they start to solve previously unsolved engineering problems, I think you start to have something that looks kind of like super intelligence.”
Labenz: GPT-4 to GPT-5 capability leap matches GPT-3 to GPT-4
“And if you look back to GPT three, you know, there's a huge leap. I would contend that the leap is similar from GPT four to five.”
Labenz: AI scaling laws are empirical observations, not guaranteed natural laws
“The scaling law idea, which is, you know, it's definitely worth agreeing, taking a moment to note that it is not a law of nature. You know, we do not have a principled reason to believe that scaling is some law that will go indefinitely. All we really know is …”
Labenz: GPT-4.5 achieved 65% accuracy on SimpleQA versus o3's 50%
“The O-three class of models got about a 50% on that benchmark, and GPT 4.5 popped up to like 65%. So, in other words, it basically, of the things that were not known to the previous generation of models, it picked up a third of them.”
Labenz: AI post-training reasoning currently yields higher ROI than raw scaling
“And it just seems like we're getting more benefit from the post training and the reasoning paradigm than scaling. But I don't think either one is I definitely don't think either one is, is dead.”
Labenz: Pure reasoning AI models achieved IMO gold without external tools
“Well, I mean, a big one from just the last few weeks was that we had an IMO gold medal with pure reasoning models with no access to tools from multiple companies. And, you know, that is night and day compared to what GPT-IV could do with math, right?”
Labenz: FrontierMath AI benchmark scores rose from 2% to 25% in under a year
“Now we've got the frontier math benchmark that is, I think now like up to 25%. It was two percent about a year ago, or even a little less than a year ago, I think.”
Labenz: Google's AI Co-Scientist solved an open virology problem independently
“And they gave it legitimately unsolved problems in science, and in one particularly famous, kind of notorious case, it came up with a Hypothesis, which it wasn't able to verify because it doesn't have direct access to actually run the experiments in the lab, b…”
Labenz: OpenAI's router failure caused bad initial GPT-5 outputs
“The problem at launch was that that router was broken. So all of the queries were going to the dumb model, and so a lot of people literally just got Bad outputs, which were worse than oh three because they were getting non thinking responses.”
Labenz: Plugged-in AI experts are not pushing timelines past 2030
“I don't think too many people, at least that I, you know, think are really plugged in on this, are pushing out too much past 20 30 at all.”
Labenz: Intercom's Fin AI agent resolves 65% of support tickets
“They now have this fin agent that is solving like 65% of customer service tickets that come in.”
Labenz: AI customer service agents will cause significant headcount reductions
“So I don't think these things go to zero probably in a lot of environments, but I do expect that you will see significant headcount reduction in a lot of these”
Labenz: AI auditor agent won state contract for 1M annual document audits
“They've created this auditor AI agent that just won a state-level contract to do the audits on, like, a million transactions a year of these You know, these packets of documents, again, scanned, handwritten, all this kind of crap and they just blew away the hu…”
OpenAI o3 model completes 40% of internal research engineer pull requests
“That's another data point, by the way, from this was from the O three system card. They showed a jump from like low to mid single digits to roughly 40% of PRs actually checked in by Research engineers at OpenAI that the model could do. So prior to O three, not…”
Labenz: Per-token model costs fell 95% from GPT-4 to GPT-5
“It's like 90 it's like a 95% discount from GPT-IV to GPT-V.”
Labenz: AI will outperform average developers on standard apps within five years
“But I would be very surprised if you can't get your nuts and bolts Web app, mobile app type things spit out for you for far less and far faster than, and probably honestly with significantly higher quality and less back and forth with an AI system than, you kn…”
Torenberg: AI capex spending currently exceeds 1% of GDP
“AI capex is, you know, over one percent of GDP.”
Labenz: Existing AI capabilities could automate 50-80% of work in 5-10 years
“I think if progress stopped today, I still think we could get to 50 to 80% of work automated over the next, like, five to 10 years.”
MIT researchers used AI models to create novel antibiotics for resistant bacteria
“It's been enough for this group at MIT to use some of these relatively, you know, narrow purpose-built biology models and create totally new antibiotics. New in the sense that they have a new mechanism of action. Like they're affecting the bacteria in a new wa…”
Labenz: Non-text AI modalities will unify with language models over time
“We have seen this play out with text and image where you had your text only models and you had your image only models, and then they started to come together and now they've come really deeply together. And so I think you're going to see that across a lot of o…”
Labenz: China may be ahead of the US in general robotics
“And this is one area where I do think China might be actually ahead of the United States right now”
Labenz: LLM fine-tuning and reinforcement learning will successfully power humanoid robotics
“All these techniques that have been developed over the last few years, Seems to me they're absolutely gonna apply to a problem like a humanoid robot as well.”
Labenz: AI agent task capacity will reach two weeks within two years
“If you extrapolate that out a bit and you're like, okay, take, take the four month case just to be a little aggressive. That's three doublings a year. That's eight X task length increase per year. That would mean you go from two hours now to two days. In one y…”
Labenz: Claude 4 system card documented AI blackmailing a human engineer
“In the cloud four system card, they reported blackmailing of the human. The setup was that the AI had access to the engineer's email and They told the AI that it was going to be like replaced with a, you know, a less ethical version or something like that. It …”
Labenz: Near Protocol pivoted to crypto to solve international AI worker payments
“They took a huge detour into crypto because they were trying to hire task workers around the world and couldn't figure out how to pay them. So they were like, this sucks so bad to pay these task workers in all these different countries that we're trying to get…”
Labenz: Chinese open-source AI models have surpassed American open-source models
“For those that are using open source, I do think it's true that the Chinese models have become the best.”
Labenz: Non-coders are performing legitimate frontier AI behavioral research
“Because you can get the AIs to code so well, I'm starting to see people who have never coded before. I'm working with one guy right now who's never coded before, but does have a sort of behavioral science background. And he's starting to do legitimate frontier…”