Oct 19, 2025 · 1h 7m · lennys-podcast
How to measure AI developer productivity in 2025 | Nicole Forsgren
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this podcast episode, host Lenny Rachitsky interviews developer productivity expert Nicole Forsgren on how artificial intelligence is transforming software engineering, Developer Experience (DevEx), and organizational performance. Nicole shares actionable frameworks from her book Frictionless for eliminating workflow friction, adapting metrics like DORA and SPACE for non-deterministic AI, and aligning engineering velocity with business value.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Lenny holds 31.5% of the talking time here. How this is scored →
Lenny as informed peer 4.3 Guest teaching 5.6 Guest disagreement 1.3 Lenny pushing back 0.3
speaking balance: gold is Lenny, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 12:23 Nicole rejecting output metrics
Nicole bluntly rejects lines-of-code metrics as a lie, warning that LLMs make it trivial to game raw volume and introduce tech debt.
Hardest push from Lenny ▶ 14:50 Lenny challenging metric obsolescenceLenny directly challenges whether DORA and SPACE frameworks have become completely out of date in an AI-assisted era.
Biggest teaching moment ▶ 28:58 Nicole correcting the speed obsessionNicole reframes speed for its own sake, schooling listeners that shipping garbage faster is worthless without sound product strategy.
Lenny holds their own ▶ 20:52 Lenny synthesizing AI workflow shiftsLenny applies domain knowledge to reframe Nicole's points on deep work, insightfully identifying that individual contributors are effectively becoming engineering managers for AI agents.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Lenny as informed peer | Guest teaching | Guest disagreement | Lenny pushing back | Why |
|---|---|---|---|---|---|---|
| Defining DevEx and Developer Productivity Beyond Raw Output | 4 | 5 | 1 | 0 | Lenny opens by asking Nicole to define DevEx and frames the intersection of productivity and engineering happiness. Nicole elaborates on why output alone is misleading and how cognitive load impacts burnout. | |
| Flow State, Cognitive Load, and Engineering Workflows with AI | 5 | 6 | 2 | 1 | Lenny brings up his own engineering background to ask about flow state. Nicole explains the counter-intuitive dynamic of how senior engineers maintain flow by delegating architecture chunks to AI agents. | |
| Why Traditional Metrics and Lines of Code Fail with AI | 3 | 7 | 3 | 0 | Nicole emphatically dismisses traditional productivity metrics as lies, pointing out how easily lines of code are gamed when LLMs generate verbose code. | |
| Adapting DORA and SPACE Frameworks for Non-Deterministic AI | 5 | 7 | 2 | 1 | Lenny asks whether frameworks like DORA and SPACE are outdated in an AI paradigm. Nicole clarifies that DORA must be used strictly for its prescribed purpose while SPACE provides an adaptable multidimensional framework. | |
| Restructuring Workdays and Deep Work Around AI Interruptions | 5 | 6 | 1 | 0 | Nicole explores research around deep work limits and speculates that AI enables productive 45-minute blocks. Lenny synthesizes this by comparing developers to engineering managers delegating to junior AI agents. | |
| The Business Case for DevEx and Starting with a Listening Tour | 3 | 6 | 1 | 0 | Nicole outlines the business value of DevEx and advises teams to begin with listening tours rather than automated tooling. | |
| Identifying Team Inefficiencies and the Balance of Speed and Strategy | 6 | 6 | 2 | 2 | Lenny asks how leaders can tell if their team is moving fast enough. Nicole explains system friction smells and cautions that shipping trash faster is useless without product strategy. | |
| Combining AI Prototyping with Product Strategy and Experimentation | 4 | 5 | 1 | 0 | Lenny and Nicole discuss how rapid prototyping and customer experimentation compress development cycles while requiring solid stakeholder alignment. | |
| Sponsor Message: Coda | 4 | 5 | 1 | 0 | Following the midroll ad read, Nicole describes measured velocity gains and the unblocking impact of AI IDEs, while Lenny brings up Andrej Karpathy's debugging anecdotes. | |
| Nicole's Book 'Frictionless' and the Seven-Step DevEx Framework | 5 | 6 | 1 | 0 | Lenny highlights Atlassian's billion-dollar acquisition of DX to underscore DevEx value. Nicole details the 7-step frictionless framework she co-authored with Abi Noda. | |
| Launching a DevEx Team, Quick Wins, and Navigating the J-Curve | 4 | 5 | 1 | 0 | Lenny draws on his experience seeing DevEx teams formed at Airbnb. Nicole details team composition, quick wins, and the J-curve of adoption. | |
| Translating DevEx Metrics and Financial Value to Leadership | 4 | 6 | 1 | 0 | Nicole demonstrates how to translate developer metrics like flaky build cleanup into executive language such as cloud cost savings and recovered capacity. | |
| Aligning AI Productivity Measurement with Executive Priorities | 5 | 6 | 1 | 0 | Lenny asks for tactical measurement advice for AI investments. Nicole explains how to align measurement strategies directly with executive priorities like margins or velocity. | |
| Designing Effective DevEx Surveys and Satisfaction Over Happiness | 4 | 6 | 2 | 1 | Lenny questions Nicole on why she dislikes happiness surveys. Nicole explains why broad happiness measures fail and why tooling satisfaction provides actionable signal. | |
| Essential AI Tooling and Applying a Product Mindset to DevEx | 4 | 5 | 1 | 0 | Lenny shares non-engineering use cases for Claude Code. Nicole emphasizes applying a strict product management mindset and lifecycle sunsetting to internal DevEx tools. | |
| AI Corner: Practical Home Design and Image Generation with LLMs | 4 | 3 | 0 | 0 | Nicole shares her personal use of image generation models for home remodeling during AI Corner, followed by book recommendations and her new role at Google. |
Statements from this episode (23)
Forsgren: Superior tools cannot overcome poor developer experience
“When DevEx is poor, everything else just isn't gonna help, right? The best processes, the best tools, the best Whatever magic you have, right? If the DevX is bad, everything kind of takes.”
Forsgren: High-toil engineering workflows drive burnout and block innovation
“If you're getting there in ways that are high toil or high friction, then at some point a developer is going to burn out. Or if it's, you know, super high cognitive load, if it's hard to even think about what you're doing, because you're concentrating on like …”
Forsgren: DevEx relies on flow state, cognitive load, and feedback loops
“One way to talk about it is kind of three key things that Have components that are important of themselves. They also kind of reinforce each other. So flow state is one of them. Cognitive load is another, and then feedback loops are another.”
Forsgren: AI shifts developer flow from syntax to high-level orchestration
“It helps them keep in the flow in terms of, instead of details and line by line writing, they're in the flow in terms of what's my goal. What are the pieces that I need to get there? How quickly can I get there? So then I can step back and kind of evaluate eve…”
Forsgren: Most developer productivity metrics are a lie
“I will say most productivity metrics are a lie.”
Forsgren: DORA metrics alone are insufficient in the AI era
“If that's all you're looking at, it's not going to be sufficient anymore because AI has now changed the way we think about feedback loops.”
Forsgren: SPACE framework adapts effectively to AI development environments
“We're actually seeing that space applies fairly well in these new emerging contexts like AI, because we still want to look at, so space is an acronym, right? So we still want to look at satisfaction. We still want to look at performance.”
Forsgren: Non-deterministic LLM code demands active evaluation over direct acceptance
“LLMs are non-deterministic, right? Now we can't just Put in a command and get something back and accept it. We really need to evaluate it. So, you know, are we seeing hallucinations? What's the reliability? Does it meet like the style that we would typically w…”
Forsgren: Humans max out at four hours of deep work daily
“Gloria Mark has done some really good work on attention and deep work, and Cubans can get about four hours of good deep work a day. And like, that's about it.”
Rachitsky: Every software engineer is becoming an AI engineering manager
“Essentially everyone, every engineer is turning into an EM, engineering manager, coordinating all of these junior AI engineers.”
Forsgren: Improve DevEx by listening to engineers before building tools
“Start with listening and not with tools and automation. So many times companies are like, well, I'm just going to build this tool or going to, I'm going to build this thing. Often you build a thing that you yourself have had a challenge with, or that like is e…”
Forsgren: Almost no engineering teams have reached their maximum velocity limits
“Most teams can move faster, right? So And also given what we know about cognitive load, not all speed gains are necessarily good. Right. Or the upside is going to be kind of limited, right? Once you hit kind of a certain point, most people are not even near th…”
Forsgren: Increasing engineering speed without strategy just ships trash faster
“You could, we can ship trash faster every single day. We need strategy and really smart decisions to know what to ship, what to experiment with what features we want to do in what order and what rollout, right? The strategy is the core piece. And then think ab…”
Forsgren: AI prototyping cuts A/B testing cycles to under a week
“Some places it used to take, you know, months to get something through production, to do A-B testing and get feedback. We can do this in a day or two, right? Definitely under a week.”
Forsgren: Better documentation and comments directly improve AI tool performance
“Turns out agents rely on good data, right? Because it's all about how they've been trained or how they've been grounded and better data gives you better outcomes, and some of that data includes documentation and comments. And the better documentation, the bett…”
Forsgren: PMs have essential DevEx skills that engineering teams often miss
“I think it's particularly, particularly relevant for PMs because if you're PMing something that involves software and building and creating software, improving DevX will only help your team. And also you have Key skills and insights and instincts that are so i…”
Forsgren: DevEx initiatives follow a J-curve before compounding benefits
“I will mention that it tends to follow something like J curve, right? So like you'll have a couple of quick wins and it'll look like a big, big win. And then you'll hit a kind of a little divot where suddenly The really obvious projects, the low hanging fruit …”
Forsgren: Frame test suite cleanup as cloud cost savings for leadership
“If we can clean up a test and build suite to a developer, they really want to hear about time saved and More reliable systems, right? There's less toil because they don't have to keep rerunning tests or kind of go clean up test suites. From the business perspe…”
Forsgren: DevEx metrics can correlate with business outcomes, not prove causation
“And then sometimes we can correlate to business outcomes and correlate is usually the best we can do here, but there can be some pretty compelling correlations in terms of speeding up time to value and increase market share, for example.”
Forsgren: Frame engineering productivity metrics using leadership's strategic terminology
“If you can solve a problem that they have, or it's like something that they're focused on, if you can slightly reframe it even, right? Like if they're calling everything developer productivity, go ahead and call it productivity. If they're calling it velocity …”
Forsgren: Limit DevEx barrier surveys to three prioritized items for clearer signal
“That can give you incredible signal because by making folks prioritize the top three things, if you let them pick everything, like it makes the data super, super messy, but three things and how often you can just come up with a score or a weighted score if you…”
Forsgren: Measure developer satisfaction rather than general happiness
“I will say I don't love a happiness survey. Because there are too many things that contribute to happiness. Happiness is a lot, right? So happiness is work. Happiness is family. Happiness is hobbies. Happiness is weekends. Happiness. There are so many things t…”
Forsgren: Run DevEx initiatives and productivity metrics like software products
“I think something that's important to think about in general is to bring a product mindset to any type of DevEx improvements that are happening, and also the metrics that we kind of collect and capture. And by that, I mean, we want to identify a problem, right…”