Sep 4, 2022 · 1h 4m · lennys-podcast
The role of AI in new product development | Ryan J. Salva (VP of Product at GitHub)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of Lenny's Podcast, host Lenny Rachitsky interviews Ryan J. Salva, VP of Product at GitHub, about the incubation, engineering, and organizational scaling of GitHub Copilot. Salva details how GitHub partnered with OpenAI to build the AI pair programmer, shares frameworks for transitioning research moonshots into production, and explores the future of software engineering in an AI-driven era.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Lenny holds 21.6% of the talking time here. How this is scored →
Lenny as informed peer 4.4 Guest teaching 5.6 Guest disagreement 0.2 Lenny pushing back 0.1
speaking balance: gold is Lenny, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 52:00 Forceful rejection of AI replacing human developers
Ryan firmly pushes back against the premise that AI will replace developers, adamantly stating GitHub will never support auto-generating code without a reasoning human at the keyboard.
Hardest push from Lenny ▶ 28:23 Lenny pressing for a concrete horizon definitionLenny challenges Ryan to specify whether Horizons 2 and 3 have concrete year-based definitions like Amazon's framework rather than general abstractions.
Biggest teaching moment ▶ 19:10 Origin story of OpenAI cloning GitHub reposRyan educates Lenny on how Copilot originated accidentally when OpenAI traffic spikes initially looked like a denial-of-service attack on GitHub infrastructure.
Lenny holds their own ▶ 1:00:20 Lenny showcasing sci-fi literature expertiseWhen Ryan brings up the film Arrival, Lenny immediately identifies that it was adapted from author Ted Chiang's short stories and recommends his wider bibliography.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Lenny as informed peer | Guest teaching | Guest disagreement | Lenny pushing back | Why |
|---|---|---|---|---|---|---|
| Ryan Salva's Background: Aesthetics, Philosophy, and Creative Tools | 3 | 3 | 0 | 0 | Lenny kicks off the interview with a lighthearted inquiry into Ryan's non-traditional background in philosophy of aesthetics. Ryan elaborates on viewing software as a creative medium, maintaining a collaborative and warm tone. | |
| Transitioning Leadership: From Microsoft Infrastructure to GitHub | 4 | 4 | 0 | 0 | Lenny asks about the organizational transition from leading Microsoft internal engineering systems to moving over to GitHub after the acquisition. Ryan details his motivation to get closer to the broader developer community. | |
| Demystifying GitHub Copilot and Sustaining the Flow State | 3 | 6 | 0 | 0 | Lenny prompts Ryan for a fundamental overview of GitHub Copilot. Ryan provides a clear technical breakdown of Codex-powered multi-line autocomplete and the psychological importance of preserving developer flow state. | |
| Real-World Copilot Applications: Education and Codebase Navigation | 5 | 5 | 0 | 0 | Lenny shares real-world user examples like centering divs and coding tutorials. Ryan expands with an anecdote about high school students building tools for small businesses and using Copilot to navigate unfamiliar codebases. | |
| The Origin Story: From Accidental Traffic Spike to Inline Autocomplete | 4 | 7 | 0 | 1 | Ryan details the origin story of Copilot, revealing how OpenAI unexpectedly hammered GitHub's infrastructure cloning repos, leading to the Arctic Code Vault partnership and experimenting with inline autocomplete. | |
| Model Optimization: Latency Thresholds and Prompt Engineering | 4 | 7 | 0 | 0 | Lenny probes into the optimization work between research models and production. Ryan explains empirical latency testing, revealing that ~200ms is the sweet spot for unobtrusive developer suggestions. | |
| GitHub Next and the Three Horizons Innovation Framework | 4 | 5 | 1 | 1 | Lenny asks if the Three Horizons framework uses rigid calendar definitions. Ryan reframes the framework around levels of ambiguity and confidence rather than strict calendar dates. | |
| Graduating Moonshots: Transitioning R&D to Production Squads | 5 | 7 | 0 | 0 | Lenny observes that corporate R&D teams often fail to graduate products and asks how GitHub succeeded. Ryan outlines their strategy of embedding researchers into product squads during technical preview. | |
| Organizational Blueprint for Successful Product Handoffs | 5 | 6 | 0 | 0 | Ryan details key handoff principles, including avoiding calendar-based exits for researchers, ensuring EPD owns the roadmap, and handling the cultural shift to engineering fundamentals. Lenny validates these pain points from product experience. | |
| Responsible AI: Ethical Guardrails and Content Filtering | 6 | 7 | 0 | 0 | Ryan discusses content filtering and ethical guardrails using the AI pair programmer persona. Lenny demonstrates tech history expertise by referencing Microsoft's infamous Tay chatbot incident. | |
| The Evolution of Software Engineering in the Age of AI | 5 | 6 | 1 | 0 | Lenny brings up a cited statistic about 40% of code being AI-written. Ryan corrects and contextualizes the stat, clarifying it applies specifically to Python, and discusses how AI abstracts lower-level syntax to elevate creative problem solving. | |
| Scaling Hurdles: GPU Constraints and Sustaining Community Trust | 4 | 7 | 1 | 0 | Ryan details hardware bottlenecks around specialized GPUs and the operational necessity of managing community trust regarding public training data. He firmly emphasizes that Copilot must augment rather than replace thinking engineers. | |
| Strategic Portfolio Management and Capacity Allocation | 4 | 6 | 0 | 0 | Ryan provides a clear breakdown of portfolio resource allocation: 5-10% on moonshots, 25-30% on operations, and 60% on incremental enhancements. Lenny appreciates the concrete operational ratios. | |
| Lightning Round: Book Recommendations, Media, and Key Influences | 6 | 3 | 0 | 0 | During the lightning round, Ryan mentions the film Arrival and Lenny displays strong knowledge by connecting it to Ted Chiang's sci-fi short stories. Ryan also shares his favorite PM interview question and praises Copilot creator Uga D'Amour. |
Statements from this episode (21)
Salva: GitHub Copilot is powered by OpenAI's Codex, derived from GPT-3
“Copilot is essentially that magnified by many lines of code. It is multi-line autocomplete that is
Fundamentally powered by an AI model called Codex, which is a derivative of another kind of one that you might be familiar with, GPT-III.”
Salva: Copilot sustains developer flow by eliminating Stack Overflow and doc lookups
“Copilot.
Helps developers stay in the flow by bringing all of that information into the editor, preventing them from having to go check out documentation or watch a tutorial or go to Stack Overflow and, you know, either find an answer or worse, have to ask a q…”
Salva: AI will be core to the developer toolchain by 2026
“Students build not only the kind of the tool, the software that the business needs, and then get to put that on their resume and their, you know, application for college and university, but they also get to learn by kind of using the tools that likely are goin…”
Salva: Copilot helps developers build mental maps of unfamiliar codebases
“One of the, you know, really magical pieces of Copilot here is that, that AI is collecting context of the application that you're going into. And so it can help you build that mental map and learn the code base, even if it's a language that you're already fami…”
Code is better suited for AI models than natural languages
“In fact, They're kind of nice from an AI perspective because they're relatively constrained in terms of their semantics, right? The number of words, and I put that in scare quotes, as it were, that can be expressed in Python, for example, is much smaller than …”
OpenAI scraping GitHub repositories initially appeared to be a DoS attack
“And at the time, one of the teams that I was responsible for was GitHub's infrastructure team. You know, the team responsible for our data centers, our reliability, our uptime. And we noticed one day that we were getting hammered. I mean, absolutely hammered w…”
GitHub provided its Arctic Code Vault to OpenAI for model training
“And so what we did is just like the year before that, we had actually created a snapshot. Of GitHub's public code for what we call the Arctic code vault, right?... Well, we took that same data snapshot and we brought it to our friends over at OpenAI to see lik…”
Salva: AI code completion latency sweet spot is roughly 200 milliseconds
“It seems like right now it's around 200 milliseconds. So depending upon where you're in the world, your latency can go up or down a little bit from there, but it seems like the sweet spot is somewhere around 200 milliseconds.”
Salva: Copilot originated inside GitHub Next's exploratory R&D team
“The original team that was working on co-pilot At GitHub was, you know, it's a team that we call GitHub Next. And essentially their job is to work on second and third horizon projects. What some folks might call moonshots, right? Things that we never really ex…”
Salva: Innovation horizons measure ambiguity and confidence, not calendar dates
“Not necessarily a concrete definition. Like for me, I usually ballpark it as first horizon is the next year, second horizon, the next three years, third horizon, next five years. But we generally think of it more as like a measure of ambiguity and confidence l…”
Salva: Early R&D teams should not be held to revenue or production standards
“Don't expect any, anything out of them that is going to turn into a moneymaker or something that is going to be beholden to fundamentals around security, privacy, uptime, you know, accessibility, all that groupy kind of stuff upfront. They need space to create…”
Salva: GitHub embedded Copilot researchers into production squads for knowledge transfer
“We made an intentional decision to take some of the researchers who were in the next team and for a finite period of time, move them over to create a new EPD squad, right? We want them to be researchers, but we need to do knowledge transfer, and we need to act…”
Salva: Production teams must own the product roadmap, not R&D teams
“It's critical that the team who is taking over from the R&D shop feels like they have control over their own future.
you can't
Really delegate roadmap to an R&D team.
The team who's responsible for maintaining the product, for building the product, who has th…”
GitHub Copilot launched without content filters before adopting basic word blocklists
“When we first started out, we didn't really have any filter on Copilot whatsoever in the very, very, very early days. And then eventually we're like, okay. It needs to be slightly more controlled experience. We need to edit out, you know, some of the most egre…”
Salva: GitHub Copilot generates 40% of code for Python developers
“And so just to put a fine point on that stat, it is 40% is specifically for Python developers. It candidly, it varies depending upon the language.”
GitHub Copilot requires rare GPUs with limited global supply
“Co-pilot for both training and operating the models requires some very rare and unique GPUs that there's not a lot of global supply of.”
Scaling Copilot required scaling product management more than engineering
“And so the amount of kind of give and take between developers and us as a product team, Has really required us to scale up more of the product team than it has the engineering team.”
Salva: GitHub Copilot Will Never Replace Developers
“Copilot is not a replacement for a developer. It will never be.”
Salva: Product teams should allocate 5-10% of capacity to moonshots
“As a general rule, as a general principle, I certainly try to make sure that we're always reserving some capacity for bold, audacious experimental research projects. You can kind of think of those, like those really uncertain bets as being five to 10% of the t…”
Salva evaluates PM candidates by having them teach something in one minute
“So I ask them to teach me something new in one minute. And so I'll usually like, I'll pull up my phone and I'll like start the timer. I'll give them a second to think about it and start the timer. And they're graded on three different criteria. So one is compl…”