Dec 1, 2023 · 50m · lennys-podcast
The future of AI in software development | Inbal Shani (CPO of GitHub)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode, GitHub Chief Product Officer Inbal Shani explores how generative AI is transforming software engineering by elevating developers toward high-level systems architecture, establishing frictionless enterprise workflows, and redefining developer productivity.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Lenny holds 27.1% of the talking time here. How this is scored →
Lenny as informed peer 2.7 Guest teaching 5.2 Guest disagreement 1.4 Lenny pushing back 0.4
speaking balance: gold is Lenny, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 12:23 Decisive rejection of engineering downsizing
Inbal directly rejects the premise that AI efficiency should lead to cutting engineering headcount, forcefully stating that humans remain irreplaceable in the software lifecycle.
Hardest push from Lenny ▶ 37:55 Challenging innovation lab effectivenessLenny challenges the track record of dedicated innovation teams, explicitly citing high-profile failures like Facebook NPE and Google's incubator initiatives.
Biggest teaching moment ▶ 31:05 Demystifying LLMs and safety-critical systemsInbal draws on her aerospace and automotive background to correct the common assumption that general LLMs will swallow every domain, demonstrating why safety-critical industries require specialized, niche hybrid models.
Lenny holds their own ▶ 22:45 Lenny challenges conventional engineering metrics with DORALenny challenges standard output metrics like lines of code, citing specific engineering research frameworks like Nicole Forsgren's DORA framework to ground the discussion on developer productivity.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Lenny as informed peer | Guest teaching | Guest disagreement | Lenny pushing back | Why |
|---|---|---|---|---|---|---|
| What Is Overhyped and Underhyped in AI Software Engineering | 3 | 6 | 2 | 1 | Lenny opens by asking what is overhyped versus underhyped in AI development. Inbal reframes common industry rhetoric, arguing firmly that generative AI replacing human engineers is overhyped while automated comprehensive testing across systems is severely underhyped. | |
| The Evolving Role of Developers and Hardware Demands | 2 | 6 | 1 | 0 | Lenny asks about the 3-5 year trajectory of software development and Copilot. Inbal emphasizes her core principle that Copilot remains an assistant rather than autonomous pilot, educating on the architectural shift junior devs must make toward systems thinking and hardware optimization. | |
| Real-World Adoption Metrics and Impact of GitHub Copilot | 4 | 5 | 1 | 0 | Lenny cites Shopify's codebase statistics to highlight rapid adoption, prompting Inbal to reel off detailed internal GitHub metrics on organization adoption, code review acceleration, and developer retention. | |
| Protecting Developer Flow and Maximizing Team Happiness | 3 | 7 | 3 | 1 | When Lenny brings up whether efficiency gains mean companies will lay off 25% of their software engineers, Inbal immediately pushes back with a firm clarification that human engineers cannot be cut, breaking down developer time allocation to show how little time is actually spent coding. | |
| Dogfooding Culture and Internal Product Testing at GitHub | 2 | 5 | 1 | 0 | Lenny inquires how GitHub's product teams work differently given early access to cutting-edge AI. Inbal explains GitHub's deep dogfooding culture where non-technical teams like legal and finance also operate directly within GitHub repositories and PR workflows. | |
| Designing Frictionless AI Experiences for Software Engineers | 4 | 6 | 1 | 1 | Lenny and Inbal discuss product design and productivity metrics. Lenny brings up Nicole Forsgren's DORA framework, while Inbal explains why raw speed or lines of code are misleading metrics, advocating instead for measuring time-to-value and developer happiness. | |
| Sponsor: Help Bar by Chameleon | 1 | 4 | 2 | 1 | Following the sponsor break, Lenny asks about viral demos where ChatGPT generates complete apps from napkin sketches. Inbal clarifies that these tools function primarily as communication and alignment aids rather than end-to-end autonomous production systems. | |
| Programming Abstraction Layers and Retaining Joy in Coding | 3 | 5 | 1 | 0 | Lenny shares an anecdote about an engineer nostalgic for writing basic algorithms by hand. Inbal contextualizes AI tools against historical programming abstractions like C to Java to Python, explaining how engineers choose their preferred layer of abstraction. | |
| The Future of Hybrid AI and Domain-Specific Models | 3 | 6 | 2 | 0 | Lenny asks if monolithic LLMs are the permanent endgame or if new architectures will replace them. Inbal draws on her aerospace and robotics background to argue that generalized LLMs will give way to a hybrid model involving specialized, safety-critical niche AI. | |
| Fostering Organic Innovation and Experimentation at GitHub | 3 | 5 | 1 | 1 | Lenny probes how GitHub structures teams to reliably capture major breakthrough products like Copilot. Inbal explains why rigid time-allotment structures fail for organic innovation, describing instead customer-driven research incubation and fail-forward culture. | |
| Inside GitHub Next: Bridging Applied Research and Production | 3 | 5 | 1 | 1 | Lenny compares GitHub Next to corporate innovation labs like Facebook NPE and Google's incubators that struggled to deliver. Inbal explains GitHub Next's deliberate bridge between applied research and immediate production feasibility from day zero. | |
| Career Progression and Skillsets for Chief Product Officers | 2 | 5 | 1 | 0 | Lenny asks how Inbal cultivated the competencies required to become CPO at a major tech company. Inbal breaks down the shift from standard product management to holistic organizational influence, systems thinking, and business acumen. | |
| Failure Corner: Learning to Navigate Change Management at TomTom | 3 | 5 | 2 | 0 | Lenny introduces the Failure Corner segment. Inbal renames it to Learning Corner, reflecting on her early management tenure at TomTom where pushing aggressive change without cultural context and stakeholder buy-in backfired. | |
| Celebrating GitHub Milestones and Looking to the Future | 2 | 3 | 1 | 0 | Lenny conducts the lightning round, asking about recommended books, favorite shows, and go-to interview questions, concluding with Inbal's personal leadership motto on embracing risk. |
Statements from this episode (16)
GitHub CPO: 92% of software developers already use AI tools
“We know that something like 92% of developers are already using AI tools.”
GitHub CPO: Generative AI will not replace developers in the near future
“Some of the things that are overhyped is that generative AI will replace humans. I don't see that happening in the near future. The way I think about it, you always need that human in the loop because AI cannot replace innovation, right? That creative spark, t…”
GitHub CPO: AI lets junior developers focus on systems architecture earlier
“The junior developers, when they start, usually we expect them to be able to write simple code. But if now there is an AI assistant that is helping them writing code, they can spend more time from the get-go understanding the system, understanding the environm…”
GitHub Copilot reaches over 37,000 organizations and 1.5 million developers
“We have over 37,000 organization and more than 1.5 million developers that are using Copilot.”
GitHub surveys find developers write code 55% faster using Copilot
“And they're writing codes based on our surveys, 55% faster.”
GitHub CPO: Accenture retained 88% of code suggested by GitHub Copilot
“I think Accenture is a recent one that we got where they had 88% of suggested code was retained.”
GitHub CPO: Companies cannot cut engineering headcount due to AI tools
“You cannot cut your people.
You have to have a human in the loop.
Copilot is a copilot, is not a pilot.”
GitHub CPO: Most developers spend under 25% of their time coding
“So most developers spend less than 25, some say less than 20% of their time writing code.”
GitHub's finance, legal, and HR teams run their operations on GitHub
“So for example, our finance team is using discussions and posts and PRs and reposts to communicate our AR numbers and so on and so forth. We have our legal team using, we have our HR team. If they like it or not, it's a different question, but the idea is that…”
GitHub CPO: Developers reject AI tools requiring extra steps or waiting
“If it's an extra tool, and if you need to ask for it, and if you need to ask for it, or if you need to wait for it, then developers will not adopt it.”
GitHub began prototyping Copilot with OpenAI's GPT models in 2020
“In 2020, we had a group of GitHub engineers that were working with the design team and we opened AI GPT to really figure out how we're going to build that and how it can work to help developers to do their job more efficiently, more productively, improve their…”
GitHub CPO: Raw coding speed fails as a developer productivity metric
“The most easiest one is time, but time is, it's funny what I'm going to say, but time is not quantifiable. As a success metrics, because you can write really bad code really fast.”
GitHub CPO: Sketch-to-code AI is for collaboration, not building production software
“I'm thinking about that as a better collaboration tool versus a production tool. Because right now, a lot of the time where we see challenges in articulating ideas is the communication. It's the clarity of thoughts. So if we can leverage AI to improve collabor…”
GitHub CPO: Developers should treat AI tools as technical abstraction layers
“There are areas that I will still go and write in C even today. I don't trust someone inverting metrics for me in an efficient way if it needs to run on a very small CPU. So I'll do that myself. On the other hand, there is going to be elements that I'm going t…”
GitHub CPO predicts AI will evolve into hybrid, multi-model systems
“I might be wrong, but I think that eventually we will find ourselves in the world of hybrid models and multi-models where there will be several LLM models coming together because each one of them will have their own benefit. And then there's going to be a leas…”
GitHub CPO: Corporate research fails when it becomes overly academic or tactical
“What happens in teams that are not successful, at least from what I've seen, one, that the team is becoming basically another university. They write papers, but nothing is coming out of that. So nothing Find its walls to production. So if you don't introduce t…”