Feb 8, 2024 · 1h 8m · lennys-podcast
Inside OpenAI | Logan Kilpatrick (head of developer relations)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this interview, Lenny Rachitsky speaks with Logan Kilpatrick, Head of Developer Relations at OpenAI, about the company's high-agency internal culture, prompt engineering best practices, and the evolution of custom GPTs. Logan shares strategic advice for startup founders building defensible AI applications beyond simple chat interfaces, alongside insights into upcoming platform developments like third-generation embeddings and autonomous agents.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Lenny holds 33.1% of the talking time here. How this is scored →
Lenny as informed peer 4.5 Guest teaching 4.6 Guest disagreement 1.4 Lenny pushing back 1.4
speaking balance: gold is Lenny, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 49:20 Logan dismisses hyperbolic GPT-5 expectations
Logan firmly pushes back against the public expectation that GPT-5 will solve every problem, characterizing it as a normal and predictable productivity tool.
Hardest push from Lenny ▶ 38:15 Lenny presses on OpenAI's lack of standard OKRsLenny pushes on how OpenAI aligns its massive team on product roadmaps without using the standard OKR frameworks prevalent in the tech industry.
Biggest teaching moment ▶ 22:57 Logan breaks down the mechanics of prompt contextLogan directly demonstrates why Lenny's custom Tyler Cowen prompt failed to yield stellar questions due to a lack of explicit contextual data provided to the model.
Lenny holds their own ▶ 15:58 Lenny demonstrates granular knowledge of enterprise GPT workflowsLenny proves his practical expertise by detailing custom GPT configurations built at Chime for marketing automation and statistical experiment readouts.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Lenny as informed peer | Guest teaching | Guest disagreement | Lenny pushing back | Why |
|---|---|---|---|---|---|---|
| Inside OpenAI During the Board and Leadership Crisis | 2 | 3 | 1 | 1 | Lenny opens by inquiring about the November 2023 board crisis at OpenAI. Logan provides an insider narrative on company transparency and the swift return to normal operations. | |
| Novel AI Interfaces and Strategic Defensibility for Startups | 4 | 5 | 2 | 2 | Lenny draws on his product management background to ask about startup defensibility against OpenAI. Logan clearly differentiates general assistant products from defensible vertical specialized applications. | |
| Driving Internal Efficiency and Specialized Enterprise GPTs | 6 | 4 | 1 | 1 | Lenny provides concrete examples from Chime on internal GPT workflows for ad copy and experiment analysis. Logan elaborates on academic findings regarding consultant efficiency gains and internal planning GPTs. | |
| Deconstructing Prompt Engineering and Optimizing AI Context | 5 | 7 | 2 | 2 | Lenny tests a specific Tyler Cowen interview prompt on Logan. Logan breaks down why models struggle without explicit background context and explains human-like communication mechanics. | |
| The Custom GPT Ecosystem, Tooling, and Platform Evolution | 5 | 4 | 1 | 1 | Lenny checks the GPT store live while discussing leading applications like Canva and Runway's Universal Primer. Logan outlines the democratization of tool calling and upcoming monetization. | |
| Sponsor Message: Arcade Interactive Product Demos | 4 | 6 | 1 | 2 | Lenny contrasts OpenAI with slower institutions like Apple and NASA, probing planning rituals. Logan explains high-agency hiring and the mathematical necessity of keeping AI research teams small during GPU shortages. | |
| Future Frontiers: Multimodality, Autonomous Agents, and Demystifying GPT-5 | 4 | 7 | 3 | 2 | Lenny asks how builders should anticipate GPT-5 capabilities. Logan firmly pushes back on overhyped expectations, clarifying that GPT-5 will follow predictable scaling laws rather than perform magical feats. | |
| Enterprise Tier Features and the Launch of Generation 3 Embeddings | 6 | 6 | 1 | 1 | Lenny brings up enterprise adoption and his custom search tool LennyBot.com. Logan explains the underlying mechanics of Generation 3 vector embeddings and their cost reductions. | |
| Product Strategies for Founders and Expanding Beyond Chat | 5 | 4 | 1 | 1 | Lenny shares his family's experience using Visual Electric to generate art for their television. Logan emphasizes that moving beyond basic chat interfaces into purpose-built canvases offers founders huge advantages. | |
| Lightning Round: Books, Sleep Science, Life Mottos, and AI Humor | 4 | 2 | 1 | 1 | Lenny and Logan exchange recommendations during the lightning round, comparing sleep mask brands and discussing Khan Academy's educational AI philosophy. | |
| Submitting Actionable Bug Reports, Daily AI Usage, and Practical Tips | 4 | 3 | 1 | 1 | Lenny offers practical advice on expensing AI subscriptions and incorporating daily use. Logan asks the developer community for reproducible bug reports with shared chat links. |
Statements from this episode (16)
Thanksgiving 2023 was OpenAI's first planned company-wide break after launching ChatGPT
“OpenAI had been pushing for a really long time since ChatGPT came out and that was supposed to be like the first, one of the first weeks that like the whole company had like taken time away to like actually reset and have a break.”
Another company would have built frontier AI if OpenAI collapsed
“Sort of on the world scale, like, you know, somebody else will build a model if open AI disappeared and continue towards this progress of general intelligence.”
Kilpatrick: A board crisis in 5-10 years would be far more damaging
“Fast forward like five or 10 years of something like this would have happened. And we sort of hadn't gone through the hopeful upcoming like word transformation and sort of all those changes that are going to happen. I think it would have been a little bit or p…”
OpenAI models will likely never match vertical tools like Harvey
“Our models are probably never going to be as capable as some of the things that Harvey's doing because like our goal and our mission is really to solve this like very general use case.”
Startups building general-purpose AI agents will directly compete with OpenAI
“I talked to a lot of developers who are building like, you know, just general purpose assistance and like general purpose agents and stuff like that. And I think it's cool and it's a good idea. I think like the challenge for them is like, they are going to end…”
OpenAI will not build highly verticalized AI products like sales agents
“Versus like, we're not going to launch like some of these like very verticalized products. Like we're not going to launch like an AI sales agent. Like that's just not what we're building towards.”
AI unlocks at least a 50% efficiency improvement in software engineering
“I think engineering is actually like one of the highest leverage things that you could be using AI to do today. And like really unlocking like, Probably on the order of at least a 50% improvement, especially for some of the like lower hanging fruit software en…”
Future LLMs will automatically expand short user prompts into detailed descriptions
“This will happen with text models. You can imagine a world where you go into ChatGPT and you say, write me a blog post about AI. It automatically will go and be like, let me generate a much higher fidelity description of what this person really wants, which is…”
OpenAI plans to launch GPT Store monetization in Q1 2024
“Like I think monetization when it comes to the store later this quarter, I think is going to be extremely exciting. Like when people can get paid based on who's using their GPTs, that's going to be a huge unlock and like open a lot of people's eyes to the oppo…”
OpenAI internal teams do not use OKRs
“I actually don't think open AI is like a big OKR company. Like, I don't think teams do OKRs today”
OpenAI views revenue merely as a proxy to acquire more compute
“Even if revenue is a goal, it's like revenue is not actually the goal. Revenue is a proxy for getting more compute, which is then like actually what helps us get towards getting more GPUs so that we can, you know, train better models and like actually get to t…”
OpenAI deliberately keeps its core research team small
“The actual research team who like, you know, sort of see the most of the innovation that happens at OpenAI is intentionally small. They're not Like, you know, most of the growth that OpenAI has seen is around like our customer facing roles, our engineering rol…”
Adding AI researchers decreases group productivity when GPU capacity is constrained
“In a world where you're constrained by the amount of GPU capacity that you have as a researcher, which is the case for OpenAI researchers, but also researchers everywhere else, like Each new researcher that you add is actually, like, a net productivity loss fo…”
GPT-4 was OpenAI's first model with reliably predictable capabilities before training
“GPT-IV was the first model that we trained where we could reliably predict the capabilities of that model beforehand. Based on the amount of compute that we were going to put into it, and you could actually, we did like a scientific study to show like, hey, th…”
GPT-5 will normalize as a useful tool rather than changing everything
“GPT-V is like surely going to be extremely useful and like solve some whole new echelon of Problems. Hopefully they'll be faster. Hopefully it'll be better on all these ways, but like fundamentally the same problems that exist in the world are still going to b…”
AI products moving beyond standard chat interfaces will gain competitive edges
“You're going to have an edge on other people. If you're providing AI that's not accessible in a chat bot, like people are using a ton of chat and like, it's a really valuable service area. Like it's clearly valuable because people are using it, but I think pro…”