Apr 25, 2025 · 30m · tbpn
Google's AI Comeback in Their Own Words - Logan Kilpatrick
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Google's Logan Kilpatrick discusses the rapid acceleration of generative AI, highlighting how Google's full-stack infrastructure, multimodal model breakthroughs like Gemini 2.5 Pro, and developer-first tooling are transforming user interfaces and application economics.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The hosts as informed peer 5.2 Guest teaching 4.4 Guest disagreement 1.9 The hosts pushing back 1.8
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 1:49 Logan directly rejects the incremental progress narrative
Logan openly challenges John's claim that post-GPT-4 developments have been mostly incremental, asserting that multimodal breakthroughs are vastly underappreciated.
Hardest push from the hosts ▶ 1:14 John questions whether recent AI improvements are truly breakthroughJohn challenges the prevailing acceleration narrative by arguing that model releases after GPT-4 have felt more like incremental optimizations rather than major step functions.
Biggest teaching moment ▶ 20:09 Logan reframes prompting as a software bugWhen Jordi asks how to teach consumers better prompt habits, Logan firmly reframes the question, arguing that demanding complex prompts from users is a fundamental failure of AI product design.
The host holds their own ▶ 5:11 John synthesizes market strategy and architectural trade-offsJohn demonstrates sharp market analysis by citing Ben Thompson's strategy framework and exploring the operational tensions between API infrastructure and consumer apps.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Pacing the AI Acceleration and Model Release Cadence | 5 | 5 | 4 | 4 | John challenges whether recent AI progress represents genuine order-of-magnitude breakthroughs or merely incremental product improvements since GPT-4. Logan explicitly pushes back, explaining that human expectations have inflated and multimodal capabilities represent massive leaps. | |
| Pushing the Pareto Frontier with Full-Stack Hardware and Model Efficiencies | 6 | 4 | 2 | 2 | The hosts demonstrate strong industry knowledge referencing Ben Thompson's thesis on OpenAI's enterprise-consumer tension and Google's full-stack advantage down to silicon. Logan articulates a four-factor economic framework benefiting builders. | |
| Navigating AI Evaluation Challenges and the Rise of Vibe Evals | 4 | 5 | 1 | 1 | Jordi asks about the disconnect between standard benchmarks and real-world user experience. Logan educates the hosts on the inherent difficulty of evaluation, comparing model evals to corporate performance reviews and explaining the necessity of vibe evals like LMSYS. | |
| Foundational Research Horizons: Compounding Gains from Pre-Training to Reasoning | 6 | 4 | 2 | 2 | John demonstrates solid technical domain knowledge regarding the data wall, RL scaling, and program synthesis. Logan clarifies that pre-training improvements act as an exponential capability multiplier for downstream reasoning models. | |
| Low-Level Inference Optimization and Scaling Gemini Under High Demand | 6 | 3 | 1 | 1 | John contextualizes low-level optimization by citing Google's historical V8 JavaScript engine development and recent DeepSeek inference innovations. Logan confirms that meeting massive inference demand for 2.5 Pro requires round-the-clock systems engineering. | |
| Fostering Internal Innovation: Google Labs, NotebookLM, and AI Studio | 5 | 4 | 1 | 1 | John asks about the viability of Google's 20% time culture and internal product cannibalization using Gmail's creation as an analogy. Logan explains how Google Labs under Josh Woodward incubates breakout products like NotebookLM and AI Studio. | |
| Designing Seamless AI Interfaces: Moving Beyond the Burden of Prompting | 5 | 6 | 3 | 2 | Jordi asks how to train consumers to prompt better, but Logan forcefully rejects the premise, arguing that requiring user prompt engineering is a product flaw rather than a consumer education problem. | |
| Inside Logan's Daily Workflow: Gemini App vs AI Studio Screen Sharing | 5 | 5 | 2 | 2 | John asks about Logan's personal day-to-day workflow across tools and side projects. Logan distinguishes between AI Studio's raw developer playground and the Gemini consumer app, emphasizing live screen sharing as a superior context-gathering mechanism. | |
| Exploring AI Hardware and Evaluating AI-Assisted Engineering Talent | 5 | 4 | 1 | 1 | The hosts bring up specialized AI hardware and viral controversies around candidates cheating on LeetCode technical interviews using AI. Logan highlights that modern developer evaluations must assess candidates' AI-assisted output and tool fluency. |
Statements from this episode (19)
Kilpatrick: AI leaps go underappreciated without consumer product experiences
“It's actually hard to appreciate some of those moments if there isn't a product experience that brings it to life. And I actually think that's been a lot of the gap.”
Kilpatrick: AI models now outperform humans on most vision tasks
“If you look at like multimodal, like the fact that the models can like with better, better than just from a multimodal input perspective, better than humans are at like most vision tasks, like The number of products and like things that that unlocks is like tr…”
Kilpatrick: Google's Full-Stack Control Extends From Silicon to Model Delivery
“Google controls. From a product perspective, like all the way to how the models are delivered to how the models are trained down to the silicon. So like you can make decisions assuming a bunch of those things are going to be true, which is like a lot of folks …”
Kilpatrick: Pushing the AI Pareto Frontier Uniquely Increases Builder Margins
“And like the margin for people building AI products actually goes up every, like the farther you push the Pareto frontier, the more money builders get to make, which is like such a interesting and like unique thing about this AI moment that I actually don't th…”
Kilpatrick: ChatGPT's consumer success was driven by its API business
“I think part of my core worldview is a lot of the reason that ChatGPT has been as successful as it has
Is because there was an API business built around it.”
Kilpatrick: AI inference costs dropped 99% over two years
“Cost of AI down 99% over the last two years.”
Kilpatrick: Chatbot Arena captures user vibes rather than evaluating scientific truth
“Like, Ella Marina is basically capturing vibes. It's like, how do people feel about this thing? It's not scientific. They're not, like, not actually evaling that the model is saying things that are true. It's like, how do humans feel about this response?”
Kilpatrick: Gemini 2.5 Pro Relied on Pre-Training Innovations, Not Just RL
“But I think if you look at like a, an example of this in practice, like 2.5 pro is actually an example where it wasn't just like RL scaling that made that model better. Yes, RL was part of the story, but like there was also a bunch of pre-training innovation a…”
Kilpatrick: Pre-Training Isn't Dead; Gains Multiply Through Post-Training and RL
“And this is why, like, I don't subscribe to the, like pre-training is, you know, dead and all that stuff, because the more work that you can do at the pre-training level, those capabilities, as you do post-training and as you give the models RL capability, it'…”
Kilpatrick: Large High-Demand AI Models Require Massive Inference Investments
“Especially with larger models and especially with models that have lots of demand, like there's no world where you can get away with not putting a large order of magnitude of investment into inference.”
Kilpatrick: Heavy Gemini 2.5 Pro Demand Forces Continuous Inference Optimization
“Our team who's Like actually working around the clock right now to make it so that people can keep scaling with 2.5 pro because there's so much demand and we're having to like, you know, as a an artifact of the constraints that we're under, like we're having t…”
Kilpatrick: Multimodal Foundation Models Match or Beat Domain-Specific Vision Models
“Relative to today where you can literally just write a prompt and send images or videos to the model and have it do those tasks like with basically, you know, near or better accuracy than you would get from domain specific models is absolutely fascinating.”
Kilpatrick: Real-Time Audio and Video Is Next Major AI UX Iteration
“I don't think we've seen like across other product services, people actually invest in like real-time audio and real-time video and image stuff. And I think that's like the next iteration of the UX of how people are going to interact with AI models.”
Kilpatrick: Burdening AI users with prompt crafting is a product bug
“Yeah, I actually, my personal take on this is I think this is a bug of this current AI moment where like the, if you look at like what is the ideal case is the models and the products like pull out what they need to from the user in order to create value for t…”
Kilpatrick refuses to build AI products that require changing user behavior
“And this is maybe too like absolutist of like a product perspective, but like, I will not build a product that like we have to try to convince a consumer to change their behavior.”
Kilpatrick: In 10 years, AI interfaces will look eerily similar to texting
“And I actually think if we've, if we fast forward like. 10 years, I do think there's going to be a lot of those experiences which like look eerily similar to the way that they do today, because it's just like so ingrained in like human culture, like how, like …”
Kilpatrick: Screen sharing eliminates the manual context-gathering friction in AI UX
“Right now the product experience of using AI is I need to go and find all the context that's relevant for the model. And get it into this text box somewhere or get it into this list of files somewhere. And like the beautiful thing about screen sharing is like …”