Kilpatrick: Google hot-swaps experimental models without allowing developers to pin old versions
Logan Kilpatrick · Gemini 2.0 Flash and Flash Thinking: the new SOTA models for the agentic era · Feb 28, 2025 · at 6:21
Logan Kilpatrick (Product Lead at Google AI Studio) explains how Google manages experimental Gemini model releases and why developers should not deploy them in production environments.
0:00 / 0:22exact quote · 22.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Framing it as experimental is, is really to try to signal that like one, you shouldn't use this thing in production. like it's You know, rate limits are usually the limitation to stop people from doing this, but to like the model might change. And in certain cases, like we come out with a new rev of the model, we literally just like hot swap the old one with a new version of it. Like there's no opportunity to stay on the old one.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Logan Kilpatrick
Kilpatrick: Generative UI will be the killer use case for diffusion LLMs
“But I do think that's going to be the killer use case will be like this generative UI experience that doesn't exist today because the models just take too long to generate tokens.”
Kilpatrick: Google's strategy is building Gemini as one single unified model
“Like we're here to make one model and like that model is Gemini.
And like, I think you do need to just trust this point, like to make the capabilities work in some cases, like you do need to have these forks that like go off and make that capability and harde…”
Kilpatrick: Gemini's SOTA video performance resulted from reasoning, not video engineering
“With reasoning is a great example of this where like multimodal with video understanding ended up like having this huge, like it's having this beautiful moment.
The model is like soda out of the box because of all the reasoning capabilities that were baked in…”
Next billions of AI users will onboard via SMS and phone
“I don't think those people are going to come in through, you know, some front end website somewhere. Like those people are going to come in through audio from a telephone to texting to email. Like that's just the most obvious outcome.”
Embeddings will not scale for cross-session memory in real-time AI systems
“I don't think that like embeddings are going to be able to scale to, I think they work well for some of this as like kind of the MVP version of the experience, but I think you're going to need a different experience and it's Probably something like really smar…”
Kilpatrick: Live AI APIs create severe vendor lock-in due to bespoke infra
“I think if you look at a lot of the live API infrastructure right now, like you really do need to commit that you're like gonna, you know, there's, it's not easily interoperable between different model providers. Like everyone's infrastructure is all bespoke a…”