Kilpatrick: Reasoning will solve multi-item retrieval in long context
Logan Kilpatrick · Gemini 2.0 Flash and Flash Thinking: the new SOTA models for the agentic era · Feb 28, 2025 · at 11:43
Logan Kilpatrick discusses how reasoning models will overcome existing multi-needle retrieval limitations in Gemini's million-token context windows.
0:00 / 0:46exact quote · 46.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And like, it feels like, again, like back to this, the thread around these capabilities, like it feels like long context with reasoning is like finally going to be that thing where like, it actually just like blows the lid off of it. And like, it makes the use case really come to life because like the challenges historically has been long context. If you read the original paper that we put out with long context landed, it was, you know, the models are pretty good at like, needle in a haystack like tasks of like attending to, you know, one or two things or a small number of things in the million to two million token context window. But as you scale that up to like, you know, find me a hundred things in this two hour video, it becomes really hard for the model to do that. And I think reasoning is the moment where this starts to change, where the model's actually able to go and look over that number of things in a single context window, which is really cool.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Logan Kilpatrick
Kilpatrick: Generative UI will be the killer use case for diffusion LLMs
“But I do think that's going to be the killer use case will be like this generative UI experience that doesn't exist today because the models just take too long to generate tokens.”
Kilpatrick: Google's strategy is building Gemini as one single unified model
“Like we're here to make one model and like that model is Gemini.
And like, I think you do need to just trust this point, like to make the capabilities work in some cases, like you do need to have these forks that like go off and make that capability and harde…”
Kilpatrick: Gemini's SOTA video performance resulted from reasoning, not video engineering
“With reasoning is a great example of this where like multimodal with video understanding ended up like having this huge, like it's having this beautiful moment.
The model is like soda out of the box because of all the reasoning capabilities that were baked in…”
Next billions of AI users will onboard via SMS and phone
“I don't think those people are going to come in through, you know, some front end website somewhere. Like those people are going to come in through audio from a telephone to texting to email. Like that's just the most obvious outcome.”
Embeddings will not scale for cross-session memory in real-time AI systems
“I don't think that like embeddings are going to be able to scale to, I think they work well for some of this as like kind of the MVP version of the experience, but I think you're going to need a different experience and it's Probably something like really smar…”
Kilpatrick: Live AI APIs create severe vendor lock-in due to bespoke infra
“I think if you look at a lot of the live API infrastructure right now, like you really do need to commit that you're like gonna, you know, there's, it's not easily interoperable between different model providers. Like everyone's infrastructure is all bespoke a…”