context window

“The idea that like a hundred K or even a million tokens is going to solve all our problems. And like, you just ask your questions and dump in all context naively is ridiculous to me. I think it's a really important area of product and research work.”

“I think in two years it'll be reasonably solved in terms of the size of context windows. It'll be usable. Like, I'd be happy to take that bet, but we'll see. Like, I think the context windows are going to get very large.”

“I'm more of the belief that it just opens up the set of trade-offs you can make between retrieval, more sophisticated retrieval and model reasoning by having a larger context window versus saying, like, we don't need any ability to work with a specific data se…”

“There are also things where it requires iterations. You need to like decide what to do, interact with the environment, get that back. So this whole idea of chaining and agents, I don't like, That's less around context windows and more around interacting with t…”

“And then it seems lucky that a lot of people will end up in the ten million plus range in the next a year or two or, you know, some reasonable timeframe ahead.”

“I mean, I think there's always this thought of that, like, you know, the models will have near infinite context windows and you can just give them everything and they can figure things out automatically. But I don't think, you know, like they're anywhere close…”

“We're not seeing that with AI models today. They kind of, they're born into a world for, and they exist for a very short context window, and then they just, like, disappear. And yeah, there are things that you can kind of do to, like, continue them, but it's v…”