context window
also referred to as: context windows
4 statements across 3 episodes · 0 bullish · 1 bearish · 3 people on the record · first statement Jun 19, 2025 by Andrej Karpathy · across every show →
Everything said about context window, oldest first
Jun 19, 2025
Continuous knowledge consolidation remains unsolved in LLM research and development
“LLMs don't natively do this, and this is not something that has really been solved in the R&D of LLMs, I think. And so context windows are really kind of like working memory, and you have to sort of program the working memory quite directly because they don't …”
Feb 6, 2026 negative
French-Owen: Context window limits remain the primary bottleneck for coding agents
“I mean, I still think context window is like probably the number one limit. Like if you look at cloud code executing, it's delegating to all these different context windows. At the end of the day, when each one comes back, it's like getting some sort of summar…”
Apr 29, 2026 neutral
Apr 29, 2026 neutral
Hassabis: Information retrieval costs remain non-trivial even in massive context windows
“Even though we're working on machines, not biological brains, and so potentially you could have, you know, millions or tens of millions size context window or memory, and it can be perfect. There's still a cost to looking it up and finding the right thing that…”