LLM context window
also referred to as: llm context windows
2 statements across 1 episodes · 0 bullish · 2 bearish · 1 people on the record · first statement Nov 6, 2023 by June Park · across every show →
Everything said about LLM context window, oldest first
Nov 6, 2023 negative
Park: LLMs struggle with attention drop in the middle of long prompts
“Larger context window does confuse models, right? So we, some of my colleagues are actually doing more rigorous studies on this, where You can have a really long prompt, but model really focuses on the first few lines and the last few lines, and whatever comes…”
Nov 6, 2023 negative
Park: Expanding LLM context windows cannot replace external agent memory
“And even if that limitation were to go away in the future, processing a lot of really long-term context window is really inefficient and also ineffective when you're trying to prompt these models for a really narrowly defined behavioral assets.”