Park: LLMs struggle with attention drop in the middle of long prompts
June Park · Inside AI Town: What AI Can Teach Us About Being Human · Nov 6, 2023 · at 39:24
June Park, researcher and author of the Generative Agents paper, discusses the limitations of massive LLM context windows during an a16z panel on AI agents.
0:00 / 0:20exact quote · 20.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Larger context window does confuse models, right? So we, some of my colleagues are actually doing more rigorous studies on this, where You can have a really long prompt, but model really focuses on the first few lines and the last few lines, and whatever comes in between, its attention drops significantly, right?”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from June Park
Park: Expanding LLM context windows cannot replace external agent memory
“And even if that limitation were to go away in the future, processing a lot of really long-term context window is really inefficient and also ineffective when you're trying to prompt these models for a really narrowly defined behavioral assets.”
Park: GPT-3 simulated COVID-19 discussions without prior pandemic training data
“We basically asked GPT-III to create a community that has to talk about COVID and vaccination, vaccination policy. And you would wonder, it shouldn't be able to do that in theory, because it doesn't know anything about COVID. It doesn't know anything about the…”
Park: Bank of England explores AI agent simulations for economic policy
“For instance, if you're, in fact, some of the places that I'm visiting now are More places like banks, like the Bank of England and so forth, where these places, they need to test their policies before they run roll out new comic policies, or many of my collea…”
Park: AI agents will progress first in soft-edge problem spaces
“My bet, it's a bit of a hot take, is my bet is in the early days of agent development, I think we'll see a lot of progress that's going to be made first in sort of the soft edge problem spaces.”