Context Compaction
topic on 2 shows · 3 statements across 3 episodes
3 statements about Context Compaction, every show
Biderman: Harmless enterprise queries on frontier models cost thousands of dollars
“And now you can solve these tasks with frontier models and compaction. And when you ask them to do so, they will consume thousands of dollars for queries that we think are harmless. That every employee in the company would be able to answer.”
Chase: AI agent context compaction must retain recent messages for flow
“You want to keep around like the last N messages, let's say the last like 10 or so messages, because if you compact everything, it actually like throws it off completely.”
Nelle: Sub-agents provide an architectural boundary to compress agent context cleanly
“And then the other reason to use sub-agents is we want, ah, contexts to be sort of summarized, reduced down at a sub-agent level.
That's a really neat boundary at which to compress that rollout and testing into a final message that that agent writes that then…”