Context Caching

topic on 1 show · 2 statements across 2 episodes

2 statements about Context Caching, every show

Martin: Context caching solves cost and latency, but not context rot

“I do think an important and subtle point here is that caching doesn't solve the long context problem. So it, of course, solves the problem of, like, latency and cost, but if you still have a 100,000 tokens in context whether it's cached or not, the LM is utili…”

Lance Martin Sep 11, 2025 ▶ 37:46 Context Engineering for Agents - Lance Martin, LangChain

LATENT SPACE Assertion Not checkable as stated

Huber: Context caching improves cost and speed but ignores context rot

“And yeah, they're using context caching and that certainly helps, but like their cost and speed, but like isn't helping the context raw problem at all.”

Jeff Huber Aug 19, 2025 ▶ 23:48 Long Live Context Engineering - with Jeff Huber of Chroma

← every entity, every show

Turn any episode into a week of clips.

This entire site, thousands of episodes across every show transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Context Caching

2 statements about Context Caching, every show

Turn any episode into a week of clips.