prompt caching
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jun 19, 2025 by Sander Schulhoff · across every show →
Everything said about prompt caching, oldest first
Jun 19, 2025
Schulhoff: Place task context at prompt beginning for caching and focus
“Usually I will put my additional information at the beginning of the prompt. And that is helpful for two reasons. One, it can get cached. So subsequent calls to the LM with that same context at the top of the prompt are cheaper because the model provider store…”