Caching
topic on 1 show · 2 statements across 2 episodes
2 statements about Caching, every show
Swix: Proprietary LLM caching creates vendor lock-in
“I feel like this is definitely a form of lock-in because you ideally want to be able to run prompts across multiple providers and all that. And yeah, caching is a hard problem. Like, I think ultimately, like, you control your destiny if you can run your own op…”
Embeddings will not scale for cross-session memory in real-time AI systems
“I don't think that like embeddings are going to be able to scale to, I think they work well for some of this as like kind of the MVP version of the experience, but I think you're going to need a different experience and it's Probably something like really smar…”