Opinion
Shah: Claude Code avoids indexing because vendors profit from high token usage
“They could index the code and cursor does, but plot code does not. And I believe this is also because they are not incentivized to utilize less tokens by the end user, because they're just offsetting the cost of indexing, et cetera, to tokens and time that the…”
Insight
Shah: OpenClaw memory fails because LLMs frequently skip search tool calls
“The way OpenClaw has with QMD or with whatever memory plugin you use, it inherently relies on tools to search through these memory.md files that it prepares. So, you know, like, what did I decide about the API? Then agent will decide to search, and sometimes i…”
Assertion Supported
Shah: OpenClaw's 15-message replay fails prompt caching and costs 10x more
“The way OpenClaw does it is it essentially sends back the last 15 messages in the conversation and it essentially uses that back and forth. And I mean, the approach itself is not ideal because you will, like, you are not doing any, like, you're not utilizing a…”
Opinion
Shah: Most vector databases become too slow or expensive at scale
“So, so, better databases, like, at that time, and even now, except a few, like, TurboPuffer and Chroma, Were not really scalable for the scale I was getting to where they were either getting too slow or too expensive to run.”
Opinion
Shah: Triplet-Based Knowledge Graphs Degrade AI Memory Performance
“And we think that triplets actually Lead to worse performance because you have to traverse them a lot to get to any information.”
Assertion Supported
Shah: Supermemory outperformed Claude Code and OpenClaw benchmarks by almost 50%
“So the Claude code one performed the worst, and OpenClaw slightly more than that, and SuperMemory is the highest, and you can see that, you know, it's like a pretty significant difference, like almost 50%.”
Opinion
Shah: LongMemEval is good but reward-hacks memory extraction volume
“LongMemival is a very good benchmark because it calculates, it tests for the right things. It tests for the right things, the data set is great, everything is really good in LongMemival, except for the fact that you can win LongMemival by remembering as many t…”
Opinion
Shah: LoCoMo benchmark is flawed because models pass via context dumping
“Locomo is, like, really bad. And people actually, it's not just me, like, pretty much everyone knows that Locomo is not the right thing to judge. So the reason is, like, Locomo is essentially testing for retrieval capacity. And it's not testing for the real me…”
Insight
Shah: Pure retrieval AI memory fails on non-literal questions
“Traditionally you have this retrieval thing that happens. So you get a question, you retrieve something, and an answer is generated based on that. But we think that it will not work for most non-literal questions, like find the best monitor for me. If I've nev…”
Assertion Supported
Shah: No other memory provider offers hybrid memory and raw RAG fallback
“Essentially, you will return the memories first, and then if there's any raw chunks that match up, we also return those to make sure the agent knows just enough information to answer the question. And no other provider does this right now”
Disclosure
Shah: Supermemory infrastructure costs only two cents per million tokens
“For us, we have optimized infrastructure down to, it costs us two cents for a million tokens, because we have our own model, our own database, our own everything.”
Insight
Shah: True LLM User Understanding Requires Four Memory Capabilities
“So, to make an LLM truly good at understanding a user, you have to handle four things. One is knowledge updates, so you have to invalidate, steal knowledge, and build on top of it, which is different from the storing vectors. You have to have some sort of temp…”
Assertion Supported
Shah: Supermemory dynamically injects under 2,000 tokens via deterministic hooks
“So we basically turned the tools based approach to hooks based approach that actually is dealing from the super memory graph, which keeps the content fresh. So it handles updates, et cetera. So now, you know, for every tool call or for every user message, a ho…”
Assertion Not checkable as stated
Shah: Supermemory was among 2024's fastest-growing open-source projects
“I open sourced SuperMemory during BuildSpace and it Like the project absolutely blew up. It got like, you know, it was one of the biggest projects, like fastest growing projects of 20, 24.”
Disclosure
Shah: Supermemory Pivoted from Consumer App to Memory Infrastructure
“And I was like, this consumer app is probably not useful anymore. This core infrastructure is more useful now. And that is our core business now. And that's what we offer.”
