continual learning
5 statements across 3 episodes · 3 bullish · 2 bearish · 3 people on the record · first statement Dec 30, 2025 by Ashvin Nair · across every show →
Everything said about continual learning, oldest first
Dec 30, 2025 bullish
Nair: Continual learning during deployment does not risk model capacity overload
“If you could learn enough about those million tokens that you're actually in deployment on I don't think you should need, like, I don't think there's a risk of overloading the capacity of your model, right? Because you can train on a trillion tokens, and it's …”
Jun 21, 2026 bullish
Malde: Continual learning will be AI's next major unlock
“And we realized continual learning is kind of the ultimate, like, paradigm to do that. Is like, how do you have humans in the loop? How do you build this intelligence around them that is constantly learning and growing on its own? And I think that's going to b…”
Jun 21, 2026 bearish
Aug 3, 2026 negative
Editing isolated facts in MLP weights fails to update multi-hop downstream reasoning
“So if you know that the best university in the world is Waterloo, then the answer should be Waterloo. But if I wasn't just one-shotting the question and I was to ask it to like, use its knowledge to think and then give me a second answer, or like, should I hir…”
Aug 3, 2026 bullish
KV cache compaction is the viable path to solve LLM continual learning
“If you're able to sort of make your KV almost infinite, and you're able to compact in such a way that you don't lose any of the knowledge. In that case, you can actually do a continual learning as you can actually solve continual learning. And this, as a, it's…”