Biderman: AI efficiency and frontier intelligence cannot be decoupled
Dan Biderman · The AI Memory Problem: Why Long Context Isn’t Enough — Dan Biderman, Engram Co-founder & CEO · Jul 13, 2026 · at 45:46
Dan Biderman, co-founder and CEO of Engram, discusses AI scaling and architecture while cooking on the Latent Space podcast.
0:00 / 0:31exact quote · 31.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The point for me is, the principle is, any kind of, like, efficiency and intelligence, they cannot really be decoupled. Sometimes people think if you're building something that's more efficient, that can save you dollars, therefore you're not in the premium category, you're in the, ah, you know, you're making the cheaper product. And that and intelligence, this is just you know, purely wrong, right? So, The more you can do with less, the more ambitious tasks you can solve longer term.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Dan Biderman
Biderman: AI-native companies will amass trillions of internal tokens within 18 months
“In 18 months, many companies would have maybe trillions of tokens, which of internal company data, proprietary data. I'm talking about like maybe trillions. It sounds exaggerated, but I don't think it's an impossibility if they're really AI native.”
Biderman: Hard engineering tasks will require test-time gradient updates
“We think that eventually part of the solution for very hard tasks in, in science and engineering and defense and all that stuff will involve some form of gradient based updates during during doing these long horizon tasks.”
Biderman: In 18 months, data scale will require weight-based learning
“Other parts of it are bets that in 18 months from now, the scale of the data will require the methods that we know from pre-training work.”
Biderman: PC hardware will soon run near-trillion-parameter models locally
“And in the long, long term, I do think these things will actually run on people's devices, and we're seeing right now the new hardware on personal computers is already, ah, you know, soon approaching the ability to run inference on close to trillion parameters…”
Biderman: Engram trains models to decide what to memorize vs keep in notes
“The way to work on it is to train models both, to train models to manage it themselves, and that's an active area for us. Have the model know, like, without any explicit supervision signal to determine this kind of stuff I can pull from my brain, and that kind…”
Biderman: Manually partitioning LLM memory vs retrieval becomes unmanageable whack-a-mole
“And now the thing is, if you start manually, heuristically saying this is in, this is out, then it becomes a whack-a-mole. Every, every person in every enterprise has different data, and you can really very easily pick and choose what goes in and what goes out…”