Mamba
product on 5 shows · 7 statements across 5 episodes · said 40 times in 16 episodes since 2024
Latent Space 30 
No Priors 4 
the MAD Podcast 4 
the a16z Podcast 1 
20VC 1
Mentions by year, every show
tap a year for its mentions
Latent Space 30the MAD Podcast 4No Priors 420VC 1the a16z Podcast 1
2026 3 mentions in 3 episodes 1 per episode
2025 15 mentions in 5 episodes 3 per episode
- Building Jamba 3B: the tiny Hybrid Transformer State Space Reasoning Model - Barak Lenz, CTO of AI21
- How Zyphra went all-in on AMD + Why Devs feel faster with AI but are slower — with Quentin Anthony
- The AI Agenda: GPT5 leaks and the business of AI News — Steph Palazzolo, The Information
- Information Theory for Language Models: Jack Morris
- Building the Next Generation of Conversational AI
- every mention in 2025, scene by scene →
2024 22 mentions in 8 episodes 3 per episode
- Building an open AI company - with Ce and Vipul of Together AI
- No Priors Ep. 70 | With Cartesia Co-Founders Karan Goel & Albert Gu
- The Four Wars of the AI Stack - Dec 2023 Recap
- 2024 in Post-Transformer Architectures: State Space Models, RWKV [Latent Space LIVE! @ NeurIPS 2024]
- AGI, The Future of AI Agents And The Next Wave of Opportunities in AI | Richard Socher, CEO, You.com
- The Winds of AI Winter (Q2 Four Wars of the AI Stack Recap)
- Ethan Mollick: Why OpenAl Abandons Products, The Biggest Opportunities They Have Not Taken | E1184 · 20VC with Harry Stebbings
- A Comprehensive Overview of Large Language Models - Latent Space Paper Club
- every mention in 2024, scene by scene →
every mention on every show, scene by scene, with the transcript →
7 statements about Mamba, every show
Transformers perform all Bayesian tasks, Mamba does most, and MLPs fail
“Transformer does everything. Mamba does most of it. LSTMs do only partially, and MLPs fail completely.”
Groq hardware is too inflexible to run alternative architectures like Mamba SSMs
“Grok is very inflexible hardware so that you kind of, if you want to do like a Mamba SSM on it, you're going to have a really hard time because instead of having like a low level CUDA compiler, like everything is like Designed it at the hardware level, right?”
Lenz: Hybrid Transformer models are here to stay for long context
“If I had to guess hybrid models are here to stay just because the efficiency without sacrificing the performance is, is too much to give up. You know, the attention is so expensive. The quadratic cost and the linear memory cost is so much that I think for long…”
Lenz: Middle attention placement at 1:8 ratio optimizes hybrid models
“Putting it the first or the last performed worse than in the middle. And one to eight was good enough. You know, you might get very slight improvements with one to six, but it was marginal, maybe within the standard deviation.”
Cheah: Hybrid SSM-transformer models outperform pure baselines of both
“None of us understand why a hybrid with a state-based model, the RWA state space, and transformer performs better than the baseline of both. It's like when you train one, you expect, and then you replace, you expect the same results. That's our pitch. That's o…”
Gu: Mamba successfully applied state-space models to language modeling
“Recently proposed a model called Mamba which was kind of brought these to language modeling and showed really good results there.”