AdamW
topic on 3 shows · 3 statements across 3 episodes · said 6 times in 2 episodes since 2024
Latent Space 3 
the a16z Podcast 3 
the MAD Podcast
Mentions by year, every show
tap a year for its mentions
the a16z Podcast 3Latent Space 3
2025 3 mentions in 1 episode
2024 3 mentions in 1 episode
every mention on every show, scene by scene, with the transcript →
3 statements about AdamW, every show
RLVR and GRPO will stabilize into canonical algorithms similar to AdamW
“It's similar to, you know, optimizers with Adam. So Adam is, I mean, right now there's Adam W. There was SGD and all the other RMS prop and how they were called. And they kind of all converge to Adam W by adding more and more tricks. And I think that's the sam…”
Bakouch: Novel optimizer speedups are exaggerated due to undertuned AdamW baselines
“And what they find is that the speed up is greatly, greatly exaggerated. And mostly because often people like undertone the Adam W baseline.”