Softmax
topic on 2 shows · 4 statements across 2 episodes · said 6 times in 1 episodes since 2025
the a16z Podcast 6 
Latent Space
Mentions by year, every show
tap a year for its mentions
the a16z Podcast 6
2025 6 mentions in 1 episode
every mention on every show, scene by scene, with the transcript →
4 statements about Softmax, every show
Fused GPU kernels cannot eliminate inter-GPU communication in tensor parallelism
“Well, I mean, like a fused kernel can't save you. Like here with the tensor parallelism, you're the half, the matrix is one GPU and the other half is on another. And if I need the entire matrix in order to do like a non-linear operation on the next step, which…”
Shear: AI alignment requires pre-training on full game-theoretic manifolds
“You have to get it to, it has to be trained on the full manifold of every possible game theoretic situation, every possible team situation, every possible making teams, breaking teams, changing the rules, not changing the rules, all of that stuff. And then it …”
Shear: Softmax is building multi-agent RL simulations for alignment
“So that's our goal. It's like big multi-agent reinforcement learning simulations, which create a surrogate model for alignment.”
Shear: Softmax is building toward a peer-based human and AI future
“So we have this awesome suite of AI tools used by us and our AI brethren who care about each other and want to build a glorious future together. I think that would be a really beautiful future. And it's the one we're trying to build.”