Softmax
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Aug 3, 2026 by Ali Taha · across every show →
Everything said about Softmax, oldest first
Aug 3, 2026 negative
Fused GPU kernels cannot eliminate inter-GPU communication in tensor parallelism
“Well, I mean, like a fused kernel can't save you. Like here with the tensor parallelism, you're the half, the matrix is one GPU and the other half is on another. And if I need the entire matrix in order to do like a non-linear operation on the next step, which…”