temperature zero inference
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jul 29, 2024 by Eugene Yan · across every show →
Everything said about temperature zero inference, oldest first
Jul 29, 2024
Eugene Yan: GPU floating-point math makes temperature-zero inference non-deterministic
“For GPUs with floating points, and you push it through so many calculations, and so many met miles, the floating points aren't just not gonna be precise. So that's why even if temperature is zero, it's not gonna be the same throughout, ah, for multiple request…”