inference compute optimal training
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jan 1, 2025 by Shawn Wang · across every show →
Everything said about inference compute optimal training, oldest first
Jan 1, 2025
Swyx: Inference compute optimality creates different scaling laws than Chinchilla
“Chinchilla paper is compute optimal training, but what is not stated in there is it's pre-trained compute optimal training. And once you start caring about inference, compute optimal training, you have a different scaling law and in a way that we did not know …”