Inference Compute Optimal Training
topic on 1 show · 1 statements across 1 episodes
1 statements about Inference Compute Optimal Training, every show
Swyx: Inference compute optimality creates different scaling laws than Chinchilla
“Chinchilla paper is compute optimal training, but what is not stated in there is it's pre-trained compute optimal training. And once you start caring about inference, compute optimal training, you have a different scaling law and in a way that we did not know …”