Chinchilla scaling laws
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Dec 28, 2023 by Arthur Mensch · across every show →
Everything said about Chinchilla scaling laws, oldest first
Dec 28, 2023 positive
Mensch: Overtraining models past Chinchilla limits lowers inference costs
“If you take into account the fact that your model Should also be efficient at inference time. You probably want to go far beyond the Cinchilla scaling low. So it means you want to overtrain the model. So train on more tokens than would be optimal for performan…”