inference cost
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Dec 28, 2023 by Arthur Mensch · across every show →
Everything said about inference cost, oldest first
Dec 28, 2023 positive
Mensch: Overtraining models past Chinchilla limits lowers inference costs
“If you take into account the fact that your model Should also be efficient at inference time. You probably want to go far beyond the Cinchilla scaling low. So it means you want to overtrain the model. So train on more tokens than would be optimal for performan…”