Chinchilla paper
3 statements across 3 episodes · 1 bullish · 0 bearish · 3 people on the record · first statement Jun 25, 2024 by Josh Albrecht · across every show →
Everything said about Chinchilla paper, oldest first
Jun 25, 2024 positive
Albrecht: Cost-aware tuning reveals scaling laws for all hyperparameters
“So by doing that, we can see the scaling laws or not just, you know, the scaling laws from like the, you know, chinchilla paper, the scaling laws for all parameters. We can see how does the number of layers change with this? How does the You know, the learning…”
Jul 23, 2024 neutral
Jan 1, 2025
Swyx: Inference compute optimality creates different scaling laws than Chinchilla
“Chinchilla paper is compute optimal training, but what is not stated in there is it's pre-trained compute optimal training. And once you start caring about inference, compute optimal training, you have a different scaling law and in a way that we did not know …”