SmolLM2
product on 1 show · 3 statements across 2 episodes · said 2 times in 1 episodes since 2024
Mentions by year, every show
tap a year for its mentions
Latent Space 2
every mention on every show, scene by scene, with the transcript →
3 statements about SmolLM2, every show
Bakouch: SmolLM 2 scored random on MMLU until 6.5 trillion tokens
“And this is basically until like 6.5 trillion of tokens, which is a lot, to be honest. Until this amount of token, the MMLU in the QA format, meaning that the model have to select which answer is, the model have to output, for example, the right answer is A, o…”
Ben Allal: Small models continue improving when trained on 11T tokens
“For example, smaller than one was trained only on one trillion tokens, but this model is trained on 11 trillion tokens. And we saw that the performance kept improving. The models didn't really plateau me training. Which I think is really interesting. It shows …”