Soldaini: Frontier AI labs limit final pre-training runs to two months
Luca Soldaini · Open Source AI Strikes Back — Inside Ai2’s OLMo 3 ‘Thinking" · Nov 20, 2025 · at 47:21
Luca Soldaini, AI researcher at Ai2, explains typical computational constraints and duration limits in pre-training frontier language models.
0:00 / 0:11exact quote · 11.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I think it's standard practice among the frontier labs to try to cap your big final pre-training run to two months not more than that.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Luca Soldaini
Soldaini: Most open AI models are open weights, not open source
“Majority of models that get release I think the best term to describe them is open weights. Your Quinn, your Gemma, your Lama you know, Kimi it's what gets release is a set of weights that correspond either to the final state of model, that's the most common, …”
Soldaini: AI scaffolding allows people outside frontier labs to drive capabilities
“If the scaffolding is what really moves a lot of like from, you know, broad capability model to like something that actually has meaningful impact, that scaffolding is not just like, oh, only the labs of people are trained models can do it. Like the number of …”
Ai2 releases OLMo 3 with full training recipes, data, and intermediate checkpoints
“We're not just releasing the final models. We're releasing, you know, the entire recipe we followed to get this model. So the data, the intermediate states, the evaluation frameworks, all the details, all the bits that people need to know to make models like O…”
Soldaini: Flawed long-context model architecture cannot be saved by good data
“But they're like technical decisions in how you set up your model that you can have the best data in the world. And your model will not be able to reason over many, many tokens. So it doesn't matter in the sense that you can't train the model on bad data, but …”