pre-train
2 statements across 2 episodes · 1 bullish · 1 bearish · 2 people on the record · first statement Jun 11, 2024 by Mike Conover · across every show →
Everything said about pre-train, oldest first
Jun 11, 2024 bearish
Conover: Economic incentives to pre-train commodity foundation models from scratch are diminishing
“The incentives, the economic incentives for companies to train their own foundation models, I think, are diminishing. So the, like, window in which you are the dominant pre-train, and let's say that you spend five to forty million dollars, you know, for like a…”
Apr 15, 2025 bullish
Pokrass: AI model gains now driven by post-training, not larger pre-trains
“We find that actually a significant amount of the gains come from new post-training techniques. So I think in the past the narrative is that you need to pre-train these larger and larger models to get better performance, and we're finding that we're able to sq…”