Taylor: Pre-training a fourth-best foundation model offers highly questionable ROI
Bret Taylor · Bret Taylor: A Vision for AI’s Next Frontier · Apr 15, 2025 · at 1:01:10
Bret Taylor, OpenAI Board Chair and Sierra co-founder, discusses the economic viability and capital dynamics of training foundation models versus frontier AI models.
0:00 / 0:21exact quote · 21.4s
720p mp4 · rendered on demand · StarZero watermark
“If you're pre-training a foundation model that's the fourth best that's gonna cost you a lot of money, and the return on that investment is, is probably fairly questionable, because why use your fourth best large language model versus a frontier model or an open source one for Meta?”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Bret Taylor
Taylor: The craft of software engineering will completely change by 2027
“The actual craft of software engineering that we're doing right now I think it's literally a fact that it'll be completely different two years from now.”
Taylor: The craft of software engineering will be completely different by 2027
“And if I look at the actual craft of software engineering that we're doing right now I think it's literally a fact that it'll be completely different two years from now.”
Taylor: Generating AI code in human-designed programming languages is a local maximum
“It feels like a local maximum in a really obvious way to me, which is you have a bunch of code written by people written in programming languages that were designed to make it easy for people to tell a computer what to do.”
Taylor: Treating software engineers as machine operators will make development more robust
“I have a strong suspicion that if we designed these systems with the role of a software engineer in mind being an operator of a machine rather than the author of the code, we could make the process Much more robust and much more productive.”
Taylor: AI model market will consolidate into a few massive capex players
“I think we have probably have too many people building models right now. There's already been some consolidation actually of companies being folded into Amazon and Microsoft and others. But I do think it will play out a bit like the cloud infrastructure busine…”
Taylor: Pre-training startups have pharma-like costs without the pharma business model
“I would have a hard time personally, like, funding a startup that says, I'm gonna do pre-training, you know, it's, ah, it's, I don't really know, like, what's your differentiation in this marketplace, and I think a lot of those companies you're already seeing,…”