Inference-Time Compute Does Not Require Densely Interconnected Supercomputers
Aidan Gomez · No Priors Ep. 91 | With Cohere Co-Founder and CEO Aidan Gomez · Nov 21, 2024 · at 29:28
Aidan Gomez, CEO of Cohere, breaks down how shifting from pre-training scaling to inference scaling reshapes data center architecture and chip requirements.
0:00 / 0:11exact quote · 11.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“If we have a new avenue, which is inference time compute, That doesn't require this densely interconnected supercomputer. It's fine to have nodes. You can do a lot more locally and less distributed.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Aidan Gomez
Humanity Will Achieve Broadly Capable AGI but Will Not Build God
“We will build AGI if what you mean is very useful, generally capable technology that can do a lot of the stuff that humans can do and flex into a lot of different domains. If what you mean is, you know, are we gonna build God? No.”
Falling AI API Prices Stem From Price Dumping, Not Commoditization
“I don't think that models are actually getting commoditized. I think what you see is you see price dumping. And so you see people giving it out for free, giving it out at a loss, giving out at zero margin. And so they see the prices coming down and they assume…”
Cohere Will Not Build a Direct ChatGPT Competitor
“We're not going to build a ChatGPT competitor. What we want to build is a platform and a series of products to enable enterprises to adopt this technology and make it valuable.”
Current LLM Tech Alone Requires Five Years of Economic Integration Work
“Even if we didn't train a single new language model, like, okay, all the data centers blow up. We can't improve the LLM. We only have what we have today. There's a half decade of work to go integrate this into the economy, to build all these things, to build t…”
GPT-4 Class Enterprise Models Now Cost $10M to $20M to Train
“What we've seen is today you can build a model that's as good as GPT-IV in all the things that enterprises might care about. For ten million dollars, twenty million dollars, like just orders of magnitude less than what was spent to develop that model. And so i…”
Cohere Strategically Lags Frontier Labs by Six Months to Avoid $7B Burn
“So that's the strategy is don't lead. Don't burn, you know, three, five, seven billion dollars a year to be at the front, be six months behind. And offer something to market to enterprises that actually fits their needs at a price point that makes sense for th…”