Gomez: Industry took two to three years to realize language model scaling worked
Aidan Gomez · Aidan Gomez: What No One Understands About Foundation Models | E1191 · 20VC with Harry Stebbings · Aug 19, 2024 · at 20:56
Aidan Gomez is the co-founder and CEO of Cohere and a co-author of the seminal 2017 Transformer paper 'Attention Is All You Need'. He discusses the lag between early research breakthroughs and tech industry realization.
0:00 / 0:17exact quote · 17.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“With language modeling and the whole scaling project, I thought the world would catch on way faster to that piece. It started to become really obvious, but then it was two, three years before everyone woke up, and it sort of hit the world.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Aidan Gomez
Aidan Gomez: Becoming dependent on cloud providers is dangerous for AI startups
“It's really dangerous when you make yourself a subsidiary of your cloud provider.”
Gomez: OpenAI's AGI effort is taking a backseat to consumer products
“At least like lately, or in the new OpenAI they're like a product company. They're like hardcore building a consumer product. That is their objective, and it's working. People love that product. It's, you know, a household name at this point. So I think in the…”
Aidan Gomez: AI agent developers are disadvantaged without owning model architecture
“If you're not able to actually transform the model to be better at the thing that you care about, if you're not the one building the model, if you're just a consumer of the model you're structurally disadvantaged to build that product.”
Aidan Gomez: AI Will Not Cause Mass Unemployment
“The fears around displacement and replacement, both on the consumer side, where we're all gonna get addicted to chatting to these chatbots, and on the workplace, the end of work, you know, there's gonna be mass unemployment. I can't see that happening.”