Douwe Kiela: AI Model Parameter Sizes Will Stop Growing Rapidly
Douwe Kiela · 20VC: The Biggest AI Leaders on What Matters More; Model Size or Data Size & Where Does The Value in AI Accrue; to Startups or to Incumbents · Sep 15, 2023 · at 13:29
Douwe Kiela, co-founder and CEO of Contextual AI, discusses AI scaling laws and model parameter limits on The 20VC podcast.
0:00 / 0:16exact quote · 16.6s
720p mp4 · rendered on demand · StarZero watermark
“I think Sam Altman had this interesting quote where he was saying that he thought models would stop growing in size. GPT-IV kind of hit this ceiling. I think that's probably right, but not really because size doesn't matter. It's just that data size matters even more than model size.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Douwe Kiela
Douwe Kiela: Incumbent AI firms promote existential risk to entrench regulatory advantage
“So the people who are pushing this narrative are really the people who are benefiting from this being the narrative. So these are the incumbent AI companies who are, who want to have either the market regulated. Right. Because then they benefit because they ca…”
Kiela: Believing open-source AI can match OpenAI is incredibly naive
“I would like it to be true that with open source, we could just keep up with all of that. But I think that's just incredibly naive. Open AI has this very deep understanding of how people want to use language models. Basically nobody else has, and they have thi…”
Douwe Kiela: The leaked 'no AI moat' Google memo is completely wrong
“I think in terms of data modes and maybe you, you've seen this come by actually, there was this Google memo from an internal Google employee who had written that open AI and Google have no mode. I think for me as a AI researcher, when I read that memo, I was l…”
Douwe Kiela: Getting struck by lightning is more likely than AI extinction
“Probably the chance of me getting hit by lightning, like right now is much higher than that happening.”
Douwe Kiela: EU AI regulation will completely destroy innovation
“What Europe is going to try to do is overregulate everything and just completely destroy
innovation.”
Douwe Kiela: AI will displace the majority human workforce within 5-10 years
“If you look at how OpenAI and Anthropic and these places define AGI, it's as systems achieving capabilities that allow them to effectively do the work of humans for the majority of economically valuable human tasks. Then we're not that far away from it. And so…”