Narayanan: No AI model has meaningfully surpassed GPT-4 in 18 months
Arvind Narayanan · Arvind Narayanan: AI Scaling Myths, The Core Bottlenecks in AI Today & The Future of Models | E1195 · 20VC with Harry Stebbings · Aug 28, 2024 · at 44:46
Princeton CS professor Arvind Narayanan evaluates the state of frontier AI models following the launch of OpenAI's GPT-4.
0:00 / 0:10exact quote · 10.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And what we've seen in the nearly year and a half since GPT-IV came out is that we haven't really had models That have surpassed it in a meaningful way.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Arvind Narayanan
Narayanan: AI model scaling cycles are ending
“We're not gonna have too many more cycles, possibly zero more cycles, of a model that's almost an order of magnitude bigger in terms of the number of parameters than what came before, and thereby more powerful.”
Narayanan: Skeptical GPT-5 will yield a leap comparable to GPT-4
“Are we going to see a GPT-V that's as big a leap over GPT-V as GPT-V was over GPT-V? I'm frankly skeptical.”
Narayanan: Recursive AI scaling using synthetic data will not work
“And the other way to look at synthetic data is, okay, you take one trillion tokens, you train a model on it, and then you output 10 trillion tokens, so you get to the next bigger model, and then you use that to output a hundred trillion tokens. You know, I'll …”
Narayanan: GPT-4 passing bar and medical exams meant nothing for actual practice
“So when GPT-IV came out and OpenAI claimed that it passed the bar exam and the medical licensing exam people were very excited slash scared about what this means for doctors and lawyers, and the answer turned out to be approximately nothing, right? Because it'…”