Alexandr Wang: Achieving AGI will take multiple decades of solving individual problems.
Alexandr Wang · No Priors Ep. 65 | With Scale AI CEO Alexandr Wang · May 22, 2024 · at 34:53
Scale AI CEO Alexandr Wang shares his most contrarian AI belief on No Priors with Sarah Guo and Elad Gil.
0:00 / 0:49exact quote · 49.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“My biggest belief here is that the path to AGI is is one that looks a lot more like curing cancer than developing a vaccine. And what I mean by that is I think that the path to build AGI is going to be in, in, you know, you're going to have to solve a bunch of small problems that, where you don't get that much positive leverage between solving one problem to solving the next problem. And there's just sort of, you know, it's like curing cancer, which is you have to then zoom into each individual cancer and solve them independently. And eventually over a multi decade timeframe, we're going to look back and realize that we've, you know, built HGI, we've cured cancer, but the path to get there will be this like, you know, quite plodding road of solving individual capabilities and building individual sort of data flywheels to support this end mission.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Alexandr Wang
Wang: The path to AGI resembles curing cancer, not a vaccine
“My biggest belief here is that the path to AGI is is one that looks a lot more like curing cancer than developing a vaccine. And what I mean by that is I think that the path to build AGI is going to be in, in, you know, you're going to have to solve a bunch of…”
Wang: AI gets no positive transfer across modalities like video to text
“My understanding there's no positive transfer from learning in one modality to other modalities. So like training off of a bunch of video doesn't really help you that much with your text problems and vice versa.”
Wang: No strong evidence that video creates useful AI world models
“I don't think there's strong scientific evidence of that yet. Maybe there will be eventually.”
Alexandr Wang: Cross-modality training produces no positive transfer between video and text.
“I think the main thing, fundamentally, is I think there's very limited generality that we get from these models and even for multimodality, for example my understanding there's no positive transfer from learning in one modality to other modalities. So like tra…”
Wang: Standard academic benchmarks are contaminated by training data overfitting
“Most of the benchmarks that we as a community look at... Academic benchmarks that are what the industry used to measure the performance of these algorithms are fraught with issues. Many of the models are overfit on these benchmarks. They're sort of in the trai…”
Alexandr Wang: Data abundance is the fundamental bottleneck for post-GPT-4 models.
“The key to the scaling of these large language models and the, you know, these language models in general is the ability to scale data. And I think that one of the fundamental bottlenecks to, you know, what's, what's in the way of us getting from GPT-IV to GPT…”