Alexandr Wang
13 statements across 2 episodes · 3 bullish · 6 bearish · 1 people on the record · first statement May 22, 2024 by Alexandr Wang · said 6 times in 5 episodes since 2024 · across every show →
On the record as a speaker too: Alexandr Wang's record, appearances and statements → this page counts the times other people say the name.
Mentions by year
brought up most by Sarah Guo (3), Pedro Franceschi (1), Melisa Tokmak (1), Joshua Xu (1)
tap a year for its mentions
2026 1 mention in 1 episode
2025 2 mentions in 1 episode
Everything said about Alexandr Wang, oldest first
May 22, 2024 bearish
May 22, 2024 negative
Alexandr Wang: Academic AI benchmarks are contaminated and models are overfit.
“The academic benchmarks that are what the industry used to measure the performance of these algorithms are fraught with issues. Many of the models are overfit on these benchmarks. They're sort of in the training data sets of these models.”
May 22, 2024 bearish
Alexandr Wang: Multimodality faces a scarcity of quality data for personal agents.
“So multimodality as an entire space is one where for the same reasons that we've like exhaust a lot of the internet data, there's a lot of scarcity for good multimodal data that can empower these personal agents and these personal
Use cases.”
May 22, 2024 neutral
Alexandr Wang: Data abundance is the fundamental bottleneck for post-GPT-4 models.
“The key to the scaling of these large language models and the, you know, these language models in general is the ability to scale data. And I think that one of the fundamental bottlenecks to, you know, what's, what's in the way of us getting from GPT-IV to GPT…”
May 22, 2024 positive
Alexandr Wang: Human-AI teams will outperform standalone models for a long time.
“The question is, is a human plus a model together going to be able to produce better output than a model alone? And I think that'll be the case for A very, very, very long time. That, that humans are still, you know, human intelligence is complementary to mach…”
May 22, 2024 positive
Alexandr Wang: High-quality frontier data is 10,000x more valuable than enterprise data.
“One of the things that every, you know, all the model developers understand well, but the enterprises understand super well is that you know, not all data is created equal and high quality data or frontier data is, is, can be, you know, 10,000 times more valua…”
May 22, 2024 neutral
Alexandr Wang: Achieving AGI will take multiple decades of solving individual problems.
“My biggest belief here is that the path to AGI is is one that looks a lot more like curing cancer than developing a vaccine. And what I mean by that is I think that the path to build AGI is going to be in, in, you know, you're going to have to solve a bunch of…”
May 22, 2024 bearish
Alexandr Wang: GPT-4 was too early a model to sustain application hype.
“GPT-IV, I think, as a model, was a little early of a technology for us to have this entire hype wave around, and I think we, you know, the community very quickly discovered all the limitations of GPT-IV... It was probably a few generations too early of a model…”
May 22, 2024 negative
Alexandr Wang: Multimodality is a lateral move; the industry needs smarter models.
“So, you know, we got multi-modality capability.
That's exciting.
It's more of a lateral expansion of the models, and the industry needs smarter models.
We need GPT-V, or we need Gemini-II, or whatever that, those models are going to be.
and so to me it was, y…”
May 22, 2024 positive
Alexandr Wang: Scale AI will launch recurring held-out LLM benchmark leaderboards.
“So one is that we're going to launch these private held out evaluations and have leaderboards associated with these evals for the leading LLMs in the ecosystem. And we're going to rerun this contest periodically. So every few months we're going to do a new set…”
May 22, 2024
Alexandr Wang: Held-out benchmarks reveal several AI models underperform their reported scores.
“So we, one of the things we did is we published DSM-I-K, which was a held out eval. So we basically produced a new evaluation of the math capabilities of models. That there's no way it would ever exist in the training data set to really see how much of the, ho…”
May 22, 2024 neutral
Alexandr Wang: The AI industry has exhausted all easy internet training data.
“And we've sort of, as a community, we have, we've had easy data, which is all the data on the internet and we've kind of exhausted all the easy data, and now it's about, you know, forward data production that has high supervisory signal that is basically very …”
Jul 11, 2024 negative
Wang: Standard academic benchmarks are contaminated by training data overfitting
“Most of the benchmarks that we as a community look at... Academic benchmarks that are what the industry used to measure the performance of these algorithms are fraught with issues. Many of the models are overfit on these benchmarks. They're sort of in the trai…”