Data Wall
topic on 5 shows · 5 statements across 5 episodes
the Knowledge Project 
Lenny's Podcast 
No Priors 
Big Technology 
20VC
5 statements about Data Wall, every show
Taylor: Reasoning models generate net new ideas, breaking the data wall
“What's really interesting about, you know, reasoning and reasoning models is I think I feel really optimistic these models are generating net new ideas, and so it really affords the opportunity to break through some of these, the data wall as well.”
Nguyen: Post-training scaling avoids data walls through infinite learnable tasks
“The scaling in post-chaining itself is not hitting the wall, and that's because Basically, we went from, like, raw data sets from pre-trained models to infinite amount of tasks that you can teach the model in the post-training world via reinforcement learning.…”
Alex Wang: AI performance plateau is caused by hitting a data wall
“And I think a lot of the plateau that we've recently seen can almost be explained at a very high level from hitting kind of a data wall.”
Frontier AI scaling will hit a 'data wall' before GPT-5
“And I think the main thing we're going to learn is between 4.5 and five level models, we're going to hit What's known as the data wall, which is to say that as you make these models bigger, you need more and more data to keep training them. And we're running o…”
Sarah Guo believes the AI data wall is solvable via synthetic data
“The number of cheap available tokens on the internet we have used, and now we have to go figure out how to go get more, collect more in the world, or more likely, like, you know or in combination with generating synthetic data. That still feels like a bits not…”