Big Labs
topic on 5 shows · 9 statements across 9 episodes
More or Less 
Latent Space 
the a16z Podcast 
TBPN 
20VC
9 statements about Big Labs, every show
Sam Lessin: Big Frontier AI Labs Are Economically 'Fucked'
“This isn't a technical commentary, but from like a business and economics perspective, I think all the big labs are fucked.”
Horowitz: Big AI labs stumble due to employee addiction to public validation
“Why did the big labs, you know, smack themselves in the face? Well, it's because they need that external validation for their employees, because their employees are addicted to it.”
Weinberg: AI Researchers Leave Big Labs When Strategic Direction Changes
“Researchers really genuinely do care about pushing research forward, and I think what ends up happening sometimes is the leaders at some of these companies kind of change the direction of the company, and the researchers are no longer interested in that direct…”
Morcos: Major AI labs fail to holistically integrate pre-, mid-, and post-training
“Something that you don't see happen even at the big labs because they have entirely separate teams, right? There's a free training team. There's a mid training team. There's a post training team. And like the mid training team is a customer of the free trainin…”
Swyx: Big Frontier AI Labs Are Probably Two Years Ahead of Open Research
“I'm not sure any of it is going to bear fruit, because the big labs are probably, like, two years ahead of us in the open research”
Model training from scratch will concentrate mostly in major AI labs
“As AI improves, like, fewer people probably need to train models from scratch. It gets concentrated more and more in, in different, like, in the big labs.”
Swyx: Frontier labs view new learning rate optimizers as unimportant
“Most people at the big labs who I asked about this say that it's cute, but it's not something that matters.”
Lambert: Frontier Labs Lack Visibility into Cross-Model RLHF Sensitivity
“I think big labs are so over-indexed, are indexed on their own base models, so they don't know, like, what's swapping between CloudBase or GPT-IV-Base, how that would change any notion of preference or what you do with RLHF.”