Neural Network Architecture
topic on 5 shows · 6 statements across 6 episodes
Latent Space 
No Priors 
Catalyst 
the MAD Podcast 
the a16z Podcast
6 statements about Neural Network Architecture, every show
Kolter: Neural network architectures matter less than commonly believed
“I actually think architectures don't matter as much as everyone else thinks they do.”
Battaglia: Data pipelines matter more than neural network architectures today
“I don't feel that neural network architectures these days tend to be the sort of exciting part. It's usually more of like the training and the sort of data how you handle the data and that kind of thing.”
Ermon: Diffusion LLMs Can Reuse Standard Architectures and Datasets
“Well, you can use architectures. I think that at least the shapes you, that, that can be leveraged. So you don't have to reinvent and necessarily completely different neural network architectures can, a lot of the data can be used. Like I think perhaps there a…”
Karpathy: Model architecture is no longer the fundamental bottleneck in AI
“I don't think that the neural network architecture is like holding us back fundamentally anymore. It's like not the bottleneck, whereas I think in the previous, before Transformer, it was a bottleneck, but now it's not the bottleneck. So now we're talking a lo…”
Chen: Algorithmic bias comes from incomplete datasets, not neural network architecture
“When people talk about bias in algorithms, they're mostly talking about this phenomenon, which is the human researcher or the human programmer selected an incomplete data set, and therefore you got biased results, as opposed to somehow the architecture of the …”