AI Model Architecture
topic on 1 show · 1 statements across 1 episodes
1 statements about AI Model Architecture, every show
Soldaini: Flawed long-context model architecture cannot be saved by good data
“But they're like technical decisions in how you set up your model that you can have the best data in the world. And your model will not be able to reason over many, many tokens. So it doesn't matter in the sense that you can't train the model on bad data, but …”