Vision Transformers
topic on 1 show · 1 statements across 1 episodes
1 statements about Vision Transformers, every show
Morcos: Soft inductive biases become harmful past 1M data points in vision
“Turns out in the small data regime, and when I say small data here, I mean, say less than 500,000 data points. And this was in the context of image self-supervised learning. So in that small data regime, this is super helpful. And where this paper's actually b…”