full attention models
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Oct 11, 2025 by Barak Lenz · across every show →
Everything said about full attention models, oldest first
Oct 11, 2025 bearish
Lenz: Full attention models will decline as sequence lengths rise
“I can definitely see sequence length rising, and I can't see full attention models being as prominent as they are today. So, so, so at least they'll have less full attention layers and I hope they'll have more innovations like Mumbai.”