attention
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jul 8, 2025 by Dwarkesh Patel · across every show →
Everything said about attention, oldest first
Jul 8, 2025 neutral
Patel: No researchers have bypassed transformer quadratic attention scaling
“And while people have found modifications to the transformer, which make the constant time overhead of attention, reduce the constant time overhead on attention to like, you find these little hacks with mixture of experts or latent attention, nobody has gotten…”