Latent Attention
topic on 1 show · 1 statements across 1 episodes
1 statements about Latent Attention, every show
Patel: No researchers have bypassed transformer quadratic attention scaling
“And while people have found modifications to the transformer, which make the constant time overhead of attention, reduce the constant time overhead on attention to like, you find these little hacks with mixture of experts or latent attention, nobody has gotten…”