Efficient Attention
topic on 1 show · 1 statements across 1 episodes
1 statements about Efficient Attention, every show
Yi Tay: Efficient attention research consistently failed to eliminate self-attention
“There was this whole big era, which I was also involved in this era, where people try to like undermine the attention as much as possible. Like they try to remove it, simplify it, make it efficient, like this whole Like, efficient attention era. At the end of …”