attention mechanism
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Sep 6, 2023 by Ben Gilbert · across every show →
Everything said about attention mechanism, oldest first
Sep 6, 2023 neutral
Transformer attention computational cost scales quadratically with input prompt length
“Traditionally, you'd say this is very, very inefficient, and it actually means that the larger your context window, aka token limit, aka prompt length, gets, the more computationally expensive it gets on a quadratic basis. So doubling your input means quadrupl…”