compute per token
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jul 23, 2024 by Thomas Scialom · across every show →
Everything said about compute per token, oldest first
Jul 23, 2024 neutral
Scialom: Fixed compute per token in transformers forces models to generate extra tokens to think
“We are lacking of flexibility for pre-training architecture, transformers, where We spend the same amount of compute per token. And so, because of that, how can you, like, mitigate this? By generating more tokens, so more thoughts, more compute, because you ha…”