token generation
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jul 5, 2026 by Jonathan Ross · across every show →
Everything said about token generation, oldest first
Jul 5, 2026
Ross: Splitting LLM pre-fill and generation across different chips is a mistake
“What we realized was, and this is what most people get wrong when they're trying to do this themselves. They'll, they'll take the reading of tokens or what's called pre-fill, and they'll do that on one piece of hardware, and then they'll put the generation of …”