prefill
also referred to as: pre-fill
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jul 29, 2026 by Misha Smulyanski · across every show →
Everything said about prefill, oldest first
Jul 29, 2026
Smulyanski: LLM decode workloads remain bandwidth-bound even across large batch sizes
“Pre-fill is generally very compute bound. Right because you basically do, ah, like, attention, ah, you do, ah, ah, a lot of, ah work for, you know, for, you know, all the tokens that you are fetching, right? You can, ah, you fetch the weights while I'm once an…”