accelerator
2 statements across 2 episodes · 0 bullish · 0 bearish · 2 people on the record · first statement Jul 29, 2026 by Misha Smulyanski · across every show →
Everything said about accelerator, oldest first
Jul 29, 2026
Smulyanski: LLM decode workloads remain bandwidth-bound even across large batch sizes
“Pre-fill is generally very compute bound. Right because you basically do, ah, like, attention, ah, you do, ah, ah, a lot of, ah work for, you know, for, you know, all the tokens that you are fetching, right? You can, ah, you fetch the weights while I'm once an…”
Jul 30, 2026
Dean: 1,000x energy penalty for moving data forces machine learning batching
“I mean, I think the example you raised of a thousand X difference in bringing moving data versus actually computing on it in, in terms of energy is, is a pretty significant one. And it shapes a lot of aspects of what we do in machine learning. Because if you d…”