disaggregated serving
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Aug 3, 2026 by Ali Taha · across every show →
Everything said about disaggregated serving, oldest first
Aug 3, 2026 bullish
Ultra-fast network interface cards could deliver 100x speedups in disaggregated AI inference
“If you were to somehow be able to, in like this theoretical dreamland, have extremely fast NICs, you could, in theory, spare that HBM and you could just transfer KVCache trans, like directly from one node to another. This would give you like almost a hundred X…”