NICs
2 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Aug 3, 2026 by Ali Taha · across every show →
Everything said about NICs, oldest first
Aug 3, 2026 neutral
Network interface card speed is the primary bottleneck in large-scale AI serving
“I think the answer is just faster next, like faster network chip communications. It seems to me that like more and more memory is the bottom, like you want to have larger models. Right now, when you're doing serving at large, you have to transfer KV cache from…”
Aug 3, 2026 bullish
Ultra-fast network interface cards could deliver 100x speedups in disaggregated AI inference
“If you were to somehow be able to, in like this theoretical dreamland, have extremely fast NICs, you could, in theory, spare that HBM and you could just transfer KVCache trans, like directly from one node to another. This would give you like almost a hundred X…”