NVLink
3 statements across 3 episodes · 1 bullish · 1 bearish · 3 people on the record · first statement Mar 6, 2024 by Soumith Chintala · said 15 times in 8 episodes since 2023 · across every show →
Mentions by year
brought up most by Kyle Kranen (3), George Hotz (3), Soumith Chintala (2), Jonathan Frankle (2), Varun Mohan (1), Philip Kiely (1), Eugene Cheah (1)
tap a year for its mentions
2026 4 mentions in 2 episodes 2 per episode
2025 1 mention in 1 episode
2024 7 mentions in 4 episodes 2 per episode
- State of the Art: Training 70B LLMs on 10,000 H100 clusters
- Supervise the Process of AI Research — with Jungwon Byun and Andreas Stuhlmüller of Elicit
- Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI
- [LLM Paper Club] Llama 3.1 Paper: The Llama Family of Models
- every mention in 2024, scene by scene →
2023 3 mentions in 1 episode
Everything said about NVLink, oldest first
Mar 6, 2024 bullish
Chintala: Nvidia's primary competitive moat is NVLink interconnect, not GPU silicon
“The mode that Nvidia has right now, I feel like, is that they're, they have the interconnect that no one else has. Like, AMD GPUs are pretty good. I'm sure there's very silicon that is not bad at all, but, like, the interconnect like, NVLink is uniquely awesom…”
Mar 8, 2026 neutral
Scaling beyond eight NVIDIA H100 GPUs requires inter-GPU communication over InfiniBand
“The maximum NVLink domain, domain for H 100 for most DGX H 100 Is eight GPUs, right? So if you scaled up past that, You're gonna have to figure out ways to handle the fact that now for the GPUs to communicate, you have to do it over InfiniBand, which is still …”
Aug 3, 2026 bearish
Tensor parallelism is a poor fit for local multi-device AI setups
“Tensor parallelism is not a good fit for local AI because it assumes a very high bandwidth interconnects like NVLink was, you know, They might be forced to do something like pipeline parallelism, which we're never going to do unless we're doing some kind of mu…”