InfiniBand, every mention
9 scenes · ← back to InfiniBand
tap a year for its mentions
every year anyone Jonathan Frankle 4Evan Conrad 3Kyle Kranen 2Soumith Chintala 1Akshat Bubna 1
Verbatim, from the transcripts: the passages where InfiniBand comes up
The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO
- ▶ 33:28 Akshat Bubna When you run multi-handle training on, on modal, uh, RDMA, uh, I, I think Melanox, uh, is, or InfiniBand is, like, uh, is, you also use RDMA, uh, but basically it's a way to bypass, uh,
Agent Inference at the "Speed of Light" — How NVIDIA moves like a $4.3 Trillion Startup
- ▶ 30:37 Kyle Kranen You're gonna have to figure out ways to handle the fact that now for the GPUs to communicate, you have to do it over InfiniBand, which is still very fast, but is not as fast as NVLink. 2 times in the scene
SF Compute: Commoditizing Compute
- ▶ 26:14 Evan Conrad That's not always the case because you just have logistical problems, um, like, you get a cluster and parts of the InfiniBand fabric are broken, and there's, like, um, some issue with some switch somewhere,
- ▶ 29:31 Evan Conrad Um, it's like the InfiniBand cables or the mix or like whatever, or you need a bunch of like generators or you're, you don't have data center space or like, but like there's always some bottleneck somewhere else.
- ▶ 34:33 Evan Conrad Yeah, like, I just don't really think this is gonna ever be more efficient than a fully interconnected cluster with Infiniband, or, you know, whatever sort of next spec might be.
State of the Art: Training 70B LLMs on 10,000 H100 clusters
- ▶ 10:54 Jonathan Frankle And, you know, we've dealt with the weirdest things from, you know, our InfiniBand cables getting stolen from the data center twice, like in boxes before they arrived at the data center, like, you know, porch pirate basically had stolen… 2 times in the scene
- ▶ 13:34 unnamed speaker Um, they use unified fabric manager nodes, um, which manage the infinite band network, and you talk a little bit about your networking.
- ▶ 23:20 Jonathan Frankle Somehow I'm still asking them about which InfiniBand provider they used this time, um, to see if this is part of the bad batch of cables I encountered on that cloud provider. 2 times in the scene
Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI
- ▶ 7:13 Soumith Chintala Um, similarly, GPU has several levels of memory, and then you have, like, different levels of network hierarchies, NVLink plus, like, uh, InfiniBand or Rocky or something like that, right?