PyTorch, every mention
35 scenes (2024), the whole family · ← back to PyTorch
tap a year for its mentions
every year 2024 anyone Andrej Karpathy 27Shawn Wang 26George Hotz 25Chris Lattner 11comfyanonymous (Comfy) 6Alessio Fanelli 6Evan Feinberg 5Thomas Sohmers 4Batuhan Taskaya 4Ankur Goyal 4
Verbatim, from the transcripts: the passages where PyTorch comes up
2024 in Post-Transformer Architectures: State Space Models, RWKV [Latent Space LIVE! @ NeurIPS 2024]
[Paper Club] Weight Streaming on Wafer-Scale Clusters (w/ Sarah Chieng of Cerebras)
- ▶ 39:36 Sarah Chieng Um, okay, so the Cerebris Graph Compiler, um, the CGC right here integrates with machine learning frameworks, such as, such as TensorFlow and PyTorch.
Why Compound AI + Open Source will beat Closed AI — with Lin Qiao, CEO of Fireworks AI
- ▶ 2:14 Shawn Wang You ran the PyTorch team at Meta for a number of years, and we previously had Sumith Chintala on, and I think we were just all very interested in, like, the, the history of Gen.E.I. 8 times in the scene
- ▶ 6:59 Shawn Wang When you and I chatted about like the origins of fireworks, it was originally envisioned more as a PyTorch platform, and then later became much more focused on generative AI. 9 times in the scene
- ▶ 9:50 Alessio Fanelli What was the transition from you start focus on PyTorch and like people want to understand the framework, get it live. 2 times in the scene
- ▶ 14:59 Lin Qiao Because we wrote PyTorch code, we know we basically have a special PyTorch build for that, uh, together with a custom kernel we wrote. 2 times in the scene
- ▶ 29:08 Shawn Wang I understand that's also why PyTorch won.
[Paper Club] Upcycling Large Language Models into Mixture of Experts
Production AI Engineering starts with Evals
- ▶ 1:23:17 Ankur Goyal In particular, in DSPY's case, code that looks a lot like PyTorch code. 4 times in the scene
llm.c's Origin and the Future of LLM Compilers - Andrej Karpathy at CUDA MODE
- ▶ 2:03 Andrej Karpathy And then, um, you, you've all worked with PyTorch, of course, right? 5 times in the scene
- ▶ 4:06 Andrej Karpathy So, uh, let's think about, like, what is PyTorch offering you, really? 6 times in the scene
- ▶ 5:39 Andrej Karpathy So for example, layer norm here is like a PyTorch layer, and, uh, we'd like to basically port this over to C. 6 times in the scene
- ▶ 9:05 Andrej Karpathy And then we can verify that PyTorch code is identical to the C code, and everything is great, and we're just running in C. 2 times in the scene
- ▶ 18:14 Andrej Karpathy There's no need for Python, no need for PyTorch. 6 times in the scene
- ▶ 21:29 Andrej Karpathy Like, if PyTorch is, especially when push compiles, it's a bit like TCC for software, it's a compiler. 2 times in the scene
Answer.ai & AI Magic with Jeremy Howard
- ▶ 39:31 Jeremy Howard Um, and then, you know, FSTP is this very complicated library in PyTorch, which not particularly well documented.
- ▶ 44:37 Jeremy Howard Also, the PyTorch team have created this Torch AO project on quantization, um, and so there's a, yeah, big overlap now between, kind of, the fast AI and answer AI and CUDA mode communities of people, um,
[LLM Paper Club] Llama 3.1 Paper: The Llama Family of Models
- ▶ 1:10:55 unnamed speaker You can set the torch seed and the inference seed.
The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka
- ▶ 40:14 unnamed speaker So if I, like, Sling, like, SlingPyTorch, okay, great.
- ▶ 49:44 Yi Tay How much I think that people inside Google don't care about what people think outside Google, uh, like, I kind of feel like, okay, we were a bit like, like, I think, I don't think we considered, uh, uh, uh, I mean, not like, forever not… 2 times in the scene
- ▶ 1:11:57 Yi Tay A little bit of like, if you, if you propose like some very complicated thing for like, for like, easy in PyTorch.
How to train a Million Context LLM — with Mark Huang of Gradient.ai
- ▶ 26:32 Mark Huang The other PyTorch implementations outside of ECContext, they, um, they just didn't really work. 3 times in the scene
High Agency Pydantic over VC Backed Frameworks — with Jason Liu of Instructor
A Comprehensive Overview of Large Language Models - Latent Space Paper Club
- ▶ 24:12 unnamed speaker So, things like, um, the libraries that we're using, um, JAX, PyTorch, TensorFlow, amongst others, um, there's this idea of distributed training, which means that, um, can we use multiple GPUs to train our models so that we are able to…
Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI
- ▶ 0:27 unnamed speaker I was actually listening to your interview on the Gradient podcast, so if people want to know more about, like, the history of Smith, uh, the history of PyTorch, uh, they can go to that podcast. 3 times in the scene
- ▶ 4:26 unnamed speaker Yeah, we kind of touched on PyTorch in a lot of episodes. 6 times in the scene
- ▶ 10:49 unnamed speaker PyTorch has been such a Switzerland versus just making Meta hardware go bird? 11 times in the scene
- ▶ 19:47 unnamed speaker Who are some of the other FAIR, PyTorch alumni that are building cool companies? 4 times in the scene
- ▶ 29:59 unnamed speaker Like, what's the most interesting usage of PyTorch that you're seeing, maybe, outside of this little bubble? 5 times in the scene
- ▶ 1:01:43 Soumith Chintala Um, they fund a huge amount of the PyTorch, uh, development.
- ▶ 1:29:21 Soumith Chintala Yeah, I mean, first, like, ah, the very interesting reason I invested in Osmo is because Alex Wilsko, the founder of Osmo, also was, like, a, um, before PyTorch, there was Torch, and Alex Wilsko actually worked on Torch.
A Brief History of the Open Source AI Hacker - with Ben Firshman of Replicate
- ▶ 40:34 Ben Firshman So they want to be able to use this stuff without having to, like, figure out all the internals of the models and, you know, like, touch PyTorch and whatever.
- ▶ 1:16:37 Ben Firshman You don't need to be like, you know, the metaphor here is that you don't need to be digging down into like, uh, this sort of PyTorch level if you don't want to in the same way as a software engineer in the nineties.
The AI-First Graphics Editor - with Suhail Doshi of Playground AI
- ▶ 1:03:34 Suhail Doshi We don't, yeah, we don't even, no, we don't, I think we use very basic tools, like, you know, Slurm for scheduling, and just normal PyTorch, PyTorch Lightning, that kind of thing.