Chintala: No AI company currently feels they have sufficient compute
Soumith Chintala · Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI · Mar 6, 2024 · at 49:43
Meta AI Engineering Lead Soumith Chintala discusses compute scarcity across frontier AI labs and tech companies.
0:00 / 0:24exact quote · 24.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“If you don't have enough compute, you figure out how to make do with smaller models, but like, no one as of today, I think, would feel like they have enough compute. I don't think, like, I've heard any company within the AI space be like, oh yeah, like, we feel like we have sufficient compute, and we couldn't have done better.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Soumith Chintala
Chintala: George Hotz's TinyGrad requires major breakthroughs to match PyTorch
“There's no, like, I don't think, like, unless we have, like, great breakthroughs, like, George's vision is achievable, like, or, like, he should be thinking about a narrower problem, such as, I'm only gonna make this for, like, work for self-driving car con ne…”
Chintala: Apple's MLX will fail server-side due to lack of differentiation
“If they end up expanding onto the server side, and they'll probably build something like PyTorch as well, right? Like, eventually, that'll where it will land. And I think there, they will kind of fail on the, like, lack of differentiation. Like, it wouldn't be…”
Chintala: Inference Moats from Fast CUDA Kernels Last Only Months
“I think, like, Together and Fireworks and all these people are trying to build some faster CUDA kernels and faster, like, you know, hardware kernels in general. But those modes only last for a month or two. Like, these ideas quickly propagate.”
Chintala: Simple AI frameworks must accept long compile times
“You can write a very simple framework but then you also should be willing to eat the long compile times of, like, searching for that optimal performance at runtime.”
Chintala: Nvidia's primary competitive moat is NVLink interconnect, not GPU silicon
“The mode that Nvidia has right now, I feel like, is that they're, they have the interconnect that no one else has. Like, AMD GPUs are pretty good. I'm sure there's very silicon that is not bad at all, but, like, the interconnect like, NVLink is uniquely awesom…”
Chintala: LLM inference market will become a low-margin laundromat business
“My view of the LLM inference market in general is that it's like the laundromat model. Like you, the margins are going to drive down towards the bare minimum, like It's gonna be all kinds of arbitrage between how much you can get the hardware for, and then how…”