Patel: NVIDIA's inference moat relies on hardware rather than software
Dylan Patel · AI Semiconductor Landscape feat. Dylan Patel | BG2 w/ Bill Gurley & Brad Gerstner · Bg2 Pod · Dec 23, 2024 · at 13:53
Dylan Patel (SemiAnalysis) explains to Bill Gurley and Brad Gerstner why NVIDIA's software moat is weaker in AI inference compared to training.
0:00 / 0:07exact quote · 7.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“NVIDIA's moat in, in inference is actually A lot smaller on software but it's a lot bigger on, hey, they just have the best hardware.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Dylan Patel
Patel: Every semiconductor company except NVIDIA is terrible at software
“I would say every semiconductor company in the world sucks at software except for NVIDIA, right?”
Patel: AMD lacks software talent and refuses to fund internal GPU clusters
“AMD is really good, but they're missing software. AMD has no clue how to do software, I think. They've got very few developers on it. They won't spend the money to build a GPU cluster for themselves so that they can develop software.”
Patel: Initial NVIDIA GPU cloud deployments suffer a 5% failure rate
“Google's brought in a level of reliability that NVIDIA GPUs don't have. You know, the dirty secret is to go ask people what the reliability rate of GPUs is in the cloud or in a deployment. It's like, oh God, it is not, they're reliable-ish, like, but like, esp…”
Patel: Meta and Microsoft may take free cash flows close to zero
“I think Meta and Microsoft may even take their free cash flows close to zero and just spent.”
Patel: NVIDIA's Jensen Huang only plans 12 to 18 months ahead
“Well, the funny thing is a lot of people at NVIDIA will say Jensen doesn't plan more than a year or year and a half out. Because they change things and they'll deploy them out that fast, right? No semi, every other semiconductor company takes years to deploy, …”
Patel: NVIDIA is cutting Blackwell margins to compete with custom ASICs
“Like with Blackwell, not only is it way, way, way faster, anywhere from 10 to 15 times on really large models for inference, because they've optimized it for very large language models, they've also decided, hey, we're gonna cut our margin, too, somewhat, beca…”