Nov 7, 2024 · 36m · no-priors
No Priors Ep. 89 | With NVIDIA CEO Jensen Huang
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
NVIDIA CEO Jensen Huang discusses the transition to full-stack accelerated computing, the rise of industrial-scale AI factories, and how agentic flywheels and embodied robotics are transforming science and technology.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 12.3% of the talking time here. How this is scored →
The hosts as informed peer 3.1 Guest teaching 3.4 Guest disagreement 1.1 The hosts pushing back 0.2
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 19:07 Jensen rejects single-bottleneck framing
When Sarah presents a multiple-choice list of potential scaling bottlenecks, Jensen bluntly dismisses narrowing it down by stating 'Everything' is hard and nothing about these scales is normal.
Hardest push from the hosts ▶ 19:15 Sarah counters Jensen's bottleneck pessimismAfter Jensen insists that nothing about megacluster scaling is normal or easy, Sarah immediately pushes back with 'But nothing is impossible.'
Biggest teaching moment ▶ 10:56 Jensen clarifies NVIDIA already builds full data centersIn response to Sarah asking if NVIDIA might eventually build full data centers, Jensen explains that NVIDIA already builds end-to-end supercomputing centers to validate software rather than relying on PowerPoint specs.
The host holds their own ▶ 5:58 Elad presents proprietary token price compression dataElad demonstrates rigorous market analysis by citing his team's internal findings that GPT-4 equivalent inference token costs dropped 240x in just 18 months.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Full-Stack Co-Design, Data Center Networking, and CUDA Acceleration | 5 | 4 | 1 | 0 | Elad demonstrates domain expertise by introducing proprietary research on the 240x drop in token costs for GPT-4 equivalents over 18 months. Jensen elaborates on the underlying hardware dynamics, explaining Dennard scaling limits, full-stack co-design, and NVLink virtual GPUs. | |
| Infrastructure Disaggregation and the Hierarchy of AI Models | 3 | 4 | 1 | 0 | Sarah asks about infrastructure fungibility between training and inference workloads. Jensen educates the hosts on how older training infrastructure naturally cascades down into inference and model distillation. | |
| The Data Center as the New Unit of Computing | 2 | 4 | 2 | 0 | When Sarah asks if NVIDIA should build entire data centers, Jensen gently corrects the premise by stating they already build complete operational supercomputer data centers internally before disaggregating components for customers. | |
| Rapid Engineering and Deployment of xAI's Colossus Supercluster | 2 | 3 | 0 | 0 | The hosts prompt Jensen on the rapid bringup of xAI's Colossus cluster. Jensen provides an admiring, highly detailed breakdown of the staging, simulation, and hardware deployment logistics. | |
| Scaling Bottlenecks and the Multi-Vendor Race for General Intelligence | 3 | 3 | 2 | 2 | Sarah offers multiple choices for the primary scaling bottleneck (capital, energy, supply), but Jensen rejects the framing by declaring 'Everything' is abnormal. Sarah interjects to remind him that nothing is impossible. | |
| The Paradigm Shift to AI Factories and Token Generation | 4 | 4 | 1 | 0 | Elad frames NVIDIA's exponential market cap surge with detailed financial comparisons. Jensen reframes the business model, explaining that computing has shifted from storage data centers to token-producing AI factories. | |
| Embodied Robotics and Enterprise Agent Ecosystems | 2 | 4 | 2 | 0 | Jensen counters the common industry narrative that enterprise SaaS platforms will be disrupted by AI, arguing instead that incumbents like SAP and ServiceNow sit on gold mines for autonomous domain agents. | |
| The Groundswell of Generative AI Across All Scientific Disciplines | 2 | 4 | 1 | 0 | Jensen provides an authoritative monologue on the underappreciated groundswell of generative AI in hard sciences, drawing parallels to his early observations of AlexNet in computer vision. | |
| Universal Algorithmic Foundations and Personal AI Utilization | 5 | 1 | 0 | 0 | Sarah articulates a high-level thesis about universal algorithmic foundations bridging computer science across physical scientific disciplines, which Jensen enthusiastically validates before sharing his personal AI workflow. |
Statements from this episode (21)
Huang: AI compute will follow hyper Moore's Law over the next decade
“Over the next 10 years our hope is that we could double or triple performance every year at scale. Not at chip. At scale. And to be able to, therefore, drive the cost down by a factor of two or three, drive the energy down by a factor of two, three every singl…”
Huang: Scaling AI Compute Requires Controlling Both Algorithms and System Hardware
“The new way of doing scaling are all kinds of things associated with co-design unless you can modify or change the algorithm to reflect the architecture of the system or change and then change the system to reflect the architecture of the new software and go b…”
Huang: NVIDIA Bought Mellanox to Treat Data Center Networks as Compute Fabrics
“That, that's the reason why we bought Mellanox and started fusing InfiniBand and MVLink in such an aggressive way.”
Gil: GPT-4 Equivalent Token Costs Dropped 240x in 18 Months
“Over the last 18 months or so, the cost of a million tokens going into a GPT-IV equivalent model has basically dropped 240 X.”
Huang: NVIDIA Improved Hopper Performance on LLaMA 5x in One Year
“CUDA made it possible for us to iterate so quickly, just in the last year, and then we just went back and benchmarked when Lama first came out, we've improved the performance of hopper by a factor of five without the layer on top ever changing.”
Huang: Sam Altman recently decommissioned OpenAI's Volta GPUs
“Sam was just telling me that he had decommissioned Volta just recently.”
Huang: ChatGPT is largely inferenced on same systems used for training
“And most of ChatGPT I believe are inferenced on the same type of systems that we're trained on just recently.”
Huang: Tiny language models will achieve superhuman performance in narrow domains
“And we're going to see superhuman tasks in one little tiny domain from a little tiny, tiny, tiny model.”
Jensen Huang: The new unit of computing is the entire data center
“You know, I say that the new unit of computing is the data center.”
Huang: NVIDIA plans to build at least five more supercomputers in 2025
“Next year, we're going to build easily five more.”
Huang: NVIDIA has never abandoned a single piece of software
“We've never given up on a piece of software.”
Huang: xAI's 100,000 GPU cluster is the largest single unit built
“Decide to build this 100,000 GPU super cluster, which is, you know, the largest of its kind in, in one unit.”
Jensen Huang: NVIDIA can set up complete data centers in 30 days
“If you're interested in a data center and just have to give me a space and some power, some cooling, You know, and we'll, we'll help you set it up within, call it, 30 days.”
Huang: No laws of physics limit scaling superclusters to 1M GPUs
“Nothing is, yeah, no laws of physics limits but everything is going to be hard.”
Huang: NVIDIA could not have built Hopper without AI chip designers
“We can't, we couldn't build Hopper without it.”
Jensen Huang: Computing marginal costs fell 1,000,000x over the last decade
“We've driven down the marginal cost of computing down probably by a million X in the last 10 years, to the point that we just, hey, let's just let the computer go exhaustively write the software.”
Jensen Huang: AI data centers are token-producing factories, not storage
“These new data centers we're creating are not data centers. They don't, they're not multi-tenant. They tend to be single tenant. They're not storing any of our files. They're just, they're producing something and they're producing tokens.”
Huang: The world is close to artificial general robotics
“In a lot of ways, we've, we're close to artificial general intelligence, but we're also close to artificial general robotics.”
Huang: Incumbent SaaS platforms will flourish with AI agents, not be disrupted
“People, people say that these SaaS platforms are going to be disrupted. I actually think the opposite. That they're sitting on a gold mine, that, that they're going to be this flourishing of agents that are going to be specialized in Salesforce specialized in,…”
Huang: In 2-3 years, every science breakthrough will use generative AI
“If we give ourselves another couple, two, three years, the world's going to change. There's not going to be one paper There's not going to be one breakthrough in science, one breakthrough in engineering where generative AI isn't at the foundation of it. I'm fa…”
Jensen Huang: I don't learn anything without first consulting AI
“I don't learn anything without first going to an AI, you know. Why learn the hard way? Just go directly to an AI. I go directly to ChatGPT or, you know, sometimes I do perplexity just depending on just the formulation of my questions, and I just start learning…”