Oct 13, 2024 · 1h 21m · bg2-pod
Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner · Bg2 Pod
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of BG², NVIDIA CEO Jensen Huang joins hosts Brad Gerstner and Clark Tang to discuss NVIDIA's full-stack computing strategy, the economic transformation of $1 trillion in data center infrastructure, and the exponential growth of AI reasoning models. Huang delivers insights into hardware-software integration, rapid cluster deployments like xAI's Memphis supercomputer, open-source model dynamics, and the future of human workers acting as managers of AI agents.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Brad and Bill hold 23.8% of the talking time here. How this is scored →
Brad and Bill as informed peer 5.0 Guest teaching 5.5 Guest disagreement 1.8 Brad and Bill pushing back 1.4
speaking balance: gold is Brad and Bill, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 30:30 Jensen dismisses analyst forecasts as historic failure
Jensen forcefully calls out Wall Street analyst consensus from early 2023, declaring it the single greatest failure of forecasting the world has ever seen.
Hardest push from Brad and Bill ▶ 29:40 Brad presses Jensen on the Cisco 2000 bubble comparisonBrad directly confronts Jensen with skepticism from critics who argue that current AI infrastructure spending is an unsustainable fiber-like overbuild destined for a boom and bust.
Biggest teaching moment ▶ 13:15 Jensen educates on parallel vs serial processing architectureJensen reframes Brad's Intel analogy by demonstrating why parallel computing requires fundamentally different transistor trade-offs and specialized mathematical algorithm layers.
Brad and Bill hold their own ▶ 39:03 Brad lays out detailed OpenAI financial comparisons to Google and MetaBrad demonstrates command of market data by comparing OpenAI's revenue run-rate, user metrics, and valuation multiples directly against Google and Meta's historical IPO figures, prompting Jensen to praise his command of history.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Brad and Bill as informed peer | Guest teaching | Guest disagreement | Brad and Bill pushing back | Why |
|---|---|---|---|---|---|---|
| Welcome and Opening Remarks at NVIDIA HQ | 4 | 5 | 1 | 1 | Brad opens with friendly banter and sets the stage on scaling intelligence to AGI. Jensen takes over with a deep technical overview of how NVIDIA drove computing costs down 100,000x across the full stack. | |
| NVIDIA's Full-Stack Advantage and Competitive Moat | 5 | 6 | 2 | 1 | Clark and Brad ask about NVIDIA's moat compared to prior years. Jensen explains that viewing chips solely through FLOPs is obsolete, detailing how the entire machine learning data flywheel must be accelerated. | |
| Combinatorial Advantage vs. Custom ASICs and Intel Comparison | 5 | 7 | 2 | 2 | Brad brings up custom ASICs and compares NVIDIA to Intel at its peak. Jensen delivers an architectural breakdown of serial versus parallel computing, explaining why domain-specific libraries like cuDNN make parallel architectures unique. | |
| Extending the Moat from Training to Inference | 4 | 6 | 1 | 1 | Clark questions whether NVIDIA's moat persists in inference. Jensen outlines how training fleets naturally transition into free inference install base and describes the extreme bandwidth demands of time-to-first-token. | |
| AI Infrastructure as a Single Data Center Computer | 5 | 5 | 2 | 2 | Brad mentions his dinner with Andy Jassy regarding Amazon's Trainium/Inferentia. Jensen explains that NVIDIA views the entire data center as a single computer and undertakes immense annual integration across major cloud providers. | |
| Market Making, Ecosystem Alignment, and Supplier Relations | 4 | 5 | 2 | 1 | Brad asks about ecosystem alignment and long-term supplier relationships in Asia. Jensen clarifies that NVIDIA acts as an open market maker rather than an adversarial share taker. | |
| Addressing CapEx Concerns and the $1T Datacenter Modernization | 6 | 6 | 3 | 4 | Brad confronts Jensen with skeptic arguments comparing current AI CapEx to the 2000 Cisco telecom bubble. Jensen dismisses the comparison from first principles, arguing $1T of CPU data centers must be modernized alongside new AI factories. | |
| The Rise of OpenAI and Model Provider Economics | 7 | 4 | 1 | 1 | Brad displays deep market recall comparing OpenAI's valuation, revenue trajectory, and user metrics to Google's IPO, earning Jensen's praise. Jensen emphasizes OpenAI's velocity and the real-world adoption of AI across sciences. | |
| Dissecting the AI Stack: Models, AI, and Applications | 5 | 6 | 2 | 2 | Brad asks if open source is commoditizing model builders into consolidation. Jensen separates the abstraction layers between raw models, AI capabilities, and specialized applications, comparing it to GPUs versus accelerated computing. | |
| Building xAI's Memphis Supercluster in 19 Days | 5 | 6 | 2 | 1 | Brad asks about rumors surrounding xAI's Memphis cluster. Jensen playfully rejects the dinner causation rumor before praising Elon Musk's singular execution in deploying a 100k GPU liquid-cooled supercomputer in 19 days. | |
| Scaling Laws, Multi-Modality, and Million-GPU Clusters | 4 | 6 | 2 | 1 | Brad asks if NVIDIA's future hinges on single clusters reaching millions of GPUs. Jensen clarifies that asynchronous distributed training and massive test-time compute will drive multi-million GPU demand regardless of cluster topology. | |
| Inference-Time Reasoning and Test-Time Compute (o1 / Strawberry) | 6 | 5 | 1 | 1 | Brad discusses OpenAI's o1 reasoning model and his personal use tutoring his son. Jensen affirms that inference compute will surge by a billion times as models engage in runtime simulation and reflection. | |
| Internal AI Deployment, Company Culture, and Productivity | 5 | 5 | 2 | 1 | Brad asks about NVIDIA's internal culture, high revenue per employee, and management structure. Jensen explains how AI agents already co-design chips and refutes the premise that productivity gains inevitably cause workforce layoffs. | |
| Macroeconomic Productivity and Humans as CEOs of AI Agents | 6 | 5 | 1 | 1 | Brad cites decades of US macroeconomic productivity data to ask if AI will spark a massive expansion. Jensen agrees, noting that everyday workers will act like CEOs managing fleets of specialized AI agents just as he manages his executive staff. | |
| AI Safety, Alignment Systems, and Application-Level Regulation | 5 | 6 | 2 | 1 | Brad asks about safety coordination with Washington. Jensen argues that existing application-level agencies like the FDA, FAA, and NHTSA should regulate AI in their domains rather than a single overarching regulatory council. | |
| Open Source vs. Closed Source Models and Nemotron | 5 | 6 | 2 | 1 | Brad asks about open source versus closed source dynamics and NVIDIA's Nemotron. Jensen rejects an either/or framing and uses a padded room analogy to explain why diverse models like Nemotron are essential for reward evaluation and synthetic data. | |
| Reflection on NVIDIA's Journey, Leadership, and Staying Relevant | 4 | 4 | 2 | 1 | Brad asks Jensen whether he is having fun and how long he can sustain his intense pace. Jensen pushes back on the expectation that work must always be fun, explaining that staying relevant through continuous learning with AI keeps him going. |
Statements from this episode (45)
Huang: Personal AGI assistants will arrive soon and improve over time
“Soon in some form. Yeah, soon in some form, and that, that that assistant will get better over time. That's the beauty of technology as we know it, and so, so I think in the beginning it'll, it'll be quite useful but not perfect, and then it gets more and more…”
Huang: NVIDIA Lowered Marginal Compute Costs by 100,000x Over 10 Years
“You know, a lot of this is happening because we drove the marginal cost of computing down by a 100,000 X over the course of 10 years.”
Huang: Moore's Law Alone Would Have Delivered Only 100x Improvement
“Moore's law would have been about a hundred X.”
Huang: AI Inference and Post-Training Are Now Just as Hard as Pre-Training
“People used to think that pre-training was hard, and inference was easy. Now everything is hard.”
Jensen Huang: AI Is Incorporating Fast and Slow Thinking Models
“There must be a concept of fast thinking, and slow thinking, and reasoning, and reflection, and iteration, and simulation, and all that and that now it's coming in.”
Jensen Huang: Curating data to train AI requires AI itself
“A lot of people don't even realize that it takes AI to curate data to teach an AI. And that AI alone is pretty complicated.”
Jensen Huang: Language models will be involved in everything
“Language models are going to be involved in everything. It took us, took the industry enormous technology and effort to train a language model, to train these large language models. Now we're using a large language model in every single step of the way.”
Huang: Parallel processing favors transistor volume over individual transistor speed
“Serial processing requires every transistor to be excellent. Parallel processing requires Lots and lots of transistors to be more cost effective. I'd rather have 10 times more transistors, 20% slower. Than 10 times less transistors, 20% faster.”
Huang: Accelerated parallel computing cannot rely on generic compilers across architectures
“What people don't realize is that you can have, ah, three different ISAs, CPU ISAs, they all have their own C compilers, you could take software and compile down to that ISA. That's not possible in accelerated computing. That's not possible in parallel computi…”
Huang: cuDNN revolutionized deep learning by powering frameworks like PyTorch
“So we revolutionized deep learning because of our domain specific library called QDNN. Without QDNN, nobody talks about QDNN because it's one layer underneath PyTorch and, you know, and TensorFlow and back in the old days, CAFE and Theano and now Triton, and t…”
Huang: NVIDIA's moat in inference will be greater than in training
“And I'm sure I said it would be greater.”
Huang: Models built on NVIDIA run on NVIDIA without modification
“If you built it on this architecture, Without any consideration, it will run on this architecture, ok? You could still go and optimize it for other architectures, but at the very minimum, since it's already been architect, you know, built on NVIDIA, it will ru…”
Huang: Software updates make Hopper GPUs 2x to 4x better over time
“Now, we also put a lot of energy into continuously reinventing new algorithms so that when the time comes, the Hopper architecture is two, three, four times better than when they bought it, so that, that you, that infrastructure continues to be really effectiv…”
Huang: OpenAI recently decommissioned its Volta GPU infrastructure
“And I think Sam was just telling me that, that they had just decommissioned the Volta infrastructure that they have at OpenAI recently, and so, so I think it's, we leave behind this trail of install base.”
Huang: The entire data center is now the fundamental unit of computing
“The data center is now the unit of computing. To me, when I think about a computer, I'm not thinking about that chip. I'm thinking about this thing. That's my mental model, and all the software, and all the orchestration, all the machinery that's inside.”
Huang: NVIDIA delivers 2x-3x performance and cost improvements every single year
“We're trying to build a brand new one every single year, and every single year we deliver two or three times more performance. As a result, every single year, we reduce the cost by two or three times. Every single year, we improve the energy efficiency by two …”
Huang: NVIDIA never tracks or discusses market share internally
“NVIDIA is a market maker, not share taker. If you look at our company slides, we don't show, not one day does this company talk about market share, not inside.”
Huang: NVIDIA shares roadmaps years ahead with CSPs building ASICs
“When we work with all the GCPs Azure, we present our roadmap to them years in advance. They don't present their ASIC roadmap to us, and it doesn't ever offend us.”
Huang: Almost every software application will be highly machine-learned in the future
“Almost every single application, Word, Excel, PowerPoint, Photoshop, Premiere, you know, AutoCAD. You give me your favorite application. That was all hand, hand engineered. I promise you it will be highly machine learned in the future.”
Huang: $1T in legacy data center infrastructure must be modernized in 4-5 years
“And we just know that we have a trillion dollars with the data centers that we have to modernize. And so right now as we speak, if we were to have a trajectory over the next four or five years to modernize that old stuff, that's not unreasonable.”
Huang: The new market for AI factories will probably reach a few trillion dollars
“So there's a whole layer of computing fabric, a whole layer of what I call AI factories that the world has to make that doesn't exist today at all. So the question is how big is that? Unknowable at the moment, probably a few trillion dollars.”
Gerstner: Altimeter and NVIDIA invested in OpenAI's $6.5B round
“Open AI. Raised, as you know, six and a half billion dollars this week at like a hundred and fifty billion dollar valuation. We both participated.”
Gerstner: OpenAI has 250M weekly users, double Google at IPO
“They have two hundred and fifty million, yeah, two hundred and fifty million weekly average users, which we estimate is twice the amount Google had at the time of its IPO.”
Huang: AI will deliver immense value long before achieving AGI
“The one thing that I know is that, that AI's, AI's gonna have a roadmap of capabilities over time, and that, and roadmap of capabilities over time is gonna be quite spectacular. And along the way, long before it even gets to anybody's definition of AGI, we're …”
Gerstner: OpenAI has escape velocity to fund future models, unlike competitors
“OpenAI clearly has hit that escape velocity. They can fund their own future. It's not clear to me that many of these other companies can.”
Jensen Huang: Foundation models are necessary but not sufficient for AI
“A model is an essential ingredient for artificial intelligence. It's necessary, but not sufficient.”
Jensen Huang: GPU makers have no clue how to build accelerated computing
“Somebody who's really, really good at building GPUs have no clue how to be an accelerated computing company.”
Huang: Elon Musk Is Singular in Engineering and Large-Scale Construction
“Elon is singular in this understanding of, Engineering, and construction, and large systems, and marshaling resources.”
Huang: xAI Brought 100,000-GPU Cluster from Delivery to Training in 19 Days
“NVIDIA, NVIDIA's infrastructure and computing infrastructure and all that technology, to training, 19 days.”
Huang: A 100,000-GPU Cluster Is Easily the Fastest Supercomputer on Earth
“A 100,000 GPUs, that's, you know, easily the fastest supercomputer on the planet. That's one cluster.”
Huang: Supercomputers Normally Take Three Years to Plan and One Year to Deploy
“A supercomputer that you would build would take normally three years to plan. And then they deliver the equipment, and it takes one year To get it all working.”
Huang: Asynchronous distributed computing and federated learning will be invented for AI
“My sense is that distributed training will have to work. And my sense is that, that distributed computing will be invented. And some form of federated learning and distributed, you know asynchronous distributed computing is going to be discovered.”
Huang: The AI industry will with certainty require millions of GPUs
“If you just did that math and you compound it with, you add, you compound that with four X per year, On model size and computing size. And then on the other hand, demand continues to grow in usage. Do we think that we need millions of GPUs? No doubt. Yeah, tha…”
Huang: Scaling large AI training jobs would have failed without Megatron
“Without Megatron. That we developed with some seven years ago now. The scaling of these large training jobs wouldn't have happened.”
Huang: AI inference compute is about to increase by a billion times
“It's about to go up by a billion times.”
Huang: NVIDIA could not design Hopper, Blackwell, or Rubin chips without AI agents
“Our cyber security system today can't run without Without our own agents. We have agents helping to design chips. Hopper wouldn't be possible. Blackwell wouldn't be possible. Ruben don't even think about it.”
Gerstner: NVIDIA generates roughly $4M revenue and $2M FCF per employee
“Nvidia is in a league of its own, really you know, at about four million of revenue per employee, about two million of profits or free cash flow per employee.”
Huang: NVIDIA aims for 50,000 employees and 100M AI assistants
“I'm hoping that NVIDIA someday will be a 50,000 employee company with a hundred million, you know, AI assistants. And they're in every single group.”
Huang: AI productivity gains lead to higher corporate growth, not layoffs
“The part that is overlooked is when companies become more productive, Using artificial intelligence. It is likely that it manifests itself into either better earnings, or better growth, or both. And when that happens, the next email from the CEO is likely not …”
Huang: People will all become CEOs of AI agents
“And so, so I think that that's the thing that, that people are going to learn is that they're all going to be CEOs. They're all going to be CEOs of AI agents.”
Huang: AI should be regulated at application level by existing agencies
“It's necessary to have regulation for important technologies but it's also don't overreach to the point where some of the regulation ought to be done, or most of the regulation ought to be done at the applications. The FAA, NHTSA, FDA, you name it, right? All …”