Assertion Supported AI assessment confidence: 95% certainty 4/5 debate potential 1/5

Huang: Andrew Ng contacted NVIDIA around 2012 to train neural nets on GPUs

Jensen Huang · No Priors Ep. 13 | With Jensen Huang, Founder & CEO of NVIDIA · Apr 25, 2023 · at 11:40

Jensen Huang is the founder and CEO of NVIDIA, discussing the early inflection points of deep learning with Elad Gil and Sarah Guo.

0:00 / 0:22exact quote · 22.9s

▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark

“Around 20 12, I guess, and it was because simultaneously Andrew Eng reached out to Bill Daly, our chief scientist to work on a way to get the neural network model that they were working on onto GPU so that they could, instead of using thousands of CPU servers, they could use a few GPUs.”

quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →

The claim, plainly

Around 2012, Andrew Ng reached out to NVIDIA chief scientist Bill Dally to get neural network models running on GPUs to replace thousands of CPU servers with a few GPUs.

a quote alone can be ambiguous, so this restates it with the references and context of the conversation around it resolved. AI-written; the tape below the player is the source.

Assessment

Historical accounts from then-NVIDIA Chief Scientist Bill Dally, researcher Bryan Catanzaro, and published research papers confirm that around 2011–2012, Andrew Ng and Bill Dally discussed running deep neural network training on GPUs to replace large CPU clusters. Following discussions regarding the Google Brain system (which utilized 1,000 machines and 16,000 CPU cores), Dally assigned Catanzaro to collaborate with Ng's Stanford group. This collaboration culminated in an ICML 2013 paper demonstrating that commodity GPU systems could replicate and scale Google's 1-billion-parameter CPU-based model using just 12 to 64 GPUs.

Bill Dally's 2023 SVEC Hall of Fame statement detailing his 2010–2011 discussions with Andrew Ng and assigning Bryan Catanzaro to replicate Ng's cat-finding model on GPUs instead of 16,000 CPUs
Stanford and NVIDIA's joint ICML 2013 paper ('Deep learning with COTS HPC systems') authored by Adam Coates, Bryan Catanzaro, Andrew Ng, and others demonstrating matching and scaling Google's CPU cluster performance with 3 to 16 GPU servers
Bryan Catanzaro's documented research timeline confirming Bill Dally tasked him in 2011–2012 to collaborate directly with Andrew Ng's group at Stanford

Checked against

[1] svec.is [2] svec.is [3] caltech.edu [4] caltech.edu [5] unitbv.ro [6] semanticscholar.org [7] twimlai.com [8] principus.si

"Deep learning with COTS HPC systems" "Bryan Catanzaro" "Bill Dally" "Andrew Ng" "Stanford" GPU "Google" "Andrew Ng" "Bill Dally" GPU 16000 1000 machines "Bill Dally" "Andrew Ng" "cat finding" "16,000" "Deep learning with COTS HPC systems" "16,000" "Google" "Bryan Catanzaro" "Bill Dally" "Andrew Ng" GPU "Bill Dally" "Andrew Ng" "breakfast" GPU

assessed as of 2026-09-03

This assessment is an AI reading of a public statement, made against the cited references with the confidence shown. It can be wrong. The quote, the tape and the references are all on this page; if something is off, use report an error below. full disclaimer →

Mentioned

Andrew Ng Bill Daly GPU Nvidia

⚑ Spot an error? Report it

More from Jensen Huang

Opinion

Huang: DeepSeek paper was Silicon Valley's most important recent read

“This is, you know, let's face it, Deep Seek was probably the single most important paper that most Silicon Valley researchers read from in the last couple years.”

Jensen Huang Jan 8, 2026 ▶ 33:30 NVIDIA’s Jensen Huang on Reasoning Models, Robotics, and Refuting the “AI Bubble” Narrative

Opinion

Huang: Tech companies should not lobby governments to regulate competitors

“As a policy, as a practice, I don't think companies ought to go to governments to advocate for the regulation on other companies and other industries, just in practice. Their intentions Are clearly deeply conflicted. And their intentions are clearly, you know,…”

Jensen Huang Jan 8, 2026 ▶ 25:56 NVIDIA’s Jensen Huang on Reasoning Models, Robotics, and Refuting the “AI Bubble” Narrative

What-if

Huang: Without energy growth policies, US would have lost AI revolution

“If not for this entire change in, in sentiment, About energy growth in our country. We can all concede now. We would have handed this industrial revolution to somebody else.”

Jensen Huang Jan 8, 2026 ▶ 54:27 NVIDIA’s Jensen Huang on Reasoning Models, Robotics, and Refuting the “AI Bubble” Narrative

Prediction Not checkable as stated

Huang: Natural gas is probably the only viable power solution this decade

“We're going to need nuclear. Wind is not going to be enough. Solar is not going to be enough. Let's just all acknowledge that we'll take it. We'll take everything we can. But the fact that matters, I think for the next decade, natural gas, you know, is probabl…”

Jensen Huang Jan 8, 2026 ▶ 55:25 NVIDIA’s Jensen Huang on Reasoning Models, Robotics, and Refuting the “AI Bubble” Narrative

Prediction Not checkable as stated

Huang: Chinese military is unlikely to use American tech

“And so that American technology, although general purpose it is unlikely to be used by their military because their military is too smart, just as our military is too smart to use their technology.”

Jensen Huang Jan 8, 2026 ▶ 1:01:05 NVIDIA’s Jensen Huang on Reasoning Models, Robotics, and Refuting the “AI Bubble” Narrative

Assertion Contradicted

Huang: China contributes more to open source than any other country

“No country in the world contributes more to open source than China.”

Jensen Huang Jan 8, 2026 ▶ 1:04:09 NVIDIA’s Jensen Huang on Reasoning Models, Robotics, and Refuting the “AI Bubble” Narrative

Huang: Andrew Ng contacted NVIDIA around 2012 to train neural nets on GPUs

More from Jensen Huang

Turn any episode into a week of clips.