Jul 23, 2026 · 1h 12m · mad
Cerebras CEO: Why GPUs Can't Do Fast Inference
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Cerebras Systems Co-Founder and CEO Andrew Feldman joins Matt Turck on The MAD Podcast to discuss the architectural limits of traditional GPUs, the engineering breakthroughs behind Cerebras' massive wafer-scale engine, and why ultra-fast AI inference speed is essential for the future of reasoning models and agentic workflows.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 20.3% of the talking time here. How this is scored →
Matt as informed peer 4.5 Guest teaching 5.6 Guest disagreement 1.9 Matt pushing back 2.5
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 1:01:15 Direct rejection of CUDA moat narrative
Guest immediately rejects the host's premise that CUDA remains a formidable moat, citing that leading models like Gemini and Anthropic are trained without it and that switching cloud providers requires just eight keystrokes.
Hardest push from Matt ▶ 16:53 Devil's advocate on AI market demand and VC fundingHost directly challenges the sustainability of chip demand, questioning whether orders from AI labs are artificially inflated and financed by venture capital and private equity.
Biggest teaching moment ▶ 42:46 Detailed breakdown of memory bandwidth and decode sequentialityGuest educates the host on memory bandwidth limitations by comparing the data movement for a single 70B parameter inference token to transferring 100 HD movies from memory to compute.
Matt holds his own ▶ 58:00 Host articulates disaggregated AWS Trainium and Cerebras architectureHost demonstrates acute technical domain expertise by accurately describing how AWS Trainium handles the parallel pre-fill step while Cerebras handles the sequential decode step in a disaggregated system.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| The AI Speed Revolution and Tokens Per Second | 3 | 4 | 1 | 1 | Host sets the stage with clarifying questions on speed metrics and offers an apt UX analogy regarding broadband. Guest explains tokens per second per user and why latency matters for agentic workflows. | |
| Navigating the AI Chip Landscape and Multi-Silicon Ecosystem | 5 | 5 | 2 | 2 | Host demonstrates strong industry knowledge by citing specific silicon names like Google TPU, AWS Trainium, and OpenAI's Jalapeno project with Broadcom. Guest outlines the ASIC landscape and highlights NVIDIA's acquisition of Grok as validation of Cerebras's vision. | |
| Geopolitics, China's AI Strategy, and Edge vs. Cloud Compute | 4 | 5 | 2 | 2 | Host prompts discussion on China's AI stack, citing Huawei and DeepSeek. Guest educates on geopolitical constraints, comparing French nuclear power and Chinese grid infrastructure against US data center power bottlenecks. | |
| AI Market Dynamics and Three Hidden Hardware Shortages | 5 | 6 | 3 | 5 | Host presents a strong devil's advocate pushback on market valuation crashes and potential artificial demand driven by VC-funded labs. Guest counters the bubble hypothesis by walking through three specific hardware supply bottlenecks: HBM DRAM, TSMC CoWoS packaging, and 3nm foundry capacity. | |
| The Cerebras Origin Story and Wafer-Scale Innovation | 4 | 6 | 1 | 2 | Host guides Guest through Cerebras's origin story and its 10-year journey, probing into technical burn rates. Guest delivers a masterclass on why SRAM on wafer-scale chips eliminates the data movement bottleneck that plagues traditional GPUs. | |
| Technical Deep Dive: Why Wafer-Scale Beats GPUs at Inference | 5 | 7 | 1 | 2 | Host asks Guest to explain token generation mechanisms simply and probes into pre-fill versus decode phases. Guest explains the sequential nature of decode, illustrating that moving model weights is equivalent to shuffling 100 HD movies per generated word. | |
| Reasoning Models, Verification, and Multimodal Performance | 5 | 6 | 1 | 2 | Host asks informed questions covering RL training, model-data parallelism, verification guardrails, and multimodality. Guest details why distributed compute on GPUs requires complex tensor parallelism, whereas Cerebras's large memory footprint simplifies training flow. | |
| Cloud Strategy, Data Center Expansion, and Eradicating the CUDA Moat | 5 | 6 | 4 | 4 | Host brings up NVIDIA's CUDA moat as a potential barrier to entry. Guest directly rejects the premise, pointing out that state-of-the-art models like Gemini and Anthropic are trained without CUDA and that switching in the cloud takes eight keystrokes. |
Statements from this episode (16)
Feldman: Tokens per second per user is the right AI speed metric
“The right metric is tokens per second per user. That that's how fast you get the first token all the way through the last token in, in your response.”
Feldman: Cerebras sales were 10x higher than Groq's at acquisition
“And we were the fastest at it, and the largest, and, you know, our sales were more than 10 times the Grox, and they paid twenty billion dollars for the number two collector.”
Feldman: Cerebras signed an OpenAI compute deal worth over $20 billion
“Remember, we did a huge deal. This is probably the largest deals in Silicon Valley history north of twenty billion dollars.”
Feldman: Chinese open-source AI models trail GPT, Anthropic, and Gemini
“They are behind in chips. But their approach was at the next level is open source models where they're producing some extraordinary models. Not as good as GPT or Anthropic or Google's Gemini, but very good.”
Feldman: Agentic AI workflows are driving CPU demand through the roof
“And so, as we do more and more AI work, and more and more agentic work, we're making more and more calls to CPUs, and therefore the demand for CPUs is through the roof.”
Feldman: AI inference is bottlenecked by data movement, causing GPU slowness
“In inference in AI, it's the exact opposite. You move a huge amount of data, all the weights, from memory to compute, and you need one calculation to generate the next word. And then you have to do it again. So all the time is dominated by the movement of data…”
Feldman: Cerebras built a 46,000 square millimeter wafer-scale chip
“The biggest chip that had ever been built before us was 800 square millimeters. 840 to be exact. And this is 46,000.”
Feldman: Cerebras burned $8M monthly for 18 months before building successful chips
“And we had a, an 18 month period where we were spending eight million a month and we couldn't build them.”
Feldman: Cerebras completed the largest semiconductor IPO in history on May 14th
“And so when we rang the bell and we went public on May 14th this year and the largest semiconductor IPO in history and we did something unusual.”
Feldman: GPUs suffer from high failure rates and infant mortality
“The JPs have a huge failure rate, so I'm sure you guys have spoken about this. Infant mortality is enormous, and they fail all the time.”
Feldman: Cerebras moves weights to compute ~2,500x faster than standard GPUs
“And so the speed of moving waits to compute is about two and a half thousand times faster here than on a Wilben GP.”
Feldman: Cerebras serves second-tier AI labs for model training
“We do RL and we do traditional training too. Not for the largest models, for the largest lab, but for the next tier.”
Feldman: Leading AI labs paused video generation development due to compute costs
“Obviously, what follows that Is video, because a video is just a collection of images. But that takes an enormous amount of compute right now. And that's one of the reasons it's been sort of set aside by the leading labs. So unbelievably computation intensive.”
Feldman: Cerebras signed a 760-megawatt multi-year compute deal with OpenAI
“The deal is 760 megawatts, 250 megawatts in 26 on a multi-year lease. An additional 250 megawatts in 27, on a multi-year lease, and an additional in 28, a multi-year lease.”
Feldman: Nvidia CUDA lost 70% of frontier AI model training market share
“I think two years ago every state of the art model was trained in a Cuda flow. And right now, Gemini is trained without Cuda. Anthropical is trained without Cuda. Open AI as strange as could. So in a one or two year period, they lost 70% share. Of training mod…”