Jan 29, 2025 · 1h 0m · news
Jonathan Ross: DeepSeek Special - How Should OpenAI and the US Government Respond | E1253 · 20VC with Harry Stebbings
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this emergency episode of 20VC, Groq co-founder Jonathan Ross joins host Harry Stebbings to analyze the technical and geopolitical implications of China's DeepSeek model, arguing that model commoditization will dismantle proprietary software moats and fundamentally reshape the economics of AI compute.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Harry holds 15.8% of the talking time here. How this is scored →
Harry as informed peer 4.6 Guest teaching 5.3 Guest disagreement 2.2 Harry pushing back 3.0
speaking balance: gold is Harry, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 7:48 Ross refutes the copycat narrative
Ross firmly interrupts and rejects Harry's framing that Chinese companies simply duplicate Western models, arguing they created genuinely innovative reinforcement learning techniques.
Hardest push from Harry ▶ 30:51 Harry challenges Ross on taking steroids in an AI arms raceHarry aggressively pushes back on Ross's moral stance against IP theft, arguing that competing in a global AI race against adversary nations requires pragmatism rather than idealism.
Biggest teaching moment ▶ 42:09 Ross explains why inference will dwarf training computeRoss educates Harry on the fundamental economics of AI hardware, showing why 95% of long-term compute spend will shift to inference rather than model training.
Harry holds his own ▶ 38:15 Harry shares his Nvidia stock buy thesis following the 16% dipHarry demonstrates strong financial and industry expertise by explaining how he bought Nvidia shares during a massive panic drop, capitalizing on the exact inference demand thesis discussed.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Harry as informed peer | Guest teaching | Guest disagreement | Harry pushing back | Why |
|---|---|---|---|---|---|---|
| Emergency Episode Setup & Conversational Banter | 2 | 3 | 1 | 1 | Harry welcomes Ross and asks broad intro questions about DeepSeek and Ross's background. Ross playfully praises Harry's podcast go-to-market strategy before setting the scene with a Sputnik metaphor. | |
| Unpacking the Mechanics of AI Model Distillation | 3 | 6 | 1 | 1 | Ross delivers a technical breakdown of scaling laws, AlphaGo Zero self-play, and how DeepSeek distilled OpenAI models. Harry asks clarifying questions to help unpack distillation for listeners. | |
| Debunking the Chinese Copycat Narrative | 4 | 5 | 3 | 3 | Ross firmly refutes the narrative that Chinese AI merely copies Western models, pointing out simple automated reinforcement learning innovations. Harry pushes back with specific questions on reward modeling research. | |
| Export Control Loopholes and Accidental Subsidies | 5 | 5 | 2 | 4 | Harry cites Josh Griffin's tweet regarding US export law violations to challenge Ross's premise. Ross explains how cloud remote access creates a massive loophole in GPU export controls. | |
| The Reality of Data Privacy with Chinese AI | 3 | 6 | 2 | 3 | Harry cuts through indirectness to ask if DeepSeek is a CCP control instrument. Ross explains soft data deletions in tech and how CCP compliance works for Chinese tech entities. | |
| Groq's Response: Privacy-Safe DeepSeek Hosting | 4 | 5 | 2 | 2 | Ross outlines Groq's hardware privacy guarantees, explaining that Groq stores no data on hard drives. Harry brings up Hamilton Helmer's 7 Powers framework, finding common ground. | |
| Demystifying the $500 Billion Stargate Plan | 5 | 6 | 3 | 4 | Harry asks whether DeepSeek's efficiency ridicules Sam Altman's $500B Stargate vision and cites Jensen Huang's revenue split. Ross counters by drawing on Google TPU history to argue $500B is actually not enough spending for inference. | |
| Carrot vs. Stick: Countering Chinese AI Subsidies | 4 | 5 | 2 | 3 | Harry introduces the parallel of Chinese EV subsidization disrupting Europe. Ross advocates for automated reciprocal subsidies and notes that open-source models invariably beat proprietary ones. | |
| The Existential Threat to OpenAI's Business Model | 4 | 5 | 2 | 3 | Harry questions if open sourcing would cannibalize OpenAI's core revenue. Ross explains how OpenAI could leverage its dominant brand power to switch to open source from a position of strength. | |
| Mapping Moats: Meta and Microsoft | 5 | 5 | 2 | 3 | Harry rejects standard cloud provider comparisons, pointing out that LLMs lack switching costs. Ross maps Hamilton Helmer's 7 Powers to Big Tech incumbents like Microsoft and Meta. | |
| Geopolitical Risk-Taking: The US vs. Europe and China | 6 | 4 | 4 | 7 | Harry forcefully challenges Ross's principled stance against corporate theft, arguing that in a global AI race you must take 'steroids' if your opponent does. Ross maintains moral repulsion toward IP theft while admitting government intervention may be required. | |
| Unleashing Europe: Building a Thousand Station Fs | 5 | 5 | 2 | 2 | Harry references visiting Station F in Paris. Ross proposes that European policymakers create 1,000 Station F hubs to inject risk-taking entrepreneurial culture across the continent. | |
| The VC Panic: Why Foundation Model Startups Must Pivot | 5 | 5 | 2 | 3 | Harry voices venture capital anxiety over foundation model write-downs. Ross candidly advises foundation model startups to pivot toward product and user experience, highlighting Suno and Perplexity as examples. | |
| Jevons Paradox in AI Compute | 8 | 6 | 3 | 4 | Harry displays strong market conviction by sharing that he bought Nvidia stock during a 16% market drop based on an inference usage thesis. Ross explains Jevons Paradox using 1860s steam engines and Satya Nadella's tweets. | |
| Training vs. Inference: The Tech Industry's Misconception | 5 | 7 | 2 | 3 | Ross details why inference will account for 95% of AI compute long-term and explains Mixture of Experts (MoE) sparsity in DeepSeek R1 versus Llama. Harry asks probing questions on Nvidia's defensibility. | |
| Meta's Fine-Tuning, Synthetic Data, and Locked Signups | 4 | 6 | 2 | 2 | Ross explains how Meta achieved performance gains via fine-tuning on high-quality data rather than retraining from scratch, and notes that DeepSeek locked signups due to inference compute shortages. | |
| Stargate, Scale Economies, and OpenAI's Brand Moat | 5 | 5 | 3 | 4 | Harry bluntly asks if Stargate's $500B figure is bullshit. Ross breaks down the capEx math and argues OpenAI's true defensible moat lies in its consumer brand equity. | |
| The Threat of Automated Zero-Day Exploits | 5 | 7 | 2 | 2 | Harry asks about Elon Musk's xAI and seeks clarification on zero-day exploits. Ross warns about automated nation-state cyberattacks enabled by LLMs and deniable warfare. | |
| Where Does Value Accrue? Wrapper Apps & Craftsmanship | 6 | 5 | 2 | 4 | Harry cites specific developer tools like bolt.new and lovable to question where value accrues if app creation becomes trivial. Ross cites Charles Eames on craftsmanship and product detail. |
Statements from this episode (56)
Ross: DeepSeek spent far more distilling OpenAI than on training
“They spent a lot more distilling or scraping the OpenAI model.”
Ross: OpenAI will lose the closed model battle and should open source
“I can't speak for Sam Altman or OpenAI, but if I was in that position, I would be gearing up to open source my models in response. Because it's pretty clear you're gonna lose that, so you might as well try and win all the users and the love from open sourcing.”
Ross: Most AI companies rely on identical data providers
“Most people actually, most don't realize this. Most companies have access to roughly the same amount of data. They buy them from the same data providers.”
Ross: DeepSeek's ultra-low training budget claims are false marketing
“This model was supposedly trained on a smaller number of GPUs and a much, much tighter budget I think the way that it's been put is less than the salary of many of the executives at Meta, and that's not true. It's actually, there's an element of marketing invo…”
Ross: DeepSeek Used Better Training Data Than Meta Via OpenAI Distillation
“Now, more recently, Meta has been training on more GPUs, but Meta hasn't been using as much good data as DeepSeq, because DeepSeq was doing reinforcement learning using OpenAI.”
Ross: High-Quality Data Lets AI Models Beat Traditional Token Scaling Laws
“What most people don't realize is that assumes that the data quality is uniform. If the data quality is better, then you can actually get away with training on fewer tokens.”
Ross: DeepSeek scraped OpenAI data while developing unique RL techniques
“And all of that said, they did a lot of really innovative things. So that's what makes it so complicated because on the one hand, they kind of just scraped the OpenAI model. On the other hand, they came up with some unique reinforcement learning techniques,”
Ross: OpenAI does not need to distill DeepSeek because OpenAI remains superior
“They don't need to because they're actually better still. They're a little bit better. So they could, but why would they?”
Ross: Renting cloud GPUs is the biggest loophole in export controls
“Why would they try and smuggle in GPUs when all they'd have to do is log into any cloud provider and rent GPUs? This is like the biggest gaping hole in the whole way that export control is done. You can literally log in. You can swipe credit card, whatever, an…”
Ross: OpenAI accidentally subsidized DeepSeek via unprofitable API tokens
“Open AI Was effectively subsidizing accidentally the training of this model because they were using open AI, right? And, you know, rumors are that open AI may not be completely profitable yet in terms of every token in the API, like on the subscriptions maybe,…”
Ross: Groq blocks IP addresses originating from China
“One of the problems, so we actually block IP addresses from China.”
Ross: IP blocking fails to enforce AI export restrictions
“It's also a little bit fruitless because, you know, someone could just like rent a server anywhere and then log into us from there. Right? And then there's nothing we can check. So I don't know that IP addresses are really the right way to do it anyway. I thin…”
Ross: Groq decided in 2016 to avoid China for commercial reasons
“And so, in 2016, when Grok started, we decided that we were not going to do business in China. This was not a geopolitical decision. This was purely commercial.”
Ross: Foreign companies in China cannot extract net profits
“And the formula is actually pretty simple. You're not allowed to make net money. You're allowed to spend more money in China. But the moment that you start to become profitable or anywhere near profitable, all of a sudden there's a thumb on the scale.”
Ross: Chinese laws force companies to hand over data and censor content
“But at the same time, they also require that you hand over all data. And not only that, they also require that certain answers be in a form that they find acceptable.”
Ross: Groq previously refused to run Chinese AI models
“So we up until recently refused to run any Chinese models.”
Ross: Groq hardware has no hard drives and stores zero data
“So we store nothing. Like, there is no, like, delete or whatever, like, there is just, we store nothing. We don't even have hard drives, right? Like, we just, we have DRAM, and when the power goes off, everything goes away, right?”
Ross: CCP will pressure DeepSeek to harvest user data
“The CCP is probably going to be taking the safeties off the weapons. They're going to be like, why are you making this model open source? Please direct your data towards us. Go win a bunch of customers this way, but now we want the data, right? And so they're …”
Ross: DeepSeek was created by a hedge fund, not the state directly
“Remember deep seek is a real, I mean, it's a hedge fund. They're doing this themselves and they're just influenced by the CCP”
Ross: DeepSeek R1 won't be talked about in six months
“Are we going to be talking about DeepSeq for the next six, or R-one for the next six months? And the answer is absolutely not.”
Ross: DeepSeek R1 proves base AI models are commoditized
“This has just made it absolutely nakedly clear that the models are commoditized, right?”
Ross: OpenAI's unmatched brand power will sustain them long-term
“They've got amazing brand power, like no, no one else in this space, and that's gonna serve them for a really long time, right?”
Ross: $500B AI infrastructure plan is not enough spending
“Actually, I don't think it's enough spending.”
Ross: Google spent 10x to 20x more on AI inference than training
“At Google, we always ended up spending 10 to 20 times as much on the inference as training back when I was there.”
Ross proposes automated matching subsidies to deter foreign AI and tech intervention
“There needs to be some sort of Automated response of like, if you do this, we will respond. If you subsidize this industry, we will automatically subsidize the equivalent industry. Just automatic. So don't do it because there's no benefit to you.”
Ross: OpenAI is losing its pricing power to DeepSeek
“Especially for the pricing, because they're losing their pricing power on this.”
Ross: OpenAI Could Switch Backend to DeepSeek and Retain Users
“Most people think of them synonymously as AI. They could just switch to deep seek and people would still use them. It's brand. It's one of the seven powers.”
Ross: LLMs Have Zero Switching Costs Compared to Linux
“Linux has switching costs, and I think what we've discovered is LLMs have no switching costs whatsoever.”
Ross: Meta Will Not Scrape OpenAI Data Like DeepSeek Did
“The question is, are they willing to scrape open AI like DeepSeek did? And I don't think they are. I, they've been super careful on everything that they've been doing.”
Ross: AI Competitors Will Set Aside Morals to Win
“And I think that's going to happen. I think people will like you cannot lose.”
Ross: European risk aversion hampers tech ecosystem relative to US
“So what I, for me, you know, watching everything, it feels like with Europe, there's a lack of a willingness to take risk, right? There's a black mark if you get it wrong, like everything's about downside protection. Whereas in the U S it's like, that was a gr…”
Ross: Chinese tech relies on state-backed IP theft from Western firms
“And then you look at China, China practices R D T research development theft. It's just part of the culture. And it's not just against Western company. It's against each other too. The difference is if you're a Western company, then the government Steals from …”
Ross: Europe should build 1,000 Station F startup hubs by next year
“I would say by the end of this year, you should have 100 station Fs. And by the end of next year, you should have a thousand.”
Jonathan Ross: AI models are engines, products are cars
“Model is it's a piece of machinery. It's an engine, but what is the car? What is the experience?”
Ross: Generative AI Is Fundamentally Different From the Internet
“Is AI the next internet? And I'm like, absolutely not, because the internet is an information age technology. It's about duplicating data with high fidelity and distributing it. It's what telephone does. It's what internet does. It's what the printing press di…”
Ross: Perplexity Is Perfectly Positioned as AI Hallucination Rates Drop
“So when I look at perplexity as being perfectly positioned for the moment that the hallucination or really confabulation rate comes down because the moment that these models get good enough where you don't have to check the citations anymore, that's going to o…”
Ross: Compute costs dropped 1,000x per decade while spending rose 100x
“So for the last five to six decades, like clockwork, once a decade, the cost of compute has gone down a thousand X. People buy 100,000 X as much compute spending a hundred times as much. So every decade they spend a hundred times as much. So you make it cheape…”
Ross: Cheaper AI models trigger net spikes in Groq developer count
“And so what's really happening is every time one of these models gets cheaper, we see our developer count just skyrocket. It just like goes up and then it comes back down a little bit, but the slope is higher than when it started.”
Ross: Better AI models create a self-reinforcing compute demand loop
“So better models create more demand for inference. Then has people going, I should train a better model, and the cycle continues.”
Harry Stebbings bought Nvidia stock following post-DeepSeek market drop
“I just bought a shitload of NVIDIA, because they dropped 16% on the thesis that the increasing efficiency means that obviously we wouldn't need as much NVIDIA chips.”
Ross: Nvidia is more valuable because of DeepSeek, not less
“It's actually more valuable. Thanks to deep seek, not less valuable.”
Ross: AI token cost drops consistently spark significant demand growth
“And so what's happened is every time we've seen the cost of tokens for a particular level of quality of models come down, we've actually seen the demand grow significantly.”
Ross: Specialized AI inference chip startups benefit NVIDIA's stock
“I don't know that Nvidia will ever see it this way, but I do think that those of us focusing on inference and building stuff specifically for that are probably the best thing that's ever happened for Nvidia stock because we'll take on the low margin, high volu…”
Ross: The entire AI industry will shift to Mixture of Experts architecture
“What you're going to see is everyone else starting to use this MOE approach.”
Ross: AI companies will use massive GPU clusters to generate synthetic data
“What you're going to see now is now that everyone has seen this deep seek architecture, they're going to go great. I have hundreds of thousands of GPUs. I'm now going to use a lot of them to create a lot of synthetic data. And then I'm going to train the bejes…”
Ross: DeepSeek restricted signups due to a shortage of inference compute
“So, they ran out of compute, and this is why, this is the other reason why chip startups are gonna do just fine, because they ran out of inference compute.”
Ross: OpenAI will have a much stronger brand in three years
“Much stronger. I think they're going to double down on that, and they're going to focus on it.”
Ross: Generative AI disruption will happen faster than future tech waves
“The generative age, we're gonna speed run it faster than whatever comes next, because we know what it looks like.”
Ross: xAI's hardware strategy is sound, but proprietary model bet looks worse
“I'd feel better about my bet on building out more hardware. I would feel worse about trying to build out my own model.”
Ross: Google announced first zero-day exploit found by an LLM
“Google just announced recently the first zero day exploit found by an LLM that was previously unknown.”
Ross: Cyber defense must be automated to withstand AI attackers
“And now the defense has to be automated, because there's no way to keep up with automated attackers.”
Ross: Software creation will shift to non-coding prompt engineers
“Now you're not going to have to implement things. You're going to be able to prompt engineer your way through things just as we moved from hardware engineers to software engineers and sped up productivity, right? You're now just going to be able to have a prom…”
Ross: As AI software creation becomes trivial, craftsmanship will command value
“And so what you're going to see now is because it's so easy to come up with something that just kind of works. It's a little embarrassing, but it kind of works. People are really going to value well-crafted, high quality, products.”