Jul 20, 2026 · 1h 28m · news
The Open-Source AI Reality | How Token Costs Will Fall 10X & Usage Will Explode 100X | Lin Qiao
Clips from this episode (6)
Short vertical cuts produced from the tape, captions burned in. Where a cut lands on a statement from the ledger, its card says so.
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of 20VC, host Harry Stebbings interviews Lin Qiao, co-founder and CEO of Fireworks AI, exploring the economics of generative AI, the strategic imperative of open-source models over monolithic AGI, physical infrastructure constraints, and insights on scaling high-growth AI startups.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Harry holds 17.8% of the talking time here. How this is scored →
Harry as informed peer 5.7 Guest teaching 5.3 Guest disagreement 2.2 Harry pushing back 4.6
speaking balance: gold is Harry, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 32:27 Direct confrontation over routing layer defensibility
Lin resists Harry's suggestion that standalone routing is a permanently valuable independent layer, acknowledging automated self-evolving systems will replace manual routers.
Hardest push from Harry ▶ 24:16 Harry challenges SaaS moat collapse with legal enterprise realityHarry pushes back firmly against Lin's thesis that application moats have dissolved, citing lengthy enterprise legal sales cycles and custom implementation barriers.
Biggest teaching moment ▶ 1:09:00 Lin breaks down ASIC and chip tape-out requirementsLin educates Harry on hardware economics, demonstrating that custom chip design requires completely frozen, mature software workloads before committing to tape-out.
Harry holds his own ▶ 11:10 Harry calculates frontier lab valuation vs open-source realityHarry synthesizes open-source efficiency gains, cost differentials, and workflow coverage to challenge the fundamental market valuation of frontier labs.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Harry as informed peer | Guest teaching | Guest disagreement | Harry pushing back | Why |
|---|---|---|---|---|---|---|
| Lin Qiao's Career Journey and Founding Fireworks AI | 4 | 2 | 1 | 2 | Harry sets up the interview citing background diligence with Eric Vishria and Dima, asking Lin about starting a company at 48. Lin shares her technical background in distributed systems and seven years at Meta learning people management. | |
| Positioning Fireworks AI: Inference, Specialized Intelligence, and Enterprise Data | 6 | 6 | 3 | 5 | Harry questions why inference and specialized intelligence are not commodities and pushes Anthropic's enterprise moat as a counterexample. Lin counters with Jensen Huang's insight that every business is specialized because proprietary data is private. | |
| Open-Source Model Quality and the Viability of Frontier Labs | 7 | 6 | 2 | 6 | Harry asks if frontier model companies are dramatically overvalued if open-source models solve 90% of enterprise workflows at 15x lower cost. Lin details how open models allow enterprises to retain weight control and hill-climb on private data. | |
| AI Economics: Scaling to Bankruptcy, and One-Size-Fits-One Optimization | 6 | 6 | 3 | 5 | Lin outlines the concept of startups and incumbents 'scaling into bankruptcy' with high inference COGS. Harry pushes back asking if OpenAI's aggressively price-reduced frontier models eliminate the open-weight advantage. | |
| Geopolitical AI Risk, Guardrails, and Vertical Model Adoption | 6 | 5 | 2 | 5 | Harry raises national security concerns regarding Chinese dominance on open LLM leaderboards and potential export restrictions. Lin argues that every company needs custom guardrails regardless of origin and open ecosystems naturally decentralize. | |
| Disruption of SaaS Application Moats and Rapid Model Iteration | 6 | 6 | 3 | 6 | Harry challenges the idea that SaaS moats have collapsed by citing long enterprise legal sales cycles at portfolio company Leya vs Harvey. Lin points out that workflow harnesses must be co-trained with underlying specialized models to prevent errors. | |
| Base Model Step Functions, Government AI Infrastructure, and Monopoly Risks | 5 | 4 | 2 | 4 | Harry asks if foundation AI should be treated as national infrastructure or partially government-owned. Lin warns against single-company monopolies, comparing base intelligence to utilities while defending specialized intelligence. | |
| The AI Routing Layer and Autonomous Self-Evolving Systems | 7 | 6 | 4 | 7 | Harry directly probes whether routing layers like OpenRouter or Requesty will be disintermediated if companies build autonomous routing internally. Lin concedes that standalone routers might be redundant once self-evolving automated routing matures. | |
| Transitioning Beyond Coding to Enterprise Co-Work and Consumer AI | 6 | 5 | 2 | 5 | Harry asks bluntly about customer concentration risk if Cursor churns. Lin details the diversification from coding to enterprise co-work and consumer recommendation engines across multiple verticals. | |
| Token Volume Explosions and AI Supply Chain Bottlenecks | 6 | 6 | 2 | 5 | Harry reacts incredulously to Lin predicting a 20-100x explosion in token count, noting that Capex bubble fears would be groundless under that demand. Lin explains that physical supply chain bottlenecks in energy and chips throttle delivery. | |
| Startup Agility, Platform Specialization, and Enterprise AI Expenditure | 7 | 5 | 3 | 6 | Harry presses on whether AI infrastructure players must go full-stack and why Nvidia's Jensen Huang does not expand into Fireworks' platform layer. Lin clarifies that Nvidia enters models purely to clear supply bottlenecks without building cloud inference. | |
| Long-Term Token Economics and Infrastructure Cost Deflation | 6 | 6 | 3 | 6 | Harry references Marc Benioff's developer spend data and asks whether token prices will truly fall when they have remained stubbornly high. Lin explains how task-level token efficiency and hardware supply easing will lead to a 10x cost reduction driving 100x usage. | |
| Quality-First Inference, Zero KLD, and Bit-Wise Model Equivalence | 6 | 7 | 3 | 5 | Harry asks if Fireworks is viewed as more expensive than low-cost providers like Together AI. Lin reframes the pricing dynamic, introducing the concept of zero KLD and exact bit-wise equivalence between training and inference. | |
| Gross Margins in AI Infrastructure and Growth-Stage Priorities | 6 | 6 | 2 | 5 | Harry asks if 30-40% gross margins represent the permanent new normal for AI infrastructure. Lin explains that low initial margins represent an intentional constraint choice to prioritize hypergrowth and geographic expansion over premature optimization. | |
| Custom Data Center Architecture and Heterogeneous Hardware Clusters | 6 | 7 | 2 | 4 | Lin outlines hardware disaggregation between FLOPs-intensive pre-fill and SRAM-intensive token generation using Groq ASICs. Harry connects this to his background diligence with Groq CEO Jonathan Ross. | |
| Physical Data Center Execution Constraints and Global Competition | 6 | 5 | 2 | 5 | Harry asks whether rapid physical infrastructure buildouts give China a structural advantage over US permitting hurdles. Lin agrees on civil engineering velocity but notes specialized operations talent is still scarce globally. | |
| Hardware and Model Depreciation Cycles and Long-Term Margin Strategy | 6 | 7 | 3 | 5 | Lin breaks down how hardware depreciation cycles have compressed from six years to annual multi-SKU releases, mismatching rapid model obsolescence. Harry synthesizes the implication that model speed outstrips hardware depreciation schedules. | |
| Market Dynamics and Owning Intelligence vs. Renting | 5 | 5 | 2 | 5 | Harry asks whether inference markets are winner-take-all like ridesharing or multi-player like hyperscale cloud. Lin argues that enterprises inevitably transition from renting general intelligence to owning and optimizing proprietary models. | |
| Sovereign Models and National AI Independence | 6 | 7 | 3 | 6 | Harry asks why startups avoid chip manufacturing if Meta, OpenAI, Anthropic, and DeepSeek build their own custom silicon. Lin educates him on silicon workload stabilization: taping out chips requires unchanging model architectures that premature startups cannot freeze. | |
| System Level Infrastructure and Architecture Bottlenecks | 6 | 6 | 2 | 5 | Harry cites Groq's Jonathan Ross claiming HBM memory bandwidth is the primary bottleneck. Lin counters that the broader architectural bottleneck is system-level co-design across 10-trillion parameter clusters, celebrating Fireworks hitting $800M run-rate potential. | |
| Strategic Executive Hiring and AI Leadership Traits | 5 | 4 | 1 | 3 | Harry inquires into the hiring of former Salesforce president George Hu. Lin explains how maintaining high curiosity alongside deep executive seniority allowed them to accelerate go-to-market without bureaucratic ossification. | |
| Quick Fire: Scaling Velocity and Culture of Extreme Ownership | 4 | 3 | 1 | 2 | In the quick fire round, Harry shares 20VC hiring profiles while Lin articulates the culture of extreme ownership and hiring talent that rejects artificial organizational boundaries. | |
| Quick Fire: Leadership Lessons from Jensen Huang | 4 | 4 | 1 | 2 | Lin reflects on leadership lessons learned from Jensen Huang, highlighting instant responsiveness and context-driven decision making. She shares her past mistake of underinvesting in marketing. | |
| Quick Fire: Untapped Enterprise Segments | 4 | 3 | 1 | 2 | Lin outlines untapped opportunities in traditional enterprise segments like insurance and banking, concluding with her core thesis that every enterprise must eventually own its software intelligence stack. |
Statements from this episode (43)
Lin Qiao: A single-company monopoly on AI makes no sense
“What I don't want to see is there's only one company owns intelligence. I think that doesn't make sense to me.”
Lin Qiao: Collaborative AI tools define the current market shift
“I think last year is the year of coding, and this year is the year of co-work.”
Stebbings wrote a $10M check for Fireworks after a 15-minute meeting
“This is 20 VC with me, Harry Stebbings, and in the hot seat today, a founder who I wrote a ten million dollar check for after just a 15 minute meeting, Lin Kuo, founder of Fireworks.”
Lin Qiao: AI token costs will drop 10x in three years, driving 100x usage
“I do think the cost of token will go down drastically. 10 X cost reduction in the next three years, and this 10 X cost reduction will drive a hundred X usage.”
Lin Qiao: Fireworks AI will not enter the application layer
“We absolutely are not going to move into application layer. Very clear to us.”
Fireworks hired former Salesforce president George Hu
“Lin just hired George Hu. He was the former president of Salesforce.”
Stebbings: Fireworks can become a $500 billion company
“If so, fireworks, honestly, It can be a five hundred billion dollar company.”
Lin Qiao: Most world data is private enterprise data, not public internet
“If you think intelligence is a derivative of data, then majority of the data is actually not used for training a general intelligence model. The training data is coming from public internet and the label data. Public internet is very small. Corpus of data comp…”
Lin Qiao: Enterprise data will never be shared with third parties
“It will never get shared with anyone else because this is company's proprietary IP.”
Lin Qiao: The future of AI is private, specialized intelligence
“We believe the future of the frontier of the intelligence, actually private intelligence, our specialized intelligence.”
Qiao: Anthropic relies on a single-AGI belief over specialization
“I view Anthropic as a company fully believing AGI.”
Lin Qiao: Corporate Uniqueness and User Data Intelligence Cannot Be Replicated Externally
“Every single company is doing something unique that justify their existence, and this something unique is deeply baked into their product design, is deeply baked into their software design and system building, and that's deeply baked into the data and the inte…”
Fireworks AI uses open-source models internally for hiring, finance, and coding
“So within, ah, within Firewalls, obviously, we dock for our own product. We use OpenModel to drive our recruiting process, candidate sourcing, and the feedback collection. We use OpenModel to even drive some internal finance processes. Obviously, for coding, w…”
Lin Qiao: Fine-tuned open models outperform general models on specialized enterprise tasks
“And the model intelligence has password threshold is much easier to steer, especially with small amount of data. A small amount of unique data a particular company has, and then we can hill climb towards your eval, and often time, the end result of hill climbi…”
Lin Qiao: Product-market fit no longer guarantees business durability in AI SaaS
“During SaaS time, product market fit and the durable business almost are equivalent to each other. The hardest thing is find product market fit, and then once you find it just scales as fast as you can. Because CPU is a commodity. The infrastructure you build …”
Lin Qiao: Fireworks has achieved 5x to 10x inference cost reductions
“What we have seen in the past is five times to 10 times cost reduction.”
Lin Qiao: AI future will feature millions of specialized models, not AGI
“I really believe the future will not be a few small number of AGI models dominating the world. I really believe the future will be, it may be scary, but I think that's true, it will be millions of specialized models, one per application, per use case.”
Lin Qiao: US can build its own open-source AI ecosystem if China restricts access
“I do believe, in terms of talent density and resources, I do believe U.S. Will be able to build that open system by ourselves, and we should.”
Lin Qiao: AI coding models have eliminated software implementation as a competitive barrier
“In the past, it requires tens of very strong product engineers and PMs to convert from idea to implementation to production scale. Multiple quarters of years of investment. That's a deep mode. And today, one person, a few weeks, can possibly launch their ideas…”
Lin Qiao: Almost all AI coding startups now fine-tune custom models
“In coding space, Cursor probably is one of the pioneers starting to tune their model, and now almost all coding companies tune their own models.”
Lin Qiao: Major AI base IQ leaps will occur every 9-12 months
“I see those as every year or every three quarters, there's a major leap.”
Qiao: Enterprise AI frontier is routing architecture, not single models
“So again, my thinking of what is the frontier is not just this one model. The frontier could be your special routing mechanism for your business, and you decompose that based on, hey, in order to fulfill this task, and you need a highly intelligent layer, mayb…”
Qiao: AI model routing will evolve into fully automated self-learning systems
“We also think there's a space to build a automatic routing system that can learn by itself, and that compound with automatic tuning system eventually. We think it should all be automated, and then you can see a self-evolving system based on what flow through y…”
Qiao: Standalone model router services like OpenRouter won't be needed long-term
“You probably don't. We're not there yet, but I do think this is, this could be area of innovation.”
Lin Qiao: Fireworks runs distributed RL across 5-6 global data center regions
“We've designed a fully distributed system. We run across five, six data center regions globally, and tap into scattered GPUs, and they are able to run massive jobs, our jobs.”
Lin Qiao: All major AI coding companies use Fireworks infrastructure
“I think all major coding companies are on us.”
Lin Qiao: Fireworks processes over 40 trillion tokens daily
“Today we process more than 40 trillion tokens a day. So majority of those tokens are coming from a customized model, not from off-the-shelf models, are coming from a customized model.”
Qiao: Fireworks daily token count could grow 20x-100x by next year
“Anywhere ranging from 20 to a hundred X could be possible.”
Qiao: Cursor was a single-digit million-dollar company before 1000x growth
“I would say Curso is the first company they have decided to work with us. Early on. I still remember when they worked with us, they were single digit million dollar. Very small. That's only two years ago. They grew by a hundred, a thousand X over two years.”
Lin Qiao: token costs will fall drastically and cheaper infrastructure will invite far more usage
“I do think the cost of token will go down drastically. High price will invite a lot of people coming in to solve the problem, and it will invite competition. Competition will bring down the cost, and then eventually will lead into a very economical solution, r…”
Fireworks achieves exact bit equivalence between AI training and inference
“Between the training system and the inference system, when models move over, We have bit equivalence. So, as in, the numerics are fully the same. We do not lose a bit of accuracy.”
Qiao: Fireworks could optimize gross margins now but chooses rapid expansion
“If our focus is only optimized growth margin, we absolutely can do that, but we are sacrificing the speed of growth because we want to go everywhere.”
Lin Qiao: AI hardware vendors now release three SKUs per year
“And now within a year from one vendor alone, we have three SKUs.”
Lin Qiao: Legal AI market consolidated from many startups to two
“Take legal, for example. I was on a dinner table, and interesting, it seems like there were a lot of those companies around two years ago, but now it's pretty much two.”
Lin Qiao: Nations will build sovereign AI models like power grids
“I definitely see that possibility. I also see, if we think about the general intelligence model as the electricity layer, as a power line, every country should own their own power line, right?”
Meta Has Been Building Custom Silicon Chips for Over Five Years
“I think Meta has been building their chips for more than five years, way more than five years.”
Qiao: Generative AI workloads are too dynamic to justify custom silicon
“We're still in the early stage of workload maturity for now to warrant a chip that will be durable.”
The AI Industry Lacks Systems Designed for 10-Trillion Parameter Models
“Great system designed for 10 trillion parameter models today.”
Stebbings: 20VC hires mostly immigrants because British people lack strong work ethic
“Pretty much only hire immigrants. British people don't work very hard.”
Qiao: AI market will shift from token maxing to ROI maxing
“In the next couple of years, as AI is getting more and more into production, there will be a lot of focus. In getting that clarity and getting that discipline out. The token maxing is just a thing in time, but we're quickly moving to ROI maxing, which is about…”