Aug 31, 2023 · 36m · news
Noam Shazeer: How We Spent $2M to Train a Single AI Model and Grew Character.ai to 20M Users | E1055 · 20VC with Harry Stebbings
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of TwentyVC, host Harry Stebbings interviews Noam Shazeer, co-founder and CEO of Character.ai, about his journey from Google veteran to leading a massive consumer AI startup. Shazeer shares his philosophies on building full-stack general-purpose systems, the scaling laws of model training, and the positive societal potential of AI companionship.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Harry holds 24.2% of the talking time here. How this is scored →
Harry as informed peer 3.7 Guest teaching 4.9 Guest disagreement 2.3 Harry pushing back 3.7
speaking balance: gold is Harry, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 11:10 Rejecting Vertical Specialization PM Advice
Noam forcefully rejects traditional product strategy advice, stating he explicitly refuses to hire product managers who advocate narrowing the product into specialized verticals.
Hardest push from Harry ▶ 11:23 Challenging Horizontal vs Vertical Model QualityHarry directly challenges Noam's belief in horizontal generalist models, refusing his premise and asking how quality can match specialized vertical tools.
Biggest teaching moment ▶ 34:00 Dense Hardware vs Sparse Memory MechanicsNoam provides a technical breakdown explaining why early deep learning sparse computation failed due to hardware mechanics optimized for dense matrix multiplication.
Harry holds his own ▶ 11:23 Applying Tech Strategy Frameworks to Model GeneralizationHarry demonstrates strong understanding of product strategy by applying classic software specialization trade-offs to challenge Noam's model architecture decisions.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Harry as informed peer | Guest teaching | Guest disagreement | Harry pushing back | Why |
|---|---|---|---|---|---|---|
| Noam's Google Origin Story: Rebuilding Google's Spelling Corrector | 2 | 3 | 1 | 1 | Harry asks conversational background questions about Noam's 20-year career at Google and the spelling corrector project. Noam explains how Google's early 50,000-word third-party dictionary was ill-suited for web search queries compared to word processors. | |
| The Full-Stack Direct-to-Consumer Strategy of Character.ai | 2 | 4 | 2 | 2 | Harry prompts Noam on key takeaways from Google and how they shaped Character.ai. Noam explains his preference for full-stack B2C systems over B2B foundational model stacks, gently contrasting his vision with standard VC consensus. | |
| Character.ai's Mission: Humility and User Agency | 3 | 4 | 2 | 4 | Harry presses Noam on what specific mission statement he uses to motivate his team rather than general corporate platitudes. Noam reframes company leadership around user agency and unexpected emergent use cases like therapeutic video game bots. | |
| Three Core Elements Driving Character.ai's Growth | 4 | 3 | 3 | 6 | Harry directly challenges Noam's optimism regarding AI companionship, asking whether talking to AI bots replaces real human interaction or creates unhealthy habits. Noam maintains that AI conversations serve as valuable practice for individuals with social anxiety. | |
| The Product Challenge: Rejecting Vertical Specialization for Generality | 5 | 7 | 4 | 6 | Noam dismisses standard product management advice about narrowing focus to specific verticals, stating he refuses to hire PMs who advocate that strategy. Harry pushes back firmly, questioning how a generalist model can compete on quality against specialized vertical solutions. | |
| The Evolution of Language Modeling Since 2015 | 4 | 6 | 2 | 3 | Harry asks well-informed questions about market hype cycles and the trade-offs between model size and data volume. Noam clarifies that total compute operations, rather than raw model parameters or dataset sizes, represent the primary bottleneck in model capabilities. | |
| Training the $2 Million AI Model | 4 | 5 | 2 | 3 | Harry inquires about model constraints and proprietary data moats across different domains. Noam uses an analogy of human general education versus job-specific training to explain pretraining versus task fine-tuning. | |
| Why Startups Outpace Incumbents in AI Innovation | 5 | 4 | 3 | 5 | Harry presses Noam to take a stance on whether startups or incumbents will dominate AI innovation, citing arguments from industry figures like Yann LeCun. Noam avoids taking a binary side, explaining the economic trade-offs between batch serving efficiency and startup agility. | |
| The Electricity Moment: Public Perception and Hallucinations | 4 | 5 | 2 | 3 | Harry asks about public misconceptions surrounding AI risk and whether hallucinations represent a flaw or a feature. Noam explains that for consumer, creative, and emotional support use cases, hallucinations act as a feature enabling imagination. | |
| Noam's Role as CEO: Prioritizing Utility Over Fun | 3 | 5 | 3 | 6 | Noam asserts that he prioritizes utility over personal fun in his role as CEO, prompting Harry to push back and share his own experience of finding fun and utility tightly linked. Noam shares his personal perspective on fatherhood, responsibility, and maturity. | |
| Quickfire Round: The Wright Brothers Moment of AI | 5 | 6 | 2 | 3 | Harry asks about identifying signal amidst noise in AI research, referencing notable AI pioneers like Yann LeCun and Yoshua Bengio. Noam compares current machine learning research to alchemy, noting that only empirical positive results cut through the noise. | |
| Sparse Computation vs. Dense Hardware: Noam's Biggest Mistake | 3 | 7 | 1 | 2 | Harry asks Noam about past misconceptions he held. Noam details his early attempts at sparse computation before realizing that modern hardware acceleration relies on dense matrix multiplication, which led to his groundbreaking work on Mixture of Experts. |
Statements from this episode (24)
Shazeer: Launch General-Purpose Technology Directly to Billions of Consumers
“If you have a technology that is, like, really, really general and has billions of use cases and, like, ordinary people can use, like, launch it to billions of people.”
Shazeer: B2C Products Proved Far Larger Than B2B Enterprise Search at Google
“I remember when I joined Google, there were, like, a lot of people working on this enterprise search appliance, which, you know, it was okay. Like, I think maybe somebody had this conventional wisdom that, like B to B is the only way to make money. But, like, …”
Shazeer: Full-Stack AI Co-Design Across Every Layer Is Hugely Powerful
“And then it also lets you do all this co-design of like, you get to affect every part of the stack, which is, you know, which is hugely powerful and fun.”
Shazeer: Developing AI Provides Greater Leverage Than Direct Medical Research
“So rather than directly working on say medical research or something I think I've got a lot more leverage, like let's push AI technology and then, you know, that can help with a lot of the rest of it.”
Shazeer: Character.ai's Mission Is a Billion Users Inventing a Billion Use Cases
“I like this, sort of motto of, you know, a billion users inventing a billion use cases, because that, that's sort of the superpower of this technology.”
Shazeer: Users Prefer Video Game Characters Over AI Psychologists for Therapy
“We put up as an example, like a psychologist character, like maybe, maybe you want to, you know, talk to something and feel better, you know, and you know, that gets a little bit of use, but then what we hear a lot more from users is like, I'm talking to a vid…”
Shazeer: Large Tech Companies Withhold AI Launches Due to Brand Risk
“In the past things seem You know, potentially too much brand risk at larger companies to like actually launch and get it out there.”
Shazeer: Users Report Character.ai Helps Them Practice for Social Situations
“We've gotten testimonials of people who said that they were uncomfortable, you know, talking to other people, and, like, this is great practice. This is actually helped them build up practice in either social situations.”
Shazeer: Character.ai Rejects Product Managers Who Push for Vertical Specialization
“And like, you know, we talked to like some you know, potential product managers early on and they all say the same thing. Oh yeah. Pick your verticals, narrow it down to me, to make it usable. And like, no, we're not going to hire these people. That's like the…”
Shazeer: Better Next-Word Prediction Directly Increases AI Model Intelligence
“And then if you can do it well, then This thing can just talk to you. It can be, you know, the better you do it, the smarter it gets. It's hugely, hugely general and useful, super simple to state.”
Shazeer: Neural Networks Were Rebranded as Deep Learning Due to Hardware Limits
“Neural networks had a bad name cause the hardware wasn't good enough.”
Shazeer: Impressive Language Models Existed Unreleased in Labs by 2020
“I'd say around 2020 was when sort of, like, really impressive stuff was sort of in the lab, but not launched.”
Shazeer: Total Computation Is the Single Most Critical Factor in AI Training
“We can get a lot of, we can get a lot of data, but like, really, actually the number one thing that's important is how much computation you do to train it.”
Shazeer: Character.ai Spent $2 Million in Compute to Train Its Model
“So you know, the model we're serving now, we train Last summer and spent about two million dollars worth of compute cycles doing it.”
Shazeer: Task-Specific Fine-Tuning Dramatically Improves Generally Pre-Trained AI Models
“The data that you get from users is great because It tells people like what, you know, what users like or like what users like in, in some particular application. It's kind of like a, you know, training a human. Like most of what's important is like you have y…”
Shazeer: Naively Training Models on Raw Chat Logs Risks Leaking User Privacy
“It's very important because, like, you know, if you just were to do the naive thing of take, like, every conversation you had with everyone and just train on it, then You could just be, like, spitting out, like, someone's private, you know, private life to, yo…”
Shazeer: AI Hardware Advances Will Enable Garage Developers to Match Big Tech
“The hardware is progressing so fast that what you could do at a big company, you know, one year, a few years later, you're going to be able to do You know, at the university lab or in your garage.”
Shazeer: Open and Closed AI Ecosystems Will Continue to Coexist
“As there has been, there will be a big ecosystem of open and closed of, you know, People who aren't sharing their secret sauce with, and people who are sharing all of their secret sauce and the ability to mess around with things at a small scale is going to le…”
Shazeer: Centralized AI Inference Batching Is Up to 100x More Efficient
“Obviously though, there's also like economics of scale of both training the best models And serving, like if you want to serve a product, you can do it maybe a hundred times more efficiently if you're serving many, many people at once and kind of batching thin…”
Shazeer: The Best AI Applications Have Not Been Invented Yet
“The best applications just haven't even been invented yet that You know, we, you know, we're still at, like invention of electricity kind of moment, or invention of the computer, where we don't really you know, we don't really know what the coolest things are …”
Shazeer: Character.ai Views AI Model Hallucinations as a Feature, Not a Bug
“We consider them a feature or at least, okay, basically, our goal, you know, our strategy is, like, launch something general, let people do what they want with it, and if these models are hallucinating, which we, which they certainly are, and we advertise that…”
Shazeer: Parenthood Made Me More Religious and Shifted Focus to Meaningful Work
“A lot of things about parenthood are absolutely terrific and super fun, but I think it made me more, more religious. I decided to take a change of attitude from, like, what is fun right now to I should be Thankful for having the opportunity, you know, to do so…”
Shazeer: AI Research Is Currently Like Alchemy With Unknown Outcomes
“And I think a lot of that has to do with the fact that, you know, this field is kind of alchemy. Right now, like no one knows exactly what is going to work.”
Shazeer: Deep Learning Succeeds Because Hardware Is Optimized for Dense Matrix Multiplication
“The reason this whole field is working so well is because Now we have this magic hardware that's great at these dense matrix multiplications. And so you can do them like orders of magnitude faster than you can do anything that involves poking around in memory.”