Sep 25, 2023 · 17m · a16z
Universally Accessible Intelligence with Character.ai's Noam Shazeer
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In an a16z stage interview, Character.AI Co-founder and CEO Noam Shazeer discusses how consumer-driven conversational AI accelerates the path to Artificial General Intelligence (AGI). Alongside a live interactive comparison with an AI chatbot trained on his persona, Shazeer explores compute scaling, full-stack model architecture, and the transition into an era of universally accessible intelligence.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The host as informed peer 3.5 Guest teaching 4.3 Guest disagreement 1.8 The host pushing back 1.8
speaking balance: gold is the host, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 4:16 Rejection of the global AGI safety pause premise
Noam cheekily deflects the existential risk question by proposing a pause only long enough to bring more H100 GPUs online.
Hardest push from the host ▶ 10:07 Questioning generalist models versus niche domain modelsSarah uses VC market data from a16z to challenge Noam on why a single model beats domain-specific startups in areas like mental health.
Biggest teaching moment ▶ 13:15 Fermi estimation of global compute capacityNoam breaks down exact hardware counts and floating-point operations per second to educate the host on global compute math per human.
The host holds their own ▶ 3:53 Contextualizing scaling laws across industry peersSarah demonstrates domain expertise by framing Noam's take on scaling laws alongside identical positions from Mira Murati at OpenAI and Dario Amodei at Anthropic.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| Title Sequence and Important Disclosures | 2 | 3 | 2 | 1 | Sarah introduces the guest and asks standard friendly icebreaker questions regarding Duke and leaving Google. Noam explains early LLM developments at Google, internal corporate renaming, and leaving due to big company brand risk. | |
| Question 3: Scaling Laws and AGI Existential Risk | 3 | 4 | 2 | 2 | Sarah connects Noam's scaling perspectives with other industry leaders like Mira Murati and Dario Amodei. Noam dismisses safety pause doom-mongering with a joke about needing four months to acquire more H100s and outlines progress in training costs. | |
| Consumer Engagement and Parasocial AI Relationships | 4 | 5 | 3 | 2 | Sarah presents precise platform engagement metrics including 20 billion messages and 2-hour daily usage averages. Noam reframes the entertainment industry as parasocial imaginary friends and argues model hallucination is a feature rather than a flaw for companions. | |
| Generalist vs Specialised Domain AI Models | 5 | 4 | 2 | 3 | Sarah uses a16z startup insights to question Character's generalist model approach compared to specialized edtech or mental health startups. Noam explains why specialized rule-tuning fails to generalize and defends full-stack model development. | |
| Theory of Mind and Global Compute Scaling | 4 | 6 | 1 | 1 | Sarah quotes research on Theory of Mind and references Noam's quote on universally accessible intelligence. Noam performs back-of-the-envelope calculations on global Nvidia H100 output to demonstrate compute accessibility. | |
| Fundamental Breakthroughs and Compute Value | 3 | 4 | 1 | 2 | Sarah pushes on whether current Transformer scaling is sufficient or if new breakthroughs are needed to reach AGI. Noam highlights that operational compute costs are vastly lower than human time value. |
Statements from this episode (16)
Shazeer: Dialogue is the world's number one pastime
“Dialogue, like it's world's number one pastime”
Shazeer is not currently afraid of AGI destroying humanity
“No. Not yet. Not yet. I think there's a lot of possibility of you know, a lot of potential benefits and yeah, we're going to work on it as the technology improves.”
Character.ai's model cost $2M to train, now $500k
“The model we're serving now, we, you know, cost us about, like, two million dollars worth of compute cycles to train last year, and could probably repeat it for, like, half a million now.”
Character.ai will launch a significantly smarter model by late 2023
“We're going to launch something, tens of IQ points smarter hopefully by the by the end of the year.”
Entertainment is a $2T industry built on imaginary friends
“Entertainment is like this two trillion dollar a year industry. And like the dirty secret is that entertainment Is imaginary friends that don't know you exist.”
LLM hallucination is a feature, not a bug, for AI companions
“Like you want to launch something that's a doctor, It's going to be a lot slower because you want to be really, really, really careful about not providing like false information, but friend, you can do like really fast. Like it's just entertainment. It makes t…”
AI companion entertainment is ready for immediate explosive adoption
“If I, like, I want to push this technology ahead fast, like, that's what I want to go with, because like, A, it's, you know, it's ready for an explosion, like right now, not like in five years when we solve all the problems, but like, now.”
Narrow AI use cases tempt builders into non-generalizable rules
“The more you get to like mission critical, you know, particular use case, the more you get tempted into like writing particular rules and like doing things that will not generalize well.”
Character.AI aims for AGI by building consumer products at scale
“Our goal is to be like an AGI company and a product first company. And the way to do that is by picking the right product that forces us to work on the right things, things that generalize, make the model smarter, make it, like, do what people, you know, what …”
Frustration over inability to launch AI products drove Google exit
“You know, some people are motivated by publishing, you know, like, I was frustrated I couldn't launch at Google.”
Theory of mind in AI is an emergent property of scale
“Yeah, just make the thing smarter. It's gonna have a better theory of mind, and I think that's definitely something massively, massively important. It seems like one of these emergent properties that just Is gonna, going to come with scale.”
AI will improve massively from scaling without new breakthroughs
“And without any breakthroughs, like it's going to get like massively better as everyone just Kind of scales up to use it, and there will be more breakthroughs because now, you know, like all the scientists in the world are like working on like making this stuf…”
Top-tier AI capabilities will reach academic labs in a few years
“Like, you know, we're going to see like a huge amount of innovation and, you know, what, what's possible in the largest companies now can be possible in, you know, in somebody's academic lab or garage in a few years.”
AI capable of curing cancer is only a few years away
“I'd love to get to the point where you can just ask it how to cure cancer or something, you know. I mean, it's, you know, it seems a few years away for now, but, you know, like.”