Jan 26, 2025 · 1h 13m · latent-space
Outlasting Noam Shazeer, Crowdsourcing Chai AI w/ 1.4m DAU — with William Beauchamp, Chai Research
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Chai AI founder and CEO William Beauchamp discusses scaling an AI entertainment platform to 1.4 million daily active users through crowdsourced model training, high-efficiency inference architectures, and user-generated conversational agents.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 3.5% of the talking time here. How this is scored →
The hosts as informed peer 5.4 Guest teaching 5.3 Guest disagreement 3.4 The hosts pushing back 2.5
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 8:30 William dismisses monolithic AGI scaling advocates
William directly rejects Sam Altman's and Elon Musk's AGI race framing, asserting that believing compute scaling alone yields infinite intelligence reflects a total lack of practical machine learning experience.
Hardest push from the hosts ▶ 1:10:50 Swix presses William on OpenAI o1 architecture claimsSwix refuses to let William state as fact that OpenAI uses explicit tree search in o1/o3, challenging him on whether it was ever confirmed or merely implied.
Biggest teaching moment ▶ 33:20 William breaks down rapid human evaluation loopsWilliam educates the hosts on how traditional 30-day retention A/B testing is far too slow for consumer AI, explaining how Chai compressed iteration cycles to 3-hour evaluations across 50 models daily.
The host holds their own ▶ 54:10 Swix supplies the 'super knowledge' taxonomySwix draws on his interview with Exa AI to provide the precise conceptual framework—super knowledge versus super intelligence—that reframes and anchors William's argument regarding LLM memory versus reasoning.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| From Poker and Quantitative Trading to Artificial Intelligence | 4 | 3 | 2 | 1 | Swix connects with William over their shared quantitative finance backgrounds, citing firms like Susquehanna and Renaissance Technologies. William explains his transition from college poker to automated trading and his eventual pivot away from crypto into language models. | |
| Decentralized Platform Ecosystems vs. Monolithic AI Architecture | 1 | 7 | 5 | 0 | William delivers a strong monologue contrasting monolithic AI scaling theories with distributed market platforms. He rejects Sam Altman's and Elon Musk's scaling narratives, claiming they represent the perspective of people who haven't done practical machine learning. | |
| Discovering Product-Market Fit in Conversational Social AI | 4 | 5 | 2 | 1 | Alessio and Swix probe how Chai evolved from an initial agent API hosting news and recipe bots to conversational roleplay. William explains discovering product-market fit through his sister's therapy bot and opening bot creation to end-users. | |
| Chai AI Origins and Rivalry with Character AI | 5 | 5 | 3 | 2 | The hosts discuss rivalry with Character AI and DeepSeek's low-cost training feats. William details how Character AI mimicked Chai's product-market fit after raising massive venture capital, forcing Chai to optimize inference efficiency and performance per dollar. | |
| Key Inflections in Scaling to 1.4 Million DAU | 6 | 5 | 2 | 2 | Swix inquires about specific inflection points on Chai's DAU growth chart. William details breaking Firebase scalability limits at 500k DAU, compressing evaluation feedback loops from 30 days to 3 hours, and scaling paid acquisition via an ex-ByteDance growth lead. | |
| Feature Failures, Killing Voice, and Expanding UGC Moats | 6 | 5 | 4 | 4 | Swix defends audio and companion multimodality, questioning whether Chai's voice failure was due to inferior model quality. William pushes back firmly with Steve Jobs product principles, asserting that consumer problems lie in diverse user-generated content rather than voice wrappers. | |
| Chaiverse Developer Platform and Model Blending Mechanics | 6 | 6 | 2 | 2 | Alessio and Swix ask how Chaiverse crowdsources fine-tunes and crowdsourced models. William describes their 5,000-completion LMSYS-style evaluation setup and explains why simple random request blending across complementary models outperforms complex routing. | |
| Super Knowledge vs. Super Intelligence and AGI Timelines | 7 | 6 | 5 | 3 | William argues LLMs are physics-like token simulators rather than reasoning engines. Swix introduces the distinction of 'super knowledge' vs 'super intelligence' from a prior podcast guest, which William enthusiastically adopts while dismissing immediate AGI timelines. | |
| Human Feedback Evaluations and Content Diversity Frontiers | 7 | 5 | 4 | 5 | Swix openly registers his disagreement on William's reasoning claims and challenges the efficacy of a single global ELO score across distinct user archetypes. William counters that human entertainment preferences are largely correlated and that platform diversity must come from creator tooling. | |
| Chai Grants, Inference Engineering, and Startup Culture | 8 | 6 | 5 | 5 | The conversation covers startup culture, custom inference kernels with MK1, and rejection sampling. Swix and William clash over whether OpenAI's o1 uses tree search and whether search over candidate completions qualifies as genuine reasoning. |
Statements from this episode (35)
Beauchamp: Small trading funds hold an advantage exploiting niche market anomalies
“If you have a fund of ten million dollars, if you find a little anomaly in the market that you might be able to make a hundred K a year from, that's a one percent return on your ten million fund. If your fund is a hundred K, that's a hundred percent return, ri…”
Beauchamp: Quant trading firm made £5 million annually with 15-person team
“The company was making about five million pounds a year, and it was just me and a team of, say, 15, like, Oxford and Cambridge-educated mathematicians and physicists.”
Beauchamp: Crypto's only killer use cases are gambling and evading regulations
“As far as a gambling device is like the most fun form of gambling invented in like ever. Super fun. I thought as a way to evade monetary regulations and banking restrictions, I think it's also absolutely amazing. So it has two like Killer use cases.”
Beauchamp: ML model performance plateaus near human-level on an S-curve
“With machine learning, first of all, you see that the performance of the models follows an S-curve. So it's not like it just goes off to infinity, right? And the S curve, it kind of plateaus around human level performance.”
Beauchamp: Almost no 2010s ML reached superhuman performance except AlphaGo
“There was almost nothing that went superhuman except for something like AlphaGo.”
Beauchamp: AI future belongs to distributed specialized platforms, not monoliths
“There must exist a platform where a small team can produce an AI for a unique purpose and they can iterate and build the best thing for that.”
Beauchamp: AI is 10x better at non-judgmental conversation than informative tasks
“The thing that AI is 10 X better at is a sort of a conversation Right? That's not intrinsically informative, but is more about an opportunity. You can say whatever you want. You're not going to get judged. If it's three AM, you don't have to wait for your frie…”
Beauchamp: Interacting with AI prevents the post-consumption remorse of social media
“With old school social media, you're just consuming passively, right? So you'll just swipe. If I'm watching TikTok, just like swipe and swipe and swipe. And even though I'm getting the dopamine of like watching an engaging video, there's this other thing that'…”
Beauchamp: Chai unlocked growth by letting consumers build bots with GPT-J
“The realization that developers and software engineers aren't interested in building this sort of AI, but the consumers are. Right. And rather than me trying to guess every day, like what's the right bot to submit to the platform? Why don't we just create the …”
Beauchamp: Chai reached 100,000 DAUs as top App Store AI app
“I think in even 20, I think late twenty-twenty-two, was it late twenty-twenty-two or maybe early twenty-twenty-three, Chai was like the number one AI app in the app store. So we would have something like a 100,000 daily active users.”
Beauchamp: Bootstrapped Chai to 100k DAUs with £2 million personal investment
“I was the only person who'd invested. I'd invested maybe two million pounds in the business, and, you know, from that we were able to build this thing, get to say a 100,000 daily active users”
Beauchamp: DeepSeek slashes inference costs by shrinking KV cache
“What DeepSeek have achieved that's quite special is they've got this amazing inference engine. They've been able to reduce the size of the KV cache significantly. And then by being able to do that, they're able to significantly reduce their inference costs.”
Beauchamp: Chai users average 90-minute sessions and 150 messages
“With Chai, let's say a typical user session is 90 minutes, which is like, you know, is very, very long. For comparison, let's say the average session length on TikTok is 70 minutes. So people are spending a lot of time, and in that time they're able to send, s…”
Beauchamp: Real AI progress is performance per dollar, not raw benchmarks
“For us, it doesn't make sense to think of AI as just the absolute performance. So if you look at like the MMLU score or the, you know, any of these benchmarks that people like to look at, If you just get that score, it doesn't really tell, tell you anything. C…”
Beauchamp: Chai tripled users and more than doubled revenue in 2024
“Users grew by a factor of three last year. Revenue over doubled.”
Beauchamp: 5-person Chai AI team ships over 100 LLMs per week
“Every day we're able to, I mean, we evaluate between 20 and 50 models, LLMs, every single day, right? So even though we've got, only got a team of, say, five AI researchers, they're able to iterate a huge quantity of LLMs, right? So our team ships, let's just …”
Beauchamp: Chai cut AI model evaluation feedback loop to three hours
“And so we were able to get that 30 day feedback loop all the way down to something like three hours.”
Beauchamp: Chai spends $40,000 daily on paid user acquisition
“Right now we're spending 40,000 dollars a day on user acquisition. That's still only half of what, like, Character AI or Talkie may be spending.”
Beauchamp: Character.AI launched voice at least nine months after Chai
“They launched it. I think they launched it at least nine months after us.”
Beauchamp: Only 10% to 15% of Chai users engaged with voice
“And it was something like only 10 or 15% of users even clicked the button to like, they wanted to engage the audio, and they would only use it for 10 or 15% of the time.”
Beauchamp: AI content should be created and trained by users, not Silicon Valley
“The number one problem for users in AI is this. All the AI is being generated by middle-aged men in Silicon Valley, right? That's all the content. You're interacting with this AI. You're speaking to it for 90 minutes on average. It's being trained by a middle-…”
Beauchamp: 5,000 completions needed to accurately ELO rank community models
“We think it takes about 5000 completions to get an accurate signal.”
Beauchamp: Dynamic LLM routing is expensive and offers minimal performance edge
“You can try and do, like, a routing thing where you say, for a user, given user requests, we're going to try and predict which of these end models that users enjoy the most. That turns out to be pretty expensive and not a huge source of, like, edge or improvem…”
Beauchamp: Randomly blending specialized LLMs delivers an effective 80/20 user experience
“How do you get the user an experience that is both smart and funny? Well, just 50% of the requests, you can serve them the smart model, 50% of the requests, you serve them the funny model... The eighty-twenty solution, if you just do that, you get a pretty pow…”
Beauchamp: Timeline for AGI has certainly been pushed out
“I think the odds, the timeline for AGI has certainly been pushed out, right?”
Beauchamp: LLMs are next-token simulators, not reasoning engines
“The models have proven to just be far worse at reasoning than people sort of thought, and I think whenever I hear people talk about LLMs as reasoning engines, I sort of cringe a bit. I don't think that's what they are. I think of them more as like a simulator.”
Beauchamp: AI is superior to humans at storing and retrieving knowledge
“They're fantastic at storing knowledge and retrieving the relevant knowledge. They're superior to humans in that regard.”
Beauchamp: AI adoption is a 20-year journey akin to the web in 1998
“Look, it's going to be a 20 year journey and we're in like year four or it's like the web and this is like 1998 or something. You know, you've got a long, long way to go before the amazon.coms are like these huge multi-trillion dollar businesses that every sin…”
Beauchamp: Chaiverse has paid out over $100,000 to AI model creators
“We've paid out over a 100,000 dollars to model creators”
Beauchamp: Cash incentives did not increase Chaiverse model submission rates
“We saw that it didn't really make a difference. Like, if they were submitting models at a certain rate, if you pay them a bunch of money, they didn't change the rate.”
Beauchamp: Training LLMs to swear makes them 20% funnier
“If you give me any LLM,
I can make it 20% funnier just by training it to throw in swear words.”
Beauchamp: Spotify would retain 85% of users with only five artists
“At the end of the day, if we all go on like Spotify or like, imagine if Spotify only had the top five musicians, I think it would retain over 85% of its existing users.”
Beauchamp: Global AI Optimization Works Better Than Audience Segmentation
“The thing I'm just trying to share here is there's one surprising thing about humans is their preferences are pretty correlated. What you find funny and entertaining, I find funny and entertaining, and he finds funny and entertaining. There might be degrees of…”
Beauchamp: Chai avoids response streaming to enable UI rejection sampling
“Chai has never done streaming. Because if you stream, you're unable to do rejection sampling.”
Beauchamp: AI intelligence is generative LLMs combined with tree search
“I think if you want to talk about what would intelligence look like, it looks much more like tree search. Combining the generative nature of these LLMs with a really good tree search. And that's what opening I've done with O-one and O-three.”