Beauchamp: ML model performance plateaus near human-level on an S-curve
William Beauchamp · Outlasting Noam Shazeer, Crowdsourcing Chai AI w/ 1.4m DAU — with William Beauchamp, Chai Research · Jan 26, 2025 · at 9:39
William Beauchamp, founder of Chai AI, contrasts his view of AI scaling limits with centralized scaling narratives from Sam Altman and Elon Musk.
0:00 / 0:12exact quote · 12.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“With machine learning, first of all, you see that the performance of the models follows an S-curve. So it's not like it just goes off to infinity, right? And the S curve, it kind of plateaus around human level performance.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from William Beauchamp
Beauchamp: Timeline for AGI has certainly been pushed out
“I think the odds, the timeline for AGI has certainly been pushed out, right?”
Beauchamp: Crypto's only killer use cases are gambling and evading regulations
“As far as a gambling device is like the most fun form of gambling invented in like ever. Super fun. I thought as a way to evade monetary regulations and banking restrictions, I think it's also absolutely amazing. So it has two like Killer use cases.”
Beauchamp: AI future belongs to distributed specialized platforms, not monoliths
“There must exist a platform where a small team can produce an AI for a unique purpose and they can iterate and build the best thing for that.”
Beauchamp: AI content should be created and trained by users, not Silicon Valley
“The number one problem for users in AI is this. All the AI is being generated by middle-aged men in Silicon Valley, right? That's all the content. You're interacting with this AI. You're speaking to it for 90 minutes on average. It's being trained by a middle-…”
Beauchamp: Dynamic LLM routing is expensive and offers minimal performance edge
“You can try and do, like, a routing thing where you say, for a user, given user requests, we're going to try and predict which of these end models that users enjoy the most. That turns out to be pretty expensive and not a huge source of, like, edge or improvem…”
Beauchamp: Randomly blending specialized LLMs delivers an effective 80/20 user experience
“How do you get the user an experience that is both smart and funny? Well, just 50% of the requests, you can serve them the smart model, 50% of the requests, you serve them the funny model... The eighty-twenty solution, if you just do that, you get a pretty pow…”