Beauchamp: LLMs are next-token simulators, not reasoning engines
William Beauchamp · Outlasting Noam Shazeer, Crowdsourcing Chai AI w/ 1.4m DAU — with William Beauchamp, Chai Research · Jan 26, 2025 · at 52:17
William Beauchamp, founder and CEO of Chai AI, argues against the view that current transformer LLMs perform genuine reasoning.
0:00 / 0:15exact quote · 15.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The models have proven to just be far worse at reasoning than people sort of thought, and I think whenever I hear people talk about LLMs as reasoning engines, I sort of cringe a bit. I don't think that's what they are. I think of them more as like a simulator.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from William Beauchamp
Beauchamp: ML model performance plateaus near human-level on an S-curve
“With machine learning, first of all, you see that the performance of the models follows an S-curve. So it's not like it just goes off to infinity, right? And the S curve, it kind of plateaus around human level performance.”
Beauchamp: Timeline for AGI has certainly been pushed out
“I think the odds, the timeline for AGI has certainly been pushed out, right?”
Beauchamp: Crypto's only killer use cases are gambling and evading regulations
“As far as a gambling device is like the most fun form of gambling invented in like ever. Super fun. I thought as a way to evade monetary regulations and banking restrictions, I think it's also absolutely amazing. So it has two like Killer use cases.”
Beauchamp: AI future belongs to distributed specialized platforms, not monoliths
“There must exist a platform where a small team can produce an AI for a unique purpose and they can iterate and build the best thing for that.”
Beauchamp: AI content should be created and trained by users, not Silicon Valley
“The number one problem for users in AI is this. All the AI is being generated by middle-aged men in Silicon Valley, right? That's all the content. You're interacting with this AI. You're speaking to it for 90 minutes on average. It's being trained by a middle-…”
Beauchamp: Dynamic LLM routing is expensive and offers minimal performance edge
“You can try and do, like, a routing thing where you say, for a user, given user requests, we're going to try and predict which of these end models that users enjoy the most. That turns out to be pretty expensive and not a huge source of, like, edge or improvem…”