Marcus: LaMDA Is Merely Autocomplete on Steroids, Not Sentient
Gary Marcus · Is AI Dangerously Overhyped? — With Gary Marcus · Sep 13, 2022 · at 16:59
Gary Marcus dismisses claims of AI sentience in Google's LaMDA, explaining that the system merely relies on massive training data to predict next tokens.
0:00 / 0:28exact quote · 28.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“My turn-by-turn system has more elements of what I would actually ascribe sentience than Lambda, which is really just autocomplete on steroids. That's all it is, right? You type in your phone, I will meet you at, and it guesses that, you know, you might say the restaurant because either you or other people have said that before. It's all Lambda is doing is predicting next words and sentences, and it is this massive Scale of data that makes it seem like a real thing that it just isn't.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Gary Marcus
Marcus: A death will be tied to an LLM within a year
“The prediction that I made Is basically that there will be a death tied to a large language model in the next year.”
Marcus: OpenAI's Project Orion failed and became GPT-4.5
“So OpenAI tried to build GPT-V and they had a thing called Project Orion and it actually failed. And eventually got released as GPT four and a half. So what they thought was going to be GPT five just didn't meet expectations.”
Marcus: Models like o1 are not systematically better than GPT-4
“Models like O-I are not systematically better than GPT-IV. There's, they're better in certain use cases. Especially ones where you can create data in advance.”
Marcus: 'Reasoning' AI models only mimic patterns without genuine abstractions
“Now, the reason I wouldn't call them reasoning models, though you're right that many people do, is what I think they're doing is basically copying patterns of human reasoning. They're getting data about how humans reason certain things, but the depth of reason…”
Marcus: AI scaling laws are no longer true
“So the scaling quote laws were empirical guesses about how these models work and they were true for a little while, which was amazing. And they're not true anymore, which is also amazing in a way.”
Marcus: OpenAI is not worth $300B and won't IPO at $3T
“I don't think LLMs will disappear. I think they're useful, but this, yeah, the valuations don't make sense. I mean, I don't see open AI being worth three hundred billion dollars. And you have to remember that venture capitalists have to like 10 X to be happy o…”