Prediction Held up
Marcus: A death will be tied to an LLM within a year
“The prediction that I made Is basically that there will be a death tied to a large language model in the next year.”
Assertion Supported
Marcus: OpenAI's Project Orion failed and became GPT-4.5
“So OpenAI tried to build GPT-V and they had a thing called Project Orion and it actually failed. And eventually got released as GPT four and a half. So what they thought was going to be GPT five just didn't meet expectations.”
Assertion Supported
Marcus: Models like o1 are not systematically better than GPT-4
“Models like O-I are not systematically better than GPT-IV. There's, they're better in certain use cases. Especially ones where you can create data in advance.”
Opinion
Marcus: 'Reasoning' AI models only mimic patterns without genuine abstractions
“Now, the reason I wouldn't call them reasoning models, though you're right that many people do, is what I think they're doing is basically copying patterns of human reasoning. They're getting data about how humans reason certain things, but the depth of reason…”
Opinion
Marcus: AI scaling laws are no longer true
“So the scaling quote laws were empirical guesses about how these models work and they were true for a little while, which was amazing. And they're not true anymore, which is also amazing in a way.”
Opinion
Marcus: OpenAI is not worth $300B and won't IPO at $3T
“I don't think LLMs will disappear. I think they're useful, but this, yeah, the valuations don't make sense. I mean, I don't see open AI being worth three hundred billion dollars. And you have to remember that venture capitalists have to like 10 X to be happy o…”
Opinion
Marcus: No AI lab has a technical moat; OpenAI has users
“Nobody has a technical mode. OpenAI has a user mode.”
Prediction Not checkable as stated
Marcus: Gen AI's core business model will be surveillance and ads
“The business model of gen AI will be surveillance and hyper-targeted ads, just like it has been for social media.”
Assertion Supported
Marcus: Microsoft launched Bing Chat despite negative India test feedback
“And then the other really disturbing thing is that apparently they tested in India and got, you know, customer service requests saying it's not ready for prime time. And it was, you know, still put out.”
Opinion
Marcus: The Turing test was an error in AI history
“I think that the Turing test was an error in, in AI history. I mean, Turing's obviously a brilliant man, and he, you know, made enormous contributions to computer science, but I think that the Turing test has been an exercise in fooling people. We've now solve…”
Opinion
Marcus: LaMDA Possesses No Internal Model of the World
“When I'm going through this in some depth, because when we get to Lambda, Lambda doesn't have a model of the world.”
Opinion
Marcus: LaMDA Is Merely Autocomplete on Steroids, Not Sentient
“My turn-by-turn system has more elements of what I would actually ascribe sentience than Lambda, which is really just autocomplete on steroids. That's all it is, right? You type in your phone, I will meet you at, and it guesses that, you know, you might say th…”
Insight
Marcus: Nobody knows how to prevent chatbots from being toxic or misleading
“Nobody knows, you know, how to make their chatbots constrained and not toxic and not spew misinformation.”
Opinion
Marcus: The AI field is facing the same unsolved problems as 2015
“Despite all the hype about, you know, we're so close to solving AI or whatever. We're not, we're facing the same problems.”
Opinion
Marcus: Grok 3 delivers minimal gains despite 10x compute
“So he built Grok three and by his own testimony, it was 10 times the size of Grok two. It's a little better, but it's not night and day, right? Grok two was night and day better than the original Grok. GPT four was night and day better than GPT three. GPT thre…”
Insight
Marcus: AI coding assistants regurgitate public code but cannot debug software
“That's their sweet spot is regurgitation. And so Yeah, they can build the stuff that's out there, but if you want to code things in the real world, you usually want to code something that's new, and these systems have a lot of problems with that. And another r…”
Insight
Marcus: Pure transformer models lack mechanisms for truth and inherently hallucinate
“If you look at a completely pure case of a transformer model trained on a bunch of data, It doesn't have any mechanisms for truth. Now, except the sort of accidental contingency, and there are inherent reasons why these systems hallucinate, and maybe I can, in…”
Insight
Marcus: LLMs hallucinate because they cannot distinguish individuals from categories
“The way I think about it is that these things don't understand the difference between individuals and kinds. So I actually wrote about this twenty-some years ago in my book, The Algebraic Mind, and I gave an example there, which is I said, suppose it was a dif…”
Opinion
Marcus: ChatGPT succeeded over Galactica due to RL guardrails
“Part of the reason why ChatGPT succeeded where Galactica didn't is Galactica didn't really have those guardrails at all, and so it was just, you know, very easy to get it to say terrible things, and it's harder to get ChatGPT to say terrible things because tha…”
Opinion
Marcus: We Lack the Science or Engineering to Know Proper AI Controls
“We don't even know what the right controls should be. They didn't do a good job and we don't yet have a good science or engineering practice on what it should be.”
Opinion
Marcus: AI chatbots create illusions too compelling for average people to discern
“They're too compelling in that illusion for the average person with no training in how they work to understand it.”
Insight
Marcus: Current AI is a dress rehearsal showing we are not ready for AGI
“I, I've been thinking about this whole thing as a dress rehearsal. And before we didn't know how to make a dress rehearsal. This is a dress rehearsal for AGI. And you know, the lesson of the dress rehearsal is like, we are not ready for prime time. Let us not …”
Prediction Open · timeframe Sep 2052
Marcus: Machines will convincingly conduct podcast interviews within 30 years
“And some point I'm going to say, 30 years from now. We'll be able to make machines that do podcast interviews and I won't really know. I don't really know if you're a person fake or, you know, machine faking me out or whatever at some point.”
Insight
Marcus: No established methodology exists for debugging chatbots or autonomous vehicles
“None of these chatbot systems are well-debugged. Nobody knows how to debug them, in fact. And so both the problem with GPT-III and with driverless cars is we don't actually have a methodology even for debugging it.”
Gary Marcus