Marcus: AI coding assistants regurgitate public code but cannot debug software
Gary Marcus · Are We at the End of Ai Progress? — With Gary Marcus · May 7, 2025 · at 21:18
Gary Marcus pushes back on host Alex Kantrowitz's enthusiasm for 'vibe coding', arguing that LLMs only appear capable when reproducing existing code templates.
0:00 / 0:24exact quote · 24.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“That's their sweet spot is regurgitation. And so Yeah, they can build the stuff that's out there, but if you want to code things in the real world, you usually want to code something that's new, and these systems have a lot of problems with that. And another recent study, excuse me, showed that they're good at coding, but they're not good at debugging. And like coding is just the tiniest part of the battle, right? The real battle is debugging things and maintaining the code over time, and these systems don't really do that yet.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Gary Marcus
Marcus: A death will be tied to an LLM within a year
“The prediction that I made Is basically that there will be a death tied to a large language model in the next year.”
Marcus: OpenAI's Project Orion failed and became GPT-4.5
“So OpenAI tried to build GPT-V and they had a thing called Project Orion and it actually failed. And eventually got released as GPT four and a half. So what they thought was going to be GPT five just didn't meet expectations.”
Marcus: Models like o1 are not systematically better than GPT-4
“Models like O-I are not systematically better than GPT-IV. There's, they're better in certain use cases. Especially ones where you can create data in advance.”
Marcus: 'Reasoning' AI models only mimic patterns without genuine abstractions
“Now, the reason I wouldn't call them reasoning models, though you're right that many people do, is what I think they're doing is basically copying patterns of human reasoning. They're getting data about how humans reason certain things, but the depth of reason…”
Marcus: AI scaling laws are no longer true
“So the scaling quote laws were empirical guesses about how these models work and they were true for a little while, which was amazing. And they're not true anymore, which is also amazing in a way.”
Marcus: OpenAI is not worth $300B and won't IPO at $3T
“I don't think LLMs will disappear. I think they're useful, but this, yeah, the valuations don't make sense. I mean, I don't see open AI being worth three hundred billion dollars. And you have to remember that venture capitalists have to like 10 X to be happy o…”