The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 45 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 1 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Marcus: A death will be tied to an LLM within a year
“The prediction that I made Is basically that there will be a death tied to a large language model in the next year.”
Marcus: OpenAI's Project Orion failed and became GPT-4.5
“So OpenAI tried to build GPT-V and they had a thing called Project Orion and it actually failed. And eventually got released as GPT four and a half. So what they thought was going to be GPT five just didn't meet expectations.”
Marcus: Models like o1 are not systematically better than GPT-4
“Models like O-I are not systematically better than GPT-IV. There's, they're better in certain use cases. Especially ones where you can create data in advance.”
Marcus: 'Reasoning' AI models only mimic patterns without genuine abstractions
“Now, the reason I wouldn't call them reasoning models, though you're right that many people do, is what I think they're doing is basically copying patterns of human reasoning. They're getting data about how humans reason certain things, but the depth of reason…”
Marcus: AI scaling laws are no longer true
“So the scaling quote laws were empirical guesses about how these models work and they were true for a little while, which was amazing. And they're not true anymore, which is also amazing in a way.”
Marcus: OpenAI is not worth $300B and won't IPO at $3T
“I don't think LLMs will disappear. I think they're useful, but this, yeah, the valuations don't make sense. I mean, I don't see open AI being worth three hundred billion dollars. And you have to remember that venture capitalists have to like 10 X to be happy o…”
Marcus: No AI lab has a technical moat; OpenAI has users
“Nobody has a technical mode. OpenAI has a user mode.”
Marcus: Gen AI's core business model will be surveillance and ads
“The business model of gen AI will be surveillance and hyper-targeted ads, just like it has been for social media.”
Marcus: Microsoft launched Bing Chat despite negative India test feedback
“And then the other really disturbing thing is that apparently they tested in India and got, you know, customer service requests saying it's not ready for prime time. And it was, you know, still put out.”
Marcus: The Turing test was an error in AI history
“I think that the Turing test was an error in, in AI history. I mean, Turing's obviously a brilliant man, and he, you know, made enormous contributions to computer science, but I think that the Turing test has been an exercise in fooling people. We've now solve…”
Marcus: LaMDA Possesses No Internal Model of the World
“When I'm going through this in some depth, because when we get to Lambda, Lambda doesn't have a model of the world.”
Marcus: LaMDA Is Merely Autocomplete on Steroids, Not Sentient
“My turn-by-turn system has more elements of what I would actually ascribe sentience than Lambda, which is really just autocomplete on steroids. That's all it is, right? You type in your phone, I will meet you at, and it guesses that, you know, you might say th…”
Marcus: Nobody knows how to prevent chatbots from being toxic or misleading
“Nobody knows, you know, how to make their chatbots constrained and not toxic and not spew misinformation.”
Marcus: The AI field is facing the same unsolved problems as 2015
“Despite all the hype about, you know, we're so close to solving AI or whatever. We're not, we're facing the same problems.”
Marcus: Grok 3 delivers minimal gains despite 10x compute
“So he built Grok three and by his own testimony, it was 10 times the size of Grok two. It's a little better, but it's not night and day, right? Grok two was night and day better than the original Grok. GPT four was night and day better than GPT three. GPT thre…”
Marcus: AI coding assistants regurgitate public code but cannot debug software
“That's their sweet spot is regurgitation. And so Yeah, they can build the stuff that's out there, but if you want to code things in the real world, you usually want to code something that's new, and these systems have a lot of problems with that. And another r…”
Marcus: Pure transformer models lack mechanisms for truth and inherently hallucinate
“If you look at a completely pure case of a transformer model trained on a bunch of data, It doesn't have any mechanisms for truth. Now, except the sort of accidental contingency, and there are inherent reasons why these systems hallucinate, and maybe I can, in…”
Marcus: LLMs hallucinate because they cannot distinguish individuals from categories
“The way I think about it is that these things don't understand the difference between individuals and kinds. So I actually wrote about this twenty-some years ago in my book, The Algebraic Mind, and I gave an example there, which is I said, suppose it was a dif…”
Marcus: ChatGPT succeeded over Galactica due to RL guardrails
“Part of the reason why ChatGPT succeeded where Galactica didn't is Galactica didn't really have those guardrails at all, and so it was just, you know, very easy to get it to say terrible things, and it's harder to get ChatGPT to say terrible things because tha…”
Marcus: We Lack the Science or Engineering to Know Proper AI Controls
“We don't even know what the right controls should be. They didn't do a good job and we don't yet have a good science or engineering practice on what it should be.”
Marcus: AI chatbots create illusions too compelling for average people to discern
“They're too compelling in that illusion for the average person with no training in how they work to understand it.”
Marcus: Current AI is a dress rehearsal showing we are not ready for AGI
“I, I've been thinking about this whole thing as a dress rehearsal. And before we didn't know how to make a dress rehearsal. This is a dress rehearsal for AGI. And you know, the lesson of the dress rehearsal is like, we are not ready for prime time. Let us not …”
Marcus: Machines will convincingly conduct podcast interviews within 30 years
“And some point I'm going to say, 30 years from now. We'll be able to make machines that do podcast interviews and I won't really know. I don't really know if you're a person fake or, you know, machine faking me out or whatever at some point.”
Marcus: No established methodology exists for debugging chatbots or autonomous vehicles
“None of these chatbot systems are well-debugged. Nobody knows how to debug them, in fact. And so both the problem with GPT-III and with driverless cars is we don't actually have a methodology even for debugging it.”
Marcus: OpenAI's o3 hallucinates more than preceding models
“I'll give you just one more example is O three apparently hallucinates more than the models that came before it.”
Marcus: Microsoft study suggests chatbot usage impairs human critical thinking
“Well, Microsoft did a study, in fact, suggesting that critical thinking was getting worse as a function of them.”
Marcus: AlphaFold is a neurosymbolic model and AI's greatest achievement
“And also alpha fold is actually a neurosymbolic model, and it's probably the best thing that AI ever did.”
Marcus: We Have No Way to Truly Debug AI Systems
“I think that's actually the deepest problem here is we have no way to debug these systems, really. We have the band-aids of, like, reinforcement learning and things like that that are so indirect.”
Marcus: LLMs will connect to all major software within two years
“You have companies like Adept that are spending their whole time trying to quote, add all the world, connect all of the world's software to large language models. So if not now, soon enough. I think that that's right. Like, you know, two years if it's not toda…”
Marcus: AI is not close to fooling experts in hour-long conversations
“Well, I don't think we're that close to a system that's going to be able to, you know, if I have an hour with it, let's say just to be conservative. I don't think we're that close to a victory there.”
Marcus: AI deployment will not pause unless Congress steps in
“The bottom line is what's driving it, and it's not that likely, unless Congress steps in, that there will be a pause.”
Marcus: Generative AI Art Is Fully Derivative Sampling on Steroids
“And what these systems are doing is kind of like amazing enhanced version of sampling where you don't even recognize what the samples are anymore. It's all derivative.”
Marcus: Google's LaMDA Has No Sensors Perceiving the Physical World
“Lambda actually has fewer sensors than my watch. My watch has a lot of sensors and lamb doesn't really have anything sensing the real world, except for its linguistic input.”
Marcus: Building sentient machines is possible, but we do not know how
“I am in no way arguing that it is not possible to build a sentient machine. I don't think we know how to do it, and I don't think we're clear enough on what it would consist of, but I'm not making the argument that it's impossible.”
Marcus: Future AGI systems will require internal world models
“So I'm pretty confident that we'll have models of the world, internal ideas about how the world works. I don't see how to build an AI without it.”
Marcus: LLM-powered misinformation as a service will soon be widespread
“If what you want to do is to put out like 10,000 versions of something on Twitter, something untrue and find one that sticks, then misinformation as a service, which is how they might call it in the tech industry is a pretty damn powerful technique. And if it …”
Marcus: No companies using GPT technology are breakout successes
“So yeah, there's a ton of companies that are trying to use GPT's technology. I don't know any of them that are You know, breakout successes.”
Marcus: Generative image software is relatively easy to copy
“The issue there is the software itself is relatively easy to copy.”
Marcus: Vals AI benchmark shows LLM accuracy under 10% on financial charts
“Where they looked at things like, can you pull out a chart based on a series of financial statements, SEC statements from a bunch of companies and these systems all claimed to do it, but accuracy was under 10%. And overall on this new benchmark, accuracy was a…”
Marcus: AI Language Models Produce Convincing Text That Is Often Untrue
“If you talk about another domain, like language synthesis, it turns out that current systems can make very convincing language, but it's often bullshit. That doesn't matter in the same way in art. So in, if you have a system make up a news story, even if it's …”
Marcus: AI Art Generators Lack True Understanding or Intentionality
“And it's worth realizing that the art system doesn't really understand what it's doing. It's just correlating words with images in its database. It doesn't have the same intentionality about the objects as a person does, but it can still be a very effective te…”
Marcus: AI Will Change Art but Humans Will Remain Creative Filters
“I think it probably will change art. I think in general that humans are still going to be like the creative inspiration and are often going to be filtering things.”
Marcus: Evolution primes humans to mistake conversational machines for people
“There's no machinery in our brains innately to tell us the difference between a person and machine. And what happens is that in evolutionary perspective, anything that could talk was probably a person, right?”
Marcus: Very interested in raising venture capital for neurosymbolic AI
“I'm very interested in that. Let's put it that way.”
Marcus: An amateur Go player beat KataGo 14 out of 15 games
“There was another mind-blowing study this week that showed that one of the best Go programs, KataGo, could be fooled by some silly little strategy that would be obvious to a human player. But, you know, somebody was able to follow this strategy and, like, an a…”