The Ledger, every show

“We are not going to get to human level AI by just scaling up LLMs. This is just not going to happen.”

“LLMs are not the path to human level intelligence. LLMs work for discrete worlds. They don't work for continuous, high dimensional worlds, which is the case for video. And this is why LLMs do not understand the physical world and cannot be used in their curren…”

“There is this idea, somehow, the desire to, and the ability to dominate, is linked with intelligence. A statement that a lot of people are making, including Jeff Hinton recently, That somehow, as soon as a machine becomes intelligent, it becomes uncontrollable…”

“So this is predicated on an assumption that is just false, which is the existence of a hard takeoff. Right? So the fact that the minute you turn on a super intelligent AI system is going to take over the world. And it's going to escape your control and it's go…”

“Their technology is not particularly innovative from the scientific point of view. It's very well engineered. So they put together, you know, large scale scaled up system with trained with very well curated data. I mean, they know what they're doing. But in te…”

“Whatever you can hear from some of my more adventurous colleagues it's not going to happen within the next two years. There's absolutely no way in hell to, you know, pardon my French.”

“Well, so first of all, there is no question that eventually AI systems will understand the world in similar ways that, that humans do perhaps better ways but there will not be autoregressive large language models as a type that we're now talking about.”

“So my prediction is that within a few years, nobody in their right mind would use auto-reversive LLMs. They'll go away in favor of something more sophisticated and controllable. They can plan its answer as opposed to just produce one word after the other react…”

“So I understand why, why he might have wanted to leave, but I don't agree with him at all with the whole sort of, you know, probability of human extinction or whatever.”

“And so until we build systems that have some internal model of the world, it allows them to kind of simulate the world if you want, some way of generating actions on the world to use tools like a calculator or something of that, or interrogating a database or …”

“Inventing new things, you know, requires a type of skill and abilities that you're not gonna get from NLMs.”

“The people who share their research are going to move faster than the ones that don't.”

“We also know that they can hallucinate facts that aren't true but they're really, in their purest form, they are incapable of inventing new things.”

“No, not, not in the current form. I mean, and whatever form of AI would be able to do that will not be LLMs. They might use LLMs as one component.”

“One simple way of getting NNMs to kind of appear to reason is chain of thought, right? So you basically tell them to generate more tokens than they really need to in the hope that in the process of generating those tokens, they're going to devote more computa…”

“Everybody's talking about agentic systems. Nobody has any idea how to build them other than basically regurgitating plans that have, the system has already been trained on.”

“We are not going to get to human level AI by just scaling up LLMs. This is just not going to happen.”

“Whatever you can hear from some of my more adventurous colleagues it's not going to happen within the next two years. There's absolutely no way in hell to, you know, pardon my French. The, you know, the idea that we're going to have, you know, a country of gen…”

“And it tells you clearly that we're not going to get to human level AI by just training on text. It's just not a rich enough source of information.”

“Is the right way to approach this problem to train better and better video generation systems? And my answer to this is absolutely no. The problem of understanding the world does not go through the solution to the, to generating video at the pixel level.”

“Open source progress is faster and you know, a lot more innovation can take place in the open source world, which the proprietary world may have a hard time catching up with.”

“If it's text, they will regurgitate solutions to puzzles. They will, you know, give you answers to questions you may have. It's mostly retrieval. There's a very tiny bit of reasoning, but really not much and that's an important limitation.”

“The smartest LLMs are not as smart as your house cat. And it's really true.”

“So you may have heard of O-one from OpenAI, and there is kind of similar work at Meta and other places where this sort of very basic forms of reasoning that consists in having an LLM produce lots of different sequences of words, and then having a way of search…”

“You still want to do a PhD if you're an entrepreneur, or at least a masters, because you want to really sort of learn deep. I mean, you might be doing this by yourself. You don't have to, but it's useful because you learn more about, you know, what exists out …”

“So we're not going to run out of jobs. Economists that I talk to tell me, We're not going to run out of jobs because we're not going to run out of problems. But we're going to find better solutions to problems with the help of AI.”

“They would be so pervasive, so much will ride on those systems that I don't think anyone will accept that those assistants be behind the event horizon in a private company. They will insist that the infrastructure is open.”

“So I'm not against regulation. There should be regulation of AI products, particularly the ones that Involve making critical decisions for people, but regulating or slowing down research is complete nonsense.”

“The scenario I think will happen, and I'm certainly rooting for, is the scenario I described earlier, where you have some sort of open platform for base LLMs. So base LLMs basically would be seen as a basic infrastructure. TCP, IP, Linux, Apache, completely op…”

“So those systems do not have anywhere close to human level intelligence. Okay. Despite what you might think, we are kind of fooled into thinking it because those systems are very fluent with language, but their ability to think, to understand how the world wor…”

“The second fallacy is that there is this idea somehow that the desire to, and the ability to dominate is linked with intelligence, right? So this is a statement that a lot of people are making, including, you know, my friend Jeff Hinton recently. That somehow,…”

“It's not like if you get it wrong, it's going to destroy humanity.”

“They would be so pervasive, so much will ride on, on those systems that I don't think anyone will accept that those assistant being behind Event Horizon in a private company, they will insist that the infrastructure is open.”

“Autoregressive LLMs basically don't do planning, or a very simple form of it.”

“So I think the proprietary approaches will actually fall behind.”

“No economist believes we're going to run out of jobs because no economist believes that we're going to run out of problems to solve or requirement for human creativity and human communication and stuff like that. So, you know, this is going to create as many j…”

“So, for example, there's a narrative, a very, very common narrative that AI is the culprit for a lot of the bad side effects of social networks in the past. And in fact, it's completely backwards. AI is the solution to those problems.”

“Regulating or slowing down research is complete nonsense. It's just obscurantism.”

“The short answer is it's not a, Particularly big step towards you know, more like human, human level intelligence.”

“They've become sort of a contract research house for Microsoft to some extent.”

“You can't, absolutely cannot do research in a startup. You just can't because you just don't have the funds, right?”

“The economic return after 10 years or so, or nine years that Google has gotten from DeepMind, you know, is not clear. It's worth their investment.”

“Because the understanding that those, the current systems have of you know, the underlying reality that language expresses is extremely shallow. So those systems have only been trained with text a huge amount of text. So they can regurgitate texts that they've…”

“Most of the technology underlying techniques used in ChatGPT have been invented at Google and Meta.”

“So if you have autonomous AI systems that work by optimizing objectives and they have the ability to predict they are going to have emotions. It's inseparable from autonomous intelligence.”

“Human artists are Absolutely authorized to get inspired and straight down copy someone else's style. that happens absolutely all the time in in art. And so would it make sense to apply a different rule for, let's call them artificial artists, right? That, tha…”

“There are people out there in Europe that, that have argued for the fact to kind of study neurons and neurons and neural networks natural neural networks in all the details, and then build giant supercomputers to simulate this very accurately, and somehow they…”

“The problem is, from the engineering point of view, nobody has actually demonstrated that spiking neurons actually work. Like there is no image recognition systems that are based on spiking neural networks. Why build chips for something we don't know works? So…”