The Wisdom Wall

“Inventing new things, you know, requires a type of skill and abilities that you're not gonna get from NLMs.”

“Uh, we also know that they can hallucinate facts that aren't true, uh, but they're really, in their purest form, they are incapable of inventing new things.”

“you can't, you can't, absolutely cannot do research in a startup. You just can't because you just don't have the, the funds, right?”

“Because the understanding that those, the current systems have of, uh, you know, the underlying reality that language expresses is extremely shallow. So those systems have only been trained with text, uh, a huge amount of text. Um, so they can regurgitate texts that they've seen and, you know, interpolate for new…”

“human artists are Absolutely authorized to get inspired and straight down copy, uh, someone else's style. Um, that happens absolutely all the time in, um, uh, in art. And so would, would it make sense to apply a different rule for, let's call them, uh, artificial artists, right? That, that generate things. Um, like,…”

“it's, it's basically, um, you know, why we, we had super impressive, you know, autonomous driving demos 10 years ago, um, But we still don't have level five self-driving cars, right? Um, it's the last mile that's really difficult, uh, so to speak, uh, for cars. You know, it's, you know, the last few, that was not…”

“there's not going to be like a day Before which there is no AGI and after which we, we have AGI. This is not going to be an event. Um, it's going to be continuous conceptual ideas that as time goes by are going to be made bigger and to scale and going to work better, and it's not going to come from a single entity.…”

“the economic model The only one I know outside of, uh, universities and philanthropy is, uh, industry research lab inside of a large company that has, that is profitable and is, uh, sufficiently well-established in this market that it can think for the long term and invest in fundamental research.”

“how much of, uh, human knowledge is present and described in text? And my answer to this is a tiny portion. Like most of human knowledge is not actually language related. It's completely non-linguistic.”

“LLMs do not have that. They don't have any internal model of the world that allows them to predict.”

“what characterizes intelligence is the ability to predict, first of all, and then the ability to use those prediction Those predictions as a tool to plan by predicting the consequences of actions you might take. Prediction is the essence of intelligence.”

“the type of learning that we are currently able to reproduce in machine, which is supervised learning and reinforcement learning do not seem to, uh, Reflect what we observe in humans and animals. There is another type of learning, another paradigm of learning that seems to take place in humans and animals that allows,…”

“currently what we can do in machine learning is more like the system when the stuff that, you know, here is an input, here is an output, uh, that does not re require, uh, reasoning if you want.”

“face detection can be learned in minutes. If you're a baby, your vergence is bad. Your, your, um, your focus, uh, is, is basically fixed at a relatively short range. So the only thing you see during the first weeks of your life are, are faces and nipples, essentially. Um, so, you know, and then, and then you have a…”

“a lot of reasoning in, uh, certainly in animals and in humans is not logical reasoning. It's, it's, it's basically simulation. Or analogical reasoning, which kind of similar.”

“Like most of human knowledge is not represented in any text, uh, in existence.”

“most of what we learn as, as humans and animals and in, in the future that machine will learn, uh, is learned in this kind of self-supervised manner, basically by watching the world go by and by, you know, taking an action once in a while, but in a, in a, you know, non-task specific way, learning how the world works,…”

“a big issue there is, is that when, when humans or animals reason, We don't do it in token space. In other words, when we reason, we don't have to, you know, generate a text that expresses our solution, um, and then generate another one, and then generate another one, and then among other ones we, we produce, pick the…”

“I don't like the term AGI, artificial general intelligence, because I think human intelligence is very specialized. So if you want to designate, uh, the type of intelligence that we observe in, in humans by general, that's a complete misnomer.”

“Whereas if you are Meta or Google, you could think about like, you know, putting out a system of this type that, you know, is going to spew nonsense. Um, and, you know, because you are a large company, you have a lot to lose by, you know, people kind of making fun of you for that. Uh, and it's not clear what the…”

“beyond the impressive demos, actually deploying systems that are reliable is where things tend to falter in, in the use of computers and technologies and particularly AI. This is not you. Um, it's, it's basically, um, you know, why we, we had super impressive, you know, autonomous driving demos 10 years ago, um, but we…”

“there is no intelligence without learning. And, uh, in the engineering world, it's almost true as well. And it's probably because I'm either lazy or not smart enough to, that I think that as human engineers, we cannot actually directly conceive and construct an intelligent machine. We have to, Build a machine that can…”

“Well, um, I mean, I agree with Danny that current current AI systems are, you know, very specialized, and that makes them very brittle because they're trained for one task or maybe a collection of tasks.”

“Uh, multi-layer, you know, deep neural nets are hierarchical Bayesian systems. If you kind of view them the right way, and there are certain forms of them that are actually explicitly Bayesian.”

“All those things would be built entirely by hand. Okay. Completely engineered. And that was essentially that, that necessity was one of the reason of the, the kind of decrease in interest in sort of, uh, good old fashioned AI.”