The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 30 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Large language models are not the path to human-level intelligence
“LLMs are not the path to human level intelligence. LLMs work for discrete worlds. They don't work for continuous, high dimensional worlds, which is the case for video. And this is why LLMs do not understand the physical world and cannot be used in their curren…”
LLMs primarily perform data retrieval and possess very little actual reasoning
“If it's text, they will regurgitate solutions to puzzles. They will, you know, give you answers to questions you may have. It's mostly retrieval. There's a very tiny bit of reasoning, but really not much and that's an important limitation.”
The smartest LLMs are not as smart as a house cat
“The smartest LLMs are not as smart as your house cat. And it's really true.”
OpenAI o1's search-based reasoning approach is highly inefficient
“So you may have heard of O-one from OpenAI, and there is kind of similar work at Meta and other places where this sort of very basic forms of reasoning that consists in having an LLM produce lots of different sequences of words, and then having a way of search…”
AI entrepreneurs should pursue a PhD or master's degree to learn deeply
“You still want to do a PhD if you're an entrepreneur, or at least a masters, because you want to really sort of learn deep. I mean, you might be doing this by yourself. You don't have to, but it's useful because you learn more about, you know, what exists out …”
Society will not run out of jobs because human problems are limitless
“So we're not going to run out of jobs. Economists that I talk to tell me, We're not going to run out of jobs because we're not going to run out of problems. But we're going to find better solutions to problems with the help of AI.”
AI will amplify human intelligence to help solve major global problems
“That's the best reason also to work on AI, because AI is going to amplify human intelligence. I mean, the overall intelligence of humanity, if you want. So I think that that's the key to solving a lot of the problems that we have.”
Achieving human-level AI within five to ten years is overly optimistic
“So reach human level intelligence within a decade. That may be optimistic, right? Five to 10 years would be if everything goes great, all the plans that we're, we've been making will succeed. We're not going to encounter unexpected obstacles, but that is almos…”
Future AI infrastructure must be built collaboratively as a repository of knowledge
“AI is going to become a kind of common infrastructure, which people will use as a repository of all human knowledge, and this cannot be built by a single entity. It's going to have to be a collaborative project. With training being distributed all around the w…”
Hardware rivals struggle against NVIDIA because of its dominant software stack
“Training is dominated by NVIDIA at the moment. There's going to be other players, but they have a hard time competing because of the software stack, basically. Their hardware may be really good, but the software stack is is a challenge.”
The top AI business model is fine-tuning open-source models for verticals
“The most likely business model that has to do with AI is taking a open source foundation model, like LAMA, which is the data open source system, which is used everywhere now, right? Every, almost every startup uses it even large companies. So take an open sour…”
Open source models and platforms will dominate the AI ecosystem by 2029
“So five years from now, the world is going to be dominated by open source platforms.”
Smart glasses will replace smartphones for interacting with technology and AI
“Smart glasses. Yeah. I mean, yeah, there's almost no question.”
Future human workers will act like managers directing AI systems
“We're going to, we're all going to be a boss. We're all going to be like those high level managers. We're going to tell our AI systems what to do. But we're not going to have to do it ourselves necessarily.”
Domestic robots and autonomous cars require AI systems to learn from video
“We're going to have, at some point, domestic robots and you know, self-driving cars and things like this once we figure out how to get the system to learn how the real world works from video.”
The only way to understand intelligence is to build an intelligent machine
“For the problem that really has been my obsession for a long time is discovering the mysteries of uncovering the mysteries of intelligence. And as an engineer, I think the only way to do this is to build a machine that is intelligent, right?”
A lack of human intelligence is the root cause of global problems
“For almost every problem that we have, the cause is really a lack of knowledge or intelligence by humans. We're making mistakes. We're making mistakes because we're not smart enough to figure out we have a problem, because we're not smart enough to figure out …”
Self-supervised learning is the primary breakthrough behind modern chatbots
“Self-supervised learning is what has become
Very prominent over the last five, six years and is, is really the main component or the main contribution to the success of things like chatbot and natural language understanding systems.”
Reinforcement learning is highly inefficient outside of simulated environments like games
“It's very inefficient because the system has to try many things before it gets the correct answer. And so, It's very inefficient. It requires many, many, many trials. And so it works really well for games. You know, you, it's very efficient. If you want to tra…”
Autoregressive models succeed on text because language is discrete and finite
“And the reason it works for text and not for other things is because text is discrete. So there is a finite number of possible things that can happen, right? There's a finite number of words in the dictionary. There's a, you know, so if you can discretize your…”
The next major frontier in AI is learning world models from video
“This is what a lot of us consider the next Challenge in AI. So basically have systems that can learn how the world works by watching videos.”
Autoregressive LLMs are System 1, while objective-driven AI is System 2
“LLMs are system one. The architecture I'm describing, which I call objective driven AI, is system two.”
The computing infrastructure required for AI inference vastly exceeds training
“It's a lot of computing infrastructure. It's actually much bigger than the infrastructure for learning.”
LLM inference costs dropped 100x in two years, far outpacing Moore's law
“I think the cost of inference for LLM has gone down by a factor of a hundred in two years. I mean, it's just, it's amazing, right? It's way faster than Moore's law.”
Scientific progress relies on technological advances enabling data collection
“Scientists, you try to understand the world. Engineers, you try to create new things, and very often, if you want to understand the world, you need to create new things. The progress of science very much is linked with progress in technology that allows to Col…”
DeepMind was founded on the premise that reinforcement learning leads to AGI
“There was a big wave of interest in reinforcement learning about, you know, a dozen years ago, and companies like DeepMind set themselves up with the idea that reinforcement learning was going to be the key element towards building truly intelligent machines.”
Artificial neurons resemble biological neurons like airplane wings resemble bird wings
“We use that term, it's an abuse of language, because those neurons are not really neurons like in the brain. They are to real neurons as an airplane wing is to a bird wing, ok?”
Intelligence requires existing skills, rapid learning, and zero-shot problem-solving
“So the combination of those three things, you know, having already a number of skills that you know, experience with solving problems and accomplishing tasks, being able to learn new tasks really quickly with a few trials and then the next step is being able t…”
LLMs are self-supervised models constrained to predict preceding words only
“Chatbots are or LLMs, large language models, are a special case of that, where you train a system to predict a word,
But you only allow it to look at the words that precede it, you know, that are to the left of it.”
Deep learning and 1980s backpropagation remain the foundation of modern AI
“Deep learning, which is really the foundation of pretty much all of AI today. So basically, neural networks with multiple layers, right? The idea of this goes back to the 19 eighties and backpropagation. That's still the basic foundation of everything we do.”