Reinforcement learning is highly inefficient outside of simulated environments like games
Yann LeCun · WTF is Artificial Intelligence Really? | Yann LeCun x Nikhil Kamath | People by WTF Ep #4 · Nikhil Kamath · Nov 27, 2024 · at 43:38
Meta Chief AI Scientist Yann LeCun explains the fundamental sample inefficiency of reinforcement learning compared to other AI paradigms.
0:00 / 0:23exact quote · 23.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“It's very inefficient because the system has to try many things before it gets the correct answer. And so, It's very inefficient. It requires many, many, many trials. And so it works really well for games. You know, you, it's very efficient. If you want to train a system to play chess or go or things like that, poker, reinforcement learning is great because you can have the system play millions of games against itself or copies of itself.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Yann LeCun
Large language models are not the path to human-level intelligence
“LLMs are not the path to human level intelligence. LLMs work for discrete worlds. They don't work for continuous, high dimensional worlds, which is the case for video. And this is why LLMs do not understand the physical world and cannot be used in their curren…”
LLMs primarily perform data retrieval and possess very little actual reasoning
“If it's text, they will regurgitate solutions to puzzles. They will, you know, give you answers to questions you may have. It's mostly retrieval. There's a very tiny bit of reasoning, but really not much and that's an important limitation.”
The smartest LLMs are not as smart as a house cat
“The smartest LLMs are not as smart as your house cat. And it's really true.”
OpenAI o1's search-based reasoning approach is highly inefficient
“So you may have heard of O-one from OpenAI, and there is kind of similar work at Meta and other places where this sort of very basic forms of reasoning that consists in having an LLM produce lots of different sequences of words, and then having a way of search…”
AI entrepreneurs should pursue a PhD or master's degree to learn deeply
“You still want to do a PhD if you're an entrepreneur, or at least a masters, because you want to really sort of learn deep. I mean, you might be doing this by yourself. You don't have to, but it's useful because you learn more about, you know, what exists out …”
Society will not run out of jobs because human problems are limitless
“So we're not going to run out of jobs. Economists that I talk to tell me, We're not going to run out of jobs because we're not going to run out of problems. But we're going to find better solutions to problems with the help of AI.”