The Ledger, every show

“If you believe that the technology that we are building, AGI, could potentially be so capable as to do every single task that people do, does it mean that it might unemploy everyone? Well, I don't know, but it's not impossible.”

“So it's better to think about it in terms of compute efficiency rather than in terms of, can it get there at all? I think at this point, the answer is obviously yes.”

“The appetite for compute is truly endless as now clearly seen, but we realized that we will need a lot. And a nonprofit was, wouldn't be the way to get there. Wouldn't be able to build a large cluster with a nonprofit.”

“Because if you imagine how an AGI should look like, it has to be some kind of a big engineering project that's using a lot of compute, right? Even if you don't know how to build it, what that should look like, you know that this is the ideal you want to strive…”

“I think maybe the most surprising, if I had to pick one, it would be the fact that when I speak to it, I feel understood.”

“As we train them, they gain more and more insight into the true nature of the human world. And their insight will continue to deepen.”

“I would actually point out that the main thing that's lost when you switch to the smaller models is reliability. I would argue that at this point it is reliability that's the biggest bottleneck to these models being truly useful.”

“I do think though that as models continue to get larger and better, then they will unlock new and unprecedentedly valuable applications. So yeah, the small models will have their niche for the less interesting applications, which are still very useful.”

“The day will come when you have models which can do science autonomously, like build, deliver on big science projects.”

“So the most near term limit to scaling is obviously data. This is well known and some research is required to address it. Without going into the details, I'll just say that the data limit can be overcome and progress will continue.”

“These are fairly well-known ideas in AI that the cortex of humans and animals are extremely uniform. And so that further supports the, yeah, like you just need one, you need big, uniform architecture. That's all you need.”

“I think that will happen when those systems become reliable in such a way as to be very autonomous. Right now, those systems are clearly not autonomous.”

“Technology is already reproducing using the minds of people who copy ideas from previous generation of technology. So I claim that the reproduction is already there.”

“It doesn't seem implausible. It doesn't seem at all implausible that we will have computers, data centers that are much smarter than people. And by smarter, I don't mean just have more memory or have more knowledge, but I also mean have deeper insight into the…”

“The reason neural networks of the time weren't good is because they were too small. So like if you try to solve a vision task with a neural network, which has like a thousand neurons, what can it do? It can't do anything. It doesn't matter how good your learni…”

“There is one specific formula right now that everyone is doing. And this formula is train a larger and larger transformer on more and more data.”

“I actually think that that is a pretty dramatic, and I would say quite a scary thing if you have an autonomously reproducing AI, if it's also very capable.”

“Neural network training can almost be seen as solving a neural equation. Solving a neural equation where every data point is an equation and every parameter is a variable.”