Superintelligence

“It doesn't look like we're going to suddenly end up in a regime where like you could train super intelligence with a really cheap recipe, as opposed to it being more of an iterative thing where like the cost keeps going down, the capabilities keep going up. Th…”

“This is why I think, like, the world is, on the whole, not going to be that different. Even with superintelligence, like, people are still going to be very fundamentally wired to care about other people, to want to be around other people to interact with other…”

“If you get a kind of weak super intelligence that is For the most part aligned, we might then be able to use that to make a more powerful form of super intelligence that is more reliably aligned.”

“Turns out there is like some big hobbling that we have unwittingly, like some, something we were doing wrong that just made these systems way less efficient than they could be. And when somebody figures out how to remove that, like maybe the current Compute is…”

“At the point where it is as good as an average human in everything, it might already be super intelligent in some key relevant domains for AI research.”

“This gives us more sort of surface area to work with. Like you can more easily understand and interact with these systems because they have human double concepts and you can talk with them.”

“We believe that everybody in the world, you know, all the billions of people in the world are going to have a super intelligence that is adapted and tailored to them, that is, enables them to accomplish their goals, knows their context, and ultimately is an ex…”

“The kind of, like, cult worship of the machine god states those people believe that, like, more is going to happen quickly than is going to happen. Eventually a lot will happen, but eventually a lot was gonna happen anyway.”

“If you can solve coding, tool use, reasoning, and multimodality, you have the keys to superintelligence. Those are the building blocks. If you can solve those four, you can automate anything that a human does in front of a computer, and it'll give humans super…”

“Our vision is not that there's going to be like some central super intelligence that solves all of science.”

“I think we'll probably get it within 20 years. That's all I'm happy to say at present.”

“And when it comes, we've no idea how to be safe.”

“I guess I'm more optimistic than I was a year or two ago, because I see that it might be possible to design these new beings so they care about us. It also might be possible to use Yoshua's technique of designing new, new beings that can't actually perform act…”

“It is not about, like, formal verification or verified AI to us. It's not just about handling or, like, kicking out the lousiness, the hallucinations, the mistakes. It's about scaling brilliance. It's about super intelligence.”

“The super intelligence is going to be the boss. You're going to be the junior thing. And so and that's what he's, that's what these guys are talking about, Brian Halligan and Jack Dorsey, which is that actually there'll be like a central AI that's making the k…”

“We had this strong thesis and vision that the intelligence will come not just from building ever smarter and ever bigger and more powerful single agents But more through what we call horizontal scaling. So we distributed systems, a lot of agents coming togethe…”

“There has to be, I think, and I think there's, I think most people now, if you look, talk to like leading edge research and stuff, sort of agree with this. They're like, look, there has to be another breakthrough to really start that, like, To get to the next …”

“I think that, that there's kind of, that seems to be totally reasonable based on the trajectory that we're on. And I would bet that, that Sam has an even a far higher bar for what his definition of, you know, intellectual or whatever the term was, then even I …”

“We are probably in the last innings before superintelligence.”

“Ideally, you are setting up your research so that as super intelligence arrives, that is a tailwind. That's also bolstering our ability to like understand the models because otherwise you're fighting a losing battle. If it's like the systems are getting more a…”

“So I think when you talk about like opening AI, building its own chips and data centers and all this stuff, that's all about like them for seeing the future where there's so many people using these things, those it's expensive to run in the data center. They'r…”

“We have the AGI we need. Let us just focus on solving the actual problems inside the organizations, and I think we can already, that's enough to automate a lot of the tasks and get huge economic value out of it. We don't actually need superintelligence for tha…”

“A candidate definition for superintelligence is when a system can do a better job being president United States, CEO of a major company, you know, running a very large scientific lab than any person can, even with the assistance of AI.”

“I think something like that is like an interesting framework for super intelligence. I think it's like a long way off, but I would love to have like a cleaner definition this time around.”

“There's a there's a whole ideology in the valley. I mean, everywhere around the valley right now that like this is the truth that there it's 100% zero sum and whoever gets there first to super super intelligence.”

“Superintelligence and AGI are really goals rather than methods, and I think that the ambition is to create superhuman performance at most or all human tasks.”

“Somebody else is just gonna develop super intelligence as well, and they're gonna get into a price war. So how long can you say, I want a party of the profit of this drug that you're gonna develop with our technology? Where like somebody else can be like, well…”

“We are giving birth to some kind of super intelligence. We don't know what it is. We don't know what kind of scale, but it's big. It's important.”

“In self-driving they have like level five full autonomy, like totally self-driving human out of the loop, but a lot of the commercial value gets created at like the level one, level two, like, all right, let's give people a working search box. Let's actually g…”

“When we start to give the next generation of the model these power tools, and they start to solve previously unsolved engineering problems, I think you start to have something that looks kind of like super intelligence.”

“I have talked to three people in the last month that are all very, very smart, successful people. Who, when you're like, what are you doing? They're like, oh, I'm working on super intelligence. I'm like, with what team? I'm like, just by myself.”

“I don't agree with the three years, but I do agree that it'll be maybe six or seven years.”

“Glasses are the natural delivery mechanism for super intelligence. Like the, it is, you need something that will see what you see, hear what you hear, and they can easily deliver information to you.”

“Like, I don't think it's, I know there's been some executives at big labs that say we'll have super intelligence in three years, but I think the truth is that it's a longer road.”

“I very much believe so, and the reason is that the question comes down to when we'll have super intelligence. Once we have super intelligence and models are better than humans at everything, then, of course, that means that humans won't be able to contribute t…”

“That we'll have super intelligence in three years. That's better than humans at everything. I think it's totally wrong.”

“My honest opinion is, I think there is some, you know, rapid, singularity, superintelligence thing that people kind of talk about. I would guess It's very hard to say. You know, nothing's impossible, but I would guess that that's not something that happens in …”

“It's, it just is obviously all marketing at this point, and so I do feel like that, you know, you gotta at some point pin people down to say like, what are you actually trying to build, and what is the outcome of that going to be?”

“I'll stand on the table here and say it's the same thing. I think it's, if there were different techniques, I think this is really what it comes down to. If there were different techniques to build AGI and super intelligence, I would say, okay, sure, you know,…”

“So I think these terms are totally meaningless. I don't know what AGI is. I don't know what super intelligence is. It sounds like a marketing term.”

“And I actually think the original sin in all of this AI You know, anthropomorphic fallacy started with Bostrom, right? It was one of these, kind of, thought experiments where, you know, when Nick Bostrom wrote Superintelligence, it was what, 2014?”

“I mean, superintelligence is coming in the next five to seven years.”

“Like once we get to super intelligence, it will be too late to align the models. Probably.”

“I think, like, 50th percentile chance of hitting some kind of super intelligence in just a small handful of years is probably reasonable, and it does sound crazy, but this is the exponential that we're on.”

“Even if we have superintelligence, I think it will take some time for its effects to be felt throughout society and the world. And I think they'll be felt sooner and faster in some parts of the world than others.”

“Superintelligence in these contexts of the large lab context is actually just being used synonymously with what AGI used to be used for.”

“Like if you really solve this oracle for organizations just for coding, you've basically built all the capabilities you need to have super intelligence.”

“The reward problem in itself is at the time I called, I thought it was AGI complete. Now I'd say it's ASI complete, but by the time you have a neural network that can accurately verify any outcome, that is probably a super intelligence.”

“I think that where I think we'll be in a couple of years from now is that there'll be kind of definitive super intelligence in, Some meaningful categories of work.”

“Actually going in and deploying it and building it for, you know, specific categories of work. There are going to be a lot of product and kind of research innovation specific to those categories that will probably make this a multi-decade thing. So I don't thi…”

“And I think what you're seeing with scaling is we will be able to certainly accomplish our collective definition of super intelligence. With the current, ah, the current path we're on with scaling laws.”

“And if we're only a couple of years away, do I think that Meta can win that race? Not if we're a couple of years away, I don't think so.”

“Even if there's no software progress out of that point, that algorithms aren't improving, just that ability to learn on the job from everything in the economy would functionally produce what looks like a super intelligence, right? No human will be, will have m…”

“Look, there, there's a lot you know, in 10 or 20 years, unless something goes hugely wrong, we'll have, like, unimaginable super intelligence, and I'm very excited to see how that goes.”

“I genuinely think super intelligence will do almost no harm. Whatever harm it will do, it will like self restore.”

“Like, we're gonna scale it, sure, we're gonna scale these things up by a few more orders of magnitude, they're gonna become more capable, but we're not gonna see superintelligence from just that. And like, yes, if we had a quadrillion dollars to train these mo…”

“Look, if there is an intelligence explosion, it has a really tough dynamic because if you're a month ahead, you will kick off this loop much faster than anybody else. And what that means is that you will you will be a month ahead to super intelligence, but nob…”

“The cost of training, the systems is declining so fast that Literally you will be able to train a super intelligence in a basement at some point in the future.”

“Clearly the messaging worked. Everyone started saying it. It absolved people from having to achieve AGI or when everyone is saying AGI is already here, yet life doesn't feel significantly different.”

“I think it's quite possible. I think it's very hard to put confident bounds on, on the numbers, but”