AI systems

“And so until we build systems that have some internal model of the world, it allows them to kind of simulate the world if you want, some way of generating actions on the world to use tools like a calculator or something of that, or interrogating a database or …”

“Well, I don't think we're that close to a system that's going to be able to, you know, if I have an hour with it, let's say just to be conservative. I don't think we're that close to a victory there.”

“Yeah, I think the deterioration, further deterioration, it's already been going for a while, of trust and authority is gonna continue, and it's gonna be driven by these systems”

“In five years, we're gonna see really strong systems across a broad set of tasks. I have some strong conviction that, that we're on a path there. After that, you know, I don't want to bin any intelligence into narrow box, whether human or AI, but we will be am…”

“I'm saying that we may enter the weird zone where that becomes a thing that people study. And I think that if like sentience is a thing, you could imagine like weird versions of it leading to certain types of misuses or problems in the system as well.”

“We need some set of tests administered by a third party for things that people would view as legitimate, like national security risks or what have you, and systems, whether proprietary like ours or otherwise, Should go through those tests before they're deploy…”

“I think that if you really scale up AI systems, and you plug them into important parts of the world, and they go wrong, the effects could be extraordinarily, like, bad and catastrophic, in the sense of some, Cascading emergent problem.”

“Well, our AI systems today are kind of operating with short term memories that are millions of numbers in length, and it feels very unintuitive. Ultimately, we want them to instead be able to bake stuff into some kind of long term storage, and that's going to …”

“Everyone is trying to figure out how they can spend more and more of their compute on reinforcement learning, because I think everyone has this intuition that the more RL you add, the more sophisticated you're going to be able to make these things, and a lot o…”

“And I'm, you know, not as convinced about Leopold that this is necessarily going to come soon, but I think, you know, there's maybe like a 10% probability that this happens within three years.”

“What some of the research that I did at OpenAI was, you know, trying to develop techniques for this, and the simple technique that we tried and showed that worked was just ask a different AI system, or even the same AI system, is there a problem with this answ…”

“I think well I heard that we actually think there are some superconductor materials. I doubt they're room temperature ones though, but I think at some point, if it's possible with physics an AI system will one day find it.”

“I think those risks would potentially grow in time. I don't think they're as substantial now compared to just the malicious use sorts of risks.”

“You cannot change the code. You can get very different answers and so it's almost a different mentality on building. Again, when it's deterministic, you ship it, and you can kind of move on. Again, you want to keep an eye on it, but, like, it'll keep doing wha…”

“The challenge with it is it requires a new way of thinking for the people building the systems. Even if more and more is coded by cursor and other parts, you still need the humans that are working on this to adopt new ways of thinking and how they build system…”

“All the systems on like that, that depend on capturing engagement on like trying to see what people do. If you try to capture that trend of the immediate reward, You're going to get a lot of it, and so it's very easy to train. If you try to say, no, I really w…”

“And if you get used to this as what a great interaction should be, then it makes you less and less able to interact with humans who have flaws, have their own needs. You have to, it's more of a give and take, and it's very healthy to learn that give and take.”

“If we strangle off the business model of those places, if we strangle off the incentives for content to create for content creators to create content, then we're actually going to end up strangling the AI systems as well, because there's, if there's no content…”

“And I think in some of the rhetoric in the last few years, you can feel that there's a little bit of like you know, a kind of creeping assumption that it is inevitable that These kinds of systems exceed our control and our capability and move beyond us as a sp…”

“Eventually the entire economy is going to run on AI systems. That's for sure. But it might take up 20 years because it's actually fairly complex.”

“And today's systems, in my opinion, are nowhere near that. Doesn't matter how many, you know, Erdos problems you solve, which for some reason, you know, I mean, you know, that's, it's good that we're doing those things, but I think it's far, far from what you …”

“And the fundamental problem that we're seeing is we don't know how to take our values or our goals and encode them into these AI systems and get them to pursue it reliably.”

“The AI systems are helpful above and beyond Google in part because they can go back and forth with you and they can help you troubleshoot and dynamically answer your questions. And so even presented with the information on Google, people often can't get all th…”

“So I think there's now of the order of a hundred AI systems for interpreting scans that have been federally approved and they're being used a lot by radiologists.”

“And already we know that AI systems are better than doctors at diagnosis.”

“I think now putting things on back on the web and then making sure that they're accessible across the different AI systems in a way that makes sense for creators and make sure that creators get compensated is going to be something which actually I think will r…”