The Wisdom Wall

“Well, I think whatever we do, uh, there will be both existential risks and individual risks. So it's not as if we have a choice between avoiding risks and confronting risks. So it's, uh, it's looking at these different alternatives and weighing up the, the risks and benefits. Um, and, and that would be some optimal…”

“once you start to engage in activism, then There is kind of mimetic pressures to, to simplify our message and to close ranks and to try to beat the people with opposing views down in the marketplace of ideas. Um, and I think we are seeing some of the beginnings of that, um, where there are these kind of campaigns, uh,…”

“From this point onward, probably AI safety is relevant not only for deployment, um, but also during training and evaluation. Like these models might be quite powerful even before they are sort of released to the general public. So that's not the only point at which safety concerns arise, but also now whilst they're…”

“turns out there is like some big, um, hobbling that we have unwittingly, like some, something we were doing wrong that just made these systems way less efficient than they could be. And when somebody figures out how to remove that, like maybe the current Compute is already enough to kind of catapult us into the super…”

“The most valuable time for that to happen is at the latest possible moment. Um, because then you would have the actual system that you're trying to align to work with.”

“So if the pause only applies to the most responsible actors, for example, then a long pause would remove the initiative from the most responsible AI developers and shift it over to the, uh, Less responsible AI developers who decide not to abide by the past”

“Another is that, um, you might with a longer pause start to build up a lot of hardware overhang. It's a like, if, if, if, if we keep building out bigger data centers and chips are getting better than a long pause would result in a situation where you now have such a massive amount of compute available that once you…”

“it's harder to sometimes see the The cost of, of limiting ourselves in all those ways, because the new innovations that would be unlocked are not yet there. Whereas like the pollution is, is there immediately, right? Or the, the person, like if a self-driving car runs over a pedestrian, like that's immediately visible,…”

“We often define our sense of self-worth on, on the idea of, of being a contributor. Like you're, you're a breadwinner, or you make like a positive difference in the lives of your friends or of society at large. You're like, you, you bring value to the world. Um, so much of our existence is kind of Um, constructed…”

“Um, I, I think for cybersecurity right now we're in a regime where attackers often win, um, but it might be that in the limit if you have sort of An AI trying to find vulnerabilities and also patch vulnerabilities and you keep making the AI stronger, like eventually maybe you reach a point where the AI, the software is…”

“Um, and patches are a lot easier to roll out in the digital space. Uh, so, so maybe there's like some cyber thing. We figure out what the vulnerabilities we can release the patch. And then, um, in, in principle, like almost immediately around the world, all the relevant systems could be patched. Now there is often a…”

“it's also conceivable that even when you do get recursive self-improvement, you still might not have An intelligence explosion. It might, there might be diminishing returns at some point. Presumably there are at some point, but it could turn out that that is close enough to where we are now that you have this massive…”

“this gives us more sort of surface area to work with. Like you can more easily understand and interact with these systems because they have human double concepts and you can talk with them.”

“In a solved world, in utopia, the utopians could have, like, extreme levels of motivation and immersion and subjective purpose. That's easy. That's like a check mark. Um, and more broadly, you can go through different plausible human values. And for some of them, you can just right off the bat say, well, yeah, sure. Of…”

“AI, you know, the goal of AI has all along been not just to automate a few specific things, but to provide the technology that allows us to automate all tasks, right? Like AI hasn't really succeeded until all intellectual labor can be done by machine.”

“the simulation hypothesis expands the space of sort of realistic possibilities and the realist, like the space of realistic futures. You might think if you're living just in a simple materialistic universe, you die. That's the end. Your brain rots. There is no more experience. And there's really not much room for other…”

“Well, I think we are starting to see the, uh, added dimensions of the alignment challenge, um, that open up once you have systems that are sophisticated enough because the, uh, space of possible strategies that you can pursue is a function of, uh, your, your cognitive capacity. Like you can think of new, clever,…”

“you're focusing there on the misuse potential that this people might choose to do bad things with AI technology. And that certainly is one big category of risk, right? But that's not primarily a technical challenge. It's more ultimately a governance challenge and an ethics challenge. Um, and that's kind of, in addition…”

“There is this phenomenon that like, before it is done, it looks really hard. Once AIs have done it, then we kind of quickly forget how, just how like impressive it was and just take it for granted.”

“we are designed in such a way that our reward system motivates us to, uh, produce more effort. At, at whatever level, like, no matter how good our situation is, we are designed to always try to want to make it better, and so we only get reward when things improve, rather than when things are, like, at a good level, to…”