AI Alignment

“You're focusing there on the misuse potential that this people might choose to do bad things with AI technology. And that certainly is one big category of risk, right? But that's not primarily a technical challenge. It's more ultimately a governance challenge …”

“If you get a kind of weak super intelligence that is For the most part aligned, we might then be able to use that to make a more powerful form of super intelligence that is more reliably aligned.”

“This gives us more sort of surface area to work with. Like you can more easily understand and interact with these systems because they have human double concepts and you can talk with them.”

“This is the number one unsolved problem in AI. It's not the tech, we're making great progress on the technical alignment problem, but we haven't made jack progress on the human alignment problem, which is that we've known since the development of Conway's laws…”

“AI alignment, don't get me wrong, is hard, but not the hardest problem. Human alignment is really the problem right now.”

“Goals are one level of alignment. You can align something around goals. The kind of goals we're talking about here are one level of alignment. You can align something around goals by like if you can explicitly articulate in concept and in description, the stat…”

“What a lot of people don't realize is this endeavor is entirely about AI alignment.”

“I definitely think we would keep using pre-training data, not just from an efficiency point of view as well, but also I think there is interesting safety angles, because by pre-training and, you know, all this human knowledge, we're implicitly creating an agen…”

“Yeah, I wouldn't view it alignment adjust like an RL problem. I think it sort of, it goes throughout the whole stack. You might, you know, for example, filter the pre-training data in some way. You might, after training, you might have classifiers that, you kn…”

“And it's I think it's a never ending pursuit because like, even, even for humans, it's not super easy to define what's, what do we consider a light? And I think as our civilization will evolve, it will, the notion of alignment and the goals of humanity will, K…”

“I'm actually really positive and bullish that there is this economic pressure in a good way To make progress on alignment because long horizon agents require it.”

“I do think at some point there will be, like, some pieces of alignment that, like, you do want to export back into pre-training because that might be a way to, like, Put them in with more strength, like, more robustness, kind of, or more core to the intelligen…”

“And since then, my estimation of how hard the problem would be has gone down significantly actually because things like language models actually do really understand human values in a core way. The problem is definitely not solved, but I'm more hopeful than I …”

“So I view the distinction between alignment and safety as alignment as being a sort of subset of safety. Obviously you want the value systems of the AIs to be in keeping with or compatible with say the US public for USAIs or for you as an individual, but that …”

“The only way to sort of reasonably approach this is to iteratively ask your model to solve alignment and safety at that stage, not, not, you know, surely you can also ask it to solve your product level problems, but like that, that's nice, but that's not the f…”

“At the moment, the idea that we build LLMs that are being coerced with good behavior is not really sustainable. Because if they cannot prove that the behavior is actually good I think we are doomed.”

“So I have this view that we need human alignment instead of AI alignment. So right now, kind of when we talk about, you know, hey, we need to align AIs with like human values, but the reality is that, you know, all the problems that exist, they all exist becau…”

“And for that second risk expressed as AI alignment, what you're dealing with fundamentally is social engineering. And when you're dealing with social engineering, you're necessarily dealing with politics.”

“Alignment is about figuring out all these edge cases and saying, don't do this, basically putting it in a sheet. And the problem is we just don't know what it'll end up doing even this world. So nobody can write all the edge cases.”

“I think a lot of what's called AI alignment today is not really aligning with what the average human being wants. It's aligning with like what the sort of Silicon Valley average sensibility is, which I don't think it's good.”