The Exchanges

Q But what is what, what is much better?

A Well, I mean, one way to think about it is you didn't need a magnifying glass to see the difference between GPT-II and, we didn't call it GPT-I, but the original GPT. And you didn't need a magnifying glass for GPT-IV as opposed to GPT-III. It was just obviously better. A lot of people thought is that we would pretty quickly see GPT-V and a lot of people raced to build it. So OpenAI tried to build GPT-V and they had a thing called Project Orion and it actually failed. And eventually got released as GPT four and a half. So what they thought was going to be GPT five just didn't meet expectations. Now they could slap any name on any model they want. And in fact, lately nobody understands how they're naming their models, but they haven't felt like any of the models that they've worked on since GPT four actually deserve the name GPT five. And it didn't meet the performance that these so-called mathematical laws required. And what I said in that paper is they're not really mathematical laws. They're not physical laws of the universe like gravity. They're just generalizations that held for a little while. Like a baby may double in weight every couple of months early in its life. That doesn't mean that by the time you're 18 years old that you're going to be 30,000 pounds. And so we had this doubling for a while and then it stopped and we can talk about why, but the reality is that's not…

AI assessment note: “you didn't need a magnifying glass... It was just obviously better.”

Q But isn't the black box the thing in the plane that tells you what actually happened?

A Well, that's a different thing, right? So a black box in a plane is actually a flight recorder that records a lot of data. But what we mean in machine learning by black box is you have a model where you have the inputs and you have the outputs. You know how you calculate them, but you don't really understand how the system gets there. So in this case, you're doing all this matrix multiplication. Nobody really understands it. And so nobody can actually give you a straightforward answer for why O three hallucinates more than GPT four. We can just observe it. That's what happens with black boxes is you, you empirically observe things, And you say, well, it does that, but you don't really know why, and you don't really know how to fix it either. Another example, just in the last couple of days is apparently Sam Altman reported, I forget the new model is, is stubborn or what was it?

AI assessment note: “Well, that's a different thing, right? So a black box in a plane”

Q And he's like, I'm not backing down. I followed the rules. Um, curious what you make of the whole situation. What do you think it says about AI that, you know, now people can use a prompt. He basically said, you know, draw this and the AI drew it. So what do you think it means that people can just use a prompt and now it's winning human art contests?

A I think we're in a whole new world on that score. You know, later we'll talk about some of my skepticism in using AI for some purposes, but there's no question that you can get a whole breed of recent software to draw amazing paintings or things that look like paintings. Um, and society has to sort out what it thinks about. I mean, it's sort of like a performance enhancing drug. Right. Um, and it's untraceable in general. Um, and so, I mean, You know, I don't, I don't know the details in this particular case and how people found out, but in general, people are going to be able to use these techniques. Um, in, you know, the 19 seventies, people started using drum machines. Uh, and started doing all kinds of stuff with electronic music. And now, you know, in, in the studio, if you're doing music, you can, you know, change notes to make them have the right pitch. You can change the timing in subtle ways and stuff like that. Um, in general, in music, we just care what we hear and we don't really care how the sausage was made as long as it's entertaining. And maybe people will take that attitude in art. Maybe they'll, they'll be upset about it. You know, my expertise is really in what can the AI do and not so much in the ethics of attribution and so forth. If you talk about another domain, like language synthesis, it turns out that current systems can make very convincing language, …

AI assessment note: “society has to sort out what it thinks about. I mean, it's sort of like a performance enhancing drug.”

Q things that I kind of wonder about this is, you know, how does, again, like I, I understand your, your perspective on what sentience is, but like one of the thoughts I've had in, in reading about it, speaking with Blake is what are humans if not for, you know, intelligent machines trained on, you know, many, many, many terabytes of historical data. So, Where do we draw the difference?

A Because we are intelligent machines, but we're very different sort of machine. Um, and it goes back to our trying to represent entities in the world and to reason upon them and to act upon them and so forth. It's just a different set of computations that we're trying to do. I am in no way arguing that it is not possible to build a sentient machine. Um, I don't think we know how to do it, and I don't think we're clear enough on what it would consist of, but I'm not making the argument that it's impossible. I'm just looking at how this system works, and this is just not what it does, right? I mean, here's another way to think about it. A lot of sentience talk is, talk about consciousness, and A lot of what we talk about is really self-reflection when we talk about consciousness. There's a general problem here that there are many terms, they're fuzzy, they're not well-defined and so forth. But part of it is about when we reflect on ourselves, we're reflecting on ourselves in a world, um, in our relation to that world. I'm thinking about, am I making clear enough answers to you? That's part of like my self-awareness circuit and am I convincing you or not? Um, Um, you know, maybe you're not completely convinced and I'm disappointed and I'm trying to think how to make you more convinced and so forth. But, but these are with respect to constructs about the world. So I have a construct…

AI assessment note: “we are intelligent machines, but we're very different sort of machine.”

Q Gary, I was going to ask you last time we spoke, you said these things were smart enough to be dangerous, but not smart enough to, you know, be safe in some, some way. Now they're a lot smarter. Do you want them to be smarter? Like what's your perspective?

A I probably wouldn't use the word smart. I don't think. Um, I mean, what I would say is they give an illusion of being smart. Um, and of course, you know, intelligence is a multidimensional thing. Intelligence is a multidimensional thing. I would say that They can be smart in the way of, like, they can play a game of chess. There was another mind-blowing study this week that showed that one of the best Go programs, KataGo, could be fooled by some silly little strategy that would be obvious to a human player. Um, but, you know, somebody was able to follow this strategy and, like, an amateur player followed this strategy and beat, you know, a top Go program, 14 to 15. So even when we think that, like, they've solved some problem, often, you know, there are these adversarial attacks. That was basically an adversarial attack. Go that reveals how shallow things are. There are some adversarial attacks on humans. I'm sure Blake is itching to make that point, and it's true. Um, but I, I think that the general level of intelligence that humans have still exceeds, um, what machines have, that it's better grounded information that humans are better able to reason over. There are flaws. I wrote a whole book called Kludge that was all about human cognitive flaws. It's not that I'm unaware of them or not, nor that I'm unconcerned about them. Um, I still would have trouble calling the kind of …

AI assessment note: “I probably wouldn't use the word smart. I don't think.”

Q Oh, they did kill Sydney, right? Sydney was this very, I don't know, raunchy AI that tried to steal Kevin Roos' wife, so that's gone.

A Yeah, I mean, they reduced what it could do, but, um, but they stuck with it in some sense. But, you know, and like OpenAI said that we're, you know, non-profit for public benefit. Now they're desperately trying to become a for-profit. That is really not particularly interested in public benefit. It's interested in, in money. And they may become a surveillance company, which I don't think is because what you're talking about with the advertising side. So basically they have a lot of private data because they have a lot of users and people type in all kinds of stuff and they may have no choice but to monetize that. And, you know, they've been showing signs of that. They hired Nakasone who used to be at the NSA. They bought a share in a webcam company and they recently announced they're trying to build a social media company. They want, you know, they look like they're on a, a path to sell your data, your very private data to, you know, whoever they care to sell.

AI assessment note: “they reduced what it could do, but, um, but they stuck with it”