The Exchanges

Q help put together an executive order around the stuff late in 20, 23. Tell me what that order actually in practical terms will do. Like, what will it slow down this process? Will it actually create actionable protections for us? Or is it just, I don't know. I mean, again, it's an executive order. It's not, it's not law, you know, a congressional law. What, what, what does it do?

A Well, so there's multiple parts to answering this question. So, I mean, this is, I think, like, a 111 pages. It was done in record time. In six months, it touches algorithmic bias, so AI that's used in current AI systems that are biased and how to deal with those issues. It deals with AI and biological weapons and needing to lock down the supply chains for where people can get dangerous materials and saying, you know, we need to handle that better. It deals with AI and the next GPT-V and GPT-VI systems. It says that if you train a system That uses more than 10 to the 26, I believe, flops or floating point operations, too technical jargon. Then you have to notify the government. That's like basically like saying, if you're building a nuclear weapon that's really powerful, the government has to know. Yeah. But to your real point, your real question you're asking is, what can that executive order do? Because it's not law, it's an executive order. It's not legally binding for making sure that all the companies have to do all these things. Um, A lot of it is changing what's called federal terms and conditions. So to get federal funding, if you're a biology lab, you will not be able to get that funding if you don't do these new sort of protective measures for the dangerous biological materials. So what that's doing is using the leverage of the government and its funding power to star…

AI assessment note: “A lot of it is changing what's called federal terms and conditions.”

Q So there's a lot of excitement around it and not that much skepticism. And so I wonder in a world where, where profit is incentivized, obviously we live in a capitalist System. You know, what would stop somebody from pursuing this at lightning speed? I mean, if they're incentivized by financial rewards to pursue it.

A Well, if they're fully incentivized to go as fast as possible, and there's no counter incentive that says you're liable, let's say for the harms that might show up, Then of course you're going to go as fast as possible. And so that's, that's why in our work, like people think that we're criticizing Sam Altman or open AI or one company, or we're criticizing AI overall. No, neither of those things are true. What we're criticizing are perverse incentives that lead to bad outcomes because we are true futurists who want the good future. And we see that to get that future, we have to change the incentives that we're currently operating with. And a good example of this is liability. So What lesson did we learn from social media? For those who don't know or remember this, in 1996, there was this thing called the Communications Decency Act, in which there's a section, famously called Section two 30, that basically gave all internet companies an immunity shield, that you would not be liable for anything that your online bulletin board, where someone posts hate speech or something like that, or tells people to commit suicide or smear someone, that you would not be liable For any of those harms. And that made sense when the internet was just a bunch of bulletin boards and it was not powered by AI, but we used that immunity shield and applied it to social media companies when they came alon…

AI assessment note: “if they're fully incentivized... and there's no counter incentive... of course you're going to go as fast”

Q exponential curve as good as it'll be in A year, five years, 10 years from now. What are we talking about? To lay it out for me, Tristan, I mean, are we talking about a world where nothing is real and everything is, I mean, it's, it's like Aldous Huxley again on steroids. We just will not be able to even know if a call from our spouse is real.

A Yeah. So AI is not going to get worse at emulating someone's voice, someone's handwriting, someone's likeness, their This, that's what generative AI does, is it gives you those capabilities. Anything that can be emulated will be emulated, and that's why it's generative AI. It's generating text, generating images, generating three D models from scratch, generating architectural designs, generating movie scripts, generating amicus briefs, generating, you know, fake articles about people. Anything that can be simulated will be in New Hampshire. Uh, someone at deep fake Joe Biden And automated some robocalls with Joe Biden's voice telling people not to vote in New Hampshire. Yeah. And I actually listened to it. To be honest, that one, I would have thought that that sounded like Joe Biden. And the point is that it's, that's the worst that it will ever be. So if you're not impressed today with where it is, just look at the growth rate of how much better and how quickly it's getting better. And what can you do in the face of that? Well, you know, by default, yes, if we live in the world that we live in today, we won't know what's true. But I was just talking to the digital minister of Taiwan, Audrey Tang, and she's talking about the need for authenticated, privileged messages. So like, anytime the government sends a message, it now comes through one number. If you get a text from the …

AI assessment note: “by default, yes, if we live in the world that we live in today”

Q All right, so what does that effort look like? Like, what do we need to do to, you know, to prevent the AI version of a nuclear catastrophe, right? Especially when so many of these AI tools are publicly available for anyone to use.

A We need there to be different norms around that, where we probably don't want to open source the really advanced AI systems that are coming. Think of it this way. For those who don't know, by the way, What we're talking about with open source AI models, which is different than open source code. Open source code is more safe and more secure because if I do Linux in an open source code way, more people look at the code, they can identify the bugs, they can improve the code. It makes the overall thing safer, more secure, more trustworthy because it's so transparent. But AI systems, uh, AI models that are open source means that anybody can retrain them to do even more dangerous things. So for example, Facebook released Lama two, their open AI model, and they tried to tune it to be safe. So if you ask it, how do I make a biological weapon? It will not answer. You would say, sorry, I can't answer that. Yeah. But once it's out there in the open, I won't go into technical details, but basically for about a hundred dollars, you can retrain all the safety controls off of it. So you can say, be the worst evil version of yourself or your evil twin personality, and it'll suddenly answer happily any questions about biological weapons. Now it's not smart enough to have accurate, really deeply accurate instructions about how to do that, but we probably don't want to be releasing Lama three, La…

AI assessment note: “we probably don't want to open source the really advanced AI systems that are coming”

Q in late 2022, early 20 23, Tristan started getting messages from people working on a different technology. You and Aza Raskin, one of your co-founders, were contacted by people working in AI, and they were They were sounding the alarm, and the analogy that you've made about this encounter really alarmed me. You compared it to, like, to the Manhattan Project. Can you describe or explain that a little bit?

A Yeah. Well, a lot of people analogize that the invention of AI is as significant as the invention of the atomic bomb. Now that might sound like alarmist or panic creating or something like that. And it's, it's not actually, and it's because of how the atomic bomb restructured the world order. It was a new kind of power that whoever had it was clearly sort of the dog on top of the food chain. And with AI, what people don't understand is if you genuinely build something that can do full artificial intelligence across all kinds of cognitive labor, meaning scientific labor, research labor, market analyst labor, financial labor, if you can out-compete all stock traders on the stock market, if you can out-compete all military strategy games, if you can out-compete anyone in writing, uh, text to influence people on the internet, If you have that AI system that can do that, that is a new kind of power in which your position in the world will be greatest relative to everybody else. And so there is a new atomic bomb project in the form of those who are racing to build towards this, you know, fullest expression of AI, which some people call artificial general intelligence. Remember that the stated mission statement of OpenAI, Anthropic, and DeepMind from the very beginning, those three companies, Have been to create artificial general intelligence. So when we got calls from some people in…

AI assessment note: “it felt like getting a call from a scientist working on the Manhattan Project”

Q now just, you know, it's an article of faith. We now know that the attention economy is a term, and we now know that tech companies make money from Capturing our attention. You know, media does, YouTube does, this show does, right? But you were thinking about this a long time before others were as a potential problem. How did those ideas begin to kind of percolate in your mind?

A Well, again, starting with my experience at Apture, I realized that for, you know, for our company to be successful, we had to increase the total amount of time people spent on the Washington Post and New York Times. And that there obviously isn't an infinite amount of human attention out there. It's almost like planetary boundaries, right? Can you run infinite economic growth on a finite planet forever? Well, so long as economic growth has environmental externalities or depletes finite resources, you can't do it forever unless you keep making scientific breakthroughs on every dimension. So, you know, I think similar to that, you can't run infinite growth of every company demanding more attention per quarter per year forever. With a finite supply of human attention. And at the time in 2007, it was, it's actually the year the iPhone came out. We started our company before the iPhone came out, but obviously what mobile phones did is they opened up the attention economy used to be people only spend a few hours a day on computers, and then they would get offline and go outside, go to a movie, do something else. And increasingly every moment of our lives became part of this attention economy sold for a commodity, like The time you sit on the toilet and you go to the bathroom and that's a boom. Now there's 10 extra minutes in the attention economy because the smartphone opened up tha…

AI assessment note: “starting with my experience at Apture, I realized that for, you know, for our company”

Q at least in the United States, because at the end of the film, it's like, it says, um, this is just a representation of what could happen in nuclear war, and in fact, it will be much, much worse than what you've seen. And I don't know, is there a world where it's worthwhile creating some, like a day after around AI, so people just understand what we're possibly facing?

A Yeah. I'm so glad you're bringing this up. And you know, what's, what's interesting about it is, you know, Reagan had military advisors saying we can win a nuclear war. We just have to keep, we got to keep building and building bombs and keep building them, have more of them. Yeah, exactly. And if one side believes that the other side actually believes that they're going to try to win a nuclear war, that's what creates the risk. Cause then everyone's on hair trick or alert for anything that looks like it could be a nuke. And then that something that's an accident, like a flock of birds comes across the radar and you almost hit the button. So what we needed to do was create a new trustworthy basis for coordination that the US and Russia would trust that they're actually so existentially terrified by Armageddon that both of them would fear everyone losing more than they fear me losing to you. Yeah. And I think what that film the day after did is it painted a picture of how everyone loses if this happens. So it This actually can have a really big impact. And the point of this from a metaphorical stance is that we as public communicators, you guy with this podcast and people who are listening to this, we have to make the dark future legible so that we can steer towards the light. If we don't have the dark future be legible and people just want to focus on AI making cancer better an…

AI assessment note: “we have to make the dark future legible so that we can steer towards the light”

Q You eventually started your own, your own company. This is in the 2000, the mid 2000 called aperture. And this was a, it was a hyperlink. You would like click over a high hover over a hyperlink and you would get like other information, right? More or less. Can you describe what that company did?

A Yeah. I started Apture in 2007 and it was going back to the original ethos of what the internet and hyperlinking was supposed to be about. You know, you could say when people click on this button or click on this image or click on this text, I want the computer to speak this out loud or to issue a dialogue prompt or to play this song or to play the piano. And it was the first really creative idea of this interlinked, what if everything was connected to everything else in the most visually expressive multimedia educational Inspiring, beautiful way. So then I learned a lesson though, because while I was interested in kind of re-inventiating this original inspiring ethos of multimedia and teaching and education into the internet, the way that Apture, which is a, you know, for-profit company, we raised venture capital, we had investors, uh, and our progress, our, our success was measured not in the number of children who were inspired by cool Media that they got to learn and watch and click on, but our progress was measured in the form of engagement. You know, we did deals with the Washington post, the New York times, the economist, and they would ask us one question, which was how much did you increase the amount of time people spent on my website? Okay. How many ad views impressions did you increase? And that's when, you know, I really saw, I think the fallacy that, that leads an…

AI assessment note: “I started Apture in 2007 and it was going back to the original ethos”

Q And you did, and you and your co-founder, Asa Raskin, Would go on to build a presentation called the AI Dilemma that you, I think you first delivered it in March of 20, uh, 23 here in the Bay Area. You presented several times since then. I've seen it live. It's available online. Anybody can watch it. What was a message that you were trying to send with this presentation?

A You know, the main point we made in the presentation is that social media was really humanity's first contact experience with a mass deployed AI because it's an AI Pointed at your thirteen-year-old's brain trying to calculate what's the perfect next TikTok video to show you, or what's the perfect next politically outrageous tweet that will keep you scrolling. And we saw how that experiment went. You know, there's a lot of really good things that social media has done, but the overall picture of what social media has done, if you really take into account the full scope of impacts and externalities, is it's also created the backdrop of a more addicted, outrage, polarized, narcissistic, breakdown of truth, You know, lower trust society, and that that is like undermining the quality of the soil underneath your feet while you have a few nice shiny toys above the soil. We knew that no matter what positive stories we were telling about social media, unless you pay attention to the incentives, which is to maximize attention and engagement, which produces that race to the bottom of the brainstem, that that was going to be the driving force of what would tell you which future we're going to land into. And if we care about which AI future we get, We're going to hear a lot of positive stories about AI developing antibiotics and solutions to climate change and new drugs for cancer. And of c…

AI assessment note: “the main point we made in the presentation is that social media was really humanity's first contact”