Brundage: Reducing AI existential risk by 0.1% preserves massive expected value
Miles Brundage · A.I. Policy and Public Perception - Miles Brundage and Tim Hwang · Y Combinator · Apr 25, 2018 · at 33:39
Miles Brundage outlines the expected utility argument for AI safety and existential risk research at Oxford's Future of Humanity Institute.
0:00 / 0:16exact quote · 16.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Reducing the probability of existential risk is super important, even if AI is you know, decades or centuries away, and even if we can only, you know, decrease the probability, you know, of that happening by, like.1% or whatever in expectation, that's, like, a huge amount of value that you're that you're protecting.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Miles Brundage
Brundage: Scalable AI vulnerability discovery could cause a catastrophic cyber attack
“It could be that you know, something truly catastrophic could happen if you sort of combine the, Scalability of AI and digital technology in general with, like, the adaptability of human intelligence for, like, finding vulnerabilities. If you put those togethe…”
Brundage: Technical transparency in AGI could enable international non-aggression pacts
“And I think if you actually had the full development of the FAT methods, and you had accountability and transparency for even general AI systems or superintelligent systems, I think that would open up the door for a lot more collaboration. If you could sort of…”
Brundage: AGI accountability may be easier than thought if corrigibility stabilizes feedback
“Corrigibility, what he calls corrigibility, and what others have called corrigibility, might actually be, like, a stable basin of attraction, in the sense that if a system, you know is designed in such a way that it's able to, like, take critical feedback, and…”
Brundage: China will not slow AI deployment over interpretability concerns
“In China, there's, like, much let, or I haven't seen as much concern about interpretability, though there are some, like, good papers coming out of China, but in terms of, like, governance, I haven't gotten the sense that they're gonna, like, hold back the dep…”
Brundage: AI Governance Lacks the Methodological Rigor of Climate Science
“People have been talking about AI AI ethics and AI governance for a long time, but there hasn't been much dialogue between, you know, this world and then the other worlds of, like, you know, science policy and public policy, and, you know, one way to think abo…”
Brundage: AI will achieve superhuman performance in StarCraft within three years
“Yeah, I think there will be Superhuman, Starcraft and Dota too, probably in that time horizon. I said in, I think early 2017 that it would be the end of, that I gave like 50% chance by the end of 2018. So this gives me more runway. I'll say, yeah, like, 70% co…”