Apr 25, 2018 · 45m · y-combinator
A.I. Policy and Public Perception - Miles Brundage and Tim Hwang · Y Combinator
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this Y Combinator podcast, host and guests Miles Brundage and Tim Hwang explore the evolving landscape of AI policy, public perception, malicious threats, and system governance. The conversation details strategies for bridging near-term technical safety and algorithmic bias with long-term artificial general intelligence (AGI) alignment and international regulatory frameworks.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The partners as informed peer 3.1 Guest teaching 3.4 Guest disagreement 0.3 The partners pushing back 0.6
speaking balance: gold is the partners, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 43:11 Tim Hwang firmly rejects closed AI research models
Tim Hwang takes a strong principled stance against withholding research publications, arguing that public policy cannot function if researchers withhold discoveries.
Hardest push from the partners ▶ 9:16 Host pushes back on public technology backlash framingThe host challenges the idea that public backlashes target underlying AI algorithms, arguing that people criticize specific product interfaces like Facebook's News Feed rather than AI.
Biggest teaching moment ▶ 18:49 Tim Hwang explains the philosophical clash inside CS over interpretabilityTim breaks down how traditional computer science determinism clashes fundamentally with empirical data-driven machine learning on the necessity of interpretability.
The partners hold their own ▶ 19:31 Host connects ML interpretability to biological empirical modelsThe host demonstrates domain synthesis by comparing empirical machine learning acceptance to statistical significance thresholds in biological sciences.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The partners as informed peer | Guest teaching | Guest disagreement | The partners pushing back | Why |
|---|---|---|---|---|---|---|
| Miles Brundage on "The Malicious Use of Artificial Intelligence" Report | 2 | 3 | 0 | 0 | The host opens with standard introductory questions regarding the title and scope of Miles's malicious AI report. Miles and Tim collaboratively explain the omni-use nature of AI and the relevance of responsible disclosure. | |
| Research Methodology and Emerging AI Threats | 3 | 4 | 0 | 0 | The host asks about research methodologies for spotting emerging threats in code. The guests take over most of the segment discussing technical paper extrapolation, deepfakes, and the difference between physical and virtual attack vectors. | |
| Public Perception vs. Reality of AI Technologies | 4 | 4 | 1 | 2 | The host challenges the framing of public panic by pointing out how users blame the News Feed rather than AI itself, and notes the friendly aesthetic design of autonomous cars. The guests expand into robustness issues and clinical healthcare applications. | |
| Defining Interpretability and Global Policy Approaches | 5 | 4 | 0 | 1 | The host prompts a clear definition of interpretability for lay listeners and contributes analogies comparing machine learning empiricism to statistical significance in biology. Tim details transatlantic regulatory divergences under GDPR. | |
| Miles Brundage on AI Policy Research and Scenario Planning | 2 | 3 | 0 | 0 | The host inquires about Miles's doctoral dissertation topics. Miles explains his scenario-planning methodology and contrasts AI policy maturity with climate science IPCC rigor. | |
| Historical Context of Technology Policy and Translation Roles | 4 | 3 | 0 | 1 | The host asks whether tech policy ecosystems have historical precedents or are novel personal-branding developments, later summarizing the dynamic as an 'arbitrage'. Tim and Miles discuss technical-to-policy translation and nuclear history. | |
| Policy in Practice: Corporate, Academic, and Institutional Dynamics at FHI | 3 | 3 | 0 | 0 | The host asks about institutional differences between private corporate policy teams at Google and academic think tanks like FHI. Miles clarifies FHI's focus on existential risk reduction and interdisciplinary rigor. | |
| Bridging Short-Term and Long-Term AI Safety Concerns | 2 | 4 | 1 | 0 | The host asks how short-term and long-term AI safety concerns intersect. Miles explains value alignment, drawing parallels to arms control verification treaties and AI corrigibility. | |
| Open Research vs. Responsible Disclosure and Future Predictions | 3 | 3 | 1 | 1 | The host clarifies a question regarding public vs private development incentives and prompts future predictions. Tim takes a firm stance in favor of open publication while Miles predicts superhuman Starcraft performance. |
Statements from this episode (16)
Hwang: Crypto hype cycle benefits AI by tempering boom-and-bust cycles
“I mean, I think it's actually, yeah, good development, right? Like, I mean, the history of AI is like all of these winners, and like, having another hype cycle to kind of balance it out might actually be a good thing.”
Brundage: Responsible disclosure norms are missing for AI adversarial vulnerabilities
“So things like responsible disclosure, when you find out about a new vulnerability is something that's pervasive in the computer security community, but hasn't yet been seriously discussed for things like adversarial examples, where you might want to say, hey,…”
Brundage: Scalable AI vulnerability discovery could cause a catastrophic cyber attack
“It could be that you know, something truly catastrophic could happen if you sort of combine the, Scalability of AI and digital technology in general with, like, the adaptability of human intelligence for, like, finding vulnerabilities. If you put those togethe…”
Hwang: The public sees non-ML robots as AI, ignoring newsfeeds
“The newsfeed assuredly is AI, right? Like, it uses machine learning. It uses the latest machine learning to do what it does. We don't really think about it as AI, right? Whereas, like, the car is, like I mean, I think a lot of robots kind of fall into this cat…”
Brundage: Offense generally defeats defense in adversarial machine learning competitions
“If you look at the, like, offense and defense and competitions on adversarial examples, like, the offense generally wins. Like, we don't really know how to make neural nets robust against deliberate or even unintentional things that could mess them up.”
Brundage: China will not slow AI deployment over interpretability concerns
“In China, there's, like, much let, or I haven't seen as much concern about interpretability, though there are some, like, good papers coming out of China, but in terms of, like, governance, I haven't gotten the sense that they're gonna, like, hold back the dep…”
Hwang: US AI regulation is domain-specific while Europe uses broad horizontal rules
“I would say in general, I think the US moves on a very case-by-case basis. So the regulatory mode is basically to say, look, in medical, that seems to be a situation where, like, there's, like, particularly high risks. And like, we want to create a bunch of re…”
Brundage: AI Governance Lacks the Methodological Rigor of Climate Science
“People have been talking about AI AI ethics and AI governance for a long time, but there hasn't been much dialogue between, you know, this world and then the other worlds of, like, you know, science policy and public policy, and, you know, one way to think abo…”
Brundage: FHI relies on DeepMind and OpenAI ties, lacks major compute
“At the Future of Humanity Institute, we have a lot of relationships with organizations like DeepMind and OpenAI and others but, you know, we don't have, like, a ton of, you know, GPUs or TPUs here like, running the latest experiments outside of, you know, some…”
Brundage: Reducing AI existential risk by 0.1% preserves massive expected value
“Reducing the probability of existential risk is super important, even if AI is you know, decades or centuries away, and even if we can only, you know, decrease the probability, you know, of that happening by, like.1% or whatever in expectation, that's, like, a…”
Brundage: Near-term AI fairness issues are fundamentally human value alignment problems
“You could frame a lot of current issues as value alignment problems, so things around bias and fairness. So I think ultimately, you know, there's a question of how do you extract human preferences, and how do you deal with the fact that humans might not have c…”
Brundage: Technical transparency in AGI could enable international non-aggression pacts
“And I think if you actually had the full development of the FAT methods, and you had accountability and transparency for even general AI systems or superintelligent systems, I think that would open up the door for a lot more collaboration. If you could sort of…”
Brundage: AGI accountability may be easier than thought if corrigibility stabilizes feedback
“Corrigibility, what he calls corrigibility, and what others have called corrigibility, might actually be, like, a stable basin of attraction, in the sense that if a system, you know is designed in such a way that it's able to, like, take critical feedback, and…”
Hwang: Open publishing should be the default standard in AI research
“I'm very pro open publishing. Like, I think, like, it should be the default, and it's like, I'm still disputing situations where I'm like, you shouldn't publish on this stuff. Just because like, I think it is actually to the benefit of everybody to know what t…”
Brundage: AI will achieve superhuman performance in StarCraft within three years
“Yeah, I think there will be Superhuman, Starcraft and Dota too, probably in that time horizon. I said in, I think early 2017 that it would be the end of, that I gave like 50% chance by the end of 2018. So this gives me more runway. I'll say, yeah, like, 70% co…”
Hwang: Meta-learning will improve significantly and automate ML architecture design
“I think meta learning will improve significantly. So this is basically treating machine learning, designing machine learning architectures as if they were their own machine learning problem. It's something that basically is done by, like, machine learning spec…”