The Ledger, every show

“I think that later on, it becomes so destabilizing that China just says, we're going to do something preemptive, like do a cyber attack on your data center. And the U.S. Might do that to China.”

“I think that form of competition would be very dangerous, and because there's a risk of loss of control, and because it might incentivize states to engage in preventive sabotage or preemptive sabotage to disable these sorts of projects.”

“If the US were pulling ahead both Russia and China may have a substantial interest in saying, hey, cut this out. Pulling ahead to develop super intelligence, which could give it a huge advantage and an ability to crush crush them. They'd say, you don't get to …”

“If you turn the pain dial all the way up for China in export controls and if AI chips are the currency of economic power in the future, then this increases the probability that they want to invade Taiwan.”

“Even so, I still think if you really tighten the export controls, you made it so that China can't get any of those chips at all, and this is your, one of your biggest priorities, they're just going to steal the weights anyway.”

“If you could have one AI do that, then you could make, you know, a 100,000 copies of these and have them perform research simultaneously. So that could lead to some very substantial acceleration in the rate of development. You might get a decade's worth of AI …”

“In the shorter term when AIs get more agential, I'd be concerned about AIs causing cyber attacks on critical infrastructure, possibly by as directed by a rogue actor. There'd also be the risk of AIs facilitating the development of bioweapons in particular pand…”

“We are finding that with the most recent reasoning models quite unlike the models from two years ago, like the initial GPT-IV, the most recent reasoning models are getting around 90th percentile compared to these expert level virologists in their area of exper…”

“Them doing brainstorming to come up with ways to make viruses more dangerous, I think that's a capability that they've had for over a year, the brainstorming part, but the implementation part seems to be fairly different.”

“So, I think in bio, actually, the I would not be surprised if in a few months there's a consensus that there expert level in many relevant ways in that we need to be doing something about that.”

“When there's very high performance on, on that benchmark that would be suggestive of something that has, say, in the ballpark of superhuman mathematician capabilities. And so I think that would revolutionize the, academy quite substantially. Because all the th…”

“That is separate from the new reasoning paradigm that has emerged in the past year which is where you train models to on math and coding types of questions with reinforcement learning, and that has a very steep slope, and I don't see any signs of that slowing …”

“This is why I think it might take a while for superintelligence to be developed, because there'll be deterrence around it later on.”

“So we have a paper out last week, we're just measuring the extent to which they're deceptive. And in the scenarios we have, like all the models were in these sorts of scenarios under, you know, slight pressure to lie, not being told to lie, but just some sligh…”

“I think voluntary commitments from AI companies are also a distraction because the companies will, you should expect most of them by default to just break those sorts of commitments if they end up going up against economic competitiveness.”

“China can have AIs that are totally aligned with them. The U S can have AIs that are totally aligned with them. You still are going to have a strategic competition between the two. This is going to they're going to need to integrate it in their militaries. The…”

“The AIs are getting very good at STEM PhD level types of topics, and that includes virology. So I think that they are sort of rounding the corner on being able to provide expert level capabilities in terms of their knowledge of the literature, Or even helping …”

“I mean, we have, you know, some places, you know, 30% plus of the employees at these top AI companies are like Chinese nationals.”

“All they need to do is do a zero day on Slack. And then they can know what DeepMind is up to in very high fidelity and OpenAI and XAI and others.”

“They're in the ballpark of, like, 10 to 20% overall. They're the very best models.”

“There has been a fair amount of generalization from training on coding and mathematics to other sorts of domains like law, for instance.”

“No, like, super weapon type stuff, but more conventional type of warfare, like drones and things like that, I expect that they'll continue to race and Probably not, maybe not even coordinate on anything like that, but that's just how things would go.”

“When performance is near the ceiling, I think that'd basically be an indication that, like, you have something like a superhuman mathematician or a superhuman STEM scientist for, in many ways, for when they're, when closed-ended questions are very useful, such…”