The Ledger, every show

“And so you're gonna have this explosive dynamic. And once we get like that loop closed, I think it would just be like, 20% growth plus.”

“Once we get AGI humans will not have high paying jobs.”

“The cost of training, the systems is declining so fast that Literally you will be able to train a super intelligence in a basement at some point in the future.”

“In the 80% of worlds or 90% of worlds where we don't get paper clipped you will get to say you worked on something really cool at a time that was really important in the history of humanity.”

“And they actually don't, they can't like learn over the course of six months, how to become a better editor for me or how to become a better transcriptor for me. And since a human hire would be able to do this, they can't. So therefore it's not AGI.”

“It's precisely because I don't have an obvious solution that I think we're many years away.”

“I think the problem has been that people underestimate how many things are truly needed to automate human labor, and so they think, like, we've got reasoning, and now that we've got reasoning, we've, like, this is what it takes to take over a job, and I think,…”

“I'm not saying this is the world I want. I'm just saying like, just like, think about it physically. If you're colonizing the galaxy, which you can do potentially after AGI. I mean, I'm not saying like it'll happen tomorrow after AGI. Right. But like, there's …”

“One is even if your labor is not worth that much the property you own is potentially worth a lot, right? Like if you own an S&P 500, And there's been explosive growth. You're like a multi multi-millionaire or the land you have is like worth a lot.”

“I would prefer for significant amounts of redistribution in this world because like the libertarian argument doesn't make sense if there's like no way you could physically pick yourself up by the bootstraps. Like your labor is not worth anything.”

“AIs will be integrated through all the firms in the economy. A firm can have property. Firms will be like largely run by AIs, even though there's nominally a human board of directors and it might not even be nominal, right? Like maybe the AIs are aligned and l…”

“Comparative advantage is totally consistent with like with human wages being below subsistence.”

“If all human wages go to zero or go below subsistence, then the only way to deal with that is through some kind of UBI rather than, you know, if you happen to sue open AI, you get a trillion dollar settlement. Otherwise you're kind of screwed.”

“Now if in future your population is Your effective labor supply is, like, largely AIs, then you just, like, this dynamic just means that, like, your inference capacity is literally your geopolitical power, right?”

“For maybe five more years, you could have, you could keep increasing the share of energy that we're spending on training data centers or the fraction of TSMC's leading edge nodes. Wafers that we dedicate to making AI chips, or even the fraction of GDP that we …”

“The media uplift paper, contrary to expectations, they found that whenever senior developers working in repositories that they understood well used AI, they were actually slowed down by 20%. Yeah. Whereas they themselves thought that they were sped up 20%,”

“I'm like, I don't know, I'm like a 20% that like we'll have some sort of intelligence explosion.”

“I think it's less like the nuclear bomb where there's a self-contained technology that is so obviously relevant to specifically this like offensive capability. And you can say, well, like there's nuclear power as well, but like neither of those three, like nuc…”

“If you have higher inference capacity, not only can you deploy AIs faster and you have more economic value that's generated, but you ha you can have a single copy, sorry, a single model learn from the experience of all of its copies. And you can have this basi…”

“From the misalignment stuff, the main thing I worry about is the AI playing us off each other rather than us playing the AIs off, off each other.”

“What I would like to see happen between the U.S. And China, basically, is, like, the equivalent of some red telephone during the Cold War, where you can communicate Look, we noticed this, especially when AI becomes a more integrated with like the economy and g…”

“And I think that that will have to be unlocked before most of the economic value of these models can be unlocked. And so by the point these labs are like worth hundred, they're already worth hundreds of billions of dollars, but by the point they're generating …”

“If you pay an employee a hundred million dollars and they're a great AI researcher and they make your compute your training or your inference one percent more efficient. Zuck is spending on the order of like eighty billion dollars a year on on compute. That's …”

“The key problem I see Is this, the models can't do on-the-job training. So if you think about a human employee you might have some. And these human employees, the good thing about them is that, you know, you train them for six months or a year, and over time …”

“I think the, this is at least an order of magnitude less efficient and less less capable than the way humans learn.”

“This, like, tacit knowledge that you build up through practice is not this, like, written instruction manual that you can just write out as a system prompt.”

“I think the bigger problem is not just the width or the width of the pool, how many different tasks you have to RL on, but it's a depth in the sense that a job doesn't involve doing a thousand different five minute tasks individually. It's the fact that you're…”

“Even if there's no software progress out of that point, that algorithms aren't improving, just that ability to learn on the job from everything in the economy would functionally produce what looks like a super intelligence, right? No human will be, will have m…”

“I mean, I still think they're, like, underpaying them. I think, like, Meta's the first company that is actually coming close to the break-even point of what the best AI researchers are actually worth the company.”

“I don't think we're just right around our corner from AGI and it's just a little additional dash of something. That's all it's going to take. I think, you know, people often ask if all AI progress stopped right now and all you could do is collect more data or …”

“I think a big bottleneck these models have is their inability to learn on the job, to have continual learning. Their entire memory is extinguished at the end of a session. There's a bunch of reasons why I think this actually makes it really hard to get human-l…”

“I still think I I'm like, I'm not confident that this will generalize to domains that are not so verifiable or text-based.”

“And the reason I don't think that's around the corner is just because there's not, there's no obvious way, at least as far as I can tell, to just slot in this online learning into the models as they exist right now.”

“Pre-training, which is this idea that you just make the model bigger that has had diminishing returns.”

“I'm expecting a fifty-fifty if I had to like make a guess, I had to make a bet. I just say, 20 32, we have like real AGI that's doing continual learning and everything.”

“Like I do think companies will continue to pour exponentially more computing to train these systems and they'll continue to do it over the next many years, because even if there's diminishing returns, the value of intelligence is so high that it's still worth …”

“If you look at things like how much energy is there in the country, how much how many chips can TSMC produce and what fraction of them are already being used by AI. Even if you look at like raw GDP, like how much money does the world have? How much wealth does…”

“Once continual learning is solved, you might have something that looks like a broadly deployed intelligence explosion, which is to say, That because if these models are broadly deployed to the economy, every copy that's like this copy is learning how to do plu…”

“Now if we're getting to the world where you got to like do a project for seven hours and then at the end of those seven hours, then we tell you, Hey, did you did you get this right? Then like the progress just goes on a bunch. Cause you've gone from like getti…”

“I'm genuinely not sure how likely an intelligence explosion is. I don't know. I'd say like. 30% chance it happens, which is crazy by the way.”

“If you do self continue to learn it, I think like you could get rid of a lot of white collar jobs at that point. And what is that worth? Like at least tens of trillions of dollars, like the wages that are paid to white collar work.”

“I think they're treating it as like a sort of like toy within the meta universe. And I don't think that's the correct way to think about AGI.”

“The reason they're not being more widely used is not because people cannot afford a couple bucks for a million tokens. The reason they're not being more widely used is just like, they fundamentally lack some capabilities. So I disagree with this focus on the c…”

“I think the long run picture is, That yes, there will be misaligned intelligences and we had to figure out a way to be robust to them.”

“More importantly, I think the future economy, once we do have these AI workers will be denominated in compute, right? Cause if computer's labor, right now, if you just think about like GDP per capita, because the individual worker is such an important componen…”

“I think the thing that's underrated is we humans have this global hive mind where the reason we can make iPhones and we can make buildings and whatever is not just intelligence and also not just agency. It's the fact that there's so much specialization. There'…”

“I think it'll probably be one of the foundation lab companies. People have been for years trying to build agents and they just haven't worked. And it makes it makes me think that that's a fundamental limitation of the current models. And so it'll just be the c…”

“So if somehow this whole deep learning paradigm is wrong and we just, like, totally missed the boat somehow, then I could see it happening and that's, I give it a 10, 20%.”