The Ledger, every show

“The media uplift paper, contrary to expectations, they found that whenever senior developers working in repositories that they understood well used AI, they were actually slowed down by 20%. Yeah. Whereas they themselves thought that they were sped up 20%,”

“Because by the end of this year, we'll get to see hopefully what a GPT-V level model looks like, and we'll learn whether we're on the path to some kind of super intelligence.”

“Then we're gonna see much more multimodal data, and I think that'll look a lot like the equivalent of supervised fine-tuning, but for a bunch of people recording their screen and doing workflows with their screen, navigating UIs. So I think you'll have agents …”

“Google is the company that actually has a successful accelerator program for AI chips already with their TPUs, which other companies are trying, but don't yet have to replace, you know, Nvidia GPUs.”

“This thing can reason, but it's making open AI ten billion dollars a year. And McDonald's and Kohl's make more than ten billion dollars a year, right?”

“The marginal cost of keeping an H 100 running is much lower than the cost of keeping a human alive for a year.”

“Right now there's ten million H 100 equivalents in the world.”

“It is the case that for the amount of compute it costs to train a system if you like set up a cluster to train a system you can usually run a 100,000 copies of that model at typical token speeds on that same cluster.”

“If you look at what it costs to train GBT for originally, I think it was like 20,008, 100 over the course of a hundred days. So I think it costs on the order of like half a million to a hundred million dollars, somewhere in that range. And I think you could tr…”

“And what's more important is that they're adding an America sized amount of power every couple of years. It might be more longer than every couple of years. Whereas our power production has stayed flat for the last many decades.”

“Manus seems like in the middle ground here where I wouldn't want to just generally try to harm China by tariffing batteries or cars or something. This is an application of AI and it's complimentary to American AI Foundation Labs because they're using the cloud…”

“The clause in the open AI charger says that if the board, which is nonprofit and controls the company decides that we've built AGI, then Microsoft has no leverage over, over open AI anymore.”

“So the compute used on training a frontier system has grown four X a year for, I think like the last decade.”

“We've seen the opposite trend in AI where there's like more competitors today than there were a year ago, even though it's gotten more expensive.”

“And while people have found modifications to the transformer, which make the constant time overhead of attention, reduce the constant time overhead on attention to like, you find these little hacks with mixture of experts or latent attention, nobody has gotten…”

“And so that just sets up a huge incentive for all these hyperscalers to build their own ASICs, their own accelerators that replace the Nvidia ones. Which I think will come online over the next few years from all of them.”

“I was in China a few months earlier, and it was really striking to me how dismayed the venture capital system there felt and the tech ecosystem generally, because after the 20, 21 crackdowns, people are just like really pulled back.”

“I think it'll definitely be better at reasoning, which is trivial to say, because the training methods that we've seen them talk about, like you, I'm sure you heard the talk about Q star and what it seems to be is training the model to rewarding it on getting …”

“An H 100 costs 40,000 dollars today. The yearly cost of running it is, like, thousands of dollars.”

“Right now we're scaling up the training of frontier systems, four X a year, approximately.”