The Ledger, every show

“And my contrarian take is that it wouldn't be that bad. Supply would take a shock for sure, but The best processes that we have in the West, like Intel, not that far behind, at worst, like maybe two X worst performance per watt. And the gap is just far smaller…”

“You'd have missing zero buyers for a data center that is 95% uptime. I'm that first buyer. I will buy 95% uptime.”

“If you look at, you know, Hopper to Blackwell to Rubin, and you compare like for like, what is the performance per watt of a beefload-sixteen multiply? It hasn't improved all that much.”

“If you look at TSMC five nanometer versus four versus three versus two, the performance per watt on these chips doesn't change like a dramatic amount.”

“The challenge is, all those companies, you could take your pick, Base-Ten, Fireworks together, they all focus on low latency inference. And they were pulled in that direction by one very important customer Cursor.”

“You wanted more persistence, more long horizon tasks, and now it's, to me, very obvious that the future of agentic inference is long horizon tasks. You're gonna run the machine for hours or days at a time. It doesn't matter if it spits out tokens at a hundred …”

“So long-term, I think, you know, we're going to end this year at maybe fifty-fifty background and real-time workloads, but I see this going to ninety-ten in favor of background.”

“When you want secure software, it's really a question of how many dollars did you spend on Anthropix APIs trying to break into your software. That is the best indication for how secure it is, because that's the best tool in the world.”

“And we have actually started to bring it within view, a dollar cost for these long horizon tasks that is reasonable. It's not millions, it's thousands, and maybe it could be hundreds or even tens of dollars in the near future, too. Have a definitive answer to …”

“NVLink is mandatory, I would say, for low latency inference.”

“I think I'm not holding my breath for other companies broadly to figure out NVLink quickly. It's challenging technology to figure out. It's hard to scale. It's hard to productionize.”

“And so there are other chips that definitely rank higher than NVIDIA on flops per dollar. But they may not have as much interconnect”

“Cerebris and Grok and maybe a couple others, you should think of them as accelerators. What they are really good at is being used in conjunction with a more traditional GPU-like device that critically has this off-chip memory built in.”

“NVIDIA sees the, if they just let the most deep pockets buy all the chips that Maybe hurts them in the long term if that customer ends up accruing a lot of more power. They understand that compute is power today, and so they're quite strategic about how they a…”

“I like to say there's no bad chips, there's only bad pricing. And I will make any chip work at the right price.”

“And basically there's no way to build a gigawatt data center in the United States easily anymore. Even a hundred megawatts is, is increasingly hard. It's basically impossible unless you're a very special set of customers. 10 megawatts is probably on the edge o…”

“We tell our customers, look, our average throughput is going to be very competitive, but our P 99, our 99 percentile latency is not going to be controlled. It cannot be. And in return, I'll give you unbeatable economics. And I think that's the right fit for ba…”

“The trick is that it's going to give me better access to power that no one else is going to touch, because it is so annoying to deal with that kind of outage. And if my chips are cheap enough, they're probably not going to be Nvidia racks. And if my chips are …”

“NVIDIA's pumping out five million Blackwell chips this year.”

“We all make fun of XAI for having, you know, some challenges with total flop utilization on its clusters, but the reality for the rest of the world is it's far worse. A ton of GPUs just sit in warehouses or sit In private pools allocated to a specific customer…”

“In a line, I would say the labs pay an immense premium to be three to six months ahead of everything else.”

“I don't know that the premium for being three to six months ahead is going to last that long. I mean, if you look at, like, enterprise deployments they don't move at three to six months speed.”

“When I talk about building custom chips, and they ask me, oh, so what's different? Basically, it's about sidestepping the HBM shortage and focusing on More extreme offload to other forms of memory, such as flash.”

“You want to pick something and say, I think they've underpriced the impact of how short we're going to be on HBM. We're going to push really hard in this other direction instead, which, you know, as an aside, I do think is probably the thing to attack most.”

“I think they're gonna make everything else more expensive. I think that iPhones will cut their memory, iPhones are gonna go up in price, and we're just gonna deal with it.”

“Opus Four Five was the first agent that was at all suitable for longer horizon tasks.”

“It's not the case that Fable finds a superset of all bugs in software. You would find some bugs with a very small model that you don't find with a large model.”

“The long lens view to take on this is that we have a form of intelligence that can tackle any verifiable problem. Any verifiable problem means most software. It means a lot of formal, like math proofs and similar. And it could also mean scientific discovery”

“We have not solved human taste yet, and I don't know that it fundamentally can be.”

“And I would say the original sin of Transformers is that you've taken this extremely fundamentally memory bound layer and juxtaposed it right next to a compute bound layer. It is very difficult to have a single chip that is good at both compute operations and …”

“Models have seen the entire internet many times over at this point, and there is not a whole lot more to be done on human data from the internet.”

“The signal you get from random human preference, or I guess unconditioned human preference, is not actually worth anything anymore. You want expert human preference at this point.”

“Basically, the idea is that if you want artificial general intelligence, the best way to get there is to just keep stacking specialized intelligences until you have no more gaps to fill.”

“I'm sure in six months time we'll have much better models on kernel engineering, and I'm sure the labs would tell you that they already do a lot of their kernel engineering in a fully automated way.”

“That operation runs at, you know, 70, 80% of peak utilization, and it's limited not by software, but by power. The way NVIDIA quotes peak flops is a little optimistic. You never hit that because of power throttling”

“AMD, I think, great chips overall. The challenge is that people don't understand how to program them very well.”

“I think publicly, Meta and OpenAI have bought a ton of AMD chips, and so, we're increasingly seeing that all the AMD supply is also being allocated.”

“And I think what's interesting about token consumption or AI consumption broadly is that it's no longer speculative. People buy tokens because they're immediately valuable to them. You don't hoard tokens, you use them immediately.”

“Well, first we scavenge chips, and then we scavenge power for those chips. The idea is in both cases, I do not want to be bidding against Anthropic or of an AI for a complete capacity. I'm not going to win against them, and I don't want to. I want to be more c…”

“Specifically, the KV cache is quite uncompressed right now. I think if you look at the entropy in a KV cache, it's nowhere near, it's not earning its keep. Like, we're storing many kilobytes of data in the KV cache per token. And that's probably off by an orde…”

“I don't look for lots of AI experience. I don't look for, you know, CUDA experience at all. That's actually a huge red herring. I mean, CUDA as a concept or GPS as a concept have evolved so much in the last five years. There's no point asking for 10 years of e…”

“NVIDIA is really smart about this. They don't compete with their customers. NVIDIA takes a long view on everything. Why don't they even start with the NeoCloud? Why don't they just sell computer out the back door? Well, NVIDIA is really good. Jensen is really …”

“Like, he wants to create a diverse community of NeoClouds and inference providers who are all jockeying to create demand for NVIDIA, such that if any one of them Decides to, I don't know, vertically integrate or go with AMD or any other option. He's got three …”

“We think that whenever you make something 10 times cheaper, it's a new product category, and we aspire to do that for tokens.”

“I don't think tokens are the final unit of work or intelligence, but they are what we use today, and so it's very straightforward.”

“And increasingly, I think we will have agents do some unit of work, take as many shots on goal as they can, and however many tokens they use to get there is going to be kind of a dependent variable depending on the task. So you think about, like, agents that s…”

“Agents are capable of running for an hour at a time. I wouldn't say it's days, but definitely an hour is quite suitable today.”

“A lot of people I worked with in NVIDIA in 2015, 20 16 are still there today. That company has incredible retention, and these are the best engineers, frankly, on the Silicon side, at least, I've worked with them my whole career.”