TPU

“AMD, Tranium, in many ways TPU, like all of these in my mind are all trying to build a better Reuben, right? And there's a huge amount of value in that.”

“TPUs are the second most financeable. It probably takes, I don't know, double the equity check at least. And then the rates on the rest of it are higher.”

“Google already made a deal to sell, like, 20% of their TPUs to Anthropic.”

“Serving large language model is a fundamentally different problem. Because serving it requires to run it on accelerators like GPUs or TPUs, and it is a computationally intensive process that will require a lot of engineering and ensuring that for each request,…”

“Google has two big advantages, right? It has a cache machine from search that can fund a lot of things. And then number two, it has TPU. And I cannot overstate what an advantage it is to have TPU. They've been building this chip for a long time. It's a really …”

“I think, ah, you're gonna see more and more, ah high performance and low energy inference hardware systems, because I think everyone is now realizing that inference is the key to making, you know, these agent-based systems be available to more and more people,…”

“From my vantage point, we're gonna see this proliferation, and where we have sufficient demand now, because there's so much demand for tokens, Where it makes sense to specialize at the chip level that will pay out and allow you to go through the full new produ…”

“They have, many, many of the team have designed TPU chips at Google in the past.”

“One of the advantages Google has is that their TPU can be designed in collaboration with the team building Gemini or the team of DeepMind, and so they can inform their choices back and forth, and that's an enormously powerful thing that is surprisingly relativ…”

“It's actually becoming a pretty existential risk for NVIDIA because if they don't support the NeoClouds, they're at risk of basically seeing Google Amazon and others take tremendous market share on the silicon side, right? So you would see Tranium take, take m…”

“So over time, these GPUs have, you know, gotten, you know, as good and better than TPUs, you know, as the career of the TPU, I have to admit, GPUs are now better.”

“Every GPU, every TPU, every AI chip that was serving these models were just, like, fundamentally built before this and are retrofit to serve these modern models.”

“And you have TPU, Tranium, and AMD, Which are all you know, essentially trying to be a better GPU, and today I think probably Tranium is doing the best.”

“So last month, you know, we signed a five gigawatt deal with Google and with Broadcom for TPUs starting in twenty-twenty-seven.”

“In other words, for us, for let's say our TPUs, we co-design them with a building. We co-design them with the power generation source. We co-design them with the DeepMind team that builds Gemini models. So it's the software above, the models above that, the ch…”

“My lesson from Waymo and on the AI side with TPUs, et cetera, I think to really push the curve well, particularly in areas where you have safety, regulatory, everything, you want the firsthand experience of the product feedback cycle. So I think having first p…”

“Both chief scientists talked about stacking memory right on top of the GPU or TPU, and that's gonna be a huge innovation in the coming months or years.”

“When we look in 26, Google would buy a lot more TPUs, but they can't ramp production fast enough, right? And so they have to buy tons of GPUs. And we go to 27, it applies again, right? Google simply cannot buy enough TPUs, and they have to buy tons of GPUs.”

“I think Ironwood a V seven is the peak gap between on Between TCO, between NVIDIA and TPU, right?”

“I've done the math. It could be like, it's like a trillion. It's like a trillion. It's like a trillion or something like that. Assuming it gets like 30% market share or something like that.”

“As a hardware designer for ML in particular, you're trying to design a chip starting today And that design might take two years before it even lands in a data center, and then it has to sort of be a reasonable lifetime of the chip to take you three, four, or f…”

“Seven to eight year old TPUs, Google actually disclosed this, seven to eight year old TPUs actually have 100% utilization.”

“Anthropic signed a deal for a million TPUs, right? And is deep on the whole Amazon Tranium ecosystem.”

“Yeah, so, so for the longest time Google's had, ah, one main line of TPUs, right? All made by Broadcom, and then sort of next year they've diverged it, right? Where Broadcom makes a TPU, ah, and MediaTek makes a TPU. These two TPUs are focused at different thi…”

“So being able to understand how the stack works all the way down from TPUs to research is kind of a superpower, because then you're able to kind of find these gaps in between different layers that other people weren't necessarily able to see, but also to reaso…”

“Anthropic has ordered twenty-one billion dollars worth of TPUs to train to train large claws.”

“And then when Rubin comes out, we'll know the gap. The gap is going to expand significant versus TPUs versus TPUs and all other ASICs.”

“I will be surprised if there are a lot of ASICs other than Tranium and TPU.”

“And by the way, in Tranium and TPU, we'll both run on customer owned tooling at some point. We can debate when that will happen, but the economics of success that I just described mean it's inevitable. Like no matter what the companies say, just the economics …”

“One, Meta is considering, and in very serious talks, over a billions of dollars, spending billions of dollars on TPUs in a future deal”

“The idea that NVIDIA is unstoppable because of the software and hardware connection, because we have to have GPUs. I know there's a lot of truth to that, but literally as an end user, I went right back and forth to them today. No, no issue. TPUs, GPUs, LLMs, a…”

“What we are quickly seeing is that there's going to be a highly fragmented layer of decoding chips that exist in the marketplace. Grok is one. TPU is one. Microsoft has a spin. Amazon has Inferentia. Facebook, I think, is Apparently spinning up their own silic…”

“Google trained Gemini three pro on Google's own TPUs. No mention of Nvidia chips.”

“If TPUs are not for sale, Nvidia does have a monopoly. If Nvidia truly is the only seller in the market because Google is not a seller then yes, they still extract monopoly power from every other buyer because every other buyer says, yeah, I'd love to buy TPUs…”

“Our seven and eight year old TPUs have a hundred percent utilization.”

“TPU, I'll use that example again because I know it best for certain computation, is somewhere between 10 and a hundred times more efficient per watt, and it's this watt that really matters than a CPU.”

“If that's true, then the top two coding models Gemini and Anthropic are trained on TPUs, not GPUs. And I think that's a really big deal.”

“They make a mistake thinking that a TPU or a Tranium or from Amazon or something a company develops itself has to be as good as NVIDIA. It doesn't because NVIDIA's margin is so big. It can actually be a fraction of as good as NVIDIA and still cost effective.”

“Unlike meta, which is also profitable and Zuck talks up like, yeah, we can fund this via profits, which is true, but they don't have the cloud infrastructure that Google does and they don't have the TPUs that Google does and they don't have all of the sort of …”

“They're a chip company with their Tensor Processing Units, or TPUs, which is the only real scale deployment of AI chips in the world besides Nvidia GPUs.”

“Somebody put it to me in research that if you don't have a foundational frontier model, Or you don't have an AI chip, you might just be a commodity in the AI market, and Google is the only company that has both.”

“So, the TPU was designed, verified, built, and deployed into data centers in 15 months.”

“But today, Google, it's estimated, has two to three million TPUs. For reference, Nvidia shipped, people don't know for sure, somewhere around four million GPUs last year.”

“I think within a year it might happen. There are rumors already that some NeoClouds in the coming months are gonna have TPUs.”

“They have the lowest cost of goods sold for any token of any company because they have their own vertical stack on TPUs.”

“So, people look at the TPU as a big success, and what they don't realize is that there were about three chip efforts at Google at the same time and only one of them ended up outperforming GPUs.”

“You're supposed to create a startup before the market grows. You're not supposed to come up as a startup when the market's a trillion dollars large. You know, this fallacy, and all VCs know this fallacy that a large market, if you could just take a few percent…”

“I think Google's even discussing it. Internally, I think it would require a big reorg of culture and a big reorg of like how Google Cloud works and how the TPU team works and how the JAX software team and XLA software teams work.”

“Google Clouds revenue numbers disappointed, but this was because they didn't have enough GPUs and they were actually constrained. So they missed on top line, like they didn't bring in enough revenue, but their margins actually improved. And so what that means …”

“Google is the only company, we talked about the TPU thing, they're the only company with an at-scale ASIC alternative to NVIDIA's GPUs, which should give them a meaningful cost advantage to that end, ah, to the extent that cloud compute becomes commoditized.”

“Well, we mostly, for Gemini, we mostly use our own TPUs.”

“They are very much successful inside of Google, but not much outside of Google, let's say, right?”

“I think that if you are doing it bottom up, infra to applications, you will lose because nobody will care. As they don't today, right? If you look at TPUs, they're available, they're great. Nobody cares.”

“But AMD can do training, but it's also, but in terms of maturity, the, by far the most mature software and compute is TPUs, and then it's Nvidia.”

“Google actually did this alongside Broadcom you know, and they did it before Nvidia, right? You know, today everyone's freaking out about, or not freaking out, but like everyone's like very excited about Nvidia's Blackwell system, right? It is a rack Of GPUs. …”

“Like with Blackwell, not only is it way, way, way faster, anywhere from 10 to 15 times on really large models for inference, because they've optimized it for very large language models, they've also decided, hey, we're gonna cut our margin, too, somewhat, beca…”

“Nvidia's talking about GB 200, NVL 72, TPUs go to 8000 today, right?”

“There's only one company accounts for over 70% of Google's revenue from TPUs as far as I understand, and that's Apple.”

“Like in the next six months there is a bit of a slowdown in Google TPU purchases because they have no data center space. They want more. They just literally have no data center space to put them.”

“TPUs have a very different network bandwidth to compute ratio. They have a lot more bandwidth just objectively and TPUs per chip tend to be a little bit less compute intensive and have a little bit less memory.”