ASICs

“So the environments can get more and more complex, and those environments run on CPUs. They don't run on GPUs. They don't run on ASICs. The ASICs run the model,”

“I think Asics is talking much more true product itself than many other brands that we have in our space.”

“Over time, we can go even further down the stack where we should go make our own ASICs, our own chips.”

“And then when Rubin comes out, we'll know the gap. The gap is going to expand significant versus TPUs versus TPUs and all other ASICs.”

“I will be surprised if there are a lot of ASICs other than Tranium and TPU.”

“So even if they gave it to you for free, you only have two gigawatts to work with, your opportunity cost is so insanely high, you would always choose the best perf per watt.”

“What is the overhead of an NVIDIA GAM instruction, right? It's like 16%. So like you're essentially buying a, Matrix multiplication machine. So, like, it doesn't really make sense to specialize it that much.”

“And so that just sets up a huge incentive for all these hyperscalers to build their own ASICs, their own accelerators that replace the Nvidia ones. Which I think will come online over the next few years from all of them.”

“It only works if you really know which chip you're going to do. If you don't, then it's a little harder. So it makes, in my head, it makes more sense for inference where you've already established it, but for training there's so much, like, experimentation.”

“Well, traditionally, they're very loyal, you know, and it's hard to get a consumer that was running in ASICs for 10 years or Saucony for 10 years to switch brands. I think that's changing dramatically. I think people are open to trying different brands now. An…”

“After ultra marathoners came triathletes which Hoka became the most worn shoe ahead of Asics, Nike, and Saucony beginning in 2017.”

“Chips exist to run software, and if you control your software stack, building a chip that's tailor-made, a special purpose chip, we'll call them ASIC, tailor-made for that software stack, you can get performance gains over general purpose compute, like a sort …”

“Right now I'm wearing black Asics, and then I have a pair of flip-flops that I wear, and those are the only two pairs.”

“So I think that's sort of where, Where proof of replication falls. Somewhere in between something which has high, you know, barriers of entry, such as something which requires ASICs you know, versus something that is just so convenient for Amazon to do and par…”