TPU

“I think Google's even discussing it. Internally, I think it would require a big reorg of culture and a big reorg of like how Google Cloud works and how the TPU team works and how the JAX software team and XLA software teams work.”

“TPU, I'll use that example again because I know it best for certain computation, is somewhere between 10 and a hundred times more efficient per watt, and it's this watt that really matters than a CPU.”

“Our seven and eight year old TPUs have a hundred percent utilization.”

“Seven to eight year old TPUs, Google actually disclosed this, seven to eight year old TPUs actually have 100% utilization.”

“Serving large language model is a fundamentally different problem. Because serving it requires to run it on accelerators like GPUs or TPUs, and it is a computationally intensive process that will require a lot of engineering and ensuring that for each request,…”

“TPUs are the second most financeable. It probably takes, I don't know, double the equity check at least. And then the rates on the rest of it are higher.”