Tensor Processing Unit, every mention
26 scenes · ← back to Tensor Processing Unit
tap a year for its mentions
every year anyone Ben Gilbert 23David Rosenthal 3
Verbatim, from the transcripts: the passages where Tensor Processing Unit comes up
Google Part III: The AI Company. Google is amazingly well-positioned... will they win in AI? (Audio) · Acquired
- ▶ 3:27 Ben Gilbert They're a chip company with their Tensor Processing Units, or TPUs, which is the only real scale deployment of AI chips in the world besides Nvidia GPUs.
- ▶ 1:45:29 David Rosenthal Matrix multiplication, tensor multiplication, a tensor processing unit, you might say.
- ▶ 1:46:00 Ben Gilbert So enter, David, as you said, the tensor processing unit made just for neural networks that is far more efficient from GPUs at the time, with the trade-off that you can't really use it for anything else.
- ▶ 1:46:28 Ben Gilbert So the big idea behind the TPU, if you're trying to figure out, like, what was the core insight, they use reduced computational precision.
- ▶ 1:47:18 Ben Gilbert The whole thing is quite clever behind the TPU.
- ▶ 1:47:21 Ben Gilbert The other thing that has to happen with the TPU is it needs to happen now, because it's very clear speech to text is a thing.
- ▶ 1:47:49 Ben Gilbert So, the TPU was designed, verified, built, and deployed into data centers in 15 months.
- ▶ 1:48:19 Ben Gilbert The other thing they did is they fit the TPU into the form factor of a hard drive.
- ▶ 1:48:28 Ben Gilbert You just pop out a hard drive and you pop in a TPU without needing to do any physical re-architecture.
- ▶ 1:49:21 Ben Gilbert The other thing to know about the TPUs is they were done in time for the AlphaGo match.
- ▶ 1:49:25 Ben Gilbert So that match ran on a single machine with four TPUs in Google Cloud.
- ▶ 1:49:36 Ben Gilbert That's the TPU.
- ▶ 1:49:43 Ben Gilbert TPUs and GPUs look a lot more similar than they used to.
- ▶ 1:49:49 Ben Gilbert But today, Google, it's estimated, has two to three million TPUs.
- ▶ 1:50:14 Ben Gilbert The TPU is a giant deal in AI in a way that I think a lot of people don't realize.
- ▶ 1:50:44 Ben Gilbert And they built it in such a way that it doesn't just have to run on TPUs.
- ▶ 1:55:18 David Rosenthal All the efforts that are coming out of AlexNet and then the TPU project of parallelization.
- ▶ 3:28:30 Ben Gilbert But in Google Cloud, you can use TPUs, which they make a ton of, and everyone else is desperately begging NVIDIA for allocations to GPUs.
- ▶ 3:28:41 Ben Gilbert So if you're willing to not use CUDA and build on Google stack, they have an abundant amount of TPUs for you.
- ▶ 3:37:30 Ben Gilbert When working with Google on the TPU versus Nvidia's 80%, but that's still a huge difference to play with.
- ▶ 4:04:16 Ben Gilbert To Jonathan Ross, original team member on the TPU, and today the founder and CEO of Grok, that's Grok with a Q, making chips for inference.
Nvidia Part III: The Dawn of the AI Era (2022-2023) (Audio) · Acquired
- ▶ 43:16 David Rosenthal To train these models is prohibitively expensive, and, you know, even Google at this point in time, this is when they start building their own chips, TPUs, because, you know, they're still buying tons of hardware from NVIDIA, but they're…
- ▶ 1:58:55 Ben Gilbert Google and the TPU.
- ▶ 2:09:47 Ben Gilbert I know we've been harping on NVIDIA, but Google TPUs are also manufactured at volume.
- ▶ 2:10:03 Ben Gilbert But it's certainly not an industry standard to use TPUs the way that it is to use NVIDIA's hardware.
- ▶ 2:10:09 Ben Gilbert I suspect a lot of the volume of the TPUs is being used internally by Google for BARD, for doing stuff in Google search.