GPU

“In the second game, there was this famous move called move 37, which was creative. It was original. It actually wasn't completely original. It was a one in 10,000 game move. It had been in the canon of games that we're trained on. But when we went back and pla…”

“And so GPUs and LPUs combined ended up giving better performance across the performance curves.”

“So over time, these GPUs have, you know, gotten, you know, as good and better than TPUs, you know, as the career of the TPU, I have to admit, GPUs are now better.”

“The LPU and GPU, as mentioned, they, they're better at different parts of what's, what's called the decoder layer of an LLM. The GPU is better at the attention portion, and the LPU is better at sort of applying the weights, which is the thing that gets trained…”

“When LLMs first started to become a thing, I remember getting a phone call from the CEO of GitHub basically saying, I need a bunch of GPUs. We've now gotten LLMs to be able to do code completion, even though, you know, we're part of Microsoft and everything yo…”