GPT-2

“So I think that we continue to feel like graphics and these foundation models for anything really related to pixels, but also definitely images continues to be very under invested. It feels a little like graphics is in like this GPT two moment, right? Like eve…”

“We do also use, like, T-Five-based models, even, even now but started, yeah, started with GPT-II.”

“At the time of that post, we were using, in LL and that's in 30% less memory, and we were 20% faster in training, just the truth.”

“You can train it on a single node of H-one-hundreds in about 24 hours, and that costs roughly 600 dollars.”

“If you try to do the reasoning paradigm on top of GPT-II, I don't think it would have gotten you almost anything.”

“I'd say that maybe the neural net we produced, you know, it's a 40 B neural net trained on, you know, like 13 trillion base pairs or something like that. The results to be felt like GPT one, maybe starting to be GPT two level, right? It's like accessible or, a…”

“Yeah, like I would say that robotics is in kind of like the GPT-one to GPT-two area right now.”