GPT 1

“You know, what started as the unsupervised sentiment neuron paper turned into GPT-ONE and then eventually GPT-whatever.”

“And you can actually, in fact, Take a GPT one or two model and with a few, I mean, few lines of code almost, you can transform it into the latest let's say deep seek version, 3.2 architecture. It's not like a big leap. It's still the same as scaffold.”

“Yeah, like I would say that robotics is in kind of like the GPT-one to GPT-two area right now.”

“GPT-ONE really was Sora-ONE for this modality. And the progress we've made kind of in the last 18 months getting to this 3.5 moment, right, is really compressed compared to how long it took to go from GPT-ONE to 3.5 in the language domain.”

“So in June of 2018, OpenAI releases a paper describing how they have taken the transformer and developed a new approach of pre-training them on very large amounts of general text on the internet, and then fine tuning that general pre-training to specific use c…”

“I find it helpful to frame these in terms of like GPT one, two, three, four, five capabilities, right? And I think most people would agree we're somewhere between GPT one and two, right?”

“I'd say that maybe the neural net we produced, you know, it's a 40 B neural net trained on, you know, like 13 trillion base pairs or something like that. The results to be felt like GPT one, maybe starting to be GPT two level, right? It's like accessible or, a…”

“And the core idea behind GPT-ONE is that if you have a transformer and you apply this super simple objective of guessing the next token, guessing the next word, that that would be enough signal that you could actually have something that would be able to gener…”

“They pave the way. Like, just sort of like a irrational conviction to this vision of scaling, and I think that, that was driven by, I remember talking to Ilya about this stuff way before GPT-One, you know, like in the Transformer times, around that time becaus…”

“There's an example of an AI-native companionship product where they actually did this test where they used GPT-one, GPT-two, GPT-three, and they actually tracked the cohort of user retention using each of the models. And it's very clear as the model quality im…”

“I think where we are in the trajectory of Sora right now is like, this is the GPT-I of these, this new paradigm of visual models, and that we're really looking at the fundamental research into making these way better, making it a way better engine that can pow…”

“GPT-I had roughly a hundred and twenty million parameters that it was trained on. GPT-II had 1.5 billion. GPT-III had a hundred and seventy five billion, and GPT-IV, OpenAI hasn't announced, but it's rumored that it has about 1.7 trillion parameters that it wa…”