GPT

“And so I actually feel like we kind of went backwards for years and not to be honest, I mean, I'm a bit sad about this now, but I kind of got so disappointed and dissuaded by like, It felt like these bigger lab, much bigger labs, you know, like fast.ai had onl…”

“I organized a chat for both of us with Kate Metz in the New York Times, and Kate Metz answered, sorry, and Alec answered this question for Kate, and Kate just like, so how did, you know, GPT come about? And he said, well, I was pretty sure that pre-training on…”

“Maybe open source models are being as used as GPT is at this point in, like, all kinds of, in a very fragmented way. Like, in aggregate, all the open source models together are probably being used as much as GPT is. Maybe, you know, close to that.”

“There's a whole model that's been trained in a different way. So there's probably a whole lot of tasks it's probably better at than you know, GPT and Gemini and Claude.”

“So, like, GPT doesn't have very good air recovery ability. And so, because of this, it will go into loops and do the same thing over and over and over again, whereas Claude does not do this.”

“Anything where it just seems simple and like, you can do one off and you don't need to bring a ton of context into it. You can typically use like sonnet or GPT for that. Anything where I feel like if I was going to try to implement it myself and I would need t…”

“Maybe even if we tested this with LALAMA architecture, like, there's sort of like a GPT++ architecture, like, I would guess that can store better data just because the kind of numerical flow is a little bit better, the nonlinearities are maybe, like, A little …”

“Historically, what we have seen from our previous initiatives is that, okay, maybe the best GPT or best cloud is the top model, but there might be very small gap with the model just below it. And then it becomes a cost performance trade off so that users can k…”

“I think whether GPT, 5.2 codex max or Opus 4.5 is going to do better. It's probably doesn't matter. It's just a matter of like, which one's going to have all the information about what's in the lab and how much will it cost? How long will it take?”

“What this suggests about the GPT training data is that the self-reflection data has now actually become pretty much core to the training of all frontier models, because we're seeing that happen in non-instruct models across the board.”

“And so then I tried this across models, and I saw consistently across Grok, and GPT, and Gemini, that you were seeing this phenomena where models will, like, self-correct themselves quite a bit, and these were, like, non-thinking models. They were, like, the s…”

“In fact, like, you know, GPT found a proof to an unsolved Erdos problem, and our competitor Harmonic, you know, Aristotle you know, verified it.”

“This is a thing that has been tried internally within Google, and it led to Google missing GPT.”

“We here in this demo that we gave at hot chips we're showing GPT OSS running at over 4000 400 TPS, which is just mind blowing.”