The Ledger, every show

“Like GPT-Five in some way I can be considered as like, oh, 3.1. It's a little bit of like, you know, iteration of like the same thing and the same concept”

“When we trained GPT-IV, we were pretty underwhelmed internally, and then there was a lot of moments, oh, we trained this small, we spent a lot of money on it, and it's kind of like, you know, pretty dumb, at least, like, you know, we have GPT-IV, GPT-III alrea…”

“I think, like, in a way, I think it's getting more and more to be a thing of the past as the models are getting smarter and smarter. This is becoming less of a thing, but I think a few years back, and especially in GPT-IV days, this was the thing.”

“Language models do on their own, like fundamental level is they are often called as next token prediction machines. And that's not completely accurate in the age of reinforcement learning, but they still operate on mostly on tokens that are mostly text.”

“O-one like, to be perfectly honest, it was really mostly good at solving puzzles and like maybe a few kind of thinking problems here and there, but it wasn't like, it wasn't a very useful model. It was almost more like a technology demonstration.”

“Like, coding agents are at the moment the first, like, pretty successful agentic products built on top of AI.”

“From my perspective, and again, this is just how my brain works, that the 20 12 ImageNet results, like, weren't that significant.”

“It is like, yeah, it is some like risk of losing IP, but I think the risk of not doing the right thing and of people not being informed about research and not being able to do the best research is much higher in my personal opinion and how, how I approach thos…”

“As far as I know, like our O-one release mostly caught a lot of us labs by surprise. They didn't have like similarly advanced RL research program to my knowledge, basically no one.”

“The first thing that is important to know and understand, RL is hard. Like, conceptually, if you think about it, and there's still a lot of depth to it, but very conceptually, mathematically speaking, pre-training is dead simple.”

“I generally think something that we are doing, like, pre-training today is necessary. I think something that, like, we are doing RL today is necessary, and there will surely be a few things more, and like, we have a lot of, Very ambitious research programs on …”

“And like, I don't like in terms of a pure RL, I don't think like really pure RL makes sense. RL needs Pre-training to be successful. And I think pre-training, as I said before, needs RL to be successful as well.”

“Where you can have like a high rezoning model and the low rezoning models. And this is like in the end, the same model. You just, we just tweak the parameter, which says we want you to think longer or shorter.”

“We all work on a very few projects total. There are not that many projects. OpenAI is not trying to do everything. We are not trying to like have portfolio. We are trying to have like multiple different bets. Always the idea is we do a few core things really, …”

“Top down structuring of research doesn't work in research organizations. I really don't believe in it because like you are not kind of hiring some of the smartest people in the world and open air has incredibly, incredibly smart people. To kind of tell them wh…”

“I think I am pretty heavy user of ChatGPT right now, happily paying like 200 dollars a month for it”

“And what he said at the beginning of 2019 was to train large generative model on all data we can and then do reinforcement learning on it. That was the OpenAI research Plan at the beginning of 2019. And this is exactly what we are doing today.”

“Like what we, what OpenAI is doing is not exactly GRPO. It is slightly different in many different ways, but like some parts are definitely similar.”

“The models I can think for, like, 30 minutes, hour, two hours these days on certain, certain types of Tasks and problems like even, even, even longer than that.”

“This is not what I am aware, at least, like, not, not, not what OpenAI is doing at the moment”

“So I, at least until, until we have a really good safeguards, I don't think we should try to do that in anything like as complex and large scale as ChatGPT.”

“I don't think like you can just tell them all like, A few show with a few good things to do, and it will do them all needs to deeply understand its action and consequences to really be able to choose the right thing.”

“And it's I think it's a never ending pursuit because like, even, even for humans, it's not super easy to define what's, what do we consider a light? And I think as our civilization will evolve, it will, the notion of alignment and the goals of humanity will, K…”

“I think that, that, that difference is here, like answering a question usually means you already know the answer and you just elicit the answer, you know, and the process of reasoning is Getting to the answer that you don't know, and usually the longer you spe…”

“What chain of thought is, is their thinking process verbalized using human words and human concepts.”

“The project I was working on was focused on dexterous manipulation, which was back then and still continues to be an elusive challenge for trained policies.”

“The truth is in research at OpenAI, which is like around, like slightly less than 600 people at the moment, everyone knows everything really, really it does.”

“In a good way, the good way to do RL is if you balance those things. So if you kind of give cookies half of the time and punish the other half of the time, but this is almost like a mathematical kind of kind of aspect of it.”

“And that's kind of, like, the only way how to, like, really teach agents to, like, learn to react to changes in the environment is through reinforcement learning.”

“Like whenever, even when I started like 20, 2019, the reinforcement learning was kind of fashionable at that moment, although not like very successful, but we were, the reinforcement learning was able to solve a lot of games, but the bottleneck was there that …”

“There are many more bits usually in the targets than in the labels and studying the structure of targets itself. It yields much more learning and much more intelligence than learning the mapping itself. So like spending a whole compute on just learning the dat…”

“In some way you can say it's a limitation of reinforcement learning, but when I was thinking about it, I realized a lot of that happens in human systems as well. There are a lot of like incentive system and reward systems and even, even happens in workplaces a…”

“If you talk to someone from 10 years ago and show them chat GPD from today, they would probably call it AGI”

“We definitely use codecs a lot for coding, and this is only getting, getting better.”

“I think we used like specifically programming puzzles for a while as a, Very nice research test bed of our ideas. Those are nice problems to experiment on them, and they weren't ever, like, considered part of the product, but it's, those are pretty, like, comp…”

“We did also like IOI, International Olympic Informatics earlier this year, and Adcoder Heuristics competition as well, where we went second, we, behind a single human that is also a Polish person that used to be employed by OpenAI some time ago.”