post-training

“And unfortunately, a lot of labs that are doing it are not quite, like, publishing papers about it.”

“So, so I do think that post-training is what makes this, like, really big lab models are, like, different.”

“If like, if we were to train a model that was 10 times as big, like all this post-training stuff would change. But the pre-training And mid training and long contacts, I think would actually become looking pretty similar.”

“First, I think a lot of progress has been made on this front since in the last few years. I think this is mostly around post-training, around search, use search tools and then make search calls, then they would have access to that new information.”

“We're going to be having a bit of a swing back and forth between pre-training and post-training.”

“Pre-training is still the foundation and like, you can never post-train your way out of a week-based model.”

“RL is now the foundation of really all post training. It's all done by RL.”

“I think the framework I like is thinking about it as training data. Except you're just training the model at runtime. It is training data. And because you, the model's learning at inference time, the total amount of training data is far lower, right? Like the …”