post-training

“For post-training, what's really f- or one of the reasons I really like post-training is it feels more like an art than maybe even, like, other areas of research, because you kind of have to make all these trade-offs, right?”

“And it just seems like we're getting more benefit from the post training and the reasoning paradigm than scaling. But I don't think either one is I definitely don't think either one is, is dead.”

“It seems as if pre-training is comparatively less of a focus than it was before, partly because, like, you have this exciting new direction of, well, new, newish direction of post-training where they've done so much about reasoning”

“A lot of this stuff is quite synergistic. You develop a better model. You, like, use post-training stuff to make it better. You get a load of data of the model actually being used successfully or not. A lot of that can probably go into pre-training next time.”

“For the text models, there's always going to be this like really big fat free training step that like you have to invest in here. And then even the post training side is like, You know, it's not the, it's not like the easiest thing. Like it's, you know we all,…”