Rejection Sampling

“Chai has never done streaming. Because if you stream, you're unable to do rejection sampling.”

“Llama started their RLHF process with this to get some signal out of preference data. That preference data went into a reward model, and then the reward model did a good enough ranking that it was, like, essentially superpowered instruction tuning based on rew…”

“Full blown, like RLHF is, you know, definitely something we're going to look into that, but there is like several many steps that you can have in between that significantly can increase your quality. So for example, I mean like even using something like a reje…”