Human Evaluators
topic on 2 shows · 2 statements across 2 episodes
2 statements about Human Evaluators, every show
Angelopoulos: Human evaluators prefer longer AI responses given equal content
“It's true that people vote for longer responses, you know, preferentially over shorter responses, even given the same contents or well-known human bias.”
Foody: AI models will identify and ignore mistakes in human evaluator data
“Well, I think the models will be able to delineate between the valuable human knowledge and the human knowledge that's not valuable. And that maybe you have doctors that create like a bunch of evals for this particular task and the model realizes like, wow, li…”