Canary Strings
topic on 1 show · 1 statements across 1 episodes
1 statements about Canary Strings, every show
Glaese: Open-source benchmarks cannot use canary strings to avoid contamination
“There's like multiple avenues, but like the problems are sourced from open source repos. So it's not just like when we usually publish evaluations, we publish evaluations, and then we add canary strings to ensure that, you know, they are easily filtered out at…”