needle in the haystack tests
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Mar 7, 2025 by Misha Laskin · across every show →
Everything said about needle in the haystack tests, oldest first
Mar 7, 2025 negative
Needle-in-a-Haystack Tests Are Crude for Evaluating True Context Understanding
“So it's not just having long context, it's whether your model truly understands what's inside the context, and needle in the haystack tests are pretty crude and not very effective way of testing this kind of capability.”