coding benchmarks
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Nov 8, 2025 by Alex Shaw · across every show →
Everything said about coding benchmarks, oldest first
Nov 8, 2025 negative
Shaw: Current AI coding benchmarks rely on redundant bespoke test harnesses
“In fact, every single benchmark that gets released at least I would say maybe all coding benchmarks that get released at this point are some form of instruction container tests with some bespoke harness that was coded up That feels very analogous to every othe…”