Public Benchmarks
topic on 3 shows · 3 statements across 3 episodes
Latent Space 
Capital Allocators 
20VC
3 statements about Public Benchmarks, every show
Fitzpatrick: AI performance grew 40% to 60% over two years
“If you look at all the public benchmarks, models have increased 40 to 60% in performance over the last two years.”
Goyal: Publishing Public Benchmarks Is Marketing, Not Product Improvement
“It's just that the value proposition of publishing an eval is completely orthogonal to the value proposition of building evals in service of building a good product. I think the purpose of publishing benchmarks is marketing, and it's good marketing.”
Mogelof: Private Markets Generate Hundreds of Basis Points Over Publics
“I'd also tell you that the value add is much more significant, whether it's in credit or infrastructure, real estate, there's anywhere from a couple hundred to hundreds of basis points of excess returns that you can generate relative to public benchmarks.”