Flops
topic on 4 shows · 5 statements across 5 episodes
Latent Space 
No Priors 
the a16z Podcast 
20VC
5 statements about Flops, every show
Sohmers: AI hardware over-indexes on raw FLOPS instead of memory bandwidth
“Everyone else was focusing on the wrong things. They were just trying to have more and more flops when memory bandwidth, memory capacity were the real, real bottlenecks.”
Lazarte: LLM usage measured in FLOPS grew 100x over two years
“LLM usage has grown a hundred x in the last two years. So if you measure like in the number of flops, it went from like one extra flop to a hundred extra flops now. And like, of course, this data is not very precise, but it's directionally correct.”
Mensch: FLOP count is the wrong metric for regulating AI models
“Pre-market conditions like flops, the number of flops that you do to create a model is definitely not the right way of doing of measuring the performance of a model.”
FLOP counts do not necessarily correlate with wall-clock runtime
“Flops or floating point operations don't necessarily correlate with runtime. There are other factors like memory reading and writing, parallelism, and so on.”