Model Performance
topic on 3 shows · 4 statements across 4 episodes
Latent Space 
Lenny's Podcast 
No Priors
4 statements about Model Performance, every show
Chip Huyen: Sampling strategy is very underrated for boosting model performance
“Sampling strategy, I think is something extremely important. It can have you boost the performance in a huge way and very, very underrated.”
Morcos: Power-law scaling yields diminishing returns for every 10x data increase
“Power law scaling is terrible. It means that every time you 10 X your data, you get a diminishing marginal return on performance.”
Schulhoff: Ensembling prompts and models via majority voting improves performance
“So if you have a bunch of different models you're asking and then you take the final result or the most common result as your final result, you can often get better performance overall.”
Steinberger: AI performance requires trading off training compute against inference compute
“Well, so you can think of model performance as some function of training compute times some function of inference time compute. Now those are specific functions that are just scaling law things that you can like model, but the general Way to think about it is …”