Loss Function
topic on 3 shows · 3 statements across 3 episodes
the Y Combinator Startup Podcast 
Latent Space 
No Priors
3 statements about Loss Function, every show
Enforcing physics as hard constraints in AI models is computationally intractable
“Making it a hard constraint is not tractable, whereas adding it as a loss function. And of course, there's still the balancing of that loss with the data we have.”
Karpathy: Model architecture is no longer the fundamental bottleneck in AI
“I don't think that the neural network architecture is like holding us back fundamentally anymore. It's like not the bottleneck, whereas I think in the previous, before Transformer, it was a bottleneck, but now it's not the bottleneck. So now we're talking a lo…”
Schönlieb: Exactly minimizing training loss often hurts neural network generalization
“You do not necessarily need to solve your optimization problem, your training exactly. And maybe sometimes, or most of the time you actually don't want it, want to save it exactly because you only have a finite amount of training examples. And so when you thin…”