zeroth order optimization
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Oct 1, 2024 by Jeff Schmidt · across every show →
Everything said about zeroth order optimization, oldest first
Oct 1, 2024 negative
Schmidt: Zeroth-Order Optimization Requires 1,000x More Computation Than Backpropagation
“What we discovered is that backprop is still being like, you really do still need to be doing back propagation to find the optimal point of the loss. And it's just like zeroth order is like, what, like a thousand, like it worked, but it was like, you needed li…”