backpropagation
3 statements across 3 episodes · 0 bullish · 2 bearish · 3 people on the record · first statement Dec 8, 2017 by Frank Chen · across every show →
Everything said about backpropagation, oldest first
Dec 8, 2017 bearish
Geoffrey Hinton: Backpropagation is a dead end for reaching general intelligence
“He says, I think we need a reboot. I think the path that I set everybody on, and fundamentally an algorithm called backpropagation, I think that's a dead end. I think if we want general intelligence, we're going to need to invent a new technique. We're going t…”
Jan 2, 2019
Shanahan: Three technical factors drive the machine learning revolution
“What's driving the whole machine learning revolution, if we can call it that is I mean, there are three things, and one is Moore's Law, so the availability of a huge amount of computation, and in particular the development of GPUs, or the application of GPUs t…”
Oct 1, 2024 negative
Schmidt: Zeroth-Order Optimization Requires 1,000x More Computation Than Backpropagation
“What we discovered is that backprop is still being like, you really do still need to be doing back propagation to find the optimal point of the loss. And it's just like zeroth order is like, what, like a thousand, like it worked, but it was like, you needed li…”