Backpropagation
topic on 2 shows · 4 statements across 4 episodes
WTF is with Nikhil Kamath 
the a16z Podcast
4 statements about Backpropagation, every show
Deep learning and 1980s backpropagation remain the foundation of modern AI
“Deep learning, which is really the foundation of pretty much all of AI today. So basically, neural networks with multiple layers, right? The idea of this goes back to the 19 eighties and backpropagation. That's still the basic foundation of everything we do.”
Schmidt: Zeroth-Order Optimization Requires 1,000x More Computation Than Backpropagation
“What we discovered is that backprop is still being like, you really do still need to be doing back propagation to find the optimal point of the loss. And it's just like zeroth order is like, what, like a thousand, like it worked, but it was like, you needed li…”
Shanahan: Three technical factors drive the machine learning revolution
“What's driving the whole machine learning revolution, if we can call it that is I mean, there are three things, and one is Moore's Law, so the availability of a huge amount of computation, and in particular the development of GPUs, or the application of GPUs t…”
Geoffrey Hinton: Backpropagation is a dead end for reaching general intelligence
“He says, I think we need a reboot. I think the path that I set everybody on, and fundamentally an algorithm called backpropagation, I think that's a dead end. I think if we want general intelligence, we're going to need to invent a new technique. We're going t…”