gradient descent
2 statements across 2 episodes · 1 bullish · 0 bearish · 2 people on the record · first statement Sep 27, 2024 by Shunyu Yao · across every show →
Everything said about gradient descent, oldest first
Sep 27, 2024 positive
Yao: Reflexion replaces scalar RL rewards with verbal gradient descent
“I think one way to think of reflection is that the traditional idea of reinforcement learning is you have a scalar reward, and then you somehow back propagate the signal of the scalar reward. To the rest of your neural network through whatever algorithm, like …”
Jan 23, 2026
Yi Tay: Gradient descent learning paradigm is AI's bottleneck, not architecture
“It's not architecture itself. That's, that there's a problem that we, that is more of like the learning paradigm itself rather than the architecture itself. I think the architecture is just basically like the interface between the learning algorithm and the to…”