recurrent neural networks
2 statements across 2 episodes · 1 bullish · 0 bearish · 2 people on the record · first statement Apr 25, 2023 by Noam Shazeer · across every show →
Everything said about recurrent neural networks, oldest first
Apr 25, 2023 positive
Transformers Beat Recurrent Models by Processing Whole Sequences at Once
“The magic of transformer kind of like convolutions is that you get to process the whole sequence at once. I mean, it still talks, you know, it's still a function of like the, you know, the predictions for the later words are dependent on what the earlier words…”
Aug 1, 2024 neutral
Vinyals: RNNs and LSTMs Never Remembered Beyond a Few Hundred Words
“We come from a world where we had recurrent neural networks and LSTMs that actually had infinite memory, although it was not very capable, right? You, the models in, in practice, they never remember more than a few hundred words or so.”