transformer architecture
also referred to as: transformer architectures
2 statements across 2 episodes · 1 bullish · 0 bearish · 2 people on the record · first statement May 15, 2023 by Yann LeCun · across every show →
Everything said about transformer architecture, oldest first
May 15, 2023 positive
LeCun: Transformer scaling with self-supervised learning surprised AI researchers
“The fact that self-supervised learning methods applied to transformer architectures Work amazingly well, and they work, you know, way beyond what we could have expected. The fact that we can do basically train systems to understand language, translate language…”
Sep 1, 2025
Frosst: Core transformer architecture has remained largely unchanged for a decade
“The models themselves, like transformer architecture, which is the original model that, yeah, that was introduced in Hasn't changed very much, right? Like all, the whole industry is still using transformers. We've changed the way we train them, but the model a…”