RoPE
topic on 1 show · 1 statements across 1 episodes · said 7 times in 1 episodes since 2024
Mentions by year, every show
tap a year for its mentions
Latent Space 7
2024 7 mentions in 1 episode
every mention on every show, scene by scene, with the transcript →
1 statements about RoPE, every show
Huang: Modern LLMs have shifted from ALiBi to RoPE scaling
“Some of the newer architectures don't actually employ it a lot. I think the last architecture that actually really employed it was the Mosaic MPT model class, and then almost all the models these days are all rope scaling, and then effectively you can use yarn…”