Switch Transformer, every mention
2 scenes · ← back to Switch Transformer
tap a year for its mentions
Verbatim, from the transcripts: the passages where Switch Transformer comes up
[Paper Club] Upcycling Large Language Models into Mixture of Experts
- ▶ 1:04 Ethan He Switch transformer is the first model that surpassed the one trillion model size. 2 times in the scene
- ▶ 24:34 Ethan He But the problem here is, uh, we actually found the mix-throughs approach didn't work as well as expected, because the, the original model, the original switch transformer from Google uses a softmax and topk for a reason. 3 times in the scene