Looping
topic on 1 show · 1 statements across 1 episodes
1 statements about Looping, every show
Looping provides parameter-free FLOPS, whereas MoE architectures provide FLOPS-free parameters
“In mixture of experts, you have flops free parameters. So, so parameters that they're not actually bringing any flops. And in, in like looping, you have parameter free flops where you don't have extra parameters for the extra flops that you're throwing on this…”