MOPs
topic on 1 show · 1 statements across 1 episodes
1 statements about MOPs, every show
Training frontends matter little if attention and MLP kernels are highly optimized
“Most of that is an attention and MOPs, right? So if you have good kernels for attention, MOPs and norms and so on, then it doesn't much matter what the front end to, you know, send tensors to and from those kernels is”