Mixture of Experts
2 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement May 4, 2023 by Kelvin Guu · across every show →
Everything said about Mixture of Experts, oldest first
May 4, 2023 positive
Guu: Model Surgery Research Enables Unique Modularity Over Retrieval and MoE
“I feel that This is a very exciting area for research because it provides a different kind of modularity from retrieval augmented models or mixture of experts, one that actually allows a kind of generalization that's very interesting.”
May 4, 2023
Guu: Adapting LLMs to Code Requires Specialized Training Over Pure Retrieval
“So if you just want to get very precise factoid information, like what is my Wi-Fi password, retrieval augmentation is going to be very good. But if, for example, you're trying to adapt a language model to a new enterprise, and they have some kind of a special…”