in-context learning
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jan 18, 2024 by Beyang Liu · across every show →
Everything said about in-context learning, oldest first
Jan 18, 2024 neutral
Liu: Fine-tuning medium models can harm their in-context learning ability
“And if you fine-tune a medium-sized-ish model, sometimes it loses the ability to do effective in context learning, because I think the intuition is, it's devoting more, more of its parameter space to, kind of, like, memorizing the training set so it can do bet…”