Few Shot Prompting
topic on 2 shows · 6 statements across 4 episodes
6 statements about Few Shot Prompting, every show
Chase: Few-shot prompting works better than detailed instructions for agent trajectories
“I'm pretty bullish on it, to be honest, for a few reasons. Like, one, I think it can maybe help for more complex things, but then also, two, like, it's a form of prompting, and prompting is just Communicating with the model what you want it to do. And sometime…”
Schulhoff: Few-Shot Exemplar Order Can Shift Model Accuracy From 0% to 90%
“How you order your exemplars in the prompt is super important. And we've seen this move accuracy from like zero percent to 90%, like Zero to state of the art on some tasks, which is just ridiculous”
Schulhoff: LLMs Rely More on Prompt Structure Than Exemplar Labels
“There are a number of papers which have found that the label of the exemplar doesn't really matter, and the model reads the exemplars and cares more about structure than label.”
Swix: Few-Shot Prompting Is Not Always Superior to Zero-Shot Templates
“Few shot is not necessarily better than zero shot is, which is counterintuitive because you're working harder.”
Carlini never uses few-shot prompting for personal language model queries
“I don't because usually when I want the answer, I just, I want to get the answer.”
Chase: Few-shot prompting effectively builds procedural memory for AI agents
“So on the procedural side, I think the main thing that we see people doing and that we think is pretty effective is few-shot prompting and maybe fine-tuning for how to use for how to use tools, because that's basically what it comes down to. What's the right w…”