Zero-shot
Just ask, with no examples.
Prompting with only an instruction and no worked examples, relying on the model’s general training. Works well for common tasks; degrades on unusual formats or domain-specific conventions.
Try zero-shot first. If it fails, that is information: either the task is unusual to the model, or your instruction is ambiguous. Add examples to disambiguate, not as a first resort.
Zero-shot works well enough on common tasks that it should always be the first attempt, and its failures are informative in a specific way. If a model cannot do a simple task zero-shot but succeeds with two examples, the instruction was probably ambiguous rather than the model incapable — which is a prompt bug you can fix in a minute.
The baseline that saves the example set
Classifying support tickets into eight categories.
Modern models are good at this with just category names and definitions, and maintaining an example set has a real ongoing cost as the taxonomy evolves. Establish the zero-shot number before committing to curate examples.
- paperFinetuned language models are zero-shot learners ↗
Instruction tuning: training on many tasks phrased as instructions improved results on tasks the model had not been trained on.
- paperLanguage models are few-shot learners ↗
Introduces GPT-3 at 175B parameters and shows that scale alone produced in-context learning: performance from examples in the prompt, with no gradient updates.
Few-shot · In-context examples
Show a handful of examples and let the model pattern-match.
Prompt Engineering
The craft of writing instructions a model reliably follows.
Instruction Following
The model’s ability to actually obey what you asked.