Few-shot examples: how to select and order them
Zero-shot prompting gives the model only instructions and relies on what it learned in training. Few-shot prompting adds a few examples of input and correct output, so the model sees exactly what you mean. Examples are the fastest fix for format, tone, labelling and borderline cases, but they also bias the model toward copying them, so use 2 to 5 varied, realistic examples, include a tricky case, balance the labels, and for large or varied tasks, select the examples for each request based on similarity.
Zero-shot
In zero-shot prompting, you give no examples. "Classify this text as positive or negative: your t-shirt is killer." The model answers from its training, and in this case it says positive, because somewhere in what it learned, "killer" in that kind of sentence is a compliment.
Zero-shot works well when the task is common and the definition is clear: sentiment, translation, summarisation, general questions. Start with zero-shot, because it is the simplest and the cheapest, and add examples only when the results show you need them.
Few-shot
In few-shot prompting, you include a few worked examples before the real input.
Classify each email as "urgent" or "not urgent".
Email: My payment failed twice and I need this resolved now, my account will be suspended tomorrow.
Label: urgent
Email: Can you send me your product brochure?
Label: not urgent
Email: System error, I lost all my saved data after the update.
Label: urgent
Email: Please update my subscription to the annual plan.
Label:
The examples show the model your labels, your definition of urgent, and the exact output format, which is often more effective than a paragraph explaining all three.
When examples help most
- Format: the exact structure, length or style you want, especially when it's unusual.
- Your own categories: labels that mean something specific in your business, like what counts as "urgent" for your support team.
- Tone and voice: how replies should sound.
- Borderline cases: showing how to handle the cases people argue about.
- Specialised tasks: domain-specific extraction or classification the model hasn't seen much of.
When examples hurt
Examples are powerful, and that is also their risk.
- Copying: the model may copy the examples' wording, length or structure too closely, so every output starts to look like example 2.
- Label bias: if 4 of your 5 examples are "urgent", the model leans toward "urgent".
- Order effects: the last example often has the most influence.
- Narrowing: examples that are all the same type make the model worse at other types.
- Cost: every example is sent on every call, so 10 long examples add up.
- Overfitting your tests: if your examples come from your test set, your tests will look better than real performance.
How many examples
There is no fixed number, but these are good starting points:
- 2 to 3 for format and tone.
- 3 to 5 for classification, with at least one per label.
- More only if your tests show they help, and then consider dynamic selection or fine-tuning instead.
Test with zero, two and five examples on the same evaluation set, and keep the smallest number that gives the best results.
How to choose them
- Use real inputs, not invented ones, with sensitive details removed. Real inputs have the typos, length and mess of what your users send.
- Cover the variety: different lengths, topics, languages and styles of input.
- Include at least one tricky case, and one where the right answer is "can't tell" or "not applicable" if that's an option.
- Balance the labels roughly to match what you expect, and don't let one label dominate.
- Keep them correct. A wrong example teaches the wrong thing very efficiently.
- Keep them separate from your test set.
How to order and format them
- Use exactly the same format in every example as you want in the output.
- Put examples in clear delimiters, like
<example>tags, so the model doesn't confuse them with the real input. - Mix the labels, rather than grouping all examples of one label together.
- If one example matters most, for example the closest to typical inputs, put it last.
Dynamic example selection
When your inputs vary a lot, a fixed set of examples can't cover them. Instead, keep a larger library of good examples, say 100 to 500, and for each request select the few most similar to the current input.
The usual way is with embeddings: embed every example once, embed the incoming request, and pick the closest examples by cosine similarity, which is the same machinery as retrieval in RAG. Add a rule to keep the selected examples diverse, so you don't get five near-identical ones.
This gives you relevant examples for each request without sending hundreds of examples on every call.
Examples or fine-tuning?
If you need dozens of examples in every prompt to get the behaviour you want, you are paying for those tokens on every call, and the behaviour still isn't fully reliable. That is the point to consider fine-tuning, which teaches the behaviour once instead of repeating it in every prompt.
From the floor
In the hands-on prompt session, I wrote three labelled restaurant reviews, two positive and one negative, and asked the model to classify a new one, "The pizza was cold and tasteless". It answered negative, and the model even noticed that some of my example labels looked off and asked whether I had mislabelled them on purpose, because I had. That is a good reminder that the model reads your examples carefully, so a wrong example is not harmless.
The urgent and not urgent example above is the one we used in the cohort for few-shot email classification, because almost every company has a support inbox where that decision matters.