Zero-Shot vs Few-Shot for Classification: When Examples Win

A solar-panel installer routes about 400 support tickets a week through a single support inbox. Someone on the ops team built a ChatGPT prompt that sorts them into five buckets: Billing, Scheduling, Technical, Warranty, and Account. It worked on the first fifty tickets and then started mislabeling the interesting ones. “App shows zero production but the inverter light is green” landed in Account. “You charged me twice for the same monitoring plan” landed in Warranty. The labels were defined, but not demonstrated.

Classification is the one job where showing the model two examples per label usually beats writing a cleverer definition. Definitions describe a category; examples reveal its boundaries.

The Symptom: A Classifier That Keeps Changing Its Mind

Zero-shot classification, where you name the categories and hand over no examples, has a specific failure mode: overlap. Labels that a human distinguishes by context, like Billing and Warranty, read as near-synonyms to a model working from definitions alone. It guesses, and each guess is reasonable in isolation but inconsistent across a batch. Run the same ticket twice and you may get two different labels.

The tell is the ambiguous middle. Tickets that clearly belong to Billing or Technical get sorted fine. The ones that mention money and hardware, or hardware and an account setting, drift. Those are exactly the tickets a support team most needs routed correctly.

Zero-Shot Prompt vs Few-Shot Prompt

Same ticket set, same five labels. The difference is whether the model gets to see the boundary being drawn.

Classify each support ticket into one of these categories:
Billing, Scheduling, Technical, Warranty, or Account.

Billing = payment and invoice issues.
Scheduling = appointments and site visits.
Technical = problems with the system or hardware.
Warranty = claims covered by the warranty.
Account = login, profile, and account settings.

Tickets:
1. App shows zero production but the inverter light is green.
2. Charged twice for the same monitoring plan.
3. Need to move Thursday's site survey to next week.
...

Return the category for each ticket.

You are a support-ticket classifier for a residential solar
installer. Assign exactly one label per ticket.
Labels: BILLING, SCHEDULING, TECHNICAL, WARRANTY, ACCOUNT, UNSURE.

Labeled examples
"Panel output dropped to zero after yesterday's storm." -> TECHNICAL
"App shows zero production but the inverter light is green." -> TECHNICAL
"Charged twice for the same monitoring plan." -> BILLING
"Invoice total does not match my contract price." -> BILLING
"Need to move Thursday's site survey to next week." -> SCHEDULING
"Can someone come out to check a loose roof mount?" -> SCHEDULING
"My 12-year panel warranty: is a cracked junction box covered?" -> WARRANTY
"Replacement inverter for a unit still under warranty." -> WARRANTY
"Cannot log in, password reset email never arrives." -> ACCOUNT
"Update the email on my monitoring account." -> ACCOUNT

Disambiguation rules
- If a ticket mentions money AND hardware, label TECHNICAL when
  the ask is to diagnose or repair, BILLING when the ask is about
  the charge itself.
- If no example clearly matches, output UNSURE rather than guessing.

Output format: one line per ticket, "id: LABEL". No explanations.

Tickets:
1. App shows zero production but the inverter light is green.
2. Charged twice for the same monitoring plan.
...

The zero-shot prompt asks the model to build the category boundaries from a sentence each. The few-shot prompt shows the boundaries with two boundary-adjacent examples per label and then states the tie-break rule that covers the rest.

Why Examples Beat Definitions

• Examples encode the boundary, not the center. A definition tells the model what Billing is; an example of a hardware-plus-money ticket tells it where Billing stops. Boundaries are where classifiers fail.
• They anchor the output vocabulary. Showing labels in ALL_CAPS inside the examples makes the model emit that exact string instead of “Technical Support” one time and “technical” the next.
• They encode your house rules. Every support team has a rule the labels do not express, such as “diagnosis request beats billing mention.” A single labeled example teaches that rule faster than a paragraph.
• They give the model an escape hatch. With no UNSURE option, a zero-shot classifier must force every ambiguous ticket into a bucket. Few-shot lets you define what “I cannot tell” looks like.
• They make results stable. With a fixed, labeled set in the prompt, the same ticket tends to get the same label across runs, which is the minimum bar for routing.

Two examples per class is usually the sweet spot. One example leaves the boundary ambiguous; six examples per class bloat the prompt and start to bias the model toward whatever the examples share, such as a specific product name.

When Zero-Shot Is Actually Enough

The installer’s ticket flow fails on four of the five rows, which is why zero-shot kept drifting. Compare it with a task like labeling 20 survey responses as Positive or Negative: the labels barely overlap, volume is low, and consistency matters less. That task can stay zero-shot and lose nothing.

Building Your Own Labeled Examples

You do not need a large training set to write a few-shot prompt. You need four to ten tickets you have already labeled by hand, chosen to sit near the boundaries.

1. Pull your misclassifications. The tickets the model got wrong are the most valuable examples you own; they are, by definition, boundary cases.
2. Write two per label. Pick the clearest case and the most confusing case for each category.
3. Add one tie-break rule. State the single rule that resolves the most common overlap, in plain language.
4. Lock the output format. Specify the exact label strings and the exact response shape, such as id: LABEL.
5. Include UNSURE. Reserve one label for genuinely ambiguous items so the model does not guess just to fill the slot.

A reusable skeleton for any classification task looks like this:

You are a {{task}} classifier. Assign exactly one label per item.
Labels: {{LABEL1}}, {{LABEL2}}, {{LABEL3}}, UNSURE.

Labeled examples
"{{example}}" -> {{LABEL}}
"{{boundary example}}" -> {{LABEL}}
"{{example}}" -> {{LABEL}}
"{{boundary example}}" -> {{LABEL}}

Rules
- {{tie-break rule for the most common overlap}}
- If no example clearly matches, output UNSURE.

Output format: one line per item, "id: LABEL". No explanations.

Items:
{{pasted items}}

Run it once, spot-check the boundary tickets, and add any new misclassification as an example before you scale up. Grep the output for UNSURE and route those to a human; that queue is where your next batch of examples comes from. When you later move the same classifier into a reusable assistant, the labeled examples belong in the system prompt so they persist across every call, and for categories that truly need reasoning before a label, a short reasoning step before the final answer can help without changing the output format.

Sources & Notes

The zero-shot versus few-shot comparison reflects standard prompting practice and observed behavior with GPT-4-class models in 2024–2025. The solar installer and its tickets are illustrative; the categories and counts are invented for the example. Few-shot prompting improves consistency but does not guarantee accuracy, and a human review step remains advisable for routing decisions. For the narrower single-example case, see how one-shot compares: One-Shot vs Few-Shot Prompting.