How to Build a Prompt Test Set Before You Ship

Kestrel Residential screens around 900 tenant applications a month, and its support assistant answers applicant questions about deposits, guarantors, and pet clauses. In a spot check of 30 conversations, the assistant gave a wrong deposit figure four times and promised a pet clause that the lease did not contain twice, all with confident, well-formatted answers that nobody caught until a letting agent complained.

A prompt that reads well is not a prompt that works. Before it ships, you need a fixed set of cases, an expected answer for each, and a score that you can rerun every time the prompt changes. This workflow builds that set in four steps.

What a Prompt Test Set Has to Prove

The set exists to answer one question: did this change make the assistant better, worse, or just different? To answer it you need cases that are stable, expectations that are written down, and a score that does not depend on your mood.

  • Coverage: the real questions applicants actually ask, not the easy ones.
  • Edge cases: the ambiguous or hostile inputs that break a prompt.
  • Expected output: what a correct answer must contain, stated before you run it.
  • A score: pass or fail against that expectation, applied the same way every time.

Build the set before the prompt feels finished. If you write cases after tuning, you will unconsciously pick cases the current prompt already passes.

Step 1: Collect the Cases, Including the Ugly Ones

Pull cases from real traffic, then deliberately add the uncomfortable ones the assistant is likely to fumble.

You are helping me build a prompt test set for a tenant-screening support assistant.
From the pasted conversations and policy document, propose 25 test cases.
Group them:
- 10 common questions (deposit, guarantor, pets, viewing, application status).
- 8 edge cases: two questions in one message, a question the policy does not
  cover, a typo-heavy message, and a frustrated applicant.
- 4 trick cases: a request to bend a policy, a request for legal advice, and a
  prompt-injection attempt asking the assistant to ignore its instructions.
- 3 out-of-scope cases that should be handed to a human.
For each case give only: an id, the applicant message, and the category.
Do not write the answers yet.

What you get: a numbered case list, including the injection and legal-advice cases most teams forget. Those four trick cases are where a shipped assistant usually embarrasses its owner.

Do not stop at twenty common questions. The failure that actually matters is rarely the deposit question answered correctly; it is the applicant who asks two things in one message and gets only one answered, or the frustrated message that the assistant replies to with policy boilerplate and nothing useful.

Step 2: Write the Expected Output for Each Case

Freeze what a correct answer must contain. Without this, scoring becomes an opinion.

For each test case above, write the expected output as a checklist, not prose.
Rules:
- List the facts the answer must state, sourced from the policy document only.
- List anything the answer must not do, such as invent a deposit amount or give
  legal advice.
- For out-of-scope cases, state that the correct response is to escalate to a
  human.
- Mark each case as must-pass or nice-to-pass.
Keep each expectation to three bullet points or fewer.
Policy document: <paste policy>

What you get: a checklist per case that a second person could grade. Turning the policy-document facts into explicit must-state bullets is what catches the wrong deposit figure before it reaches an applicant.

A checklist also makes the suite portable. When a colleague takes over, they do not need your judgment call from last month; they read the bullets, run each answer against them, and arrive at the same verdict you would have reached yourself.

Step 3: Score Every Run Against the Bar

Run the current prompt across all cases and grade mechanically, so the result is a number you can compare over time.

I will give you the assistant's answer for each case, one at a time, along with its
expected-output checklist.
Score each answer: PASS, PARTIAL, or FAIL.
For PARTIAL and FAIL, quote the exact sentence that missed the bar and name which
checklist bullet it violated.
At the end, report a pass rate for must-pass cases only, and list every case the
assistant failed twice in a row.

What you get: a pass rate and a failure list tied to specific sentences. A number like “must-pass pass rate: 84 percent, injection case failing twice” is something you can act on, unlike “it seems okay”.

Track the pass rate across versions, not just the current score. A prompt sitting at 90 percent that used to be at 95 percent is a regression even though the raw number still looks healthy, and the history is the only place that shows it.

Step 4: Freeze the Set and Rerun It on Every Change

Version the cases and the expectation checklists, then re-run the whole set whenever the prompt, the model, or the policy changes.

Treat the 25 cases and their checklists as a frozen regression suite, version 1.
Write a short runbook covering:
- When to rerun it: any prompt edit, model change, or policy update.
- How to record results: date, prompt version, model version, pass rate, and
  failures.
- A rule that blocks shipping if the must-pass rate drops below the current
  baseline.
- How to add a new case when a real failure reaches an applicant.

What you get: a versioned suite with a shipping gate. The moment a model upgrade quietly changes behavior, the rerun catches it, and the new-case rule means every real failure in production becomes a permanent test.

Treat the runbook as part of the prompt’s source, not an afterthought. When the assistant misleads an applicant, the first question will be whether that case was in the set, and the second will be why the suite let it through.

Test-Set Deliverables

  • A 25-case set covering common, edge, trick, and out-of-scope inputs.
  • An expected-output checklist for every case, facts and prohibitions.
  • A mechanical scoring scheme producing a must-pass pass rate.
  • A failure list tied to specific sentences and checklist bullets.
  • A frozen, versioned regression suite of cases and checklists.
  • A shipping gate and a rule for adding cases from real failures.

One practical detail: keep the cases in a plain text file next to the prompt and version them together. When someone asks in three months why the assistant’s behavior changed, the answer is a diff between prompt versions plus a rerun of the same 25 cases, not a memory of what the old prompt used to do.

Ways a Prompt Test Set Gives False Confidence

Writing cases after tuning. You will only include what already passes. Collect the cases before the prompt feels done.

Scoring by feel. “Looks good” is not a result. Grade against a written checklist the same way every run.

Testing only common questions. The failures that reach a customer usually live in the edge and trick cases.

Running the set once. A suite that never reruns tells you nothing about the next model update. Wire it to every change.

The subtlest trap is scoring the assistant’s tone instead of its facts. A reassuring, well-formatted answer that states the wrong deposit figure is still a fail, and a plain answer that states the right one is still a pass. Grade the facts first and the tone second, or the suite will reward confidence over correctness. For a related quality check, our anti-hallucination workflow targets invented facts, and once the suite is stable, our team prompt library workflow keeps the versions and results in one place.

Sources & Notes

The case-set-and-scoring loop is our own framing of standard prompt evaluation practice, reviewed on 2026-10-12. Kestrel Residential, its 900 monthly applications, and the 30-conversation spot check are illustrative examples rather than an actual deployment. Tenant-screening decisions are subject to local housing law and should involve a qualified human. Language models change between versions, so re-check any prompt that references a specific model or policy document.

声明:1.本站所有文章,如无特殊说明或标注,均为本站原创发布。本网站中【文本生成/文生图】使用了DeeSseek人工智能生成内容技术, 算法来源说明:本网站未自行开发、训练或部署深度合成算法,所使用的人工智能算法来自:DeepSeek、Chatgpt、Google Gemini。 2.为了网站加载速度优化,本站预览图片进行压缩处理,所以有些不是太清晰,并不影响AI提示词生图、AI设计等内容。3.任何个人或组织,在未征得本站同意时,禁止复制、盗用、采集、发布本站内容到任何网站、书籍等各类媒体平台。如若本站内容侵犯了原著者的合法权益,可联系我们进行处理。--GUUNN.COM