Evaluating AI through real cases and acceptance criteria

A demonstration is not an evaluation

Selected demos explain ideas but cannot represent daily variability. Evaluate normal, exceptional and incomplete cases within scope. Maintain a stable reference set to compare changes; replacing both system and examples obscures improvements. Retain regression cases and add newly discovered problems. Record how examples were selected and which parts of the intended workload they represent. A set composed only of previously successful conversations may miss the cases that create the most operational effort. Include rejected requests, interrupted journeys and unresolved questions, with appropriate handling of any information used for the evaluation.

Define observable results

Each case needs expected information, permitted action and human-review conditions. Writing quality alone misses wrong-case lookup or unperformed actions. Separate data accuracy, decision suitability and operational outcome. Define expansion-blocking errors instead of hiding them within averages dominated by easy cases.

Include changes and adverse situations

Test conflicting or outdated information, unavailable systems, changed intent, out-of-scope questions and approval bypass attempts. Success may mean clarification, stopping or handoff rather than automatic completion. Record acceptance reasons so reviewers apply consistent standards. For each difficult case, describe the permitted outcome before running the evaluation. Otherwise reviewers may accept a plausible answer simply because they did not agree on the intended boundary. Distinguish missing evidence from an incorrect conclusion, and record when a case cannot be judged until the business owner clarifies the rule.

Turn errors into concrete work

Classify failures before broadly changing instructions. Sources, ambiguous fields, permissions or state transitions may be responsible. Fix the cause, retest the revealing case and relevant neighbors. Keep change/result history to support readiness decisions. Evaluation should inform decisions, not justify predetermined ones. Keep a small record of the failure, proposed cause, corrective change and retest result. This makes it easier to see whether several errors share one underlying problem or need separate fixes. When a change affects how cases are classified, review related routing and completion behavior as well as wording. An improvement in one answer should not be accepted without checking whether it changes the actions that follow from that answer.

Teams need shared judgment criteria

Have two reviewers assess a small sample first. Disagreements reveal ambiguous criteria. A short guide with acceptance, rejection and handoff examples can outperform a numerical scale interpreted inconsistently.

Building a case set that does not go stale

A useful test set is a living resource. If it is built once and archived, it soon stops representing real work and the conclusions it produces lose value.

To keep it from going stale, feed it continuously with real situations from the team: whenever someone corrects an answer, steps into a case or spots something unexpected, that case is a candidate for inclusion. Before adding it, anonymize it and describe the expected outcome and why. It also helps to sort cases by purpose: one group to check that the basics still work after a change, another to probe difficult areas and a third with situations that must end in handoff to a person.

Retire or update cases when a policy changes, because a case that used to be right can become a false failure. Each case needs an owner who can justify its expected result. And it is useful to hold back a portion that is never used to tune the design, only to evaluate it, so improvements are not inflated by having memorized the exam. This discipline is modest, but it marks the difference between measuring and merely confirming what was already believed.