The demo passed. That tells you almost nothing about production. Enterprise AI agents that show roughly 60% success on a single run drop to about 25% measured over eight consecutive runs at production load — the same system, evaluated honestly against sustained conditions rather than a curated example.

That gap is why 89% of AI agent pilots never reach production, and why Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027. Most of those failures are not model failures. They are the discovery, late and publicly, that nobody had tested the thing properly.

QA for an agent is different from QA for ordinary software, and the differences are learnable.

Why standard testing does not transfer

Conventional software testing asserts that a given input produces an expected output. An agent's output is variable, plausible-looking, and wrong in ways that pass a superficial check. There is no exception thrown when it confidently invents a delivery date.

Three properties drive the difference:

Correctness is not binary. A response can be accurate but poorly toned, or complete but missing an escalation it should have triggered. Grading needs more than pass/fail.

Failure is silent. A wrong answer produces a successful transaction. The only signal is a customer replying — after the damage.

Performance degrades over sequences. The single-run number is not the production number, which is the specific lesson in the 60%-to-25% drop.

The four-stage test plan

• Golden set — What runs: Curated real inputs, known-good outputs · What it proves: Baseline accuracy, regression safety · Duration: Ongoing
• Shadow mode — What runs: Live traffic, output not sent · What it proves: Behaviour on real distribution · Duration: 1-2 weeks
• Draft mode — What runs: Live traffic, human approves each · What it proves: Accuracy with zero customer risk · Duration: 1-2 weeks
• Limited autonomy — What runs: Live, auto-send on qualified categories · What it proves: Sustained production performance · Duration: Ongoing

The golden set is the foundation. 100 to 300 real inputs — not invented ones — with agreed correct outputs, covering the common cases, the known exceptions and the genuinely messy ones. It runs automatically on every configuration change, and without it you cannot safely alter a prompt, a rule or a model version, because you have no way to know what you broke.

Build it from real history. Curated clean examples produce an agent that performs well on curated clean examples.

Shadow mode runs the agent against live traffic while discarding the output. It is the cheapest way to see behaviour on the actual input distribution, including the categories nobody thought to include in the golden set. What it reveals most often is not accuracy problems but coverage problems — request types the configuration never anticipated.

Draft mode is where the real learning happens. The agent writes, a human approves or edits every message, and the edit log becomes the highest-value artifact in the process — a set of labelled errors produced by work the team was doing anyway.

Limited autonomy enables auto-send only on categories that met the bar, with everything else still escalating.

What does a production-ready AI agent test plan include?

A production-ready test plan includes a golden set of real inputs run on every change, a shadow-mode period against live traffic, a draft-mode period where humans approve every output, adversarial inputs, escalation drills, and sequence testing across consecutive runs rather than single interactions.

The last two are the ones most commonly omitted, and they map to the two most damaging production failures.

Test the escalation path, not just the answers

An agent's most important behaviour is what it does when it should not act.

Escalation is where teams under-test, because it feels like an edge case. It is not — it is the control that makes the rest safe. Drill it explicitly:

1. Does it escalate when it should? Feed inputs that fall outside authority and confirm it stops.
2. Does the human receive usable context? An escalation that forces someone to reconstruct the situation from scratch has moved the work, not reduced it.
3. Does it escalate rather than guess when data is missing? Confabulation under missing data is the highest-severity failure mode.
4. Does it escalate on anomalous input? Including content attempting to redirect it — adversarial examples belong in the test set, and the correct behaviour is escalation, not compliance.

Item three deserves the most attention. An agent that says "I don't have that information, escalating" is behaving correctly. An agent that produces a plausible ETA it has no basis for is the failure that costs a customer.

Test sequences, not interactions

The 60%-to-25% gap is a sequence effect, and single-interaction testing cannot detect it.

Realistic freight interactions are multi-turn: a quote request with a follow-up question, a status thread that spans days, a carrier exchange that returns to a topic. Test those as sequences — does context hold across turns, does an early error compound, does behaviour stay consistent when the same thread is revisited?

This is also where load matters. An agent evaluated at ten messages a day may behave differently at four hundred, and peak season will find out for you if you do not.

Define the go-live bar in advance

Decide the promotion criteria before you see the results, or the results will shape the criteria.

A workable bar, per workflow: golden set accuracy above a stated threshold; draft-mode approval rate above a stated threshold sustained for a week; escalation drills passing; adversarial inputs escalating rather than complying; and no unresolved category of systematic error in the edit log.

Per workflow, not overall — because status responses and quote handling have genuinely different ceilings, and a blended bar either blocks the easy work or waves through the hard.

Then keep measuring after go-live. The test plan proves readiness at a point in time; continuous observability is what tells you it is still true in month four. Pre-production testing without ongoing sampling is how a successful launch becomes a quiet regression.

Frequently asked questions

How large should a golden set be? Typically 100 to 300 examples per workflow, weighted toward real messiness rather than volume. Coverage of exception types matters more than raw count.

How long should shadow and draft mode run? One to two weeks each is common, though the signal matters more than the calendar — draft mode is ready to end when edits stop clustering into systematic categories and become idiosyncratic.

Can we skip shadow mode and go straight to draft? You can, and many teams do. Shadow mode's unique value is revealing request types your configuration never anticipated, before humans start spending time approving outputs for them.

Who should do the grading? The people who do the work today. They are the only ones who can tell the difference between a technically correct response and the right one for that account, which is precisely the distinction that matters.

The bottom line

Single-run demos and sustained production are different measurements, separated by a gap large enough to cancel projects.

Build a golden set from real history, run shadow and draft mode against live traffic, drill the escalation path including adversarial inputs, test multi-turn sequences rather than single interactions, and set the promotion bar per workflow before you see the numbers.

Debales deploys AI agents for freight quoting, order processing, ETA updates, and multi-channel customer communication — starting in draft mode against your real traffic, with the edit log and escalation behaviour visible before anything auto-sends. Book a demo.