Bottom line first: if your WES user acceptance testing isn't run on real wave data, end to end, you haven't tested anything. I've watched this exact failure three times — three green UAT sign-offs, three go-live disasters in week one. What follows is the checklist from 30+ WES implementations. Steal it.

Why Test Data Can't Test Waves

WES (Warehouse Execution System) is not WMS. WMS logic is mostly linear, so you can click through it in UAT and learn something real. WES is the wave brain: it breaks orders into waves, pushes waves to people and machines, and reconciles the results. Its bugs live in the interactions between waves — which hand-built test data has none of.

Test data fails in three ways:

  1. It's too clean. Hand-made orders all have complete addresses, full stock, sensible quantities. Real waves carry cancellations, short picks, orders missing address fields, and multiple clients mixed together. Wave-consolidation rules and exception-routing rules exist precisely for those "dirty" orders — running clean data skips the parts that most need testing.
  2. It's too thin. Testing runs 10 orders per wave; go-live runs 200. At real volume, allocation boundary conditions change completely: not enough pickers, conveyor jams, wave priorities colliding. Ten-order tests never trigger any of that.
  3. There's no clock pressure. Testing happens at 10 AM with a mouse; go-live happens at 2 AM when a peak wave must release in five minutes. If UAT never runs against real cadence, you'll never learn what a three-minute wave-release delay actually costs.

So the iron rule: UAT data must be copied from production, anonymized, and run as-is. From our project experience: copy at least 3 complete waves, each with 80+ orders, and make sure one of them comes from your most recent peak day. Scale the numbers to your warehouse, but don't cut the "3 waves" minimum — with fewer, you can't test wave merging, wave contention, or wave queuing at all.

Three Steps to Run UAT on Real Waves

Step 1: Prepare, and freeze the baseline.

  1. Export 3 real waves from production: orders, inventory snapshot, locations, labor roster. Anonymize names, addresses, and phone numbers — but do not touch order structure, quantity distribution, or the exception ratio.
  2. Freeze this dataset as the baseline in your test environment: nobody hand-creates orders in the test database during UAT. The moment someone does, results become unrepeatable — a bug could be the system or the data, and you won't know.
  3. Write test cases in three tiers. P0 (blocker): wave release, wave cancellation, exception routing, power-loss wave recovery — if these fail, there is no go-live. P1 (major): wave consolidation, priority scheduling, task reclamation, reporting. P2 (minor): UI display, exports, permissions. From our projects, a typical case count runs 120–200, with P0 at roughly 20%. Not one P0 case may be dropped.

Step 2: Three rounds, each one tougher.

  • Round 1 — Smoke. Run 1 wave end to end. The whole team learns the flow; the goal is to surface environment, permission, and data problems early. This round is allowed to be messy. The goal is "it runs."
  • Round 2 — Regression. Run all 3 waves against real cadence, with emphasis on exception scenarios: cancel half the orders mid-wave, take a picker offline, simulate a conveyor alarm stop. The goal of this round is "find bugs." Every bug found gets fixed and its wave re-run.
  • Round 3 — Acceptance. Re-run all 3 waves with screen recording running, and have the client's key users sign off on every step. The goal is "final" — when this round passes, the UAT report is your acceptance record.

No skipping rounds. I've seen clients say "round 1 went smoothly, let's just sign off." Round 1 was smooth because it ran just 1 wave.

Step 3: Set the acceptance bar in writing, before UAT starts.

The acceptance criteria go into a document both sides sign before testing begins. No changes after. Our standard, from experience:

  • P0 pass rate: 100%. No exceptions.
  • P1 pass rate: ≥95%.
  • P2 pass rate: ≥90%.
  • P0 defect closure rate: 100%, and closure requires retesting — "the developer says it's fixed" doesn't count.
  • All 3 real waves run end to end with zero manual workarounds. ("Someone will watch this part manually after go-live" is not a pass.)

That last one matters most. Every step propped up by manual workarounds in UAT will blow up on your busiest day after go-live.

Two Go-Live Horror Stories

Story 1: Fake test data, signed off, exploded on peak week.

2023, a 3PL in East China, WES go-live. The client didn't want the hassle of copying production data, so they hand-built 50 "standard orders," ran UAT, got all green, and the boss signed off. Then the first peak-season wave hit: 320+ orders in one wave, and a boundary condition in the wave-consolidation rule — when the same SKU appears in multiple waves at once, allocated quantities get double-counted. The test data never put one SKU in two waves, so the bug sat quietly. On go-live day, pick tasks were duplicated, two pickers were sent to the same location for the same carton, and the conveyor jammed for 40 minutes. Six of us rebuilt waves manually on-site until 3 AM. During the post-mortem, the client's IT manager said, "UAT never caught it." I said, "That scenario didn't exist in your UAT data."

Story 2: Skipped the exception scenario, found out during a real blackout.

2024, an e-commerce warehouse in South China. UAT was genuinely thorough — 3 real waves, all green, signed off. But the client crossed out the "power-loss wave recovery" P0 case: "we have UPS, power won't fail." Three weeks after go-live, the industrial park did transformer maintenance, the UPS lasted 20 minutes, and the servers went down anyway. When power returned, half the waves were stuck "in execution" and half "released." Wave recovery had never run on real data — clicking "resume" threw an error. For 3 hours, 200+ staff stood idle while supervisors handed out paper pick lists. We later found the bug: the recovery logic mishandled timestamps for waves spanning midnight, and the test environment had never crossed midnight. One "simulate power loss" test case would have caught it — half a day of work. The real cost: 3 hours of standstill plus the client's trust.

The Night-Before-Go-Live Checklist

After UAT is green and before go-live, there are 10 items. Check each one off, print it, stick it on the wall:

  1. ☐ 3 real waves run and recorded, key users signed off
  2. ☐ P0 defect closure at 100%, every closure backed by a retest record
  3. ☐ Exception scenarios (cancellations, shorts, power-loss recovery, picker offline) all exercised
  4. ☐ Wave-release cadence measured: order-to-release within SLA (our experience: under 5 minutes for standard waves)
  5. ☐ Rollback plan rehearsed: if WES dies, everyone knows how to fall back to the old system or manual process
  6. ☐ Monitoring and alerts live: wave backlog, task timeouts, release failures — SMS/phone to a named person
  7. ☐ Go-live night duty roster set: developers, implementers, client IT — all three on-site
  8. ☐ Production database fully backed up, restore verified
  9. ☐ The boss has signed: UAT acceptance report AND go-live authorization, both documents
  10. ☐ "Manual workaround list" at zero: every "we'll watch this manually" item from UAT is either fixed or written into an SOP with a named owner

Item 9 deserves its own paragraph: if the boss hasn't signed, you don't go live. I've seen too many projects where the IT manager felt "it's good enough," the boss was traveling, something broke at midnight, and the boss's first sentence was "I never approved this." Signatures aren't bureaucracy — they're how responsibility is assigned. It's the cheapest item here and the easiest to skip.

One Counterintuitive Take

Here's the uncomfortable one: a 100% UAT pass rate is a warning sign, not good news.

A healthy UAT always blows up in round 2 — dirty data and exception scenarios in real waves cannot produce zero defects. If your UAT is green from start to finish, there are only two explanations: the test cases are too weak, or the data is too clean. The healthy curve: small explosions in round 1 (smoke), big explosions in round 2 (regression), fix and re-run, then green in round 3 (acceptance). Green that arrives too early means you didn't test the real thing.

So tonight, open your UAT plan and search for the words "dirty data" — cancellations, short picks, midnight-spanning waves, power-loss recovery. If they're not there, send the plan back for rework. Three extra days of UAT beats three all-nighters of firefighting after go-live. I've run that math for 30+ clients. UAT is cheaper every single time.

By Lin Xiao, WMS Implementation Consultant