It was 6:30 in the morning, and a WMS POC evaluation was about to start in our warehouse. The night shift had not left yet. The day shift was already clocking in. A truck sat at the dock waiting to unload, and the office phone would not stop ringing. The sales rep clicked through the big screen fast. Scan a carton, and the order was in. Release a wave, and the picks were sorted. The pick path drew itself across the screen like a map. The supervisors next to me kept nodding. Honestly, it looked good. But one thought kept nagging me. This was their turf, their data, their script. Of course it looked good. A demo that looks good proves nothing. What matters is what happens in our building, with our freight, on our shifts.

I have seen a vendor score full marks in the demo and then spend three months firefighting after go-live, and another look ordinary on screen and run smooth for years. The difference was never the feature list. It was what we actually tested. Here is how we do it now: setting the scenarios, running real volume, and writing acceptance into the contract.

WMS POC Evaluation Starts With Your Worst Day, Not Their Best Demo

A WMS POC evaluation does not start with booking a demo slot. It starts with writing down our hardest day. For us, that is the week before peak season. Two hundred-plus orders have to ship before ten. Returns land in the afternoon. The night shift still has to run a cycle count. We all know which links in that day break first: mixed-lot receiving, a full bin with nowhere to put the pallet, a short pick that has to be held, a rush order that forces the wave to change mid-flight.

So we write the test scenarios in the order that day actually happens, from the first truck at the dock to the last truck leaving and the shift closing. Each scenario states three things: what the starting position is, who does the work, and what counts as a pass. Take mixed-lot receiving: one truck carries three lots, but the system shows only one. How is the difference recorded? Who confirms it? Can the freight wait in a holding zone? That single scenario stopped one vendor cold. Their system could record the difference, but the ticket needed a supervisor to approve it on a desktop while the receiver waited on the dock and the truck sat blocked.

Keep the list short, eight to twelve scenarios, each one something that really happened here. Bring the veterans in. They remember the details a demo never shows: which bin sweats in winter, which customer needs a separate label, which carrier's label scans worst. Old Zhang on nights once told me: demos only show good days, and a warehouse lives or dies in a few bad minutes on a bad day.

Never let the vendor write the test scenarios. Nine times out of ten, a vendor-written scenario is tailored to their own system, every step landing on their strongest feature. We write the questions. The vendor runs them. Whoever writes the exam holds the power.

WMS POC Evaluation Runs on Real Orders, and the Math Happens Live

Once the scenarios are set, we run real order volume through them. This is where most proofs of concept fall apart: a dozen rows of clean demo data fly, everyone applauds, the contract gets signed, and then real volume arrives and the system crawls.

Our rule is simple: the orders, products, and bins all come from our own records from last month. Pick the biggest day and the messiest day. The biggest day tests whether the system can carry the load. The messiest day tests whether exception handling keeps up. Pass both, and you can sleep at night.

Here is an illustrative example, not our real figures. Say a warehouse ships 2,000 orders a day, and the test runs a 500-order sample. Pickers averaged half a minute faster per order, about four hours saved across the sample. But we asked three questions on the spot: how many orders hit an exception, how much longer did each take, and did response time slow down at peak? More than one order in ten hit an exception, each taking almost four extra minutes. The exceptions ate more time than the normal orders saved. Net saving: under one hour, not four.

What we test The demo number The real-day number
Pick speed Time per order on clean demo data Re-run on peak-day real orders, exceptions included
Exception handling Standard path in the demo Mixed lots, short picks, and rush insertions each run once
System response Speed with an empty database Re-test at full stock with peak concurrency
Reporting and close A report printed the same day Night-shift close and count differences handled end to end

The people running the test must be the people who will run the system later, not the vendor's consultants and not the one office kid who is good with computers. Let the receivers, pickers, and checkers get their hands on it. Their hands tell the truth. In one demo, a picker with eight years on the floor tried three orders and said the text was too small to read with gloves on. That one sentence beat three hours of review meetings. People are part of the system. If the interface fights them, the system fails no matter how good the engine is.

Here is the counterintuitive part: the vendor whose demo runs smoothest is not always the best vendor. Once, a demo ran with zero errors, start to finish, and that made me nervous instead of happy. When does our warehouse ever have a zero-error day? We dug in and found they had switched off every exception branch in advance. The demo ran on a road they had already paved. The system worth buying stays calm when things go wrong: a wrong order can be rolled back, a full bin gets a new home, a network drop lets the floor keep working offline. A flawless demo usually means the test was too easy, not that the system is strong.

WMS POC Evaluation Ends in the Contract, or It Didn't Happen

The run is done and the numbers are in. Now the acceptance standard goes into the contract, the step skipped most often because everyone is tired of demos and wants to decide. But hear me: whatever is not in the contract, the vendor will not own. A verbal "no problem" turns into "that needs custom development" on go-live day.

Our acceptance list is fixed now. Five items, each with a number, an owner, and a retest date. Pick speed: at peak-day real volume, how much faster per order than the old system, and what exception rate counts as a pass. Inventory accuracy: what count-difference rate we accept, and which counting method proves it. Exception handling: for mixed lots, short picks, and rush orders, how many steps the system must resolve them in, with no detour into paper logs. System stability: the response-time ceiling at peak concurrency, and the recovery steps after a network or power cut. Training and handover: real hands-on training time for every shift, plus permissions and records that match how our veterans train newcomers.

The contract also states the retest rules: how long the vendor gets to fix a failed first test, and that we re-run the same scenarios after the fix, no swapping in easier ones. Two failed rounds, and we can walk away, deposit terms spelled out. Say the hard things up front. We learned that the expensive way. One year our contract said the system would "meet expectations after go-live." Nobody could say what those expectations were. Two months of arguing followed, the money paid, everyone exhausted. Since then, our contracts carry no phrase like "meet expectations." Only numbers and scenario codes.

Before signing, one more item: the vendor leaves the scenarios and the data they ran, attached to the contract. When something goes wrong after go-live, we check it. Was it never tested, or did something change after testing? That same attachment doubles as training material for new hires.

Choosing a WMS cannot be rushed. One pretty hour in a demo room is worth nothing against 365 days of real operation. A WMS POC evaluation puts the pretty talk aside and tests the system against our hardest day, our real orders, and math done live in the room. Pass that test, and your hand stays steady when you sign. Fail it, and switching vendors now beats firefighting every day after go-live. When the next vendor walks in, hand them the scenario list before they open their laptop.

By Chen Feng, Warehouse Operations Manager