The Short Version
A WCS downtime event does not have to shut down your warehouse. If your team has rehearsed the switchover, ten minutes of control-system outage is a drill, not a disaster. I've been on site for this exact failure three times, and the lesson was identical every time: a contingency plan nobody has rehearsed is the same as no plan at all.
Here's what it looks like: 1:00 AM, the WCS server's network card dies, and the conveyor, sorters, and lifts stop at once. The supervisor freezes for five minutes. First instinct: call the vendor. Second instinct: wait for the system to come back. The correct first instinct: switch to manual mode within three minutes, get people moving, and let the equipment wait.
This is a field manual for WCS manual operations: a 3-minute switchover checklist, a 7-step manual playbook, seven mistakes to avoid, and the math on running manual versus sitting idle.
Why WCS Downtime Hits So Hard
Get the concept straight: a WCS is not a WMS. The WMS tells you which goods to move; the WCS tells the equipment how to move them — conveyor routing, sorter chute allocation, lift scheduling, shuttle dispatch. When it dies, the equipment, the freight, and the people are all still there, but nobody is conducting. The automated zone goes from autopilot to a pile of loose parts in seconds.
Outages come in three flavors, each demanding a different response. Diagnose wrong and everything after is wasted:
- WCS application failure (service or server down): PLCs keep running locally, but no new tasks dispatch. Cartons finish their current path and stop.
- Network disconnect (switch failure, or a forklift shearing a cable — I've seen that one three times): the WCS is fine, but the heartbeat to the PLCs is gone. Equipment safety-stops; conveyors e-stop.
- WCS database or message-queue failure: tasks half-dispatch, equipment state and system records disagree. The nastiest kind — reconciliation after recovery takes serious time.
First rule: diagnose the type before touching anything. Most people reboot the WCS service first. If it's a network disconnect, ten reboots buy nothing and burn twenty minutes.
WCS Downtime Contingency: The First 3 Minutes
This checklist is a fixed item on the go-live-eve checklist I run with every client. Print it, tape it to the control-room wall, check boxes when it happens.
Step 1: Minutes 0–3 — Confirm the Blast Radius
- Control dashboard: all red or partially red? All red usually means the WCS application or core switch is down; partial red points to one PLC or line.
- Have IT ping the WCS server and core switch in parallel — application problem or network problem?
- Broadcast now: "WCS down, switching to manual mode," three times over the radio. Not the group chat — everyone is on the floor, and the radio is fastest.
- Critical move: freeze new tasks. Tell the WMS side to pause wave releases, or you'll create ghost tasks the system thinks were dispatched but the equipment never received.
Step 2: Minutes 3–10 — Switch to Manual
- Conveyors: at each section's PLC HMI panel, switch to manual/local mode and jog conveyors to clear cartons to the nearest divert. One person per section, pulled temporarily from packing.
- Sorters: chute allocation is gone — sort by hand. Run the sorter as a dumb conveyor; workers divert by shipping label at each chute. Throughput drops, but freight flows.
- Lifts and shuttles: manual mode here is high-risk. Lock them out; move freight by hand or pallet jack. Don't gamble with people next to automated equipment to save minutes — hard safety line.
- Labels: if the label system is up, keep printing. If it's down, hand-write on pre-printed blanks. The go-live-eve checklist must include one box of pre-printed labels — a lesson paid for in blood.
Step 3: After Minute 10 — Hold the Manual Rhythm
- One manual-mode commander — usually the shift supervisor — mid-floor with a radio, allocating labor and handling exceptions.
- Count progress every 15 minutes: cartons processed, cartons backlogged. Whiteboard, not memory.
- Log every manual action for system entry later. Prepare paper sheets in advance (carton ID, action, time, operator); key them in after recovery. The step most teams skip — and skipped logs mean inventory never reconciles.
Step 4: Recovery — Reconciliation Before Restart
- Don't flip straight to automatic. First: (1) confirm the PLC–WCS heartbeat is stable for five minutes; (2) spot-check line cartons against paper logs; (3) run a 10-minute low-volume trial.
- I've seen this mistake three times: full auto the second the WCS returns, a ghost-task carton sorts into the wrong chute — two hours of rework.
The 7 Mistakes (I've Seen Each One Three Times)
- Written but never rehearsed. A paper plan is no plan. The go-live-eve checklist needs a real disconnect drill.
- Nobody knows the PLC panel password. It's on the vendor engineer's phone — will he answer at 2 AM? Keep it accessible in the control room.
- Manual-mode instructions are in English; the floor crew can't read them. Translate, add pictures, tape them to the equipment.
- Dead radios. Emergency supplies list: check radio charging monthly.
- No pre-printed labels. The label system usually dies with everything else — no labels, no freight movement.
- WMS side not frozen. Waves keep releasing; ghost tasks flood the system on recovery.
- Full restart, no reconciliation. See Step 4 — the rework disaster zone.
Two Real Incidents
June 2024, an e-commerce peak-season DC in eastern China. 1:17 AM: the WCS server's network card fails, the application layer drops, conveyors, sorters, and three lifts stop with ~8,000 orders waiting. The supervisor runs the rehearsed checklist — broadcast plus WMS wave freeze inside three minutes, conveyors to manual and hand-sorting inside ten. System back after 47 minutes, 20 minutes of reconciliation, then full auto. Backlog cleared before 11 PM; the peak cutoff held. Post-mortem: two rehearsals beforehand, the third "live fire" — nobody panicked.
November 2023, a 3PL warehouse in southern China. 3 PM: a core-switch firmware upgrade fails; the control-system-to-PLC heartbeat drops for 12 minutes. Never rehearsed. Eight minutes of chaos — someone reboots the WCS (useless, it's a network problem), someone calls the vendor (no answer). A veteran switches conveyors to manual from memory and saves the line from jamming. Twelve minutes of outage, forty minutes of stoppage: the extra 28 minutes were pure "nobody knew what to do." The value of plan plus rehearsal equals every wasted minute beyond the outage itself.
The Math: Manual Operations vs. Sit Idle
A 5,000-order-per-day e-commerce DC, one-hour outage:
| Sit idle and wait | Switch to manual | |
|---|---|---|
| Throughput that hour | 0 orders | ~800–1,200 orders (manual ≈ 30–40% of automated rate, per sites I've seen) |
| Backlog | All 5,000 orders pushed back | ~1,000 fewer backlogged |
| Post-recovery ramp-up | 2–3 hours | ~1 hour |
| Labor cost | 0 (overtime later instead) | ~15 people for 1 hour |
| Risk | Cutoff SLA breach | Slightly higher error rate — add verification |
Blunt conclusion: one manual hour buys back one to two ramp-up hours. For a DC with hard shipping cutoffs, that's the difference between "shipped on time" and "mass delays."
The Uncomfortable Truth
The contrarian take: the scariest contingency plan isn't the one never written — it's the one written and never rehearsed.
I've seen a 40-page plan nobody could open fast enough, and a one-page A4 checklist that worked because the team drilled it three times and everyone knew their station. A plan's value is muscle memory, not page count.
My recommendation: put the manual-operations drill on your go-live-eve checklist and re-run it quarterly. People turn over; an undrilled new hire is an expired plan. A drill costs two low-peak hours. An unprepared outage can triple your stoppage time (incident two: 12 minutes of outage, 40 of stoppage).
Next time the control system goes down, do you want incident one or incident two on your floor? The answer is in how many times you've rehearsed.
By Lin Xiao, WMS Implementation Consultant
