The same algorithm scored 99.2% in the vendor's lab. A month on your warehouse floor, it's running at 92%. Before you accuse anyone of cooking the numbers — that gap is usually not the algorithm's fault. Your warehouse and their lab are not the same world.
What does the lab look like? Even lighting, clean backdrops, posed parcels, a stationary camera. What does your warehouse look like? Skylight glare that shifts from morning to afternoon, half the tubes in a fixture burned out, dust on every conveyor, cartons crushed at the corner, tape glaring across the barcode, a worker's hand blocking the camera for a second. The algorithm didn't change. The world did.
So where do those seven points actually go? One by one.
Why does lighting hurt more than anything else? Warehouse light is the most unreasonable variable there is. A conveyor near the windows gets front light in the morning and backlight in the afternoon; overcast and sunny days are two different brightness levels entirely. Vision systems hate lighting drift — the model trained on even light, and when the light changes, the features stop matching. The fix is unglamorous but it works: give the camera its own light source, a ring light or a bar light, and push ambient light out of the equation. Don't count on the model to "adapt." Controlling light physically beats any algorithm. I watched one project lose five points of accuracy on rainy days; after they added a ring light, the fluctuation basically disappeared.
Dust is the variable everyone forgets. It's chronic — a film builds up on the lens and the light housing, and image quality degrades a little every day. The fix isn't fancy: put lens cleaning on the daily checklist. A weekly wipe does more than a model upgrade.
What about the parcels themselves? Lab parcels are new, square, labels applied straight. Real parcels? A crushed corner, a crooked label, tape reflecting across the barcode, a black polybag hiding everything. Deformation and glare are where warehouse vision AI fails most. My judgment call: for any parcel-recognition project, at least a third of the training data should be ugly parcels — deformed, damaged, reflective, stained. The uglier the better. A model trained only on pretty parcels will fail on day one. I have yet to see an exception.
And speed and occlusion? The lab camera can take its time; your conveyor won't wait. Line speed brings motion blur; a worker grabbing an item brings occlusion. Don't try to out-algorithm these — engineer around them. Size the camera's frame rate and exposure to the actual line speed, and put a human verification point at the critical stations. A vision system in a warehouse is always a productivity tool, never a replacement for people. Treat it as a replacement and you'll pay tuition.
Why does training data have to come from the real floor?
This is the point I most want to land. Too many projects train on data the vendor shot in their own lab — or "staged" in your warehouse: pick a well-lit hour, arrange some clean parcels, shoot a thousand frames, go home and train. That produces a lab champion, and the first overcast day exposes it.
Do it right: after the cameras go up, run them for two weeks in collect-only mode. Capture all three shifts, sunny and cloudy days, every ugly parcel. When labeling, prioritize the hard samples — the ones the model got wrong or scored low confidence on. Feed those back into training, iterate two or three rounds, and that's when floor accuracy actually climbs. There is no shortcut. Skip this step and 92% is your ceiling; no amount of hyperparameter tuning gets you past it.
Do false positives and false negatives cost the same?
Before writing acceptance criteria, get one thing straight: which costs more — a false positive (flagging a good parcel) or a false negative (letting a bad one through)? In nearly every warehouse scenario, the miss is more expensive. A mislabeled parcel that ships means a complaint, a return, a reship — tens of dollars at minimum. A false alarm costs a few seconds of human review.
So acceptance can't rest on one "accuracy" number. My recommendation: score false-positive and false-negative rates separately, and hold the miss rate to a bar an order of magnitude tighter. When signing, don't trust a pretty "99% overall accuracy." Ask three questions: what was the miss rate on your test set? Was the test set real floor data? Will you put the test set composition in the contract appendix? Those three questions reveal the vendor's true level fast.
What happens when the system fails?
However good the system, design for the day it doesn't work. My three rules: anything below the confidence threshold goes to a human; if the system goes down, a manual process takes over; at critical stations — outbound verification, say — vision plus human, belt and suspenders. Don't put all your eggs in one basket. AI buys you efficiency, not immunity. When something goes wrong, the boss asks "how did nobody catch this," not "what was the accuracy."
The acceptance checklist — walk it before you sign
- Test set composition: does it include hard samples from real floor conditions — deformed, reflective, occluded, low-light — and at what share?
- Split metrics: what are the false-positive and false-negative rates separately? Is the miss rate held to a tighter bar?
- Lighting plan: dedicated light source? Who owns it when ambient light changes?
- Speed match: are frame rate and exposure sized to the actual line speed? Still accurate at peak speed?
- Hard-sample loop: how often do misclassified samples flow back after go-live? Is the iteration cycle in the SLA?
- Fallback flow: what's the confidence threshold? Who handles below-threshold cases? What's the manual plan when the system is down?
- Acceptance run: in your warehouse, on your parcels, at your line speed — run a full shift before signing. Lab numbers don't count.
The lab's 99% is the algorithm's ceiling. The floor's 92% is where engineering starts. The gap between them gets closed not by a better algorithm, but by better engineering. Remember that when you sign, and it'll save you real money.
