A 3PL operator in Fontana, California spent roughly $45,000 last spring on an AI assistant that was supposed to answer "every SOP question on the floor." The demo was flawless. The vendor's pilot report claimed a 94% satisfaction rate. Thirty days after go-live, the safety manager pulled the logs and found that nearly one in four answers contained something wrong — a wrong stacking height, a wrong temperature threshold, a wrong customer packing rule. The assistant delivered every one of them with total confidence and a neatly formatted citation.
This is the part vendors don't put in the deck: a retrieval-augmented warehouse assistant doesn't fail because the language model is dumb. It fails because the engineering around it was never finished. I've now watched four of these deployments go sideways in the past year, and the failure pattern is so consistent I can practically recite it.
Where the wrong answers actually come from
The first failure point is retrieval, and it accounts for most of the damage I've seen. Warehouse documents — SOPs, equipment manuals, customer SLAs — typically get chopped into fixed-size chunks, 500 tokens each, and embedded into a vector store. An operator asks whether forklifts can charge inside the cold-chain area. The system retrieves a three-year-old battery-room policy written for an outdoor charging shed and stitches together an answer that sounds authoritative and is completely inapplicable. The chunk lost its context; the embedding only understood surface similarity.
The second failure point is document versioning, and it's sneakier. Warehouse documents change faster than IT expects. An SOP goes from v2 to v3 because a customer manager said so and someone taped a new form to the wall. The old PDF stays in the knowledge base with no "superseded" flag and no effective-date filter. The model can't tell which version is current — it just assembles the highest-scoring fragments. In Fontana, the assistant once cited "SOP v3.2, Section 4" for a stacking limit. The warehouse had never produced a v3.2. The model had fused a section from v2 with a section from v3 and invented the version number. Hallucinations are often built from half-true retrieved fragments, not from nothing.
The third failure point is the knowledge boundary. Workers ask questions outside the knowledge base: local minimum wage changes, temp agency rates next door, HR policies. A well-designed assistant should say "I don't have that — ask your supervisor." But a default-configured model hates saying it doesn't know. It fabricates, and the fabricated numbers come with decimal points, which makes them look credible.
The golden set: 50 to 100 questions before you talk models
My advice to operators now is blunt: don't start with model selection. Start with an evaluation set.
The recipe is simple. Ask shift leads and the safety manager to each list 20 "getting this wrong would hurt someone" questions: stacking limits, aisle clearances, hazmat segregation distances, cold-chain alarm thresholds, customer-specific packing rules. Then pull three months of real questions employees actually asked and take the 30 to 50 most frequent. That gives you 50 to 100 items, each paired with a "golden answer" signed off by a senior operator and whoever owns compliance — down to the clause number and the effective version.
The set does three jobs. First, run it before go-live; if exact-match accuracy against the golden answers is below 85%, don't launch. Second, re-run it after every model swap, retrieval change, or document update, so fixing one thing doesn't break another. Third — and this is the one that matters — write it into the vendor contract: acceptance accuracy, refusal rate, and citation coverage, tied to payment terms.
The most serious contract I've seen specified a 30-day pilot with at least 50 questions per day, and acceptance required ≥90% accuracy on the golden set with a refusal rate between 5% and 15%. Note the floor on refusals. An assistant that never says "I don't know" is far more dangerous than one that occasionally declines. A refusal is a health signal, not a failure.
Fix retrieval before you blame the model
When answers go wrong, most teams reach for a bigger model. In my experience, eight times out of ten the problem is retrieval, and the model is taking the blame.
Change the chunking strategy first. Stop cutting by fixed token counts. Chunk along the document's natural structure: SOPs by section and clause, equipment manuals by whole troubleshooting procedure. A clause cut in half can't be saved by a better embedding. In practice, semantic chunking is the cheapest accuracy gain available — it costs almost nothing.
Then add metadata filtering. Tag every document at ingest: version, effective date, applicable site, applicable zone. Filter to the latest effective version before doing semantic matching. Don't delete old versions — audits may need them — but mark them superseded and exclude them from retrieval by default. The Fontana site made exactly this change and the phantom version numbers largely disappeared.
One more underrated move: give high-frequency, high-risk questions a deterministic fast lane. Stacking heights, aisle widths, alarm thresholds — there are maybe two or three dozen of these. Put them in exact-match Q&A pairs that bypass vector retrieval entirely. That's not cheating. It's moving the most dangerous questions out of a probabilistic system and into a deterministic one.
Three rules that force the assistant to admit uncertainty
Even with retrieval fixed, models still confabulate occasionally. You don't govern hallucinations with hope. You govern them with three hard rules.
First, mandatory citations. Every answer must name its source document and clause, and the link has to resolve to the actual text. In evaluation, an answer without a verifiable citation is scored as wrong, full stop. This single rule forces vendors to get retrieval right, because there's nothing left to fabricate.
Second, refuse below a confidence threshold. If retrieved documents score below a relevance cutoff, or if multiple sources contradict each other, the assistant answers "no clear answer in the knowledge base — escalated to your shift lead" and routes the question to a human. Set the threshold conservatively. On a warehouse floor, the cost of a wrong answer is always higher than the cost of no answer.
Third, human spot-checks on a schedule. Month one after go-live: 20 random real Q&As checked by a human every day. Month two: 30 per week. After three stable months: 10 per week. Every miss goes back into the pipeline — retrieval problem, fix chunking and tags; document problem, update the knowledge base; boundary problem, add a refusal rule. Keep the audit trail. You'll want it when safety asks.
The warehouses that shouldn't build this
Now the part that annoys vendors. Not every warehouse should build an AI Q&A assistant.
A sub-30-person operation shouldn't. The SOPs fit in a binder, the veterans are on the floor, and shouting a question across the aisle is faster and free. A warehouse whose SOPs are themselves a mess shouldn't either — wrong, expired, contradictory documents fed into AI just produce confident confusion at machine speed. And a site with fewer than 20 questions a day shouldn't bother: the labor of maintaining the eval set, the spot-checks, and document governance will cost more than the Q&A time you save.
The fit is real for 100-plus-person sites running multiple shifts with meaningfully different SOPs per customer — high turnover, night shifts with nobody to ask, piles of customer-specific requirements. Even there, position it honestly: it's an always-available digital veteran you can look things up with, not a manager that makes judgments for you.
Before you sign anything, answer one question: can your senior operators find the latest SOP version without opening a laptop? If they can't, neither can the AI. It will just give you the wrong answer faster — with a citation that looks real.
