Here's my verdict up front: if an AI vendor contract doesn't address model decay, think twice before signing it. That accuracy number from acceptance testing? It was true on that day, under those conditions. After go-live, decay isn't a possibility — it's physics. And if the contract is silent on it, you're the one holding all the risk.
Decay Isn't a Bug. It's the Default.
In 2023 I was at a 3PL warehouse outside Phoenix — roughly 100,000 square feet, home-goods e-commerce. They'd put a vision system on the audit station of the sort line: cameras photographed each parcel's shipping label and product appearance, automatically verifying the SKU was correct. During acceptance week, the vendor arrived with a test report showing 99.2% recognition accuracy. The warehouse manager was thrilled and signed a three-year service contract.
Eight months later, the complaints started. False alarms at the audit station kept climbing. Operators had to double-check everything by hand, and throughput actually fell below what it had been before the system went in. We pulled the data: accuracy had slid to just above 95%. What happened in those eight months? Three things — and not one of them was anyone's fault.
First, the SKU mix changed. Home goods turn over fast with the seasons. In eight months they'd added more than 400 new SKUs, many of them repackaged versions of existing products. New colors, slightly tweaked dimensions, and the model started struggling. Second, seasonality. Between Thanksgiving and Christmas, parcel volume doubled, the conveyor got sped up, and the cameras' capture quality dropped. Third, a small process tweak: the warehouse replaced the audit station's fluorescent tubes with brighter LEDs. The color temperature shifted, and the model's color-based features drifted off target.
That's data distribution drift, in plain terms: the data the model sees every day slowly stops resembling the data it was trained on. Vision systems aren't the only victims. I've seen a replenishment forecasting model at an apparel warehouse double its error after consumer buying patterns shifted — the system kept ordering against demand curves from years earlier. So remember this: decay isn't a bug, it's the default. Every model running in a warehouse — vision, forecasting, anomaly detection — decays. The only question is how fast.
Without Decay Terms, the Acceptance Number Is Decoration
What made the Phoenix case painful wasn't the technology. It was the contract. Across three years of agreement, accuracy appeared exactly once — in the acceptance clause. Hit 99%, pass, done. After acceptance, the contract said nothing about accuracy. When the model dropped to 95%, the vendor's response was perfectly "reasonable": the contract covers no post-launch metrics, so retraining is a new service, billed separately.
The warehouse manager was furious when he called me. "We accepted at 99.2%, now it's 95% — how is that not a breach?" I gave him the uncomfortable answer: on the letter of the contract, it isn't. The acceptance clause governed a single day. Everything after that was unwritten. He hadn't signed "99% accuracy for three years." He'd signed "99% on acceptance day — good luck after that."
I've seen more than one contract shaped like this. AI vendors always sell the acceptance number: 99%, 99.5%, beautiful figures. But acceptance is measured on a dataset the vendor prepared with care — ideal lighting, ideal angles, ideal SKU distribution. A live warehouse is a different animal entirely.
There's another layer most people miss: vendors have no incentive to volunteer decay terms. Their business model is software licenses plus implementation services, and per-cycle retraining fees are a lovely recurring revenue stream. The faster the model decays, the more retraining they sell. Writing decay obligations into the contract — free retraining during the warranty period, for instance — means cutting their own income. If you don't raise it, they won't either. This one is on the buyer to demand at the negotiating table.
My judgment stands: a contract that talks acceptance but never mentions decay quietly transfers 100% of the decay risk to you. The vendor gets paid. You live with the consequences.
Make Retraining the Vendor's Obligation
So what should the contract actually contain? One logical chain: define the baseline, set the trigger, assign the retraining duty, spell out the consequences of persistent failure. Break any link and the whole chain fails.
Pin down the baseline metric — and the measurement method. "Accuracy not below 98%" by itself is worthless. Measured on what data? How many samples? Daily or weekly aggregation? Peak season weighted the same as off-season? Without the method written down, every dispute becomes a shouting match. My recommended language: adopt the acceptance test set as the baseline dataset, re-test quarterly against an equally sized, equally distributed set of newly labeled samples, and judge on a four-week rolling average. Once the method is in the contract, the vendor can't wave you off with "it looks fine on our end."
The trigger has to be specific. For example: rolling accuracy stays more than two percentage points below baseline for four consecutive weeks, and the vendor owes you a retraining cycle. Or a relative decline exceeding 5%. The exact figure is negotiable — but there must be a figure. Without a trigger, "retrain when it decays" is an empty sentence. Who decides what counts as decayed? You'll be arguing about that instead of fixing the model.
Retraining ownership and cost is the single most valuable clause in the agreement. Spell out: how many business days the vendor has to deliver the retrained model after a trigger; whether retraining is free during the warranty period or billed per cycle with a written cap; how many days in advance you must supply labeled data for new SKUs, and who pays for rework when your labels are bad. The Phoenix warehouse got stuck precisely here — the vendor's retraining quote came in at nearly the cost of a fresh implementation. And don't overlook the labeling trap: in many warehouses, labeling is done part-time by operators, with uneven quality. Put the labeling standard and a sampling/inspection ratio in the contract, or you'll end up with a retrained model that performs worse and two sides blaming each other.
Finally, consequences for persistent failure. Service credits are the standard tool: miss the metric in a given month, that month's fee gets cut by a stated percentage. More important is the exit clause: if the metric stays missed for three consecutive months despite retraining, you can terminate without penalty, and the vendor must hand over the model, training data, and documentation — and cooperate on migration. Without an exit clause, you're locked in: the model keeps degrading, switching vendors costs a fortune, so you renew with your nose held.
Pre-signature checklist — verify every line before you sign:
- Baseline metric, locked: the acceptance number plus the measurement method (test-set composition, sample size, measurement window, rolling-average formula).
- Decay trigger: how many points below baseline, sustained for how many weeks, triggers the vendor's retraining duty.
- Retraining timeline: business days to deliver the retrained model after a trigger, and what counts as delivered.
- Cost allocation: is retraining free during the warranty period; if billed, a per-cycle cap written into the contract.
- Data supply duty: how many days ahead you provide labeled data for new SKUs or process changes; who owns rework on bad labels.
- Monitoring reports: vendor delivers accuracy curves and drift metrics monthly or quarterly; you reserve audit rights on the raw data.
- Service credits: a concrete percentage fee reduction for any month the metric is missed.
- Exit clause: persistent misses (e.g., three months) give you penalty-free termination; the vendor hands over model, data, and docs, and cooperates on migration.
