Fall 2024, Fontana, California. A 3PL doing e-commerce fulfillment asked me to post-mortem their AI project: AMR dispatch plus vision-based verification. The board had approved $380,000 at kickoff. Vehicles and software licenses were all in there. When finance closed the books: $740,000. A 95% overrun. The CFO asked one question at the closeout meeting: where did the extra $360,000 go?
Nobody could answer. Because the budget sheet at kickoff had exactly two lines: equipment, software.
An AI project's budget is always four buckets, never two: software, hardware, labor, and data. Miss any one of them and the overrun is a certainty. Here is the budget framework I hand to every AI project before sign-off, on a first-year TCO basis:
| # | Bucket | What goes in it | Typical share of first-year TCO |
|---|---|---|---|
| 1 | Software | Licenses or subscriptions, API tokens, platform fees, year-two price hikes | 20%-30% |
| 2 | Hardware | Cameras, edge compute boxes, network upgrades, power work, install materials | 25%-35% |
| 3 | Labor | Internal hours (IT, ops, engineering), vendor implementation days, training | 20%-30% |
| 4 | Data | Data cleansing, sample labeling, master-data governance, backfill | 10%-20% |
Read that again. Software is, at best, a third of the total. If you budgeted only software, you priced the tip of the iceberg.
Bucket by bucket.
Bucket one, software. Licenses and subscriptions are the visible part. The hidden parts are three. API tokens billed per call feel trivial in a pilot at a few thousand calls a day, then production runs tens of thousands a month and the invoice doubles. Platform fees charged per camera or per concurrent stream do not scale linearly - going from 4 cameras to 12 jumps you up pricing tiers. And year-two renewals: year one is the discount, year two is the real price, with 15%-30% increases being normal. Negotiate year-three pricing before you sign and put it in an addendum.
Bucket two, hardware. Cameras and edge boxes are visible. Networks and power are not. Warehouses have Wi-Fi dead zones; an AMR rolling into one drops its connection. Adding access points, pulling fiber, tuning roaming handoffs - a few tens of thousands of dollars, gone. Old buildings short on power need a new industrial circuit: electrician on site, outage windows coordinated with operations, more money. If the site survey was a walkthrough instead of real engineering, this bucket is a guess.
Bucket three, labor. And here is the counterintuitive call: the most expensive line in an AI project is not software. It is your own people's time. A mid-size project burns three months of IT support, two months of half-time from your best ops people, a month of engineering on site. Hundreds of hours. At fully loaded cost, that routinely exceeds the software license itself. Worse, it never appears in the project budget - it hides inside departmental headcount, where overruns are invisible.
Bucket four, data. This is the bucket you cannot cut, and the one that gets cut first. Early 2025, an auto-parts 3PL in Texas launched vision inspection with a $20,000 data-cleansing budget. Once work started, they found 400,000-plus historical SKU master records: duplicates, missing images, inconsistent units. Cleansing plus labeling ended at $90,000 - more than four times the budget. Why is it untouchable? AI eats data. Dirty data means a blind model. One bad SKU master record can push vision verification false positives from 1% to 8%, and your floor team burns hundreds of extra exception-handling hours a day. The $70,000 you "saved" on data gets paid back in manual labor within three months. That money was wasted.
Five overrun traps, all from real projects:
| Trap | How it happens | Defense |
|---|---|---|
| Network and power upgrades missed | Budget covers devices only; survey was a walkthrough | Paid engineering survey before kickoff, survey fee in the budget |
| Data cleansing underestimated | Workload estimated by gut feel | Clean a 10,000-record sample, measure the unit cost, multiply |
| Internal hours never budgeted | Only vendor implementation days counted | Headcount times months, fully loaded, written into TCO |
| WMS integration fees | Assumed "there is an API, so it is fine" | Interface work quoted per day, locked before signing |
| Year-two price reset | Signed at the year-one discount | Three-year price ladder in the contract |
One more pattern worth naming: the pilot-to-production cliff. Pilots run in a clean corner of the building with the vendor's best engineer on site. Production runs across the whole floor with your night shift. Budget the production reality, not the pilot experience - double the integration days, add 50% to the data work, and assume the first month runs at 80% of target throughput. Every project I have seen that skipped this adjustment paid for it in quarter two.
Three levers that actually save money, each one checked against the math.
First, run a paid PoC before signing a multi-year deal. Spend a little to verify two numbers: can recognition accuracy hit the line the business needs, and does the system keep up with floor takt time. If the PoC fails, the hundreds of thousands after it never get spent. Cheapest risk hedge there is.
Second, do data cleansing in-house. Labeling fits well into QC staff downtime, and quality beats outsourcing. Master-data governance done jointly by IT and ops cleans the SKU master once, and every AI project after that benefits. Spend it once, amortize it across projects.
Third, stage the hardware. Pilot one zone, replicate after it runs clean. Stage the network upgrades with the pilot footprint. Lighting up the whole building on day one maximizes your cost of being wrong.
One computable conclusion to close. Before any AI project gets approved, run this formula:
Payback months = first-year TCO / monthly quantifiable savings
Monthly savings admits exactly three inputs: labor hours saved times hourly rate, error-rate reduction times average cost per error, throughput gain times marginal profit. Anything you cannot compute - "efficiency gains," "digital image" - counts as zero. Payback beyond 24 months: park it. Under 12 months: approve. Between 12 and 24: decide on strategic value.
If the Fontana project had run the four-bucket math at kickoff, the board would have seen roughly $700,000 of real TCO, not $380,000. Approve or reject, their call - but nobody would have been blindsided at closeout. A budget's job was never to make the number small. Its job is to leave overruns nowhere to hide.
By Lao Zhou, Logistics Cost Analyst
