I'll take the edge side on this one — with one condition. You need to separate your AI into two kinds: the kind that's "watching the road" and the kind that's "reading fortunes."
Road-watching can't wait a single millisecond. Fortune-reading can take an hour and nobody dies. That line is exactly where edge and cloud split. Over the last two years I've watched warehouses burn money on both sides of it: shoving everything into the cloud, then watching the whole sort line stall the moment the network hiccups; or buying a GPU server on day one to run a reporting model that only needs to run once a day.
Where to deploy inference comes down to three things fighting each other: latency, bandwidth, and cost. Whichever one a use case is most sensitive to decides where the inference should live.
Anything in Milliseconds Belongs at the Edge
Start with the road-watchers: vision inspection, AMR obstacle avoidance, robotic pick-and-place. The common thread is a closed loop measured in milliseconds. A sort line moves well over a hundred parcels a minute; a box is in the camera's field of view for a blink, and the recognition result has to come back in tens of milliseconds. A round trip to the cloud easily takes hundreds of milliseconds — by the time the answer arrives, the box is past the divert point and the pusher arm has nothing to aim at.
Then there's the bandwidth bill. A single vision camera streaming raw images around the clock generates tens of gigabytes a day. Multiply by a few stations and your monthly uplink costs stop being a rounding error. And warehouse networks are nothing like office networks: Wi-Fi dead spots, signal blocked by steel racking, forklifts chewing through cables as a daily event. Betting real-time inference on that chain is handing your line's uptime to luck.
I saw the counterexample at a 3PL in Ontario, California last year: their barcode cameras ran recognition in the cloud. One afternoon the network jittered, and the whole line slowed to manual rescans. It was just an access point swap for maintenance — four hours of throughput down by nearly a third. The owner told me afterward, with total sincerity, that every dollar saved on servers went straight into overtime pay.
AMR obstacle avoidance is even more extreme. The vehicle is moving, a person steps out in front of it, and the fused lidar-plus-vision inference has to complete locally and issue a brake command in tens of milliseconds. If that decision waited on the cloud, the vehicle would already be through the person. That's why every serious AMR vendor runs avoidance models on board — it's not a technology preference, it's physics.
The rule for this category is one line: if the inference result directly drives a physical action, it runs at the edge. And "the line keeps running when the network dies" goes into the acceptance criteria.
What Can Wait Is Cheaper in the Cloud
The other category is the fortune-readers: demand forecasting, replenishment suggestions, safety-stock calculations, monthly operations reports. These models run once and their answers stay valid for a day or a week — a few minutes of delay changes nothing. The real advantage of the cloud here isn't cheap compute; it's that you don't have to own compute at all. No machines to buy, no heat to manage, no 2 a.m. calls about a dead box.
The cloud's second advantage is training. Training eats historical data, and warehouse history already lives in the cloud: the WMS database, the order history, the labor records. Training and iterating up there is the natural fit. The edge device has exactly one job: download the trained model, run inference, and pick up new versions from the cloud on a schedule.
The hybrid architecture I see working in the field — and the one I'd hand to any small or mid-size warehouse as a starting template — is this: edge does real-time inference with local caching so it runs standalone when the network drops; the cloud does training, model distribution, and global analytics. Write the division of labor down and don't let either side grab the other's job. Roll out model updates in stages: new version to one station first, a week of clean running, then fleet-wide. Never do a "one-click whole-warehouse upgrade" — when it breaks, the whole warehouse breaks together.
And make the acceptance test physical: on go-live day, pull the network cable and watch what happens. If the line stops, the architecture is decoration, no matter how pretty the vendor's slides were.
Small Warehouses: Skip the GPU Server, Start with an Industrial PC
Now the money. People hear "AI inference" and picture a GPU server rack costing tens of thousands of dollars. I'll say something unpopular: nine out of ten small warehouses don't need one.
A single vision inspection station runs comfortably on an edge device in the class of NVIDIA's Jetson Orin Nano — the 8GB version released in 2023, tens of watts of power draw, fits inside an electrical cabinet. A GPU-equipped industrial PC, in the low thousands of dollars, can serve two or three stations. Start there: prove the ROI on one or two stations — cut the miss rate on manual re-checks, or kill the slowdown stoppages on the sort line. Once the math works, then talk about expanding stations, then about servers.
One environmental detail that gets missed constantly: dust and heat. Warehouses hit high temperatures in summer and dust gets everywhere; consumer-grade hardware thrown into that environment won't survive a peak season. When choosing edge hardware, wide operating temperature, fanless design, and dust resistance are hard requirements — don't shop on compute specs alone. That's another reason I recommend starting with an industrial PC: it's built for exactly this environment.
Who are GPU servers actually for? Operations running dozens of stations at once, or teams training their own models. If your warehouse isn't there yet, buying one means paying the power bill for a machine room — and funding the vendor's sales commission.
My verdict is straightforward: split your use cases into road-watchers and fortune-readers first — edge for the former, cloud for the latter. Start hardware with a single industrial PC, prove one station, then expand. Next time an AI vendor pitches you, ask three questions: where does inference run, does the line stop when the network dies, and how often do models update? If they can't answer those three, you can close the proposal deck.
