Late last year, a friend running a 3PL warehouse outside Columbus, Ohio called me in a panic. They'd installed a parcel damage-detection system — cameras snap photos, send them to the cloud for inference — and the diverters at the sortation chutes kept mis-sorting. Peak season was coming and they were about to lose the client. I walked the floor, didn't touch a single parameter, and told them to move inference off the cloud and back into the building. A week later, the mis-sort rate dropped from 3% to 0.4%.
The cloud wasn't the problem. That job never belonged in the cloud. Here's the math, so you don't burn money learning it the hard way.
Where the latency actually goes
Think of it like food delivery versus a kitchen in your own building. Cloud inference is the waiter boxing up the dish (the image) and shipping it to a central kitchen (the cloud), then shipping the finished plate back. Edge computing is setting up a stove right at the warehouse door — the food never leaves the building.
From shutter click to result, time goes to four places: upload, cloud queue plus inference, download, and local pre/post-processing. Just the round trip from a US warehouse to a cloud region eats 50–90 milliseconds (published measurements: ~56 ms average on the East Coast, ~90 ms on the West Coast). Add cloud API queueing and inference on top, and 400+ ms end-to-end is business as usual.
Put another way: the cloud wins on brains — bigger models, faster training, easy iteration. The edge wins on hands — it's close to the action, 1–2 ms over the LAN. Take NVIDIA's Jetson AGX Orin: the 64GB version pushes 275 TOPS at 15–60W, and a lightweight detector like YOLOv8n runs a frame in about 21 ms measured. Camera to box to actuator, the whole loop finishes under 60 ms. That's faster than a blink.
Three numbers that settle it
Number one: the latency budget. Line speed times allowable travel distance is your hard deadline. My friend's sorter ran 1.2 meters per second, with 30 centimeters between camera and divert — the parcel covers that in 250 ms. That's the budget. The cloud burns 50–90 ms on network round trip alone, 400+ ms all in, and misses the deadline every time. The edge does it in under 60 ms with room to spare.
Budgets vary wildly by task. Quick reference:
| Task | Latency budget | Where it belongs | | Chute OCR, label damage detection | under 250 ms | Edge | | AMR vision obstacle avoidance | under 100 ms | Edge, no alternative | | Hard-hat detection, forklift proximity alerts | under 500 ms | Edge | | Cross-dock photo audit, traceability archive | seconds | Cloud is fine | | Temperature forecasting, replenishment suggestions | minutes to hours | Cloud, don't overthink it |
Number two: the outage bill. Back in 2022, when I was still at an integrator, we ran vision obstacle-avoidance for 40 AMRs through the cloud. Every time the warehouse Wi-Fi roamed between access points, a few hundred milliseconds of silence, and the robots emergency-stopped on the spot. Fleet uptime for the day: 96%. The client's face said the rest. We moved inference onto the robots, kept the cloud for training and model pushes, and uptime came back to 99.5%. Rule of thumb: never assume the warehouse network stays up. Do the math on one hour of a stopped line yourself — next to that, an edge box costs pocket change.
Number three: the data bill. Cloud vision APIs charge per image — label detection runs $1.00 per thousand images in the first tier, $0.80 after that. My friend's line shot 3 photos per parcel, 40,000 parcels a day: 120,000 images a day, 3.6 million a month. First million: $1,000. The other 2.6 million: about $2,080. That's $3,080 a month, $37,000 a year, metered forever.
The edge alternative? Four industrial Orin boxes at two to three thousand dollars each — call it $10,000 one time. Payback: 10,000 ÷ 3,080, a little over three months. Everything after month three is pure savings.
The smartest setup is usually hybrid: edge handles real-time inference, the cloud handles training, model distribution, and after-the-fact analysis. Do the dirty work on site, do the thinking back at headquarters — like a restaurant chain where each location cooks, but recipe R&D lives at HQ.
One detail beginners miss: the edge box only runs inference — training still lives in the cloud. The right pattern is a closed loop. The edge flags the hard cases it got wrong or scored low on and sends those back up; the cloud retrains on them and pushes the improved model back down. Like a master training an apprentice: the apprentice works the floor, writes down whatever stumped them, and the master reviews it at night. The edge is the hands, the cloud is the brain — separate, but connected.
Three traps waiting on the edge side
Trap one: heat. An Orin pulls 60W at full load. In a 40°C warehouse summer, stuffed into a sealed cabinet with no airflow, the chip throttles and your 21 ms inference degrades toward 80 ms. Then the fans clog with dust — I'm telling you, half of edge hardware MTBF gets eaten by dust. Worst install I ever saw: the cabinet bolted right next to the sorter, vibration working the M.2 drive loose, the box dropping its OS every other day. Took a week to pin down.
Trap two: model updates. In the cloud, you tweak a threshold, hit save, and it's live everywhere. With dozens of edge boxes, it's OTA one unit at a time, and the moment versions drift, line A and line B start disagreeing with each other. No remote-ops capability? Don't touch edge.
Trap three: people. Warehouse IT knows how to reboot. When TensorRT throws an error, they stare at it. Edge boxes with nobody to raise them become e-waste in three years.
Rant time: plenty of edge projects exist not because the business needs them, but because "edge computing" sounds impressive in a boardroom. Running minute-tolerance jobs like temperature forecasting on edge hardware is just donating money to your vendor. Pseudo-requirements like that deserve to be shot on sight.
Still unsure? Ask yourself three questions in order: does your latency budget survive a cloud round trip? Can you afford to stop when the network drops? Does the monthly data bill exceed the price of one edge box? If any answer is no, pilot the edge on one line first.
So is the money worth it? Watch this one number: payback months = one-time edge hardware cost ÷ (monthly cloud bill − monthly edge ops cost). Over 18 months, keep paying the cloud. Under 12, buy the edge boxes with your eyes closed. Stuck in between? Depends how scared you are of an outage.
By Da Liu, Automation Engineer
