Rant first: Amazon's restock suggestions can't replace real demand forecasting. Last peak season it told me to stock 8,000 units of a hot SKU. I stocked 6,000 — and sold 2,300. The rest sat in an FBA warehouse for four months, and the storage fees alone were painful. That was the day I learned: for stock planning, trust the platform less than yourself, and trust your gut less than a model.
In 2025 our catalog grew from 40 to 200 SKUs and daily orders from 300 to 1,500. Planning stock in Excel by gut feel ended exactly how you'd expect: stockout rate hit 12%, one hot product went out of stock for 21 days, and I watched a competitor steal our Best Seller badge. Those 21 days cost roughly 9,000 lost orders. I've done that math. It still hurts.
So in Q1 2026, I gritted my teeth and ran real forecasting models — time series and machine learning, head to head. The conclusion might surprise you.
Time series vs machine learning: the demand forecasting comparison table first
| Time series (moving average / exponential smoothing / Prophet) | Machine learning (XGBoost etc.) | |
|---|---|---|
| Data appetite | Runs on 1–2 years of sales history | Needs 2–3 years plus external features (price, promos, holidays, traffic) |
| Feature engineering | Basically none, works out of the box | You build it: promo flags, price changes, category events, all of it |
| Our measured WAPE | 18% | 12% |
| Setup cost | Half a day in Excel | Took us 3 weeks of tuning (my midnight hours) |
| Maintenance | Near zero | Retrain after every big promo or accuracy falls off a cliff |
WAPE = forecast error as a share of actual sales; lower is better. We ran both on the same 120 SKUs with two full years of history, same test window. Fair fight.
Read the table sideways: start with the "data appetite" row — that's the gate; miss it and nothing else matters. Then "our measured WAPE" — that's the payoff. Then setup and maintenance cost — that's the price. Gate times price versus payoff: do that math and the choice makes itself.
Machine learning won by 6 points. Sounds great — but do the full math before you get excited: three weeks of tuning and feature work; retraining after every major promo or accuracy slides like a falling stock; and on long-tail SKUs, ML improved accuracy by less than 2 points — pure theater.
So here's our actual setup: the top 20 SKUs (70% of revenue) run on XGBoost, refreshed weekly. The other 100 ride on moving averages plus 15% safety stock — ten minutes of work. The hours I saved went into negotiating supplier payment terms — 30 days to 45. The freed-up cash flow is worth more than those 2 points of accuracy.
When not to touch machine learning
Here's the counterintuitive call: if you're under ~100 SKUs or ~500 orders a day, machine learning is a cannon for a mosquito.
Why? ML is a beast you feed with data. Starve it and it's worse than gut feel. We had a long-tail SKU selling 400 units a year — XGBoost confidently predicted 1,200, because it mistook one lucky mini-spike for a pattern. For SKUs like that, an honest moving average wins: stock what you sell, plus 15% safety stock, done.
Two more categories where even time series can't help: new products (no history — the model is blind) and hard-seasonal items (Halloween costumes only wake up in September, and every year's curve looks different). Skip the models for these. Watch the category rhythm, keep safety stock generous, and test with a small first batch.
Here's our own joke. In March 2026 we launched a new product — zero history. XGBoost still produced a confident "first order: 4,000 units" based on category averages. I didn't buy it and ordered 1,500. We sold 900; the other 600 still aren't cleared. If I'd listened to the model, 4,000 units would have flattened my cash flow. In the new-product phase, a model's confidence is pure theater. Small first batch, watch two weeks of sell-through, then reorder — beats any model.
One line: data is the model's food. Don't keep a beast you can't feed.
What bad demand forecasting actually costs per year
Stockouts: one day out of stock = daily units × margin per unit. Our 21-day stockout SKU moved 430 units a day at roughly $4 margin — $36,000 of margin gone in 21 days. That doesn't count the extra ad spend needed to climb back after losing the Best Seller rank.
Overstock: FBA long-term storage fees plus trapped cash. Those 3,700 leftover units sat for four months. Storage fees were the small part. The big part: $6 unit cost × 3,700 units = $22,000 of cash frozen in place — money I needed to chase new products for peak season.
How did those 3,700 units end? A 30% price cut and two months of clearance. Margin went to roughly zero, plus the ad spend to move them. That's the full cost chain of bad forecasting: stockouts lose orders, overstock freezes cash, clearance kills margin. Three cuts, all deep.
So every point of forecast accuracy is real money. Our WAPE went from 18% to 12%, stockout rate from 12% to 7% — the orders saved in one quarter paid for those 3 weeks of model work with plenty left over.
A quick note on scale: sellers often ask whether ML "pays for itself." For us, the math only worked because 20 SKUs carry 70% of revenue. If your revenue is spread flat across 200 SKUs with no clear head, the tuning cost eats the accuracy gain. Concentration is what makes the cannon worth loading.
Pitfalls: don't believe everything the model says
Three traps, all stepped in personally.
First, don't trust demo accuracy. In-sample fit numbers are paper tigers — gorgeous until they meet real future data. Only look at out-of-sample test results; everything else is theater.
Second, flag anomalies like Prime Day separately. In 2025 one SKU sold 20× its normal daily volume on Prime Day. Feed that to the model unmarked and it thinks every day is Christmas — you'll be stocked to the ceiling.
Third, every model output passes a human. XGBoost says stock 5,000; I cut it to 3,500 — because supplier lead time dropped from 45 days to 30 and the model doesn't know that. Models understand data; they don't understand the street. The final call is still yours.
Fourth, clean your data before feeding the model. Our first runs mixed returns and cancelled orders into "sales" — the model learned demand 8% higher than reality, and every forecast came out heavy. Now we strip returns and cancellations before every weekly run. Garbage in, garbage out — nowhere is it truer than forecasting.
Tested it myself, it works. Machine learning for the head SKUs, moving averages for the long tail, anomalies flagged separately, and a human signs off on every output — those four rules cost me $36,000 in tuition. A forecasting model isn't magic. It's just a coworker who can crunch numbers all night without sleeping. And no matter how good the coworker is, the boss still steers the ship.
By Ah K, E-commerce Seller (5 yrs)
