One global LightGBM model forecasts 91 days of daily demand for 500 store-item pairs. Then the part most forecasting projects skip: turning the error into safety stock, and replaying the last quarter to count what would actually have run out or been left over.
DataKaggle store-item demand: 913,000 daily sales, 10 stores × 50 items, 2013–2017. Synthetic-looking and very regular: no promotions, no stock-outs, no new items.
10.8%Q4 holdout WAPE, LightGBM
91%Of series where it beats Prophet
−16%Safety stock vs Prophet
2.5×Faster than 500 Prophet fits
Data flow · how a record moves
Daily sales to an order-up-to level
Each dot is a day of sales for one store-item. Four forecasters run side by side in a rolling backtest; the winner's errors size the safety stock, which is then tested against real Q4 demand. Click a stage to read what happens there.
01 · No peeking
Direct forecasting, tested for leaks
Every sales feature is lagged by at least the forecast horizon, 91 days. One model forecasts the whole quarter in one pass, and nothing inside the forecast window can reach its own features. The windows start on Mondays so weeks line up with order cycles.
Leakage testA unit test scrambles every sale after the cutoff and checks that the features don't change. 10 tests in total.
02 · Accuracy
One global model wins
WAPE, weighted absolute percentage error, lower is better. Grey is the Q1–Q3 average; red is the held-out Q4. LightGBM has the lowest error in all four quarters.
CV WAPE, Q1–Q3Holdout WAPE, Q4
What it learnedLags 364 and 371, the same weekday a year earlier, carry 90% of the split gain. The yearly mean (4%) and the two-year lag (2%) adjust the level.
03 · Bias
The rule without seasons
The weekday average looks reasonable until the season turns. It cannot see the spring ramp coming, then carries the summer level into autumn.
Weekday average, Q2 bias0%under-forecasts the spring ramp
Weekday average, Q4 bias0%over-forecasts autumn
LightGBM, Q4 bias0%RMSE 7.7 units
Fit time per fold0svs 87s for 500 Prophet models
MaintenanceLightGBM is one model to retrain and monitor. Prophet here is 500 separate models.
04 · Stock
Same service, less buffer
At a 95% weekly service target, each model's safety stock across all 500 store-items, then replayed on real Q4 demand. The tick marks 95%; the number on the right is the share of weeks fully served.
LightGBM holds 3,300 fewer units than Prophet (−16%) and 8,900 fewer than seasonal naive (−34%). Seasonal naive misses the target and loses almost 3× as many units. The weekday average never runs out, but only by holding over two days of demand, about 5× LightGBM's leftovers.
LimitsThe data has no promotions or stock-outs, so real demand would need censored-demand handling. Lead time is taken as one week; longer lead times scale safety stock by roughly √(lead time). The same ranking holds at 90% and 98% targets.