Forecast Accuracy Audit

Measure MAPE and bias per item family; a systematic bias means the process, not the model, is broken.

Time30 min
FormatSolo
StageForecast

Forecast Accuracy Audit: what it is and why it works

A forecast accuracy audit measures two different things: how large forecast errors are, and whether they lean consistently in one direction. Mean absolute percentage error (MAPE) averages the absolute error divided by actual demand and shows typical error size. Because simple MAPE breaks down when actuals are near zero and overweights small items, many teams use weighted MAPE, the sum of absolute errors divided by the sum of actuals. Bias is the signed error, commonly forecast minus actual summed and divided by total actual, so a positive value means over-forecasting. Both are computed per family and plotted over twelve months to see patterns rather than single-month noise.

Separating size from direction matters because they have different causes and cures. Random error of moderate size is expected and is handled with safety stock. Persistent bias is not random: it usually reflects incentives, optimistic sales inputs, unreviewed overrides or a model that misses a trend. That is why sustained bias points to the process, not to the model. A tracking signal, cumulative error divided by mean absolute deviation, flags bias automatically. The audit supplies the numbers that the monthly demand review explains, sets realistic targets by ABC/XYZ class, and provides the error inputs that safety stock calculations need.

What you need

Twelve months of archived forecasts, frozen at the lag that drives decisions (for example, one or two months before the period)
Actual demand for the same periods at the same level of aggregation
Item or family groupings, ideally aligned with ABC/XYZ classes
A clear sign convention for bias and an agreed error formula

What you get

MAPE or weighted MAPE per family, month by month
Bias per family with its direction and trend
A short list of families with persistent bias and suspected process causes
Accuracy and bias targets per family class, with a monthly review cadence

When to use it

When stockouts and overstocks alternate and nobody checks forecast quality.

How to do it, step by step

Compute MAPE per family to see typical error size.
Compute forecast bias (signed error) to see systematic over- or under-forecasting.
Plot both over twelve months — patterns, not single months, matter.
If bias is positive for months, fix the process: incentives, inputs or overrides.
Set an accuracy target per family class and review monthly.

Worked example: Finding a hidden bias in spare-parts forecasts

Illustrative scenario — figures are realistic but not from a real company.

A manufacturer of industrial compressors forecast service parts for its aftermarket business. Warehouse stock kept rising while service technicians still reported stockouts on common items. Leadership assumed the forecasting software was at fault and was evaluating a replacement.

The analyst pulled forecasts frozen one month ahead and compared them with actual shipments for twelve months across 14 part families.
Weighted MAPE ranged from 18% on filters to more than 60% on rebuild kits, which was in line with how erratic each family was.
Bias told a different story: nine of the 14 families were over-forecast in at least ten of twelve months, with overall bias around +17%.
Tracing the overrides showed that regional managers added a safety margin to their forecast inputs, and planners then added safety stock on top, buffering the same risk twice.

Result. The company removed manual margins from forecast inputs and let safety stock carry the uncertainty explicitly. Within two quarters bias fell to about +3% and parts inventory dropped by roughly 12% with no loss in fill rate. The software replacement was shelved. The audit showed that the model was largely fine; the process around it was not.

Open it in the FieldKit Supply Chain app → Get the Supply Chain template pack — join the waitlist

Common pitfalls and how to avoid them

Measuring accuracy against the latest forecast just before the period, when it no longer drives decisions.Freeze and store the forecast at the lead-time lag that triggers purchasing or production, and measure that one.
Averaging simple MAPE across items with very small or zero demand.Use weighted MAPE or a similar volume-weighted metric, and handle intermittent items separately.
Reporting accuracy without bias, so persistent over- or under-forecasting stays hidden.Always show signed bias next to error size, and track it over time.
Setting one accuracy target for all items.Set targets by segment; stable AX items can reach far lower error than erratic CZ items.

Frequently asked questions

What is the difference between MAPE and forecast bias?

MAPE measures how large errors are on average regardless of direction, so over- and under-forecasts both count as error. Bias measures the average signed error, so over- and under-forecasts cancel out and what remains is a systematic lean. A forecast can have high MAPE and no bias, which is noisy but fair, or low MAPE with a steady bias, which is precise but consistently wrong in one direction.

What is a good forecast accuracy percentage?

There is no universal figure. Achievable accuracy depends on demand stability, aggregation level, horizon and industry. Stable, high-volume families can often reach low error, while erratic or new items may never do so. The better approach is to benchmark against a naive forecast and your own history, set targets per segment, and aim for bias close to zero everywhere.

How do you calculate forecast bias?

A common formula is the sum of forecast minus actual over a period, divided by the sum of actuals, expressed as a percentage. Positive values then indicate over-forecasting and negative values under-forecasting. Some organizations use actual minus forecast instead, so state the convention clearly. A tracking signal, the running sum of errors divided by mean absolute deviation, can flag when bias exceeds a set limit.

Origin

Forecast error metrics (MAPE, bias) — operations research practice, 1960s.

Used in these playbooks

S&OP setup quarter 1 quarter

Install the monthly S&OP rhythm in one quarter: demand review, supply review, one reconciliation with real trade-offs.

Related methods

More in “Forecast”

Anticipate demand so the chain plans from facts, not hope.