Are all forecasts wrong?

You’ve probably heard the phrase “all forecasts are wrong”, suggesting that the future is unpredictable and that no forecast will ever match the actual outcome. Well, this phrase is not entirely correct, and here’s why.

When your favourite forecasting approach generates point forecasts, it usually provides a conditional mean. This means it’s giving you the average expected value based on the information you have (in rare cases, it might produce a geometric mean or median, but for simplicity, we’ll focus on the mean). Statistically speaking, the mean doesn’t match the actual values exactly because, as discussed in the previous posts, any data (like demand for a product) consists of both structure and noise. The purpose of a point forecast is to capture the structure, not to get the actual value exactly. So, your point forecast isn’t “wrong” – it is just not meant to match the actual values.

In the image below, you can see how an ETS(M,N,M) model captures the structure and produces point forecasts for some time series.

ETS(M,N,M) fitting the seasonal data

ETS(M,N,M) fitting the seasonal data

Expecting these forecasts to perfectly match future data would be unrealistic given the inherent randomness. This example shows how the model fits the data (captures the structure, purple line) and also depicts the variability around the fitted line. I could use any other model instead of ETS, and while the fit might differ, the key takeaway would remain: we’re extrapolating the main patterns based on the data we have. We can’t and shouldn’t try to forecast the noise – it’s unpredictable by definition.

This also means that point forecasts alone aren’t very useful for decision making. While they give you an idea of what to expect, they don’t fully capture the uncertainty around that expectation. To address this, we can generate quantiles, allowing us to say, for example, that in 99% of cases, demand is expected to lie in some region. Depending on your needs, you might want an upper quantile (e.g., for setting safety stock based on a 99% service level), some specific quantile (e.g., to determine the minimum electricity generation to satisfy x% demand), or upper and lower ones to create a prediction interval (e.g., for strategic decisions).

Having said that, you might come across rare cases where there’s no variability in the data, and you can produce a 100% accurate forecast. These situations are unusual and should be considered exceptions. In most cases, your point forecast will differ from the actual value, and that’s completely fine.

So, no, not all forecasts are wrong. Many are accurate in their own way – we just need to understand what they’re telling us.

P.S. There are cases, when forecasts are very wrong. The image with Godzilla in this post tries to depict that: a company forecasts that everything will go up, but there is an uncertainty outside of the window…

Leave a Reply