17.3 Other aspects of forecast uncertainty
There are other elements related to forecasting and taking uncertainty into account that we have not discussed in the previous sections. They are related to the previous sections but did not align with them well to be included there. so, we discuss them in this section.
17.3.1 Constructing intervals for intermittent demand model
When it comes to constructing prediction intervals for intermittent state space model (see Section 13), then there is an important aspect that should be taken into account. Given that the model consists of two parts: demand sizes and demand occurrence - the prediction interval should take the uncertainty from both into account. In this case we should first predict what the probability of occurrence of demand will be for the h steps ahead and then based on this probability decide, what the width of the interval should be. For example, if we estimate that the demand will occur with probability \(\hat{p}_{t+h|t} = 0.8\), then this means that we expect that in 20% of the cases we will observe zeroes. This should reduce the confidence level for the demand sizes. Formally speaking, this comes to the following equation: \[\begin{equation} F_y(y_{t+h} \leq q) = \hat{p}_{t+h|t} F_z(z_{t+h} \leq q) +(1 -\hat{p}_{t+h|t}), \tag{17.7} \end{equation}\] where \(F_y(\cdot)\) is the cumulative distribution function of demand, \(F_z(\cdot)\) is the cumulative distribution function of the demand sizes, \(\hat{p}_{t+h|t}\) is the h steps ahead expected probability of occurrence and \(Q\) is the quantile of the distribution. In the formula (17.7), we know the expected probability and we know the confidence level \(F_y(y_{t+h} \leq q)\). The unknown element is the \(F_z(z_{t+h} \leq q)\). So after regrouping elements we get: \[\begin{equation} F_z(z_{t+h} \leq q) = \frac{F_y(y_{t+h} \leq q) -(1 -\hat{p}_{t+h|t})}{\hat{p}_{t+h|t}}. \tag{17.8} \end{equation}\] For example, if the confidence level is 0.95 and the expected probability of occurrence is 0.8, then \(F_z(z_{t+h} \leq q) = \frac{0.95 - 0.2}{0.8} = 0.9375\). Assuming the distribution for the demand sizes, we can use formula (17.2) to construct the parametric prediction interval. Alternatively, we can use any other approach discussed in Section 17.2 to generate the intervals.
17.3.2 One-sided prediction interval
In some cases we might not need both bounds of the interval. For example, when we deal with intermittent demand, we know that in many cases the lower bound will be equal to zero. In fact, this will always happen, when the significance level is lower than the probability of inoccurrence \(1-p_{t+h|t}\): we will have the quantile equal to zero, because the probability of having zeroes is higher than the significance level. Another example is the safety stock calculation: we only need the upper bound of the interval and we need to make sure that the specific proportion of demand is satisfied (e.g. 95% of it). In these cases, we can just focus on the specific bound of the interval and drop the other one. Statistically speaking, this means that we cut only one tail of the assumed distribution. This has its own implications and issues in several scenarios:
- When we are interested in upper bound only and deal with positive distribution of demand (for example, Gamma, Log Normal or Inverse Gaussian), we know that the demand will always lie between zero and the constructed bound. In cases of low volume (or even intermittent) data, this makes sense, because the original data might either contain zeroes or have values close to it. The upper bound in this case will be lower than in the case of the two-sided prediction interval, because we would not be splitting the probability into two parts (for the left and the right tails);
- The combination of lower bound and positive distribution implies that the demand will be greater than the specified bound in the pre-selected number of cases (defined by confidence level). There is no natural bound from above, so from theoretical point of view this implies that the demand can be infinite;
- Upper or lower bound with real-valued distribution (such as Normal, Laplace, S or Generalised Normal) implies that the demand is either below or above the specified level respectively, without any natural limit on the other side. If Normal distribution is used on positive low volume data, then there is a natural lower bound, but the model itself will not be aware of it and will not restrict the space with the specific value.
17.3.3 Cumulative over the horizon forecast
Another related thing to consider, when producing forecasts in practice is that in some contexts the classical point forecast is not needed. Instead, the cumulative over the forecast horizon (or over the lead time) might be more suitable. The classical example is the safety stock calculation based on the lead time (time between the order of product and its delivery). In this situation we need to make sure that while the product is being delivered, we do not run out of stock, thus still satisfying the selected level of demand (e.g. 95%), but now over the whole period of time rather than on every separate observation.
In case of pure additive ADAM models, there are analytical formulae for the conditional expectations and conditional variance that can be used in forecasting. These formulae come directly from the recursive relation (5.9) Svetunkov and Petropoulos (2018): \[\begin{equation} \begin{aligned} \mu_{Y,t,h} = \text{E}(Y_{c,t,h}|t) = & \sum_{j=1}^h \sum_{i=1}^d \left(\mathbf{w}_{m_i}' \mathbf{F}_{m_i}^{\lceil\frac{j}{m_i}\rceil-1} \right) \mathbf{v}_{t} \\ \sigma^2_{Y,h} = \text{V}(Y_{c,t,h}|t) = & \left(1 + \sum_{k=1}^{h-1} \left((h-k) \sum_{i=1}^d \left(\mathbf{w}_{m_i}' \sum_{j=1}^{\lceil\frac{k}{m_i}\rceil-1} \mathbf{F}_{m_i}^{j-1} \mathbf{g}_{m_i} \mathbf{g}'_{m_i} (\mathbf{F}_{m_i}')^{j-1} \mathbf{w}_{m_i} \right) + 1 \right) \right) \sigma^2 \end{aligned}, \tag{17.9} \end{equation}\] where \(Y_{c,t,h}=\sum_{j=1}^h y_{t+j}\) is the cumulative actual value and all the other variables have been defined in Section 5.2. Based on these expectation and variance, we can construct prediction interval as discussed in Section 17.2.
In cases of multiplicative and mixed ADAM models, there are no closed forms for the conditional expectation and variance. As a result, the simulation similar to the one discussed in Section 17.1.1 is needed to produce all possible paths for the next \(h\) steps ahead. The main difference would be that before taking expectation or quantiles, the paths would need to be aggregated over the forecast horizon \(h\). This approach together with the idea of one-sided prediction interval can then be directly used for the calculation of the safety stock over the lead time.
17.3.4 Example in R
For the demonstration purposes, we consider an artificial intermittent demand example, similar to the one from Section 13.4:
y <- ts(c(rpois(20,0.25),rpois(20,0.5),rpois(20,1),rpois(20,2),rpois(20,3)))For simplicity, we apply iETS(M,Md,N) model with odds ratio occurrence:
adamModeliETS <- adam(y, "MMdN", occurrence="odds-ratio",
h=7, holdout=TRUE)
plot(adamModeliETS,7)
To make this setting closer to possible real life situation, we assume that the lead time is 7 days and we need to satisfy the 99% of demand for the last 7 observations based on our model. Thus we produce the upper bound for the cumulative values for the confidence level of 99%:
adamModeliETSForecast <- forecast(adamModeliETS, h=7,
cumulative=TRUE, interval="prediction", side="upper")Given that we deal with cumulative values, the basic plot will not be helpful, we should produce a different one (see Figure 17.8).
plot(sum(adamModeliETS$holdout), ylab="Cumulative demand",
xlab="", pch=16,
ylim=range(c(0, sum(adamModeliETS$holdout), adamModeliETSForecast$upper)))
abline(h=adamModeliETSForecast$mean, col="blue", lwd=2)
abline(h=adamModeliETSForecast$upper, col="grey", lwd=2)
Figure 17.8: Cumulative demand, expectation and the upper bound based on iETS model.
What Figure 17.8 demonstrates is that for the holdout period of 7 days, the cumulative demand was around 18 units, while the upper bound of the interval was approximately 38. Based on that upper bound we could place an order (based on what we already have in stock) and have an appropriate safety stock.
17.3.5 Confidence interval
Finally, we can construct a confidence interval for some statistics. As discussed in Section 5.2 of Svetunkov (2021c), it can be constructed for the mean, for a parameter, for fitted values etc. In our context, we might be interested in the confidence interval for the conditional h steps ahead expectation. This implies that we are interested in the uncertainty of the line, not of the actual values. This means that the confidence interval can only be constructed for the model that takes the uncertainty of parameters into account (as discussed in Section 16). The construction of confidence interval in this case relies on the normal distribution, based on Central Limit Theorem, as long as the basic assumptions for the model and CLT are satisfied (see Section 4.2 and Chapter 12 of Svetunkov (2021c)). Technically speaking, the construction of confidence interval comes to capturing the model ucnertainty discussed in Chapter 16.
17.3.5.1 Example in R
The only way how the confidence interval can be constructed for ADAM models is via the reforecast() function. Consider the example with ADAM ETS(A,Ad,N) on BJSales data as in Section 17.2.8:
adamModel <- adam(BJsales, h=10, holdout=TRUE)The confidence interval for this model can be produced either directly via reforecast() or via forecast(), which will call it for you:
adamModelConfidence <- forecast(adamModel, h=10,
interval="confidence", nsim=1000)
plot(adamModelConfidence)Note that I have increased the number of iterations for the simulation in order to get a more accurate confidence interval around the conditional expectation. This will consume more memory, as the operation involves creating 1000 sample paths for the fitted values and another 1000 for the holdout sample forecasts.
Figure 17.9: Confidence interval for the point forecast from ADAM ETS(A,Ad,N) model.
Figure 17.9 shows the uncertainty around the point forecast based on the uncertainty of parameters of the model. As can be seen, the interval is narrow, demonstrating that the conditional expectation would not change much if the sample sizes changes. The fact that the actual values are systematically above the forecast does not mean anything, because the confidence interval does not consider the uncertainty of actual values.