Chapter 13 Intermittent State Space Model
So far, we have discussed data that has regular occurrence (i.e. happening every observation). For example, daily sales of bread in a supermarket would have this regularity. However, there are time series where non-zero values do not happen on every observation. In the context of demand forecasting, this is called “intermittent demand”. The conventional example of such demand is monthly sales of jet engines: they will contain a lot of zeroes, when nobody buys the product and then all of a sudden several units when an aeroplane company decides to buy it, again followed by zeroes. However, this problem does not only apply to such expensive exotic products – the retailers face this all the time for various products, mainly when sales are recorded daily or on an even higher frequency.
One of the most straightforward definitions of intermittent demand is that it is the demand that happens at irregular frequency. This means that we cannot tell with 100% certainty, when specifically customers will buy our product. In fact, if we zoom in on the sales of any product and observe it at a higher frequency, we will most probably see intermittent demand. For example, if we work with hourly sales of products in a supermarket, we will most probably observe intermittent demand because there will be some hours of day, when supermarket will have few customers and thus will not sell some products at all. However, if we aggregate this data to a weekly level, the intermittence will typically disappear, making the demand look regular. So, intermittent demand is related to the regular one, and in many cases arises on the lower aggregation levels.
You might also meet the term “count data” (or “integer-valued data”) in a similar context, but there is a fundamental difference between the count and intermittent data. The former implies that demand can take integer values only and can be typically modelled via Poisson, Binomial, or Negative Binomial distributions. It does not necessarily contain zeroes and does not explicitly allow demand to happen at random. In this case, if there are zeroes, then it is assumed that they are just one of the possible values of a distribution. On the other hand, in the case of intermittent demand, we explicitly acknowledge that demand might not happen, but if it happens, then the value will be greater than zero. Furthermore, intermittent demand does not necessarily need to be integer-valued. For example, daily energy consumption for charging electric vehicles would typically be intermittent (because the vehicle owners do not charge them every day), but the non-zero consumption will not be an integer. Still, count distributions can be used in some cases of intermittent demand, but they do not necessarily always provide a good approximation of complex reality.
Before we move towards the proper discussion of the topic in the context of ADAM, we should acknowledge that at the heart of what follows, there lies the following model (Croston, 1972): \[\begin{equation} y_t = o_t z_t , \tag{13.1} \end{equation}\] where \(o_t\) is the demand occurrence variable, which can be either zero or one and has some probability of occurrence \(p_t\), \(z_t\) is the demand sizes captured by a model (for example, ETS), and \(y_t\) is the final observed demand. In the context of intermittent demand, this model was originally proposed by Croston (1972), but similar models (e.g. Hurdle and Zero Inflated Poisson) exist in other, non-forecasting related literature.
In this chapter, we will discuss the intermittent state space model (13.1), both parts of which can be modelled via ADAM, and we will see how they can be used, what they imply and how they connect to the conventional regular demand. If an ETS model is used for \(z_t\) then (13.1) will be called iETS. So, the iETS(M,N,N) model refers to the intermittent state space model, where demand sizes are modelled via ETS(M,N,N). ETS can also be used for the occurrence part of the model, so if the discussion is focused on the demand occurrence part of the model, \(o_t\) (as in Section 13.1), we will use “oETS” instead. Similarly, we can use the terms iARIMA and oARIMA, referring either to the whole model or just to its occurrence part. Note, however, that while ARIMA can be used in theory, it is not yet implemented for the occurrence part of the model. So we will focus the discussion in this chapter on the ADAM ETS. Furthermore, depending on how the occurrence part is modelled, the notations above can be expanded to include references to specific parts of the occurrence part of the model. This is discussed in detail in Section 13.1.
This chapter is based on Svetunkov and Boylan (2023a). If you want to know more about intermittent demand forecasting, Boylan and Syntetos (2021) is an excellent textbook on the topic, covering all the main aspects in appropriate detail.