Back in 1972, John Croston, who worked on stock control systems relying on Simple Exponential Smoothing (SES), noticed that SES forecasts were biased exactly when ordering decisions were made: right after a demand occurred. The following image shows why.

Intermittent demand consists mostly of periods with no sales, and SES treats each of them as a sale of size zero rather than as the absence of a sale. So, its forecast goes down with every zero, as if people wanted the product less and less. Then a customer buys 5 units, and SES immediately becomes optimistic. But in intermittent demand, sales rarely happen in consecutive periods, so this optimism leads to wrong ordering decisions.

The problem is not SES itself, but that it does not distinguish demand occurrence from demand sizes. Croston split the series into two parts (see the main image of the post): demand intervals (the number of periods between non-zero demands) and demand sizes (the non-zero values). He applied SES to each of them separately, and defined the final forecast as the ratio of the smoothed sizes to the smoothed intervals, i.e. the expected demand per period. Both parts are updated only when a demand happens, so the method does not change its mind with every zero, giving more stable forecasts and more reasonable ordering decisions.

Nowadays, we have many great forecasting approaches, from simple ones like TSB to more advanced ones like LightGBM. Croston’s method itself turned out to be biased (Syntetos & Boylan proposed a correction for that). Still, it works fine for manufacturers of large expensive items, with low volumes and long gaps between orders, where a stable forecast matters more than a quick reaction. I am not a big fan of the method, but I really like Croston’s original idea: split intermittent demand into demand occurrence and demand sizes. More than half a century later, this idea still works. My iETS paper used it, outperforming other simple forecasting methods on the M5 data. Türkmen et al. (2021) used the same idea with deep neural networks. The paper with Anna Sroginis showed that it also works with LightGBM, leading to substantial accuracy improvements that translate into inventory savings. So, even if you dislike Croston’s method, I bet you can improve your approach by using his original idea.

We will cover intermittent demand and the methods built on Croston’s idea in our Demand Forecasting Principles course in November.

Message The main idea behind the Croston’s method first appeared on OpenForecast.

First, why would we want non-equal weights in the first place? Demand evolves over time, and recent sales might reflect its current level better than the sales we had several years ago. So, it is only natural to give higher weights to recent observations and lower ones to older ones. Can we do that in a simple and principled way? Yes, and there is a forecasting method that does exactly that!

I’m talking about Simple Exponential Smoothing (SES), proposed by Robert Goodell Brown back in 1956 and independently by Charles Holt in 1957. It is yet another simple forecasting method, suitable for data without trend, seasonality or features. Mathematically, it is written like this:

\begin{equation}
F_{t+1} = \alpha A_{t} + (1−\alpha) F_{t}
\end{equation}

where \(F_t\) is the one-step-ahead forecast, \(A_t\) is the actual value, \(t\) is the time index, and \( \alpha \) is the smoothing parameter. Roughly speaking, \( \alpha \) defines how the weight is split between the most recent actual value and the previous forecast. If \( \alpha=0 \), the forecast ignores all new information and stays at its starting value, which could be, for example, the global average. With \( \alpha=1 \), it ignores all previous forecasts and becomes Naïve. The nice thing about SES is that \( \alpha \) also regulates how the weights are distributed over time, because the method can be rewritten as (see derivations here):

\begin{equation}
F_{t+1} = \alpha A_{t} + \alpha (1−\alpha) A_{t−1} + \alpha (1−\alpha)^2 A_{t−2} + \alpha (1−\alpha)^3 A_{t−3} + …
\end{equation}

Take \(\alpha=0.5\). The most recent observation gets the weight of 0.5, the one before it 0.5 × 0.5 = 0.25, the one before that 0.5 × 0.5² = 0.125, and so on. The weights decay exponentially. So, instead of 12 parameters, we define just one, and it determines how fast the weights decay. The chart in this post compares the equal weights of SMA(12) with SES for \( \alpha=0.2 \) and \( \alpha=0.5 \): the closer α is to zero, the more evenly the weights are spread; the closer it is to one, the more weight goes to the most recent observation, and the faster the older ones are forgotten.

There are tons of papers discussing SES, its modifications and extensions. Gardner (1985 and 2006) is the best review on the topic. And finally, SES is a predecessor of ETS, and its mechanism is used in TBATS.

How do you choose α in practice, and what do you do when SES is not enough? This is what we discuss in our Demand Forecasting Principles course.

Message Simple Exponential Smoothing: one parameter instead of many weights first appeared on OpenForecast.

One of the widely used approaches was developed by Hyndman & Khandakar (2008). It is implemented in auto.arima in R (and in StatsForecast for Python). It mixes two frameworks: stationarity tests (KPSS or ADF) to select the order of differencing d, and then information criteria for p and q. The tests are needed because, in the conventional formulation, models with different d are estimated on different sample sizes, so their information criteria cannot be compared (I explained this here). Although the algorithm works well and showed good performance in several competitions, it mixes hypothesis testing with information theory. The former tests the differences between samples, but assumes that you can make a mistake (based on your significance level), while the latter simply selects the most plausible model given data.

AutoMSARIMA does the selection differently. Because it is formulated in state space, the sample stays the same for any d, so the algorithm can compare all orders directly via information criteria, including the differencing. This follows the approach of Svetunkov & Boylan (2020), which treats order selection as a component selection problem rather than hypothesis testing. No ADF test, no KPSS test, no pre-screening for stationarity. The IC decides everything. And because nothing in this logic is tied to a specific seasonal period, it extends naturally to multiple seasonalities: each seasonal lag simply adds its own components to the model, and the same criterion decides which of them to keep, with the seasonality remaining stochastic rather than fixed.

The search itself is stepwise: the algorithm inspects the ACF/PACF of the residuals to propose a candidate order, adds it, and keeps it only if the information criterion decreases. When done, it also tries several simple SARIMA models to check whether any of them beats the selected one. The algorithm is explained in detail here.

In Python:

from fcompdata import AirPassengers
from smooth import AutoMSARIMA

# Monthly data, automatic order selection up to SARIMA(3,2,3)(3,1,3)[12]
model = AutoMSARIMA(lags=[1, 12])
model.fit(AirPassengers.y)
print(model)

The same algorithm works for the multiple seasonal ARIMA. The only thing to specify explicitly is the maximum orders to check:

from fcompdata import taylor

model = AutoMSARIMA(
lags=[1, 48, 336],
orders={"ar": [3, 2, 2],
"i": [2, 1, 1],
"ma": [3, 2, 2]}
)
model.fit(taylor.y)

The selected model returns the same attributes and supports the same methods as ADAM.

Does it actually work? I benchmarked AutoMSARIMA against statsforecast AutoARIMA, skforecast and R’s auto.arima on 5,315 series from the M1, M3 and Tourism datasets. The honest summary: point accuracy is a tie – RMSSE differs in the third decimal place across all implementations. But smooth produces the best-calibrated prediction intervals of the pack and is 3–5 times faster than the alternatives. Here is the full notebook.

Try smooth now: pip install smooth
smooth wiki.

Message smooth for Python: Automatic order selection for MSARIMA first appeared on OpenForecast.

The idea behind the Simple Moving Average (SMA) is to take the average of the last few observations and use it as a forecast for the next several steps ahead. Very crude and very simple. In fact, it has Naive (from this post) as a special case if you take the average of one most recent observation. On the other hand, if you increase the order to include all the observations, you will end up with the Global Mean. And this simple method works quite well if you have level data, i.e. no apparent strong trend, no obvious seasonality, and no other important elements of structure.

The only thing that makes it a bit harder to use in practice is the choice of the order, i.e. the number of observations to average over. Unfortunately, there is no universal answer here. But people report that the order of 12 or 13 is fine for weekly data, although it’s not completely clear why. In the academic literature, Aris Syntetos & John Boylan (2005) found that SMA(13) performed quite well on intermittent demand data, which was unexpected given the nature of the data (lots of zeroes). And almost 10 years ago, Fotios Petropoulos and I proposed a model underlying SMA with automatic order selection. We showed that it outperforms other simple benchmarks on supply chain data.

There is also some evidence from the VN2 inventory competition by Nicolas Vandeput. The benchmark there was built around a 13-week moving average with a simple seasonal adjustment, and only 25 out of 180+ participants managed to beat it. Many sophisticated ML pipelines lost to a method that predates computers.

So, if you work, for example, in retail or in supply chain, SMA is a method to consider for your sanity-check pool of models. But don’t expect miracles from it! It is still a simple method that works for level time series. Use it as a stepping stone to find a better model that has more features.

Anyone else found SMA to be a strong contender? Leave a comment – it would be interesting to see how many of you have had the same experience.

And yes, we discuss it in our “Demand Forecasting Principles” training in more detail. The next one will be held online in November, with live sessions from 2pm to 4pm UK time. We still have a few places left, so register here.

Message The Menace of ML: Simple Moving Average first appeared on OpenForecast.

The standard practice when ETS is not enough to capture the dynamics of real demand is to either switch to ARIMA or apply it to the residuals of the model. But it is possible to join the two models in one by using the Single Source of Error (SSOE) state space framework and to estimate the parameters jointly. This reduces the bias in the parameter estimates, letting the two models work as a team rather than one cleaning up after the other.

In the SSOE form, ETS and ARIMA states are stacked one after another in a large vector. For example, an ETS(A,N,A)+AR(2) model has four states: level, seasonal, and two AR components — each updated independently but responding to the same shock. Structurally, this resembles fitting ETS and then modelling its residuals with ARIMA, but with a critical difference: the parameters are estimated jointly, removing the bias that accumulates from sequential fitting.

But this flexibility comes with constraints. Some ETS and ARIMA combinations are actually redundant — they describe the same process from two angles, producing infinitely many parameter combinations with identical fit. For example, ETS(A,N,N) and ARIMA(0,1,1) are equivalent models, and combining them creates an unidentifiable model with no stable solution. So, I have come up with a few practical guidelines, which you can read about in Section 9.4 of ADAM.

Practically speaking, this is all handled under the hood of the ADAM function in Python, so you don’t need to worry about it. Here is an example:

from fcompdata import AirPassengers
from smooth import ADAM

# ETS(A,A,N) + SARIMA(0,0,0)(1,1,1)[12]
model = ADAM(
    model="AAN",
    ar_order=[0, 1],
    i_order=[0, 1],
    ma_order=[0, 1],
    lags=[1, 12],
    h=12, holdout=True
)
model.fit(AirPassengers.y)

Note how the orders are specified: per lag, with the first element referring to lag 1 and the second to lag 12. The code above fits ETS(A,A,N) + SARIMA(0,0,0)(1,1,1)[12], delegating the seasonality to the ARIMA part. The advantage of doing this via ADAM is that the result can be compared with pure ETS or ARIMA directly using information criteria. And if you don’t want to choose the orders yourself, AutoADAM will select them for you.

But do you even need the full combination? Gardner (1985) and Taylor (2010) argued that adding AR(1) tends to improve the accuracy of ETS. So, maybe you can get away with the simple ETS+AR(1) model, which is easy to apply using ADAM.

When to use ETS vs ARIMA.
How to combine them.

Try it yourself: pip install smooth

Message smooth for Python: ETS+ARIMA first appeared on OpenForecast.

One of the definitions I see on the internet: “A measure of the degree to which something may be forecast with accuracy”. This definition is so disturbing that I cannot resist ranting about it. The thing that annoys me the most is the “with accuracy”, which is really arbitrary and can mean anything. Your Naive method produces a relative RMSE of 1.0. Is that accurate enough? Does this make the time series forecastable? What if ETS produces 1.2, while LightGBM delivers 0.8? Is the data forecastable now? And when do you say “this is fine”?

But let me breathe and take a step back. The thing is, any method of measuring forecastability has assumptions behind it. Here is my list of five (did I miss any?):

1. The models you use: each time series has its own characteristics, so you can only say that the data is “forecastable” given the set of models you have. The data with trend might look unforecastable for the model that only has the level component.
2. The features/indicators that are available: if you do not know when promotions happen, some observations will look unforecastable. It’s a similar argument to (1), but more about what you include in your model. If you spend more time on feature generation and transformation, then the series that looked very hard might become quite easily forecastable.
3. The statistic you focus on: this could be the conditional mean, the median, some quantile, or the whole predictive distribution. A model can do great in terms of point forecasts but very poorly when it comes to prediction intervals. Does that make the time series unforecastable? Which one do you care more about?
4. The forecast horizon: the longer it is, the less accurate the forecasts become. One-step-ahead forecasts are easy, multiple steps ahead are hard. Which one do you use to measure forecastability?
5. The error measures: I’ll just leave this here: https://openforecast.org/category/forecasting-theory/forecast-evaluation/

Do I hear someone mentioning the coefficient of variation (CoV) as a good measure of forecastability? This is, by the way, what is used in the conventional XYZ classification. Well, let’s check the five points above against it and see what it assumes: (1) the global mean is suitable and the variance is constant; (2) no features are available; (3) conditional mean as the statistic of interest; (4) 1-step-ahead point forecast; (5) Root Mean Squared Error (the core of the CoV). So, it is suitable for a very small set of time series. If you use it universally, you might decide that some time series with a very clear structure are not forecastable. And the same exercise works for the more sophisticated measures, entropy-based ones included: run them through the five questions and see what they silently assume.

And then we come to the final point: so what? Let’s say you split your data into the “forecastable/not” categories. What are you going to do with that? The intention behind this is legitimate — you cannot babysit ten thousand SKUs equally, so you want to know where to spend your time. But the label “unforecastable” answers the wrong question. The right question is why the series is hard to forecast: missing promotion information? Wrong model? Genuinely random demand? Each of these implies a different action. It might well be that your “unforecastable” time series just require more time and effort to become forecastable again.

We don’t teach “forecastability” in our Demand Forecasting Principles course — because, as you can see, I don’t believe in it. What we do teach is everything in the list above: models, their assumptions, features, horizons, and how to evaluate forecasts properly. See details about the next course here.

This is my personal view. Happy to hear what others have to say.

Message Five assumptions behind “forecastability” first appeared on OpenForecast.

In the conventional ARIMA, taking differences is treated as a pre-processing step, where we switch from, for example, the sales of a product to the sales increase/decrease. If you had a sample of 100 observations, after first differences you are left with 99 – the first observation has nothing to be subtracted from. So yes, with the conventional approach, ARIMA(1,0,1) and ARIMA(1,1,1) are estimated on different sample sizes, and their information criteria are not directly comparable.

But! MSARIMA (and ADAM in general) is a state space model, and this pre-processing step is not needed. The differences are embedded in the model as additional components: ARIMA(0,1,1) has one, ARIMA(0,2,1) has two. The sample stays the same; what changes is the number of estimated parameters – each new component needs its initial value, and the information criterion penalises that. So the comparison stays fair: same sample, same likelihood basis, different number of parameters. I have several examples of how ARIMA is formulated in the ADAM SSOE framework.

What this implies: any ARIMA model of any orders (non-seasonal, seasonal, or multiseasonal) can be compared with other ARIMA models directly via information criteria, on the same sample of data.

This is not a new idea, by the way. I used this approach in the ARIMA algorithm I developed for the DemandWorks company (now part of Netstock) more than ten years ago, and John Boylan and I later built the State Space ARIMA on the same principle (the paper).

So, yes: if you use the conventional ARIMA (auto_arima from pmdarima, ARIMA from aeon or statsforecast in Python; stats or forecast implementations in R), you NEED to make sure that you compare models with the same order of differencing. But if you use the smooth implementation, you don’t need to worry – everything is already taken care of for you. This comes as a bonus from the specific formulation of the model.

Message On differencing of ARIMA in state space first appeared on OpenForecast.

The conventional implementations (statsmodels or pmdarima in Python, stats in R) do their job well: they estimate SARIMA via likelihood, select orders automatically, and produce sensible forecasts for series with trend and one seasonality. Monthly or quarterly demand data is their home ground. The problems start when the data has more than one cycle: the classical SARIMA formulation has a slot for exactly one seasonal lag, so for the half-hourly electricity demand you would have to pick either the hour-of-day or the day-of-week cycle and discard the other.

MSARIMA (Multiple Seasonal ARIMA) solves this by reformulating ARIMA in the Single Source of Error (SSOE) state space form. Each AR, I, and MA element becomes a state in the model, so nothing restricts you to one seasonal lag: you can have as many as you want. In the smooth package, orders are specified per lag, matched to a lags list. Here is an example on the classical taylor series (half-hourly electricity demand in England and Wales), with two seasonal cycles:

from fcompdata import taylor
from smooth import MSARIMA

# MSARIMA(3,0,1)(0,1,1)[48](0,1,1)[336] model = MSARIMA(
orders={"ar": [3, 0, 0], "i": [0, 1, 1], "ma": [1, 1, 1]},
lags=[1, 48, 336],
h=336, holdout=True
)
model.fit(taylor.y)
model.predict(h=336, interval="prediction", level=0.95)

In Python, it takes only 1.5 seconds for the function to fit the model to the data and estimate its parameters, which is fast for a double-seasonal model fit to roughly four thousand observations:

But the smooth implementation brings several additional practical benefits.

First, model.fit() accepts a matrix of external regressors via xreg, estimated jointly with the ARIMA components – no separate pre-filtering step. The logic and the code are identical to ETSX, which I covered earlier.

Second, you can swap the loss: loss=”GTMSE” or “MSEh” optimises the model directly on multistep errors, with the same shrinkage mechanism I discussed in the post on multistep losses.

Third, you don’t need to stick with the Gaussian distribution – you can choose other ones if you think that, for example, Laplace is more suitable for the data (this post).

And there is one more thing. Because MSARIMA now lives in the same state space framework as ETS, the two models can be combined into one and compared with each other directly via information criteria. But that deserves a post of its own, so stay tuned.

Read Chapter 9 on ADAM ARIMA.
Or check out the documentation in smooth wiki.

An why not try it yourself? pip install smooth

Message smooth in python: Multiple Seasonal ARIMA first appeared on OpenForecast.

Eight live sessions over four weeks, every Tuesday and Thursday from 3pm to 5pm UK time, starting 3 November. Nikos Kourentzes and I will deliver it together.

The course is built around one idea: you cannot defend a forecast you do not understand. So we teach how the models actually work, starting from the time series components, moving to simple methods, exponential smoothing, the ETS framework, intermittent demand, forecast evaluation, judgemental adjustments, and finishing with combinations. We focus on the understanding instead of how to call specific function, so that you can use that knowledge in any support system you work with (SAP, SAS, Excel etc). Workshops between sessions come in both R and Python, and we review them together at the start of the next session.

The training is for demand planners, forecasting analysts, data scientists moving into demand forecasting, and supply chain professionals who need forecasts they can trust and challenge. No prior forecasting or statistics knowledge is assumed.

£750 per person, £600 each for two or more, £500 for students.

Details and booking can be found here.

Message Demand Forecasting Principles training moved to November first appeared on OpenForecast.

It is called “Seasonal Naïve”. While the simple Naïve copies the last observed actual into the future as a forecast, the seasonal one copies the whole seasonal shape of the data and uses it as a forecast. The logic is straightforward: if you see an increase in sales every January, why not use the actual sales of January 2025 as the forecast for January 2026? Simple, easy to do, and hard to beat in some cases, especially when your demand does not have high variability.

What this method doesn’t do is filter out the noise in the data. This means that if you had a promotion-driven spike this February, Seasonal Naïve will happily copy it into next February’s forecast. So, if you have some distinct components in your time series and/or effects of explanatory variables on sales, Seasonal Naïve might not be a good choice. But it remains an essential benchmark for any seasonal data.

I actually have an anecdote related to the Seasonal Naïve. Yves Sagaert and I were working on a paper, and we decided to apply our new method to data with multiple seasonality. It worked great, better than the double seasonal exponential smoothing and ARIMA. I was really hyped and was ready to celebrate, when Yves suggested trying the Seasonal Naïve as well. It’s good that he did, because it turned out that Seasonal Naïve beat them all, including our new method, without even blinking. This was a great demonstration of a principle I had been preaching to others: if you have seasonal data, always use Seasonal Naïve as a benchmark.

In the Demand Forecasting Principles course in November, we cover simple methods like this one properly, including when to stop trusting them. Details and booking can be found here.

Message Another important Naïve method first appeared on OpenForecast.