A year has 365.25/7 ≈ 52.18 weeks on average. This is typically resolved by having 53 weeks in some years (roughly once every five to six years). If you just use a seasonal model with 52 weeks, the pattern drifts by about a day every year, and after five or six years, your Christmas peak is a whole week off.

And there is a twist. Many tools do not use ISO weeks. Some count seven-day blocks from the 1st of January (e.g. lubridate::week() in R), so the last one or two days of the year form a “week 53”. Others follow the weekdays but cut the weeks at the start of the year (e.g. WEEKNUM() in Excel by default), so the first and last weeks may contain only a few days. If weekly demand is the sum of daily sales, these short weeks show sharp drops, although nothing has happened to demand. A rough fix is to model the average daily demand per week and scale the forecast back. A better one is to aggregate daily data into ISO weeks in the first place (e.g. with lubridate::isoweek() in R or ISOWEEKNUM() in Excel), which are always full seven-day weeks.

This leaves us with the 52/53 weeks, for which there are several options, each with its own assumptions:

1. Shift the seasonal lag from 52 to 53 in years with the extra week. No additional parameters, but it assumes that week 53 behaves like week 52, which is not true if Christmas falls into it.
2. Use categorical variables for weeks of the year. This avoids that assumption, but you need to estimate the effect of week 53, which you observe once every five to six years, if at all.
3. Use fractional seasonality, e.g. TBATS (De Livera, Hyndman & Snyder, 2011: https://doi.org/10.1198/jasa.2011.tm09771), which models seasonality with harmonics and accepts the period of 52.18. This handles the drift elegantly, but you need to select the number of harmonics, and the harmonics might miss sharp peaks, like Black Friday.
4. Pre-process the data, e.g. merge week 53 with week 52. This assumes that the merged week does not matter, while it often contains the peak you care about the most.

So, which one to use? There is no universal answer. In practice, it often depends on your software and its restrictions, e.g. if it does not support categorical variables, the second option is gone. But whatever you choose, be aware of the assumptions behind it.

In the next post, I’ll tell you about the new TBATS implementation in the smooth package, which handles fractional seasonality and has some features that other implementations don’t. Stay tuned!

Message The problem with weekly seasonality first appeared on OpenForecast.

But first, why bother? Because a method just gives you a number, while a model gives you a likelihood, and with it flexibility: you can select components and features via information criteria, get predictive distributions, and extend the model however you like. So, if you derive a model underlying a method, you bring that method to another level.

A quick recap. Croston split intermittent demand into two parts: how often the product sells (demand intervals) and how much it sells when it does (demand sizes), and applied Simple Exponential Smoothing (SES) to each of them. In my previous post, I showed the simplest model for the “how often” part, which assumes that the chance of a sale does not change over time.

A more complicated statistical model was proposed by John Boylan and me in our iETS paper. It assumes that the occurrence of sales follows a Bernoulli distribution with a time-varying probability. In the “inverse-odds-ratio” version of the model, we track the expected number of periods without sales: if the gap between sales is 4 periods then the probability of a sale is 1/4. Every new zero or one is converted into an error term, and this expected number is updated via ETS(M,N,N), the model underlying SES. When the probability does not change much over time, this model can be estimated with Croston’s method, which is why it can be considered as the model underlying it.

In Python, the basic model is done like this:

from smooth import OM, ADAM

occ = OM(model="MNN", occurrence="inverse-odds-ratio")
occ.fit(y)
model = ADAM(model="MNN")
model.fit(y, occurrence=occ)
model.predict(h=10, interval="prediction")

In the paper, we also showed that this model is especially suitable for new products with demand building up. This means that the gaps between sales become shorter and shorter, i.e. the demand transforms from intermittent to regular. If you expect this to continue in the holdout period, you can use the trend model, and here how to do that in Python:

occ = OM(model="MMN", occurrence="inverse-odds-ratio")

This way, the expected gap between sales would decline in the holdout period, and the probability of a sale would grow. The image in this post shows how this works on an artificial example.

And if you know what drives sales (e.g. promotions or price changes), you can include this information in the occurrence model as well.

So, yes, there is a model underlying Croston’s method, and it allows doing things that the method itself cannot do.

Try smooth for Python: pip install smooth
Occurrence model on wiki

Message smooth in Python: iETS and the Croston’s Method first appeared on OpenForecast.

This idea goes back to Croston, who suggested splitting intermittent demand into demand occurrence and demand sizes. The main image in the post shows a typical intermittent demand series, and the one below shows its occurrence part: “1” when someone bought the product, and “0” otherwise.

We now have a series that can be modelled on its own, and there are many ways to do that, which come down to your preferences and experience: ETS, ARIMA, logistic regression, GBM, Neural Network – you name it!

One thing to avoid, though, is trying to predict the exact periods when sales will happen: when demand is truly intermittent, you can’t. What you can forecast is the probability. I had a post on that topic.

In the smooth package, there is a separate function for this, OM, which allows using ETS, ARIMA and regression elements to model demand occurrence. It supports several types of occurrence, including the odds-ratio, the inverse-odds-ratio (related to Croston’s method) and the direct one (similar to TSB). But the simplest of all is the fixed probability, which assumes that the probability of sale does not change over time. It simply takes the share of periods with sales: if the product sold in 30 out of 100 weeks, the probability is 0.3. If you read my previous post, you might notice that this is essentially 1/ADI turned into a forecast, with the same limitation of ignoring how demand evolves. Still, it is a good benchmark and a starting point for intermittent demand forecasting.

In Python, this is done like this:

from smooth import OM, ADAM

model = OM(occurrence="fixed")
model.fit(y)
fc = model.predict(h=10)

The resulting probability forecast is shown in the following image:

We can also use this directly in ADAM, which splits the demand into two parts on its own. Here is the model with ETS(M,N,N) for demand sizes and the fixed probability of occurrence, iETS(M,N,N)_F:

model = ADAM(model="MNN", occurrence="fixed")
model.fit(y)
model.predict(h=10)

The last command takes the probability from the occurrence model and multiplies it by the forecast of demand sizes. The result is the mean demand per period, just like from any other forecasting method.

Next time, I’ll tell you about the more advanced occurrence models, where the probability changes over time. So, stay tuned!

Try smooth for Python: pip install smooth
Occurrence model on wiki.

Message smooth in Python: fixed probability model for demand occurrence first appeared on OpenForecast.

I don’t know when specifically ADI was proposed, but it was popularised by Syntetos, Boylan & Croston (2005), yes, the same Croston from this post. A demand interval is the number of periods between consecutive non-zero demands, and ADI is just their mean. It lies between one (demand in every period) and, theoretically, infinity (no demand at all). But why would anyone need it?

Originally, it was used together with the coefficient of variation \(CV^2 \) to classify demand into four types (the SBC classification) and to choose between Croston’s method and SBA. The authors compared the theoretical MSE of the methods and found cut-off points: Croston does better only when \(ADI < 1.32\) and \(CV^2 < 0.49\). Otherwise, SBA wins.

My interpretation of the \(ADI < 1.32\) (this is not in the paper): SBA fixes the bias of Croston’s method, which matters most for very slow-moving products. But it does that at a cost. SBA multiplies Croston’s forecast by \( (1 − \alpha /2) \), which just lowers probability of occurrence. If demand becomes less sporadic, SBA underestimates it (e.g. 0.85 instead of 0.9 with \(\alpha = 0.1\)).

But why ADI in the first place? Because historically the main academic work on intermittent demand was carried out in spare parts and similar domains, where sales are rare even at the monthly level (the 2005 paper used 3000 series from the automotive industry). When the probability of occurrence is low, demand does not evolve much over time, so averaging the demand intervals is acceptable.

Nowadays, we face intermittent demand everywhere: in retail, energy, healthcare and other areas. The data is often collected at a more granular level (weekly or daily), and intermittent demand comes in more complex shapes than in spare parts. Averaging the intervals then hides important information. For example, if your product becomes obsolete halfway through the sample and the probability of occurrence drops from 0.5 to 0.1, ADI will be around 3.3. That corresponds to a probability of 0.3: neither the 0.5 you had nor the 0.1 you have now. It is like judging a hospital by the average temperature of all its patients.

So, ADI is a fine measure for its original purpose. But time has passed, use cases have changed, and nowadays instead of averaging the intervals, I recommend looking at how the probability of occurrence evolves over time, for example, by smoothing it (see the image in the post) or with a model that has a time-varying probability (like iETS). A plot of demand occurrence over time tells you more than ADI ever will.

We have a separate session on intermittent demand in our Demand Forecasting Principles course, where we cover this and other topics in detail.

Message What the Average Demand Interval (ADI) hides first appeared on OpenForecast.

Back in 1972, John Croston, who worked on stock control systems relying on Simple Exponential Smoothing (SES), noticed that SES forecasts were biased exactly when ordering decisions were made: right after a demand occurred. The following image shows why.

Intermittent demand consists mostly of periods with no sales, and SES treats each of them as a sale of size zero rather than as the absence of a sale. So, its forecast goes down with every zero, as if people wanted the product less and less. Then a customer buys 5 units, and SES immediately becomes optimistic. But in intermittent demand, sales rarely happen in consecutive periods, so this optimism leads to wrong ordering decisions.

The problem is not SES itself, but that it does not distinguish demand occurrence from demand sizes. Croston split the series into two parts (see the main image of the post): demand intervals (the number of periods between non-zero demands) and demand sizes (the non-zero values). He applied SES to each of them separately, and defined the final forecast as the ratio of the smoothed sizes to the smoothed intervals, i.e. the expected demand per period. Both parts are updated only when a demand happens, so the method does not change its mind with every zero, giving more stable forecasts and more reasonable ordering decisions.

Nowadays, we have many great forecasting approaches, from simple ones like TSB to more advanced ones like LightGBM. Croston’s method itself turned out to be biased (Syntetos & Boylan proposed a correction for that). Still, it works fine for manufacturers of large expensive items, with low volumes and long gaps between orders, where a stable forecast matters more than a quick reaction. I am not a big fan of the method, but I really like Croston’s original idea: split intermittent demand into demand occurrence and demand sizes. More than half a century later, this idea still works. My iETS paper used it, outperforming other simple forecasting methods on the M5 data. Türkmen et al. (2021) used the same idea with deep neural networks. The paper with Anna Sroginis showed that it also works with LightGBM, leading to substantial accuracy improvements that translate into inventory savings. So, even if you dislike Croston’s method, I bet you can improve your approach by using his original idea.

We will cover intermittent demand and the methods built on Croston’s idea in our Demand Forecasting Principles course in November.

Message The main idea behind the Croston’s method first appeared on OpenForecast.

First, why would we want non-equal weights in the first place? Demand evolves over time, and recent sales might reflect its current level better than the sales we had several years ago. So, it is only natural to give higher weights to recent observations and lower ones to older ones. Can we do that in a simple and principled way? Yes, and there is a forecasting method that does exactly that!

I’m talking about Simple Exponential Smoothing (SES), proposed by Robert Goodell Brown back in 1956 and independently by Charles Holt in 1957. It is yet another simple forecasting method, suitable for data without trend, seasonality or features. Mathematically, it is written like this:

\begin{equation}
F_{t+1} = \alpha A_{t} + (1−\alpha) F_{t}
\end{equation}

where \(F_t\) is the one-step-ahead forecast, \(A_t\) is the actual value, \(t\) is the time index, and \( \alpha \) is the smoothing parameter. Roughly speaking, \( \alpha \) defines how the weight is split between the most recent actual value and the previous forecast. If \( \alpha=0 \), the forecast ignores all new information and stays at its starting value, which could be, for example, the global average. With \( \alpha=1 \), it ignores all previous forecasts and becomes Naïve. The nice thing about SES is that \( \alpha \) also regulates how the weights are distributed over time, because the method can be rewritten as (see derivations here):

\begin{equation}
F_{t+1} = \alpha A_{t} + \alpha (1−\alpha) A_{t−1} + \alpha (1−\alpha)^2 A_{t−2} + \alpha (1−\alpha)^3 A_{t−3} + …
\end{equation}

Take \(\alpha=0.5\). The most recent observation gets the weight of 0.5, the one before it 0.5 × 0.5 = 0.25, the one before that 0.5 × 0.5² = 0.125, and so on. The weights decay exponentially. So, instead of 12 parameters, we define just one, and it determines how fast the weights decay. The chart in this post compares the equal weights of SMA(12) with SES for \( \alpha=0.2 \) and \( \alpha=0.5 \): the closer α is to zero, the more evenly the weights are spread; the closer it is to one, the more weight goes to the most recent observation, and the faster the older ones are forgotten.

There are tons of papers discussing SES, its modifications and extensions. Gardner (1985 and 2006) is the best review on the topic. And finally, SES is a predecessor of ETS, and its mechanism is used in TBATS.

How do you choose α in practice, and what do you do when SES is not enough? This is what we discuss in our Demand Forecasting Principles course.

Message Simple Exponential Smoothing: one parameter instead of many weights first appeared on OpenForecast.

The idea behind the Simple Moving Average (SMA) is to take the average of the last few observations and use it as a forecast for the next several steps ahead. Very crude and very simple. In fact, it has Naive (from this post) as a special case if you take the average of one most recent observation. On the other hand, if you increase the order to include all the observations, you will end up with the Global Mean. And this simple method works quite well if you have level data, i.e. no apparent strong trend, no obvious seasonality, and no other important elements of structure.

The only thing that makes it a bit harder to use in practice is the choice of the order, i.e. the number of observations to average over. Unfortunately, there is no universal answer here. But people report that the order of 12 or 13 is fine for weekly data, although it’s not completely clear why. In the academic literature, Aris Syntetos & John Boylan (2005) found that SMA(13) performed quite well on intermittent demand data, which was unexpected given the nature of the data (lots of zeroes). And almost 10 years ago, Fotios Petropoulos and I proposed a model underlying SMA with automatic order selection. We showed that it outperforms other simple benchmarks on supply chain data.

There is also some evidence from the VN2 inventory competition by Nicolas Vandeput. The benchmark there was built around a 13-week moving average with a simple seasonal adjustment, and only 25 out of 180+ participants managed to beat it. Many sophisticated ML pipelines lost to a method that predates computers.

So, if you work, for example, in retail or in supply chain, SMA is a method to consider for your sanity-check pool of models. But don’t expect miracles from it! It is still a simple method that works for level time series. Use it as a stepping stone to find a better model that has more features.

Anyone else found SMA to be a strong contender? Leave a comment – it would be interesting to see how many of you have had the same experience.

And yes, we discuss it in our “Demand Forecasting Principles” training in more detail. The next one will be held online in November, with live sessions from 2pm to 4pm UK time. We still have a few places left, so register here.

Message The Menace of ML: Simple Moving Average first appeared on OpenForecast.

One of the definitions I see on the internet: “A measure of the degree to which something may be forecast with accuracy”. This definition is so disturbing that I cannot resist ranting about it. The thing that annoys me the most is the “with accuracy”, which is really arbitrary and can mean anything. Your Naive method produces a relative RMSE of 1.0. Is that accurate enough? Does this make the time series forecastable? What if ETS produces 1.2, while LightGBM delivers 0.8? Is the data forecastable now? And when do you say “this is fine”?

But let me breathe and take a step back. The thing is, any method of measuring forecastability has assumptions behind it. Here is my list of five (did I miss any?):

1. The models you use: each time series has its own characteristics, so you can only say that the data is “forecastable” given the set of models you have. The data with trend might look unforecastable for the model that only has the level component.
2. The features/indicators that are available: if you do not know when promotions happen, some observations will look unforecastable. It’s a similar argument to (1), but more about what you include in your model. If you spend more time on feature generation and transformation, then the series that looked very hard might become quite easily forecastable.
3. The statistic you focus on: this could be the conditional mean, the median, some quantile, or the whole predictive distribution. A model can do great in terms of point forecasts but very poorly when it comes to prediction intervals. Does that make the time series unforecastable? Which one do you care more about?
4. The forecast horizon: the longer it is, the less accurate the forecasts become. One-step-ahead forecasts are easy, multiple steps ahead are hard. Which one do you use to measure forecastability?
5. The error measures: I’ll just leave this here: https://openforecast.org/category/forecasting-theory/forecast-evaluation/

Do I hear someone mentioning the coefficient of variation (CoV) as a good measure of forecastability? This is, by the way, what is used in the conventional XYZ classification. Well, let’s check the five points above against it and see what it assumes: (1) the global mean is suitable and the variance is constant; (2) no features are available; (3) conditional mean as the statistic of interest; (4) 1-step-ahead point forecast; (5) Root Mean Squared Error (the core of the CoV). So, it is suitable for a very small set of time series. If you use it universally, you might decide that some time series with a very clear structure are not forecastable. And the same exercise works for the more sophisticated measures, entropy-based ones included: run them through the five questions and see what they silently assume.

And then we come to the final point: so what? Let’s say you split your data into the “forecastable/not” categories. What are you going to do with that? The intention behind this is legitimate — you cannot babysit ten thousand SKUs equally, so you want to know where to spend your time. But the label “unforecastable” answers the wrong question. The right question is why the series is hard to forecast: missing promotion information? Wrong model? Genuinely random demand? Each of these implies a different action. It might well be that your “unforecastable” time series just require more time and effort to become forecastable again.

We don’t teach “forecastability” in our Demand Forecasting Principles course — because, as you can see, I don’t believe in it. What we do teach is everything in the list above: models, their assumptions, features, horizons, and how to evaluate forecasts properly. See details about the next course here.

This is my personal view. Happy to hear what others have to say.

Message Five assumptions behind “forecastability” first appeared on OpenForecast.

The conventional implementations (statsmodels or pmdarima in Python, stats in R) do their job well: they estimate SARIMA via likelihood, select orders automatically, and produce sensible forecasts for series with trend and one seasonality. Monthly or quarterly demand data is their home ground. The problems start when the data has more than one cycle: the classical SARIMA formulation has a slot for exactly one seasonal lag, so for the half-hourly electricity demand you would have to pick either the hour-of-day or the day-of-week cycle and discard the other.

MSARIMA (Multiple Seasonal ARIMA) solves this by reformulating ARIMA in the Single Source of Error (SSOE) state space form. Each AR, I, and MA element becomes a state in the model, so nothing restricts you to one seasonal lag: you can have as many as you want. In the smooth package, orders are specified per lag, matched to a lags list. Here is an example on the classical taylor series (half-hourly electricity demand in England and Wales), with two seasonal cycles:

from fcompdata import taylor
from smooth import MSARIMA

# MSARIMA(3,0,1)(0,1,1)[48](0,1,1)[336] model = MSARIMA(
orders={"ar": [3, 0, 0], "i": [0, 1, 1], "ma": [1, 1, 1]},
lags=[1, 48, 336],
h=336, holdout=True
)
model.fit(taylor.y)
model.predict(h=336, interval="prediction", level=0.95)

In Python, it takes only 1.5 seconds for the function to fit the model to the data and estimate its parameters, which is fast for a double-seasonal model fit to roughly four thousand observations:

But the smooth implementation brings several additional practical benefits.

First, model.fit() accepts a matrix of external regressors via xreg, estimated jointly with the ARIMA components – no separate pre-filtering step. The logic and the code are identical to ETSX, which I covered earlier.

Second, you can swap the loss: loss=”GTMSE” or “MSEh” optimises the model directly on multistep errors, with the same shrinkage mechanism I discussed in the post on multistep losses.

Third, you don’t need to stick with the Gaussian distribution – you can choose other ones if you think that, for example, Laplace is more suitable for the data (this post).

And there is one more thing. Because MSARIMA now lives in the same state space framework as ETS, the two models can be combined into one and compared with each other directly via information criteria. But that deserves a post of its own, so stay tuned.

Read Chapter 9 on ADAM ARIMA.
Or check out the documentation in smooth wiki.

An why not try it yourself? pip install smooth

Message smooth in python: Multiple Seasonal ARIMA first appeared on OpenForecast.

It is called “Seasonal Naïve”. While the simple Naïve copies the last observed actual into the future as a forecast, the seasonal one copies the whole seasonal shape of the data and uses it as a forecast. The logic is straightforward: if you see an increase in sales every January, why not use the actual sales of January 2025 as the forecast for January 2026? Simple, easy to do, and hard to beat in some cases, especially when your demand does not have high variability.

What this method doesn’t do is filter out the noise in the data. This means that if you had a promotion-driven spike this February, Seasonal Naïve will happily copy it into next February’s forecast. So, if you have some distinct components in your time series and/or effects of explanatory variables on sales, Seasonal Naïve might not be a good choice. But it remains an essential benchmark for any seasonal data.

I actually have an anecdote related to the Seasonal Naïve. Yves Sagaert and I were working on a paper, and we decided to apply our new method to data with multiple seasonality. It worked great, better than the double seasonal exponential smoothing and ARIMA. I was really hyped and was ready to celebrate, when Yves suggested trying the Seasonal Naïve as well. It’s good that he did, because it turned out that Seasonal Naïve beat them all, including our new method, without even blinking. This was a great demonstration of a principle I had been preaching to others: if you have seasonal data, always use Seasonal Naïve as a benchmark.

In the Demand Forecasting Principles course in November, we cover simple methods like this one properly, including when to stop trusting them. Details and booking can be found here.

Message Another important Naïve method first appeared on OpenForecast.