In many contexts, “deterministic” has a precise meaning: no randomness, no uncertainty. A deterministic solution to an optimisation problem (e.g. linear programming) implies that there are no random inputs or outputs once the model and its parameters are fixed. Forecasting is different. As Chatfield and many others have pointed out, forecasting has multiple sources of uncertainty, and there is essentially zero chance that the future will unfold exactly as any single number suggests.

Yes, some people use “deterministic” as a synonym for “point forecast”. But that label is still misleading, because a point forecast is not uncertainty-free – it is just one summary of a predictive distribution (often the conditional mean, sometimes the median or another functional).

Here’s a quick reality check you can do yourself. Take a dataset, apply your model, and write down the point forecast for the next few observations. Now add one new observation, re-estimate, and forecast again (the image in this post depicts exactly that, but with 50 forecasts produced on different subsamples of data). The point forecast will change unless you are dealing with an exotic situation with non-random data (e.g. every day, you sell exactly 100 units). So, which of the two was the “deterministic” forecast? If forecasts were truly deterministic in the strict sense, you would not get multiple plausible values from small, reasonable changes in the sample.

This happens because any forecasting method (statistical or ML) depends on data and on modelling choices: parameter estimation, feature selection, splitting rules, tuning, even decisions like “use α=0.1”. Those choices can be fixed across samples of data, but fixing them does not remove uncertainty – it only hides it. The randomness is still there in the data and in the fact that we only observe a sample of it.

So when you see someone mentioning “deterministic forecast”, it’s worth translating it mentally to: “a point forecast, probably a conditional mean”. If you care about decisions and risk, you should know that there is an uncertainty associated with this so called “deterministic forecast”, and that it should not be ignored. But this is a topic for another discussion in another post.

Message There’s no such thing as “deterministic forecast” first appeared on Open Forecasting.

JORS recently published the paper of Nikos Kourentzes and I based on a simple but powerful idea: instead of using summary statistics (like the mean RMSE of cross-validated errors), you should consider the entire distribution and choose a specific quantile. This aligns with my previous post on error measures, but here is the core intuition:

The distribution of error measures is almost always asymmetric. If you only look at the average, you end up with a “mean temperature in the hospital” statistic, which doesn’t reflect how models actually behave. Some models perform great on most series but fail miserably on a few.

What can we do in this case? We can look at quantiles of distribution.

For example, if we use 84th quantile, we compare the models based on their “bad” performance, situations where they fail and produce less accurate forecasts. If you choose the best performing model there, you will end up with something that does not fail as much. So your preferences for the model become risk-averse in this situation.

If you focus on the lower quantile (e.g. 16th), you are looking at models that do well on the well-behaved series and ignore how they do on the difficult ones. So, your model selection preferences can be described as risk-tolerant, because you are accept that the best performing model might fail on a difficult time series.

Furthermore, the median (50th quantile, the middle of sample), corresponds to the risk-neutral situation, because it ignores the tails of the distribution.

What about the mean? This is a risk-agnostic strategy, because it says nothing about the performance on the difficult or easy time series – it takes everything and nothing in it at the same time, hiding the true risk profile.

So what?

In the paper, we show that using a risk-averse strategy tends to improve overall forecasting accuracy in day-to-day situations. Conversely, a risk-tolerant strategy can be beneficial when disruptions are anticipated, as standard models are likely to fail anyway.

So, next time you select a model, think about the measure you are using. If it’s just the mean RMSE, keep in mind that you might be ignoring the inherent risks of that selection.

P.S. While the discussion above applies to the distribution of error measures, our paper specifically focused on point AIC (in-sample performance). But it is a distance measure as well, so the logic explained above holds.

P.P.S. Nikos wrote a post about this paper here.

P.P.P.S. And here is the link to the paper.

Message Risky business: how to select your model based on risk preferences first appeared on Open Forecasting.

I’ve worked with a couple of companies where there were some examples of data with drastically changing seasonal patterns over just a few years. For example, Before Covid (BC), the consumer behaviour could be different than After the Disruption (AD): people started ordering online some products that they used to buy in shops. Some practitioners told me that the seasonal patterns in their data had changed so dramatically that historical data BC had become practically useless.

Why is this important?

If you don’t recognise that the seasonal patterns can change and simply use seasonal dummy variables (for example, in regression or decision trees), you’ll run into problems, as these approaches won’t capture the evolving profile. The same applies to classical decomposition (see Section 3.2 of ADAM), since it assumes a fixed seasonal structure. In fact, any model that assumes fixed seasonality would fail in this situation, and you may not even notice (see image in the post, where the model fails to capture the profile correctly because it assumes that it is fixed).

What can we do with that?

The solution is to use approaches that allow seasonality to evolve over time. ARIMA and ETS handle this via their parameters, while STL decomposition produces a dynamic seasonal profile. In regression or decision trees, you could incorporate lagged sales to partially account for this, or bring in the seasonal component from STL/ETS as an additional feature.

All good? Not quite, because there is a nasty small grey elephant hidden in this room.

Even if your chosen method allows seasonality to evolve, you must ensure that forecasting uncertainty reflects this properly. If the seasonal pattern in your training set changed drastically, what prevents it from shifting again in the test set? This is particularly critical if you need predictive distributions, such as for setting safety stock levels or generating prediction intervals. Ignoring the fact that the seasonality might change further could make your predictive distribution narrower than expected, leading to potential lost sales.

The good news: ARIMA and ETS handle this naturally, as components’ uncertainty translates directly to the holdout variance. In ML, it is more complicated, because you would need to invest time in proper feature engineering to explicitly capture the potential seasonality changes. Unfortunately, I haven’t done much in the latter direction, so I cannot give you a good recipe. Any thoughts what to do here?

And what do you do in the situation like that?

Message Evolving seasonality first appeared on Open Forecasting.

Introduction

But first, why am I writing this post?

There’s growing interest in forecasting among data scientists, data engineers, ML experts etc. Many of them assume that they can apply their existing knowledge directly to this new area without reading domain-specific literature. As a result, we get a lot of “hit-or-miss” work: sometimes having promising ideas, but executed in ways that violate basic forecasting principles. The main problem with that is that if your experiment is not done correctly, your results might be compromised, i.e. your claims might be simply wrong.

If you’re a researcher writing forecasting-related papers, then hopefully reading this post (and the posts and papers I refer to), will help you improve your papers. This might lead to a smoother peer-review process. Also, while I can’t speak for other reviewers, if I come across a paper with similar issues, I typically give it a hard time.

I should also say that I am not a reviewer of this paper (I would not publish a review), but I merely decided to demonstrate what issues I can see when I read papers like that. The authors are just unlucky that I picked their paper…

Let’s start.

The authors apply several ML methods to retail data, compare their forecasting accuracy, and conclude that XGBoost and LightGBM outperform N-BEATS, NHITS, and Temporal Fusion Transformer. While the finding isn’t groundbreaking, additional evidence on a new dataset is always welcome.

Major issues

So, what’s wrong? Here is a list of the major comments:

1. Forecast horizon vs. data frequency:

Daily data with a 365-day forecast horizon makes no practical sense (page 2, paragraph 3). I haven’t seen any company making daily-level decisions a year in advance. Stock decisions are typically made on much shorter horizons, and if you need a year ahead forecast, you definitely do not need it on the daily level. After all, there is no point in knowing that on 22nd December 2025 you will have the expected demand of 35.457 units – it is too far into the future to make any difference. Some references:

• Athanasopoulos and Kourentzes (2023) paper discusses data frequency and some decisions related to them;
• and there is a post on my website on a related topic
2. Misuse of SBC classification:

Claiming that 70% of products are “intermittent” (page 2, last paragraph) based on SBC is incorrect. Furthermore, SBC classification does not make sense in this setting, and is not used in the paper anyway, so the authors should just drop it.

• Read more about it here.
• And there is a post of Stephan Kolassa on exactly this point
3. Product elimination and introduction is unclear (page 3):

The authors say “Around 30% of products were eliminated during training and 10% are newly introduced in validation”. It’s not clear why this was done and how specifically. This needs to be explained in more detail.

4. “Missing values” undefined:

It is not clear what the authors mean by “missing values” (page 3, “Handling Missing Values”). How do they appear and why? Are they the same as stockouts, or were there some other issues in the data? This needs to be explained in more detail.

5. Figure 1 is vague:

Figure 1 is supposed to explain how the missing values were treated. But the whole imputation process is questionable, because it is not clear how well it worked in comparison with alternatives and how reasonable it is to have an imputed series that look more erratic than the original one. The discussion of that needs to be expanded with some insights from the business problem.

6. No stockout handling discussion:

The authors do not discuss whether the data has stockouts or not. This becomes especially important in retail, because if the stockouts are not treated correctly, you would end up forecasting sales instead of demand

• For example, see this post.
7. Feature engineering is opaque:

“Lag and rolling-window statistics for sales and promotional indicators were created” (page 3, “Feature Engineering”) – it is not clear, what specific lags, what length of rolling windows, and what statistics (anything besides mean?) were created. These need to be explained for transparency and so that a reader could better understand what specifically was done. Without this explanation, it is not clear whether the features are sensible at all.

8. Training/validation setup not explained:

It is not clear how specifically the split into training and validation sets was done (page 3, last paragraph), and whether the authors used rolling origin (aka time series cross-validation). If they did random splits, that could cause some issues, because the first law of time series is not to break its structure!

9. Variables transformation is unclear:

It is not clear whether any transformations of the response variable were done. For example, if the data is not stationary, taking differences might be necessary to capture the trend and to do extrapolation correctly. Normalisation of variables is also important for neural networks, unless this is built-in in the functions the authors used. This is not discussed in the paper.

10. Forecast strategy not explained:

It is not clear whether the direct or recursive strategy was used for forecasting. If lags were not used in the model, that would not matter, but they are, so this becomes a potential issue. Also, if the authors used the lag of the actual value on observation 235 steps ahead to produce forecast for 236 steps ahead, then this is another fundamental issue, because that implies that the forecast horizon is just 1 step ahead, and not 365, as the authors claim. This needs to be explained in more detail.

• I’ve written a post about the strategies.
11. No statistical benchmarks:

At the very least, the authors should use simple moving average and probably exponential smoothing. Even if they do not perform well, this gives an additional information about the performance of the other approaches. Without them, the claims about good performance of the used ML approaches are not supported by evidence. The authors claim that they used mean as a benchmark, but its performance is not discussed in the paper.

• A post on the Simple Moving Average
• Why you should care about the exponential smoothing
12. Issues with forecast evaluation:

The whole Table 3 with error measures is an example of what not to do. Here are some of major issues:

1. There is no point in reporting several error measures – each one of them is minimised by their own statistics. The error measure should align with what approaches produce.
2. MSE, RMSE, MAE and ME should be dropped, because they are not scaled, so the authors are adding up error measures for bricks and nails. The result is meaningless.
3. MASE is not needed – it is minimised by median, which could be a serious issue on intermittent demand see this post. wMAPE has similar issues because it is also based on MAE.
4. If the point forecasts are produced in terms of medians (like in case of NBEATS), then RMSSE should be dropped, and MASE should be used instead.
5. But also, comparing means with medians is not a good idea. If you assume a symmetric distribution, the two should coincide, but in general this might not hold.
6. R2 is not a good measure of forecast accuracy. It makes some sense in regression context for linear models, but in this one, it is pointless, and only shows that the authors don’t fully understand what they are doing. Plus, it’s not clear how specifically it was calculated.
7. I don’t fully understand “demand error”, “demand bias” and other measures, and the authors do not explain them in necessary detail. This needs to be added to the paper.
8. The split into “Individual Groups” and “Whole Category” is not well explained either: it is not clear what this means, why, and how this was done.
9. And in general, I don’t understand what the authors want to do with Cases A – D in Table 3. It is not clear why they are needed, and what they want to show with them. This is not explained in the paper.
• I have a series of posts on forecast evaluation here.
13. Invalid analysis of bias measures:

Analysis of bias measures is meaningless because they were not scaled.

14. Disturbing bias of NBEATS in Figure 2:

The bias shown in Figure 2 is disturbing and should be dealt with prior to evaluation. It could have appeared due to the loss function used for training or because the data was not pre-processed correctly. Leaving it as is and blaming NBEATS for this does not sound reasonable to me.

15. No inventory implications:

The authors mention inventory management, but stop on forecasting, not showing how the specific forecasts translate to inventory decisions. If this paper was to be submitted to any operations-related journal, the inventory implications would need to be added in the discussion.

16. Underexplained performance gaps:

The paper also does not explain well why neural networks performed worse than gradient boosting methods. They mention that this could be due to the effect of missing values, but this is a speculation rather than an explanation, which I personally do not believe (I might be wrong). While the overall results make sense for me personally, if you want to publish a good paper, you need to provide a more detailed answer to the question “why?”.

Minor issues

I also have three minor comments:

1. “many product series are censored” (page 2, last paragraph) is not what it sounds like. The authors imply that the histories are short, while the usual interpretation is that the sales are lower than the demand, so the values are censored. I would rewrite this.
2. Figure 2 has the legend saying “Poisson” three times, not providing any useful information. This is probably just a mistake, which can easily be fixed.
3. There are no references to Table 2 and Figure 3 in the paper. It is not clear why they are needed. Every table and figure should be referred to and explained.

Conclusions

Overall, the paper has a sensible idea, but I feel that the authors need to learn more about forecasting principles and that they have not read forecasting literature carefully to understand how specifically the experiments should be designed, what to do, and not to do (stop using SBC!). Because they made several serious mistakes, I feel that the results of the paper are compromised and might not be correct.

P.S. If I were a reviewer of this paper, I would recommend either “reject and resubmit” or a “major revision” (if the former option was not available).

P.P.S. If the authors of the paper are reading this, I hope you find these comments useful. If you have not submitted the paper yet, I’d suggest to take some of them (if not all) into account. Hopefully, this will smooth the submission process for you.

Message Review of a paper on comparison of modern machine learning techniques in retail first appeared on Open Forecasting.

This September, we (Lancaster Centre for Marketing Analytics and Forecasting members) will deliver the Natcor course on Forecasting and Predictive Analytics at Lancaster University, UK. I’ll be acting as the course leader and I’m currently working on the schedule, aiming to make it both engaging and useful for participants.

The current plan is for the course to be delivered by our team members: Dr. Sven F. Crone, Anna Sroginis, Kandrika Pritularga, Nicos Pavlidis, Anna-Lena Sachs, and myself. The course will run over 5 days.

As for the content, we’ll start by covering the fundamentals of forecasting and gradually move on to more advanced statistical and machine learning approaches for forecasting and predictive analytics. As this course is designed specifically for PhD students, we’ll dive deeper into the material than we typically do in our Master’s level and in executive training courses. It will also be supported by real-life cases, of which the centre has accumulated many over the years, and discuss how forecasts translate to specific decisions and can be connected to the other OR-related topics.

You can find more details about the course here.

And register here (Note that NATCOR offers other excellent courses as well!).

Please share this post with your network if you know any PhD students or supervisors who might be interested.

P.S. Image in the post is partially related to what we will discuss, but is mainly here just to attract your attention 😝

Message NATCOR course on Forecasting and Predictive Analytics, September 2025 first appeared on Open Forecasting.

When Box and Jenkins wrote their book back in the 1960s, it was a very different era: computers were massive, and people worked with punch cards. To make their approach viable, Box and Jenkins developed a methodology for selecting the appropriate orders of AR and MA based on the values of the autocorrelation and partial autocorrelation functions (ACF and PACF, respectively). Their idea was that if an ARMA process generates a specific ACF/PACF pattern, then it could be identified by analysing those functions in the data. At the time, it wasn’t feasible to do cross-validation or rolling origin evaluation, and even using information criteria for model selection was a challenge. So, the Box–Jenkins approach was a sensible option, producing adequate results with limited computational resources, and was considered state of the art.

Unfortunately, as the M1 competition later showed (see my earlier post), the methodology didn’t work well in practice. Simpler methods that didn’t rely on rigorous model selection actually performed better. But in fact, the winning model in the competition was ARARMA by Emanuel Parzen (https://doi.org/10.1002/for.3980010108). His idea was to make the series stationary by applying a low-order, non-stationary AR to the data, then extract residuals and select appropriate ARMA orders using AIC. Parzen ignored the Box–Jenkins methodology entirely – he didn’t analyse ACF or PACF and instead relied fully on automated selection. And it worked!

So why didn’t the Box–Jenkins methodology perform as expected? In my monograph Forecasting and Analytics with ADAM, I use the following example to explain the main issue: “All birds have wings. Sarah has wings. Thus, Sarah is a bird.” But Sarah, as shown in the image attached to this post, is a butterfly.

The fundamental issue with the Box–Jenkins methodology lies in its logic: if a process generates a specific ACF/PACF, that doesn’t mean that an observed ACF/PACF must come from that process. Many ARMA and even non-ARMA processes can generate exactly the same autocorrelation structure.

Further developments in ARIMA modelling have shown that ACF and PACF can only be used as general guidelines for order selection. To assess model performance properly, we need other tools. All modern approaches rely on information criteria for ARIMA order selection, and they consistently perform well in forecasting competitions. For example, Hyndman & Khandakar (2008) use AIC for ARMA order selection, while Svetunkov & Boylan (2020) apply AIC after reformulating ARIMA in a state space form. The former is implemented in the forecast package in R and the StatsForecast library in Python (thanks to Nixtla and Azul Garza); the latter is available in the smooth package in R. I also discuss another ARIMA order selection approach in Section 15.2 of my book.

Long story short: don’t use the Box–Jenkins methodology for order selection. Use more modern tools, such as information criteria.

P.S. See also my early post on ARIMA, discussing what is wrong with it.

Message Fundamental Flaw of the Box-Jenkins Methodology first appeared on Open Forecasting.

What is this paper about?

Intermittent demand is a common challenge in sectors like supply chain and retail. But the key issue is that zeroes in sales can happen for two fundamentally different reasons (see one of my previous posts):

• Nobody wanted to buy the product (naturally occurring zeroes),
• Nobody could buy the product (artificially occurring due to stockouts, etc).

However, forecasting methods are typically unaware of this distinction and treat both types equally. This can lead to inaccurate forecasts and poor decisions. On top of that, existing classification schemes for intermittent demand (such as SBC) use arbitrary thresholds and rely on choosing between forecasting methods like Croston and SBA. There’s a clear need for smarter, more flexible tools that can distinguish between types of demand and make classifications practical.

In this paper, we introduce a two-stage, model-based framework called “Automatic Identification of Demand” (AID), designed to bring more clarity and accuracy to demand classification. The first stage uses a data-driven approach to detect artificially occurring zeroes. Once those are accounted for, the second stage classifies the demand into one of six categories based on key characteristics: whether the demand is regular or intermittent, whether it consists of count or fractional values, and whether intermittent demand is smooth or lumpy in nature. AID detects stockouts by analysing demand intervals using the Geometric distribution, then flags the demand as one of those six types based on several simple statistical models.

We applied AID to a retailer dataset covering over 31,000 products with weekly sales across three stores. Based on that, we generated several features and tested multiple approaches (local level, pooled regression, and LightGBM) to see whether their accuracy improved. We found that:

1. Correcting for stockouts significantly improved the accuracy of all approaches;
2. Using a mixture approach (separating demand into sizes and occurrences) yielded large gains in accuracy, regardless of the forecasting method used;
3. Further splitting the data by demand categories (e.g., regular vs. intermittent, smooth vs. lumpy) provided additional, though more modest, benefits.

We argue that these three principles are universally valuable for forecasting, no matter what approach you use. If you face intermittent demand, at a minimum, consider detecting stockouts and then using the mixture approach.

Hope you find this paper useful. Let me know what you think in the comments.

Message Svetunkov & Sroginis (2025) – Model Based Demand Classification first appeared on Open Forecasting.

Here it is or here.

This paper, written by Xiaoqian Wang, Rob Hyndman, Feng Li, and Yanfei Kang, is a 50-year literature review on the topic of forecast combinations. The authors conduct a thorough review of the literature in this area. They begin by discussing implications for point forecasts, covering different combination methods such as linear and non-linear approaches, learning-based combinations, pooling, and more. Then, they shift their focus to probabilistic forecast combinations, exploring what it means to combine quantiles and how to make them better calibrated. As expected, the paper ends with conclusions, but the authors go further, summarising some of the gaps in the literature – a helpful starting point for those interested in forecasting research.

I admit that this paper has nothing to do with Christmas, but I feel it’s a fitting way to say “Good bye” to the year 2024. While we’ve seen remarkable developments in machine learning over the past year, I feel that some people are starting to lose sight of basic forecasting principles. This paper discusses one of the important ones: combinations often produce more robust forecasts than individual models, explained here in great detail.

Merry Christmas, Happy New Year, and see you in 2025!

Message A paper to read over the Xmas holiday: Wang et al. (2023) – Forecast combinations: An over 50-year review first appeared on Open Forecasting.

The featured image in this post shows two forecasting approaches: one that tries to predict when demand happens (the yellow line), and the other one that tries capturing the structure of the demand and extrapolates it (the blue line). The green line shows the values in the holdout, and the RMSE indicates the error of the two approaches. Apparently, the straight line is better in this example. Let’s discuss why.

Just a reminder, intermittent demand is the demand that happens at irregular frequency. By definition, we cannot know when a person will come to our store and buy the product. We operate with probabilities in this case, and can say sometimes that the probability of purchase goes up or down due to some factors (seasonality, holidays, promotion etc). When a spherical ML expert in vacuum hears about probability, the first thing that pops to their mind is the “decision boundary” for classification task. Why not set some threshold and say that if the probability is higher than that, the product will be bought and in the other case it won’t?

Well, while this works in classification, it typically doesn’t make sense in demand forecasting.

First, there’s not much structure to capture in intermittent demand besides the basic level, external factors, such as promotions and calendar effects, and occasional trend. Yes, some of them might change the probability of occurrence, and, for example, show that a product will be bought on Monday with 90% probability. This still does not mean that the product will be indeed bought. Saying that it will is just informed guessing, not forecasting.

Second, point forecast is supposed to capture the structure and filter out the noise (see this post). In case of intermittent demand, the structure consists of two parts: expected occurrence (probability) and demand sizes. If we substitute the probability with zeroes and ones based on some threshold, we’ll end up overfitting the noise, but on a different level than usually: the future is uncertain and we can never say for sure what will happen and when, yet we would be playing a guessing game, hoping to be correct. It is like tossing a coin, trying to guess how it will land next time. If you want to have an expectation in that experiment, you should have probability, not a sequence of zeroes and ones.

Third and most important, working with intermittent demand, we typically want to solve a specific problem. The classical example is inventory management, in which case we don’t care whether customers will come and buy our product on Monday, instead of Tuesday. We care about having enough product on shelves to satisfy customers throughout a period of time, while our product is being delivered (lead time). So, the goal in this case is to identify the appropriate safety stock level based on the current stock and thus get an estimate of the demand over lead time, not to predict when people come and how much they will buy. Focusing on the point forecast in this setting is a futile task.

So, when working with intermittent demand, don’t waste your time on trying to forecast when the demand will happen. Focus instead on getting the structure correctly and then understanding what is needed by decision makers and how it will be used.

Message Intermittent demand: don’t try to predict WHEN it will happen first appeared on Open Forecasting.

Naive is a forecasting method that uses the last available observation as a forecast for the next ones. It does not have any parameters to estimate, it does not require training, it can be applied to the sample of any data (even if you only have one observation). When you deal with a regular demand, it makes perfect sense to use Naive as a benchmark, because it costs nothing in terms of computational time, you get a forecast of demand, and if you cannot beat it, you should rethink your forecasting process.

However, in case of intermittent demand, the demand itself does not happen on every observation. As a result, when the Naive copies the last available value, it can either reproduce either a proper non-zero demand, or just the absence of demand. The latter implies that nobody bought our product today, and nobody will do in the next week or whatever the forecasting horizon we use. In the following image, Naive will be the most accurate forecasting method, because in the training set, the final observation was zero, and in the test set we did not have any sales:

Naive forecast on intermittent demand

But is this useful? To answer this question, we need to understand what specifically we are forecasting when we deal with demand with zeroes.

As discussed in a previous post, zeroes can occur for different reasons: some of them happen because nobody came to buy the product (naturally occurring zeroes), while the others appear because there was some sort of disruption (e.g. a stockout) or a product was discontinued (artificially occurring zeroes). The two situations are fundamentally different, but if we work with the sales data exclusively (no stock information), it can be hard to tell the difference between them. Naive might work perfectly in both cases, forecasting no sales for the next few observations, and it can be 100% right in some cases. But the problem is that this is not useful. If we indeed cannot beat Naive on the data with zeroes, it does not mean that we should use it, because there is a chance that we have stockouts in the holdout period. If that’s the case, we might be doing something fundamentally wrong. After all, “we will not sell anything” is in general a simple statement, but not ordering products based on that could be a mistake, because “no sales” is not the same as “no demand”. In fact, if Naive indeed performs very well on your series with zeroes, this might indicate that your evaluation is wrong and you need to clean the data, removing the discontinued and out of stock items from the evaluation.

There are three lessons here:

1. we should forecast demand, not sales;
2. we should measure accuracy on the data with naturally occurring zeroes – do data cleaning before setting up your evaluation;
3. it’s better to use a benchmark that tries capturing demand, not the one that reproduces sales.

Arguably, a more helpful benchmark forecast would be the one in the following image:

Forecast for intermittent demand from the SMA

The forecast above was generated using the Simple Moving Average, and it tells us that there is a demand for the product over the next 13 days. Yes, it is less accurate than Naive, but it gives an estimate of the expected demand, not the expected sales.

Message Why Naive is not a good benchmark for intermittent demand first appeared on Open Forecasting.