First things first, why am I writing this post? I want to share failures with the community, because I am tired of all the success stories. It is okay not to win, and this happens much more often than it seems.

Now, about the paper. In the first round of revisions, four reviewers looked at it and provided their comments. We expanded the paper accordingly, making it now 46 pages long (ouch!). We introduced inventory simulations and showed how using some basic principles improves forecasting accuracy and can lead to a reduction in inventory costs.

In the second round, the AE added one more reviewer. After careful consideration, two of the reviewers recommended major revisions, while the other two suggested a strong rejection, claiming that the paper does not make new and significant contributions to the production research literature.

Obviously, I disagree with this evaluation. Based on the reviewers’ comments, I have a feeling they didn’t read the paper in full (their main concerns relate to Section 3, and some of these could have been resolved if they had reached Section 5). But this probably also means that the paper in its current state is too big and needs to be rewritten to become more focused. Maybe this is what confused the reviewers.

So, what’s next?

We will amend it to address the reviewers’ comments, shorten it a bit to make it more focused, and then submit to another OR-related journal.

And while we are doing that, I have updated the arXiv version of the paper to show what we did after the first round, and here is a brief summary of the main findings:

• Using a stockout dummy variable and capturing the level of data correctly (removing the effect of stockouts) improves the accuracy of forecasting approaches;
• The stockouts detection should be done for both the training and the test sets. If the series with stockouts are not removed from the test set, the forecasts would be evaluated incorrectly;
• Splitting the demand into demand sizes and demand occurrence, producing forecasts for each of the parts and then combining the result substantially improves the accuracy;
• Using the feature for regular/intermittent demand improves the forecasting accuracy, but does not seem to impact the inventory performance. Note that this separation is straightforward in AID: if after removing the stockouts, there are some zeroes left, the demand is identified as intermittent;
• The further split into smooth/lumpy leads to slight improvements in terms of accuracy, without a substantial impact on the inventory;
• The split into count/fractional demand does not bring value in terms of forecasting accuracy or inventory performance.

Message AID paper rejected from the IJPR first appeared on Open Forecasting.

Several papers I reviewed claim that demand can be either intermittent or lumpy. They then mention the Syntetos-Boylan-Croston (SBC) classification and use the thresholds from Syntetos et al. (2005: ) to do some things with ML methods. Sounds reasonable?

No! And here’s why.

Actually, I’ve already explained this in a previous post, but let me summarise the main points again.

First, intermittent demand is the demand that happens at irregular frequency. That’s the definition John Boylan and I came up with in our paper (this one). But even before that, the literature generally agreed: if you observe naturally occurring zeroes (e.g., no one wants to buy a product), then the demand is intermittent – even if there’s only one zero in the data.

Now, Syntetos et al. (2005) specifically studied intermittent demand and proposed a classification to help choose between Croston’s method and SBA. Their classification includes four types (see image in the post):

1. Erratic but not very intermittent
2. Smooth
3. Lumpy
4. Intermittent but not very erratic

The thresholds they used (ADI=1.32 and CV²=0.49) were only intended to guide the choice between Croston and SBA. And “lumpy”, as you can see, is just a special case of intermittent demand!

Yes, you can classify intermittent demand into “lumpy” and “smooth”, but this separation is not well-defined. Use a different classification (e.g., this paper) and you’ll get different results. In fact, practically speaking, your ML approach likely doesn’t need this classification at all.

So, here are a two things you should NOT DO:

1. Saying that demand can be “intermittent” or “lumpy” – the latter is a subset of the former.
2. Use ADI=1.32 and/or CV²=0.49 to categorise demand, unless you’re selecting between Croston and SBA. And let’s be honest, you’re probably not doing that. So forget about it!

And honestly, stop overusing SBC! Lately, I’ve seen more harm than good from it. If you really want to use it, make sure you’ve read carefully and understood the original paper.

But if you don’t know what you are doing, SBC is not for you!

Message SBC is not for you! first appeared on Open Forecasting.

My presentation was based on this paper. And here are the slides:
2025-05-01-IMA-OR

Message 5th IMA and OR Society Conference first appeared on Open Forecasting.

What is this paper about?

Intermittent demand is a common challenge in sectors like supply chain and retail. But the key issue is that zeroes in sales can happen for two fundamentally different reasons (see one of my previous posts):

• Nobody wanted to buy the product (naturally occurring zeroes),
• Nobody could buy the product (artificially occurring due to stockouts, etc).

However, forecasting methods are typically unaware of this distinction and treat both types equally. This can lead to inaccurate forecasts and poor decisions. On top of that, existing classification schemes for intermittent demand (such as SBC) use arbitrary thresholds and rely on choosing between forecasting methods like Croston and SBA. There’s a clear need for smarter, more flexible tools that can distinguish between types of demand and make classifications practical.

In this paper, we introduce a two-stage, model-based framework called “Automatic Identification of Demand” (AID), designed to bring more clarity and accuracy to demand classification. The first stage uses a data-driven approach to detect artificially occurring zeroes. Once those are accounted for, the second stage classifies the demand into one of six categories based on key characteristics: whether the demand is regular or intermittent, whether it consists of count or fractional values, and whether intermittent demand is smooth or lumpy in nature. AID detects stockouts by analysing demand intervals using the Geometric distribution, then flags the demand as one of those six types based on several simple statistical models.

We applied AID to a retailer dataset covering over 31,000 products with weekly sales across three stores. Based on that, we generated several features and tested multiple approaches (local level, pooled regression, and LightGBM) to see whether their accuracy improved. We found that:

1. Correcting for stockouts significantly improved the accuracy of all approaches;
2. Using a mixture approach (separating demand into sizes and occurrences) yielded large gains in accuracy, regardless of the forecasting method used;
3. Further splitting the data by demand categories (e.g., regular vs. intermittent, smooth vs. lumpy) provided additional, though more modest, benefits.

We argue that these three principles are universally valuable for forecasting, no matter what approach you use. If you face intermittent demand, at a minimum, consider detecting stockouts and then using the mixture approach.

Hope you find this paper useful. Let me know what you think in the comments.

Message Svetunkov & Sroginis (2025) – Model Based Demand Classification first appeared on Open Forecasting.

Message Why do zeroes happen? A model-based view on demand classification first appeared on Open Forecasting.

The mistake? Using error measures based on the Mean Absolute Error (MAE). This is a crime against the humanity when working with intermittent demand. I’ve explained this issue multiple times before (here, here, and here), but it appears that this idea needs to be repeated over and over again. Let me explain.

MAE is minimised by the median. In the case of intermittent demand, the median can often be zero. If you use MAE (or scaled measures like MASE or sMAE) to evaluate forecasts and compare, for example, Croston, TSB, ETS, and an Artificial Neural Network (ANN), you may find the ANN outperforming the others. However, this could simply mean that the ANN produces forecasts closer to zero than the alternatives. This is not what you want for intermittent demand! The goal is to capture the structure correctly and produce conditional mean forecasts (typically). Instead, by relying on MAE, you might conclude: “We won’t sell anything in the next two weeks”, implying that there’s no need to stock products. This is apparently wrong and unhelpful.

Attached to this post is a figure showing three forecasts for an intermittent demand series:

• The blue line represents the mean of the data;
• The green line is a forecast from an Artificial Neural Network;
• The red line is the zero forecast.

In the figure’s legend, you’ll see error measures indicating that the zero forecast performs best in terms of MAE, followed by the ANN, and lastly, the mean forecast. Based on MAE, the conclusion would be: “We won’t sell anything, so don’t bother stocking the product”. But this outcome occurs solely because 12 out of 20 values in the holdout are zeros, making the median zero as well.

On the other hand, RMSE provides a more reasonable evaluation, showing that the mean of the data is more informative and preferable to the other methods.

The brief summary of this post is: *Don’t use MAE-based error measures for intermittent demand!* (Insert as many exclamation marks as you’d like!)

P.S. Actually, as a general rule, avoid using MAE for evaluating methods that produce mean forecasts. For more details, check out this post.

P.P.S What frustrates me a lot is that the reviewers of those papers did nothing to fix this issue, which means that they are clueless about that as well.

Message Don’t use MAE-based error measures for intermittent demand! first appeared on Open Forecasting.

The featured image in this post shows two forecasting approaches: one that tries to predict when demand happens (the yellow line), and the other one that tries capturing the structure of the demand and extrapolates it (the blue line). The green line shows the values in the holdout, and the RMSE indicates the error of the two approaches. Apparently, the straight line is better in this example. Let’s discuss why.

Just a reminder, intermittent demand is the demand that happens at irregular frequency. By definition, we cannot know when a person will come to our store and buy the product. We operate with probabilities in this case, and can say sometimes that the probability of purchase goes up or down due to some factors (seasonality, holidays, promotion etc). When a spherical ML expert in vacuum hears about probability, the first thing that pops to their mind is the “decision boundary” for classification task. Why not set some threshold and say that if the probability is higher than that, the product will be bought and in the other case it won’t?

Well, while this works in classification, it typically doesn’t make sense in demand forecasting.

First, there’s not much structure to capture in intermittent demand besides the basic level, external factors, such as promotions and calendar effects, and occasional trend. Yes, some of them might change the probability of occurrence, and, for example, show that a product will be bought on Monday with 90% probability. This still does not mean that the product will be indeed bought. Saying that it will is just informed guessing, not forecasting.

Second, point forecast is supposed to capture the structure and filter out the noise (see this post). In case of intermittent demand, the structure consists of two parts: expected occurrence (probability) and demand sizes. If we substitute the probability with zeroes and ones based on some threshold, we’ll end up overfitting the noise, but on a different level than usually: the future is uncertain and we can never say for sure what will happen and when, yet we would be playing a guessing game, hoping to be correct. It is like tossing a coin, trying to guess how it will land next time. If you want to have an expectation in that experiment, you should have probability, not a sequence of zeroes and ones.

Third and most important, working with intermittent demand, we typically want to solve a specific problem. The classical example is inventory management, in which case we don’t care whether customers will come and buy our product on Monday, instead of Tuesday. We care about having enough product on shelves to satisfy customers throughout a period of time, while our product is being delivered (lead time). So, the goal in this case is to identify the appropriate safety stock level based on the current stock and thus get an estimate of the demand over lead time, not to predict when people come and how much they will buy. Focusing on the point forecast in this setting is a futile task.

So, when working with intermittent demand, don’t waste your time on trying to forecast when the demand will happen. Focus instead on getting the structure correctly and then understanding what is needed by decision makers and how it will be used.

Message Intermittent demand: don’t try to predict WHEN it will happen first appeared on Open Forecasting.

Naive is a forecasting method that uses the last available observation as a forecast for the next ones. It does not have any parameters to estimate, it does not require training, it can be applied to the sample of any data (even if you only have one observation). When you deal with a regular demand, it makes perfect sense to use Naive as a benchmark, because it costs nothing in terms of computational time, you get a forecast of demand, and if you cannot beat it, you should rethink your forecasting process.

However, in case of intermittent demand, the demand itself does not happen on every observation. As a result, when the Naive copies the last available value, it can either reproduce either a proper non-zero demand, or just the absence of demand. The latter implies that nobody bought our product today, and nobody will do in the next week or whatever the forecasting horizon we use. In the following image, Naive will be the most accurate forecasting method, because in the training set, the final observation was zero, and in the test set we did not have any sales:

Naive forecast on intermittent demand

But is this useful? To answer this question, we need to understand what specifically we are forecasting when we deal with demand with zeroes.

As discussed in a previous post, zeroes can occur for different reasons: some of them happen because nobody came to buy the product (naturally occurring zeroes), while the others appear because there was some sort of disruption (e.g. a stockout) or a product was discontinued (artificially occurring zeroes). The two situations are fundamentally different, but if we work with the sales data exclusively (no stock information), it can be hard to tell the difference between them. Naive might work perfectly in both cases, forecasting no sales for the next few observations, and it can be 100% right in some cases. But the problem is that this is not useful. If we indeed cannot beat Naive on the data with zeroes, it does not mean that we should use it, because there is a chance that we have stockouts in the holdout period. If that’s the case, we might be doing something fundamentally wrong. After all, “we will not sell anything” is in general a simple statement, but not ordering products based on that could be a mistake, because “no sales” is not the same as “no demand”. In fact, if Naive indeed performs very well on your series with zeroes, this might indicate that your evaluation is wrong and you need to clean the data, removing the discontinued and out of stock items from the evaluation.

There are three lessons here:

1. we should forecast demand, not sales;
2. we should measure accuracy on the data with naturally occurring zeroes – do data cleaning before setting up your evaluation;
3. it’s better to use a benchmark that tries capturing demand, not the one that reproduces sales.

Arguably, a more helpful benchmark forecast would be the one in the following image:

Forecast for intermittent demand from the SMA

The forecast above was generated using the Simple Moving Average, and it tells us that there is a demand for the product over the next 13 days. Yes, it is less accurate than Naive, but it gives an estimate of the expected demand, not the expected sales.

Message Why Naive is not a good benchmark for intermittent demand first appeared on Open Forecasting.

First, a quick note: it’s a mistake to define intermittent demand simply as “demand with zeroes”. That definition is incomplete and can be misleading. As some of you know, the definition I prefer is that intermittent demand occurs at irregular frequencies. This means the zeroes in such demand are unpredictable and happen simply because nobody wanted to buy the product on a specific day. Unless you know precisely who will buy and how much, you can’t predict if there will be demand that day. These zeroes can be considered “naturally occurring”.

But zeroes can also happen for other reasons. People might want to buy a product, but it may be unavailable. This typically happens due to stockouts, caused by either incorrect safety stock levels, supply chain disruptions (e.g., a container ship running aground), or a product being discontinued by the company. Sometimes, zero sales occur because a store was closed for a holiday, a gas leak, a flood, or another unexpected event. These types of zeroes are explainable and sometimes even predictable, so we can call them “artificially occurring”.

Furthermore, zeroes may appear at the start of a time series if a product was recently introduced and lacks a sales history. These zeroes don’t provide useful information for forecasting.

Some zeroes might also occur seasonally, for example, for Christmas-related products. These too can be classified as artificially occurring because, while there may be a small demand for such items, it’s usually unreasonable to sell them just to satisfy a handful of customers.

Finally, errors in the system can result in zeroes. For example, sales might not have been recorded correctly, leading to either zeroes or missing values (sometimes treated as zeroes). These can also be categorized as “artificially occurring”.

Demand with only artificially occurring zeroes isn’t intermittent; it is regular demand with issues.

Having said that, the reality is often more complex. You can easily have intermittent demand with stockouts, and distinguishing between naturally and artificially occurring zeroes in such cases can be challenging.

But why bother?

If your goal is to forecast demand (not just sales), you need to address artificially occurring zeroes in your data. When applying models, you should indicate which observations should either be ignored or treated differently. Similarly, when measuring the performance of your models (e.g., forecasting accuracy), you should evaluate them on data without artificially occurring zeroes. Otherwise, you’ll end up testing which model forecasts stockouts or system failures better, rather than actual demand. This ties into the well-known principle: “You should forecast demand, not sales”. In the case of intermittent demand, this is not only difficult but also extremely important.

Here’s an example of a time series N27364 from the M5 competition (Makridakis et al., 2022):

Series N27364 from the M5 dataset

This isn’t a unique case, most time series in the M5 dataset have stockouts. In this specific example, gaps in sales are apparent and likely caused artificially. If we train a model on this data, it might those zeroes into account and produce inaccurate demand forecasts, e.g. lower point forecasts than necessary. The problem worsens if the test set also contains stockouts, as the selected model would be the one that forecasts artificially occurring zeroes better. Using such a model in decision-making could be harmful, leading to erroneous decisions like discontinuing products that actually sell well.

As a final note, Stephan Kolassa has given excellent presentations on the challenges of forecasting in retail. He has shared insightful examples of the complexities of tracking sales and stock. For instance, he discussed this topic in one of the CMAF webinars and in this short video.

Message Why zeroes happen first appeared on Open Forecasting.

Intermittent demand comes in different flavours: sometimes zeroes occur frequently with low demand volumes, while other times the volumes are high with occasional zeroes. Demand patterns can also change over time, with demand either becoming obsolete (more zeroes) or building up (fewer zeroes). How can we classify these different types of demand? Well, there is a paper on that (academic Rule 34)!

Syntetos, Boylan & Croston (2005) developed a categorization scheme using the Average Demand Interval (ADI) and Coefficient of Variation (CV). They compared MSE performance of Croston (1972) and SBA (Syntetos & Boylan, 2005) forecasting methods, creating four categories of intermittent demand with ADI=1.32 and CV²=0.49 as cut-off values:

1. Erratic but not very intermittent
2. Smooth
3. Lumpy
4. Intermittent but not very erratic

These are distinct categories of INTERMITTENT demand, though the names of the first and last are sometimes shortened to “Erratic” and “Intermittent,” causing confusion (intermittent demand can be intermittent?). The authors recommended using Croston for (1) and SBA for the other three. The image below illustrates these categories, with ADI increasing from left to right and CV increasing from bottom to top.

Examples of intermittent demand data

But that’s not all! Kostenko & Hyndman (2006) found that the split between Croston and SBA does not form four distinct areas – the cut-off should be non-linear. While mathematically correct, this classification has not gained as much popularity as SBC because it is more complicated. There is also a reply from Syntetos, Boylan & Croston to Kostenko & Hyndman, where the authors of the original classification agree with the new cut-off but also point out that their classification is practical, while not necessarily as accurate as KH.

Furthermore, Petropoulos & Kourentzes (2015) extended the KH classification by adding Simple Exponential Smoothing for regular demand, where the average inter-demand interval equals to one.

So, we have at least 3 popular techniques. So what?

These classifications were designed for conventional intermittent demand (e.g., spare parts) assuming stable ADI and CV over time. But what if demand builds up (fewer zeroes, higher volume) or slows down? In such cases, SBC, KH, and PK would be inappropriate. Moreover, classification should serve a purpose. SBC’s original purpose was to help choosing between Croston and SBA. So, the threshold between “lumpy” and “erratic” is based on these methods’ MSE performance. Are you using these methods in your case? If not, why bother with SBC/KH/PK classifications?

In the 2024, we have more advanced models and methods, and, for example, using SBC to decide between XGBoost and Poisson regression would be unwise. You need a different classification! Or maybe you don’t need one at all, just apply competing approaches and select the most appropriate one based on the holdout performance.

So, next time you work with intermittent demand, stop for a second and think what you plan to do. SBC is useful, but don’t use it just because you don’t know what to do!

Message Intermittent demand classifications: is that what you need? first appeared on Open Forecasting.