When I test my functions in R, I usually use the M1, M3, and tourism competition datasets because they are diverse enough, containing seasonal, non-seasonal, trended, and non-trended time series of different frequencies (yearly, quarterly, monthly). The total number of these series is 5,315, which is large enough but not too heavy for my PC. So, when I run something on those datasets, it becomes like a stress test for the forecasting approach, and I can see where it fails and how it can be improved. I consider this type of test a toy experiment — something to do before applying anything to real-world data.

In R, there are the Mcomp and Tcomp packages that contain these datasets, and I like how they are organised. You can do something like this:

series <- Mcomp::M3[[2568]]
ourModel <- adam(series$x)
ourForecast <- forecast(model, h=series$h)
ourError <- series$xx - ourForecast$mean

Each series from the dataset contains all the necessary attributes to run the experiment without trouble. This is easy and straightforward. Plus, I don’t need to download or organise any data — I just use the installed package.

When I started vibe coding in Python, I realised that I missed this functionality. So, with the help of Claude AI, I created a Python script to download the data from the Monash repository and organise it the way I liked. But then I realised two things, which motivated me to package it:

1. I needed to drag this script with me to every project I worked on. It would be much easier to just run "pip install fcompdata" and forget about everything else.
2. Some series in the Monash repository differ from those in the R package.

Wait, what?! Really?

Yes. The difference is tiny — it’s a matter of rounding. For example, series N350 from the M1 competition data (T169 from the quarterly data subset) has three digits in the R package and only two if downloaded from the Monash repository (Zenodo website).

Who cares?! It's just one digit difference, right?

Well, if you want to reproduce results across different languages, this tiny difference might become your nightmare. So, I care (and probably nobody else in the world), and I decided to create a proper Python package. You can now do this in Python and relax:

pip install fcompdata

from fcompdata import M1, M3, Tourism
series = M3[2568]

The "series" object is now an instance of the MCompSeries class that has the same attributes as in R: series.x, series.h, series.xx, etc.

As simple as that!

One more thing: I’ve added support for the M4 competition data, which — when imported — will be downloaded and formatted properly. The dataset is large (100k time series), and I personally don’t like it. I even wrote a post about it back in 2020. But if I want the package to be useful to a wider audience, I shouldn’t impose my personal preferences — you should decide for yourselves whether to use it or not.

P.S. Submitting to PyPI gave me a good understanding of the submission process for Python and why it can be such a mess. My package was published just a few seconds after submission — nobody looked at it, nobody ran any tests. CRAN does a variety of checks to ensure you don’t submit garbage. PyPI doesn’t care. So, I’ve gained more respect for CRAN after submitting this package to PyPI.

Message Forecasting Competitions Datasets in Python first appeared on Open Forecasting.

DISCLAIMER: the image in the post is taken from the paper, Figure 6, showing the proportion of GRP A&E admissions, the forecast errors and two detectors.

Here’s a brief summary of what it’s about:

One of the common issues in forecasting is that there might be some serious changes in the data due to external factors (e.g. changes in consumer preferences). These changes are not always captured by the model, which can lead to reduced accuracy, increased variance, and ultimately to losses. The changepoint detection literature addresses this by trying to automatically identify such structural changes and alert analysts when intervention might be needed. This becomes especially useful when managing large numbers of time series, where visual inspection isn’t feasible.

However, most existing approaches either work directly on raw data or rely on a model, which makes their usefulness limited.

Tom Grundy and Rebecca Killick came up with a better idea: analysing forecast errors instead. They kindly invited me to join as a co-author (since I know a thing or two about forecasting). The result is an online changepoint detection mechanism that is more universal and can be applied to classical statistical forecasting models and potentially to machine learning approaches.

The paper is quite technical and includes theoretical derivations, showing that the proposed method substantially reduces detection delay compared to some conventional approaches. We also evaluated its performance with ARIMA and ETS models on simulated data and provided several examples with real time series, demonstrating how it works.

The final version of the paper is available here, while the pre-print is here.

Message Online Detection of Forecast Model Inadequacies Using Forecast Errors first appeared on Open Forecasting.

I understand why people use the term “Time series forecasting” – they want to show the type of data they work with and explain what they are doing. But there is no point in forecasting outside of time series. According to one of the definitions (I previously mention it here and here), “forecast is a scientifically justified assertion about possible states of an object in the future”, while forecasting is just a process of producing forecasts. So, the time is already embedded in the definition, and there is no need to add “time series” to it.

Furthermore, you cannot do “cross-sectional forecasting”. It wouldn’t be forecasting per se, but rather scenarios generation. For example, you can say that based on the collected cross-sectional data across several shops in our chain on Monday, the increase of price of a product should cause a decline in sales on average by some amount. You can even say what sales to expect if the price and other variables were somehow defined. But you cannot say what to expect in next week based on this data, because the cross-sectional data itself does not have dynamic element. It is good for scenario planning, but useless in forecasting.

Furthermore, when you say “time series forecasting”, you imply a wide area without any specificity. But there are many areas, where you can do forecasting which differ substantially from one to another. Are you doing “demand forecasting”, “revenue forecasting”, “price forecasting”, “volatility forecasting” or something else? While some approaches can be applied to many of these areas, they still have their specificity and own set of instruments and rules. So, you should always keep the specific domain in mind, otherwise you might do something unreasonable. For example, GAMLSS works very well in energy demand forecasting, but it does not necessarily perform as well in supply chain forecasting, where you often have short history and lots of zeroes.

But most importantly, forecasting should not be done for the sake of itself (see my old post from COVID times on this). If you “forecast a time series”, then you just do an exercise without a specific aim. Forecasting should inform decisions. Yes, you can show how cool you are, but is there anything beside that?

So, when I see statements like “This and that guy will talk in our webinar about time series forecasting”, I feel that the person has a poor understanding of forecasting itself and does not know what to talk about, because there is no specificity in such talk. It’s like presenting on the topic of “frequentist statistics” – too broad and too general.

I feel that this post might provoke some debate, so feel free to express yourselves in the comments! :)

Message Is there such thing as “Time series forecasting”? first appeared on Open Forecasting.

First things first, in forecasting, the term “seasonality” refers to any natural pattern repeating with some periodicity. For example, if you work in a hospital with A&E attendance, you know that every Monday has higher demand than other days, while weekends tend to have lower demand. This is a well known phenomenon: people have fun over the weekend and then go to hospital first thing in the morning of the work week, so if you don’t want to get stuck in the hospital, don’t injure yourselves over the weekend!

Anyway, back to the main topic of this post!

Consider the following example:

Seemingly seasonal time series

This is a daily data. Is it seasonal? Without context, you might assume so: there’s a midweek peak followed by a decline, repeating weekly, so it must be seasonal, right? But what if I told you that this data is daily LinkedIn impressions of my posts? The peaks coincide with posts releases, and it is hard to tell from the image, but I released first three posts on Thursdays and then switched to Wednesdays, shifting peaks one day forward. So, this is not a seasonal data, but instead it is a classical life cycle (which can be described, for example, by the Bass model), repeating every week. It would be seasonal if the impressions happened naturally without me releasing anything. For example, number of visitors of my website has natural seasonality, because they happen without my interventions:

The series with natural seasonality

Bringing this to a business context, I have encountered several times the “spurious seasonality”. For example, one company, working with weekly data, used a seasonal model with periodicity of 4, because they noticed that at the end of each month, people get their salary and spend it, increasing the sales of the company. However this is not true seasonality, but rather a calendar event that happens seemingly periodically, on a specific day of month (not necessarily the same one).

Why is this important?

Relying on a seasonal model (like seasonal ETS or ARIMA) in such cases poses risks. If the periodicity shifts (e.g., salaries received on a different week), the model will produce misaligned forecasts (e.g., predicting an earlier peak). To address this, you should model such events with explanatory variables instead of seasonal indices. This way you will be able to control the timing of the event in your model and adjust it if needed.

So, next time you see a pattern that looks seasonal, think whether it happens naturally, or whether you are dealing with the spurious seasonality. Remember, the context is always important!

Message Seasonal or not? first appeared on Open Forecasting.