Risky business: how to select your model based on risk preferences

A distribution of some error measures across models

What do you use for model selection? Do you select the best model based on its cross-validated performance, or do you use in-sample measures like AIC? If so, there is a way to improve your selection process further. JORS recently published the paper of Nikos Kourentzes and I based on a simple but powerful idea: … Read more

Review of a paper on comparison of modern machine learning techniques in retail

A couple of days ago, I noticed a link to the following paper in a post by Jack Rodenberg: https://arxiv.org/abs/2506.05941v1. The topic seemed interesting and relevant to my work, so I read it, only to find that the paper contains several serious flaws that compromise its findings. Let me explain. Introduction But first, why am … Read more

Online Detection of Forecast Model Inadequacies Using Forecast Errors

Figure 6 from the paper, showing the proportion of GRP A&E admissions, the forecast errors and two detectors.

There’s a large and fascinating area in time series analysis called “changepoint detection”. I hadn’t worked in this area before, but thanks to Rebecca Killick and Thomas Grundy, I contributed to the paper “Online Detection of Forecast Model Inadequacies Using Forecast Errors“, which has just been published in the Journal of Time Series Analysis. DISCLAIMER: … Read more

Svetunkov & Sroginis (2025) – Model Based Demand Classification

Stockouts detection algorithm, Figure 2 from the paper https://doi.org/10.48550/arXiv.2504.05894

For the last year, Anna Sroginis and I have been working on a paper, trying to modernise demand classification schemes and make them useful in the brave new era of machine learning. We have finally wrapped it up and submitted it to a peer-reviewed journal. But the temptation to share was too strong, so we … Read more

Point Forecast Evaluation: State of the Art

I have summarised several posts on point forecasts evaluation in an article for the Foresight journal. Mike Gilliland, being the Editor-in-Chief of the journal, contributed to the paper a lot, making it read much smoother, but preferred not to be included as the co-author. This article was recently published in the issue 74 for Q3:2024. … Read more

Staying Positive: Challenges and Solutions in Using Pure Multiplicative ETS Models

Authors: Ivan Svetunkov, John E. Boylan Journal: IMA Journal of Management Mathematics Abstract: Exponential smoothing in state space form (ETS) is a popular forecasting technique, widely used in research and practice. While the additive error ETS models have been well studied, the multiplicative error ones have received much less attention in forecasting literature. Still, these … Read more

iETS: State space model for intermittent demand forecasting

Authors: Ivan Svetunkov, John E. Boylan Journal: International Journal of Production Economics Abstract: Inventory decisions relating to items that are demanded intermittently are particularly challenging. Decisions relating to termination of sales of product often rely on point estimates of the mean demand, whereas replenishment decisions depend on quantiles from interval estimates. It is in this … Read more

Multi-step Estimators and Shrinkage Effect in Time Series Models

Authors: Ivan Svetunkov, Nikos Kourentzes, Rebecca Killick Journal: Computational Statistics Abstract: Many modern statistical models are used for both insight and prediction when applied to data. When models are used for prediction one should optimise parameters through a prediction error loss function. Estimation methods based on multiple steps ahead forecast errors have been shown to … Read more

Story of “Probabilistic forecasting of hourly emergency department arrivals”

The paper Back in 2020, when we were all siting in the COVID lockdown, I had a call with Bahman Rostami-Tabar to discuss one of our projects. He told me that he had an hourly data of an Emergency Department from a hospital in Wales, and suggested writing a paper for a healthcare audience to … Read more