From the conclusions article in the M4 competition IJF special issue:
“The forecasting spring began with the M4 Competition, where a complex hybrid approach combining statistical and ML elements came first, providing a 9.4% improvement in its sMAPE relative to that of the Comb benchmark.”
Makridakis and Petropoulos (2020)

Hi Vangelis,

Thank you for you comment.

Indeed, we do agree to disagree. We don’t need to have the same opinion on the topic, and I’m just expressing mine based on the observations I made. I know that you have a reasonable view on the problem, although I’m not sure that Spyros shares your opinion on the topic.

Anyway, good luck with M5!

Glad you like the set-up of M5. We also think it is reasonable and that it represents reality, at least as much as a competition allows in order for it to be relatively simple and comprehensible. I think M5 has a lot of potential and I am looking forward to reviewing its results.

Regarding M4, I think we’ve already “agreed we disagree” in a previous post of yours about both the accuracy measures used in the competition and the way its results were interpreted.

I won’t comment on the M4 measures here as I’ve already done that in your previous post.
I would just like to make it clear that the finding of M4 was not that “ML does not work”. It was that pure ML methods, trained in a series by series fashion, are less accurate than traditional, statistical approaches. We have highlighted however the importance of exploiting ML elements for applying cross-learning, i.e., learning from multiple series to predict the individual ones. In fact, this is what Slawek (1st place) and Pablo (2nd place) did and this is exactly why they managed to win the competition. If it wasn’t for ML algorithms, cross-learning wouldn’t be possible to apply.

You claim that the setting of the competition itself was not suitable for ML in the first place because you cannot expect non-linearity in yearly, quarterly or monthly series, which are also relatively short to allow effective training. This is not true. ML methods do not have to be necessarily fitted to each series individually. Those who did that, failed to get a high score, but the ones that trained their models across all 100,000 series of the competition got the highest scores. It is not about having long, non-linear data. It is about finding relationships between the series. Also, it is not about using ML methods. It is about the way you use them. Is this a super car or a super bike? I don’t know, but it works nice.

In my point of view, M4 promoted the utilization of ML in time series forecasting, showing how ML methods should be used to extract information from multiple series. Its large data-set allowed for such experimentation and lot of research is done in this area, expanding from ML to deep learning (see the excellent work done by Boris and his colleagues here https://arxiv.org/abs/1905.10437). Personally, I love ML and I don’t have anything against its use. I am also pretty sure that the winner of M5 will utilize a ML-based method.

Finally, I would like you to know that I appreciate your current workload, making it difficult for you to participate. On the other hand, given that CMAF is the biggest forecasting center in the UK, it would be reasonable for some of its members to participate (maybe some PhD students under the supervision of yours and other senior researchers?). M5 would be an excellent opportunity for forecasters to do some forecasting and experiment with the tools they’ve been developing for such purposes.