Don’t forget about bias!

So far, we’ve discussed forecasts evaluation, focusing on the precision of point forecasts. However, there are many other dimensions in the evaluation that can provide useful information about your model’s performance. One of them is bias, which we’ll explore today. Introduction But before that, why should we bother with bias? Research suggests that bias is […]

Avoid using MAPE!

Frankly speaking, I didn’t see the point in discussing MAPE when I wrote recent posts on error measures. However, I’ve received several comments and messages from data scientists and demand planners asking for clarification. So, here it is. TL;DR: Avoid using MAPE! MAPE, or Mean Absolute Percentage Error, is a still-very-popular-in-practice error measure, which is […]

What does “lower error measure” really mean?

“My amazing forecasting method has a lower MASE than any other method!” You’ve probably seen claims like this on social media or in papers. But have you ever thought about what it really means? Many forecasting experiments come to applying several approaches to a dataset, calculating error measures for each method per time series and […]

Error Measures Flow Chart

In order to help master students of Lancaster University Managemen Science department, I have developed a flow chart, that acts as a basic guide on what error measures to use in different circumstances. This is not a complete and far from perfect flow chart, and it assumes that the decision maker knows what intermittent demand […]