Chapter 17 Uncertainty about the model form

In this Chapter, we discuss more advanced topics related to regression modelling. In a way, this part builds upon elements of Statistical Learning (see, for example, the textbook of Hastie et al., 2009) and focuses on how to select variables for regression model. We start with a fundamental idea of bias-variance trade-off, which lies in the core of many selection methods. We then move to the discussion of information criteria, explaining what they imply, after that - to several existing variable selection approaches, explaining their advantages and limitations. Furthermore, we discuss combination approaches and what they mean in terms of parameters of models. We finish this chapter with an introductory discussion of regularisation techniques (such as LASSO and RIDGE).

References

• Hastie, T., Tibshirani, R., Friedman, J., 2009. The Elements of Statistical Learning, Springer series in statistics. Springer New York, New York, NY. https://doi.org/10.1007/978-0-387-84858-7

The software behind this book. The methods described here are implemented in the greybox package for R and Python, which are free and open source.

Want to learn this with us? We teach these methods to practitioners on Demand Forecasting Principles, a four-week online course, and run other courses in forecasting, statistics and analytics.