Chapter 17 Uncertainty about the model form
In this Chapter, we discuss more advanced topics related to regression modelling. In a way, this part builds upon elements of Statistical Learning (see, for example, the textbook of Hastie et al., 2009) and focuses on how to select variables for regression model. We start with a fundamental idea of bias-variance tradeoff, which lies in the core of many selection methods. We then move to the discussion of information criteria, explaining what they imply, after that - to several existing variable selection approaches, explaining their advantages and limitations. Furthermore, we discuss combination approaches and what they mean in terms of parameters of models. We finish this chapter with an introductory discussion of regularisation techniques (such as LASSO and RIDGE).