This book is in Open Review. I want your feedback to make the book better for you and other readers. To add your annotation, select some text and then click the on the pop-up menu. To see the annotations of others, click the button in the upper right hand corner of the page

12.1 Confidence intervals

In order to take this uncertainty into account, we could construct confidence intervals for the estimates of parameters, using the principles discussed in Section 6.4. This way we would hopefully have some idea about the uncertainty of the parameters, and not just rely on average values. If we assume that CLT holds, we could use the t statistics for the calculation of the quantiles of distribution (we need to use t because we do not know the variance of estimates of parameters). But in order to do that, we need to have variances of estimates of parameters. One of possible ways of getting them would be the bootstrap used in the example above. However, this is a computationally expensive operation, and there is a more efficient procedure, which however only works with linear regression models either estimated using OLS or via Maximum Likelihood Estimation assuming Normal distribution (see Section 16). In these conditions the covariance matrix of parameters can be calculated using the following formula: \[\begin{equation} \mathrm{V}(\hat{\boldsymbol{\beta}}) = \frac{1}{n-k} \sum_{j=1}^n e_j^2 \times \left(\mathbf{X}' \mathbf{X}\right)^{-1}. \tag{12.1} \end{equation}\] This matrix will contain variances of parameters on the diagonal and covariances between the parameters on off-diagonals. In this specific case, we only need the diagonal elements. We can take square root of them to obtain standard errors of parameters, which can then be used to construct confidence intervals for each parameter \(i\) via: \[\begin{equation} \beta_i \in (b_i + t_{\alpha/2}(n-k) s_{b_i}, b_i + t_{1-\alpha/2}(n-k) s_{b_i}), \tag{12.2} \end{equation}\] where \(s_{b_i}\) is the standard error of the parameter \(b_i\). All modern software does all these calculations automatically, so we do not need to do them manually. Here is an example:

##             (Intercept)      speed
## (Intercept)   45.676514 -2.6588234
## speed         -2.658823  0.1726509

This is the covariance matrix of parameters, the diagonal elements of which are then used in the confint() method:

##                  S.E.       2.5%     97.5%
## (Intercept) 6.7584402 -31.167850 -3.990340
## speed       0.4155128   3.096964  4.767853

The confidence interval for speed above shows, for example, that if we repeat the construction of interval many times, the true value of parameter speed will lie in 95% of cases between 3.08 and 4.78. This gives an idea about the real effect in the population. We can also present all of this in the following summary (this is based on the alm() model, the other functions will produce different summaries):

## Response variable: dist
## Distribution used in the estimation: Normal
## Loss function used in estimation: MSE
## Coefficients:
##             Estimate Std. Error Lower 2.5% Upper 97.5%  
## (Intercept) -17.5791     6.7584   -31.1678     -3.9903 *
## speed         3.9324     0.4155     3.0970      4.7679 *
## Error standard deviation: 15.3796
## Sample size: 50
## Number of estimated parameters: 2
## Number of degrees of freedom: 48
## Information criteria:
##      AIC     AICc      BIC     BICc 
## 417.1569 417.4122 420.9809 421.4803

This summary provide all the necessary information about the estimates of parameters: their mean values in the column “Estimate”, their standard errors in “Std. Error”, the bounds of confidence interval and finally a star if the interval does not contain zero. This typically indicates that we are certain on the selected confidence level (95% in our example) about the sign of the parameter and that the effect really exists.