ANOVA Calculations in Multiple Linear Regression: Difference between revisions
(Created page with 'ANOVA Calculations in Multiple Linear Regression The sum of squares for the analysis of variance in multiple linear regression is obtained using the same relations as those in s…') |
No edit summary |
||
| Line 1: | Line 1: | ||
ANOVA Calculations in Multiple Linear Regression | =Reference Appendix A: ANOVA Calculations in Multiple Linear Regression= | ||
The sum of squares for the analysis of variance in multiple linear regression is obtained using the same relations as those in simple linear regression, except that the matrix notation is preferred in the case of multiple linear regression. In the case of both the simple and multiple linear regression models, once the observed and fitted values are available, the sum of squares are calculated in an identical manner. The difference between the two models lies in the way the fitted values are obtained. In a simple linear regression model, the fitted values are obtained from a model having only one predictor variable. In multiple linear regression analysis, the model used to obtained the fitted values contains more than one predictor variable. | The sum of squares for the analysis of variance in multiple linear regression is obtained using the same relations as those in simple linear regression, except that the matrix notation is preferred in the case of multiple linear regression. In the case of both the simple and multiple linear regression models, once the observed and fitted values are available, the sum of squares are calculated in an identical manner. The difference between the two models lies in the way the fitted values are obtained. In a simple linear regression model, the fitted values are obtained from a model having only one predictor variable. In multiple linear regression analysis, the model used to obtained the fitted values contains more than one predictor variable. | ||
==Total Sum of Squares== | |||
Recall from Chapter 4 on simple linear regression that the total sum of squares, , is obtained using the following equation: | Recall from Chapter 4 on simple linear regression that the total sum of squares, , is obtained using the following equation: | ||
| Line 20: | Line 21: | ||
Model Sum of Squares | ==Model Sum of Squares== | ||
Similarly, the model sum of squares or the regression sum of squares, , can be obtained in matrix notation as: | Similarly, the model sum of squares or the regression sum of squares, , can be obtained in matrix notation as: | ||
| Line 26: | Line 28: | ||
where is the hat matrix and is calculated using . | where is the hat matrix and is calculated using . | ||
Error Sum of Squares | ==Error Sum of Squares== | ||
The error sum of squares or the residual sum of squares, , is obtained in the matrix notation from the vector of residuals, , as: | The error sum of squares or the residual sum of squares, , is obtained in the matrix notation from the vector of residuals, , as: | ||
Mean Squares | ==Mean Squares== | ||
Mean squares are obtained by dividing the sum of squares with their associated degrees of freedom. The number of degrees of freedom associated with the total sum of squares, , is () since there are observations in all, but one degree of freedom is lost in the calculation of the sample mean, . The total mean square is: | Mean squares are obtained by dividing the sum of squares with their associated degrees of freedom. The number of degrees of freedom associated with the total sum of squares, , is () since there are observations in all, but one degree of freedom is lost in the calculation of the sample mean, . The total mean square is: | ||
| Line 45: | Line 49: | ||
Calculation of the Statistic | ==Calculation of the Statistic== | ||
Once the mean squares and are known, the statistic to test the significance of regression can be calculated as follows: | Once the mean squares and are known, the statistic to test the significance of regression can be calculated as follows: | ||
Revision as of 00:12, 2 July 2011
Reference Appendix A: ANOVA Calculations in Multiple Linear Regression
The sum of squares for the analysis of variance in multiple linear regression is obtained using the same relations as those in simple linear regression, except that the matrix notation is preferred in the case of multiple linear regression. In the case of both the simple and multiple linear regression models, once the observed and fitted values are available, the sum of squares are calculated in an identical manner. The difference between the two models lies in the way the fitted values are obtained. In a simple linear regression model, the fitted values are obtained from a model having only one predictor variable. In multiple linear regression analysis, the model used to obtained the fitted values contains more than one predictor variable.
Total Sum of Squares
Recall from Chapter 4 on simple linear regression that the total sum of squares, , is obtained using the following equation:
The first term, , can be expressed in matrix notation using the vector of observed values, , as:
If represents an square matrix of ones, then the second term, , can be expressed in matrix notation as:
Therefore, the total sum of squares in matrix notation is:
(31)
where is the identity matrix of order .
Model Sum of Squares
Similarly, the model sum of squares or the regression sum of squares, , can be obtained in matrix notation as:
where is the hat matrix and is calculated using .
Error Sum of Squares
The error sum of squares or the residual sum of squares, , is obtained in the matrix notation from the vector of residuals, , as:
Mean Squares
Mean squares are obtained by dividing the sum of squares with their associated degrees of freedom. The number of degrees of freedom associated with the total sum of squares, , is () since there are observations in all, but one degree of freedom is lost in the calculation of the sample mean, . The total mean square is:
The number of degrees of freedom associated with the regression sum of squares, , is . There are () degrees of freedom associated with a regression model with () coefficients, , , .... However, one degree of freedom is lost because the deviations, , are subjected to the constraints that they must sum to zero (). The regression mean square is:
The number of degrees of freedom associated with the error sum of squares is (), as there are observations in all, but () degrees of freedom are lost in obtaining the estimates of , , ... to calculate the predicted values, . The error mean square is:
The error mean square, , is an estimate of the variance, , of the random error terms, .
Calculation of the Statistic
Once the mean squares and are known, the statistic to test the significance of regression can be calculated as follows: