ANOVA for Designed Experiments

In Simple Linear Regression Analysis and Multiple Linear Regression Analysis, methods were presented to model the relationship between a response and the associated factors (referred to as predictor variables in the context of regression) based on an observed data set. Such studies, where observed values of the response are used to establish an association between the response and the factors, are called observational studies. However, in the case of observational studies, it is difficult to establish a cause-and-effect relationship between the observed factors and the response. This is because a number of alternative justifications can be used to explain the observed change in the response values. For example, a regression model fitted to data on the population of cities and road accidents might show a positive regression relation. However, this relation does not imply that an increase in a city's population causes an increase in road accidents. It could be that a number of other factors such as road conditions, traffic control and the degree to which the residents of the city follow the traffic rules affect the number of road accidents in the city and the increase in the number of accidents seen in the study is caused by these factors. Since the observational study does not take the effect of these factors into account, the assumption that an increase in a city's population will lead to an increase in road accidents is not a valid one. For example, the population of a city may increase but road accidents in the city may decrease because of better traffic control. To establish a cause-and-effect relationship, the study should be conducted in such a way that the effect of all other factors is excluded from the investigation.

The studies that enable the establishment of a cause-and-effect relationship are called experiments. In experiments the response is investigated by studying only the effect of the factor(s) of interest and excluding all other effects that may provide alternative justifications to the observed change in response. This is done in two ways. First, the levels of the factors to be investigated are carefully selected and then strictly controlled during the execution of the experiment. The aspect of selecting what factor levels should be investigated in the experiment is called the design of the experiment. The second distinguishing feature of experiments is that observations in an experiment are recorded in a random order. By doing this, it is hoped that the effect of all other factors not being investigated in the experiment will get cancelled out so that the change in the response is the result of only the investigated factors. Using these two techniques, experiments tend to ensure that alternative justifications to observed changes in the response are voided, thereby enabling the establishment of a cause-and-effect relationship between the response and the investigated factors.

Randomization

The aspect of recording observations in an experiment in a random order is referred to as randomization. Specifically, randomization is the process of assigning the various levels of the investigated factors to the experimental units in a random fashion. An experiment is said to be completely randomized if the probability of an experimental unit to be subjected to any level of a factor is equal for all the experimental units. The importance of randomization can be illustrated using an example. Consider an experiment where the effect of the speed of a lathe machine on the surface finish of a product is being investigated. In order to save time, the experimenter records surface finish values by running the lathe machine continuously and recording observations in the order of increasing speeds. The analysis of the experiment data shows that an increase in lathe speeds causes a decrease in the quality of surface finish. However the results of the experiment are disputed by the lathe operator who claims that he has been able to obtain better surface finish quality in the products by operating the lathe machine at higher speeds. It is later found that the faulty results were caused because of overheating of the tool used in the machine. Since the lathe was run continuously in the order of increased speeds the observations were recorded in the order of increased tool temperatures. This problem could have been avoided if the experimenter had randomized the experiment and taken reading at the various lathe speeds in a random fashion. This would require the experimenter to stop and restart the machine at every observation, thereby keeping the temperature of the tool within a reasonable range. Randomization would have ensured that the effect of heating of the machine tool is not included in the experiment.

Analysis of Single Factor Experiments

As explained in Simple Linear Regression Analysis and Multiple Linear Regression Analysis, the analysis of observational studies involves the use of regression models. The analysis of experimental studies involves the use of analysis of variance (ANOVA) models. For a comparison of the two models see Fitting ANOVA Models. In single factor experiments, ANOVA models are used to compare the mean response values at different levels of the factor. Each level of the factor is investigated to see if the response is significantly different from the response at other levels of the factor. The analysis of single factor experiments is often referred to as one-way ANOVA.

To illustrate the use of ANOVA models in the analysis of experiments, consider a single factor experiment where the analyst wants to see if the surface finish of certain parts is affected by the speed of a lathe machine. Data is collected for three speeds (or three treatments). Each treatment is replicated four times. Therefore, this experiment design is balanced. Surface finish values recorded using randomization are shown in the following table.

Surface finish values for three speeds of a lathe machine.

The ANOVA model for this experiment can be stated as follows:

$Y_{ij}={\mu }_i+{ϵ}_{ij}$

The ANOVA model assumes that the response at each factor level, $i$, is the sum of the mean response at the $i$th level, ${\mu }_i$, and a random error term, ${ϵ}_{ij}$. The subscript $i$ denotes the factor level while the subscript $j$ denotes the replicate. If there are $n_a$ levels of the factor and $m$ replicates at each level then $i=1,2,...,n_a$ and $j=1,2,...,m$. The random error terms, ${ϵ}_{ij}$, are assumed to be normally and independently distributed with a mean of zero and variance of ${\sigma }^2$. Therefore, the response at each level can be thought of as a normally distributed population with a mean of ${\mu }_i$ and constant variance of ${\sigma }^2$. The equation given above is referred to as the means model.

The ANOVA model of the means model can also be written using ${\mu }_i=\mu +{\tau }_i$, where $\mu$ represents the overall mean and ${\tau }_i$ represents the effect due to the $i$th treatment.

$Y_{ij}=\mu +{\tau }_i+{ϵ}_{ij}$

Such an ANOVA model is called the effects model. In the effects models the treatment effects, ${\tau }_i$, represent the deviations from the overall mean, $\mu$. Therefore, the following constraint exists on the ${\tau }_i$s:

$\underset{i=1}{\sum ^{n_a}}{\tau }_i=0$

Fitting ANOVA Models

To fit ANOVA models and carry out hypothesis testing in single factor experiments, it is convenient to express the effects model of the effects model in the form $y=X\beta +ϵ$ (that was used for multiple linear regression models in Multiple Linear Regression Analysis). This can be done as shown next. Using the effects model, the ANOVA model for the single factor experiment in the first table can be expressed as:

$Y_{ij}=\mu +{\tau }_i+{ϵ}_{ij}$

where $\mu$ represents the overall mean and ${\tau }_i$ represents the $i$th treatment effect. There are three treatments in the first table (500, 600 and 700). Therefore, there are three treatment effects, ${\tau }_1$, ${\tau }_2$ and ${\tau }_3$. The following constraint exists for these effects:

$\begin{array}{rl}\underset{i=1}{\sum ^3}{\tau }_i=& 0\\ \text{or }{\tau }_1+{\tau }_2+{\tau }_3=& 0\end{array}$

For the first treatment, the ANOVA model for the single factor experiment in the above table can be written as:

$Y_{1j}=\mu +{\tau }_1+0\cdot {\tau }_2+0\cdot {\tau }_3+{ϵ}_{1j}$

Using ${\tau }_3=-({\tau }_1+{\tau }_2)$, the model for the first treatment is:

$\begin{array}{rl}{Y}_{1j}=& \mu +{\tau }_1+0\cdot {\tau }_2-0\cdot ({\tau }_1+{\tau }_2)+{ϵ}_{1j}\\ \text{or }{Y}_{1j}=& \mu +{\tau }_1+0\cdot {\tau }_2+{ϵ}_{1j}\end{array}$

Models for the second and third treatments can be obtained in a similar way. The models for the three treatments are:

$\begin{array}{rl}\text{First Treatment}:& Y_{1j}=1\cdot \mu +1\cdot {\tau }_1+0\cdot {\tau }_2+{ϵ}_{1j}\\ \text{Second Treatment}:& Y_{2j}=1\cdot \mu +0\cdot {\tau }_1+1\cdot {\tau }_2+{ϵ}_{2j}\\ \text{Third Treatment}:& Y_{3j}=1\cdot \mu -1\cdot {\tau }_1-1\cdot {\tau }_2+{ϵ}_{3j}\end{array}$

The coefficients of the treatment effects ${\tau }_1$ and ${\tau }_2$ can be expressed using two indicator variables, $x_1$ and $x_2$, as follows:

$\begin{array}{rl}\text{Treatment Effect }{\tau }_1:& x_1=1,x_2=0\\ \text{Treatment Effect }{\tau }_2:& x_1=0,x_2=1\\ \text{Treatment Effect }{\tau }_3:& x_1=-1,x_2=-1\end{array}$

Using the indicator variables $x_1$ and $x_2$, the ANOVA model for the data in the first table now becomes:

$Y=\mu +x_1\cdot {\tau }_1+x_2\cdot {\tau }_2+ϵ$

The equation can be rewritten by including subscripts $i$ (for the level of the factor) and $j$ (for the replicate number) as:

$Y_{ij}=\mu +x_{i1}\cdot {\tau }_1+x_{i2}\cdot {\tau }_2+{ϵ}_{ij}$

The equation given above represents the "regression version" of the ANOVA model.

Comparison of ANOVA and Regression Models

It can be seen from the equation given above that in an ANOVA model each factor is treated as a qualitative factor. In the present example the factor, lathe speed, is a quantitative factor with three levels. But the ANOVA model treats this factor as a qualitative factor with three levels. Therefore, two indicator variables, $x_1$ and $x_2$, are required to represent this factor. Note that if a regression model were to be fitted to the data in the first table then a distinction is made between quantitative and qualitative factors. For regression models, the factor, lathe speed, would be used as a quantitative factor and represented with a single predictor variable. For example, if a first order regression model were to be fitted to the data in the first table, then the regression model would take the form $Y_{ij}={\beta }_0+{\beta }_1{x}_{i1}+{ϵ}_{ij}$. If a second order regression model were to be fitted, the regression model would be $Y_{ij}={\beta }_0+{\beta }_1{x}_{i1}+{\beta }_2{x}_{i1}^2+{ϵ}_{ij}$. Notice that unlike these regression models, the regression version of the ANOVA model does not make any assumption about the nature of relationship between the response and the factor being investigated.

The choice of the two models for a particular data set depends on the objective of the experimenter. In the case of the data of the first table, the objective of the experimenter is to compare the levels of the factor to see if change in the levels leads to a significant change in the response. The objective is not to make predictions on the response for a given level of the factor. Therefore, the ANOVA model is used in this case. If the objective of the experimenter were prediction or optimization, the experimenter would use a regression model and focus on aspects such as the nature of relationship between the factor, lathe speed, and the response, surface finish, so that the regression model obtained is able to make accurate predictions.

Expression of the ANOVA Model in the Form $y=X\beta +ϵ$

The regression version of the ANOVA model can be expanded for the three treatments and four replicates of the data in the first table as follows:

$\begin{array}{rl}{Y}_{11}=& 6=\mu +1\cdot {\tau }_1+0\cdot {\tau }_2+{ϵ}_{11}\text{ Level 1, Replicate 1}\\ Y_{21}=& 13=\mu +0\cdot {\tau }_1+1\cdot {\tau }_2+{ϵ}_{21}\text{ Level 2, Replicate 1}\\ Y_{31}=& 23=\mu -1\cdot {\tau }_1-1\cdot {\tau }_2+{ϵ}_{31}\text{ Level 3, Replicate 1}\\ Y_{12}=& 13=\mu +1\cdot {\tau }_1+0\cdot {\tau }_2+{ϵ}_{12}\text{ Level 1, Replicate 2}\\ Y_{22}=& 16=\mu +0\cdot {\tau }_1+1\cdot {\tau }_2+{ϵ}_{22}\text{ Level 2, Replicate 2}\\ Y_{32}=& 20=\mu -1\cdot {\tau }_1-1\cdot {\tau }_2+{ϵ}_{32}\text{ Level 3, Replicate 2}\\ & & ...\\ Y_{34}=& 18=\mu -1\cdot {\tau }_1-1\cdot {\tau }_2+{ϵ}_{34}\text{ Level 3, Replicate 4}\end{array}$

The corresponding matrix notation is:

$y=X\beta +ϵ$

where

$y=\left[\begin{array}{c}{Y}_{11}\\ Y_{21}\\ Y_{31}\\ Y_{12}\\ Y_{22}\\ .\\ .\\ .\\ Y_{34}\end{array}\right]=X\beta +ϵ=\left[\begin{array}{ccc}1& 1& 0\\ 1& 0& 1\\ 1& -1& -1\\ 1& 1& 0\\ 1& 0& 1\\ .& .& .\\ .& .& .\\ .& .& .\\ 1& -1& -1\end{array}\right]\left[\begin{array}{c}\mu \\ {\tau }_1\\ {\tau }_2\end{array}\right]+\left[\begin{array}{c}{ϵ}_{11}\\ {ϵ}_{21}\\ {ϵ}_{31}\\ {ϵ}_{12}\\ {ϵ}_{22}\\ .\\ .\\ .\\ {ϵ}_{34}\end{array}\right]$

Thus:

$\begin{array}{rl}y=& X\beta +ϵ\\ & & \\ & \left[\begin{array}{c}6\\ 13\\ 23\\ 13\\ 16\\ .\\ .\\ .\\ 18\end{array}\right]=& \left[\begin{array}{ccc}1& 1& 0\\ 1& 0& 1\\ 1& -1& -1\\ 1& 1& 0\\ 1& 0& 1\\ .& .& .\\ .& .& .\\ .& .& .\\ 1& -1& -1\end{array}\right]\left[\begin{array}{c}\mu \\ {\tau }_1\\ {\tau }_2\end{array}\right]+\left[\begin{array}{c}{ϵ}_{11}\\ {ϵ}_{21}\\ {ϵ}_{31}\\ {ϵ}_{12}\\ {ϵ}_{22}\\ .\\ .\\ .\\ {ϵ}_{34}\end{array}\right]\end{array}$

The matrices $y$, $X$ and $\beta$ are used in the calculation of the sum of squares in the next section. The data in the first table can be entered into DOE++ as shown in the figure below.

Single factor experiment design for the data in the first table.

Hypothesis Test in Single Factor Experiments

The hypothesis test in single factor experiments examines the ANOVA model to see if the response at any level of the investigated factor is significantly different from that at the other levels. If this is not the case and the response at all levels is not significantly different, then it can be concluded that the investigated factor does not affect the response. The test on the ANOVA model is carried out by checking to see if any of the treatment effects, ${\tau }_i$, are non-zero. The test is similar to the test of significance of regression mentioned in Simple Linear Regression Analysis and Multiple Linear Regression Analysis in the context of regression models. The hypotheses statements for this test are:

$\begin{array}{rlr}& H_0:& {\tau }_1={\tau }_2=...={\tau }_{n_a}=0\\ & H_1:& {\tau }_i\ne 0\text{ for at least one }i\end{array}$

The test for $H_0$ is carried out using the following statistic:

$F_0=\frac{M{S}_{TR}}{M{S}_E}$

where $M{S}_{TR}$ represents the mean square for the ANOVA model and $M{S}_E$ is the error mean square. Note that in the case of ANOVA models we use the notation $M{S}_{TR}$ (treatment mean square) for the model mean square and $S{S}_{TR}$ (treatment sum of squares) for the model sum of squares (instead of $M{S}_R$, regression mean square, and $S{S}_R$, regression sum of squares, used in Simple Linear Regression Analysis and Multiple Linear Regression Analysis). This is done to indicate that the model under consideration is the ANOVA model and not the regression model. The calculations to obtain $M{S}_{TR}$ and $S{S}_{TR}$ are identical to the calculations to obtain $M{S}_R$ and $S{S}_R$ explained in Multiple Linear Regression Analysis.

Calculation of the Statistic $F_0$

The sum of squares to obtain the statistic $F_0$ can be calculated as explained in Multiple Linear Regression Analysis. Using the data in the first table, the model sum of squares, $S{S}_{TR}$, can be calculated as:

$\begin{array}{rl}S{S}_{TR}=& y^{\prime }[H-(\frac{1}{n_a\cdot m})J]y\\ =& {\left[\begin{array}{c}6\\ 13\\ .\\ .\\ 18\end{array}\right]}^{\prime }\left[\begin{array}{ccccc}0.1667& -0.0833& .& .& -0.0833\\ -0.0833& 0.1667& .& .& -0.0833\\ .& .& .& .& .\\ .& .& .& .& .\\ -0.0833& -0.0833& .& .& 0.1667\end{array}\right]\left[\begin{array}{c}6\\ 13\\ .\\ .\\ 18\end{array}\right]\\ =& 232.1667\end{array}$

In the previous equation, $n_a$ represents the number of levels of the factor, $m$ represents the replicates at each level, $y$ represents the vector of the response values, $H$ represents the hat matrix and $J$ represents the matrix of ones. (For details on each of these terms, refer to Multiple Linear Regression Analysis.) Since two effect terms, ${\tau }_1$ and ${\tau }_2$, are used in the regression version of the ANOVA model, the degrees of freedom associated with the model sum of squares, $S{S}_{TR}$, is two.

$dof(S{S}_{TR})=2$

The total sum of squares, $S{S}_T$, can be obtained as follows:

$\begin{array}{rl}S{S}_T=& y^{\prime }[I-(\frac{1}{n_a\cdot m})J]y\\ =& {\left[\begin{array}{c}6\\ 13\\ .\\ .\\ 18\end{array}\right]}^{\prime }\left[\begin{array}{ccccc}0.9167& -0.0833& .& .& -0.0833\\ -0.0833& 0.9167& .& .& -0.0833\\ .& .& .& .& .\\ .& .& .& .& .\\ -0.0833& -0.0833& .& .& 0.9167\end{array}\right]\left[\begin{array}{c}6\\ 13\\ .\\ .\\ 18\end{array}\right]\\ =& 306.6667\end{array}$

In the previous equation, $I$ is the identity matrix. Since there are 12 data points in all, the number of degrees of freedom associated with $S{S}_T$ is 11.

$dof(S{S}_T)=11$

Knowing $S{S}_T$ and $S{S}_{TR}$, the error sum of squares is:

$\begin{array}{rl}S{S}_E=& S{S}_T-S{S}_{TR}\\ =& 306.6667-232.1667\\ =& 74.5\end{array}$

The number of degrees of freedom associated with $S{S}_E$ is:

$\begin{array}{rl}dof(S{S}_E)=& dof(S{S}_T)-dof(S{S}_{TR})\\ =& 11-2\\ =& 9\end{array}$

The test statistic can now be calculated using the equation given in Hypothesis Test in Single Factor Experiments as:

$\begin{array}{rl}{f}_0=& \frac{M{S}_{TR}}{M{S}_E}\\ =& \frac{S{S}_{TR}/dof(S{S}_{TR})}{S{S}_E/dof(S{S}_E)}\\ =& \frac{232.1667/2}{74.5/9}\\ =& 14.0235\end{array}$

The $p$ value for the statistic based on the $F$ distribution with 2 degrees of freedom in the numerator and 9 degrees of freedom in the denominator is:

$\begin{array}{rl}pvalue=& 1-P(F\le f_0)\\ =& 1-0.9983\\ =& 0.0017\end{array}$

Assuming that the desired significance level is 0.1, since $p$ value < 0.1, $H_0$ is rejected and it is concluded that change in the lathe speed has a significant effect on the surface finish. DOE++ displays these results in the ANOVA table, as shown in the figure below. The values of S and R-sq are the standard error and the coefficient of determination for the model, respectively. These values are explained in Multiple Linear Regression Analysis and indicate how well the model fits the data. The values in the figure below indicate that the fit of the ANOVA model is fair.

ANOVA table for the data in the first table.

Confidence Interval on the i^th Treatment Mean

The response at each treatment of a single factor experiment can be assumed to be a normal population with a mean of ${\mu }_i$ and variance of ${\sigma }^2$ provided that the error terms can be assumed to be normally distributed. A point estimator of ${\mu }_i$ is the average response at each treatment, ${\overline{y}}_{i\cdot }$. Since this is a sample average, the associated variance is ${\sigma }^2/m_i$, where $m_i$ is the number of replicates at the $i$th treatment. Therefore, the confidence interval on ${\mu }_i$ is based on the $t$ distribution. Recall from Statistical Background on DOE (inference on population mean when variance is unknown) that:

$\begin{array}{rl}{T}_0=& \frac{{\overline{y}}_{i\cdot }-{\mu }_i}{\sqrt{{\hat{\sigma }}^2/m_i}}\\ =& \frac{{\overline{y}}_{i\cdot }-{\mu }_i}{\sqrt{M{S}_E/m_i}}\end{array}$

has a $t$ distribution with degrees of freedom $=dof(S{S}_E)$. Therefore, a 100 ($1-\alpha$) percent confidence interval on the $i$th treatment mean, ${\mu }_i$, is:

${\overline{y}}_{i\cdot }\pm t_{\alpha /2,dof(S{S}_E)}\sqrt{\frac{M{S}_E}{m_i}}$

For example, for the first treatment of the lathe speed we have:

$\begin{array}{rl}{\hat{\mu }}_1=& {\overline{y}}_{1\cdot }\\ =& \frac{6+13+7+8}{4}\\ =& 8.5\end{array}$

In DOE++, this value is displayed as the Estimated Mean for the first level, as shown in the Data Summary table in the figure below. The value displayed as the standard deviation for this level is simply the sample standard deviation calculated using the observations corresponding to this level. The 90% confidence interval for this treatment is:

$\begin{array}{rl}=& {\overline{y}}_{1\cdot }\pm t_{\alpha /2,dof(S{S}_E)}\sqrt{\frac{M{S}_E}{m_i}}\\ =& {\overline{y}}_{1\cdot }\pm t_{0.05,9}\sqrt{\frac{M{S}_E}{4}}\\ =& 8.5\pm 1.833\sqrt{\frac{(74.5/9)}{4}}\\ =& 8.5\pm 1.833(1.44)\\ =& 8.5\pm 2.64\end{array}$

The 90% limits on ${\mu }_1$ are 5.9 and 11.1, respectively.

Data Summary table for the single factor experiment in the first table.

Confidence Interval on the Difference in Two Treatment Means

The confidence interval on the difference in two treatment means, ${\mu }_i-{\mu }_j$, is used to compare two levels of the factor at a given significance. If the confidence interval does not include the value of zero, it is concluded that the two levels of the factor are significantly different. The point estimator of ${\mu }_i-{\mu }_j$ is ${\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot }$. The variance for ${\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot }$ is:

$\begin{array}{rl}var({\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot })=& var({\overline{y}}_{i\cdot })+var({\overline{y}}_{j\cdot })\\ =& {\sigma }^2/m_i+{\sigma }^2/m_j\end{array}$

For balanced designs all $m_i=m$. Therefore:

$var({\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot })=2{\sigma }^2/m$

The standard deviation for ${\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot }$ can be obtained by taking the square root of $var({\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot })$ and is referred to as the pooled standard error:

$\begin{array}{rl}PooledStd.Error=& \sqrt{var({\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot })}\\ =& \sqrt{2{\hat{\sigma }}^2/m}\end{array}$

The $t$ statistic for the difference is:

$\begin{array}{rl}{T}_0=& \frac{{\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot }-({\mu }_i-{\mu }_j)}{\sqrt{2{\hat{\sigma }}^2/m}}\\ =& \frac{{\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot }-({\mu }_i-{\mu }_j)}{\sqrt{2M{S}_E/m}}\end{array}$

Then a 100 (1- $\alpha$) percent confidence interval on the difference in two treatment means, ${\mu }_i-{\mu }_j$, is:

${\overline{y}}_{i\cdot }-{\overline{y}}_{j\cdot }\pm t_{\alpha /2,dof(S{S}_E)}\sqrt{\frac{2M{S}_E}{m}}$

For example, an estimate of the difference in the first and second treatment means of the lathe speed, ${\mu }_1-{\mu }_2$, is:

$\begin{array}{rl}{\hat{\mu }}_1-{\hat{\mu }}_2=& {\overline{y}}_{1\cdot }-{\overline{y}}_{2\cdot }\\ =& 8.5-13.25\\ =& -4.75\end{array}$

The pooled standard error for this difference is:

$\begin{array}{rl}PooledStd.Error=& \sqrt{var({\overline{y}}_{1\cdot }-{\overline{y}}_{2\cdot })}\\ =& \sqrt{2{\hat{\sigma }}^2/m}\\ =& \sqrt{2M{S}_E/m}\\ =& \sqrt{\frac{2(74.5/9)}{4}}\\ =& 2.0344\end{array}$

To test $H_0:{\mu }_1-{\mu }_2=0$, the $t$ statistic is:

$\begin{array}{rl}{t}_0=& \frac{{\overline{y}}_{1\cdot }-{\overline{y}}_{2\cdot }-({\mu }_1-{\mu }_2)}{\sqrt{2M{S}_E/m}}\\ =& \frac{-4.75-(0)}{\sqrt{\frac{2(74.5/9)}{4}}}\\ =& \frac{-4.75}{2.0344}\\ =& -2.3348\end{array}$

In DOE++, the value of the statistic is displayed in the Mean Comparisons table under the column T Value as shown in the figure below. The 90% confidence interval on the difference ${\mu }_1-{\mu }_2$ is:

$\begin{array}{rl}=& {\overline{y}}_{1\cdot }-{\overline{y}}_{2\cdot }\pm t_{\alpha /2,dof(S{S}_E)}\sqrt{\frac{2M{S}_E}{m}}\\ =& {\overline{y}}_{1\cdot }-{\overline{y}}_{2\cdot }\pm t_{0.05,9}\sqrt{\frac{2M{S}_E}{m}}\\ =& -4.75\pm 1.833\sqrt{\frac{2(74.5/9)}{4}}\\ =& -4.75\pm 1.833(2.0344)\\ =& -4.75\pm 3.729\end{array}$

Hence the 90% limits on ${\mu }_1-{\mu }_2$ are $-8.479$ and $-1.021$, respectively. These values are displayed under the Low CI and High CI columns in the following figure. Since the confidence interval for this pair of means does not included zero, it can be concluded that these means are significantly different at 90% confidence. This conclusion can also be arrived at using the $p$ value noting that the hypothesis is two-sided. The $p$ value corresponding to the statistic $t_0=-2.3348$, based on the $t$ distribution with 9 degrees of freedom is:

$\begin{array}{rl}pvalue=& 2\times (1-P(T\le |t_0|))\\ =& 2\times (1-P(T\le 2.3348))\\ =& 2\times (1-0.9778)\\ =& 0.0444\end{array}$

Since $p$ value < 0.1, the means are significantly different at 90% confidence. Bounds on the difference between other treatment pairs can be obtained in a similar manner and it is concluded that all treatments are significantly different.

Residual Analysis

Plots of residuals, $e_{ij}$, similar to the ones discussed in the previous chapters on regression, are used to ensure that the assumptions associated with the ANOVA model are not violated. The ANOVA model assumes that the random error terms, ${ϵ}_{ij}$, are normally and independently distributed with the same variance for each treatment. The normality assumption can be checked by obtaining a normal probability plot of the residuals.

Mean Comparisons table for the data in the first table.

Equality of variance is checked by plotting residuals against the treatments and the treatment averages, ${\overline{y}}_{i\cdot }$ (also referred to as fitted values), and inspecting the spread in the residuals. If a pattern is seen in these plots, then this indicates the need to use a suitable transformation on the response that will ensure variance equality. Box-Cox transformations are discussed in the next section. To check for independence of the random error terms residuals are plotted against time or run-order to ensure that a pattern does not exist in these plots. Residual plots for the given example are shown in the following two figures. The plots show that the assumptions associated with the ANOVA model are not violated.

Normal probability plot of residuals for the single factor experiment in the first table.

Plot of residuals against fitted values for the single factor experiment in the first table.

Box-Cox Method

Transformations on the response may be used when residual plots for an experiment show a pattern. This indicates that the equality of variance does not hold for the residuals of the given model. The Box-Cox method can be used to automatically identify a suitable power transformation for the data based on the relation:

$Y^{\ast }=Y^{\lambda }$

$\lambda$ is determined using the given data such that $S{S}_E$ is minimized. The values of $Y^{\lambda }$ are not used as is because of issues related to calculation or comparison of $S{S}_E$ values for different values of $\lambda$. For example, for $\lambda =0$ all response values will become 1. Therefore, the following relation is used to obtain $Y^{\lambda }$ :

$Y^{\lambda }=\{\begin{array}{c}\frac{Y^{\lambda }-1}{\lambda {\dot{y}}^{\lambda -1}}\\ \dot{y}\ln y\end{array}\begin{array}{c}\lambda \ne 0\begin{array}{c}\\ \end{array}\\ \lambda =0\end{array}$

where $\dot{y}={\ln }^{-1}[(1/n)\underset{}{\overset{}{}}\ln y]$. Once all $Y^{\lambda }$ values are obtained for a value of $\lambda$, the corresponding $S{S}_E$ for these values is obtained using $y^{\lambda \prime }[I-H]y^{\lambda }$. The process is repeated for a number of $\lambda$ values to obtain a plot of $S{S}_E$ against $\lambda$. Then the value of $\lambda$ corresponding to the minimum $S{S}_E$ is selected as the required transformation for the given data. DOE++ plots $\ln S{S}_E$ values against $\lambda$ values because the range of $S{S}_E$ values is large and if this is not done, all values cannot be displayed on the same plot. The range of search for the best $\lambda$ value in the software is from $-5$ to $5$, because larger values of of $\lambda$ are usually not meaningful. DOE++ also displays a recommended transformation based on the best $\lambda$ value obtained as per the second table.

Recommended Box-Cox power transformations.

Confidence intervals on the selected $\lambda$ values are also available. Let $S{S}_E(\lambda )$ be the value of $S{S}_E$ corresponding to the selected value of $\lambda$. Then, to calculate the 100 (1- $\alpha$) percent confidence intervals on $\lambda$, we need to calculate $S{S}^{\ast }$ as shown next:

$S{S}^{\ast }=S{S}_E(\lambda )(1+\frac{t_{\alpha /2,dof(S{S}_E)}^2}{dof(S{S}_E)})$

The required limits for $\lambda$ are the two values of $\lambda$ corresponding to the value $S{S}^{\ast }$ (on the plot of $S{S}_E$ against $\lambda$). If the limits for $\lambda$ do not include the value of one, then the transformation is applicable for the given data. Note that the power transformations are not defined for response values that are negative or zero. DOE++ deals with negative and zero response values using the following equations (that involve addition of a suitable quantity to all of the response values if a zero or negative response value is encountered).

$\begin{array}{rlr}& y(i)=& y(i)+\left|y_{min}\right|\times 1.1\text{ Negative Response}\\ & y(i)=& y(i)+1\text{ Zero Response}\end{array}$

Here $y_{min}$ represents the minimum response value and $\left|y_{min}\right|$ represents the absolute value of the minimum response.

Example

To illustrate the Box-Cox method, consider the experiment given in the first table. Transformed response values for various values of $\lambda$ can be calculated using the equation for $Y^{\lambda }$ given in Box-Cox Method. Knowing the hat matrix, $H$, $S{S}_E$ values corresponding to each of these $\lambda$ values can easily be obtained using $y^{\lambda \prime }[I-H]y^{\lambda }$. $S{S}_E$ values calculated for $\lambda$ values between $-5$ and $5$ for the given data are shown below:

$\begin{array}{cccc}\lambda & & S{S}_E& \ln S{S}_E\\ & & & \\ -5& & 5947.8& 8.6908\\ -4& & 1946.4& 7.5737\\ -3& & 696.5& 6.5461\\ -2& & 282.2& 5.6425\\ -1& & 135.8& 4.9114\\ 0& & 83.9& 4.4299\\ 1& & 74.5& 4.3108\\ 2& & 101.0& 4.6154\\ 3& & 190.4& 5.2491\\ 4& & 429.5& 6.0627\\ 5& & 1057.6& -6.9638\end{array}$

A plot of $\ln S{S}_E$ for various $\lambda$ values, as obtained from DOE++, is shown in the following figure. The value of $\lambda$ that gives the minimum $S{S}_E$ is identified as 0.7841. The $S{S}_E$ value corresponding to this value of $\lambda$ is 73.74. A 90% confidence interval on this $\lambda$ value is calculated as follows. $S{S}^{\ast }$ can be obtained as shown next:

$\begin{array}{rl}S{S}^{\ast }=& S{S}_E(\lambda )(1+\frac{t_{\alpha /2,dof(S{S}_E)}^2}{dof(S{S}_E)})\\ =& 73.74(1+\frac{t_{0.05,9}^2}{9})\\ =& 73.74(1+\frac{{1.833}^2}{9})\\ =& 101.27\end{array}$

Therefore, $\ln S{S}^{\ast }=4.6178$. The $\lambda$ values corresponding to this value from the following figure are $-0.4686$ and $2.0052$. Therefore, the 90% confidence limits on are $-0.4689$ and $2.0055$. Since the confidence limits include the value of 1, this indicates that a transformation is not required for the data in the first table.

Box-Cox power transformation plot for the data in the first table.

Experiments with Several Factors - Factorial Experiments

Experiments with two or more factors are encountered frequently. The best way to carry out such experiments is by using factorial experiments. Factorial experiments are experiments in which all combinations of factors are investigated in each replicate of the experiment. Factorial experiments are the only means to completely and systematically study interactions between factors in addition to identifying significant factors. One-factor-at-a-time experiments (where each factor is investigated separately by keeping all the remaining factors constant) do not reveal the interaction effects between the factors. Further, in one-factor-at-a-time experiments full randomization is not possible.

To illustrate factorial experiments consider an experiment where the response is investigated for two factors, $A$ and $B$. Assume that the response is studied at two levels of factor $A$ with $A_{\text{low}}$ representing the lower level of $A$ and $A_{\text{high}}$ representing the higher level of $A$. Similarly, let $B_{\text{low}}$ and $B_{\text{high}}$ represent the two levels of factor $B$ that are being investigated in this experiment. Since there are two factors with two levels, a total of $2\times 2=4$ combinations exist ($A_{\text{low}}$ - $B_{\text{low}}$, - $B_{\text{high}}$, $A_{\text{high}}$ - $B_{\text{low}}$, - $B_{\text{high}}$). Thus, four runs are required for each replicate if a factorial experiment is to be carried out in this case. Assume that the response values for each of these four possible combinations are obtained as shown in the third table.

Two-factor factorial experiment.

Interaction plot for the data in the third table.

Investigating Factor Effects

The effect of factor $A$ on the response can be obtained by taking the difference between the average response when $A$ is high and the average response when $A$ is low. The change in the response due to a change in the level of a factor is called the main effect of the factor. The main effect of $A$ as per the response values in the third table is:

$\begin{array}{rl}A=& Averageresponseat{A}_{\text{high}}-Averageresponseat{A}_{\text{low}}\\ =& \frac{45+55}{2}-\frac{25+35}{2}\\ =& 50-30\\ =& 20\end{array}$

Therefore, when $A$ is changed from the lower level to the higher level, the response increases by 20 units. A plot of the response for the two levels of $A$ at different levels of $B$ is shown in the figure above. The plot shows that change in the level of $A$ leads to an increase in the response by 20 units regardless of the level of $B$. Therefore, no interaction exists in this case as indicated by the parallel lines on the plot. The main effect of $B$ can be obtained as:

$\begin{array}{rl}B=& Averageresponseat{B}_{\text{high}}-Averageresponseat{B}_{\text{low}}\\ =& \frac{35+55}{2}-\frac{25+45}{2}\\ =& 45-35\\ =& 10\end{array}$

Investigating Interactions

Now assume that the response values for each of the four treatment combinations were obtained as shown in the fourth table. The main effect of $A$ in this case is:

$\begin{array}{rl}A=& Averageresponseat{A}_{\text{high}}-Averageresponseat{A}_{\text{low}}\\ =& \frac{40+10}{2}-\frac{20+30}{2}\\ =& 0\end{array}$

Two factor factorial experiment.

It appears that $A$ does not have an effect on the response. However, a plot of the response of $A$ at different levels of $B$ shows that the response does change with the levels of $A$ but the effect of $A$ on the response is dependent on the level of $B$ (see the figure below). Therefore, an interaction between $A$ and $B$ exists in this case (as indicated by the non-parallel lines of the figure). The interaction effect between $A$ and $B$ can be calculated as follows:

Interaction plot for the data in the fourth table.

$\begin{array}{rl}AB=& Averageresponseat{A}_{\text{high}}\text{-}{B}_{\text{high}}and{A}_{\text{low}}\text{-}{B}_{\text{low}}-\\ & Averageresponseat{A}_{\text{low}}\text{-}{B}_{\text{high}}and{A}_{\text{high}}\text{-}{B}_{\text{low}}\\ =& \frac{10+20}{2}-\frac{40+30}{2}\\ =& -20\end{array}$

Note that in this case, if a one-factor-at-a-time experiment were used to investigate the effect of factor $A$ on the response, it would lead to incorrect conclusions. For example, if the response at factor $A$ was studied by holding $B$ constant at its lower level, then the main effect of $A$ would be obtained as $40-20=20$, indicating that the response increases by 20 units when the level of $A$ is changed from low to high. On the other hand, if the response at factor $A$ was studied by holding $B$ constant at its higher level than the main effect of $A$ would be obtained as $10-30=-20$, indicating that the response decreases by 20 units when the level of $A$ is changed from low to high.

Analysis of General Factorial Experiments

In DOE++, factorial experiments are referred to as factorial designs. The experiments explained in this section are referred to as general factorial designs. This is done to distinguish these experiments from the other factorial designs supported by DOE++ (see the figure below). The other designs (such as the 2 level full factorial designs that are explained in Two Level Factorial Experiments) are special cases of these experiments in which factors are limited to a specified number of levels. The ANOVA model for the analysis of factorial experiments is formulated as shown next. Assume a factorial experiment in which the effect of two factors, $A$ and $B$, on the response is being investigated. Let there be $n_a$ levels of factor $A$ and $n_b$ levels of factor $B$. The ANOVA model for this experiment can be stated as:

$Y_{ijk}=\mu +{\tau }_i+{\delta }_j+{(\tau \delta )}_{ij}+{ϵ}_{ijk}$

where:

• $\mu$ represents the overall mean effect

• ${\tau }_i$ is the effect of the $i$th level of factor $A$ ($i=1,2,...,n_a$)

• ${\delta }_j$ is the effect of the $j$th level of factor $B$ ($j=1,2,...,n_b$)

• ${(\tau \delta )}_{ij}$ represents the interaction effect between $A$ and $B$

• ${ϵ}_{ijk}$ represents the random error terms (which are assumed to be normally distributed with a mean of zero and variance of ${\sigma }^2$)

• and the subscript $k$ denotes the $m$ replicates ($k=1,2,...,m$)

Factorial experiments available in DOE++.

Since the effects ${\tau }_i$, ${\delta }_j$ and ${(\tau \delta )}_{ij}$ represent deviations from the overall mean, the following constraints exist:

$\underset{i=1}{\sum ^{n_a}}{\tau }_i=0$

$\underset{j=1}{\sum ^{n_b}}{\delta }_j=0$

$\underset{i=1}{\sum ^{n_a}}{(\tau \delta )}_{ij}=0$

$\underset{j=1}{\sum ^{n_b}}{(\tau \delta )}_{ij}=0$

Hypothesis Tests in General Factorial Experiments

These tests are used to check whether each of the factors investigated in the experiment is significant or not. For the previous example, with two factors, $A$ and $B$, and their interaction, $AB$, the statements for the hypothesis tests can be formulated as follows:

$\begin{array}{rl}& \begin{array}{cc}1.& \end{array}{H}_0:{\tau }_1={\tau }_2=...={\tau }_{n_a}=0\text{ (Main effect of }A\text{ is absent)}\\ & \begin{array}{cc}& \end{array}{H}_1:{\tau }_i\ne 0\text{ for at least one }i\\ & \begin{array}{cc}2.& \end{array}{H}_0:{\delta }_1={\delta }_2=...={\delta }_{n_b}=0\text{ (Main effect of }B\text{ is absent)}\\ & \begin{array}{cc}& \end{array}{H}_1:{\delta }_j\ne 0\text{ for at least one }j\\ & \begin{array}{cc}3.& \end{array}{H}_0:{(\tau \delta )}_{11}={(\tau \delta )}_{12}=...={(\tau \delta )}_{n_a{n}_b}=0\text{ (Interaction }AB\text{ is absent)}\\ & \begin{array}{cc}& \end{array}{H}_1:{(\tau \delta )}_{ij}\ne 0\text{ for at least one }ij\end{array}$

The test statistics for the three tests are as follows:

1)${(F_0)}_A=\frac{M{S}_A}{M{S}_E}$

where $M{S}_A$ is the mean square due to factor $A$ and $M{S}_E$ is the error mean square.

2)$(F_0{)}_B=\frac{M{S}_B}{M{S}_E}$

where $M{S}_B$ is the mean square due to factor $B$ and $M{S}_E$ is the error mean square.

3)$(F_0{)}_{AB}=\frac{M{S}_{AB}}{M{S}_E}$

where $M{S}_{AB}$ is the mean square due to interaction $AB$ and $M{S}_E$ is the error mean square.

The tests are identical to the partial $F$ test explained in Multiple Linear Regression Analysis. The sum of squares for these tests (to obtain the mean squares) are calculated by splitting the model sum of squares into the extra sum of squares due to each factor. The extra sum of squares calculated for each of the factors may either be partial or sequential. For the present example, if the extra sum of squares used is sequential, then the model sum of squares can be written as:

$S{S}_{TR}=S{S}_A+S{S}_B+S{S}_{AB}$

where $S{S}_{TR}$ represents the model sum of squares, $S{S}_A$ represents the sequential sum of squares due to factor $A$, $S{S}_B$ represents the sequential sum of squares due to factor and $S{S}_{AB}$ represents the sequential sum of squares due to the interaction $AB$. The mean squares are obtained by dividing the sum of squares by the associated degrees of freedom. Once the mean squares are known the test statistics can be calculated. For example, the test statistic to test the significance of factor $A$ (or the hypothesis $H_0:{\tau }_i=0$) can then be obtained as:

$\begin{array}{rl}{(F_0)}_A=& \frac{M{S}_A}{M{S}_E}\\ =& \frac{S{S}_A/dof(S{S}_A)}{S{S}_E/dof(S{S}_E)}\end{array}$

Similarly the test statistic to test significance of factor $B$ and the interaction $AB$ can be respectively obtained as:

$\begin{array}{rl}{(F_0)}_B=& \frac{M{S}_B}{M{S}_E}\\ =& \frac{S{S}_B/dof(S{S}_B)}{S{S}_E/dof(S{S}_E)}\\ {(F_0)}_{AB}=& \frac{M{S}_{AB}}{M{S}_E}\\ =& \frac{S{S}_{AB}/dof(S{S}_{AB})}{S{S}_E/dof(S{S}_E)}\end{array}$

It is recommended to conduct the test for interactions before conducting the test for the main effects. This is because, if an interaction is present, then the main effect of the factor depends on the level of the other factors and looking at the main effect is of little value. However, if the interaction is absent then the main effects become important.

Example

Consider an experiment to investigate the effect of speed and type of fuel additive used on the mileage of a sports utility vehicle. Three speeds and two types of fuel additives are investigated. Each of the treatment combinations are replicated three times. The mileage values observed are displayed in the fifth table.

Mileage data for different speeds and fuel additive types.

The experimental design for the data in the fifth table is shown in the figure below. In the figure, the factor Speed is represented as factor $A$ and the factor Fuel Additive is represented as factor $B$. The experimenter would like to investigate if speed, fuel additive or the interaction between speed and fuel additive affects the mileage of the sports utility vehicle. In other words, the following hypotheses need to be tested:

$\begin{array}{rl}& \begin{array}{cc}1.& \end{array}{H}_0:{\tau }_1={\tau }_2={\tau }_3=0\text{ (No main effect of factor }A\text{, speed)}\\ & \begin{array}{cc}& \end{array}{H}_1:{\tau }_i\ne 0\text{ for at least one }i\\ & \begin{array}{cc}2.& \end{array}{H}_0:{\delta }_1={\delta }_2={\delta }_3=0\text{ (No main effect of factor }B\text{, fuel additive)}\\ & \begin{array}{cc}& \end{array}{H}_1:{\delta }_j\ne 0\text{ for at least one }j\\ & \begin{array}{cc}3.& \end{array}{H}_0:{(\tau \delta )}_{11}={(\tau \delta )}_{12}=...={(\tau \delta )}_{33}=0\text{ (No interaction }AB\text{)}\\ & \begin{array}{cc}& \end{array}{H}_1:{(\tau \delta )}_{ij}\ne 0\text{ for at least one }ij\end{array}$

The test statistics for the three tests are:

1.${(F_0)}_A=\frac{M{S}_A}{M{S}_E}$

where $M{S}_A$ is the mean square for factor $A$ and $M{S}_E$ is the error mean square

2.${(F_0)}_B=\frac{M{S}_B}{M{S}_E}$

where $M{S}_B$ is the mean square for factor $B$ and $M{S}_E$ is the error mean square

3.${(F_0)}_{AB}=\frac{M{S}_{AB}}{M{S}_E}$

where $M{S}_{AB}$ is the mean square for interaction $AB$ and $M{S}_E$ is the error mean square

Experimental design for the data in the fifth table.

The ANOVA model for this experiment can be written as:

$Y_{ijk}=\mu +{\tau }_i+{\delta }_j+{(\tau \delta )}_{ij}+{ϵ}_{ijk}$

where ${\tau }_i$ represents the $i$th treatment of factor $A$ (speed) with $i$ =1, 2, 3; ${\delta }_j$ represents the $j$th treatment of factor $B$ (fuel additive) with $j$ =1, 2; and ${(\tau \delta )}_{ij}$ represents the interaction effect. In order to calculate the test statistics, it is convenient to express the ANOVA model of the equation given above in the form $y=X\beta +ϵ$. This can be done as explained next.

Expression of the ANOVA Model as $y=X\beta +ϵ$

Since the effects ${\tau }_i$, ${\delta }_j$ and ${(\tau \delta )}_{ij}$ represent deviations from the overall mean, the following constraints exist. Constraints on ${\tau }_i$ are:

$\begin{array}{rl}\underset{i=1}{\sum ^3}{\tau }_i=& 0\\ \text{or }{\tau }_1+{\tau }_2+{\tau }_3=& 0\end{array}$

Therefore, only two of the ${\tau }_i$ effects are independent. Assuming that ${\tau }_1$ and ${\tau }_2$ are independent, ${\tau }_3=-({\tau }_1+{\tau }_2)$. (The null hypothesis to test the significance of factor $A$ can be rewritten using only the independent effects as $H_0:{\tau }_1={\tau }_2=0$.) DOE++ displays only the independent effects because only these effects are important to the analysis. The independent effects, ${\tau }_1$ and ${\tau }_2$, are displayed as A[1] and A[2] respectively because these are the effects associated with factor $A$ (speed). Constraints on ${\delta }_j$ are:

$\begin{array}{rl}\underset{j=1}{\sum ^2}{\delta }_j=& 0\\ \text{or }{\delta }_1+{\delta }_2=& 0\end{array}$

Therefore, only one of the ${\delta }_j$ effects are independent. Assuming that ${\delta }_1$ is independent, ${\delta }_2=-{\delta }_1$. (The null hypothesis to test the significance of factor $B$ can be rewritten using only the independent effect as $H_0:{\delta }_1=0$.) The independent effect ${\delta }_1$ is displayed as B:B in DOE++. Constraints on ${(\tau \delta )}_{ij}$ are:

$\begin{array}{rl}\underset{i=1}{\sum ^3}{(\tau \delta )}_{ij}=& 0\\ \text{and }\underset{j=1}{\sum ^2}{(\tau \delta )}_{ij}=& 0\\ \text{or }{(\tau \delta )}_{11}+{(\tau \delta )}_{21}+{(\tau \delta )}_{31}=& 0\\ {(\tau \delta )}_{12}+{(\tau \delta )}_{22}+{(\tau \delta )}_{32}=& 0\\ \text{and }{(\tau \delta )}_{11}+{(\tau \delta )}_{12}=& 0\\ {(\tau \delta )}_{21}+{(\tau \delta )}_{22}=& 0\\ {(\tau \delta )}_{31}+{(\tau \delta )}_{32}=& 0\end{array}$

The last five equations given above represent four constraints, as only four of these five equations are independent. Therefore, only two out of the six ${(\tau \delta )}_{ij}$ effects are independent. Assuming that ${(\tau \delta )}_{11}$ and ${(\tau \delta )}_{21}$ are independent, the other four effects can be expressed in terms of these effects. (The null hypothesis to test the significance of interaction $AB$ can be rewritten using only the independent effects as $H_0:{(\tau \delta )}_{11}={(\tau \delta )}_{21}=0$.) The effects ${(\tau \delta )}_{11}$ and ${(\tau \delta )}_{21}$ are displayed as A[1]B and A[2]B respectively in DOE++.

The regression version of the ANOVA model can be obtained using indicator variables, similar to the case of the single factor experiment in Fitting ANOVA Models. Since factor $A$ has three levels, two indicator variables, $x_1$ and $x_2$, are required which need to be coded as shown next:

$\begin{array}{rl}\text{Treatment Effect }{\tau }_1:& x_1=1,x_2=0\\ \text{Treatment Effect }{\tau }_2:& x_1=0,x_2=1\\ \text{Treatment Effect }{\tau }_3:& x_1=-1,x_2=-1\end{array}$

Factor $B$ has two levels and can be represented using one indicator variable, $x_3$, as follows:

$\begin{array}{rl}\text{Treatment Effect }{\delta }_1:& x_3=1\\ \text{Treatment Effect }{\delta }_2:& x_3=-1\end{array}$

The $AB$ interaction will be represented by all possible terms resulting from the product of the indicator variables representing factors $A$ and $B$. There are two such terms here - $x_1{x}_3$ and $x_2{x}_3$. The regression version of the ANOVA model can finally be obtained as:

$Y=\mu +{\tau }_1\cdot x_1+{\tau }_2\cdot x_2+{\delta }_1\cdot x_3+{(\tau \delta )}_{11}\cdot x_1{x}_3+{(\tau \delta )}_{21}\cdot x_2{x}_3+ϵ$

In matrix notation this model can be expressed as:

$y=X\beta +ϵ$

where:

$y=\left[\begin{array}{c}{Y}_{111}\\ Y_{211}\\ Y_{311}\\ Y_{121}\\ Y_{221}\\ Y_{321}\\ Y_{112}\\ Y_{212}\\ .\\ .\\ Y_{323}\end{array}\right]=X\beta +ϵ=\left[\begin{array}{cccccc}1& 1& 0& 1& 1& 0\\ 1& 0& 1& 1& 0& 1\\ 1& -1& -1& 1& -1& -1\\ 1& 1& 0& -1& -1& 0\\ 1& 0& 1& -1& 0& -1\\ 1& -1& -1& -1& 1& 1\\ 1& 1& 0& 1& 1& 0\\ 1& 0& 1& 1& 0& 1\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ 1& -1& -1& -1& 1& 1\end{array}\right]\left[\begin{array}{c}\mu \\ {\tau }_1\\ {\tau }_2\\ {\delta }_1\\ {(\tau \delta )}_{11}\\ {(\tau \delta )}_{21}\end{array}\right]+\left[\begin{array}{c}{ϵ}_{111}\\ {ϵ}_{211}\\ {ϵ}_{311}\\ {ϵ}_{121}\\ {ϵ}_{221}\\ {ϵ}_{321}\\ {ϵ}_{112}\\ {ϵ}_{212}\\ .\\ .\\ {ϵ}_{323}\end{array}\right]$

The vector $y$ can be substituted with the response values from the fifth table to get:

$y=\left[\begin{array}{c}{Y}_{111}\\ Y_{211}\\ Y_{311}\\ Y_{121}\\ Y_{221}\\ Y_{321}\\ Y_{112}\\ Y_{212}\\ .\\ .\\ Y_{323}\end{array}\right]=\left[\begin{array}{c}17.3\\ 18.9\\ 17.1\\ 18.7\\ 19.1\\ 18.8\\ 17.8\\ 18.2\\ .\\ .\\ 18.3\end{array}\right]$

Knowing $y$, $X$ and $\beta$, the sum of squares for the ANOVA model and the extra sum of squares for each of the factors can be calculated. These are used to calculate the mean squares that are used to obtain the test statistics.

Calculation of Sum of Squares for the Model

The model sum of squares, $S{S}_{TR}$, for the regression version of the ANOVA model can be obtained as:

$\begin{array}{rl}S{S}_{TR}=& y^{\prime }[H-(\frac{1}{n_a\cdot n_b\cdot m})J]y\\ =& y^{\prime }[H-(\frac{1}{18})J]y\\ =& 9.7311\end{array}$

where $H$ is the hat matrix and $J$ is the matrix of ones. Since five effect terms (${\tau }_1$, ${\tau }_2$, ${\delta }_1$, ${(\tau \delta )}_{11}$ and ${(\tau \delta )}_{21}$) are used in the model, the number of degrees of freedom associated with $S{S}_{TR}$ is five ($dof(S{S}_{TR})=5$).

The total sum of squares, $S{S}_T$, can be calculated as:

$\begin{array}{rl}S{S}_T=& y^{\prime }[I-(\frac{1}{n_a\cdot n_b\cdot m})J]y\\ =& y^{\prime }[I-(\frac{1}{18})J]y\\ =& 10.7178\end{array}$

Since there are 18 observed response values, the number of degrees of freedom associated with the total sum of squares is 17 ($dof(S{S}_T)=17$). The error sum of squares can now be obtained:

$\begin{array}{rl}S{S}_E=& S{S}_T-S{S}_{TR}\\ =& 10.7178-9.7311\\ =& 0.9867\end{array}$

Since there are three replicates of the full factorial experiment, all of the error sum of squares is pure error. (This can also be seen from the preceding figure, where each treatment combination of the full factorial design is repeated three times.) The number of degrees of freedom associated with the error sum of squares is:

$\begin{array}{rl}dof(S{S}_E)=& dof(S{S}_T)-dof(S{S}_{TR})\\ =& 17-5\\ =& 12\end{array}$

Calculation of Extra Sum of Squares for the Factors

The sequential sum of squares for factor $A$ can be calculated as:

$\begin{array}{rl}S{S}_A=& S{S}_{TR}(\mu ,{\tau }_1,{\tau }_2)-S{S}_{TR}(\mu )\\ =& y^{\prime }[H_{\mu ,{\tau }_1,{\tau }_2}-(\frac{1}{18})J]y-0\end{array}$

where $H_{\mu ,{\tau }_1,{\tau }_2}=X_{\mu ,{\tau }_1,{\tau }_2}{(X_{\mu ,{\tau }_1,{\tau }_2}^{\prime }{X}_{\mu ,{\tau }_1,{\tau }_2})}^{-1}{X}_{\mu ,{\tau }_1,{\tau }_2}^{\prime }$ and $X_{\mu ,{\tau }_1,{\tau }_2}$ is the matrix containing only the first three columns of the $X$ matrix. Thus:

$\begin{array}{rl}S{S}_A=& y^{\prime }[H_{\mu ,{\tau }_1,{\tau }_2}-(\frac{1}{18})J]y-0\\ =& 4.5811-0\\ =& 4.5811\end{array}$

Since there are two independent effects (${\tau }_1$, ${\tau }_2$) for factor $A$, the degrees of freedom associated with $S{S}_A$ are two ($dof(S{S}_A)=2$).

Similarly, the sum of squares for factor $B$ can be calculated as:

$\begin{array}{rl}S{S}_B=& S{S}_{TR}(\mu ,{\tau }_1,{\tau }_2,{\delta }_1)-S{S}_{TR}(\mu ,{\tau }_1,{\tau }_2)\\ =& y^{\prime }[H_{\mu ,{\tau }_1,{\tau }_2,{\delta }_1}-(\frac{1}{18})J]y-y^{\prime }[H_{\mu ,{\tau }_1,{\tau }_2}-(\frac{1}{18})J]y\\ =& 9.4900-4.5811\\ =& 4.9089\end{array}$

Since there is one independent effect, ${\delta }_1$, for factor $B$, the number of degrees of freedom associated with $S{S}_B$ is one ($dof(S{S}_B)=1$).

The sum of squares for the interaction $AB$ is:

$\begin{array}{rl}S{S}_{AB}=& S{S}_{TR}(\mu ,{\tau }_1,{\tau }_2,{\delta }_1,{(\tau \delta )}_{11},{(\tau \delta )}_{21})-S{S}_{TR}(\mu ,{\tau }_1,{\tau }_2,{\delta }_1)\\ =& S{S}_{TR}-S{S}_{TR}(\mu ,{\tau }_1,{\tau }_2,{\delta }_1)\\ =& 9.7311-9.4900\\ =& 0.2411\end{array}$

Since there are two independent interaction effects, ${(\tau \delta )}_{11}$ and ${(\tau \delta )}_{21}$, the number of degrees of freedom associated with $S{S}_{AB}$ is two ($dof(S{S}_{AB})=2$).

Calculation of the Test Statistics

Knowing the sum of squares, the test statistic for each of the factors can be calculated. Analyzing the interaction first, the test statistic for interaction $AB$ is:

$\begin{array}{rl}{(f_0)}_{AB}=& \frac{M{S}_{AB}}{M{S}_E}\\ =& \frac{0.2411/2}{0.9867/12}\\ =& 1.47\end{array}$

The $p$ value corresponding to this statistic, based on the $F$ distribution with 2 degrees of freedom in the numerator and 12 degrees of freedom in the denominator, is:

$\begin{array}{rl}pvalue=& 1-P(F\le {(f_0)}_{AB})\\ =& 1-0.7307\\ =& 0.2693\end{array}$

Assuming that the desired significance level is 0.1, since $p$ value > 0.1, we fail to reject $H_0:{(\tau \delta )}_{ij}=0$ and conclude that the interaction between speed and fuel additive does not significantly affect the mileage of the sports utility vehicle. DOE++ displays this result in the ANOVA table, as shown in the following figure. In the absence of the interaction, the analysis of main effects becomes important.

The test statistic for factor $A$ is:

$\begin{array}{rl}{(f_0)}_A=& \frac{M{S}_A}{M{S}_E}\\ =& \frac{S{S}_A/dof(S{S}_A)}{S{S}_E/dof(S{S}_E)}\\ =& \frac{4.5811/2}{0.9867/12}\\ =& 27.86\end{array}$

The $p$ value corresponding to this statistic based on the $F$ distribution with 2 degrees of freedom in the numerator and 12 degrees of freedom in the denominator is:

$\begin{array}{rl}pvalue=& 1-P(F\le {(f_0)}_A)\\ =& 1-0.99997\\ =& 0.00003\end{array}$

Since $p$ value < 0.1, $H_0:{\tau }_i=0$ is rejected and it is concluded that factor $A$ (or speed) has a significant effect on the mileage.

The test statistic for factor $B$ is:

$\begin{array}{rl}{(f_0)}_B=& \frac{M{S}_B}{M{S}_E}\\ =& \frac{4.9089/1}{0.9867/12}\\ =& 59.7\end{array}$

The $p$ value corresponding to this statistic based on the $F$ distribution with 2 degrees of freedom in the numerator and 12 degrees of freedom in the denominator is:

$\begin{array}{rl}pvalue=& 1-P(F\le {(f_0)}_B)\\ =& 1-0.999995\\ =& 0.000005\end{array}$

Since $p$ value < 0.1, $H_0:{\delta }_j=0$ is rejected and it is concluded that factor $B$ (or fuel additive type) has a significant effect on the mileage. Therefore, it can be concluded that speed and fuel additive type affect the mileage of the vehicle significantly. The results are displayed in the ANOVA table of the following figure.

Analysis results for the experiment in the fifth table.

Calculation of Effect Coefficients

Results for the effect coefficients of the model of the regression version of the ANOVA model are displayed in the Regression Information table in the following figure. Calculations of the results in this table are discussed next. The effect coefficients can be calculated as follows:

$\begin{array}{rl}\hat{\beta }=& {(X^{\prime }X)}^{-1}{X}^{\prime }y\\ =& \left[\begin{array}{c}18.2889\\ -0.2056\\ 0.6944\\ -0.5222\\ 0.0056\\ 0.1389\end{array}\right]\end{array}$

Therefore, $\hat{\mu }=18.2889$, ${\hat{\tau }}_1=-0.2056$, ${\hat{\tau }}_2=0.6944$ etc. As mentioned previously, these coefficients are displayed as Intercept, A[1] and A[2] respectively depending on the name of the factor used in the experimental design. The standard error for each of these estimates is obtained using the diagonal elements of the variance-covariance matrix $C$.

$\begin{array}{rl}C=& {\hat{\sigma }}^2{(X^{\prime }X)}^{-1}\\ =& M{S}_E\cdot {(X^{\prime }X)}^{-1}\\ =& \left[\begin{array}{cccccc}0.0046& 0& 0& 0& 0& 0\\ 0& 0.0091& -0.0046& 0& 0& 0\\ 0& -0.0046& 0.0091& 0& 0& 0\\ 0& 0& 0& 0.0046& 0& 0\\ 0& 0& 0& 0& 0.0091& -0.0046\\ 0& 0& 0& 0& -0.0046& 0.0091\end{array}\right]\end{array}$

For example, the standard error for ${\hat{\tau }}_1$ is:

$\begin{array}{rl}se({\hat{\tau }}_1)=& \sqrt{C_{22}}\\ =& \sqrt{0.0091}\\ =& 0.0956\end{array}$

Then the $t$ statistic for ${\hat{\tau }}_1$ can be obtained as:

$\begin{array}{rl}{t}_0=& \frac{{\hat{\tau }}_1}{se({\hat{\tau }}_1)}\\ =& \frac{-0.2056}{0.0956}\\ =& -2.1506\end{array}$

The $p$ value corresponding to this statistic is:

Confidence intervals on ${\tau }_1$ can also be calculated. The 90% limits on ${\tau }_1$ are:

$\begin{array}{rl}=& {\hat{\tau }}_1\pm t_{\alpha /2,n-(k+1)}\sqrt{C_{22}}\\ =& {\tau }_1\pm t_{0.05,12}\sqrt{C_{22}}\\ =& -0.2056\pm 0.1704\end{array}$

Thus, the 90% limits on ${\tau }_1$ are $-0.3760$ and $-0.0352$ respectively. Results for other coefficients are obtained in a similar manner.

Least Squares Means

The estimated mean response corresponding to the $i$th level of any factor is obtained using the adjusted estimated mean which is also called the least squares mean. For example, the mean response corresponding to the first level of factor $A$ is $\mu +{\tau }_1$. An estimate of this is $\hat{\mu }+{\hat{\tau }}_1$ or ($18.2889+(-0.2056)=18.0833$). Similarly, the estimated response at the third level of factor $A$ is $\hat{\mu }+{\hat{\tau }}_3$ or $\hat{\mu }+(-{\hat{\tau }}_1-{\hat{\tau }}_2)$ or ($18.2889+(0.2056-0.6944)=17.8001$).

Residual Analysis

As in the case of single factor experiments, plots of residuals can also be used to check for model adequacy in factorial experiments. Box-Cox transformations are also available in DOE++ for factorial experiments.

Factorial Experiments with a Single Replicate

If a factorial experiment is run only for a single replicate then it is not possible to test hypotheses about the main effects and interactions as the error sum of squares cannot be obtained. This is because the number of observations in a single replicate equals the number of terms in the ANOVA model. Hence the model fits the data perfectly and no degrees of freedom are available to obtain the error sum of squares. For example, if the two factor experiment to study the effect of speed and fuel additive type on mileage was run only as a single replicate there would be only six response values. The regression version of the ANOVA model has six terms and therefore will fit the six response values perfectly. The error sum of squares, $S{S}_E$, for this case will be equal to zero. In some single replicate factorial experiments it is possible to assume that the interaction effects are negligible. In this case, the interaction mean square can be used as error mean square, $M{S}_E$, to test hypotheses about the main effects. However, such assumptions are not applicable in all cases and should be used carefully.

Blocking

Many times a factorial experiment requires so many runs that all of them cannot be completed under homogeneous conditions. This may lead to inclusion of the effects of nuisance factors into the investigation. Nuisance factors are factors that have an effect on the response but are not of primary interest to the investigator. For example two replicates of a two-factor factorial experiment require eight runs. If four runs require the duration of one day to be completed then the total experiment will require two days to be completed. The difference in the conditions on the two days may lead to introduction of effects on the response that are not the result of the two factors being investigated. Therefore, the day is a nuisance factor for this experiment. Nuisance factors can be accounted for using blocking. In blocking, experimental runs are separated based on levels of the nuisance factor. For the case of the two-factor factorial experiment where the day is a nuisance factor, separation can be made into two groups or blocks -runs that are carried out on the first day belong to block 1, and runs that are carried out on the second day belong to block 2. Thus, within each block conditions are the same with respect to the nuisance factor. As a result, each block investigates the effects of the factors of interest, while the difference in the blocks measures the effect of the nuisance factor. For the example of the two factor factorial experiment, a possible assignment of runs to the blocks could be - one replicate of the experiment is assigned to block 1 and the second replicate is assigned to block 2 (now each block contains all possible treatment combinations). Within each block, runs are subjected to randomization (i.e., randomization is now restricted to the runs within a block). Such a design, where each block contains one complete replicate and the treatments within a block are subjected to randomization, is called randomized complete block design.

In summary, blocking should always be used to account for the effects of nuisance factors if it is not possible to hold the nuisance factor at a constant level through all of the experimental runs. Randomization should be used within each block to counter the effects of any unknown variability that may still be present.

Example

Consider the experiment of the fifth table where the mileage of a sports utility vehicle was investigated for the effects of speed and fuel additive type. Now assume that the three replicates for this experiment were carried out on three different vehicles. To ensure that the variation from one vehicle to another does not have an effect on the analysis, each vehicle is considered as one block. See the experiment design in the following figure.

Randomized complete block design for the experiment in the fifth table using three blocks.

For the purpose of the analysis, the block is considered as a main effect except that it is assumed that interactions between the block and the other main effects do not exist. Therefore, there is one block main effect (having three levels - block 1, block 2 and block 3), two main effects (speed -having three levels; and fuel additive type - having two levels) and one interaction effect (speed-fuel additive interaction) for this experiment. Let ${\zeta }_i$ represent the block effects. The hypothesis test on the block main effect checks if there is a significant variation from one vehicle to the other. The statements for the hypothesis test are:

$\begin{array}{rlr}& H_0:& {\zeta }_1={\zeta }_2={\zeta }_3=0\text{ (no main effect of block)}\\ & H_1:& {\zeta }_i\ne 0\text{ for at least one }i\end{array}$

The test statistic for this test is:

$F_0=\frac{M{S}_{Block}}{M{S}_E}$

where $M{S}_{Block}$ represents the mean square for the block main effect and $M{S}_E$ is the error mean square. The hypothesis statements and test statistics to test the significance of factors $A$ (speed), $B$ (fuel additive) and the interaction $AB$ (speed-fuel additive interaction) can be obtained as explained in the example. The ANOVA model for this example can be written as:

$Y_{ijk}=\mu +{\zeta }_i+{\tau }_j+{\delta }_k+{(\tau \delta )}_{jk}+{ϵ}_{ijk}$

where:

• $\mu$ represents the overall mean effect

• ${\zeta }_i$ is the effect of the $i$th level of the block ($i=1,2,3$)

• ${\tau }_j$ is the effect of the $j$th level of factor $A$ ($j=1,2,3$)

• ${\delta }_k$ is the effect of the $k$th level of factor $B$ ($k=1,2$)

• ${(\tau \delta )}_{jk}$ represents the interaction effect between $A$ and $B$

• and ${ϵ}_{ijk}$ represents the random error terms (which are assumed to be normally distributed with a mean of zero and variance of ${\sigma }^2$)

In order to calculate the test statistics, it is convenient to express the ANOVA model of the equation given above in the form $y=X\beta +ϵ$. This can be done as explained next.

Expression of the ANOVA Model as $y=X\beta +ϵ$

Since the effects ${\zeta }_i$, ${\tau }_j$, ${\delta }_k$, and ${(\tau \delta )}_{jk}$ are defined as deviations from the overall mean, the following constraints exist.
Constraints on ${\zeta }_i$ are:

$\begin{array}{rlr}& \underset{i=1}{\sum ^3}{\zeta }_i=& 0\\ & \text{or }{\zeta }_1+{\zeta }_2+{\zeta }_3=& 0\end{array}$

Therefore, only two of the ${\zeta }_i$ effects are independent. Assuming that ${\zeta }_1$ and ${\zeta }_2$ are independent, ${\zeta }_3=-({\zeta }_1+{\zeta }_2)$. (The null hypothesis to test the significance of the blocks can be rewritten using only the independent effects as $H_0:{\zeta }_1={\zeta }_2=0$.) In DOE++, the independent block effects, ${\zeta }_1$ and ${\zeta }_2$, are displayed as Block[1] and Block[2], respectively.

Constraints on ${\tau }_j$ are:

$\begin{array}{rlr}& \underset{j=1}{\sum ^3}{\tau }_j=& 0\\ & \text{or }{\tau }_1+{\tau }_2+{\tau }_3=& 0\end{array}$

Therefore, only two of the ${\tau }_j$ effects are independent. Assuming that ${\tau }_1$ and ${\tau }_2$ are independent, ${\tau }_3=-({\tau }_1+{\tau }_2)$. The independent effects, ${\tau }_1$ and ${\tau }_2$, are displayed as A[1] and A[2], respectively. Constraints on ${\delta }_k$ are:

$\begin{array}{rlr}& \underset{k=1}{\sum ^2}{\delta }_k=& 0\\ & \text{or }{\delta }_1+{\delta }_2=& 0\end{array}$

Therefore, only one of the ${\delta }_k$ effects is independent. Assuming that ${\delta }_1$ is independent, ${\delta }_2=-{\delta }_1$. The independent effect, ${\delta }_1$, is displayed as B:B. Constraints on ${(\tau \delta )}_{jk}$ are:

$\begin{array}{rlr}& \underset{j=1}{\sum ^3}{(\tau \delta )}_{jk}=& 0\\ & \text{and }\underset{k=1}{\sum ^2}{(\tau \delta )}_{jk}=& 0\\ & \text{or }{(\tau \delta )}_{11}+{(\tau \delta )}_{21}+{(\tau \delta )}_{31}=& 0\\ & {(\tau \delta )}_{12}+{(\tau \delta )}_{22}+{(\tau \delta )}_{32}=& 0\\ & \text{and }{(\tau \delta )}_{11}+{(\tau \delta )}_{12}=& 0\\ & {(\tau \delta )}_{21}+{(\tau \delta )}_{22}=& 0\\ & {(\tau \delta )}_{31}+{(\tau \delta )}_{32}=& 0\end{array}$

The last five equations given above represent four constraints as only four of the five equations are independent. Therefore, only two out of the six ${(\tau \delta )}_{jk}$ effects are independent. Assuming that ${(\tau \delta )}_{11}$ and ${(\tau \delta )}_{21}$ are independent, we can express the other four effects in terms of these effects. The independent effects, ${(\tau \delta )}_{11}$ and ${(\tau \delta )}_{21}$, are displayed as A[1]B and A[2]B, respectively.

The regression version of the ANOVA model can be obtained using indicator variables. Since the block has three levels, two indicator variables, $x_1$ and $x_2$, are required, which need to be coded as shown next:

$\begin{array}{rlr}& \text{Block 1}:& x_1=1,x_2=0\\ & \text{Block 2}:& x_1=0,x_2=1\\ & \text{Block 3}:& x_1=-1,x_2=-1\end{array}$

Factor $A$ has three levels and two indicator variables, $x_3$ and $x_4$, are required:

$\begin{array}{rlr}& \text{Treatment Effect }{\tau }_1:& x_3=1,x_4=0\\ & \text{Treatment Effect }{\tau }_2:& x_3=0,x_4=1\\ & \text{Treatment Effect }{\tau }_3:& x_3=-1,x_4=-1\end{array}$

Factor $B$ has two levels and can be represented using one indicator variable, $x_5$, as follows:

$\begin{array}{rlr}& \text{Treatment Effect }{\delta }_1:& x_5=1\\ & \text{Treatment Effect }{\delta }_2:& x_5=-1\end{array}$

The $AB$ interaction will be represented by $x_3{x}_5$ and $x_4{x}_5$. The regression version of the ANOVA model can finally be obtained as:

$Y=\mu +{\zeta }_1\cdot x_1+{\zeta }_2\cdot x_2+{\tau }_1\cdot x_3+{\tau }_2\cdot x_4+{\delta }_1\cdot x_5+{(\tau \delta )}_{11}\cdot x_3{x}_5+{(\tau \delta )}_{21}\cdot x_4{x}_5+ϵ$

In matrix notation this model can be expressed as:

$y=X\beta +ϵ$

or:

$\left[\begin{array}{c}17.3\\ 18.9\\ 17.1\\ 18.7\\ 19.1\\ 18.8\\ 17.8\\ 18.2\\ .\\ .\\ 18.3\end{array}\right]=\left[\begin{array}{cccccccc}1& 1& 0& 1& 0& 1& 1& 0\\ 1& 1& 0& 0& 1& 1& 0& 1\\ 1& 1& 0& -1& -1& 1& -1& -1\\ 1& 1& 0& 1& 0& -1& -1& 0\\ 1& 1& 0& 0& 1& -1& 0& -1\\ 1& 1& 0& -1& -1& -1& 1& 1\\ 1& 0& 1& 1& 0& 1& 1& 0\\ 1& 0& 1& 0& 1& 1& 0& 1\\ .& .& .& .& .& .& .& .\\ .& .& .& .& .& .& .& .\\ 1& -1& -1& -1& -1& -1& 1& 1\end{array}\right]\left[\begin{array}{c}\mu \\ {\zeta }_1\\ {\zeta }_2\\ {\tau }_1\\ {\tau }_2\\ {\delta }_1\\ {(\tau \delta )}_{11}\\ {(\tau \delta )}_{21}\end{array}\right]+\left[\begin{array}{c}{ϵ}_{111}\\ {ϵ}_{121}\\ {ϵ}_{131}\\ {ϵ}_{112}\\ {ϵ}_{122}\\ {ϵ}_{132}\\ {ϵ}_{211}\\ {ϵ}_{221}\\ .\\ .\\ {ϵ}_{332}\end{array}\right]$

Knowing $y$, $X$ and $\beta$, the sum of squares for the ANOVA model and the extra sum of squares for each of the factors can be calculated. These are used to calculate the mean squares that are used to obtain the test statistics.

Calculation of the Sum of Squares for the Model

The model sum of squares, $S{S}_{TR}$, for the ANOVA model of this example can be obtained as:

$\begin{array}{rlr}& S{S}_{TR}=& y^{\prime }[H-(\frac{1}{n_a\cdot n_b\cdot m})J]y\\ & =& y^{\prime }[H-(\frac{1}{18})J]y\\ & =& 9.9256\end{array}$

Since seven effect terms (${\zeta }_1$, ${\zeta }_2$, ${\tau }_1$, ${\tau }_2$, ${\delta }_1$, ${(\tau \delta )}_{11}$ and ${(\tau \delta )}_{21}$) are used in the model the number of degrees of freedom associated with $S{S}_{TR}$ is seven ($dof(S{S}_{TR})=7$).

The total sum of squares can be calculated as:

$\begin{array}{rlr}& S{S}_T=& y^{\prime }[I-(\frac{1}{n_a\cdot n_b\cdot m})J]y\\ & =& y^{\prime }[H-(\frac{1}{18})J]y\\ & =& 10.7178\end{array}$

Since there are 18 observed response values, the number of degrees of freedom associated with the total sum of squares is 17 ($dof(S{S}_T)=17$). The error sum of squares can now be obtained:

$\begin{array}{rl}S{S}_E=& S{S}_T-S{S}_{TR}\\ =& 10.7178-9.9256\\ =& 0.7922\end{array}$

The number of degrees of freedom associated with the error sum of squares is:

$\begin{array}{rl}dof(S{S}_E)=& dof(S{S}_T)-dof(S{S}_{TR})\\ =& 17-7\\ =& 10\end{array}$

Since there are no true replicates of the treatments (as can be seen from the design of the previous figure, where all of the treatments are seen to be run just once), all of the error sum of squares is the sum of squares due to lack of fit. The lack of fit arises because the model used is not a full model since it is assumed that there are no interactions between blocks and other effects.

Calculation of the Extra Sum of Squares for the Factors

The sequential sum of squares for the blocks can be calculated as:

$\begin{array}{rl}S{S}_{Block}=& S{S}_{TR}(\mu ,{\zeta }_1,{\zeta }_2)-S{S}_{TR}(\mu )\\ =& y^{\prime }[H_{\mu ,{\zeta }_1,{\zeta }_2}-(\frac{1}{18})J]y-0\end{array}$

where $J$ is the matrix of ones, $H_{\mu ,{\zeta }_1,{\zeta }_2}$ is the hat matrix, which is calculated using $H_{\mu ,{\zeta }_1,{\zeta }_2}=X_{\mu ,{\zeta }_1,{\zeta }_2}{(X_{\mu ,{\zeta }_1,{\zeta }_2}^{\prime }{X}_{\mu ,{\zeta }_1,{\zeta }_2})}^{-1}{X}_{\mu ,{\zeta }_1,{\zeta }_2}^{\prime }$, and $X_{\mu ,{\zeta }_1,{\zeta }_2}$ is the matrix containing only the first three columns of the $X$ matrix. Thus

$\begin{array}{rl}S{S}_{Block}=& y^{\prime }[H_{\mu ,{\zeta }_1,{\zeta }_2}-(\frac{1}{18})J]y-0\\ =& 0.1944-0\\ =& 0.1944\end{array}$

Since there are two independent block effects, and ${\zeta }_2$, the number of degrees of freedom associated with $S{S}_{Blocks}$ is two ($dof(S{S}_{Blocks})=2$).

Similarly, the sequential sum of squares for factor $A$ can be calculated as:

$\begin{array}{rl}S{S}_A=& S{S}_{TR}(\mu ,{\zeta }_1,{\zeta }_2,{\tau }_1,{\tau }_2)-S{S}_{TR}(\mu ,{\zeta }_1,{\zeta }_2)\\ =& y^{\prime }[H_{\mu ,{\zeta }_1,{\zeta }_2,{\tau }_1,{\tau }_2}-(\frac{1}{18})J]y-y^{\prime }[H_{\mu ,{\zeta }_1,{\zeta }_2}-(\frac{1}{18})J]y\\ =& 4.7756-0.1944\\ =& 4.5812\end{array}$

Sequential sum of squares for the other effects are obtained as $S{S}_B=4.9089$ and $S{S}_{AB}=0.2411$.

Calculation of the Test Statistics

Knowing the sum of squares, the test statistics for each of the factors can be calculated. For example, the test statistic for the main effect of the blocks is:

$\begin{array}{rl}{(f_0)}_{Block}=& \frac{M{S}_{Block}}{M{S}_E}\\ =& \frac{S{S}_{Block}/dof(S{S}_{Blocks})}{S{S}_E/dof(S{S}_E)}\\ =& \frac{0.1944/2}{0.7922/10}\\ =& 1.227\end{array}$

The $p$ value corresponding to this statistic based on the $F$ distribution with 2 degrees of freedom in the numerator and 10 degrees of freedom in the denominator is:

$\begin{array}{rl}pvalue=& 1-P(F\le {(f_0)}_{Block})\\ =& 1-0.6663\\ =& 0.3337\end{array}$

Assuming that the desired significance level is 0.1, since $p$ value > 0.1, we fail to reject $H_0:{\zeta }_i=0$ and conclude that there is no significant variation in the mileage from one vehicle to the other. Statistics to test the significance of other factors can be calculated in a similar manner. The complete analysis results obtained from DOE++ for this experiment are presented in the following figure.

Analysis results for the experiment in the example.

Use of Regression to Calculate Sum of Squares

This section explains the reason behind the use of regression in DOE++ in all calculations related to the sum of squares. A number of textbooks present the method of direct summation to calculate the sum of squares. But this method is only applicable for balanced designs and may give incorrect results for unbalanced designs. For example, the sum of squares for factor $A$ in a balanced factorial experiment with two factors, $A$ and $B$, is given as follows:

$\begin{array}{rlr}& S{S}_A=& \underset{i=1}{\sum ^{n_a}}{n}_{b}n{({\overline{y}}_{i..}-{\overline{y}}_{...})}^2\\ & =& \underset{i=1}{\sum ^{n_a}}\frac{y_{i..}^2}{n_{b}n}-\frac{y_{...}^2}{n_a{n}_{b}n}\end{array}$

where $n_a$ represents the levels of factor $A$, $n_b$ represents the levels of factor $B$, and $n$ represents the number of samples for each combination of $A$ and $B$. The term ${\overline{y}}_{i..}$ is the mean value for the $i$th level of factor $A$, $y_{i..}$ is the sum of all observations at the $i$th level of factor $A$ and $y_{...}$ is the sum of all observations.

The analogous term to calculate $S{S}_A$ in the case of an unbalanced design is given as:

$S{S}_A=\underset{i=1}{\sum ^{n_a}}\frac{y_{i..}^2}{n_{i.}}-\frac{y_{...}^2}{n_{..}}$

where $n_{i.}$ is the number of observations at the $i$th level of factor $A$ and $n_{..}$ is the total number of observations. Similarly, to calculate the sum of squares for factor $B$ and interaction $AB$, the formulas are given as:

$\begin{array}{rlr}& S{S}_B=& \underset{j=1}{\sum ^{n_b}}\frac{y_{.j.}^2}{n_{.j}}-\frac{y_{...}^2}{n_{..}}\\ & S{S}_{AB}=& \underset{i=1}{\sum ^{n_a}}\underset{j=1}{\sum ^{n_b}}\frac{y_{ij.}^2}{n_{ij}}-\underset{i=1}{\sum ^{n_a}}\frac{y_{i..}^2}{n_{i.}}-\underset{j=1}{\sum ^{n_b}}\frac{y_{.j.}^2}{n_{.j}}+\frac{y_{...}^2}{n_{..}}\end{array}$

Applying these relations to the unbalanced data of the last table, the sum of squares for the interaction $AB$ is:

$\begin{array}{rlr}& S{S}_{AB}=& \underset{i=1}{\sum ^{n_a}}\underset{j=1}{\sum ^{n_b}}\frac{y_{ij.}^2}{n_{ij}}-\underset{i=1}{\sum ^{n_a}}\frac{y_{i..}^2}{n_{i.}}-\underset{j=1}{\sum ^{n_b}}\frac{y_{.j.}^2}{n_{.j}}+\frac{y_{...}^2}{n_{..}}\\ & =& (6^2+4^2+\frac{{(42+6)}^2}{2}+{12}^2)-(\frac{{10}^2}{2}+\frac{{60}^2}{3})\\ & & -(\frac{{54}^2}{3}+\frac{{16}^2}{2})+\frac{{70}^2}{5}\\ & =& -22\end{array}$

which is obviously incorrect since the sum of squares cannot be negative. For a detailed discussion on this refer to [Searle].

Example of an unbalanced design.

The correct sum of squares can be calculated as shown next. The $y$ and $X$ matrices for the design of the last table can be written as:

$y=\left[\begin{array}{c}6\\ 4\\ 6\\ 12\\ 42\end{array}\right]\text{ and }X=\left[\begin{array}{cccc}1& 1& 1& 1\\ 1& 1& -1& -1\\ 1& -1& 1& -1\\ 1& -1& -1& 1\\ 1& -1& 1& -1\end{array}\right]$

Then the sum of squares for the interaction $AB$ can be calculated as:

$S{S}_{AB}=y^{\prime }[H-(1/5)J]y-y^{\prime }[H_{\stackrel{~}{}AB}-(1/5)J]y$

where $H$ is the hat matrix and $J$ is the matrix of ones. The matrix $H_{\stackrel{~}{}AB}$ can be calculated using $H_{\stackrel{~}{}AB}=X_{\stackrel{~}{}AB}{(X_{\stackrel{~}{}AB}^{\prime }{X}_{\stackrel{~}{}AB})}^{-1}{X}_{\stackrel{~}{}AB}^{\prime }$ where $X_{\stackrel{~}{}AB}$ is the design matrix, $X$, excluding the last column that represents the interaction effect $AB$. Thus, the sum of squares for the interaction $AB$ is:

$\begin{array}{rlr}& S{S}_{AB}=& y^{\prime }[H-(1/5)J]y-y^{\prime }[H_{\stackrel{~}{}AB}-(1/5)J]y\\ & =& 368-339.4286\\ & =& 28.5714\end{array}$

This is the value that is calculated by DOE++ (see the first figure below, for the experiment design and the second figure below for the analysis).

Unbalanced experimental design for the data in the last table.

Analysis for the unbalanced data in the last table.