Design Evaluation and Power Study

In general, there are three stages in applying design of experiments (DOE) to solve an issue: designing the experiment, conducting the experiment and analyzing the data. The first stage is very critical. If the designed experiment is not efficient, you are unlikely to obtain good results. It is very common to evaluate an experiment before conducting the tests. A design evaluation often focuses on the following four aspects:

• The alias structure. Are main effects and 2-way interactions in the experiment aliased with each other? What is the resolution of the design? • The orthogonality. An orthogonal design is always preferred. If a design is non-orthogonal, how are the estimated coefficients correlated? • The optimality. A design is called “optimal” if it can meet one or more of the following criteria: o D-optimality: minimize the determinant of the variance-covariance matrix. o A-optimality: minimize the trace of the variance-covariance matrix. o V-optimality: minimize the average prediction variance in the design space. • The power (or its inverse, the Type II error). Power is the probability of detecting an effect when it is indeed active. A design with low power for main effects is not a good design.

In the following sections, we will discuss how to evaluate a design according to the above four aspects.