## Three-Level Main-Effects Designs Exploiting Prior Information About Model Uncertainty

### Pi-Wen Tsai

### Institute of Statistical Science, Academia Sinica, Taipei, Taiwan, ROC.

*pwtsai@stat.sinica.edu.tw* Steven G. Gilmour

### School of Mathematical Sciences, Queen Mary, University of London, Mile End Road, London E1 4NS, UK.

*s.g.gilmour@qmul.ac.uk* Roger Mead

### School of Applied Statistics, The University of Reading, PO Box 240, Earley Gate, Reading RG6 6FN, UK.

*r.mead@reading.ac.uk*

SUMMARY

In this paper, we consider experiments for which experimenters have some prior information on the probability of effects being non-negligible in advance of collecting their data. We suggest using this prior information as a tool to help experimenters to decide the appropriate run-sizes that experimenters should use. In addition, we introduce a criterion for selecting orthogonal, or nearly orthogonal, main effects designs, with robustness to interactions as a secondary con- sideration under particular priors. We will show that this criterion exploiting prior information about model uncertainty can lead to more appropriate designs reflecting experimenters’ prior belief on the importance of each effect.

*Some key words: prior information; factor screening; orthogonal array; projection; Bayesian*
D-optimal designs.

**1** **Introduction**

In industrial experiments, when several factors are of interest, screening designs are widely used to find the few “active” factors which have major effects. Traditionally, design and analysis of screening experiments have been restricted to main effects only by assuming all interactions are small and therefore negligible. However, it may be that, when modelling the data, a few inter- actions are included in the model while some factors may be dropped completely. Motivated by Hamada and Wu (1992), Tsai, Gilmour and Mead (2000) used a two-stage analysis that consid- ers interactions as well as main effects for the analysis of screening experiments with complex aliasing patterns. First, the main effects model is fitted to identify “active” factors. Secondly, a stepwise regression procedure is used for selecting a suitable fitted model that may contain some interactions as well as main effects, but which obeys functional marginality (McCullagh and Nelder, 1989). It has been emphasised that, for this two-stage analysis, it is important that designs used for screening experiments can project onto good lower dimensional designs for various sets of factors and thus provide reasonably efficient parameter estimates for a range of possible models.

*To explore projection properties of screening designs, Tsai et al. (2000) introduced a cri-*
terion, denoted byQ(Γ^{(k)}) for a k-factor design, to compare orthogonal, or nearly orthogonal,
main effects designs, with robustness to interactions as a secondary consideration. This criterion
averages an approximation toAs-efficiency over lower-dimensional projections of the design.

It has been showed that designs with lowerQ(Γ^{(k)}) are more likely to have efficient projections
and on average can provide better parameter estimates over a range of models than designs with
higherQ(Γ^{(k)}*). For detailed information on the Q-criterion, the reader is referred to Tsai et al.*

(2000).

Here, we extend their method and consider the case when in advance of running an ex- periment, experimenters have some prior knowledge about the probabilities of effects being non-negligible. We will introduce a general framework for the prior probabilities of effects

being non-negligible for the two-stage analysis. An 18-run three-level designs with six factors will be used as an example used to demonstrate how to use these prior knowledge as a tool to help experimenters to decide the appropriate experimental run sizes that they should use.

In addition, to incorporate the prior knowledge into design process, we generalise theQ(Γ^{(k)})
criterion and propose a new criterion exploiting prior information about model uncertainty for
selecting orthogonal, or nearly orthogonal, main effects designs, with robustness to interactions
as a secondary consideration. In this paper, we will study the relationship between this criterion
and the the generalised minimum aberration criterion. For simplicity, we compare this criterion
with theG2-aberration criterion proposed by Tang and Deng (1999) for designs with two levels.

We will demonstrate that the efficiency-typeQ(Γ^{(k)}) criterion is the same as the G_{2}-aberration
criterion (1999) is a good surrogate for model robustness for designs with two levels. Therefore,
theQB(Γ^{(k)}) criterion which generalised the Q criterion is a good criterion for selecting designs
when experimenters have some prior believe of each factorial effect being non-negligible.

One of the frequently used three-level orthogonal arrays in industrial experiments is the
L18(3^{7}), an 18-run design for seven three-level factors. Wang and Wu (1995) and Cheng and
Wu (2001) studied the projection properties of theL18(3^{7}) orthogonal array when projected onto
sets of three and four factors. Cheng and Ye (2004) permuted the levels of the projected designs
obtained from theL18(3^{7}) orthogonal array to generate some additional orthogonal main effects
*designs. Alternatively, Tsai et al. (2000) used a columnwise procedure to generate all possible*
three-level main effects designs for 3 to 6 factors in 18 runs. It has been shown that, by using
their procedure, more designs can be found. The best 18-run designs with six factors that they
generated have better projection properties and are more efficient than those obtained by the
procedure of Cheng and Ye (2004) and those from theL_{18}(3^{7}). Details and applications of the
*Q-criterion were given by Tsai, et al. (2000, 2004).*

*Here, we list some best six-factor main effects designs in 18 runs obtained by Tsai et*
*al. (2000) and compare them with those obtained by Cheng and Ye (2004) and those obtained*
from the L18(3^{7}) orthogonal array under various priors. We will show that, depending on the

prior knowledge, more appropriate designs can be found among the class of good designs. The best designs constructed in their way are very robust to prior specifications, so experimenters can have more confidence in using them.

The paper is organised as follows. In Section 2, we introduce the prior probabilities for the
two-stage analysis. In Section 3, we introduce the QB(Γ^{(k)})-criterion to assess designs under
model uncertainty for some given prior. A comparison of an 18-run experiment is in Section 4.

Finally, in Section 5, we provide some concluding remarks.

**2** **Prior probabilities**

In this paper, we consider experiments for which experimenters have some prior information on the probabilities of effects being non-negligible in advance of collecting their data. The idea of using prior probability of particular factors being non-negligible for the analysis of designed ex- periments with complex aliasing was first introduced by Box and Meyer (1993). They proposed a model-discrimination criterion, based on posterior probabilities, for designing follow-up ex- periments that allow maximum discrimination among the plausible models. Chipman, Hamada and Wu (1997) suggested a more elaborate Bayesian analysis approach for analysing data from designs with complex aliasing. They used the hierarchical variable structure to reduce the model space, and implemented variable selection using the stochastic search variable selection (SSVS) methods, which sample from the posterior probability distribution over the model space. Re- cently, Beattie, Fong and Lin (2002) proposed a two-stage Bayesian model selection strategy for supersaturated designs, which can be summarized as a combination of SSVS (the first stage) to find the active factors and intrinsic Bayes factors (the second stage) to test whether their effects are significant.

Here, we do not use fully Bayesian methods. Instead, we propose a method to exploit experimenters’ prior beliefs for selecting orthogonal, or nearly orthogonal, main effects designs,

with robustness to interactions as a secondary consideration. The two-stage analysis is used to analyse the data for screening experiments. First, the main effects model is fitted to identify

“active” factors. Secondly, a stepwise regression procedure is used for selecting a suitable fitted model that may contain some interactions as well as main effects, but which obeys functional marginality (McCullagh and Nelder, 1989). Functional marginality means that every term in the model must be accompanied by all terms marginal to it, whether these are large or small.

We draw a distinction between functional marginality and the strong heredity principle, defined by Chipman (1996) to be “the belief that for an interaction to be active. . . both corresponding main effects must also be active.” Marginality defines the class of models that might be fitted, whereas strong heredity describes a set of prior beliefs about which effects might be large.

Unfortunately, these two concepts have become blurred in the literature, but the distinction is crucial to the work presented here. We believe models that break functional marginality should not be formulated or used unless some good practical reason can be given for so doing. The reader who is interested in this issue can refer to Nelder (1997, 1998).

For a three-level design withk factors, we consider only first- and second-order effects when thek-factor second-order model is the maximal model of interest. Assume that there is a “true model” that explains the relationship between the response and the factors. Assume further that experimenters have some prior knowledge about the probabilities of each effect being in the true model in advance of running an experiment. Letπ1denote experimenter’s prior belief that

*“a linear term*xi *is in the true model”, which is defined as*
Pr(δ(xi) = 1) = π1,

whereδ(xi) indicates whether the linear effect of factor i is in the true model.

Under the functional marginality rule, a quadratic effect term may be in the model only if
the linear effect of the same factor is already in the model. Therefore, we defineπ2as the prior
*probability that a quadratic effect is in the true model given that the linear effect of the same*

*factor is in the model, so that*

Pr(δ(x^{2}_{i}) = 1 | δ(xi)) =

π2 ifδ(xi) = 1;

0 ifδ(xi) = 0.

Similarly, we defineπ3 *as the prior probability that an interaction is in the true model given*
*that the linear effects of both the factors involved are in the model, so that*

Pr(δ(xixj) = 1 | δ(xi), δ(xj)) =

π3 ifδ(xi) = δ(xj) = 1;

0 otherwise.

These definitions imply that effects of the same type have the same probability of being in the true model, and that the event that a linear effect is in the true model is independent of the event that any other factor’s linear effect is in the true model. Also, for a given set of linear effects being in the model, the inclusion of second order effects in the model is independent of each other.

Note that the definition ofπ1 does not denote the probability that the linear effect of factor
i is large, but rather the probability that term xi*, or any effect to which it is marginal, is large.*

This is different from the prior probabilities defined by Chipman (1996). Detail discussion on the relationship between the definition above and strong heredity and weak heredity principles defined by Chipman (1996) can be found in Appendix A.

Given prior believe of each factorial effect being in the model, one can explicitly calculate the prior probability of each model being the true model. LetPr(Ms) denote such prior prob- ability of modelMs being the true model under a specific prior, which is computed as the joint probability of each individual effect being in the true model.

Pr(Ms) = π_{1}^{a}(1 − π1)^{k−a}π_{2}^{b} (1 − π2)^{a−b} π_{3}^{c}(1 − π3)^{a(a−1)}^{2} ^{−c}, (1)
wherea, b and c are the numbers of linear, quadratic and interaction effects that are in the model,
respectively.

We suggest use this information as a reference to help experimenters to decide the appro- priate run-sizes that they should use. First, we compute the probabilities of each of the models being the true model. Second, we add up the probabilities for models with a specific number of parameters. This sum represents the probability that a model with this number of parameters is the true model. Finally, a histogram of these added probabilities over various numbers of parameters is plotted. Experimenters can use this histogram plot as a reference to decide if the experimental run-size they are going to use is appropriate for their prior beliefs.

**Example**

Consider an 18-run experiment for six factors in which each factor has three levels. When the
experimenters’ prior beliefs are, for example, π_{1} = 1, π_{2} = 1 and π_{3} = 0.5, we calculate
the prior probability of each model being the true model. We then add up the probabilities for
models with the same number of parameters. Figure 1 gives the histogram plot of the added
probabilities of models with different numbers of parameters, from 1 to 28, being the true model
whenπ1 = 1, π2 = 1 and π3 = 0.5. The value beside the dash line in the plot is the cumulative
probability for the models with the number of parameters not greater than a given value being
the true model. For example, Figure 1 shows that there is just a 15% chance that a model with no
more than 18 parameters will be the true model for this prior. Clearly, an 18-point experiment
is not suitable to accommodate such prior. It is better to increase the experimental run-size or to
allow for follow-up experiments. By contrast, if the experimenters’ prior beliefs areπ1 = 0.8,
π_{2} = 0.7 and π_{3} = 0.3, Figure 2 shows that there is a 97.6% chance that a model with no
more than 18 parameters will be the true model. It can be concluded that an 18-point design is
appropriate for this prior.

It is clearly seen that this histogram plot can be used a useful tool to help experimenters to decide the appropriate run-sizes for their experiments. If experimenters believe that most factors will have non-negligible effects and models with a larger number of parameters are more likely to be the true model, it is better to increase the experimental run-size or to allow for follow-up

0 5 10 15 20 25 30 0

5 10 15 20 25

50%

15.1%

85%

98.2%

Number of parameters

Probability (x100)

Figure 1: Probabilities for models with various numbers of parameters being the true model for a three-level design for six factors whenπ1 = 1, π2 = 1 and π3 = 0.5.

0 5 10 15 20 25 30

0 5 10 15 20 25

97.6%

84.2%

55%

Number of parameters

Probability (x100)

Figure 2: Probabilities for models with various numbers of parameters being the true model for a three-level design for six factors whenπ1 = 0.8, π2 = 0.7 and π3 = 0.3.

experiments to accommodate their prior belief.

**3** **Design criterion**

To incorporate experimenters’ prior belief into design process, we define a new criterion for
selecting orthogonal, or nearly orthogonal, main effects designs, with robustness to interactions
as a secondary. This new criterion, denoted by QB(Γ^{(k)}), is defined as an average of an ap-
proximation toAs-efficiency, ignoring the intercept, over all eligible models with each model,
weighted by the prior probability of the model.

As defined in Section 2, we use{π1,π2,π3} to represent, respectively, experimenters’ prior probability that a linear effect is in the true model, the prior probability that a quadratic effect is in the true model given that the linear effect of the same factor is in the model, and the prior probability that an interaction effect is in the true model given that the linear effects of both the factors involved are in the model. The probability that modelMsis the true model for a given prior can then be computed as the joint probabilities of these effects being in the true model, as defined in equation (1).

For ak-factor three-level design, suppose that there are n0possible models under functional
marginality rule for the two-stage analysis. Among them, models with more thanN parameters
are not estimable for anN -point design. Those models whose number of parameters is greater
than*N is called the non-eligible models, and they will not be formulated in the second stage of*
the model-fitting procedure. In the two-stage analysis, one of eligible model will be the final
fitted model. Therefore, we define an adjusted probability, denoted by ˜Pr(Ms), to represent the
*probability that “a particular model is the best of the eligible models”. The adjusted probability*

is defined as

Pr(M˜ s) =

Pr(Ms) if|Ms| < N
Pr(Ms) + _{m}^{γ} if|Ms| = N

0 if|Ms| > N

(2)

where|Ms| is the number of parameters in model Ms, γ is the sum of the prior probabilities for all non-eligible models, andm is the number of models with exactly N parameters. Here, we reallocate the probabilities from non-eligible models between all models one size smaller.

Clearly, if a model is more likely to be the best model, the model should be given more weight in the design criterion.

Thus, the QB(Γ^{(k)}) criterion can be computed as an average of an approximation to As-
efficiency, ignoring the intercept, over the eligible models with each model, weighted by the
adjusted probability that the model is the best model, which is

QB(Γ^{(k)}) =

n0

X

s=1 v

X

i=1 v

X

j=0

rij Ms(i, j) ˜Pr(Ms) (3)
where v = 2k + ^{}^{k}_{2}^{} is the maximal number of effects of interest for a k-factor three-level
design, and n0 is the number of eligible models. SymbolMs(i, j) is an indicator representing
whether termsi and j are in the model Ms, which is

Ms(i, j) =

1 if termsi and j are both in model Ms

0 otherwise Symbolrij, fori, j = 0, . . . , v, is defined as

rij = 1 aii

a^{2}_{ij}
aiiajj

,

where aij are the elements in the information matrix for the k-factor second order model whose values are determined by the selection of designs. rijis a kind of measure for the non- orthogonality among effects for the second-order polynomial model of interest.

Letpij denote the sum of the adjusted probabilities for models containing both termsi and j, which is

pij =

n0

X

s=1

Pr(M˜ s)Ms(i, j).

ThenQB(Γ^{(k)}) criterion defined in (3) can also be written as
QB(Γ^{(k)}) =

v

X

i=1 v

X

j=0

rij pij,

Clearly, theQB(Γ^{(k)})-criterion does not require the inversion of the information matrices for the
models of interest. It is simple and computationally inexpensive, and therefore it can quickly
provide us with information on the worth of a design over a wide range of models.

This QB(Γ^{(k)})-criterion generalised the Q(Γ^{(k)}*)-criterion defined by Tsai et al. (2000)*
which weighs all models equally. Here, we study the relationship between this criterion and
the the generalised minimum aberration criterion. For simplicity, we compare this criterion
with theG2-aberration criterion proposed by Tang and Deng (1999) for designs with two levels.

Let Xd be anN × k matrix of 1’s and −1’ for an N run (N even) design for k two-level
factors, where each column hasN/2 1’s and N/2 −1’s corresponding to a factor’s main effect
and each row represents a factor-level combination. Consequently, the s-factor interactions
oare represented by the componentwise product of s columns of Xd. s = {1, . . . , k}. Let
Bs be the sum of squares all the componentwise products of s-columns of Xd divided by N^{2}.
The minimum G_{2}-aberration criterion proposed by Tang and Deng (1999) is to sequentially
minimizeB1,B2,. . . , Bk.

For ak-factor two-level design, when the k-factor second-order model is the maximal model
of interset, we have all the diagonal entries of the information matrix of the second-order model
equalN , i.e., aii = N , for i = 0, . . . , k +^{}^{k}_{2}^{}. Thus,rij = ^{a}_{N}^{2}^{ij}3. LetF = ^{}^{k}_{2}^{}, and

n0 = n(δ(0)) be the number of total models for a k-factor two levels designs, n0 =^{P}^{k}_{i=0}^{}^{k}_{i}^{}2(^{i}^{2});

n1,0 = n(δ(xi)) be the total number of models that contain main effect of factor i, n1,0 =

P_{k−1}

i=0

k−1 i

2(^{i+1}^{2} );

n2,0 = n(δ(xi), δ(xj)) be the total number of models that contain the main effects of factors i
andj, n2,0 =^{P}^{k−2}_{i=0} ^{}^{k−2}_{i} ^{}2(^{i+2}^{2} ) ;

n_{2,1} = n(δ(xi), δ(xixj)) be the total number of models that contain main effects of factor i
and a two-factor interaction involving the same factor,n2,1 =^{P}^{k−2}_{i=0} ^{}^{k−2}_{i} ^{}2(^{i+2}^{2} )^{−1};
n_{3,1} = n(δ(xi), δ(xjxl)) be the total number of models that contain the main effect of factor i

and a two-factor interaction not involving the same factor,n3,1 =^{P}^{k−3}_{i=0} ^{}^{(k−3)}_{i} ^{}2(^{(i+3)}^{2} )^{−1};
n_{3,2} = n(δ(xixj), δ(xixl)) be the total number of models that contain two interactions that

have one factor in common,n3,2 =^{P}^{k−3}_{i=0} ^{}^{k−3}_{i} ^{}2(^{i+3}^{2} )^{−2};

n4,2 = n(δ(xixj), δ(xlxm)) be the total number of models that contain two interactions that
have no factor in common,n4,2 =^{P}^{k−4}_{i=0} ^{}^{k−4}_{i} ^{}2(^{i+4}^{2} )^{−2}.

Then theQ(Γ^{(k)}*) defined by Tsai et al. (2000) can be written as*
Q(Γ^{(k)}) = 1

n_{0}N {kn1,0+ F n2,1+ [2n2,0+ n2,1+ 2(k − 2)n3,2] B2+ 6n3,1B3+ 6n4,2B4}
(4)
which is a linear combination of B2, B3 andB4 with decreasing weights. It showns that the
G2-aberration criterion can provide a good surrogate for model robustness.

Consider an alternative model space for ak-factor two-level design with k main effects and
f two-factor interactions, where f is a positive integer such that f ≤ F . Under this model
space, we haven_{0} = n_{1,0} = n_{2,0} = ^{}^{F}_{f}^{}, n_{2,1} = n_{3,1} = ^{}^{F −1}_{f −1}^{} = _{F}^{f}n_{0}, and n_{3,2} = n_{4,2} =

_{F −2}

f −2

= _{F}^{f(f −1)}_{(F −1)}n0. Therefore, the corresponding criterion which averages of an approximation
toAs-efficiency, ignoring the intercept, over the the models withk main effects and f two-factor
interactions is as

1 N

(

k + f +

"

2 + f

F + 2(k − 2)f (f − 1) F (F − 1)

#

B_{2}+ 6f

FB_{3}+ 6 f (f − 1)
F (F − 1)B_{4}

)

which is the same as the results in Cheng, Deng and Tang (2002) that the minimum G2-
aberration criterion sequentially minimizing B_{1}, B_{2}, . . . , Bk indirectly takes efficiencies into
account and provides a kind of overall measure of the partial aliasing and correlation among
factorial effects.

TheQB(Γ^{(k)}) criterion which generalised the Q criterion with each model weighted by the
prior probability of all the models. Therefore, if we partition the the sum of the adjusted proba-
bilities for models containing both termsi and j to be qi,j the sum of the adjusted probabilities
for models containing i main terms and j interactions. As done in (4), the QB(Γ^{(k)}) can be
written as

Q(Γ^{(k)}) = 1

q0N {kq1,0+ F q2,1+ [2q2,0+ q2,1+ 2(k − 2)q3,2] B2+ 6q3,1B3 + 6q4,2B4} (5) which is a linear combination ofB2,B3andB4 whereqi,j are determined by the given prior.

There is a conceptual similarity between thisQB-criterion and the BayesianD-criterion sug- gested by DuMouchel and Jones (1994) who proposed a Bayesian modification of D-optimal designs to reduce the dependence of D-optimal designs on the choice of an assumed model for looking at designs that allow the precise estimation of some primary terms while providing detectability for the potential terms. Other useful criteria could be constructed based on similar ideas. The BayesianD criterion is perhaps more widely applicable than the QBcriterion, which we recommend only for main effects designs when the robustness to interactions is a secondary consideration. However, this criterion which work with the inverse of a matrix is computation- ally more expensive and allows us to search over only a few designs, whereas for the specific problem of obtaining main effects designs, theQB criterion allows us to consider all possible designs.

**4** **A case study**

Consider an experiment described by Logothetis (1990), who gave a detailed analysis of ex-
perimental data from a plasma etching process. The aim of the experiment was to identify the
effects of six factors, labelled F_{1}–F_{6}, on the etch rate (in10^{−10}m/min) of the aluminum-silicon
layer placed on the surface of an integrated circuit. The six factors, each at three levels, were
assigned to the columns of an L18(3^{6}) array given in Table 1. For each of the 18 experimen-

Table 1: An electronics experiment
*Factor*

*Run* F_{1} F_{2} F_{3} F_{4} F_{5} F_{6} *Mean*

1 − − − − − − 4900

2 − − + + 0 0 5664

3 − 0 0 0 0 0 5777

4 − 0 − − + + 6224

5 − + + + + + 9195

6 − + 0 0 − − 5569

7 0 − − 0 0 + 12810

8 0 − 0 + − + 5391

9 0 0 0 + + − 9591

10 0 0 + − 0 − 5074

11 0 + + − − 0 5565

12 0 + − 0 + 0 5203

13 + − + 0 + − 4999

14 + − 0 − + 0 8540

15 + 0 − + − 0 10750

16 + 0 + 0 − + 12143

17 + + − + 0 − 6024

18 + + 0 − 0 + 8806

tal runs, two wafers were tested simultaneously at two levels of over-etch time. Finally, the
etch rate was measured at five fixed locations on each wafer. Table 1 gives the mean etch rate
across the 10 observations for each of the 18 experimental runs. As analysed by Tsai, Gilmour
and Mead (1996), the final model selected for the interpretation of the data is a model which
contains linear effects ofF1, F2, F5 and F6 and the linear by linear interaction ofF1 andF5.
*Logothetis (1990), Fearn (1992), Tsai et al. (1996) and Cox and Reid (2000) discussed other*
aspects of the analysis of these data.

Table 3 lists some of the best designs for six factors in 18 runs generated by the procedure in
*Tsai et al. (2000). It has been shown that, by using their procedure, more designs are generated.*

All the designs can all be generated by their procedure are equally efficient for fitting the main effects model and any sub-model of it. Nevertheless, for the two-stage analysis that considers

interactions in addition to main effects, these designs are different. The best 18-run designs
with six factors that they generated have better projection properties and are more efficient than
those obtained by the procedure of Cheng and Ye (2004) and those from theL_{18}(3^{7}). Design
25 in Table 3 is the preferred design with 6 factors in Cheng and Ye (2004). L18 2 is one of the
two six-factor designs that is obtained fromL_{18}(3^{7}). The other is in Table 1 and will be referred
*to as L18 1 in Table 3. Tsai et al. (2004) noted that the designs studied in Cheng and Ye (2004)*
and obtained from theL_{18}(3^{7}) always project onto designs for three factors that are formed by
putting two regular 3^{3−1} factorials together. Design 11 in Table 3 is the best design generated
*by Tsai et al. (2000) that has this three-factor projection property.*

We useQB(Γ^{(6)}) to compare these orthogonal main effects designs for various priors. Table
3 gives the corresponding values of QB(Γ^{(6)}) for the designs given in Table 2. The boldface
forQB(Γ^{(6)}) indicates that the design has the lowest QB(Γ^{(6)}) for the particular prior. Clearly,
depending on the prior knowledge, slightly more appropriate designs can be found, and the de-
signs chosen by the modified criterion are clearly better than the one actually used and are better
than the best design in Cheng and Ye (2004) under several priors. In general, designs with small
values of Q(Γ^{(k)}) tend to have small values of QB(Γ^{(k)}). For picking up the appropriate class
of designs, exact specification of prior probabilities is not crucial. The best designs constructed
in this way are very robust to prior specifications, so experimenters can have great confidence
in using them. Nonetheless, they can use their prior probabilities to select a specific best design
for any particular experiment.

**5** **Conclusion**

In this paper, we studied a good way to assess orthogonal main effects designs, with robustness
to interactions as a secondary consideration when experimenters have some prior information
about the probability of effects being non-negligible in advance of collecting their data. In this
paper, we used theQB(Γ^{(k)})-criterion to compare all the 440 three-level main effects designs

Table 2: Some orthogonal main effects designs for six factors in 18 runs

Design 1

−−−−− 0

−− 0 + + −

− 0 − 0 0 −

− 0 + −+ +

−+ 0 0 −+

−+ + + 0 0 0−+ − 0 − 0−+ 0 −+

0 0− + + + 0 0 0 0 0 0 0 +− + −−

0 + 0−+ 0 +−− 0 + 0 +− 0 + 0 + + 0 0−−−

+ 0 + +− 0 + +−− 0 + + + + 0 +−

Design 2

−−−− 0 −

−− 0 + − 0

− 0 − 0 −+

− 0 + − + 0

−+ 0 0 + −

−+ + + 0 + 0− 0 0 + + 0−+ −−+

0 0− + + 0 0 0 + 0 0− 0 +− + −−

0 + 0− 0 0 +−− 0 0 0 +−+ + + − + 0 0−−−

+ 0 0 + 0 + + +−− + + + + + 0− 0

Design 3

−−−−− 0

−− 0 + 0 −

− 0 − 0 + −

− 0 + − 0 +

− + 0 0 −+

− + + + + 0 0− 0 0 0 0 0−+ − + − 0 0−+ 0 + 0 0 + 0− 0 0 +−+ −−

0 + 0− + + +−− 0 + + +−+ + −+

+ 0 0−−−

+ 0 0 + + 0 + +−− 0 0 + + + 0 0−

Design 4

−−−− 0 −

−− 0 0 − +

− 0 −+ + +

− 0 + + − 0

− + 0 −+ 0

− + + 0 0 − 0− + −+ 0 0− + + 0 + 0 0− 0 + − 0 0 0 0 0 0 0 +−−− + 0 + 0 +−−

+−− 0 − 0 +− 0 + + − + 0 0− 0 + + 0 +−−−

+ +−+ 0 0 + + + 0 + +

Design 5

−−−− 0 0

−− 0 0 −−

− 0 −+ + +

− 0 + + − 0

−+ 0 −+ −

−+ + 0 0 + 0− + −+ + 0− + + 0 − 0 0−−−−

0 0 0 0 0 0 0 +− 0 + 0 0 + 0 +− + +−− 0 − + +− 0 + + 0 + 0 0− 0 + + 0 + 0 +− + +−+ 0 − + + +−− 0

Design 8

−−−− 0 −

−− 0 + −+

− 0 − 0 − 0

− 0 + −+ +

−+ 0 0 + −

−+ + + 0 0 0− 0 0 + 0 0−+ −− 0 0 0− + + + 0 0 + 0 0− 0 +− + −−

0 + 0− 0 + +−− 0 0 + +−+ + + − + 0 0−−−

+ 0 0 + 0 0 + +−−+ 0 + + + 0−+

Design 11

−−−−−−

−−+ 0 + 0

− 0 − + + +

− 0 0 − 0 0

−+ 0 0 −+

−+ + + 0 − 0− 0 0 0 − 0−+ + −+

0 0− 0 − 0 0 0 +− + − 0 +−− 0 + 0 + 0 + + 0 +−− + 0 0 +− 0 − + + + 0 0 +−−

+ 0 + 0 0 + + +− 0 + − + + +−− 0

Design 25

−−−−−−

−−+ 0 0 +

− 0 −+ 0 0

− 0 0 − + +

− + 0 0 − 0

− + + + + − 0− 0 0 + − 0−+ + − 0 0 0− 0 −+

0 0 +− 0 − 0 +−− + 0 0 + 0 + 0 + +−−+ + + +− 0 − 0 0 + 0 0 +−−

+ 0 + 0 + 0 + +− 0 0 − + + +−−+

L18 2

−−−−−−

−− 0 0 + +

− 0 −+ 0 +

− 0 + −+ 0

− + 0 + − 0

− + + 0 0 − 0−−+ + 0 0− + − 0 + 0 0 0 0 0 0 0 0 + +−−

0 +− 0 − + 0 + 0−+ − +− 0 + 0 − +− + 0 − 0 + 0− 0 + − + 0 0−− + + +−− 0 0 + + + + + +

Table 3: Comparison ofL18(3^{6}) designs under several priors

Q_{B}(Γ^{(6)}) Design

(π_{1}, π_{2}, π_{3}) D1 D2 D3 D4 D5 D8 D11 D25 L18 1 L18 2

(1, 1, 0.5) **272.15** 273.60 272.97 276.06 273.09 276.05 276.76 282.57 293.11 293.12
(1, 1, 0.2) 192.90 191.41 193.06 194.14 192.13 **191.35** 192.63 195.39 201.00 194.24
(0.8, 0.9, 0.2) 130.75 129.79 130.81 131.20 130.50 **129.79** 130.54 131.51 134.90 131.21
(0.8, 0.9, 0.3) 152.72 **151.79** 152.86 153.67 152.33 151.90 152.78 154.71 159.01 154.65
(0.8, 0.8, 0.6) 200.62 200.72 201.04 203.00 **200.31** 201.52 202.28 206.56 212.34 209.26
(0.8, 0.7, 0.3) 139.59 **138.64** 139.72 140.20 139.77 139.05 139.95 140.84 146.13 142.63
(0.8, 0.6, 0.6) **189.97** 190.13 190.39 192.01 190.34 191.30 192.04 195.16 202.04 200.13
(0.8, 0.5, 0.7) **199.35** 199.96 199.84 201.69 199.97 201.48 202.09 205.52 212.83 212.12
(0.7, 0.7, 0.4) 128.19 **127.53** 128.33 128.89 128.23 127.86 128.57 129.71 133.92 131.03
(0.6, 0.9, 0.5) 120.04 119.54 120.15 120.87 119.56 **119.53** 120.06 121.87 124.23 121.24
(0.67, 0.8, 0.7) 135.38 135.12 135.57 136.63 **134.87** 135.27 135.80 138.33 141.14 138.30
(0.67, 0.7, 0.5) 132.79 **132.29** 132.95 133.71 132.71 132.63 133.29 134.91 138.89 136.13

*with six factors in 18 runs generated by using the procedure by Tsai et al. (2000). We showed*
that, depending on the prior knowledge, more appropriate designs can be found among the class
of good designs. The best designs constructed in this way are very robust to prior specifications,
so experimenters can have more confidence in using them.

In this paper, the QB(Γ^{(k)})-criterion is used as a secondary consideration for orthogonal
main effects designs. Clearly, it can be used in exactly the same way for nearly orthogonal
designs. However, there is a question about how far we can go towards expecting large inter-
actions. If their prior probability becomes larger than that for quadratic effects, then we should
be using a different class of designs from orthogonal main effects plans. If this is the case, we
could use theQBcriterion as a primary, rather than a secondary criterion, and we could use this
criterion in an exchange algorithm to construct designs in a way similar to BayesianD-optimal
designs. However, the objectives for such designs will be different from those given in this
paper and we have not explored this possibility.

**References**

Beattie, S. D., Fong, D. K. H. and Lin, D. K. J. (2002) A two-stage Bayesian model selection
**strategy for supersaturated designs. Technometrics, 44, 55-63.**

*Box, G. E. P. and Meyer, R. D. (1986) An analysis for unreplicated fractional factorials. Tech-*
**nometrics, 28, 11-18.**

Cheng, C. S., Deng, L. W. and Tang, B. (2002) Generalized minimum aberration and design
**efficiency for nonregular fractional factorial designs. Statistica Sinica. 12, 991-1000.**

Cheng, S. W. and Wu, C. F. J. (2001) Factor screening and response surface exploration.

**Statistica Sinica, 11, 553-604.**

Cheng, S. W. and Ye, K. Q. (2004) Geometric isomorphism and minimum aberration for fac-
*torial designs with quantitative factors. To appear in The Annals of Statistics.*

*Chipman, H. (1996) Bayesian variable selection with related predictors. Canadian Journal of*
**Statistic, 24, 17-36.**

Chipman, H., Hamada, M. and Wu, C. F. J. (1997) A Bayesian variable-selection approach for
**analyzing designed experiments with complex aliasing. Technometrics, 39, 372-381.**

*Cox, D. R. and Reid, N. (2000) The Theory of the Design of Experiments. London: Chapman*

& Hall.

DuMouchel, W. and Jones, B. (1994) A simple Bayesian modification of D-optimal designs
**to reduce dependence on an assumed model. Technometrics, 36, 37-47.**

Fearn, T. (1992) Box-Cox transformations and the Taguchi method: an alternative analysis of
**a Taguchi case study. Applied Statistics, 41, 553-559.**

Hamada, M. and Wu, C. F. J. (1992) Analysis of designed experiments with complex aliasing.

**Journal of Quality Technology, 24, 130-137.**

*Logothetis, N. (1990) Box-Cox transformations and the Taguchi method. Applied Statistics,*
**39, 31-38.**

*McCullagh, P. and Nelder, J. A. (1989) Generalized Linear Models, 2nd edition. London:*

Chapman & Hall.

*Nelder, J. A. (1997) Letter to the Editors: Functional marginality is important. Applied Statis-*
**tics, 46. 281-282.**

Nelder, J. A. (1998) The selection of terms in response-surface models: how strong is the
**weak-heredity principle? American Statistician, 52, 315-318.**

Tang, B. and Deng, L. Y. (1999) MinimumG2-aberration for non-regular fractional factorial
**designs. The Annals of Statistics, 27, 1914-1926.**

Tsai, P.-W., Gilmour, S. G. and Mead, R. (1996) Letter to the Editors: An alternative analysis
**of Logothetis’s plasma-etching data. Applied Statistics, 45, 498-503.**

Tsai, P.-W., Gilmour, S. G. and Mead, R. (2000) Projective three-level main effects designs
**robust to model uncertainty. Biometrika, 87, 467-475.**

Tsai, P.-W., Gilmour, S. G. and Mead, R. (2004) Some new three-level orthogonal main effects
*designs robust to model uncertainty. To appear in Statistica Sinica.*

Wang, J. C. and Wu, C. F. J. (1995) A hidden projection property of Plackett-Burman and
**related designs. Statistica Sinica, 5, 235-250.**

**Appendix A**

Chipman (1996) defined the prior for a term of a given order being active is conditional on all terms of its lower order such as

Pr (xiis active) = p

Pr^{}x^{2}_{i} is active| δ(xi)^{}=

p0 ifδ(xi) = 0 p1 ifδ(xi) = 1

Pr (xixj is active| δ(xi), δ(xj)) =

p00 ifδ(xi) = δ(xj) = 0 p01 ifδ(xi) = 0 and δ(xj) = 1

orδ(xi) = 1 and δ(xj) = 0 p11 ifδ(xi) = δ(xj) = 1

In our definition,π1 is defined as the probability that a linear effectxi is in the true model,
which is the probability that term xi, or any effect to which it is marginal, is large. This is
the joint probability of the following events such thatxi is active andx^{2}_{i} is active and anyxixj

interaction, forj 6= i and j = 1, . . . , k, is active, which can be written as π1 = p + (1 − p)p0+ (1 − p)(1 − p0)[p01p + p00(1 − p)] +

(1 − p)(1 − p0){1 − [p01p + p00(1 − p)]}[p01p + p00(1 − p)] +

· · · + (1 − p)(1 − p0){1 − [p01p + p00(1 − p)]}^{k−2}[p01p + p00(1 − p)]

= p + (1 − p)p0+ (1 − p)(1 − p0)^{n}1 − {1 − [p01p + p00(1 − p)]}^{k−1}^{o}

π_{2} is defined as the probability that a quartic effect is in the true model given that the linear
effect of the same factor is in to model, so that

π2 = p1 (6)

Similarly, π3 is defined as the probability that an interaction effect is in the true model given that the linear effects of both the factors involved are in to model, so that

π3 = p11 (7)

Under the strong heredity principle defined by Chipman (1996) that the belief that for an interaction to be active both corresponding main effects must also be active, we havep0 = p00 = p01= 0. Therefore,

π1 = p. (8)

Under the weak heredity principle that the belief that for an interaction to be active one of the corresponding main effects must also be active, we havep00 = 0. Thus

π1 = p + (1 − p)p0+ (1 − p)(1 − p0)^{n}1 − [1 − p01p]^{k−1}^{o} (9)