OPTIMALITY ASPECTS OF DESIGNS

V.K. Gupta and Rajender Parsad I.A.S.R.I., Library Avenue, New Delhi-110012

1. Introduction Careful experimentation is an essential part of any problem of decision-making. An experiment is a device or means of getting an answer to the problem under consideration. In any sphere of scientific research, the general procedure that the experimenter has to adopt is to formulate the hypothesis according to the problem whose answer is to be sought for and then to verify the formulated hypothesis directly or by the consequences of the treatments used. This verification necessitates the collection of observations and the design of experiments is essentially the pattern of the observations to be collected. Perhaps it was John Stuart Mill who first gave clear prescriptions on how to carry out experiments and classified them into spontaneous experiments, what we would call now observational studies and artificial experiments namely controlled experiments. Mill and others were of the firm belief that controlled experimentation was better if the subject matter allowed it. The breakthrough into a more versatile approach to experimental design came with the work of Sir Ronald A. Fisher and his followers, notably, Frank Yates, at the Rothamsted Experimental Station. A number of useful concepts like balancing, orthogonality and alliasing were introduced. The usefulness of combinatorial arrangements for designing statistical experiments was evident to Sir R.A. Fisher and under his leadership, the construction and application of such designs enjoyed tremendous expansion. Fisher, his students and followers were responsible for much of the development during1930’s and 1940’s. Mathematicians and mathematical statisticians like R.C. Bose gave general structures to classes of these designs and invented general methods of constructing them. The aim of any experiment, by and large, is to compare a number of treatments on the basis of the responses produced in the experimental material. Whenever one is faced with the necessity of accepting one out of a set of alternative decisions, one has to conduct an experiment to generate data using which a statistical decision may be based. Experimental designs are, therefore, useful for either estimating some unknown parameters in the model or testing certain hypotheses (or assertions) about these unknown parameters, generally some contrasts about the treatment effects. The confidence and accuracy with which treatment differences can be assessed depend to a large extent upon the size of the experiment in the sense of number of experimental units used and also on the inherent variability present in the experimental material, besides the variability arising because of the application of treatments. In order that it may be possible to select an optimum decision procedure, the choice of the experimental design must also be optimum in the sense that the design is good in terms of some meaningful statistical criterion or a family of criteria with respect to a given problem and in a given class of designs defined by the number of treatments and other design parameters depending upon the experimental setting. This is how the problem of optimal designing of experiments arises.

Optimality Aspects of Designs

In the initial stages of development of experimental designs, emphasis was laid on constructing designs that were in some sense as symmetric as possible in their treatment of the statistical parameters of interest (for example, balanced incomplete block designs, Latin square designs, Youden square designs, etc.). This effort perhaps stemmed from three factors. Such designs yielded information matrices (coefficient matrices of the reduced normal equations for least squares estimation of estimable functions of treatment effects) that, especially in the pre-computer age, made statistical calculations easy; the designs had aesthetic appeal to mathematicians and often had algebraic or geometric representations that helped one to understand and construct them; and the symmetric treatment of parameters of interest seemed a reasonable property that made such designs yield statistical estimators that looked intuitively as accurate as possible for the given number of observations. The advent of high speed computers and importance and need to choose and adopt an experimental design that is best according to some well defined statistical criterion, led to the development of subjects like optimality of designs. The theory of optimal designs was almost non-existent till about the end of the Second World War, except for a remarkable early paper by Smith (1918) and the important paper of Wald (1943). Prof. Jack K. Kiefer (1958) initiated the serious and rigorous work on optimality aspects of designs. For the given experimental situation and an inference problem, there are a number of designs available, called a class of designs, which can be used to achieve some specified set of objectives. In the sequel, we attempt to expose the problem of optimality aspects of designs through the following examples: Example 1: Consider an experimental situation where an agricultural scientist is interested in making all possible paired comparisons among the v = 8 treatments tried in the experiment. The experimenter has n = 56 experimental units available with him. The 56 experimental units are not homogeneous and can be grouped into b = 14 blocks of k = 4 homogeneous experimental units each. Therefore, here the appropriate class of designs

for estimating all the = 28 paired comparisons (treatment contrasts) among 8

treatments is D(v, b, k), where D(v, b, k) is the class of all connected designs in which v (=8) treatments are arranged in b (=14) blocks each of size k (=4). The class of designs D contains many designs, two of which are as given below:

⎟⎟⎠

⎞⎜⎜⎝

⎛28

D1: v = 8, b = 14, r = 7, k = 4, λ = 3 D2: v = 8, b = 14, k = 4, r1 = r2 = … = r7 =

6, r8 = 14 1 2 3 4 5 6 7 3 4 5 6 7 1 2 1 2 3 4 5 6 7 1 2 3 4 5 6 72 3 4 5 6 7 1 5 6 7 1 2 3 4 2 3 4 5 6 7 1 3 4 5 6 7 1 24 5 6 7 1 2 3 6 7 1 2 3 4 5 4 5 6 7 1 2 3 4 5 6 7 1 2 38 8 8 8 8 8 8 7 1 2 3 4 5 6 8 8 8 8 8 8 8 8 8 8 8 8 8 8

In D1, the arrangement of 8 treatments into 14 incomplete blocks forms a balanced incomplete block (BIB) design [A BIB design is an arrangement of v treatments in b blocks of size k (<v) such that every treatment appears at most once in a block, every treatment appears in exactly r blocks and every pair of treatments appears together in λ blocks]. Using the design D1, all the 28 elementary contrasts are estimated with same

420

Optimality Aspects of Designs

variance = (5/15)σ2. Here σ2 is the intra block variance. The average variance of all the 28 elementary contrasts is (140)/(15 x 28))σ2. On the other hand, using the design D2,

out of a total of 28 elementary contrasts, = 21 elementary contrasts among the first 7

treatments are estimated with same variance (6/15)σ

⎟⎟⎠

⎞⎜⎜⎝

⎛27

2 and each of the remaining 7 contrasts of treatment 8 with each of the first 7 treatments are estimated with variance (4/15)σ2. The average variance of all the 28 elementary contrasts is (154)/(15 x 28))σ2. Thus, one can see that if the interest of the experimenter is in estimating all the 28 elementary contrasts, then D1 is better than D2 on the basis of the average variance of the best linear unbiased estimates of all the possible elementary contrasts (This criterion is also known as A-optimality). However, if the treatment 8 is a control treatment and 1,2,…,7 are test treatments and the interest of the experimenter is to make test treatments vs control comparisons and comparisons among test treatments are of no interest, then the average variance of all the 28 estimated elementary treatment contrasts is not a meaningful criterion. In this case the average variance of the 7 estimated elementary contrasts of treatment 8 with each of the first 7 treatments is a meaningful criterion. On comparing D1 with D2 one finds that the design D2 is a better design than D1 for making test treatments-control comparisons. Example 2: Consider another experimental situation where the experimenter is interested in making all the possible comparisons among the v = 5 treatments tried in the experiment. The experimenter has n = 21 experimental units, which can be arranged in b = 7 blocks of arbitrary sizes. Therefore, here the appropriate class of designs for

estimating all the =10 paired comparisons among 5 treatments is D(v, b, n), where

D(v, b, n) is the class of all connected block designs in which v(=5) treatments are arranged in b(=7) blocks such that total number of experimental units is n(=21). The class of designs D contains many designs, two of which are

⎟⎟⎠

⎞⎜⎜⎝

⎛25

D1: v = 5, b = 7, k = 3, r1 = r2 = r3 = r5 = 4, r4 = 5

D2: v = 5, b = 7, r1 = 5, r2 = r3 = r4 = r5 = 4, k1 = k2 = 4, k3 = k4 = k5 = k6 = 2, k7 = 5

1 1 2 2 1 1 3 2 2 1 1 1 1 1 3 4 3 4 2 2 4 3 3 2 3 4 5 2 5 5 5 5 3 4 4 4 4 3 5 5 4 5

Both the designs D1 and D2 are variance balanced [A connected design is said to be a variance balanced design if it permits the estimation of all the possible paired comparisons among the v treatments with the same variance; A design is said to be connected if it allows the estimation of all the possible elementary contrasts through the design. For a connected design, the rank of the information matrix pertaining to the estimation of estimable functions of treatment effects is v – 1, where v is the number of

421

Optimality Aspects of Designs

treatments]. Using D1 and D2, all the 10 elementary contrasts are estimated with variances (21/35)σ2 and (20/35) σ2 respectively. Thus we see that D2 is a better design than D1 on the basis of average variance of the estimated elementary contrasts. Although both the designs are variance balanced and estimate all the elementary contrasts with same variance, yet the variance from both the designs is not same, suggesting thereby that all variance balanced designs are not equally efficient and a choice is to be made among the variance balanced designs as well. Example 3: Suppose an experimenter is interested in comparing 10 treatments via 15 experimental units arranged in 5 blocks of size 3 each. Therefore, here the class of designs is D (v, b, k) for v = 10, b = 5, k = 3. The class of designs D contains many designs, two of which are:

D1 D2 1 3 5 7 9 1 3 5 7 9 2 4 6 8 10 2 4 6 8 10 3 5 7 9 1 3 1 7 9 2

The determinant of the variance-covariance matrix (or the generalized variance) of the Best Linear Unbiased Estimator (BLUE) of the linear functions of treatment effects is (35)/100 using D1 and (35)/40 using D2 assuming error variances same from both the designs. Thus D1 is better than D2 as per the determinant criterion [This criterion is also known as D-optimality]. Example 4: Consider an experimental setting where the experimenter has to compare v = 4 treatments via n =15 experimental units arranged in b = 5 blocks of size k = 3 each. Here also class of designs is D(v, b, k) for v = 4, b = 5, k = 3. The two designs considered, among the many designs in the class D, are:

D1 D2

1 1 1 2 1 1 1 1 2 1 2 2 3 3 2 2 2 3 3 1 3 4 4 4 3 3 4 4 4 1 For both the designs the positive (non-zero) eigenvalues of the information matrix are 8/3, 11/3, 11/3 and 8/3, 8/3, 8/3 respectively. D2 is a variance-balanced design, but D1 is not. If we decide to choose a design using the criterion that the smallest eigenvalue of the information matrix of the design is maximized, then the two designs are equivalent. This criterion, in fact, amounts to choosing that design for which the maximum variance of the BLUE of any normalized treatment contrast is minimum [This criterion is also known as E-optimality]. However, on the average variance and generalized variance criterion, the design D1 is better than the design D2. This example also shows that a non-variance-balanced design may sometimes score over a variance-balanced design on the basis of some well-defined criteria. In many exploratory experiments, the primary interest of the experimenter is not to estimate arbitrary treatment contrasts, but rather to optimally estimate the differences in

422

Optimality Aspects of Designs

the effects that treatments under study have on the various experimental units. In other words, we are primarily interested in paired treatment comparisons of the type τi-τj and an optimal design may seek to minimize the maximum variance of the corresponding BLUEs. Such a criterion may not be exclusively a function of the eigenvalues of the underlying C-matrix. For example, let

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−−−−

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−−

=51411094913

1477770707

21 CC ,

be two competing C-matrices. It may be checked that both C1 and C2 have eigenvalues equal to θ1 =7 and θ2=21. The variance of BLUEs of all paired comparisons obtained by using C1 are respectively (2/7)σ2, (1/7)σ2, (1/7) σ2 while those obtained by using C2 are respectively (5/49) σ2, (10/49) σ2, (13/49) σ2. Thus, one can see that when we talk of all paired comparisons, the maximum variance of the BLUEs of paired comparisons have no relationship with the smallest eigenvalue. Example 5: An experimenter is interested in generating a mating design for comparing 7-inbred lines on the basis of their gca effects. A mating design for diallel crosses experiment, D, with 21 crosses can be obtained by writing all possible pairs of treatments within a block of the BIB design, D0, with parameters v = b = 7, r = k = 3, λ = 1 and treating the treatments as lines and paired treatments as crosses. Here the number of crosses is v = 21, b = 7, r = 1, k = 3. The designs, with rows as blocks, are

D0 D D*

1 2 4 1x2 1x4 2x4 1x7 2x6 3x5 2 3 5 2x3 2x5 3x5 1x2 3x7 4x6 3 4 6 3x4 3x6 4x6 2x3 1x4 5x7 4 5 7 4x5 4x7 5x7 3x4 2x5 1x6 5 6 1 5x6 1x5 1x6 4x5 3x6 2x7 6 7 2 6x7 2x6 2x7 5x6 4x7 1x3 7 1 3 1x7 3x7 1x3 6x7 1x5 2x4

The matrix of the design D is NNKGC ′−= −1 )7

(37

77 JIC 1−= , and the variance of

the BLUE of any elementary contrast among lines (gca effects) is 2σ76 . Here s

an identity matrix of order v and a vxv matrix of all elements ones, respectively, and is the per plot variance.

vv JI , i

2σ

Another mating design generated through a different method is D*. The

matrix of the design D* is NNKGC ′−= −1 )( 77 JIC71

314

−= , and the variance of the

423

Optimality Aspects of Designs

B.L.U.E. of any elementary contrast among lines (gca effects) is 2σ73 . Thus, one can

see that the design D* estimates the gca effects with twice the precision as obtained through the design D although both the designs are variance balanced for estimating any normalized contrast of gca effects. Example 6: An experimenter is interested in comparing v = 4 treatments. There are n = 24 experimental units available with the experimenter. These 24 experimental units can be arranged in b = 6 blocks of sizes k = 4 each such that there are two rows and two columns in each of the blocks. The affordable replication of the treatments is r = 6. Two competing designs in a given class D(v, b, k, p, q), with v treatments arranged in b blocks of size k = pq each, the k experimental units within each block arranged into p rows and q columns to accommodate the heterogeneity nested in two directions within each block, are:

D1: v = 4, b = 6, r = 6, p = q = 2.

1 2 1 3 1 4 1 2 1 3 1 4 3 4 4 2 2 3 3 4 4 2 2 3

D2: v = 4, b = 6, r = 6, p = q = 2.

1 2 1 3 1 4 2 3 2 4 3 4

2 1 3 1 4 1 3 2 4 2 4 3

Both the designs, D1 and D2, are variance balanced. The variance of the BLUE of any elementary treatment contrast using design D1 is = ,

and using design D2 is = . It is thus seen that using the design D2 there is a 50% reduction in the variance of the BLUEs of the elementary treatment contrasts.

)ˆˆ( ji ttVar − ,4... 1,j i =≠∀,2σ

)ˆˆ( ji ttVar − ,4... 1,j i =≠∀,2/2σ

Example 7. Consider an equireplicated variance balanced block design with nested rows and columns D1 for v = 4 treatments arranged in b = 6 blocks of size k = 4 and with number of rows p = 2 and number of columns q = 2 in each block:

D1

1 2 1 3 1 4 2 3 2 4 3 4

2 1 3 1 4 1 3 2 4 2 4 3

Consider another non-proper variance balanced block design with nested rows and

columns D2 for v = 4, b = 4, block sizes k1 = 9 and k2 = k3 = k4 = 4, and with number of

rows and columns as p1 = q1 = 3 and pj = qj = 2, j = 2,3, 4:

424

Optimality Aspects of Designs

D2

1 2 3 1 4 2 4 3 4

2 3 1 4 1 4 2 4 3

3 1 2

D1 has 24 experimental units while D2 has 21 experimental units, but the variance of any estimated elementary treatment contrast obtained through either of the designs is σ2/2, where σ2 denotes the error variance. Thus, it may be seen that for same precision of the BLUEs of elementary contrasts, the number of experimental units required for design D2 is smaller than that in D1. Through these examples we have thus introduced the problem. The choice of an appropriate design for a particular setting depends upon (a) the inference problem whose answer is sought for; (b) the class of designs in which choice is to be made (the class should have more than one design); and (c) the criterion or criteria to be used for the selection of the design. Some commonly used optimality criteria are A-optimality, D-optimality, E-optimality, MV-optimality, S-optimality, MS optimality, φp - optimality, universal optimality, etc. The mathematical formulation of the problem is difficult and is beyond the scope of this course. Some important results are, however, given below: 2. Optimality Results Block Designs Theorem 1: For given v, b, k, a BIB design, whenever existent, is A-optimal for estimating all elementary contrasts in the class of all connected incomplete block designs. Theorem 2: For given v, b, k, a BIB design, if existent, is E-optimal. Theorem 3: For given v, b, k, a BIB design, if existent, is D-optimal for inferring on any full set of orthonormalized treatment contrasts in the class of all binary designs. Theorem 4: For given v, b, k, a most balanced group divisible design is type-1 or type-2 optimal. Theorem 5: A balanced incomplete block design extended by m disjoint binary blocks such that mk < v is E-optimal in D(v, b+m, k).

Theorem 6: A balanced incomplete block design from which m disjoint blocks (v/k2 ≤ m ≤ v/k) are deleted is E-optimal in D(v, b-m, k). Theorem 7: A most balanced group divisible design with m groups extended by less than (v - m)/k disjoint and binary blocks compatible with partition provided by the groups is E-optimal.

425

Optimality Aspects of Designs

Kiefer (1958) introduced Balanced Block Designs (BBD's) as a generalization of BIB designs to allow k > v. Definition (Kiefer, 1958): A design d∈D (v, b, k) is called BBD if

(i) for all i = 1,…,v rnb

jdij =∑

=1

(ii) for all i≠ m = 1,…,v. λ=∑=

dmjb

jdij n n

1

(iii) [ ] vk - ndij < 1.

Kiefer (1958) proved the A-, D-, E- optimality of BBD's in D (v, b, k). Subsequently, Kiefer (1975) showed that a BBD, whenever existent, is universally optimal in D (v, b, k). For k < v, a BBD is a BIB design. Das and Dey (1990) showed that (i) is redundant for v > 2. Sufficient conditions have been obtained to establish the E-optimality of block designs in a class of designs, which allows blocks to be of unequal sizes. Similarly, the concept of BBD has been extended to Generalized Binary Balanced Block Designs and Binary Balanced Block Designs to study the optimality aspects of block designs with unequal block sizes by Gupta, Das and Dey(1991). It has been shown that for given v, b, n, a Binary Balanced Block Design, whenever existent, is Universally optimal. For given v, b, k1,…,kb, a Generalized Binary Balanced Block Design, whenever existent, is universally optimal. The optimality aspects have also been studied under Heteroscedastic models by Gupta(1995). Row-Column Designs The earliest results on optimality of row-column designs are perhaps due to Wald (1943) who proved that any latin square design is D-optimum. Subsequently Kiefer(1975) studied the universal optimality of GYD's under regular settings and A-, D-, and E-optimality of GYD's under non-regular settings over the class D(v, p, q). The following results were obtained by Kiefer(1975). (1) In regular settings, a GYD is always universally optimal (2) In non-regular settings, a GYD is

(a) always A-optimal (b) always E-optimal (c) D-optimal unless v = 4.

Cheng (1981) obtained similar results for PYD's over D(v, p, p). Kiefer (1975) made the following remark "We do not know in which non-regular settings the GYD is still universally optimal". Das (1989) answered the query of Kiefer (1975) in negative. We now have the following theorem:

426

Optimality Aspects of Designs

Theorem 8: In non-regular settings, a GYD is never universally optimal in D(v, p, q). Jacroux (1982) obtained E-optimal row-column designs by adding one or two disjoint columns of treatments to a Youden Square design. Singh and Gupta (1991) generalized this result to obtain new E-optimal row-column designs. The result is given in the following theorem: Theorem 9: Let v) qp,D(v, d ρ== be a Youden and let d* be the row-column design obtained by adding x disjoint and binary columns of treatments to d, where 1 ≤ x < v/p. Then d* is E-optimal over D(v, p, q), where xqq += .

References Cheng, C.S (1978). Optimal designs for the elimination of multi-way heterogeneity. Ann.

Statist., 6, 1262-1272.

Cheng, C.S. (1981). Optimality and construction of pseudo Youden designs. Ann. Statist., 9, 201-205.

Chernoff, H. (1953). Locally optimal designs for estimating parameters. Ann.Math. Statist.,24, 586-602.

Das, A. (1989). Some investigations on optimal designs. Unpublished Ph.D. thesis, I.A.R.I., New Delhi.

Das,A. and Dey,A.(1989). A note on balanced block designs. J.Statist.Plann.Inf., 22, 265-268.

Das,A. and Dey, A.(1990). Optimality of row-column designs. Cal. Statist. Assoc. Bull., 39, 63-72.

Eccelston, J.A. and Hedayat, A. (1974). On the theory of connected designs: characterization and optimality. Ann. Statist., 2, 1238-1255.

Ehrenfeld, S. (1955). On the efficiency of experimental designs. Ann. Math. Statist., 26, 247-255.

Elfuing, G. (1952). Optimal allocation in linear regression theory. . Ann. Math. Statist., 23, 255-262.

Gupta,V.K.(1995). Universally optimal generalised binary balanced block designs of type alpha. Sankhya, B57, 420-427.

Gupta,V.K., Das,A. and Dey,A.(1991). Universal optimality of block designs with unequal block sizes. Statist.Prob.Letters, 11, 177-180.

Jacroux,M.(1982). Some E-optimal designs for one-way and two-way elimination of heterogeneity. J.Roy.Statist.Soc., B44, 253-261.

Jacroux,M.(1983). Some minimum variance block designs for estimating treatment differences. J.Roy.Statist.Soc., B45, 70-76.

Kiefer, J.(1958). On the non-randomized optimality and randomized non-optimality of symmetrical designs. . Ann. Math. Statist., 29, 675-699.

427

Optimality Aspects of Designs

Kiefer, J.(1959). Optimum experimental designs. J.Roy. Statist. Soc., B21, 272-313.

Kiefer,J.(1974). General equivalence theory of block designs(Approx. Theory). Ann.Statist., 2, 849-879.

Kiefer, J.(1975). Construction and optimality of generalized Youden designs. In a Survey of Statistical Designs and Linear Models (J.N.Srivastava, ed.), 333-353, North Holland, Amsterdam.

Shah, K.R. (1960). Optimality criteria for incomplete block designs. Ann. Math. Statist., 31, 791-794.

Shah, K.R. and Sinha, B.K. (1988). Theory of optimality og designs. Marcell Dekker.

Singh,R. and Gupta,V.K.(1991). E-optimal designs for two-way elimination of heterogeneity. Sankhya, B53(1), 97-104.

428