Algebraic Statistics in Design of Experiments

Algebraic statistics in Design of Experiments

Maria Piera Rogantin DIMA – Universit`adi Genova – [email protected]

Torino, September 2004 3. Indicator function and orthogo- Table of contents nality (a) Orthogonality 1. General aspects (b) Complex coding for full factorial (a) What is the Design of experiments? designs

(b) Design & Ideal of the design (c) Indicator function of a fraction

2. Fractions and confounding (e) Generation of fractions with a given orthogonal structure (a) Fraction of a full factorial design (f) Regular fractions (b) Space of the responses on the fraction 4. Models on a fraction and term- orders (c) Identifiability of a model (a) Identifiable sub-models and initial (d) Fractions and identifiable linear orders models (b) Model curvature (e) Confounding of subspaces (c) Block term-order and factor screening 1 PART I:

General aspects

Pistone G., Wynn H. P.(1996). Generalized confounding with Gr¨obner bases, Biometrika, 83(1): 653–666.

Robbiano L. (1998). Gröbner Bases and Statistics, Gröbner Bases and Applications (Proc. of the Conf. 33 Years of Gröbner Bases), Buchberg- er, B. & Winkler, F. ed., Cambridge University Press, 179–204.

Pistone G., Riccomagno E. and Wynn H. P. (2001). Algebraic Statistics: Computational Commutative Algebra in Statistics, Chapman&Hall.

2 1.1 What is the Design of experiments? ∗

An example: the factorial design

Push-oﬀ force of a spark control valve. (Wu & Hamada. 2000. Experiments. Whiley & Son, p.247)

Inﬂuence of three factors on the response: - weld time (0.3 - 0.5 - 0.7 seconds) - pressure (15 - 20 - 25 psi) - moisture (0.8 - 1.2 - 1.8 percent)

Each factor has three levels. We consider them as ordinal levels: low - medium - high; The levels are coded by integer numbers.

full factorial design Each treatment corresponds to a point in Z3. ∗Full Design 3 In particular, the experiment is realized on fewer treatments ∗ (because of problems with costs, times, practical constraints, setting the factors and measuring the responses . . . )

How to choose the treatments? Which fraction for “the best” study of the responses?

time pressure moisture -1 -1 -1 -1 0 0 -1 1 1 0 -1 0 0 0 1 0 1 -1 1 -1 1 1 0 -1 1 1 0 fractional factorial design or fraction ∗Fraction

4 Design and functions on the design∗ Response Factors Push-oﬀ force (lb) Time Pressure Moisture 111.1 -1 -1 -1 131.0 -1 0 0 65.4 -1 1 1 125.5 0 -1 0 46.9 0 0 1 113.7 0 1 -1 72.5 1 -1 1 141.1 1 0 -1 134.2 1 1 0

Linear inﬂuence of the factors and their interactions:

Force = θ0 + θ1 T + θ2 P + θ3 M + θ12 T · P + θ13 T · M + ... the θ’s signify the “importance” of the terms w.r.t. the response.

∗General design

5 Full factorial designs and fractions: notations

• Ai = {aij : j = 1,...,ni} factors m m aij levels coded by rational numbers Q or complex numbers C

m m m • D = A1 × . . . × Am ⊂ Q (or D ⊂ C ) with N = j=1 nj points full factorial design Q

• A fraction is a subset F ⊂D;

6 Responses on a design f : D 7→ R (functions deﬁned on D) ∗

“Design” indicates either “Full factorial design” or “Fractional factorial design” or ...

• Xi : D ∋ (d1,...,dm) 7→ di projection, frequently called factor

α α1 αm • X = X1 · · · Xm , αi

The term Xα has order k if in α there are k non-null values: Xα is an interaction of order k (binary case: order = degree)

Deﬁnitions:

1 • Mean value of f on D, ED(f): ED(f) = #D d∈D f(d) P • A contrast is a response f such that ED(f) = 0.

• Two responses f and g are orthogonal on D if ED(f g) = 0. ∗General design

7 The polynomial complete regression model∗

• L = {(α1,...,αm) : αi

• complete regression model:

For all di ∈ D and for the observed value yi

α yi = θα X (di) αX∈L whit θα ∈ R or θα ∈ C.

In vector notation, Y =(y1,...,yn) response measured on the points of D:

α Y = θα X αX∈L α • Z = [X (d)]d∈D,α∈L matrix of the complete regression model

• θ =(θα)α∈L vector of the coeﬃcients

Matrix notation of the complete regression model Y = Zθ

∗General design

8 On the full factorial design: the complete regression model is identiﬁable, i.e. there is a unique solution w.r.t. θ: θˆ = Z−1 Y (the matrix Z is a square full rank matrix)

On a fraction: there is not a unique solution w.r.t. θ α (Z = [X (d)]d∈F,α∈L has less rows than columns) θˆ = (Z′Z)−Z′ Y

9 In general, for linear regression models of the form:

Y = W θ + R where W is a matrix of “explicative variables” and R is a vector of residuals

Even if θˆ = (W ′W )−W ′ Y is not unique, the approximation of the response Y through the “explicative variables” is always unique:

Yˆ = W (W ′W )−W ′ Y

R Yˆ is the orthogonal projection of Y in the linear space Y=^ W ^θ = W (W'W)- W' Y = P Y generate by the columns of W . Wθ

Moreover, if Y is a multivariate random variable with normal distribution Y ∼ N (W θ, σ2I) then θˆ = (W ′W )−W ′ Y is a solution of maximum likelihood.

10 Example: 2 × 3 full factorial design∗ -1 -1 -1 0 A = {−1, 1}, n =2 -1 1 1 1 D = A2 = {−1, 0, 1}, n2 =3 1 -1 1 0 1 1

2 2 monomial responses: 1, X1, X2,X2 , X1X2,X1X2 L = {(0, 0), (1, 0), (0, 1), (0, 2), (1, 1), (1, 2)} complete regression model: 2 2 Y = θ00 + θ10X1 + θ01X2 + θ02X2 + θ11X1X2 + θ12X1X2

2 2 1 X1 X2 X2 X1X2 X1X2 1 −1 −11 1 −1 1 −100 0 0 matrix of the complete regression model Z = 1 −1 1 1 −1 −1 1 1 −1 1 −1 1 1100 0 0 1111 1 1

∗Full Design

12 1.2 Design and design ideal∗

The application of computational commutative algebra to the study of estimability, confounding on the fractions of factorial designs has been proposed by Pistone & Wynn (Biometrika 1996).

1st idea Each set of points D ⊆ Qm is the set of the solutions of a system of polynomial equations; we assume that each solution is simple.

2nd idea Each real function defined on D is a polynomial function with coefficients into the field of real number R. It is a restriction to D of a real polynomial.

∗General Design

13 A design D is a ﬁnite set of distinct points in a m-dimensional ﬁeld km ∗

The deﬁning ideal of the design (or design ideal) I(D) is the set of all polynomials on k[x1, . . . , xm] that vanish on D.

• The design D is a variety

• The design ideal I(D) is radical

• I(D) is the intersection of the design ideals of each point of D

∗General design

14 Operations with designs & Operations with ideals∗

• Product of designs.

m m D1 ⊂ k 1 D2 ⊂ k 2 I (D1 ×D2) =< I1, I2 > m +m D1 ×D2 ⊂ k 1 2 τ term ordering on k[x1, . . . , xm1+m2] τ1, τ2 t.o. restricted to

k[x1, . . . , xm1] and k[x1, . . . , xm2]

G1,τ1, G2,τ2, G-bases of I (D1), I (D2)

G = g1, g2 | g1 ∈ G1,τ1 g2 ∈ G2,τ2 n o is G-basis of I (D1 ×D2)

∗General design

15 • Restriction of a design∗

m D ⊂ k I + J ideal in k[x1, . . . , xm]

I = I(D) I + J = {f + g | f ∈ I g ∈ J} J ideal in k[x1, . . . , xm] Variety(I + J) = D∩ Variety(J)

∗General design

16 • Union of designs∗

m D1, D2 ⊂ k τ term ordering on k[x1, . . . , xm], m D1 ∪D2 ⊂ k G1, G2 G-bases of I (D1), I (D2)

A G-basis of I (D1 ∪D2) is

G = {g1g2 | g1 ∈ G1 g2 ∈ G22}

∗General design

17 1.3 The full factorial design.∗ Polynomial representation

The full factorial design D corresponds to the set of solutions of the system

1 n1 = n1− k (X1 − a11) ··· (X1 − a1n1 ) = 0 X1 k=0 ψ1k X1 . (X2 − a21) ··· (X2 − a2n2 ) = 0 .  . or  . P  .  .   n −1  (Xm − am1) ··· (Xm − amn ) = 0  nm m k m Xm = k=0 ψmk Xm     P In the previous examples:

2 (X1 − 1)(X1 +1) = 0 X1 = 1 rewriting or 3 X2 (X2 − 1)(X2 +1) = 0 X = X2 rules 2

3 X1 (X1 − 1)(X1 +1) = 0 X1 = X1 rewriting 3 X2 (X2 − 1)(X2 +1) = 0 or X2 = X2 rules  ( 1)( +1) = 0  3 =  X3 X3 − X3  X3 X3 ∗Full Design  18 Space of the real functions on the full factorial design R(D)∗

For a full factorial design each function is represented in a unique way by an identiﬁed complete regression model (i.e. as a linear combination of constant, simple terms and interactions):

R(D) = θ Xα , θ ∈ R  α α  αX∈L  In general, for full factorial designs or fraction: 

• R(D) is a Hilbert vector space (classical results derive from this structure) 1 The scalar product is: = ED(f g) = N d∈D f(d)g(d) P • R(D) is a ring (algebraic statistical approach) The products are reduced with the rules derived by the polynomial representation of the full factorial design:

ni−1 ni k Xi = ψik Xi , ψik ∈ R for i = 1, . . . , m kX=0 ∗Full design

19 Orthogonal decomposition of the space of responses on the full factorial design∗

R(D) can be decomposed in orthogonal subspaces corresponding to the constants, to the simple factors, to the interactions of order 2, 3, . . . , m: m m R(D) = H0 ⊕ Hi ⊕ Hij ⊕···⊕ H12···m . iX=1 i,jX=1

H = span Y , j = 1, . . . , m I  ij  iY∈I  where   – I subset of {1, . . . , m}. 2 ni−1 – 1,Yi1,Yi2,...,Yni−1 orthogonalization of 1,Xi,Xi ,...,Xi

∗Full Design

20 PART II:

Fractions and confounding

Pistone G., Wynn H. P. (1996). Generalized confounding with Gr¨obner bases, Biometrika, 83(1): 653–666.

Pistone G., Riccomagno E. and Wynn H. P. (2001). Algebraic Statistics: Computational Commutative Algebra in Statistics, Chapman&Hall.

Galetto F., Pistone G., Rogantin M. P. (2003). Confounding revisited with commutative computational algebra. Journal of Statistical Planning and Inference. 117, p. 345-363. 21 2.1 Fractions of a full factorial design F ⊂ D ∗

A fraction is a subset of a full factorial design, F ⊂D.

All the fractions are obtained by adding equations (generating equations) to restrict the set of solutions.

In the ﬁrst example:

X1 (X1 − 1)(X1 +1) =0 X2 (X2 − 1)(X2 +1) =0  X3 (X3 − 1)(X3 +1) =0    1 1 1 1 X1X2X3 − X1X2 + X1X3 + X2X3 + 3X1 + 3X2 − 3X3 + 3 =0   ∗Fraction

22 The use of fractions induces a confounding.∗

• Most of the terms of the complete regression model are not identiﬁ- able.

• Some simple or interaction terms of the complete regression model computed on the fraction points equals a linear combination of other terms.

In the ﬁrst example: 2 2 2 X1 = −2X1X2 − X2 +2X1X3 +2X2X3 − X3 + X1 + X2 − X3 +2 on each point of the displayed fraction.

∗Fraction

23 2.2 Space of the responses on the fraction R(F)∗

(Pistone & Wynn, Biometrika 1996)

Algebraic methods allows to ﬁnd bases of the vector space of responses on the fraction, namely of the quotient space k[x1, . . . , xm]/I(F)

- τ a term-ordering, - Gτ,F the Gr¨obner basis of I(F) - LT Gτ,F the set of the leading terms of Gτ,F β β α α Estτ (F) = x : x is not divisible for any x , x ∈ LT Gτ,F n o

R(F) = θ Xβ , θ ∈ R dim R(F) = #Est (F) = #F  β β  τ βX∈M  Let M be the set of exponents of the elements of Est ∗Fraction

24 ∗ Estτ (F) is a hierarchical set of terms:

β α α α if x ∈ Estτ (F) and x divides x , then x ∈ Estτ (F)

(or order ideal or standard set of power products)

Each response f on F can be written as an unique linear combination of elements of Estτ (F): β NFτ,F(f) = cβ(f) X βX∈M remainder of f w.r.t. division by Gτ,F , representative of the equivalence class of f in k[x1, . . . , xm]/I(F)

∗Fraction

25 2.3 Identiﬁability of a model∗

Y = (y1,...,yn) vector of responses

β Bτ = X (d) matrix of the model d∈F,β∈M elementsh i of Estτ (F) valued on the fraction points

The linear system of equation β Y = θβ X or Y = Bτ θ βX∈M has one and only one solution w.r.t. θ: ˆ −1 θ = Bτ Y we say that the polynomial model is identiﬁable by the fraction

Sub-models of an identiﬁable model are identiﬁable ∗Fraction

26 Decomposition of the response space of the full design R(D)∗

(Galetto, Pistone, Rogantin, JSPI, 2003)

The vector space R(D) of the responses on the full factorial D can be decomposed into two orthogonal sub-spaces:

• the space R(F) of the identiﬁable responses on F

• the space of the null responses on F

∗Full Design

27 2.3 Fractions and linear polynomial models∗

1. Direct problem: given a F, what are the linear polynomial models which can be identiﬁed by F? or What are the possible hierarchical sets of terms E which are bases of the vector space k[X]/I(F)?

2. Inverse problem: given a hierarchical linear polynomial model Y , what are the minimal fractions F ⊂D which identify Y ? E hierarchical set of terms associated to Y , E the set of its exponents:

β Y = θβ X βX∈E or What are the fractions F ⊂D such that E is a basis of the vector space k[X]/I(F)?

∗Fraction

28 1. Direct problem:∗

• Using G-bases:

given I(F), for every term-order τ there is a unique Estτ (F).

τ are inﬁnite but the Estτ (F) are ﬁnite.

If a model is identiﬁable w.r.t. a term-ordering then it is identiﬁable w.r.t. any term-ordering.

• Using indicator function (see below): If the cardinality of a list of orthogonal terms equals #F then it is a basis of the vector space k[X]/I(F)

∗Fraction

29 2. Inverse problem∗

• Given E there exists a fraction s.t. E is a basis of the vector space k[X]/I(F) Let Distr(E) be the set of such fractions. It is easily computable, but in general such fractions have not good statistical properties.

• There are hierarchical sets of terms E not derived from G-basis method: ∄ τ s.t. E = Estτ (F)

• Let Sol(E) be set of all the fractions s.t. E is a basis of the vector space k[X]/I(F) Distr(E) ⊆ Sol(E, ∀τ) ⊆ Sol(E, ∃τ) ⊆ Sol(E) where: - Sol(E, ∀τ) is the set of fractions s.t. Estτ (F) = E - Sol(E, ∃τ) is the set of fractions s.t. ∃ τ with Estτ (F) = E

In general the inclusions are strict. (Robbiano,1998)

∗Fraction

30 2.4 Confounding of interaction subspaces on F ∗ (Galetto, Pistone, Rogantin, JSPI 2000)

This deﬁnition of confounding does not relate to the non-identiﬁability of the single monomial responses but it regards the confounding of the full interaction sub-space

An interaction space has clear meaning only when it is fully identiﬁable on the fraction.

Problems arise with multilevel factors; in the binary case the dimension of each interaction sub-space is 1. Example: three-level factors

H1: subspace of the ﬁrst factor 2 on D: dim(H1) = 2; it is generated by an orthogonalization of X1,X1

H12: subspace of the interaction between the ﬁrst two factors 2 2 2 2 on D: dim(H12) = 4 ; it is generated by an orthogonalization of X1X2,X1 X2,X1X2 ,X1 X2 ∗Fraction

31 Identiﬁability of subspaces of a fraction∗

• HJ is identiﬁable on F if n dim H on F equals dim H on D = J J #J are simultaneously identiﬁable on if • HJ1,...,HJk F k k dim on equals dim on HJi F HJi D iX=1 iX=1

∗Fraction

32 Confounding of an interaction subspace HI with a set of subspaces simultaneously identiﬁable∗ HJ1,...,HJk

is confounded on with the spaces • HI F HJ1,...,HJk ( + + = ( )) if: HJ1 . . . HJk 6 R F a) + + HI ⊆ HJ1 . . . HJk b) the set is minimal for the property a) {HJ1,...,HJk}

• HI is totally confounded with the simultaneously identiﬁable spaces if HJ1,...,HJk it is confounded and if:

c) HI is identiﬁable,

d) = + + . HI HJ1 . . . HJk

∗Fraction

33 An algebraic method for studying the confounding of a interaction sub-space HI is based on the system of the Normal Forms of the basis ∗ of HI on the design D. advantage: it is easily implementable within symbolic computation softwares disadvantage: it does not display all possible confounding relations but it applies only to the controlled subspaces, i.e. spaces having a basis contained in Estτ (F) but: different choices of the term-ordering τ allow to find different confounding relations

∗Fraction

34 PART III:∗

Indicator function and orthogonality

Fontana R., Pistone G. and Rogantin M. P. (2000). Classiﬁcation of two-level factorial fractions, J. Statist. Plann. Inference 87(1), 149–172.

Ye K. Q. (2003). Indicator function and its Application in two-level factorial designs, The Annals of Statistics. 31(3).

Pistone G., Rogantin M. P. (2004). Complex coding for multilevel factorial designs. Submitted.

∗Fraction

35 3.1 Orthogonality∗ of factors : “all level combinations appear equally often” of responses in a vector space, based on a scalar or Hermitian product:

= ED(f g) = 0 Two orthogonal responses are not confounded and the estimators of their coeﬃcients in a model are not correlated.

Vector orthogonality is aﬀected by the coding of the levels, while factor orthogonality is not.

If the levels are coded with the complex roots of the unity the two notion of orthogonality are essentially equivalent

∗Fraction

36 3.2 Complex coding for full factorial designs∗

Pistone G. and Rogantin M. P. (2004)

ω We code the n levels of a factor A with the 1 complex solutions of the equation ζn = 1: ω 2π 0 ωk = exp i k for k = 0,...,n − 1 n ω 2

[ ] the residue of mod ; especially, for integer , ( )h = k n k n h ωk ω[hk]n

2π The mapping Zn ∋ k ↔ exp i n k is a group isomorphism on the multi- plicative group of C.

Recoding is a polynomial of degree n in both directions.

∗Fraction

37 The full factorial design D, as a subset of Cm, is deﬁned by the system of equations∗ nj ζj − 1 = 0 for j = 1, . . . , m

The set of all responses on the full design C(D) is a complex Hilbert space with Hermitian product

= ED(f g)

XαXβ = X[α−β], where [·] denotes the modulo operation extended to L.

The set of all the monomial response on the full factorial design: α {X ,α ∈ L} L = {(α1,...,αm) : αi

0 α 1. ED(X ) = 1, and ED(X ) = 0 for α 6= 0;

α β [α−β] 1 if α = β 2. ED(X X ) = ED(X ) = 0 if α 6= β ∗Full design

38 Example∗

Integer coding

−1 −1 −1 −1 0 0 The fraction is deﬁned by: −1 1 1  0 −1 0  X1 (X1 − 1)(X1 +1) =0 0 0 1 1 +1 =0  0 1 1  X2 (X2 − )(X2 )  −    1 −1 1  X3 (X3 − 1)(X3 +1) =0  1 0 −1    1 1 0     X X X − X X + X X + X X + 1X + 1X − 1X + 1 =0   1 2 3 1 2 1 3 2 3 3 1 3 2 3 3 3   Complex coding The fraction is deﬁned by: ω0 ω0 ω0 ω0 ω1 ω1 3 ζ1 − 1=0 ω0 ω2 ω2 3  ω1 ω0 ω1  ζ2 − 1=0 ω ω ω  3 1 1 2 ζ3 − 1=0  ω1 ω2 ω0    ω ω ω    2 0 2   2 2  ω2 ω1 ω0  ζ1 ζ2 ζ3 − 1=0  ω ω ω   2 2 1      ∗Fraction

39 3.3 Indicator function F of a fraction∗ It is a response deﬁned on the full factorial design D such that 1 if ζ ∈F F (ζ) = 0 if ζ ∈ D r F It is represented as the polynomial: α F = bα X αX∈L whose terms are orthonormal on the full factorial design.

The coeﬃcients bα satisfy the following properties:

1 α #F • bα = N ζ∈F X (ζ) and especially b0 = N P

• bα = b[−α] because F is real valued.

Important statistical features of the fraction can be read out from the form of the polynomial representation of the indicator function. ∗Fraction

40 Fractions with replicates and Counting function∗

A fraction with replicates Frep that we denote by Frep, can be considered a multi-subset of a full factorial design D, that is a list of r points.

The counting function R is a response on the full factorial design showing the number of replicates of a point ζ.

The coeﬃcients of the representation of R as: α R(ζ) = cα X (ζ) . αX∈L are: 1 r cα = Xα(ζ) and especially c = N ∅ N ζ∈FXrep

∗Fraction

41 3.4 Results about orthogonality∗

1. A simple term or an interaction term Xα is a contrast on F if and only if cα = c[−α] = 0.

2. Two simple or interaction terms Xα and Xβ are orthogonal on F if and only if c[α−β] = c[β−α] = 0;

3. If Xα is a contrast then, for any β and γ such that α = [β − γ] or α =[γ − β], Xβ is orthogonal to Xγ.

∗Fraction

42 Some other results about orthogonality∗

(following from the structure of the roots of the unity as cyclical group)

α Let X be a term with level set Ωs on the full factorial design D.

1. If s is prime, then the term Xα is a contrast if and only if its s levels appear equally often

2. If the vector of replicates is a combination with positive weights of α indicators of subgroups or laterals of subgroups of Ωs, then X is a contrast.

∗Fraction

43 3.5 Generation of fractions with a given orthogonal structure∗

The coeﬃcients of the indicator function of F are related according to:

bα = bβ b[α−β] α ∈ L βX∈L

Let O be the set of exponents corresponding to the given orthogonal structure (e.g. all the simple terms mutually orthogonal: O corresponds to all interactions of order two)

The solutions of the system of:

bα = β∈L bβ b[α−β] ∀α ∈ L b = 0 ∀α ∈ O  α P are the coeﬃcients of the indicator functions of all the fractions with such a structure and then the points of such a fraction can be derived. ∗Fraction

44 We computed all the fraction with mutually orthogonal simple terms of:∗

• 24 and 25 designs

• 34 and 2 × 33 designs

Problems:

• classes of equivalence for permutations of factors and levels

• for multilevel case: symbolic softwares with operations on the complex roots of the unity are not available.

∗Fraction

45 3.6 Regular fractions ∗

A fraction F is regular if:

• all the factors have the same number of levels n

• their deﬁning equations are of the form α X = e(α) e(α) ∈ Ωn, α ∈H where the set of exponents H can be completed to be a subgroup L of L.

In a regular fraction the interactions Xα e Xβ are

either orthogonal or (totally) confounded

∀ α,β ∈ L . ∗Fraction

46 Indicator function and regular fractions (Pistone, Rogantin, 2004)∗

The fraction F is regular with deﬁning equations Xα = e(α), α ∈ L.

if and only if the indicator function of F is 1 F (ζ) = e(α) Xα(ζ) ζ ∈ F l αX∈L e(α) (all the coeﬃcients are of the type l )

Example

The indicator function of the fraction considered before is: 1 F = 1 + X2X2X + X X X2 3 1 2 3 1 2 3 We check that it is a regular fraction. ∗Fraction

47 PART IV:∗

Models on a fraction and term-orders

Holliday T., Pistone G., Riccomagno E. and Wynn H. (1999). The application of computational algebraic geometry to the analysis of designed experiments: a case study, Comput. Statist., 14.2, p.213–231

Bates R., Giglio B., Riccomagno E. and Wynn H. (1998). Gr¨obner basis methods in polynomial modelling. Improceeding of COMPSTAT 98, ed. R. Payne p. 179–184

∗Fraction

48 4.1 Identiﬁable sub-models and initial orders∗

A strategy to ﬁnd a parsimonious model (few terms well interpolating the response):

1. Fix the term order type (e.g. DegRevLex)

2. Choose an initial order for the factors and compute Estτ

3. Repeat (2.) for all initial orders τ

4. Consider the sub-model given by E = τ Estτ (note that {1}⊆E) T

5. Use statistical methods to choose parsimonious models from E

An example of 4-factor design with 23 points in Holliday, Pistone, Riccomagno, Wynn (1999) ∗Fraction 49 4.2 Model curvature∗

Basis of the vector space of the responses on F derived from Gr¨obner basis method:

Est (F) = c Xβ τ  β  βX∈M    β The corresponding saturated identiﬁable model is Y = β∈M θˆβ X P ˆ −1 and the coeﬃcients are θ = Bτ Y

β Deﬁne the ﬁtted polynomial model function f(x) = β∈M θˆβ x P Hessian matrix of f(x): ∂2f(x) Hf (x) = (∂xi ∂xj )

∗Fraction

50 A measure of smoothness:

2 2 ′ φ = trace Hf (d) = θˆ Q θˆ dX∈F where Q is a non-negative matrix depending only on B and F.

Choice of a good model: vary the term-order τ and choose the corresponding model with smallest curvature

Notice that the curvature depend on the response values Y (through θˆ).

51 An example: A 3-factor design with 16 points∗

Y P B A Two interpolators’ bases: 271.4 500 396 8 EstLex = 2 3 4 2 2 2 2 2 3 3 2 4 268.9 500 403 13 {1,A,A ,A ,A ,B,BA,BA ,B ,B A,B A ,B ,B A ,B , P } 282.8 500 404 3 266.2 800 402 8 EstDegRevLex = 297.5 200 402 8 2 2 2 3 2 2 2 3 295.1 500 645 8 {1,A,B,P,A ,BA,PA,B ,PB,P ,A ,BA ,PA ,B A,PBA,B } 262.2 500 151 8 269.4 500 405 8 274.8 350 248 5 Curvatures: 291.1 350 550 5 φLex = 17374.9 266.9 650 248 5 285.4 650 555 5 φDegRevLex = 257.6 261.0 350 252 5 276.5 350 551 11 263.1 650 254 11 Reduced model via stepwise regression: 280.4 650 550 11 {1,PA2,A3, AP }

∗Fraction

52 4.3 Block term-order and factor screening

Statistical and practical methods can give prior information on the rele- vance of the factors: e.g. A’s factors have less inﬂuence on the response than B’s factors

Let M(τA) and M(τB) be the matrices for the term-order corresponding to the A’s and B’s factors.

The block matrix:

M(τA) 0 " 0 M(τB) # represents the matrix for a term-order corresponding to the A and B’s factors, whit A’s factors less relevant than B’s