<<

DESIGNING – An Overview1 Gregory F. Gruska Maureen S. Heaphy

ABSTRACT DEPENDENT VARIABLE or RESPONSE - What we end Statistical experiments are commonly used to identify up with: those variables, which are evaluated for changes factors affecting and productivity. The analysis of caused by the . The output of interest is called collected during these experiments is most frequent- the dependent or response variable. For example the ly recognized as the analysis of (ANOVA). response could be the diameter, surface finish, RPM, Unfortunately there is little attention shown, in the litera- torque and so on. ture or in practice by the engineer, to the designing of FACTOR LEVEL - The factor level is the number of val- experiments. That is, one needs to plan an experiment ues of the factor (independent variable) be set within prior to collecting any data. This paper addresses certain the experiment. For quantitative factors, each tested value characteristics of an experiment that are prerequisites to becomes a level; e.g., if the experiment is to be conduct- conducting a meaningful experiment. Different types of ed at four different speeds, then the factor speed has four are discussed as well as the advantages and dis- levels. Switch on or off denotes two levels for the switch advantages of each. factor. An experiment can be defined, in a general sense, as CELL – That portion of the test environment which is “a trial made to confirm or disprove something, or to likely to be more homogeneous; i.e. those samples which demonstrate some known truth . . .” In other words, it is have the same settings for each of the factors. the conducting of tests to answer specific questions. – Samples, which have the same set- However, in line with statistical thinking, the first step tings for each of the factors within the test sequence. should be the designing or planning of the experiment REPEATS – The number of times the entire test within a PDSA cycle2. sequence is implemented. A designed experiment has advantages over the classic BACKGROUND FACTORS – In addition to the factors change one variable at a time (AB-BA) approach. First of that are varied in a controlled fashion, the experimenter all, the designed experiment can detect if there are any may be aware of certain background variables that might interactions among the variables, but the classic approach affect the outcome of the tests. These background vari- cannot. Next, the approach uses all of the data ables must be considered in the planning of the experi- simultaneously in analyses of the data whereas the classic ment, so that: uses only a subset of the data. The reduced sample size • the possible effects due to background variables do present in the subset reduces the power of the analysis in not affect information obtained about the factors of the classic method. Finally, the obtained from a primary interest; or designed experiment are valid over a wide of con- • some information about the effects of the back- ditions (environment rich) as compared to the results ground variables can be obtained. from the classic approach which are valid only for the – There may be variables of which actual test conditions (environment free). the experimenter is unaware that have an effect on the outcome of the experiment. To increase the likelihood TERMINOLOGY that the effect of these variables is balanced out, an Certain terms are defined so that their usage in later experiment should always be randomized. This involves discussions will be understood. assigning the occurrence of the controlled conditions in a INDEPENDENT FACTOR - What we start out with: the purely chance fashion. To eliminate from the experi- conditions that are deliberately varied in a con- ment, variables which are not specifically controlled as trolled manner are called the independent factors or inde- factors or "blocked out" by block designs (see below) pendent variables. These factors may be quantitative fac- should be randomized. Randomization also assures valid tors such as time or temperature, which can be varied estimates of experimental error and makes possible the along a continuous scale, or they may be qualitative fac- application of statistical tests of significance and the con- tors such as different machines or a switch turned on/off. struction of confidence intervals. Continued on page 9

8 ASQ DIVISION NEWSLETTER, VOL. 19, NO. 2 DESIGNING EXPERIMENTS Continued from page 8

PREREQUISITES TO CONDUCTING • hold constant - although the noise variable might EXPERIMENT have an influence on the dependent variable, it There are certain characteristics of an experiment that should have the same effect throughout the experi- are prerequisites to conducting a meaningful experiment. ment if we hold it at a fixed level. Consequently the Some of these are: effect of the independent variables on the depen- • The experiment should have well defined dent variable can be quantified. Note assumptions: objectives. These should include identifying the 1) we can hold the noise variable constant and 2) factors and their ranges; choosing experimental the noise and signal variables are independent. procedure and equipment; and stating the applica- • assign randomly - again the noise variable might bility of the results. influence the dependent variable but by assigning it • As much as possible, effects of the independent randomly and by replicating the experiment, the factors should not be obscured by other variables. effects of the independent variable can be mea- This is accomplished by designing the experiment sured. Note assumptions: 1) we can randomize with such that the effects of uncontrolled variables are respect to the noise variables, and 2) the noise and minimized. signal variables are independent. • As much as possible the experiment should be free • include as independent variables - if the noise from bias. This involves the use of randomization variables are identifiable and can be controlled then and replications. they could be considered to be independent • The experiment should provide a measure of preci- variables. sion (experimental error), unless it is known from Statistical control of a noise variable involves previous experimentation. Replications provide the measuring the level of that noise variable for each experi- measure of precision while randomization assures mental test and then using a statistical technique called the of the measure of precision. Analysis of . This will enable us to "back out" • The expected precision of the experiment should be the effects of the noise variable and quantify the effect of sufficient to meet the defined objectives. There gen- the independent variables on the response variable. erally is a trade-off between the expense of addi- tional experimentation and the precision of the TYPES OF DESIGNS results. These trade-offs should be examined prior Once the prerequisites to conducting an experiment to the collection of data. Also, greater precision may are met, the type of experimental design can be chosen. be obtained by use of blocked designs when appro- These designs have certain relationships to the purposes, priate. needs, and physical limitations of experiments. They also have certain advantages in economy of experimentation SIGNAL AND NOISE and yield straightforward and unbiased estimates of Consider the variables of interest (factors) as generat- experimental effects and valid estimates of precision. ing a signal reflected in the response variable. Consider There are a number of ways by which experiment the effect of all other (background) variables in the test designs might be classified: environment to be noise. To increase the and • By the number of experimental factors to be investi- effectiveness of measuring the effect of the independent gated (e.g., single-factor vs. multifactor designs) variables, either the signal can be increased or the noise • By the structure of the experiment design (e.g., can be controlled. blocked designs vs. randomized designs); or • By the kind of information the experiment is pri- CONTROL OF NOISE marily intended to provide (e.g., estimates of effects The noise or nuisance variables are those factors that or estimates of variability). might interfere with the of the effect of the The types of designs to be discussed are Randomized, independent variable. There are basically four ways to Block, , Factorial, and Nested. control the effects of noise variables. Three methods are experimental control and the fourth is statistical control. The three methods of experimental control are:

Continued on page 10

ASQ STATISTICS DIVISION NEWSLETTER, VOL. 19, NO. 2 9 DESIGNING EXPERIMENTS Continued from page 9

10 ASQ STATISTICS DIVISION NEWSLETTER, VOL. 19, NO. 2 DESIGNING EXPERIMENTS Continued from page 10

• The effects of individual differences are minimized. DISADVANTAGES - There are three main disadvan- CAR I II III IV tages of a randomized block design: • It may be difficult to minimize the ‘within’ block FR AADA variability for a large number of factor levels. Position FL AACD • For a Fixed Effect Model (i.e., models where factors RR DBBB levels tested represent all levels of interest), bias is RL DCBC present in the test for factor level effects. • Each level of the factor must be run in each block. Brands Randomly Assigned to a Car This drawback can be overcome by using an incom- plete randomized block design. With this , tire brand A happens to LATIN SQUARE DESIGN be assigned to the front positions only and is not A Latin square design is appropriate when there are assigned to vehicle 111. Again from the test results, the two variables that the experimenter wishes to block out. effect of tire brand will not be isolated since there is dri- These two variables are sources of noise and could affect ver/vehicle influence and tire position. Even if each tire the test results. These two noise variables also thought of brand is restricted to being assigned only once on each as non-homogeneous conditions, are secondary factors in car, test results still could be influenced by tire position. the test whereas a third variable is the primary factor or The solution for this problem is the Latin square. The signal of interest. The use of the Latin square is to associ- primary interest is the four tire brands but the two noise ate the primary factor with the two noise variables in a variables, driver/vehicle and tire position, need to be prescribed fashion. blocked out. An acceptable assignment is shown below. APPLICABILITY - There are a number of situations where two noise variables need to be blocked out. The noise variables might be machines, positions, operators, days. A classic example is one where testing is to be CAR I II III IV done to measure tread loss on four brands of tires, using FR CDAB four similar vehicles. Let the four tire brands be identified Position FL BCDA as A,B,C,D and the four vehicles as 1, 11, 111, IV. There RR ABCD are four tires of each brand to be tested. In setting up the RL DABC test one possible assignment of tire brand and vehicle could be to have one brand assigned to one car as shown Each Brand Assigned to Each Vehicle and to Each below. Position

CAR I II III IV ADVANTAGES - The primary advantage of a Latin FR ABCD square design is the ability to block out the effects of two Position FL ABCD noise variables thereby isolating the influence of the RR ABCD primary factor. With this type of design it is possible to RL ABCD estimate and compare the effects of the two blocked variables. Also, the analysis is considered to be relatively One Brand Assigned to a Car simple. DISADVANTAGES - One disadvantage is that the design assumes no interactions exist among the three With this assignment, the test results cannot be variables. A second item, that initially seems to be a attributed to tire brand differences only since there might restriction, is there must be the same number of levels for be driver/vehicle influence. To eliminate the driver/ all three variables. If this were not the case, then an vehicle influence, one might decide to randomly assign incomplete Latin square, called a Youden square, would the sixteen tires (4 tires of each of 4 brands) to the be appropriate. four vehicles. The results-of such and assignment could be as shown as follows. Continued on page 12

ASQ STATISTICS DIVISION NEWSLETTER, VOL. 19, NO. 2 11 DESIGNING EXPERIMENTS Continued from page 11