A Single Session, Laboratory Primer on Taguchi Methods
Total Page:16
File Type:pdf, Size:1020Kb
 
											Load more
										Recommended publications
									
								- 
												![Harmonizing Fully Optimal Designs with Classic Randomization in Fixed Trial Experiments Arxiv:1810.08389V1 [Stat.ME] 19 Oct 20](https://docslib.b-cdn.net/cover/7263/harmonizing-fully-optimal-designs-with-classic-randomization-in-fixed-trial-experiments-arxiv-1810-08389v1-stat-me-19-oct-20-167263.webp)  Harmonizing Fully Optimal Designs with Classic Randomization in Fixed Trial Experiments Arxiv:1810.08389V1 [Stat.ME] 19 Oct 20Harmonizing Fully Optimal Designs with Classic Randomization in Fixed Trial Experiments Adam Kapelner, Department of Mathematics, Queens College, CUNY, Abba M. Krieger, Department of Statistics, The Wharton School of the University of Pennsylvania, Uri Shalit and David Azriel Faculty of Industrial Engineering and Management, The Technion October 22, 2018 Abstract There is a movement in design of experiments away from the classic randomization put forward by Fisher, Cochran and others to one based on optimization. In fixed-sample trials comparing two groups, measurements of subjects are known in advance and subjects can be divided optimally into two groups based on a criterion of homogeneity or \imbalance" between the two groups. These designs are far from random. This paper seeks to understand the benefits and the costs over classic randomization in the context of different performance criterions such as Efron's worst-case analysis. In the criterion that we motivate, randomization beats optimization. However, the optimal arXiv:1810.08389v1 [stat.ME] 19 Oct 2018 design is shown to lie between these two extremes. Much-needed further work will provide a procedure to find this optimal designs in different scenarios in practice. Until then, it is best to randomize. Keywords: randomization, experimental design, optimization, restricted randomization 1 1 Introduction In this short survey, we wish to investigate performance differences between completely random experimental designs and non-random designs that optimize for observed covariate imbalance. We demonstrate that depending on how we wish to evaluate our estimator, the optimal strategy will change. We motivate a performance criterion that when applied, does not crown either as the better choice, but a design that is a harmony between the two of them.
- 
												  Optimal Design of Experiments Session 4: Some TheoryOptimal design of experiments Session 4: Some theory Peter Goos 1 / 40 Optimal design theory Ï continuous or approximate optimal designs Ï implicitly assume an infinitely large number of observations are available Ï is mathematically convenient Ï exact or discrete designs Ï finite number of observations Ï fewer theoretical results 2 / 40 Continuous versus exact designs continuous Ï ½ ¾ x1 x2 ... xh Ï » Æ w1 w2 ... wh Ï x1,x2,...,xh: design points or support points w ,w ,...,w : weights (w 0, P w 1) Ï 1 2 h i ¸ i i Æ Ï h: number of different points exact Ï ½ ¾ x1 x2 ... xh Ï » Æ n1 n2 ... nh Ï n1,n2,...,nh: (integer) numbers of observations at x1,...,xn P n n Ï i i Æ Ï h: number of different points 3 / 40 Information matrix Ï all criteria to select a design are based on information matrix Ï model matrix 2 3 2 T 3 1 1 1 1 1 1 f (x1) ¡ ¡ Å Å Å T 61 1 1 1 1 17 6f (x2)7 6 Å ¡ ¡ Å Å 7 6 T 7 X 61 1 1 1 1 17 6f (x3)7 6 ¡ Å ¡ Å Å 7 6 7 Æ 6 . 7 Æ 6 . 7 4 . 5 4 . 5 T 100000 f (xn) " " " " "2 "2 I x1 x2 x1x2 x1 x2 4 / 40 Information matrix Ï (total) information matrix 1 1 Xn M XT X f(x )fT (x ) 2 2 i i Æ σ Æ σ i 1 Æ Ï per observation information matrix 1 T f(xi)f (xi) σ2 5 / 40 Information matrix industrial example 2 3 11 0 0 0 6 6 6 0 6 0 0 0 07 6 7 1 6 0 0 6 0 0 07 ¡XT X¢ 6 7 2 6 7 σ Æ 6 0 0 0 4 0 07 6 7 4 6 0 0 0 6 45 6 0 0 0 4 6 6 / 40 Information matrix Ï exact designs h X T M ni f(xi)f (xi) Æ i 1 Æ where h = number of different points ni = number of replications of point i Ï continuous designs h X T M wi f(xi)f (xi) Æ i 1 Æ 7 / 40 D-optimality criterion Ï seeks designs that minimize variance-covariance matrix of ¯ˆ ¯ 2 T 1¯ Ï .
- 
												  Advanced Power Analysis WorkshopAdvanced Power Analysis Workshop www.nicebread.de PD Dr. Felix Schönbrodt & Dr. Stella Bollmann www.researchtransparency.org Ludwig-Maximilians-Universität München Twitter: @nicebread303 •Part I: General concepts of power analysis •Part II: Hands-on: Repeated measures ANOVA and multiple regression •Part III: Power analysis in multilevel models •Part IV: Tailored design analyses by simulations in 2 Part I: General concepts of power analysis •What is “statistical power”? •Why power is important •From power analysis to design analysis: Planning for precision (and other stuff) •How to determine the expected/minimally interesting effect size 3 What is statistical power? A 2x2 classification matrix Reality: Reality: Effect present No effect present Test indicates: True Positive False Positive Effect present Test indicates: False Negative True Negative No effect present 5 https://effectsizefaq.files.wordpress.com/2010/05/type-i-and-type-ii-errors.jpg 6 https://effectsizefaq.files.wordpress.com/2010/05/type-i-and-type-ii-errors.jpg 7 A priori power analysis: We assume that the effect exists in reality Reality: Reality: Effect present No effect present Power = α=5% Test indicates: 1- β p < .05 True Positive False Positive Test indicates: β=20% p > .05 False Negative True Negative 8 Calibrate your power feeling total n Two-sample t test (between design), d = 0.5 128 (64 each group) One-sample t test (within design), d = 0.5 34 Correlation: r = .21 173 Difference between two correlations, 428 r₁ = .15, r₂ = .40 ➙ q = 0.273 ANOVA, 2x2 Design: Interaction effect, f = 0.21 180 (45 each group) All a priori power analyses with α = 5%, β = 20% and two-tailed 9 total n Two-sample t test (between design), d = 0.5 128 (64 each group) One-sample t test (within design), d = 0.5 34 Correlation: r = .21 173 Difference between two correlations, 428 r₁ = .15, r₂ = .40 ➙ q = 0.273 ANOVA, 2x2 Design: Interaction effect, f = 0.21 180 (45 each group) All a priori power analyses with α = 5%, β = 20% and two-tailed 10 The power ofwithin-SS designs Thepower 11 May, K., & Hittner, J.
- 
												  Approximation Algorithms for D-Optimal DesignApproximation Algorithms for D-optimal Design Mohit Singh School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332, [email protected]. Weijun Xie Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA 24061, [email protected]. Experimental design is a classical statistics problem and its aim is to estimate an unknown m-dimensional vector β from linear measurements where a Gaussian noise is introduced in each measurement. For the com- binatorial experimental design problem, the goal is to pick k out of the given n experiments so as to make the most accurate estimate of the unknown parameters, denoted as βb. In this paper, we will study one of the most robust measures of error estimation - D-optimality criterion, which corresponds to minimizing the volume of the confidence ellipsoid for the estimation error β − βb. The problem gives rise to two natural vari- ants depending on whether repetitions of experiments are allowed or not. We first propose an approximation 1 algorithm with a e -approximation for the D-optimal design problem with and without repetitions, giving the first constant factor approximation for the problem. We then analyze another sampling approximation algo- 4m 12 1 rithm and prove that it is (1 − )-approximation if k ≥ + 2 log( ) for any 2 (0; 1). Finally, for D-optimal design with repetitions, we study a different algorithm proposed by literature and show that it can improve this asymptotic approximation ratio. Key words: D-optimal Design; approximation algorithm; determinant; derandomization. History: 1. Introduction Experimental design is a classical problem in statistics [5, 17, 18, 23, 31] and recently has also been applied to machine learning [2, 39].
- 
												  Contributions of the Taguchi Method1 International System Dynamics Conference 1998, Québec Model Building and Validation: Contributions of the Taguchi Method Markus Schwaninger, University of St. Gallen, St. Gallen, Switzerland Andreas Hadjis, Technikum Vorarlberg, Dornbirn, Austria 1. The Context. Model validation is a crucial aspect of any model-based methodology in general and system dynamics (SD) methodology in particular. Most of the literature on SD model building and validation revolves around the notion that a SD simulation model constitutes a theory about how a system actually works (Forrester 1967: 116). SD models are claimed to be causal ones and as such are used to generate information and insights for diagnosis and policy design, theory testing or simply learning. Therefore, there is a strong similarity between how theories are accepted or refuted in science (a major epistemological and philosophical issue) and how system dynamics models are validated. Barlas and Carpenter (1992) give a detailed account of this issue (Barlas/Carpenter 1990: 152) comparing the two major opposing streams of philosophies of science and convincingly showing that the philosophy of system dynamics model validation is in agreement with the relativistic/holistic philosophy of science. For the traditional reductionist/logical empiricist philosophy, a valid model is an objective representation of the real system. The model is compared to the empirical facts and can be either correct or false. In this philosophy validity is seen as a matter of formal accuracy, not practical use. In contrast, the more recent relativist/holistic philosophy would see a valid model as one of many ways to describe a real situation, connected to a particular purpose.
- 
												  A Review on Optimal Experimental DesignA Review On Optimal Experimental Design Noha A. Youssef London School Of Economics 1 Introduction Finding an optimal experimental design is considered one of the most impor- tant topics in the context of the experimental design. Optimal design is the design that achieves some targets of our interest. The Bayesian and the Non- Bayesian approaches have introduced some criteria that coincide with the target of the experiment based on some speci¯c utility or loss functions. The choice between the Bayesian and Non-Bayesian approaches depends on the availability of the prior information and also the computational di±culties that might be encountered when using any of them. This report aims mainly to provide a short summary on optimal experi- mental design focusing more on Bayesian optimal one. Therefore a background about the Bayesian analysis was given in Section 2. Optimality criteria for non-Bayesian design of experiments are reviewed in Section 3 . Section 4 il- lustrates how the Bayesian analysis is employed in the design of experiments. The remaining sections of this report give a brief view of the paper written by (Chalenor & Verdinelli 1995). An illustration for the Bayesian optimality criteria for normal linear model associated with di®erent objectives is given in Section 5. Also, some problems related to Bayesian optimal designs for nor- mal linear model are discussed in Section 5. Section 6 presents some ideas for Bayesian optimal design for one way and two way analysis of variance. Section 7 discusses the problems associated with nonlinear models and presents some ideas for solving these problems.
- 
												  A Comparison of the Taguchi Method and Evolutionary Optimization in Multivariate TestingA Comparison of the Taguchi Method and Evolutionary Optimization in Multivariate Testing Jingbo Jiang Diego Legrand Robert Severn Risto Miikkulainen University of Pennsylvania Criteo Evolv Technologies The University of Texas at Austin Philadelphia, USA Paris, France San Francisco, USA and Cognizant Technology Solutions [email protected] [email protected] [email protected] Austin and San Francisco, USA [email protected],[email protected] Abstract—Multivariate testing has recently emerged as a [17]. While this process captures interactions between these promising technique in web interface design. In contrast to the elements, only a very small number of elements is usually standard A/B testing, multivariate approach aims at evaluating included (e.g. 2-3); the rest of the design space remains a large number of values in a few key variables systematically. The Taguchi method is a practical implementation of this idea, unexplored. The Taguchi method [12], [18] is a practical focusing on orthogonal combinations of values. It is the current implementation of multivariate testing. It avoids the compu- state of the art in applications such as Adobe Target. This paper tational complexity of full multivariate testing by evaluating evaluates an alternative method: population-based search, i.e. only orthogonal combinations of element values. Taguchi is evolutionary optimization. Its performance is compared to that the current state of the art in this area, included in commercial of the Taguchi method in several simulated conditions, including an orthogonal one designed to favor the Taguchi method, and applications such as the Adobe Target [1]. However, it assumes two realistic conditions with dependences between variables.
- 
												  Design of Experiments (DOE) Using the Taguchi ApproachDesign of Experiments (DOE) Using the Taguchi Approach This document contains brief reviews of several topics in the technique. For summaries of the recommended steps in application, read the published article attached. (Available for free download and review.) TOPICS: • Subject Overview • References Taguchi Method Review • Application Procedure • Quality Characteristics Brainstorming • Factors and Levels • Interaction Between Factors Noise Factors and Outer Arrays • Scope and Size of Experiments • Order of Running Experiments Repetitions and Replications • Available Orthogonal Arrays • Triangular Table and Linear Graphs Upgrading Columns • Dummy Treatments • Results of Multiple Criteria S/N Ratios for Static and Dynamic Systems • Why Taguchi Approach and Taguchi vs. Classical DOE • Loss Function • General Notes and Comments Helpful Tips on Applications • Quality Digest Article • Experiment Design Solutions • Common Orthogonal Arrays Other References: 1. DOE Demystified.. : http://manufacturingcenter.com/tooling/archives/1202/1202qmdesign.asp 2. 16 Steps to Product... http://www.qualitydigest.com/june01/html/sixteen.html 3. Read an independent review of Qualitek-4: http://www.qualitydigest.com/jan99/html/body_software.html 4. A Strategy for Simultaneous Evaluation of Multiple Objectives, A journal of the Reliability Analysis Center, 2004, Second quarter, Pages 14 - 18. http://rac.alionscience.com/pdf/2Q2004.pdf 5. Design of Experiments Using the Taguchi Approach : 16 Steps to Product and Process Improvement by Ranjit K. Roy Hardcover - 600 pages Bk&Cd-Rom edition (January 2001) John Wiley & Sons; ISBN: 0471361011 6. Primer on the Taguchi Method - Ranjit Roy (ISBN:087263468X Originally published in 1989 by Van Nostrand Reinhold. Current publisher/source is Society of Manufacturing Engineers). The book is available directly from the publisher, Society of Manufacturing Engineers (SME ) P.O.
- 
												  Minimax Efficient Random Designs with Application to Model-Robust DesignMinimax efficient random designs with application to model-robust design for prediction Tim Waite [email protected] School of Mathematics University of Manchester, UK Joint work with Dave Woods S3RI, University of Southampton, UK Supported by the UK Engineering and Physical Sciences Research Council 8 Aug 2017, Banff, AB, Canada Tim Waite (U. Manchester) Minimax efficient random designs Banff - 8 Aug 2017 1 / 36 Outline Randomized decisions and experimental design Random designs for prediction - correct model Extension of G-optimality Model-robust random designs for prediction Theoretical results - tractable classes Algorithms for optimization Examples: illustration of bias-variance tradeoff Tim Waite (U. Manchester) Minimax efficient random designs Banff - 8 Aug 2017 2 / 36 Randomized decisions A well known fact in statistical decision theory and game theory: Under minimax expected loss, random decisions beat deterministic ones. Experimental design can be viewed as a game played by the Statistician against nature (Wu, 1981; Berger, 1985). Therefore a random design strategy should often be beneficial. Despite this, consideration of minimax efficient random design strategies is relatively unusual. Tim Waite (U. Manchester) Minimax efficient random designs Banff - 8 Aug 2017 3 / 36 Game theory Consider a two-person zero-sum game. Player I takes action θ 2 Θ and Player II takes action ξ 2 Ξ. Player II experiences a loss L(θ; ξ), to be minimized. A random strategy for Player II is a probability measure π on Ξ. Deterministic actions are a special case (point mass distribution). Strategy π1 is preferred to π2 (π1 π2) iff Eπ1 L(θ; ξ) < Eπ2 L(θ; ξ) : Tim Waite (U.
- 
												  Interaction Graphs for a Two-Level Combined Array Experiment Design by DrJournal of Industrial Technology • Volume 18, Number 4 • August 2002 to October 2002 • www.nait.org Volume 18, Number 4 - August 2002 to October 2002 Interaction Graphs For A Two-Level Combined Array Experiment Design By Dr. M.L. Aggarwal, Dr. B.C. Gupta, Dr. S. Roy Chaudhury & Dr. H. F. Walker KEYWORD SEARCH Research Statistical Methods Reviewed Article The Official Electronic Publication of the National Association of Industrial Technology • www.nait.org © 2002 1 Journal of Industrial Technology • Volume 18, Number 4 • August 2002 to October 2002 • www.nait.org Interaction Graphs For A Two-Level Combined Array Experiment Design By Dr. M.L. Aggarwal, Dr. B.C. Gupta, Dr. S. Roy Chaudhury & Dr. H. F. Walker Abstract interactions among those variables. To Dr. Bisham Gupta is a professor of Statistics in In planning a 2k-p fractional pinpoint these most significant variables the Department of Mathematics and Statistics at the University of Southern Maine. Bisham devel- factorial experiment, prior knowledge and their interactions, the IT’s, engi- oped the undergraduate and graduate programs in Statistics and has taught a variety of courses in may enable an experimenter to pinpoint neers, and management team members statistics to Industrial Technologists, Engineers, interactions which should be estimated who serve in the role of experimenters Science, and Business majors. Specific courses in- clude Design of Experiments (DOE), Quality Con- free of the main effects and any other rely on the Design of Experiments trol, Regression Analysis, and Biostatistics. desired interactions. Taguchi (1987) (DOE) as the primary tool of their trade. Dr. Gupta’s research interests are in DOE and sam- gave a graph-aided method known as Within the branch of DOE known pling.
- 
												  Practical Aspects for Designing Statistically Optimal ExperimentsJournal of Statistical Science and Application 2 (2014) 85-92 D D AV I D PUBLISHING Practical Aspects for Designing Statistically Optimal Experiments Mark J. Anderson, Patrick J. Whitcomb Stat-Ease, Inc., Minneapolis, MN USA Due to operational or physical considerations, standard factorial and response surface method (RSM) design of experiments (DOE) often prove to be unsuitable. In such cases a computer-generated statistically-optimal design fills the breech. This article explores vital mathematical properties for evaluating alternative designs with a focus on what is really important for industrial experimenters. To assess “goodness of design” such evaluations must consider the model choice, specific optimality criteria (in particular D and I), precision of estimation based on the fraction of design space (FDS), the number of runs to achieve required precision, lack-of-fit testing, and so forth. With a focus on RSM, all these issues are considered at a practical level, keeping engineers and scientists in mind. This brings to the forefront such considerations as subject-matter knowledge from first principles and experience, factor choice and the feasibility of the experiment design. Key words: design of experiments, optimal design, response surface methods, fraction of design space. Introduction Statistically optimal designs emerged over a half century ago (Kiefer, 1959) to provide these advantages over classical templates for factorials and RSM: Efficiently filling out an irregularly shaped experimental region such as that shown in Figure 1 (Anderson and Whitcomb, 2005), 1300 e Flash r u s s e r 1200 P - B 1100 Short Shot 1000 450 460 A - Temperature Figure 1. Example of irregularly-shaped experimental region (a molding process).
- 
												  Optimal DesignLECTURE 11 Optimal Design The issue of how to optimally design experiments has been around for a lont time, extending back to at least 1918 (Smith). Before we start discussing the highlights of this topic we need to set up the problem. Note that we are going to set it up for the types of situations that we have enountered, although in fact the problem can be set up in more general ways. We will define X (n×p) as our design matrix, β(p×1) our vector of regression parameters, y(n×1) our vector of observations, and (n×1) our error vector giving y = Xβ + . We assume will be iid with mean zero and Cov() = σ 2 I . As before, we have βˆ = (X X)−1 X y Var(β)ˆ = σ 2(X X)−1 ˆ yˆx = xβ 2 −1 Var(yˆx ) = σ x(X X) x . The design problem consists of selecting row vectors x(1×p), i = 1, 2,...,n from the design space X such that the design defined by these n vectors is, in some defined sense, optimal. We are assuming that n is fixed. By and large, solutions to this problem consist of developing some sensible criterion based on the above model and using it to obtain optimal designs. One of the first to state a criterion and obtain optimal designs for regression problems was Smith(1918). The criterion she proposed was: minimize the maxi- mum variance of any predicted value (obtained by using the regression function) over the experimental space. I.e., min max Var(yˆx ).