Design of Experiments Design of Experiments

Design of Experiments Course Information Package Design of Experiments Course Information Package

Table of Contents

DOE Course Overview...... 1 Course Registration...... 2 DOE Course Descriptions...... 3 Operations Analyst Forum...... 14 Professional References...... 16 Points of Contact...... 20

NOTE: Laptop computers are REQUIRED for most courses. You must be able to load the Design Expert software so a government laptop with temporary admin privileges or personal laptop is recommended. DOE COURSE OVERVIEW

Course (Hours) Course Title Target Audience TW 105 (1-2) Executive Overview of the Science Center, Flag, and SES of Test (DOE for Executives) audiences

TW 110 (4) Leaders’ Introduction to the Scence Wing, Group, and Squadron of Test (DOE for Leaders) CC, DO, and CA

TW 211 (16) DOE for Software-Intensive SPO, Test and Software Systems Engineers

DOE for Aircrew (4) DOE for Aircrew Aircrew and Unit Project Officers

TW 210 (16) Introduction to DOE Test and Project Engineers, Project Managers, Operations Suitability Analyts TW 310 (40) Statistical Foundations for DOE SPO Engineers, Oversight (DOE Foundations) Action Officers, DOE Practitioners, and Test Design Engineers

TW 311 (40) DOE for Characterization (DOE I) DOE Practitioners and Test Project and Design Engineers

TW 410 (40) DOE for Fractional Factorials and DOE Practitioners and Test Response Surfaces (DOE II) Design Engineers

TW 411 (40) Advanced Topics in Design of Advanced DOE Practitioners Experiments (DOE III)

1 Course Registration

Air Armament Academy (A3) via AFKN CoP DOE training opportunities are scheduled through the A3 – College of Test and Evaluation: https://afkm.wpafb.af.mil/community/views/home.aspx?Filter=OO-ED-AA-A2 Click on “TE - College of Test & Evaluation” link in the middle of the page

Air Armament Academy via AF Portal Log into A3 from the Air Force Portal: https://www.my.af.mil Once logged in, select WORKSPACE found at the top left On the WEB FAVORITES tab, select the EDIT button On the EDIT WEB FAVORITES pop-up, select CREATE NEW FAVORITE found at the top right Link name can be whatever you choose (A3 Outside Mil Domain, Cool DOE Links, etc.) Under URL input the following link: https://wwwd.my.af.mil/afknprod/ASPs/CoP/OpenCoP.asp?Filter=OO-ED-AA-A2 Select SUBMIT Close the pop-up and your workspace page should reload with your recently named link You should now be able to access the A3 website by logging in through the AF Portal

53d Wing via AFKN CoP 53d WG personnel can register for courses at the 53d WG CoP: https://afkm.wpafb.af.mil/community/views/home.aspx?Filter=OO-TE-AC-56

53d Wing via SharePoint 53d WG personnel can register using the 53 TMG TR SharePoint: https://eglin.eis.af.mil/53tmg/TTI/default.aspx

Non-96TW/53WG Personnel Training DoD personnel (military or civilian outside the 96th TW or 53d WG) are eligible to attend DOE courses on a space-available basis. Advisory and Assistance Support (A&AS) contractors, with their clients’ approval, may also attend the courses. Travel and text books are funded by the student’s unit. We supply course slides, software to load for training purposes, and free instruction.

Slides and course data files, reading, and support material USAF DOE CoP: https://afkm.wpafb.af.mil/community/views/home.aspx?Filter=OO-TE-MC-79

USAF DOE SharePoint: https://eglin.eis.af.mil/53tmg/DOE/default.aspx 2 DOE COURSE DESCRIPTIONS

Executive Overview of the Science of Test (DOE for Executives)

Target Audience: Center, Flag, and SES audiences; general base acquisition, science, and engineering students wanting an introductory overview of DOE.

Pre-requisite: None

Duration: 1-2 hours

Description: This introductory course demonstrates a powerful methodology for test and evaluation: Design of Experiments (DOE). The case for DOE begins with a presentation of the central challenge of test – how to draw enough samples to make the correct fielding decision, while simultaneously testing across a broad battle-space. DOE answers four fundamental questions: how many trials, under what conditions, executed in what order, and what are defensible conclusions? DOE history is discussed along with a series of case studies to illustrate how it works. The course features test examples showing how DOE often reduces testing cost by 30-80% while increasing knowledge; conversely the course discusses several cases where tests had too few assets and should have been augmented. The concluding portion of the course addresses challenges of leading change that must be dealt with to successfully alter the way an organization typically conducts tests.

Objectives: Personnel who complete this training will: • Understand the central challenge of test and the basic DOE process • Understand leadership’s critical role in deploying the DOE concept throughout an organization • Understand the potential of implementing DOE to “right size” each test • Understand barriers which must be overcome to successfully conduct testing

3 Leader’s Introduction to the Science of Test (DOE for Leaders)

Target Audience: Wing, Group, Squadron Leaders

Prerequisite: None

Duration: 4 hours

DOE for Aircrew

Target Audience: Test Aircrew and Unit Project Officers

Prerequisites: None

Duration: 4 hours

Description: DOE for Aircrew describes the process and principles of test design from a test aircrew/execution perspective to promote effective test team interaction and understanding. Along with an introduction to statistical fundamentals, the course emphasizes the need to understand the problem under test, how to map the test item process, and brainstorming techniques to determine factors and response variables.

4 Introduction to DOE Target Audience: Project managers, unit project officers, and operations suitability analysts. Test/project engineers and test project managers leading a team conducting a designed experiment. While graduates will not be practitioners of DOE, they will be conversant with unique features of DOE tests and able to facilitate discussions among practitioners and experts. Scientists and engineers wanting to explore whether DOE holds promise in their area of expertise will also benefit from this course. Other students who may profit include technical supervisors of R&D, test and evaluation, product qualification, or digital simulation.

Prerequisite: some familiarity with how experiments or tests are usually conducted is helpful, but not required

Duration: 16 Hours

Description: This course surveys the science of test process to deal with more than one variable (e.g., weather, target signatures, aircraft profile, etc.) and their effects on the MOE or metric. This practical introduction covers the general approach to testing using a statistically sound method for choosing the test set and an analysis method for objectively discovering the variables significantly affecting effectiveness. The course topics include an introduction to the science of test, hypothesis testing and sample size determination, constructing test matrices using factorial designs, and statistical model building. The graduate will be able to design and analyze simple factorial experiments with up to four variables, and know what methods are being employed by the test engineers and OA’s. We use Excel and Design Expert and a course project is integrated throughout to enforce the methods presented.

5 Foundations for Statistical Design of Experiments

Target Audience: Test/project engineers, and test project managers leading a team conducting a designed experiment. While graduates will not be practitioners of DOE, they will be conversant with unique features of DOE tests and able to facilitate discussions among practitioners and experts. Scientists, and Engineers wanting to explore whether DOE holds promise in their area of expertise will also benefit from this course. Other students who may profit include technical supervisors of R&D, test and evaluation, product qualification, or digital simulation

Prerequisite: an academic degree with a minimum of four courses in mathematics or supervisor recommendation. A previous course in statistics is helpful but not required.

Duration: 40 hours

Description: satisfying the need for statistics to deal with variation in test data and the concepts of the scientific method (hypothesis testing), this course introduces multifactor designs for varying multiple test conditions simultaneously while retaining the ability to link cause-to-effect. It explores design options with factors at two or more levels, including strategies for running a fraction of these combinations, designs for nonlinear response spaces (response surface methods), and a number of design augmentation schemes to resolve ambiguities in test outcomes. Advanced approaches to experimental design including optimal designs, split plot designs, and space filling designs will be demonstrated. Throughout the course, the subsequent analysis and model-fitting properties of the contemplated designs will be explored. Assumptions underlying analysis will be discussed, but not stressed. Just prior to the capstone laboratory exercise, other options in experimental design will be outlined, including the use of random effects factors and covariates. Students become familiar with the powerful statistics program Design Expert 8 (www.statease.com for free 45-day download) used in the Practitioner DOE series. The course material is conveyed through the use of lecture, group discussion, student exercises, group laboratories, and electronic reading materials.

Objectives: Personnel who complete this training will: • Understand need for statistics in test and evaluation • Understand hypothesis testing, error types, and how to design defined- risk tests 6 • Understand the mechanics and benefits of two-level factorial designs • Understand the efficiencies of and approach to testing using fractional factorial designs • Be able to analyze factorial and response to surface designs • Be able to develop response surface designs understand reasons for selecting these designs. • Be able to plan, design, execute, and analyze a test using statistically based designed experiments • Be familiar with special-use design options such as split plot and optimal designs

Important Information: • Laptop computers are REQUIRED. Units may be able to temporarily issue one with a local login and admin privileges to load software in class. Personal laptops are allowed and may be used. • The required text for DOE is Montgomery’s Design and Analysis of Experiments, 8th edition, ISBN 978-1118146927. Textbooks are provided to 53d Wing personnel only. All other personnel, please obtain your own copy. If available, extra books may be borrowed during the class only. A recommended site for purchasing the book is: http://www.bestwebbuys. com/Design-and-Analysis-of-Experiments-ISBN-9781118146927?isrc=-rd.

7 DOE for Characterization (DOE I)

Target Audience: Test project engineers, test engineers, scientists, and analysts

Prerequisite: DOE Foundations or consent of instructor (advanced undergraduate, 1st year graduate)

Duration: 40 Hours

Description: This is the second of three courses in DOE, building up to the general factorial with any number of variables at any number of levels. Well- designed experiments allow you to obtain reliable results faster, easier, and with fewer resources than with poorly-designed experiments. You will learn how to plan, conduct and analyze experiments efficiently for effective use in this course. Using examples from military T&E and the student’s own problem field, the course series has a strong applied flavor and is intended to graduate practicing experimentalists. The course objectives are to learn how to plan, design and conduct experiments for characterizing systems or processes, and analyze the resulting data to obtain objective conclusions. The techniques emphasized will be metrics of a well-designed experiment, the 2-level full factorial, fractional factorials, design augmentation, blocking, and the mixed-level fractional factorial with any number of variables at any number of levels. Students use the statistics program Design Expert 8 (Design Expert - www.statease.com for free 45 day download) to design and analyze tests, including the week long course project.

Objectives: Personnel who complete this training will: • Use Design Expert to design, graph, analyze, and report on data sets • Use sound experimental or test techniques in planning, designing, and executing experiments • Have a solid grasp of the assumptions and methods of empirical modeling using least squares regression • Design and analyze 2k replicated full factorial, single replicate full and fractional factorial designs, as well as general factorial and mixed level fractional factorial designs. • Understand how to control for nuisance sources of variability with blocking • Understand the capability of optimal designs to provide test matrix solutions for special needs conditions.

8 • Know the importance and mechanics of sequential testing including an initial screening experiment followed by augmentation to decouple model terms, estimate second order relationships, and validate the model • Know how to develop alternative design choices for a given test and compare the design alternatives using a myriad of metrics for well- designed experiments

9 DOE for Response Surfaces (DOE II)

Target Audience: Test project engineers, test engineers, scientists, and operational analysts

Prerequisite: DOE I or consent of instructor (advanced undergraduate, 1st year graduate)

Duration: 40 Hours

Description: DOE II is the third week of a series of experimental design and analysis courses covering the essentials of a recommended strategy for test and evaluation. This week extends experimental design concepts to provide a full complement of test matrix alternatives capable of accommodating many variables over multiple testing periods (missions, days, and sessions) with a variety of objectives to include characterization, prediction and optimization. Efficient fractional factorial designs for 2- or more level variables are addressed. Another primary topic is response surface methods, including designs for nonlinear systems and strategies to find optimal settings for the test variables. Multiple effectiveness measure studies are also covered. In addition, we discuss proper experimentation and analysis practices when the principle of randomized run order is not practical (split plot designs) or when the collected data is either binary or is continuous and skewed. Design Expert is the primary software program used for this course, but JMP will be demonstrated as well. The graduate of this 3-course sequence will be fully trained in Design Expert to practice the basic concepts of real-world DOE and able to exercise the discipline independently.

Objectives: Personnel who complete this training will: • Comprehend the use of DOE across military test projects by studying over a dozen example cases in detail • Know the details behind the methods for 2k and fractional 2k-p designs by review and in-depth study • Use Design Expert to design, examine, and analyze factorials and fractional factorials • Know the design and analysis techniques for fitting nonlinear input/ output relationships – 2nd order models. • Understand the methods for optimization of second order systems or experiments involving multiple responses • Know the split plot design and analysis approach, necessary when complete randomization is not practical

10 • Understand various modeling alternatives for non-normal response data, including binary and heavily skewed data • Understand the benefits of optimal designs for mixed level fractions, constrained input spaces, high-order models, and special case augmentation

11 Advanced Topics in Design of Experiments (DOE III)

Target Audience: Energized practitioners with reasonable experience who successfully completed DOE Foundations, I, and II

Prerequisite: DOE Foundations, I, and II

Duration: 40 Hours

Description: This course covers methods seasoned DOE practitioners require to handle testing aberrations. The content is geared towards equipping students with tools for attacking the more challenging problems, where either the variable composition or underlying response surface is complex, or violations of our standard assumptions and principles occur. The intent is to dig deeper into the characteristics and qualities of the experimental designs and explore the possibilities engaging computer generated algorithms to assist the analyst in building or augmenting designs for a specific purpose. A second objective is to discover alternative methods of empirical model building and analysis, when the data is messy. The final objective of this course is to provide a testing strategy for test conduct environments with hard-to-change factors. The format of the course will blend lecture, case studies along with extensive use of analytical software. Both Design Expert and JMP software will be used. Format will be lecture with example problems, small team assigned problems, reading for seat and laboratory formulations. We will be browsing through topics selected from the following, depending on instructor and student interest: • Empirical Data Modeling, handling Observational Studies • Optimal Designs • Space-filling Designs • Robust Parameter Design • Second-order Designs • Multiple Responses • Measurement Systems • Sequential Design & Augmentation • Split Plot Design and Analysis • Second-order Designs/Modeling • Nested Factors and Analysis of Covariates (ANCOVA) • Discrete and Non-normal Response Modeling • Augmentation/Validation/Prediction • Random Effects and Mixed Models

12 Important Information: • Laptop computers are REQUIRED. Units may be able to temporarily issue one with a local login and admin privileges to load software in class. A trial version of Design Expert (www. statease.com) and JMP (www.jump.com) must be installed to complete the coursework. Personal laptops are allowed and may be used. • Loaner textbooks are provided for students during class on a limited basis. The primary text for this course is Myers, Montgomery, and Anderson-Cook’s “Response Surface Methods”, 3rd edition. Wu and Hamada’s “Experiments: Planning, Analysis, and Parameter Design Optimization” is an alternate reference. A recommended site for purchasing the books is: http://www.bestwebbuys.com/books/search?isrc=b-compare- srchbox&t=ISBN&q=0201775700

13 OPERATIONS ANALYST FORUM

OAs and Engineers The OA Forum is an informal weekly discussion at Eglin in one form or another. It’s been in existence since 1999 with the intended purpose of: • continuation training in DOE to keep graduate skills sharp • problem-solving venue for geo-separated OAs with challenges, questions, or in search of a second opinion • mentoring opportunity for young guns seeking a bigger view of the OR world • a resource to enable Col Pete Vanden Bosch’s immortal motto “Seek professional help!”

Consider speaking at our Forum or presenting from your desk via DCO and dial-in. Chalk talks with minimal slides are preferred, though formal briefs are fine as well. Contact the host volunteers with presentations you’d like to give. Remember that a broader vision of operations research (OR) is one of our goals. You may talk about your unit, mission, and specialties.

Our facilitator-host volunteers: Ms. Teresa English of AFRL’s Munitions Directorate (English, Teresa CIV USAF AFMC AFRL/RWPLT [email protected]) Ms. Deb Busby of 53d Wing’s 28 TES (Busby, Deborah CIV USAF ACC 28 TES/EAA [email protected])

Required: Phone and Access to Internet and DCO account (preferred.) Guest login allowed currently. Account obtained at: https://www.dco.dod.mil/ When: Mondays except holidays 1400-1500ish Central Time (+1 Eastern -1 Mountain -2 Pacific -4 Hawaiian) Where: bldg 1 CR 220 Eglin AFB Where: https://connect.dco.dod.mil/eglindoe2010s Dial-in: Provided after Forum start via DCO (Opsec)

14 Forum Data Sets & Slides USAF Design of Experiments Community of Practice (CoP) https://afkm.wpafb.af.mil/community/views/home.aspx?Filter=OO-TE-MC-79 Root Folder » OA Forums » Eglin DOE CT and OA Forum » Forum Slides and Material (by date)

Past Forum Recordings Log into DCO, or create account. https://www.dco.dod.mil/ Choose “Public Recordings” on the left under Web Conferencing, Type in “EGLIN OA” on the right beside “Meeting/Recording Search” Consider making the OA Forum a resource for your testers/analysts. A persistent issue in OR is networking, especially for small shops of OAs. We do have a T&E focus, but try to talk about OR tools and OR jobs several times a year. The Forum is painless, can be done at the desk (or anywhere with Internet access), and will field questions from the attendees.

15 PROFESSIONAL RESOURCES

Design and Analysis Of Experiments Douglas C. Montgomery (2012), 8th edition, hardcover with 704 pp.; ISBN 978-1118146927 Includes a review of elementary statistical methods, designs for experiments with a single factor, incomplete block designs, factorial designs, multifactor designs and nested arrangements, randomization restrictions, regression analysis as a methodology for analysis of unplanned experiments, and response surface methodology. Probably the most widely used graduate and undergraduate text; Montgomery’s work includes numerous worked actual engineering examples, a wealth of web-supplementary material, and a satisfying depth of mathematical detail to show the underlying theory. We have used this book exclusively for more than 19 years to teach DOE and it has few peers.

Practical Nonparametric Statistics W. C. Connover (1999) 3rd Ed. Hard or soft cover; ISBN-13: 978-0471160687 Standard statistics make assumptions about how the data are distributed (usually normally), then give results based on the assumed distribution. Two big problems are that the distribution buried in the analysis may not be the right one, and that the assumption might not even be visible in the analysis. “Nonparametric statistics” make fewer assumptions about the parent distribution, and are robust in giving correct outcomes when assumptions are violated. Using the nonparametric technique side-by-side with the classical methods serves as an excellent check on the reliability of the classic results. For the novice practitioner, this book is the broadest catalog I know of how-to for nonparametric statistics: when each analysis applies and how to apply it. Even more, it gives insight into how most of the tests work. That gives the reader a better chance to understand each technique’s strengths, weaknesses, and applicability. For the student, including self-taught, it’s a clear and well-organized textbook. The exercises are varied and generally meaningful, and half have answers (though little discussion of how the answers were derived).

Empirical Model-Building and Response Surfaces George E. Box and Norman R. Draper (1987), hardcover with 669 pp.; ISBN 0-471-81033-9 The book begins from first principles and logically develops basic ideas, finally moving to a discussion of important research findings. Engineers, quality technologists, chemists, physicists, biologists, botanists, agricultural scientists, psychologists, as well as statisticians, will find many things of value within this text.

16 Probability and Statistics in Engineering and Management Science William W. Hines and Douglas C. Montgomery (1990), 3rd edition, hardcover with 732 pp.; ISBN 0-471-60090-3 Here is a completely up-to-date intermediate level text that develops in one volume major concepts with emphasis on applications of probability and statistics to engineering and management systems.

Introduction to Statistical Quality Control Douglas C. Montgomery (1991), 3rd edition, hardcover; ISBN 0-471-303534-X This modern practice of statistical quality control is given a strong engineering orientation in this book. It is designed as a textbook for students in engineering, management, and technology who are taking a first course in quality control at the college level.

Applied Regression Analysis Norman Draper and Harry Smith (1981), 2nd edition, hardcover with 709 pp.; ISBN 0-471-02995-5 This text presents a complete discussion of regression modeling with emphasis on concepts and applications It is particularly strong in regression diagnostics and violations of assumptions.

Response Surface Methodology Process and Product Optimization Using Designed Experiments Raymond H. Myers and Douglas C. Montgomery (2009), 3rd Ed. hardcover with 700 pp.; ISBN 978-0-470-17446-3 The process of identifying and fitting an appropriate response surface model from experimental data. The book deals with the exploration and optimization of response surfaces and integrates the three topics: Statistical Experimental Design Fundamentals, Regression Modeling Techniques, and Elementary Optimization Methods. Throughout, the text uses the elegant matrix algebra theory underlying these methods so this book is recommended for those comfortable in matrix theory. The third edition is complete with updates that capture the important advances in the field of experimental design including the generalized linear model, optimal design and design augmentation, space filling designs, and fraction of the design space prediction variance metrics allowing different designs to be compared and contrasted. With the first third devoted to classical design of experiments, this single volume can be used to teach both basic DOE as well as its extension into modeling nonlinear surfaces and system optimization.

17 Experimental Design Cochran and Cox (1957) 2nd Edition, soft cover; ISBN 978-0471545675 This classic reference volume has been superseded by improved treatments and examples (see Box and Montgomery) but contains many good descriptions and motivations for basic concepts like blocking, randomization, split plot designs, etc. Illustrates that the best of factorial designs were statistically mature long before popular use of these methods by practicing engineers and scientists.

Statistics for Experimenters George E. P. Box, William G. Hunter, and J. Stuart Hunter (2004), 2nd Ed. hardcover with 638 pp.; ISBN 0-471-09315-7 Every experimenter should have this classic text in his library. The authors are well known across the world and their presentation of design types is extremely thorough! Box, in particular, began as a practicing industrial chemist and was drawn to DOE to solve the many vexing problems of achieving manufacturing mixtures with multiple, conflicting qualities like cost, yield, time, profit, effectiveness, etc… Rewritten and updated, this new edition adopts the same approach as the initial landmark edition through worked examples, understandable graphics, and appropriate use of the leading statistics packages. Box begins each section with a problem to be solved and hen develops both the theory and practice of its solution, building on what has come before. Updated with recent advances in the field of experimental design, Box discusses computer algorithmic approaches to design augmentation, split plot designs for robust product optimization, multi-response optimization both graphically and numerically, Bayesian approaches to model selection, and projection properties of complex Plackett Burman screening designs that make them desirable in sparse processes dominated by a few major causal factors. Replete with physical, biological, social, chemical, and engineering examples, this book is an invaluable reference for all practicing experimentalists.

Experiments Planning, Analysis, and Parameter Design Optimization C. F. Jeff Wu, Michael Hamada (2000) 1st Ed, Hardcover; ISBN 0-471-25511-4 This text is a superb supplement to more classically oriented works on DOE. The past two decades have seen major progress in the field of experimental design for product and process improvement. This work explores some fields not fully covered in the more standard texts including the use of Orthogonal Arrays as a solution to mixed design fractionation, that have fallen out of favor since Taguchi Methods were largely discredited in the late 1980s. Wu also contains an excellent discussion and worked examples of the Generalized Linear Model and optimal design of experiments. Wu also treats Bayesian model selection via Gibbs Samplers and some unorthodox, but revealing ways of examining and evaluating design tables. Each chapter is motivated by a real experiment.

18 The Lady Tasting Tea: How Statistics Revolutionized Science in the 20th Century David Salsburg, PhD (2002) 1st Ed, Hard or Soft cover; ISBN-10: 0805071342 Amazon -- Science is inextricably linked with mathematics. Statistician David Salsburg examines the development of ever-more-powerful statistical methods for determining scientific truth in The Lady Tasting Tea, a series of historical and biographical sketches that illuminate without alienating the mathematically timid. Salsburg, who has worked in academia and industry and has met many of the major players he writes about, shares his subjects’ enthusiasm for problem solving and deep thinking. His sense of excitement drives the prose, but never at the expense of the reader; if anything, the author has taken pains to eliminate esoteric and ephemera from his stories.

This might frustrate a few number-head readers, but the abundant notes and references should keep them happy in the library for weeks after reading the book. Ultimately, the various tales herein are unified in a single theme: the conversion of science from observational natural history into rigorously defined statistical models of data collection and analysis. This process, usually only implicit in studies of scientific methods and history, is especially important now that we seem to be reaching the point of diminishing returns and are looking for new paradigms of scientific investigation. “The Lady Tasting Tea” will appeal to a broad audience of scientifically literate readers, reminding them of the humanity underlying the work. I particularly prize the insights into the men and women who created these techniques, the problems they faced, and how they wrestled intuition into polished mathematics.

19 POINTS OF CONTACT

Dr Jim Simpson, Chief Operations Analyst 53d Test Management Group, Eglin AFB, FL [email protected] 850-882-0619

Mr Greg Hutto, Chief Operations Analyst 96th Test Wing, Eglin AFB, FL [email protected] 850-882-5349

Mr Alex Sewell, Lead Operations Analyst 28th Test and Evaluation Squadron, Eglin AFB, FL [email protected] 850-882-1845

Dr Charles Listak, Director Test Training 53d Test Management Group, Eglin AFB, FL [email protected] 850-882-2288