CRM D0010079.A3/1Rev June 2004 The Art of Military Experimentation Brian McCue 4825 Mark Center Drive • Alexandria, Virginia 22311-1850 Approved for distribution: June 2004 SLQ Kl- Helen T. Karppi, Director (Acting) &( Expeditionary Systems & support Team - Integrated Systems and Operations Division This document represents the best opinion of CNA at the time of issue. It does not necessarily represent the opinion of the Department of the Navy. Approved for Public Release; Distribution Unlimited. Specific authority: NO0014-00 -D-0700. For copies of this document call: CNA Document Control and Distribution Section at 703-824-21 23. Copyright O 2004 The CNA Corporation Contents Preface ..................................................................................................................................... 1 Introduction ............................................................................................................................ 3 The key to the art of military experimentation .................................................................. 5 The plan of the work ........................................................................................................... 5 Experiments ............................................................................................................................ 7 Thought-experiments .......................................................................................................... 9 A thought-experiment of Galileo’s ................................................................................. 9 Thought-experiments’ larger role ................................................................................ 10 Physical experiments.......................................................................................................... 11 Galileo’s weight-dropping experiment: proving a point ............................................. 11 The Cavendish experiment: a measurement................................................................ 11 The Michelson-Morley experiment: a negative result ................................................. 12 Rutherford discovers the nucleus: an instance of serendipity..................................... 13 Operational experiments................................................................................................... 14 Milgram measures the “social distance” ....................................................................... 16 Artificial worlds .............................................................................................................. 17 Operational irreducibility.............................................................................................. 18 Military operational experiments.................................................................................. 19 Serendipity in military experimentation ...................................................................... 20 A hierarchy of military experiments................................................................................... 21 Military thought-experiments........................................................................................ 21 A question for discussion ............................................................................................... 23 War games ...................................................................................................................... 24 Limited technical assessments....................................................................................... 31 Limited objective experiments...................................................................................... 32 Advanced warfighting experiments .............................................................................. 33 Experimentation in real-world operations ................................................................... 33 Models, reality, theory, realism, garbage, and truth......................................................... 37 Models................................................................................................................................. 37 Reality ................................................................................................................................. 40 Theory................................................................................................................................. 42 Theory in physical experiments .................................................................................... 43 Theory in operational experiments .............................................................................. 44 The perils of inadequate theory .................................................................................... 47 Realism ................................................................................................................................ 49 Example: penetration, thrust, and swarm .................................................................... 49 The perils of serendipity................................................................................................ 51 Garbage in, garbage out .................................................................................................... 52 Truth................................................................................................................................... 53 Some theories with military applicability........................................................................... 57 Probability theory, and statistics........................................................................................ 58 i Search theory...................................................................................................................... 60 Detection theory ................................................................................................................ 62 Information theory ............................................................................................................ 67 Queueing theory ................................................................................................................ 69 Game theory....................................................................................................................... 71 Lanchester’s attrition theory ............................................................................................. 72 General comments on the theories .................................................................................. 76 Devising new theories ........................................................................................................ 78 Dimensional analysis and scaling .................................................................................. 78 Creativity and inspiration .............................................................................................. 83 Methodology.......................................................................................................................... 85 Hypotheses ......................................................................................................................... 85 Indicators and instances................................................................................................ 87 Capabilities versus technologies.................................................................................... 88 Measurements .................................................................................................................... 88 Base case versus experimental case ................................................................................... 91 Resetting ............................................................................................................................. 91 Statistics .............................................................................................................................. 93 Iteration.......................................................................................................................... 96 Serendipity.......................................................................................................................... 97 The role of modeling and simulation ............................................................................... 97 Validation ....................................................................................................................... 97 Non-predictive uses of simulation models .................................................................... 98 Using simulation models in military experimentation ................................................ 99 Design and analysis of a military experiment.................................................................. 101 A basic view of the experimentation process.................................................................. 101 A more sophisticated view of experimentation.............................................................. 102 Questions and ideas ..................................................................................................... 103 Focused study............................................................................................................... 105 Observation .................................................................................................................. 106 Reconstruction ............................................................................................................