Building Better Games for National Security Policy Analysis Towards a Social Scientific Approach
Total Page:16
File Type:pdf, Size:1020Kb
Dissertation Building Better Games for National Security Policy Analysis Towards a Social Scientific Approach Elizabeth M. Bartels This document was submitted as a dissertation in March 2020 in partial fulfillment of the requirements of the doctoral degree in public policy analysis at the Pardee RAND Graduate School. The faculty committee that supervised and approved the dissertation consisted of Steven Popper (Chair), Stacie Pettyjohn, and Yuna Wong. PARDEE RAND GRADUATE SCHOOL For more information on this publication, visit http://www.rand.org/pubs/rgs_dissertations/RGSD437.html Published 2020 by the RAND Corporation, Santa Monica, Calif. R® is a registered trademark Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions.html. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Abstract This monograph proposes an approach to game design grounded in logics of inquiry from the social sciences. National security gaming practitioners and sponsors have long been concerned that the quality of games and sponsors’ ability to leverage them effectively to shape decision making is highly uneven. This research leverages literature reviews, semi-structured interviews, and archival research to develop a framework that describes ideal types of games based on the type of information they generate. This framework offers a link between existing treatments of philosophy of science and the types of tradeoffs that a designer is likely to make under each type of game. While such an approach only constitutes necessary, but not sufficient, conditions for games to inform research and policy analysis, this work aims to offer pragmatic advice to designers, sponsors and consumers about how design choices can impact what is learned from a game. iii Table of Contents Abstract ......................................................................................................................................... iii Figures ........................................................................................................................................... ix Tables ............................................................................................................................................ xi Summary ..................................................................................................................................... xiii Acknowledgments ..................................................................................................................... xvii CHAPTER 1 Introduction: Games for National Security Policy Analysis and How to Improve Them . 1 What Is a National Security Policy Analysis Game? ............................................................................... 1 Why Use “National Security Policy Analysis Games”? ...................................................................... 2 Elements of a National Security Policy Game ..................................................................................... 4 Defining Games in Relation to Analytical Tools ................................................................................. 6 Application of National Security Policy Games ....................................................................................... 8 Typologies of Applications of National Security Policy Games ........................................................ 10 The Limits and Misuses of Policy Gaming ........................................................................................ 12 Building Better Games for National Security Policy Analysis .............................................................. 16 What Makes a Game Good? ............................................................................................................... 17 Two Competing Explanations for the Gap ......................................................................................... 19 Towards a Scientific Approach: Defining Logics for Game Design ................................................. 22 CHAPTER 2 Study Approach ...................................................................................................................... 23 Research Context .................................................................................................................................... 23 Limits of Existing Information on Specific Games ............................................................................ 23 Limits of the Expert Community ........................................................................................................ 26 Study Approach ...................................................................................................................................... 27 Phase 1: Understanding Policy Game Design in Scientific Terms .................................................... 27 Phase 2: Initial Framework Design .................................................................................................... 27 Phase 3: Validation ............................................................................................................................. 31 Phase 4: Examples .............................................................................................................................. 33 CHAPTER 3 Towards a Social Science of Policy Games .......................................................................... 36 Policy Games as Art: A Dominant Paradigm with Problems ................................................................. 37 Pitfalls of Artistic Approaches to Policy Analysis Gaming ............................................................... 39 The Artistic Case Against Science ..................................................................................................... 41 Towards a Science for Gaming .............................................................................................................. 42 v Jackson’s Typology of Philosophies of Science ................................................................................ 44 Philosophies of Science for Gaming .................................................................................................. 48 Producing Scientific Knowledge with Games ........................................................................................ 53 The Nature of Problems Suited to Gaming and the Need for a Bayesian Approach to Certainty ..... 54 Game Evidence: Empirical or Formal? .............................................................................................. 55 Basis of Game Analysis: Differences or Mechanisms? ..................................................................... 56 Conclusion .............................................................................................................................................. 58 CHAPTER 4 Four Archetypes of Games to Support National Security Policy Analysis ....................... 60 Overview of the Four Archetypes .......................................................................................................... 62 Differentiating the Types .................................................................................................................... 63 Games with Characteristics of Multiple Types .................................................................................. 69 Connecting Philosophy of Science for Games to the Archetypes .......................................................... 69 System Exploration ............................................................................................................................ 70 Alternative Conditions ....................................................................................................................... 71 Innovation ........................................................................................................................................... 71 Evaluation ........................................................................................................................................... 72 Different Games for Different Philosophies ....................................................................................... 73 Design tradeoffs ...................................................................................................................................... 74 CHAPTER 5 Designing Games for System Exploration ............................................................................ 76 Overview of Example System Exploration Games ...............................................................................