Priority Challenges for Social and Behavioral Research and Its Modeling Paul K. Davis, Angela O’Mahony, Timothy R. Gulden, Osonde A. Osoba, Katharine Sieck C O R P O R A T I O N For more information on this publication, visit www.rand.org/t/RR2208 Library of Congress Cataloging-in-Publication Data is available for this publication. ISBN: 978-0-8330-9995-2 Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2018 RAND Corporation R® is a registered trademark. Cover: jolly/stock.adobe.com. Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Preface The Defense Advanced Research Proj ects Agency (DARPA) is invest- ing in diverse aspects of social- behavioral research and modeling. This report is part of a study to assist DARPA in defining challenges and prioritizing such investments. Its initial version was a provocative read- ahead for a workshop of prominent researchers. This final version incorporates insights from that workshop and subsequent research. Comments are welcome and should be addressed to the proj ect leaders Paul K. Davis ([email protected]) and Angela O’Mahony (aomahon1 @rand.org). The research in this report was sponsored by Dr. Jonathan Pfautz, a program man ager in DARPA’s Information Innovation Office (I20). The research was conducted within the International Security and Defense Policy and Acquisition and Technology Policy Centers of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secre- tary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense intel- ligence community. For more information on the centers, see www.rand.org/nsrd/ndri /centers/ISDP.html and www.rand.org/nsrd/ndri/centers/atp.html or contact the directors (contact information is provided on the webpage). iii Table of Contents Preface ............................................................................. iii Figures ............................................................................. ix Tables .............................................................................. xi Summary ........................................................................ xiii Acknowl edgments ............................................................ xxix CHAPTER ONE Introduction ....................................................................... 1 Objectives ........................................................................... 1 Background ......................................................................... 3 Structure of Report ................................................................ 9 CHAPTER TWO Difficulties, Challenges, and a Broad Strategy for Next Steps ..........11 Inherent Difficulties of Social- Behavioral Science and Its Modeling ......................................................................11 Complex Adaptive Systems ....................................................12 Wicked Prob lems ...............................................................13 The Dynamic Character of People and Social Systems .....................14 Obstacles Due to Shortcomings of Current Practice .........................14 Fragmentation ...................................................................14 Methods and Norms ...........................................................15 Representing the Richness of the Science ....................................17 Modularity and Composability .............................................. 20 Rethinking Model Validity ................................................... 23 v vi Priority Challenges for Social-Behavioral Research and Its Modeling CHAPTER THREE Broad Strategy for Next Steps .................................................33 CHAPTER FOUR Tightening the Theory- Modeling- Experimentation Research Cycle ..............................................................39 Background on Differences of Philosophy .....................................39 An Idealized System View Relating the Real and Model Worlds ...........44 Real ity versus the Ideal ..........................................................48 Real ity and Its Implications ...................................................48 Moving Toward the Ideal .....................................................50 CHAPTER FIVE Theory and Its Modeling .......................................................55 Assimilating Modeling Better into Social- Behavioral Science ...............55 Assuring That Modeling Is Faithful to Subtleties of Theory .................57 Toward Multivariate Causal Models ............................................59 Causality .........................................................................59 Multivariate Aspects and Nonlinearity .......................................61 Dealing Routinely with Uncertainty ...........................................62 Going Beyond Simulation: Using a Portfolio of Modeling Methods .........63 Toward Greater Generality, Synthesis, and Coherence .......................67 Issues .............................................................................67 Scale and Perspective ...........................................................67 Reconciling Models Across Type, Formalism, Resolution, and Perspective .............................................................70 Improving Scientific Practices ...................................................76 CHAPTER SIX Experimentation .................................................................79 Theory- Informed Experimentation .............................................79 Urging a Theory- Informed Approach Is Not Radical ..................... 80 To Inform with Theory Is Not to Impose a Theory........................ 80 Connections Exist with Multiresolution Modeling .........................81 Contents vii Other Issues ........................................................................82 Estimating Difficult- to- Measure Variables ...................................82 Speed, Tailoring .................................................................83 Joint Planning for Empirical and Computational Experimentation ......83 New Sources and New Technology .............................................85 Exploratory Research and Analy sis ............................................ 86 CHAPTER SEVEN Modernizing Model- Based Decision Aiding ...............................89 The Decision- Aiding Aspect of the Challenge .................................89 Deep Uncertainty................................................................90 Princi ples and Lessons Learned .................................................91 Implications for Analy sis Outputs and Communication ....................92 CHAPTER EIGHT Methods and Technology ......................................................97 Capacity for Multidimensional Uncertainty Analy sis ........................97 Variable- Structure Simulation and Emergence................................98 Knowledge Management and Heterogeneous Information Fusion .........99 Composability .................................................................101 Theory- Informed Empirical and Computational Experimentation .......102 Interactivity, Gaming, and Related Visualization ..........................104 Other ..........................................................................104 CHAPTER NINE Infrastructure, Culture, and Governance ................................. 107 Infrastructure .................................................................... 107 Cultural Mindsets ............................................................... 108 Governance and Ethics, Privacy, and Civil Liberties ....................... 109 Results from a Workshop on Ethics and Privacy.......................... 109 More General Ethical Reasons for Caution................................ 110 Final Comments ................................................................ 111 APPENDIXES A. Social and Behavioral Modeling Laboratories ........................ 113 B. Definitions ................................................................... 125 viii Priority Challenges for Social-Behavioral Research and Its Modeling C. Validation .................................................................... 129 D. Participants in Workshops ............................................... 135 Abbreviations ................................................................... 137 References ......................................................................