Dissertation Value of Information for Policy Analysis David Manheim This document was submitted as a dissertation in March 2018 in partial fulfillment of the requirements of the doctoral degree in public policy analysis at the Pardee RAND Graduate School. The faculty committee that supervised and approved the dissertation consisted of Paul Davis (Chair), Steven Popper, and Kathryn Laskey. Partial funding was provided by the Anne and James Rothenberg Dissertation Award. PARDEE RAND GRADUATE SCHOOL For more information on this publication, visit http://www.rand.org/pubs/rgs_dissertations/RGSD408.html Published 2018 by the RAND Corporation, Santa Monica, Calif. R® is a registered trademark Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions.html. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Abstract This dissertation considers the viability of applying Value of Information (VoI) methods in complex systems for policy analysis, concluding that these methods can be applied, but that different methods are appropriate in different cases. VoI is value to a decisionmaker of the difference information makes for a decision, or what a decisionmaker should pay for information. Compared to present practice, specific value of information techniques can potentially improve policy analysis and decision making. To that end, a methodology for policy applications of VoI is presented that can simplify practical application. The dissertation applied computational methods for VoI to two case studies in disparate domains: biosurveillance and detection of violent extremists. Each of the case studies outlines how Bayesian methods can be used to evaluate VoI, including a novel formulation for considering additional information sources in the systems. The case studies are used both to consider the relative benefits and drawbacks of these approaches in policy decision making, and to identify potential challenges. Approaches for identifying and avoiding these challenges and for selecting appropriate methods are integrated into the overall methodology, which is presented in the final chapter. iii Table of Contents Abstract .......................................................................................................................................... iii Table of Contents ........................................................................................................................... iv Figures............................................................................................................................................ ix Equations......................................................................................................................................... x Tables ............................................................................................................................................. xi Summary ....................................................................................................................................... xii Acknowledgments........................................................................................................................ xiv Abbreviations ............................................................................................................................... xvi 1 Introduction ............................................................................................................................... 1 1.1 Importance of VoI for Policy ............................................................................................................. 1 1.2 What is Value of Information (VoI)? ................................................................................................. 2 1.2.1 Basic Formulations of VoI ...................................................................................................... 2 1.2.2 Decision to Treat ..................................................................................................................... 3 1.2.3 Decision Trees ......................................................................................................................... 4 1.2.4 Perfect Information ................................................................................................................. 4 1.2.5 Imperfect Information ............................................................................................................. 5 1.3 Considerations on Computing VoI and Related Quantities ............................................................... 7 1.3.1 Extensions of VoI for Continuous Information ....................................................................... 7 1.3.2 Real Options ............................................................................................................................ 8 1.3.3 Algebraic Tools for Parametric Uncertainties ......................................................................... 8 1.3.4 Related Methods and Applications ....................................................................................... 12 1.4 Applying VoI and Related Methods in Policy Analyses ................................................................. 13 1.5 Research Approach .......................................................................................................................... 14 1.5.1 Methodological Research Questions ..................................................................................... 15 1.5.2 Dissertation Structure and Outline ........................................................................................ 15 1.5.3 Heterogeneous Information Fusion ....................................................................................... 15 1.5.4 Biosurveillance ...................................................................................................................... 17 2 An Overview of Policy Decisions for VoI Questions ............................................................. 19 2.1 Taxonomy for Decision making ...................................................................................................... 19 2.1.1 Conceptual Model for Decisions ........................................................................................... 19 2.1.2 Refining the Conceptual Model ............................................................................................ 20 2.2 The Role of Models in Decision Processes ...................................................................................... 22 2.2.1 Problem Formulation ............................................................................................................ 23 2.2.2 Criteria and Metric Selection ................................................................................................ 24 2.2.3 Alternative Generation .......................................................................................................... 26 2.2.4 Screening ............................................................................................................................... 27 2.2.5 Assessment ............................................................................................................................ 28 iv 2.2.6 Concluding a Decision Process ............................................................................................. 28 2.2.7 Necessity of Utility Theory in VoI Decisions ....................................................................... 29 2.2.8 Implicit Utilities and Necessity ............................................................................................. 30 2.3 Models and Decision Problems in General ...................................................................................... 31 2.3.1 Example: Flood Control Systems .......................................................................................... 33 2.3.2 Iteration and Model Selection Problems ............................................................................... 34 2.3.3 Model Selection for Information-Sensitive Decisions .......................................................... 34 2.4 Requirements for VoI Evaluation .................................................................................................... 35 2.4.1 Information Models ............................................................................................................... 35 2.5 Computational Model Characteristics .............................................................................................. 37 2.5.1 Causal Models ......................................................................................................................