COMPREHENSIVE TESTING OF IMMINENT THREAT PUBLIC MESSAGES FOR MOBILE DEVICES First Responders Group October 2014 AUTHORS (in alphabetical order) Hamilton Bean Brooke Liu Stephanie Madden Dennis Mileti Jeannette Sutton Michele Wood COMPREHENSIVE TESTING OF IMMINENT THREAT PUBLIC MESSAGES FOR MOBILE DEVICES Prepared for Department of Homeland Security Science and Technology Directorate, Support to the Homeland Security Enterprise and First Responders Group Comprehensive Testing of Imminent Threat Public Messages for Mobile Devices For information about this publication or other START research, contact NATIONAL CONSORTIUM FOR THE STUDY OF TERRORISM AND RESPONSES TO TERRORISM 8400 Baltimore Ave., Suite 250 College Park, MD 20742 Tel (301) 405-6600 [email protected] www.start.umd.edu ii Table of Contents Executive Summary .............................................................................................................................................................. 3 1. Background ................................................................................................................................................................... 7 2. Literature Review ......................................................................................................................................................... 8 3. Research Methods ....................................................................................................................................................... 12 2.1 Quantitative Experiments .......................................................................................................................................... 13 2.2 Qualitative Think-Out-Loud Interviews and Focus Groups .................................................................................... 20 2.3 Telephone Survey ....................................................................................................................................................... 21 3. Prime Research Questions Findings .......................................................................................................................... 25 3.1 Order of Message Contents ........................................................................................................................................ 25 3.3 Message Source .......................................................................................................................................................... 26 3.4 Map Inclusion ............................................................................................................................................................ 28 3.5 Relative Importance of Content Elements................................................................................................................. 29 3.6 Generalizing across Hazard Types ............................................................................................................................ 31 3.7 Message Length Efficacy ........................................................................................................................................... 34 4. Add-on Research Questions’ Findings ...................................................................................................................... 36 4.1 Introduction ................................................................................................................................................................ 36 4.2 Inclusion of a URL .................................................................................................................................................... 36 4.3 Familiarity with the WEA Service ............................................................................................................................. 38 4.4 Understanding Acronyms .......................................................................................................................................... 39 4.5 How to Best Express Time ......................................................................................................................................... 40 4.6 How to Best Express Location ................................................................................................................................... 41 4.7 Understanding of Alert and Warning Concepts........................................................................................................ 42 4.8 Optimum Level of Fear Arousal ................................................................................................................................ 43 5. Additional Research Questions’ Findings ..................................................................................................................... 43 5.1 WEA Diffusion Curve ................................................................................................................................................ 43 5.2 Guidance Mobilization Curve .................................................................................................................................... 44 5.3 Validation of Experimental Optimized Outcome Measures ..................................................................................... 44 5.4 Serendipitous Findings .............................................................................................................................................. 45 6. Optimized Messages and Templates.............................................................................................................................. 45 6.1 Optimized Test Messages ........................................................................................................................................... 45 6.2 Optimized Message Templates ................................................................................................................................... 50 References ............................................................................................................................................................................ 53 Appendix A: Experimental Designs ................................................................................................................................. 59 1 Appendix B: Experimental Messages ................................................................................................................................ 61 Appendix C: Experimental Maps ...................................................................................................................................... 66 Appendix D: Example Questionnaires from Internet and Laboratory Experiments ................................................... 67 Appendix E: Additional Experimental and Survey Statistical Results ........................................................................ 110 Appendix F: Qualitative Test Messages, Research Timeline, and Emotions .............................................................. 127 Appendix G: Qualitative Focus Groups Research Map ................................................................................................ 129 Appendix H: Selected Participants’ Comments from Qualitative Research ............................................................... 130 Appendix I: Qualitative Research Coding Sheet ............................................................................................................ 148 Appendix J: Timeline of Boulder Flood Alerts and Warnings ..................................................................................... 151 Appendix K: Demographic Distribution of Survey Sample .......................................................................................... 165 Appendix L: Survey Questionnaire ................................................................................................................................. 166 Appendix M: WEA Diffusion and Guidance Mobilization Curves .............................................................................. 188 Appendix N: Validation of Experimental Optimized Outcomes……………………….……………………………..193 2 Executive Summary This project sought to determine the optimized message contents of imminent threat wireless emergency alert (WEA) messages delivered over mobile communication devices. This report presents findings for the first WEA messages disseminated about imminent threats (i.e., first alert messages) from two research phases with U.S. adults: (1) eight experiments, seven focus groups and 50 think-out-loud interviews; and (2) a survey of an actual “real world” severe flood in Boulder, Colorado. It also integrates findings from across study methods and provides actionable guidance and considerations for optimized message contents of imminent one-hour-to- impact threat alerts delivered over mobile communication devices. Primary conclusions from the research performed to date are: 1. Short alert and warning messages are unlike any others. The optimized order of their contents is unique; their limited length constrains public understanding of the message source; it is not immediately clear for some recipients whether the message is meant for them; the key content elements of guidance (describing what to do and how to do it) and hazard (describing why they should do it) cannot be adequately communicated; and short messages cannot overcome people’s pre-event hazard-specific perceptions. Hence, to