Experimental Red Teaming to Support Integration of Information in Joint Operations Final Report PI: Dr. Gary Ackerman June 2021 University at Albany, SUNY: Albany, NY Prepared for the Strategic Multilayer Assessment Office of the Secretary of Defense Joint Staff J-39 © CART, 2021 About the Center for Advanced Red Teaming (CART) The Center for Advanced Red Teaming (CART) is an interdisciplinary Research Center within the College of Emergency Preparedness, Homeland Security and Cybersecurity at the University at Albany (SUNY). As the first academic center devoted to advancing the art and science of red teaming, CART seeks to develop research, practice, and education in this growing area of security studies. Citation Gary A. Ackerman, Douglas Clifford, Anna Wetzel, Jenna LaTourette and Hayley Peterson, “Experimental Red Teaming to Support Integration of Information in Joint Operations”, Prepared for the Strategic Multilayer Assessment, Office of the Secretary of Defense Joint Staff J-39 (University at Albany, SUNY: Albany, NY, 2021). Project Team Principal Investigator: Dr. Gary Ackerman Project Manager: Hayley Peterson Project Researchers: Douglas Clifford, Jenna LaTourette, Mike Mieses, Hayley Peterson, Anna Wetzel Research Assistants: Shane Carpenter, Rob Choudhury, Nikita Losi, Leyna Ruvola, Cory Schermerhorn, Ryan Szpicek Acknowledgement and Disclaimer The material in this report is based upon work supported by the U.S. Department of Defense, and NSI. The views and conclusions contained in this document are those of the authors and CART, and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the United States Department of Defense or NSI, inc. Contact Information For more information on this project, please contact CART at [email protected] 2 © CART, 2021 Table of Contents Executive Summary .............................................................................................................................. 4 Introduction ............................................................................................................................................ 6 Methodology ........................................................................................................................................... 7 Overall Scenario Analysis ................................................................................................................. 21 Scenario 1: Arms Support to Belligerents .......................................................................... 25 Scenario 2: US Military Exercises in the Western Balkans / Asia .............................. 31 Scenario 3: Space Junk .............................................................................................................. 33 Scenario 4: Bioweapons Laboratory Accusations ........................................................... 35 Scenario 5: Integration Leads to Instability ...................................................................... 38 Scenario 6: Crisis in the South China Sea ........................................................................... 43 Conclusion ............................................................................................................................................. 45 Appendix A: Scenario 1 Results ...................................................................................................... 49 Appendix B: Scenario 2 Results ...................................................................................................... 82 Appendix C: Scenario 3 Results ...................................................................................................... 88 Appendix D: Scenario 4 Results ..................................................................................................... 92 Appendix E: Scenario 5 Results ................................................................................................... 103 Appendix F: Scenario 6 Results ................................................................................................... 113 Appendix G: Competitive Information Environment Red Teaming Protocol………… 116 3 © CART, 2021 Executive Summary As part of the modeling and simulation phase of the IIJO effort, the Center for Advanced Red Teaming (CART) worked closely with the ICONS Project at the University of Maryland to employ two separate yet integrated human simulation approaches to test and build on the findings of earlier components of the IIJO effort. The CART portion of the simulation involved: 1. Distilling 46 propositions from the Net Assessment and TTXs into 12 explorable insights (EIs) regarding the competitive information environment. 2. Testing these EIs in six scenario-based Red Team experiments using 223 U.S.-based proxy participants from similar cultural backgrounds to actual adversary target populations (Taiwan for the Asian context and several Southeast European countries for the European region). 3. Collecting data on several measures of messaging effectiveness and analyzing this data to validate or shed new light on the EIs. The experiments yielded a number of takeaways relevant to the IIJO project. 1. The United States begins with a reputational / perception advantage over its GPC competitors, but the gap is fairly narrow between the U.S. and China in the European context. 2. Wherever it was tested (Scenarios 2, 4, and 5), there was robust evidence that GPC adversary propaganda that seeks to cast the United States in a negative light is effective in lowering attitudes towards the U.S., trust in the U.S. and U.S. influence among targeted audiences in non-GPC states (in our experiments, Taiwan and the states of southeastern Europe). Given that the perceptions of these audiences can impact their own countries’ and others’ military and political support for the United States, these findings confirm that IIJO is crucial and that the United States military needs to place significant emphasis on OIE moving forward. Furthermore, messaging to foreign populations (whether through traditional or new channels such as social media) cannot be left out of operations. 3. What countries do (as opposed to only what they say) matters. Hypocrisy by any GPC state leads to negative perceptions among target audiences, but there is no evidence that this harms the U.S. more than its GPC adversaries. 4. There is some, although not robust, evidence to indicate that messaging regarding U.S. economic success may not go as planned and could actually hurt foreign perceptions of the United States, whereas the jury is still out on the effects of similar messaging for other GPC states. 5. The effectiveness of messaging regarding the economic shortcomings of GPC adversaries like the PRC remains unclear. 6. There is no experimental evidence to suggest that emphasizing U.S. values is advantageous in its messaging and at least a possibility that doing so might negatively affect perceptions about the U.S. 7. Adopting a victimization narrative does not appear to be an effective messaging strategy in OIE, and may in fact backfire, lowering the believability of the message and perceptions of the country utilizing this approach. 4 © CART, 2021 8. There is insufficient evidence to suggest that a non-U.S. government messenger is preferable to a U.S. government messenger, but more research is required on this point. 9. There was partial support (but only in one AOR) for the proposition that uncrafted and untargeted messages are more effective in influencing perceptions about the U.S., but further research is required to determined when this finding is applicable and when it is not. 10. With the possible exception of how much the message is believed and shared, there was no experimental support for the notion that positive, proactive messages are more effective than negative, reactive messages. 11. There is some limited, provisional support for the proposition that adversary messages that attack common values between the U.S. and the target population will have a more powerful (negative) effect. 12. The proposition that messaging that resonates with current beliefs and perceptions of the target audience will have greater believability received no experimental support. There were inconclusive findings as to the effects of more resonant messaging on other measures of effectiveness, with contradictory findings in the Asian and European samples. 13. There is evidence that, in a crisis, it is better to send no message than to urge allies to refrain from escalation. 14. There can sometimes be unforeseen effects to OIE. For example, in some experiments, messaging focused on one country actually affected perceptions of other GPC states (including the state doing the messaging). In addition to the substantive findings, at the programmatic level, the experiments demonstrated how human simulation can be used to test emerging phenomena or novel ideas that arise from the insights of experts and various other knowledge artifacts developed during the course of a typical SMA study. By exposing these insights to realistic simulations involving disinterested participants at scale, the use of an integrated human simulation approach (experiments plus table-top exercises) can both validate previous findings and reveal new dynamics in complex systems like the OIE. 5 © CART, 2021 Introduction In order to address the overall IIJO1 Project goal of assessing the ways in which the Joint Force can most effectively integrate information and influence