A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis Prepared by the US Government March 2009 This primer highlights structured analytic techniques—some widely used in the private sector and academia, some unique to the intelligence profession. It is not a comprehensive overview of how intelligence officers conduct analysis. Rather, the primer highlights how structured analytic techniques can help one challenge judgments, identify mental mindsets, stimulate creativity, and manage uncertainty. In short, incorporating regular use of techniques such as these can enable one to structure thinking for wrestling with difficult questions. TABLE OF CONTENTS Introduction ................................................................................................................. 1 How To Use These Techniques .................................................................................. 5 Diagnostic Techniques ................................................................................................ 7 Key Assumptions Check ..................................................................................... 7 Quality of Information Check ............................................................................. 10 Indicators or Signposts of Change .................................................................... 12 Analysis of Competing Hypotheses (ACH) ....................................................... 14 Contrarian Techniques ............................................................................................... 17 Devil’s Advocacy ................................................................................................ 17 Team A/Team B ................................................................................................. 19 High-Impact/Low-Probability Analysis ............................................................... 22 “What If?” Analysis ............................................................................................. 24 Imaginative Thinking Techniques .............................................................................. 27 Brainstorming ..................................................................................................... 27 Outside-In Thinking ............................................................................................ 30 Red Team Analysis ............................................................................................. 31 Alternative Futures Analysis ............................................................................... 34 Strategies for Using Structured Analytic Techniques ................................................ 38 Selective Bibliography ............................................................................................... 40 INTRODUCTION THE “MIND-SET” CHALLENGE Cognitive and perceptual biases in Intelligence human perception and judgment are analysts should Using the analytic techniques contained another important reason for analysts to “be self-conscious in this primer will assist analysts in consider alternatives. As Richards Heuer about their reasoning dealing with the perennial problems and others have argued, all individuals processes. They should think about of intelligence: the complexity of assimilate and evaluate information how they make international developments, incomplete through the medium of “mental models” judgments and reach and ambiguous information, and the (sometimes also called “frames” or conclusions, not just inherent limitations of the human mind. “mind-sets”). These are experience- about the judgments Understanding the intentions and based constructs of assumptions and and conclusions capabilities of adversaries and other expectations both about the world in themselves. foreign actors is challenging, especially general and more specific domains. when either or both are concealed. These constructs strongly influence what ” — Richards Heuer, Moreover, transnational threats today information analysts will accept—that is, The Psychology of pose even greater complexity, in that data that are in accordance with analysts’ Intelligence Analysis 1 they involve multiple actors—including unconscious mental models are more nonstate entities—that can adapt and likely to be perceived and remembered transform themselves faster than those than information that is at odds who seek to monitor and contain them. with them. Finally, globalization has increased the diversity of outcomes when complex, Mental models are critical to allowing interactive systems such as financial flows, individuals to process what otherwise regional economies or the international would be an incomprehensible volume of system as a whole are in flux.2 information. Yet, they can cause analysts to overlook, reject, or forget important The first hurdle for analysts is identifying incoming or missing information that is the relevant and diagnostic information not in accord with their assumptions and from the increasing volume of ambiguous expectations. Seasoned analysts may and contradictory data that is acquired be more susceptible to these mind-set through open source and clandestine problems as a result of their expertise means. Analysts must also pierce the and past success in using time-tested shroud of secrecy—and sometimes mental models. The key risks of mind- deception—that state and nonstate actors sets are that: analysts perceive what they use to mislead. A systematic approach expect to perceive; once formed, they are that considers a range of alternative resistant to change; new information is explanations and outcomes offers one assimilated, sometimes erroneously, into way to ensure that analysts do not dismiss existing mental models; and conflicting potentially relevant hypotheses and information is often dismissed or ignored. supporting information resulting in missed opportunities to warn. 1 Richards J. Heuer, Jr., The Psychology of Intelligence Analysis (Washington: Center for the Study of Intelligence, 1999). 2 These observations were drawn from a lengthier treatment of cognitive bias found in the Sherman Kent Center’s Occasional Paper, Making Sense of Transnational Threats, Vol. 3, No. 1, October 2004. 1 Common Perceptual and Cognitive Biases Perceptual Biases Biases in Evaluating Evidence Expectations. We tend to perceive what we Consistency. Conclusions drawn from a expect to perceive. More (unambiguous) small body of consistent data engender information is needed to recognize an more confidence than ones drawn from a unexpected phenomenon. larger body of less consistent data. Resistance. Perceptions resist change even Missing Information. It is difficult to in the face of new evidence. judge well the potential impact of missing Ambiguities. Initial exposure to ambiguous evidence, even if the information gap or blurred stimuli interferes with accurate is known. perception, even after more and better Discredited Evidence. Even though information becomes available. evidence supporting a perception may be proved wrong, the perception may not quickly change. Biases in Estimating Probabilities Biases in Perceiving Causality Availability. Probability estimates are Rationality. Events are seen as part of influenced by how easily one can imagine an an orderly, causal pattern. Randomness, event or recall similar instances. accident and error tend to be rejected as explanations for observed events. For Anchoring. Probability estimates are example, the extent to which other people adjusted only incrementally in response to or countries pursue a coherent, rational, new information or further analysis. goal-maximizing policy is overestimated. Overconfidence. In translating feelings of Attribution. Behavior of others is attributed certainty into a probability estimate, people to some fixed nature of the person or are often overconfident, especially if they country, while our own behavior is attributed have considerable expertise. to the situation in which we find ourselves. Intelligence analysts must actively review • Identifying indicators of change (or the accuracy of their mind-sets by applying signposts) that can reduce the chances structured analytic techniques that will of surprise. make those mental models more explicit and expose their key assumptions. Incorporating findings derived from these The techniques found in this primer are techniques into our intelligence products designed to assist in this regard by: also serves the policymaker by: • Instilling more structure into the • Highlighting potential changes that intelligence analysis. would alter key assessments or predictions. • Making analytic arguments more transparent by articulating them and • Identifying key assumptions, challenging key assumptions. uncertainties, intelligence gaps and disagreements that might illuminate • Stimulating more creative, “out-of-the- risks and costs associated with policy box” thinking and examining alternative choices. outcomes, even those with low probability, to see if available data might • Exploring alternative outcomes for which support these outcomes. policy actions might be necessary. 2 Strategic Assumptions That Were Not Challenged 1941 World War II Japan would avoid all-out war because it recognized US military superiority. Given that US superiority would only increase, Japan might view a first strike as the only way to knock America out of the war. 1950s Korean War China would not cross the Yalu River in support of the North Korean government. Red