Intelligence Analysis Challenges and Vulnerabilities

Krizana Heuerb Trent, et al.c Hutchins, et al.d Johnsone Prematurely Formed The Vividness Inappropriate High Cognitive Secrecy versus Views Criterion Mental Set Workload Efficacy Tradeoff Willful Disregard of Absence of Evidence Environmental Potential for Error Focus on Current New Evidence Pressure Production Lack of Empathy Base-Rate Fallacy Fixation Time Pressure Time Constraints Ethnocentrism & Mirror- Oversensitivity to Recognition of Coping with Confirmation Bias, Imaging Consistency Relevant Data Uncertainty Norms, and Taboos Ignorance Anchoring Trust Data Overload Analytic Identity Rational-Actor Assessing Probability Experience Synthesizing Production-based Hypothesis or Denial of of a Scenario viewed as Multiple Sources Rewards and Rationality Expertise of Information Incentives Proportionality Bias Availability Rule Learning Insufficient Tools Analytic Training Defensive Avoidance & Similarity of Cause Tool Organizational Perception of Wishful Thinking and Effect Understanding Context “Tradecraft” Versus Scientific Conservatism in Internal vs. External Sustained Complex Human Methodology Probability Estimation Causes of Behavior Attention Judgments Presumption that Persistence of Support for One Impressions Based on Hypothesis Disconfirms Discredited Evidence Others Best-Case Analysis or Overestimating Our Worst-Case Analysis Own Importance Image and Self-Image Illusory Correlation Overconfidence in Expression of Subjective Estimates Uncertainty Inappropriate Bias Favoring Analogies & Superficial Perception of Lessons from History Centralized Direction Evoked-Set Reasoning Coping with Evidence of Uncertain Accuracy Excessive Secrecy Presumption of Unitary Bias in Favor of Action by Organizations Causal Explanations & Organizational Parochialism Note. From When is Analysis Sufficient: A Study of How Professional Analysts Judge Rigor by D. Zelik, 2007. aAdapted from Intelligence Essentials for Everyone by L. Krizan, 1999. bAdapted from Psychology of Intelligence Analysis, by R. J. Heuer, 1999. cAdapted form “Challenges for Cognition in Intelligence Analysis,” S. A. Trent, et al., in press, Journal of Cognitive Engineering and Decision Making. dAdapted from “What Makes Intelligence Analysis Difficult? A Cognitive Task Analysis of Intelligence Analysts,” S. G. Hutchins, et al., 2006, in P. L. Pirolli (Ed.) Assisting People to Become Independent Learners in the Analysis of Intelligence. eAdapted from Analytic Culture in the US Intelligence Community: An Ethnographic Study, by R. Johnson, 2005.