The Missing Piece: Why Intelligence Reform Failed After 9/11 A Dissertation Submitted to the Faculty of the Graduate School of Arts and Sciences of Georgetown University in partial fulfillment of the requirement for the degree of Doctor of Philosophy In Government By Catherine B. Lotrionte, J.D. Washington, DC July 25, 2008 Copyright 2008 by Catherine Lotrionte All Rights Reserved ii The Missing Piece: Why Intelligence Reform Failed After 9/11 Catherine B. Lotrionte, J.D. Thesis Advisor: Anthony Clark Arend, Ph.D. Abstract On September 11, 2001, the U.S. government failed to prevent a terrorist attack against the U.S. homeland. The lack of a coordinated, strategic warning of the 9/11 terrorist attacks exposed serious systemic problems within the U.S. national security system. This event resulted in a rush to reform the U.S. intelligence community. In an effort to determine the causes of the failure a number of official investigations were conducted. While these investigations were successful in providing a wealth of extensive information that would have otherwise not been available to the public, the investigations‘ conclusions and recommendations failed to fix the underlying problems that caused the intelligence failures. In fact, in some ways, their recommendations may have made the situation worse. One major reason that the 9/11 investigations failed is because they neglected to consider critical social science theories that are relevant to questions of government reform. For example, none of the investigations considered the insights from theories such as new institutionalism, normal accident theory or organizational culture theory. iii In failing to seriously explain the theoretical insights from social science theory, these investigations neglected critical information that would have improved the reform efforts and made the U.S. safer. This study applies theoretical insights from social science theory to critique the reforms and to propose a new reform agenda for the intelligence community. Importantly, this study focuses on three common criticisms of the intelligence community made by the investigations: (1) the lack of imagination on behalf of the U.S. government and, in particular, the CIA to anticipate the threat; (2) the lack of accountability within the intelligence community to hold individuals responsible for failures like 9/11; and (3) the lack of an effective intelligence collection capability within the United States. In examining these criticisms, this study applies the lessons from constructivism, new institutionalism and organizational theory and demonstrates how the 9/11 investigations failed to applied theoretical insights to properly diagnose the flaws within the intelligence community. iv Table of Contents Preface ……………………………………………………………………….1 Chapter 1: Why Intelligence Reform Failed After 9/11……………….…….13 Introduction ………………………………………………………………….15 The 9/11 Investigations ……………………………………………………...21 Three Critical Flaws of the U.S. National Security System………………….24 Issue #1: The Lack of Imagination ………………………………………….25 Issue #2: The Lack of Accountability ………………………………………..30 Issue #3: The Lack of a Domestic Intelligence Capability. …………………34 An Assessment of the post-9/11 Reform……………………………………..38 Assessing the 9/11 Investigations …………………………………… …….38 Examining Institutional Design ………………………………………………41 Defining the U.S. National Security System ……………………………...….42 Focusing on the Intelligence Community and Domestic Security……………42 The Study of Institutions ……………………………………………………..44 Defining Institutions ………………………………………………………….46 Institutions v. Organizations ………………………………………………….46 The Elements of an Institutions ……………………………………………….49 Institutionalism ………………………………………………………………..50 Applying the Theory of New Institutionalism ………………………………..51 The Elements of An Institutions: People, Structure, Culture …………………55 1. The People Operating Within the Institution ………………………………56 v 2. The Structure of the Institutions ……………………………………....…60 Centralized Structure v. Decentralized Structure …………………….…..….61 Different Structures for Different Tasks …………………………….…..…...63 Different Structures for Different Threats …………………………..………68 The Rules of an Institution ……………………………………………..……69 The Routines and Processes of an Institution ………………………….……70 3. The Culture of an Institution ………………………………………….....72 Designing Institutions ……………………………………………………….73 Why This Study? ………………………………………………………..…...75 Former Studies Focus on Individuals ……………………………………….77 Theory and Practice: The Disconnect ……………………………………….80 Academics and Policymakers: The Gap Between the Two ………………….81 Explaining the Gap …………………………………………………………..84 Academic Theories Tend to Be Imprecise …………………………………...87 Academic Theories Focus on Typical Cases ………………………………….88 Academic Theories Focus on Different Issues ……………………………….90 Theorists Have Different Jobs ………………………………………………..93 Characteristics of Policymakers May Make Theory Less Acceptable ……….94 How Theory Helps Policymakers …………………………………………….95 When Trial and Error is Too Costly …………………………………………..96 Methodology and Chapters ……………………………………………….…100 The Focus of the Study ………………………………………………………100 vi The Three Issues ……………………………………………………………102 The Methods Used in the Research …………………………………………102 The Chapters of the Study …………………………………………………..106 Audiences: Who Might Benefit From This Study ………………………….107 The Academics ………………………………………………………………107 The Policymakers ……………………………………………………………108 The General Reader ………………………………………………………….108 General Conclusions of the Study ……………………………………………109 Institutional Design of National Security Agencies ……………………….....109 Further Reform May Be Necessary ………………………………………….112 Intelligence Failures Will Likely Occur Again ……………………………...114 Academic Theories Can Help Improve Performance ……………………….114 Chapter 2: Explaining Institutional Design and Performance ……………….116 Introduction …………………………………………………………………..116 Organizational Theory ……………………………………………………….117 Political Science Theories …………………………………..……………….118 National Strategy and Institutions: The Connection …………………………120 U.S. National Security Strategy post Cold War ……………………...………124 Theory: Explaining National Security Institutions …………………………..127 Institutions Matter ……………………………………………………………130 A System Theory Approach …………………………………………………133 Normal Accident Theory …………………………………………………….134 vii An Extension of Normal Accident Theory ………………………………….145 New Institutionalism ………………………………………………….….….152 Benefits of Institutions ………………………………………………………159 The International Setting and an Institution Design ………………………...160 Institutional Design ………………………………………………………….164 Designing an Institution: The Positive and the Negative ……………………165 Social Science Theories of Culture …………………………………………..169 Difference Between the Theories on the Nature of Culture ………………….169 Differences Between the Theories on the Impact of Culture…………………172 The Study of Culture as it Relates to the Effectiveness of an Organization ....174 Towards An Institutional Theory of Culture …………………………………177 Building Upon Political Science & Organizational Theory ………………….177 A New Focus on Culture Within Political Science …………………………..179 ―New‖ Institutionalism: Progress with Culture ………………………………186 How Can Political Science & Organizational Theory Help ………………….191 Theory of Culture in a Complex System ……………………………………..193 A Case Study on Culture: The U.S. Forest Service …………………………..202 The Fit: Strategy-Design-Environment ……………………………………....204 Conclusions ……………………………………………………………..……205 Chapter 3: Institutional Innovation …………………………..…………..…..207 Introduction …………………………………………………………………..207 The Call for Imagination ……………………………………………………..211 viii Doing New Things ………………………………………………………….214 A Theory of Adaption Failure ………………………………………………216 Explaining Institutional Failure to Innovate ………………………………..222 Institutional Change: When & Why ………………………………………..226 Why Change is Not Easy or Frequent ………………………………………227 Why Change is Necessary …………………………………………………..230 Institutional Theory of Innovation …………………………………………..232 Institutional Imagination & the 9/11 Commission ………………………….233 Defining Institutional Imagination as Innovative Behavior …………………235 The Road to Institutional Innovation: A Process of Learning ……………….235 Defining Institutional Innovation as Organizational ―Sense-making‖ ………237 The Role of Institutional Structure and Culture ……………………………..244 Needs for Coordination ………………………………………………………246 The Process of Learning through Socialization ………………………………248 Defining Institutional Learning ………………………………………………250 Explaining How Socialization Leads to Institutional Learning………………255 The Institutional Locations for Innovation: ―Communities of Practice‖……..257 Defining Communities of Practice……………………………………………259 The Socialization Process …………………………………………………….260 Structural Impediments to Learning: Specialization …………………………263 Implications for researchers and Practitioners ……………………………….264 A Constructivist Perspective: Culture & Identity Within Institutions ………268 ix An Illustration of Institutional Identity & Culture: The CIA ……………….270 An Illustration of Institutional Identity & Culture: The FBI ……………….277 Flaws in Zegart‘s Assessment of Innovation ……………………………….282 An Illustration: Institutional Innovation by the CIA ………………………..285 Technical Innovations ……………………………………………………….285 Operational Innovations………………………………………..……………298 Organizational Innovations …………………………………………………305 Recognizing the Threat From al-Qaeda …………………………………….314 Conclusion …………………………………………………………………..317 Chapter 4: Accountability