Knowledge Management in the Intelligence Enterprise For a complete listing of the Artech House Information Warfare Library, turn to the back of this book. Knowledge Management in the Intelligence Enterprise Edward Waltz Artech House Boston • London www.artechhouse.com Library of Congress Cataloging-in-Publication Data Waltz, Edward. Knowledge management in the intelligence enterprise/Edward Waltz. p. cm.—(Artech House information warfare library) Includes bibliographical references and index. ISBN 1-58053-494-5 (alk. paper) 1. Military intelligence. 2. Knowledge management. 3. Military intelligence— United States. 4. Knowledge management—United States. I. Title. II. Series. UB250.W33 2003 658.4’7—dc21 2003041892 British Library Cataloguing in Publication Data Waltz, Edward Knowledge management in the intelligence enterprise. — (Artech House information warfare library) 1. Intelligence services 2. Knowledge management 3. Business intelligence I. Title 327.1’2 ISBN 1-58053-494-5 Cover design by Gary Ragaglia © 2003 ARTECH HOUSE, INC. 685 Canton Street Norwood, MA 02062 All rights reserved. Printed and bound in the United States of America. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Artech House cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark. International Standard Book Number: 1-58053-494-5 Library of Congress Catalog Card Number: 2003041892 10987654321 Dedicated to my daughter, Debbie From her wheelchair, she has shown me what the Scriptures describe as those believers who are clothed with compassion, kindness, humility, gentleness, and patience (Colossians 3:12). Contents Preface xiii 1 Knowledge Management and Intelligence 1 1.1 Knowledge in a Changing World 4 1.2 Categories of Intelligence 10 1.3 The Intelligence Disciplines and Applications 12 1.3.1 National and Military Intelligence 15 1.3.2 Business and Competitive Intelligence 17 1.4 The Intelligence Enterprise 17 1.5 The State of the Art and the State of the Intelligence Tradecraft 20 1.5.1 National and Military Intelligence 20 1.5.2 Business and Competitive Intelligence 21 1.5.3 KM 22 1.6 The Organization of This Book 23 Endnotes 24 Selected Bibliography 26 2 The Intelligence Enterprise 29 2.1 The Stakeholders of Nation-State Intelligence 29 vii viii Knowledge Management in the Intelligence Enterprise 2.2 Intelligence Processes and Products 33 2.3 Intelligence Collection Sources and Methods 35 2.3.1 HUMINT Collection 37 2.3.2 Technical Intelligence Collection 38 2.4 Collection and Process Planning 40 2.5 KM in the Intelligence Process 42 2.6 Intelligence Process Assessments and Reengineering 44 2.6.1 Balancing Collection and Analysis 45 2.6.2 Focusing Analysis-Synthesis 45 2.6.3 Balancing Analysis-Synthesis Processes 46 2.7 The Future of Intelligence 48 Endnotes 51 3 Knowledge Management Processes 55 3.1 Knowledge and Its Management 56 3.2 Tacit and Explicit Knowledge 62 3.2.1 Knowledge As Object 63 3.2.2 Knowledge As Process 68 3.2.3 Knowledge Creation Model 71 3.3 An Intelligence Use Case Spiral 74 3.3.1 The Situation 75 3.3.2 Socialization 77 3.3.3 Externalization 78 3.3.4 Combination 79 3.3.5 Internalization 79 3.3.6 Socialization 79 3.3.7 Summary 80 3.4 Taxonomy of KM 80 3.5 Intelligence As Capital 85 3.6 Intelligence Business Strategy and Models 93 3.7 Intelligence Enterprise Architecture and Applications 96 3.7.1 Customer Relationship Management 98 Contents ix 3.7.2 Supply Chain Management 98 3.7.3 Business Intelligence 100 3.8 Summary 102 Endnotes 104 4 The Knowledge-Based Intelligence Organization 107 4.1 Virtues and Disciplines of the Knowledge-Based Organization 109 4.1.1 Establishing Organizational Values and Virtues 110 4.1.2 Mapping the Structures of Organizational Knowledge 112 4.1.3 Identifying Communities of Organizational Practice 115 4.1.4 Initiating KM Projects 117 4.1.5 Communicating Tacit Knowledge by Storytelling 118 4.2 Organizational Learning 121 4.2.1 Defining and Measuring Learning 122 4.2.2 Organizational Knowledge Maturity Measurement 123 4.2.3 Learning Modes 125 4.3 Organizational Collaboration 129 4.3.1 Collaborative Culture 131 4.3.2 Collaborative Environments 133 4.3.3 Collaborative Intelligence Workflow 137 4.4 Organizational Problem Solving 142 4.4.1 Critical, Structured Thinking 143 4.4.2 Systems Thinking 147 4.4.3 Naturalistic Decision Making 150 4.5 Tradecraft: The Best Practices of Intelligence 151 4.6 Summary 153 Endnotes 153 5 Principles of Intelligence Analysis and Synthesis 159 5.1 The Basis of Analysis and Synthesis 160 5.2 The Reasoning Processes 167 5.2.1 Deductive Reasoning 167 5.2.2 Inductive Reasoning 168 x Knowledge Management in the Intelligence Enterprise 5.2.3 Abductive Reasoning 173 5.3 The Integrated Reasoning Process 175 5.4 Analysis and Synthesis As a Modeling Process 180 5.5 Intelligence Targets in Three Domains 186 5.6 Summary 190 Endnotes 191 6 The Practice of Intelligence Analysis and Synthesis 195 6.1 Intelligence Consumer Expectations 196 6.2 Analysis-Synthesis in the Intelligence Workflow 198 6.3 Applying Automation 203 6.4 The Role of the Human Analyst 205 6.5 Addressing Cognitive Shortcomings 207 6.6 Marshaling Evidence and Structuring Argumentation 209 6.6.1 Structuring Hypotheses 210 6.6.2 Marshaling Evidence and Structuring Arguments 211 6.6.3 Structured Inferential Argumentation 213 6.6.4 Inferential Networks 215 6.7 Evaluating Competing Hypotheses 223 6.8 Countering Denial and Deception 229 6.9 Summary 235 Endnotes 235 7 Knowledge Internalization and Externalization 241 7.1 Externalization and Internalization in the Intelligence Workflow 241 7.2 Storage, Query, and Retrieval Services 245 7.2.1 Data Storage 245 7.2.2 Information Retrieval 247 7.3 Cognitive (Analytic Tool) Services 249 7.4 Intelligence Production, Dissemination, and Portals 256 Contents xi 7.5 Human-Machine Information Transactions and Interfaces 264 7.5.1 Information Visualization 264 7.5.2 Analyst-Agent Interaction 265 7.6 Summary 267 Endnotes 268 8 Explicit Knowledge Capture and Combination 271 8.1 Explicit Capture, Representation, and Automated Reasoning 272 8.2 Automated Combination 275 8.2.1 Data Fusion 277 8.2.2 Data Mining 283 8.2.3 Integrated Data Fusion and Mining 288 8.3 Intelligence Modeling and Simulation 289 8.3.1 M&S for I&W 292 8.3.2 Modeling Complex Situations and Human Behavior 293 8.4 Summary 294 Endnotes 294 9 The Intelligence Enterprise Architecture 299 9.1 Intelligence Enterprise Operations 300 9.2 Describing the Enterprise Architecture 302 9.3 Architecture Design Case Study: A Small CI Enterprise 306 9.3.1 The Value Proposition 308 9.3.2 The CI Business Process 309 9.3.3 The CI Business Process Functional Flow 312 9.3.4 The CI Unit Organizational Structure and Relationships 315 9.3.5 A Typical Operational Scenario 316 9.3.6 CI System Abstraction 318 9.3.7 System and Technical Architecture Descriptions 319 9.4 Summary 322 Endnotes 324 xii Knowledge Management in the Intelligence Enterprise 10 Knowledge Management Technologies 327 10.1 Role of IT in KM 327 10.2 KM Research for National Security Applications 331 10.3 A KM Technology Roadmap 332 10.4 Key KM Technologies 335 10.4.1 Explicit Knowledge Combination Technologies 335 10.4.2 Human-Computer Tacit-Explicit Exchange Technologies 337 10.4.3 Knowledge-Based Organization Technologies 339 10.5 Summary 340 Endnotes 340 About the Author 343 Index 345 Preface In this rapidly changing and volatile world, the expectations required of those in the intelligence discipline are high—knowledge of the hidden and foreknowl- edge of the unpredictable. The consumers of intelligence—national policymak- ers, military planners, warfighters, law enforcement, business leaders—all expect accurate and timely information about the their areas of interest and threats to their security. They want strategic analyses, indications and warnings, and tacti- cal details. This book is about the application of knowledge management (KM) principles to the practice of intelligence to fulfill those consumers’ expectations. I began this manuscript shortly before the September 11, 2001, terrorist attack on the United States. Throughout the period that I was writing this manuscript, the nation was exposed to an unprecedented review of the U.S. intelligence organizations and processes; intelligence has entered our nation’s everyday vocabulary. Unfortunately, too many have reduced intelligence to a simple metaphor of “connecting the dots.” This process, it seems, appears all too simple after the fact—once you have seen the picture and you can ignore irrele- vant, contradictory, and missing dots. Real-world intelligence is not a puzzle of connecting dots; it is the hard daily work of planning operations, focusing the collection of data, and then processing the collected data for deep analysis to produce a flow of knowledge for dissemination to a wide range of consumers. From a torrent of data, real-world intelligence produces a steady stream of reli- able and actionable knowledge. Intelligence organizations have performed and refined this process to deliver knowledge long before the term knowledge man- agement became popular; today they are applying new collaborative methods and technologies to hone their tradecraft. This book focuses on those methods and technologies.