CRITICAL INFRASTRUCTURE ASSET IDENTIFICATION: a MULTI-CRITERIA DECISION SYSTEM and AVIATION CASE STUDY by CHRISTINE OGECHUKWU IZUAKOR B.S

CRITICAL INFRASTRUCTURE ASSET IDENTIFICATION: A MULTI-CRITERIA DECISION SYSTEM AND AVIATION CASE STUDY by CHRISTINE OGECHUKWU IZUAKOR B.S. DeVry University, 2010 M.S. University of Houston, 2012

A dissertation submitted to the Graduate Faculty of the University of Colorado Colorado Springs In partial fulfillment of Requirements for the degree of Doctor of Philosophy Department of Computer Science 2016

This dissertation for the Doctor of Philosophy Degree by Christine Ogechukwu Izuakor has been approved for the Department of Computer Science by

Edward Chow, Chair

Richard White, Co-Chair

Terry Boult

Bill Ayen

Chris Bronk

Date: ______

Izuakor, Christine Ogechukwu (Ph.D. Engineering – Security) Critical Infrastructure Asset Identification: A Multi-Criteria Decision System and Aviation Case Study Dissertation directed by Professor Edward Chow and Assistant Research Professor Richard White

ABSTRACT In a world where terrorism, natural disasters, and unknown unknowns threaten the security posture of various nations, comprehensive risk management remains the recommended action for protection of critical infrastructure. Furthermore, the threat landscape is rapidly evolving as critical infrastructures more heavily rely on cyber technology and are exposed to greater cyber risks. In response, effective risk management first requires a fundamental understanding of which assets are critical.

Today, countries struggle with efforts to identify these critical assets making it difficult to accurately assess risks and allocate scarce resources to protect those assets of the greatest criticality and risk.

Currently, there is not a clearly defined best practice methodology for identifying critical infrastructure. Through this research we have found that it is infeasible to even attept to estalish a oe size fits all ethod fo doig so. Istead, e eoed that critical asset identification be viewed as an objective-based decision process, where decision makers can establish tailored methodology for identifying assets based on specific objectives.

iii

The major contributions of this research are: 1) A complete, reproducible, documented, and defensible decision system (CIAid) that can be used to develop a critical infrastructure asset identification methodology by following an objective framework, 2) A methodology comparison function that leverages Technique For Order

Preference by Similarity to Ideal Solution (TOPSIS) multi-criteria decision making methodology to compare alternative methods where user constraints will not support adoption of the originally recommended CIAid methodology, 3) An aviation case study inclusive of aviation criteria analysis, expert survey validation, a sample aviation asset dataset, and an example tiered critical aviation asset list, 4) A process for selecting and applying TOPSIS to a critical infrastructure problem.

Using the CIAid tool, the Department of Homeland Security and comparable international organizations can use their own unique objectives to create customized critical infrastructure asset identification methodologies. Within the aviation sector, this research can also be used to strengthen the identification and protection of critical aviation assets.

DEDICATION

In loving memory of my dearest little cousin. Thank you for being my light. You are everything. Long live Chikosolu Ughanze.

August 1994 – April 2015

ACKNOWLEDGEMENTS

This journey has been an exciting rollercoaster of stimulating challenges, joyful discoveries, and insightful failures. I did not walk this path alone, and truly appreciate the support I have received from so many friends, family, colleagues, and associates.

First, I would like to thank my parents and Uncle Matt for motivating me to reach beyond the stars and be the best that I can be. Thanks mommy for being my number one cheerleader and prayer warrior along the way. Thanks to my wonderful siblings

Chika, Nk, and Obi for being my best friends and inspiring me every day. Thank you to my amazing cousins, uncles, and aunts. I wish I could name each and every one of you from Australia to Houston. You all mean the world to me and I appreciate you. Gech,

Tema, Bree, Tranette, Brax, CJ, Ian, and many other countless epic friends, thank you for being here for me.

I am also blessed with a wonderful team at work. Thanks to all of my colleagues and professional network for your unending support and participation in surveys, discussions, paper editing, and more. A special thanks to Mary Hickey, Dan McSweeney,

Sergei Vasilevsky, Faye Francy, and Patrick Mana for spending time to review and validate my research. Thanks to Andrea Webster for connecting me to so many helpful resources.

Endless thanks to my esteemed committee for challenging and preparing me to execute and deliver on my research goals. A special thanks to Dr. Chow for believing in me when I expressed interest in beginning this journey, and thanks to both Dr. Chow

vi and Dr. White for spending countless hours guiding me through the entire process. I

at thak ou each enough for your feedback and support.

ALL PRAISE AND GLORY TO GOD.

vii

TABLE OF CONTENTS

CHAPTER

1 INTRODUCTION ...... 1

Background and Motivation ...... 1

A Wa Foad… ...... 4

Research Scope ...... 4

Dissertation Outline ...... 5

2 LITERATURE REVIEW AND PROBLEM ANALYSIS ...... 8

Background on Critical Infrastructure Protection ...... 8

Limitations of Current Identification Methods ...... 28

Solution Exploration ...... 35

3 RESEARCH METHDOLOGY ...... 50

Required Tasks ...... 50

Success Criteria ...... 55

Validation Scoring ...... 56

Key Assumptions and Limitations ...... 56

4 INTRODUCTION AND SELECTION OF MCDM METHODOLOGY FOR APPLICATION TO CRITICAL INFRASTRUCTURE ASSET IDENTIFICATION ...... 58

Background ...... 59

Selection Methodology ...... 62 viii

Selection Process ...... 68

Selected Method: TOPSIS ...... 73

5 DEVELOPMENT OF A CRITICAL INFRASTRUCTURE ASSET IDENTIFICATION OBJECTIVE-BASED FRAMEWORK ...... 83

Input Sources and Data Gathering ...... 86

Objective Profiles ...... 88

Conclusion ...... 116

6 SYSTEM ENGINEERING FOR USER-FRIENDLY CIAID TOOL ...... 117

System Engineering Process ...... 118

System Overview ...... 119

System Lifecycle: Concept Development ...... 120

System Lifecycle: Engineering Development ...... 121

7 CIAID VALIDATION, VERIFICATION, AND EVALUATION ...... 149

Aviation Criteria Expert Verification ...... 149

CIAid Verification and Validation Interviews ...... 151

Success Criteria Evaluation...... 154

8 AVIATION CASE STUDY ...... 156

Background ...... 157

Key Cyber Issues in Aviation ...... 160

Illuminating a Fundamental Gap ...... 167

Using CIAid and Sector Specific Criteria to Lessen the Gap ...... 170

Building an Aviation Asset Dataset ...... 185

Example Critical Aviation Asset List ...... 189

Key Conclusions ...... 192

9 RESEARCH SUMMARY AND CONCLUSION ...... 194

Contributions to the CIP and Aviation Industries ...... 194

Future Research ...... 202

Conclusion ...... 208

10 REFERENCES ...... 210

11 APPENDIX ...... 222

LIST OF TABLES

TABLE

Table 1: PPD-21 Infrastructure Sectors [7] ...... 9

Table 2: CIP Law, Directives, Strategies, & Plans ...... 14

Table 3: 2003 "Out-of-Place" Assets [20, p. 11] ...... 15

Table 4: Basic Elements of Criticality Assessment Methodology [27] ...... 20

Table 5: Related Search Results of IEEE Papers ...... 22

Table 6: Methodology Classifications ...... 28

Table 7: Explanation of Success Criteria ...... 56

Table 8: Validation Scoring Scheme ...... 56

Table 9: MCDM Challenge Characteristics [64] ...... 59

Table 10: MCDM Selection Methodologies ...... 63

Table 11: Examples of MCDM Types Evaluated ...... 71

Table 12: MCDM Selection Choices for Evaluation Categories ...... 72

Table 13: TOPSIS Example Step 1 - Matrix ...... 77

Table 14: TOPSIS Example Step 2 - Normalized Matrix ...... 78

Table 15: TOPSIS Example Step 3 - Weighted Normalized Matrix ...... 78

Table 16: TOPSIS Example Step 4: PIS and NIS Determination ...... 79

Table 17: TOPSIS Example Step 5 - Distance Matrix Establishment ...... 79

Table 18: TOPSIS Example Step 6 - Similarity to NIS Calculation ...... 80

Table 19: TOPSIS Example Step 7 - Best Alternative ...... 80

Table 20: Framework Components to Support Process ...... 85 xi

Table 21: Objectives Adapted from Global CI Definitions [10] ...... 86

Table 22: Example Criteria Categories Adapted from Global CI Definitions [10] ...... 88

Table 23: Example Database Listing for Individual Objective ...... 123

Table 24: Objective Solicitation Question...... 124

Table 25: TOPSIS Scoring Scheme ...... 128

Table 26: TOPSIS Example - User Ratings ...... 133

Table 27: TOPSIS Example- Normalized and Weighted Matrix ...... 135

Table 28: TOPSIS Example - NIS and PIS ...... 136

Table 29: TOPSIS Examples - Distance Calculations...... 137

Table 30:TOPSIS Example - Distance Similarity Calculation...... 139

Table 31:TOPSIS Example - Distance Calculation with Custom Offset ...... 140

Table 32: Validation Interview Plan ...... 152

Table 33: Expert Validation Interview Scores ...... 154

Table 34: Type of Aeronautical functions serving public interest [94, p. 2] ...... 159

Table 35: Sample Theoretical Cyber Threats Applicable to Aviation Sector ...... 165

Table 36: Aviation Sector Assets ...... 168

Table 37: Transportation Sector Mobile Assets [105, pp. 105-119] ...... 169

Table 38: Brainstormed Aviation Criteria Options ...... 173

Table 39: Aircraft Types Involved in Terrorism Incidents ...... 179

Table 40: Example Tier One Critical Airports Based on Enplanements ...... 182

Table 41: Airline Operating Revenue as Percentage of Aviation GDP [110] ...... 184

Table 42: Sample Aviation Dataset- Airlines ...... 186

xii

Table 43:Aviation Criteria Used for Developing Example Aviation CI List ...... 190

Table 44:Example Tier-1 Aviation CI Asset List ...... 191

Table 45: Accepted Research Publications ...... 200

Table 46: Papers Currently Under Review for Publication ...... 200

xiii

LIST OF FIGURES

FIGURE

Figure 1: CI Asset Identification Methodology Components ...... 20

Figure 2: National Infrastructure Protection Plan Risk Management Framework [4] ..... 37

Figure 3: High-level Identification Decision Factors ...... 39

Figure 4: Dimensions for describing infrastructure dependencies [45] ...... 44

Figure 5: CIAid Framework Map ...... 53

Figure 6: Types of MCDM applied to Infrastructure Management [64] ...... 62

Figure 7: Expert Recommended MCDM Selection Process [68] ...... 64

Figure 8: Customized MCDM Selection Method for Critical Infrastructure Problem ...... 65

Figure 9: Identification Process...... 84

Figure 10: Core Objective Areas in Framework ...... 88

Figure 11: Scoping Logic Diagram ...... 90

Figure 12: Political Accommodation Logic Diagram ...... 92

Figure 13: Asset Focus Logic Diagram ...... 95

Figure 14: Impact Type Logic Diagram ...... 97

Figure 15: Impact Measurement Logic Diagram...... 98

Figure 16: Impact Measurement Logic Diagram continued ...... 99

Figure 17: Stakeholder Logic Diagram ...... 109

Figure 18: System Life Cycle Model [85] ...... 118

Figure 19: End to End CIAid Process ...... 119

Figure 20: CIAid Functional Definition ...... 123 xiv

Figure 21: CIAid Homepage ...... 142

Figure 22:Homepage Macro ...... 142

Figure 23: CIAid User Objective Solicitation ...... 143

Figure 24: CIAid User Objective Solicitation cont...... 144

Figure 25: CIAid Objective Solicitation Macro 1 ...... 144

Figure 26:CIAid Objective Solicitation Macro 2 ...... 145

Figure 27: CIAid Recommendations GUI ...... 145

Figure 28: Recommendations Macro...... 146

Figure 29: CIAid User Alternative Selections ...... 146

Figure 30: Alternative Comparison Macro ...... 147

Figure 31: CIAid Tool Output ...... 147

Figure 32: Layered Approach to Aviation Security [89, p. 143] ...... 168

Figure 33: Geographical Distribution of Responses...... 174

Figure 34: Expert Criteria Ratings ...... 175

Figure 35: Regulation of Airline Participation in Identification Efforts ...... 176

Figure 36: RAND Database Filtering ...... 178

Figure 37: Average Fatalities and Capacity per Aircraft Type ...... 180

Figure 38: Annual Airport Enplanement Trends ...... 183

Figure 39: Aviation Global GDP Impact in 2014 [91] ...... 184

1 CHAPTER I

INTRODUCTION

Background and Motivation

In 2001, hijackers exploited physical airport security vulnerabilities in an elaborate and successful attempt to kill masses, disrupting society for years to come.

Today, e-enabled aircraft and the increasing reliance of aviation infrastructure on cyber technology offer a new and intriguing attack vector for malicious intenders to target and potentially exploit in ways far greater than what was seen during 9/11. Since that unforgettable day in American history, we have been left questioning: What is critical?

Without which assets can we no longer function as a society? Today, these questions are of ever greater concern in this current threat landscape plagued with heightening terrorist attacks and growing cyber risks.

Critical infrastructure (CI) asset identification is a fundamental component of national risk management and a paramount piece to answering these questions. While growing threats and hazards have increased the need for better protection of infrastructure, budgetary constraints and resource limitations make it impractical to protect every single asset. Thus, the effective identification of the most critical assets allows protection programs to prioritize asset lists and invest scarce resources

1 appropriately. Detailed risk assessment can then be limited to those assets, for eaple, that if disupted ould hae deilitatig ipat o seuit, atioal economic security, national public health and safety, or any combination of those

attes [1].

Despite more than a decade of effort in the U.S., both internal and external audits reveal fundamental problems with even the basic task of identifying CI. The development of multiple databases and contentions over selection criteria leave our opeig uestio aguel aseed ad aises a additioal uestio: If e at idetif hats itial, ho do e ko if ee iestig sae atioal esources to protect the right infrastructure? This challenge is not unique to the U.S., as many countries around the world also struggle with identification of critical infrastructure. Its a global challenge.

Furthermore, as cyber-attacks grow in size and severity, threatening critical infrastructure through the Internet, we are forced to rethink the limitations previously placed on asset types included in the scope of identification efforts. For example, the current U.S. programs fous o fied assets ignores an asset class that is growingly susceptible to attack or weaponization: mobile assets. The exploitation of the aviation sector during 9/11 is a primary example of how mobile assets can be weaponized and used to invoke mass terror. Additionally, considering the increasing cyber connectivity of aircraft today we find that the sole focus on protecting fixed aviation asset (airports) may no longer suffice. The omission of mobile assets creates a fundamental gap in the risk management process that must be addressed in order to ensure that critical 2 infrastructure protection efforts are complete and also help improve risk management and resilience.

Existing approaches that can be leveraged for identifying critical infrastructure have been established by researchers, public sector agencies, and private sector corporations and are further discussed throughout this report. Generally, these programs follow a common fundamental process. The scope of the effort is established, followed by selection of an approach or methodology for gathering the initial asset list.

Finally, appropriate criteria are identified and applied against the asset list to determine which assets are meet criteria and are considered critical.

Though most of the programs follow this theme, there are variances in the methods used during each step in the process. For example, in the US, a consequence- based approach is used with 2-tier measurement thresholds for each of four consequence criteria categories. The asset scope is limited to fixed assets. The European program, on the other hand, considers the consequence measurements as just a subset of the identification methodology. Supplementary criteria such as proximity and capacity of assets are also considered in the EU program. Suffice it to say, the options available for criteria development and assessment scoping are abundant, and though these programs tend to have common end goals, their organization and execution processes differ. This leads to our main question: Based on stakeholder objectives, which identification method should be used to identify critical infrastructure asset? The basic answer to this question is that the most feasible methodology than can be used for identification efforts ultimately depends on the objectives and capabilities of the 3 organization conducting the assessment. These objectives may evolve over time, and the methodology used should be adaptable to those new changes.

A Way Forward…

In consideration of the numerous assessment options available and varying objectives across programs, we believe that critical asset identification should be viewed as an objective-based decision process, where decision makers can establish tailored methodology for identifying assets based on specific objectives. The multi-criteria and multi-objective nature of the problem make Multi-Criteria Decision Making

Methodology (MCDM), a subset of the Decision Theory domain, a suitable base to the solution. MCDM provides a multi-level, multi-attribute or multi-objective framework for scoring and ranking alternative decision options against multiple criteria [2]. The process includes identification of objectives, criteria, alternatives, a weighting method, aggregation, and ultimately, a decision. Successful applications of this methodology are growing across many industries, including the infrastructure management industry [3].

Through this research, we apply our own tailored version of the Technique or Order

Preference by Similarity to Ideal Solution (TOPSIS) MCDM method to establish a scope and approach, and select criteria that meets objective requirements of the decision maker. The end result is a simplified and logical decision making system.

Research Scope

While this research focuses heavily on the U.S, the greater intent of this effort is to provide a general decision system for use by any nation seeking a starting point to 4 developing an asset identification program. In addition, we provide a case study limited to the aviation sector in order to showcase the integration of sector specific criteria. This research does not aim to result in a list of prioritized critical assets based on risk, but instead provides means for identifying the most important assets that require additional risk assessment. We recognize that prioritization takes into account threat, vulnerability, and likelihood of occurrence which are measured and evaluated later in the risk assessment process. As such, we reduce the scope of this research to focus solely on identification, not risk-based prioritization.

Dissertation Outline

Chapter II begins with a detailed background on critical infrastructure protection.

Global policy frameworks are explored prior to focusing in on the current state of affairs in the United States. A review of progress leading up to the current state is described along with methodology requirements necessary for effective identification of assets.

This is followed by a review of existing identification methodologies and limitations of such methods. A deeper look at limitations and challenges specific to the U.S. program is then shared. Finally, the path to a solution is introduced. Emerging research and multi- criteria decision making are discussed, as well as, their applicability to this research problem. Chapter III explains the dissertation research methodology including all required research tasks, the success criteria, and the validation scoring requirements.

Chapter IV provides a more detailed introduction to MCDM and describes the process for selecting the appropriate MCDM methodology for application to this

5 research. The chapter progresses with a description of existing selection methods used by other researchers before introducing a customized version developed specifically for this effort. The process of developing requirements and evaluating the alternatives is noted, and the most applicable method disclosed (TOPSIS). Background on the TOPSIS method is then provided, followed by a description of methodology steps, a simplified example, and a discussion of the methods strengths and weaknesses.

Chapter V introduces the objective framework which serves as the foundation of the model developed as a result of this research. The section shares the process of data gathering and information sources, followed by a description of the main objective profiles that make up the framework: Scope, Political Accommodation, CI Definition,

Stakeholder, and Sector Identification.

Chapter VI details the system engineering process from development to delivery of a user-friendly model. The chapter starts with a system overview of the end to end process. The steps leading up to the final product are then shared. User and needs analysis are covered first. The process for developing the database, based on the objective framework, is described and finally the system model introduced.

Chapter VII provides the plan and results for validation, verification, and evaluation of the body of research. This includes the results of an online survey completed by 60+ security and aviation industry experts, as well as a series of one on one interviews held with a smaller subset of industry experts to perform a more critical review of the research.

Chapter VIII completes the core research tasks with an aviation case study. The study begins with a background on the aviation sector and key cyber security issues concerning the sector. The fundamental gap of preoccupation with fixed assets in the

U.S. program is then discussed. Further, the contribution of this research is illustrated using the aviation sector as an example. In this section, aviation criteria are developed and the results of an expert input survey shared. Using the survey insight, conclusions and criteria recommendations are drawn and shared. To further illustrate the contribution of the research, a sample aviation dataset and the process of its development is shared. The dataset is then run against recommended criteria and an example critical aviation asset list provided. The real-world application of a database and process of this nature is also discussed.

Chapter IX summarizes the original contributions of this research and then creates vision for future research including: expansion and maintenance of the objective framework, expansion of sector specific criteria development, growing the aviation asset dataset and building a real-world database, and advancement of the model and graphical user interface. The topics of automation, optimization, and adoption are also covered here. In closing, the last section provides key conclusions and a final summation of the research.

2 CHAPTER II

LITERATURE REVIEW AND PROBLEM ANALYSIS

Background on Critical Infrastructure Protection

The need for critical infrastructure (CI) asset identification was introduced through presidential directives, acts, and plans developed to guide CI protection initiatives within the United States. Other nations have taken on similar efforts to protect their own CI. This section provides an overview of program efforts in the US,

Europe, and other nations.

2.1.1 Policy Frameworks

2.1.1.1 NIPP

The National Infrastructure Protection Plan (NIPP) is the US federal government guide for risk management of CI [4]. The development of the NIPP was influenced by several directives, strategies, and policies [5]. The 2002 Homeland Security Act [6] mandated development of a CI risk management program. After several drafts, the first

NIPP was issued in 2006. Stemming as it did from the attacks of September 11, 2001, the first NIPP focused on managing CI risk from terrorist attack. As a result of Hurricane

Katia, the NIPP as updated i to iopoate a all-hazads appoah to CI risk management. The NIPP was again revised in 2013 to emphasize the current adiistatios pioit o esiliee as atiulated i PPD-21 [7]. The current plan broadly classifies sixteen CI sectors as listed in Table 1.

• Chemical • Food and Agriculture • Commercial • Government Facilities Facilities • Communications • Healthcare and • Critical Public Health Manufacturing • Information • Dams Technology • Defense Industrial • Nuclear Reactors, Base Materials, and • Emergency Services Waste • Energy • Transportation • Financial Services Systems • Water and Wasterwater Systems Table 1: PPD-21 Infrastructure Sectors [7]

At the heart of the NIPP is a five-step CI Risk Management Framework (RMF).

The essential purpose of the RMF is to assess and prioritize CI risk as a product of threat, vulnerability, and consequence. Step Two in the RMF process is identification of CI. This step sets the foundation for evaluating risks and prioritizing asset protection efforts, making the quality of information produced at this stage critical to the effectiveness of the entire program. The Department of Homeland Security (DHS) Office of

Infrastructure Protection (IP) is responsible for CI asset identification under the National

Critical Infrastructure Prioritization Program (NCIPP).

2.1.1.2 EPCIP

The European Programme for Critical Infrastructure Protection (EPCIP) provides guidance for CI risk management in Europe. The program is meant to fulfill the

euieets set foth Dietie //EC on the identification and designation of European critical infrastructures and the assessment of the need to improve their potetio [8]. The program scope is limited to the transportation and energy sectors, and calls for all-hazard consideration in CI protection efforts. While the methodology is said to incorporate an all-hazard approach including man-made, technological, and natural hazards, terrorist threats are given priority [9].

The EPCIP program phases include identification, designation, and protection of

European CI. During the Identification Phase, potential CI assets are filtered through a five-step process that consists of applying sectoral criteria, cross-cutting criteria, cross- border considerations, candidacy nomination, and final selection [9]. Similar to the

NIPP/RMF, CI asset identification lays the foundation for all subsequent phases of EPCIP.

Thus, the success of the risk management process is again dependent on the quality of output from the CI asset identification effort.

2.1.1.3 Others

CI protection is an important component of national security for other countries as well. A breadth of information can be found at cipedia.eu [10]. Notable examples include the Australian National Strategy for Critical Infrastructure Protection [11] and the Canadian Strategy for the Protection of National Critical Infrastructure [12]. The

Australia poli ad pla ais to addess all-hazads ad defies CI as, Those physical facilities, supply chains, information technologies and communication networks, which if destroyed, degraded or rendered unavailable for an extended period, would significantly impact on the social or economic wellbeing of the nation, or affect

Austalias ailit to odut atioal defese ad esue atioal seuit. “etos include Banking and Finance, Health, Food, Transport, Energy, Communications, and

Water. The Canadian version includes those sectors in addition to Safety,

Maufatuig, ad Goeet ad defies CI as, poesses, sstes, failities, technologies, networks, assets and services essential to the health, safety, security or economic well-being of Caadias ad the effetie futioig of goeet.

It is clear that identification and protection of critical infrastructure assets is relevant to multiple countries. Just as the CI definitions and associated CI sectors vary between countries and sectors, the methodology used to identify these assets also varies.

2.1.2 Critical Infrastructure Protection in the US

Critical infrastructure protection (CIP) was a concern long before 9/11. CIP was a central concern of civil defense during the Cold War from 1947 to 1991. Following the collapse of the Soviet Union, critical infrastructure protection slowly transformed from a matter of national security to one of homeland security. The change was precipitated by a growing trend in asymmetric attacks by non-state actors, including the 1993 attack on the World Trade Center, 1995 Oklahoma City Bombing, and the 1996 bombing of

Khobar Tower US military barracks in Saudi Arabia. These attacks prompted the Clinton

Administration to launch an investigation into the vulnerability of US critical ifastutue. Folloig the epot the Pesidets Coissio o Citial

Infrastructure, in 1998 President Clinton issued Presidential Decision Directive #63

(PDD-63) outlining a blueprint for critical infrastructure protection that holds to this day.

PDD-63 classified critical infrastructure into twelve sectors and assigned a lead agency within the federal government to work with industry representatives from each sector and develop a National Infrastructure Protection Plan [13].

On September 11th, , ietee hijakes sueted the atios transportation infrastructure turning passenger jets into guided missiles to inflict as much damage as the Imperial Japanese Navy on December 7th, 1941. The 9/11

Coissio ‘epot oted the supassig dispopotio of the attaks ad aked critical infrastructure protection together with containing chemical, biological, radiological, and nuclear agents to preclude further asymmetric attacks [14, p. 339].

Executive Order 13231, issued in October 2001 within weeks of 9/11, underscored the heightened concern over protecting critical infrastructure by reinforcing policies addressed previously in PDD-63 [15]. Critical infrastructure protection became law and was made a specific mission of the newly created Department of Homeland Security under the Homeland Security Act signed in November 2002. Homeland Security

Presidential Decision Directive #7 issued in December 2003 established federal guidance for protecting critical infrastructure much the same as it was specified in PDD-63, except

12 it gave greater emphasis to physical security and expanded the number of infrastructure sectors from twelve to eighteen [16].

Development of a corresponding NIPP proceeded slowly. An essential point of concern was and still is that while critical infrastructure protection is the responsibility of the public sector, a significant portion of the critical infrastructure (CI) is owned by the private sector. This makes collaboration between the two groups imperative. The

2003 National Strategy for the Physical Protection of Critical Infrastructures and Key

Assets was a forerunner to the NIPP, primarily making the case for a risk management approach through public/private partnerships [17]. Two drafts of the NIPP were circulated for industry comment in 2005, culminating in the first official release in 2006

[18, p. 21]. The 2006 NIPP endorsed a public/private partnership approach to risk management built around a Risk Management Framework (RMF). The RMF was comprised of six steps to 1) set goals, 2) identify assets, 3) assess risk, 4) prioritize countermeasures, 5) implement countermeasures, and 6) measure effectiveness [19, p.

4]. The Obama administration revised the NIPP in 2009 to expand on all-hazard and resiliency concepts, otherwise leaving it mostly unchanged [18, p. 22]. However, with increasingly more virulent cyber-attacks grabbing headlines, the Obama administration issued a second revised NIPP in 2013. The 2013 NIPP reduced the RMF from six to five steps and placed greater emphasis on cyber security and physical resilience addressed in

Presidential Policy Directive #21 (PDD-21) [4, p. 4]. Like the Homeland Security

Presidential Directive7 (HSPD-7) before it, PPD-21 adhered closely to the PDD-63 blueprint, except it reduced the number of infrastructure sectors from eighteen to

13 sixteen as listed in Table 1. These sectors provide the basis for the NIPP public/private partnerships overseeing the RMF for critical infrastructure protection. Relevant government documents that have shaped CIP in the U.S. are summarized in Table 2.

Law Directive Strategy Plan 2002 HSA 1998 PDD-63 2002 NSHS 2005 Interim NIPP 2003 HSPD-7 2003 CIP Strategy 2005 Draft NIPP 2013 PPD-21 2007 NSHS 2006 NIPP 2010 NSS 2009 NIPP 2015 NSS 2013 NIPP

Table 2: CIP Law, Directives, Strategies, & Plans

2.1.3 Cataloging Critical Infrastructure

While DHS worked to develop the National Infrastructure Protection Plan, it udetook Opeatio Liet “hield to atalog the atios itial ifastutue i advance of the US invasion of Iraq. Over the summer of 2003, DHS personnel cataloged

160 assets across various sectors it determined needed additional protection or mitigation against potential attack. Under pressure from Congress the list was expanded to 1,849 assets. DHS called it the Protected Measures Target List (PMTL) [20, p. 6].

At the same time, it was conducting Operation Liberty Shield, DHS issued a grant asking states to conduct a critical infrastructure self-assessment. The resulting data added another 26,359 to the PMTL. A review of the data determined that it was not representative of the critical infrastructure sectors [20, p. 6]. Furthermore, the state data iluded hat seeed like out-of-plae assets iludig zoos, festials, shoppig centers, and other examples shown in Table 3 [20, p. 3]. The dubious results of the 2003

14 data all ee attiuted to iial guidae gie to the states i the folloig fo: I seletig ifastutue ou should oside any system or asset that if attaked ould esult i atastophi loss of life ad/o atastophi eooi loss [20, p. 8].

Old MaDoalds pettig zoo Mall at Sears Bean Fest Nis Chek Cashig Amer. Society of Young Musicians Trees of Mystery Car Dealerships Kennel Club and Poker Room Historical Bok Sanctuary 4 Cs Fuel and Lube DPW Landfill Kangaroo Conservation Center Assyrian American Association [state] Right to Life Committee Association for the Jewish Blind [university] Insect Zoo Bourbon Festival Theological Seminary Jas “potig Goods Nestle Purina Pet Food Plant Auto Shop Veterinary Clinic Groundhog Zoo Sweetwater Flea Market High Stakes Bingo Petting Zoo [state] Community College [a] Restaurant Frontier Fun Park [a] Travel Stop Mule Day Parade Beach at End of [a] Street Amish Country Popcorn [a] Pepper and Herb Company Table 3: 2003 "Out-of-Place" Assets [20, p. 11]

In July 2004, DHS issued a second data call to correct the problems from the

2003 data call. The 2004 data call included more precise instructions in the form of

Guidelines for Identifying National Level Critical Infrastructure and Key Resources. The guidelines included more categories, subcategories, and accompanying parameters for idetifig itial ifastutue. Fo eaple, the listed ajo akig ad fiaial

etes, efieies ith efiig apait i eess of , aels pe da,

pia edial ae failities ith uiue seies, IT sstes ith aess o otol poit distiuted o oth oasts ad thoughout the out, ad oeial etes with potential economic loss impact of $10 billion or capacity of more than 35,000

15 idiiduals [20, p. 8]. States responded by submitting 47,701 additional assets to the

PMTL.

The results from the 2004 survey were combined with the 2003 data call and the

Operation Liberty Shield data to form what DHS called the National Asset Database

(NADB). The NADB tallied 77,069 assets. While DHS went to considerable efforts to verify the data, even eliminating 3,846 duplicate submissions between the 2003 and

2004 data calls, the results still contained an abundance of unusual or out-of-place assets [20, pp. 8, 9]. An audit by the DHS Inspector General noted that the NADB contained 4,055 malls, shopping centers, and retail outlets; 224 racetracks; 539 theme parks; 1,305 casinos; 234 retail stores; 514 religious meeting places; 127 gas stations;

130 libraries; 4,164 educational facilities; 217 railroad bridges; and 335 petroleum pipelines [21, pp. 2-3]. Aodig to the DH“ Ispeto Geeal, Out-of-place assets make resource allocation decisions more challenging; every possible target is not going to ise to the leel of atioal sigifiae… haig oe assets a osue desied data, akig suh pioitizatios oe diffiult [20, p. 10]. The Undersecretary for

Preparedness responded to the IG report saying DHS did not intend to have a single definitive prioritized list of critical assets and that it would not be possible or useful to develop one [18, p. 26].

Congress responded to the situation by passing the Implementing

Recommendations of the 9/11 Commission Act of 2007. The resulting law mandated creation of two databases. The first, referring to the DHS National Asset Database

(NADB), ataloged assets hose loss, iteuptio, iapait, o destutio ould 16 have a negative or debilitating effect on the economic security, public health, or safety of the Uited “tates. The seod, alled the Pioitized Citial Ifastutue List (PCIL),

ould e a lassified suset of NADB assets that if destoed o disupted, ould ause

atioal o egioal atastophi effets. The la futhe euied that DH“ epot the contents of the prioritized list to Congress annually [22, p. Sec. 1001].

DHS complied with Congress by acquiring the Infrastructure Information

Collection System (IICS). One component of the IICS was the Infrastructure Data

Warehouse (IDW). The IDW was designed to replace the NADB with an online query system making all data available to relevant public/private partners. Subsequently, the

NADB was retired effective September 2006. [23, pp. 3, 6] IDW functionality was apparently subsumed under IICS which today is accessible from the DHS Infrastructure

Protection Gateway (IP Gateway) as part of the Critical Infrastructure Technology and

Architecture (CITA) [24].

To opl ith Cogess seod euieet fo a Prioritized Critical

Infrastructure List, in 2006 DHS initiated the National Critical Infrastructure Prioritization

Program (NCIPP). The NCIPP works with states and other partners to identify and classify critical infrastructure as either level 1 or level 2 based on the consequences assoiated ith the assets disuptio o destutio. Classifiatio is ased o epeted fatalities, economic loss, mass evacuation length, and degradation of national security.

According to DHS, the overwhelming majority of the assets and systems identified through NCIPP are categorized as level 2. Only a small subset of assets meets the level 1 consequence threshold – those whose loss or damage could result in major national or 17 regional impacts similar to the impacts of Hurricane Katrina or the 9/11 attacks [25, p.

4]. Today, the National Critical Infrastructure Prioritization Program is the primary program for prioritizing critical infrastructure at the national level [4, p. 17] [5, p. 26].

But all is not well.

2.1.4 Identification Methodology Requirements

In order to better understand the challenge, we sought requirements for an effective CI asset identification methodology. The findings can be summarized in two parts: Qualitative Requirements and Quantitative Requirements.

2.1.4.1 Qualitative Requirements

Qualitative requirements refer to the soft elements required to develop methodology. For example, in 2013 the US Government Accountability Office (GAO) investigated Congressional concerns about changes to the CI asset identification methods employed by DHS/NCIPP. The 2013 GAO report [26] cited four criteria as necessary for identifying CI assets that support comparison of risk results across infrastructure sectors. The four criteria, as specified in the 2009 NIPP are 1) completeness, 2) reproducibility, 3) documentation, and 4) defensibility. These criteria have been used by numerous researchers in the evaluation of existing methodologies.

Copleteess eas the ethodolog is ssteati in examining each and every relevant asset within the set of 16 CI sectors. CI asset identification methodology is incomplete if it does not consider all potential candidates within a given set of assets.

‘epoduiilit means that results are consistent, simple, and precise to enable risk 18 comparisons between assets across different sectors. Complexity and ambiguity work against reproduiilit. Douetatio is a record of hat ifoatio is used ad how it is synthesized to geeate a isk estiate, o i this eseah ho it is used to arrive at a critical designation. Defesiilit eas that the ethodolog akes use of the professional disciplines relevant to the analysis, as well as being free from significant errors or omissions [4]. Aothe a of osideig defesiilit is to thik of alidatio ad eifiatio. I othe ods, ae e doig the ight thig, ad are we doig it ight? I ode to ase this uestio, e should defie the ight thig. I the otet of CI potetio, oe ase is that e at to keep CI fo causing catastrophic damage either through its subversion, disruption, or destruction.

From this pespetie, atastophi daage is the pia oe, ad the ight thig is peetig it fo happeig. In order to do this right, we must effectively identify CI.

2.1.4.2 Quantitative Requirements

On the other hand, quantitative criteria refer to the hard elements applied to CI asset identification. For example, Table 4 summarizes the basic components of criticality assessment methodology outlined by researchers at Central Queensland University. In

Figure 1, these quantitative requirements are further translated into a framework that aligns with CI asset idetifiatio. Its ipotat to ote that these euieets contribute immensely to the completion of the qualitative requirements outlined above and are expanded in Chapter 5.

Methodology Description Element 1. Asset A means of identifying and representing Identification assets for the purpose of criticality analysis 2. Criteria A list of factors against which asset criticality could be measured 3. Weighted A means of allocating scores to achieve a Scoring total score indicating asset criticality 4. Scoring Templates for the application of scoring Guides against each of the identified criteria 5. Application A means of applying the scoring to the criteria within an organization Table 4: Basic Elements of Criticality Assessment Methodology [27]

Scope

Systematic Unsystematic

Approach

Function-based Network-based Logic-based

Evaluation Method Criteria Selection Application Method

Figure 1: CI Asset Identification Methodology Components

2.1.4.2.1 Asset Identification Scope and Approach

Every methodology should set a specific scope and approach to the initial assessment coupled with an evaluation method. The scope of the assessment can be systematic or unsystematic based on the objective and needs of the organization.

Systematic methods take on a comprehensive approach to identification of assets and carry out a complete evaluation of the asset environment including relationships between the assets. Unsystematic methods take an individual asset level approach and do not necessarily consider all assets comprising a complete set. 20

The approach of the assessment refers to the process of establishing an initial asset baseline, and can then either be categorized as function-based, network-based, or logic-based. Function-based approaches, also referred to as mission-based approaches, begin the identification process by first identifying the functions that are critical to the mission of the organization. Assets that support those functions are then identified and evaluated against other defined criteria. Network-based approaches identify all nodes and relationships in the system and use that system mapping as a basis for the evaluation. In logic-based approaches, assets ae seleted ased o est judget of the assessor. In unsystematic approaches, this is typically the approach of choice. Under a systematic program, a logic-based approach may augment the other approach types to consider additional assets external to the original scope.

2.1.4.2.2 Criteria, Scoring, and Application

The evaluation method is organized around selecting and applying combinations of criteria to the resulting asset lists in order to distinguish critical assets from those that are not. Criteria are tailored to the organization and purpose of the identification effort.

Once criteria are established they are applied through scoring schemes, criticality matrices, and other methods in order to determine which assets meet the criticality criteria. Universal guidelines for establishing these criteria, scoring and application do not exist. Ultimately, there are various ways to combine and customize these framework components in order to establish a CI asset identification program. Existing methodology examples were found and are discussed in the next section.

2.1.5 Methodology Review

2.1.5.1 Methodology Survey

The search for CI asset identification methods was conducted on the IEEE Xplore

Digital Library [28], Homeland Security Affairs: The Journal of the NPS Center for

Homeland Defense and Security [29], Elsevier Science Direct [30], Taylor and Francis

Online [31], Google Search Engine [32], and Google Scholar [33]. A limited amount of relevant results was returned from each database. For example, a search of the phrase

itial asset idetifiatio ithi the IEEE dataase etued the esults shown in

Table 5.

Paper Type Count CI Related: Sector-specific Papers 6 CI Related: Cross-sector Compatible 0 Papers Non-CI Related Asset Identification 4 Papers Not in scope 46 Total 56 Table 5: Related Search Results of IEEE Papers

2.1.5.2 Methodology Examples

While conducting the CI asset identification survey, four examples stood out as representing each of the different CI asset identification approaches: 1) National Critical

Infrastructure Prioritization Program (NCIPP), 2) Defense Critical Infrastructure Program

(DCIP), 3) European Programme on Critical Infrastructure Protection (EPCIP), and 4)

Criticality Accessibility Recoverability Vulnerability Espyability Redundancy (CARVER2). A closer examination of each implementation here will help illuminate later gap analysis.

2.1.5.2.1 National Critical Infrastructure Prioritization Program

The NCIPP is managed by the Infrastructure Analysis and Strategy Division (IASD) within the Office of Infrastructure Protection (IP) under the National Protection and

Programs Directorate (NPPD) [25, p. 4]. Every year, IASD solicits nominations to the

NCIPP list from state homeland security and federal partners. The NCIPP list is used to establish risk management priorities that ultimately inform planning and resource decisions prescribed in the National Infrastructure Protection Plan Risk Management

Framework. The NCIPP influences resource allocations under State Homeland Security

Program (SHSP) and Urban Area Security Initiative (UASI) grant funds. In 2012, FEMA allocated $294M in homeland security grants to all 54 states and territories, plus $490M i UA“I gats to the atios highest-risk cities [25, pp. 8-10]. Thus, many stakeholders share an interest in how assets are identified and classified by NCIPP.

Initially, level 1 assets were identified by consequence thresholds, and level 2 assets identified by capacity in terms of the number of people they accommodated (e.g., building occupancy, transit ridership, utility customers, etc.). Level 1 criteria were designed to identify assets whose destruction could cause impacts on a scale similar to

9/11 and Hurricane Katrina. Level 2 criteria were designed to identify the most critical assets within each infrastructure sector. The capacity-based criteria often differed by sector, making it difficult to compare criticality across sectors, and therefore identify the highest-priority critical infrastructure on a national level. DHS was unable to compare

apples-to-apples. The soled this pole i akig leel iteia siila to level 1 criteria, but with lower thresholds. For each level, consequences were measured 23 in terms of fatalities, economic loss, mass evacuation length, and national security impact [25, p. 13]. While the new level 2 criteria facilitated comparisons across ifastutue setos, apples-to-apples, it failed to aout fo uiue fatos within the agriculture and food sector, such as the introduction of catastrophic diseases by a sigle aial. I DH“ addessed this pole itoduig speialized leel criteria for some sectors and assets. Nominated assets must also include ealisti seaios to justif thei oseuee lais. Noiated assets futhe udego adjudication providing state and federal partners the opportunity to review decisions and submit additional supporting information as necessary before the NCIPP list is finalized.

In 2010, DHS also adjusted NCIPP criteria such that an asset that only met the level 1 consequence threshold for economic impact, but no other level 1 thresholds, was added as a level 2 asset independent of the level 2 criteria. The purpose of this change was to account for instances when economic impact might be the primary impact, such as the collapse of the US financial system. Similarly, infrastructure that

eoe the sujet of edile theats fo aliious atos, ut dot otheise satisfy NCIPP criteria, would also be added as a level 2 asset. In this case, it is the threat, not the consequence that counts. Another significant change to NCIPP criteria in 2009 was expanding the classification of infrastructure to include assets, nodes, systems, and clusters. The purpose here was to capture structural relationships among some infrastructure. The need arose in 2008 after Hurricanes Gustave and Ike damaged a group of refineries causing a nationally significant disruption to petrochemicals across a

24 range of industries. The 2009 and 2010 changes to NCIPP criteria and format affected both the number of assets and composition of the NCIPP list [25, pp. 17-19]. The

Government Accountability Office (GAO) found that these changes affected the prioritization of both grant distributions and site security surveys, raising concerns aog stakeholdes. GAO also deteied that the speialized leel iteia complicated comparisons across infrastructure sectors, defeating the purpose of adopting consequence criteria in the first place [25, pp. 21-24]. Though NCIPP also prioritizes assets, their ultimate risk value is determined in the remaining steps of the

NIPP RMF.

2.1.5.2.2 Defense Critical Infrastructure Program

DCIP is a systematic, function-based method employed by the US Department of

Defense (DoD) to identify CI assets. The nine-step process begins by decomposing the mission and identifying required capabilities. Those capabilities are further broken down into task assets. The task assets are then evaluated against five criteria. Only one of the five criteria must be met in order for the asset to be nominated for advancement to the next step in the identification process. Nominated assets are validated by mission owners and then submitted to the Joint Staff for additional analysis and development of the initial Task Critical Asset (TCA) list. The initial list is then used to conduct interdependency analysis for identification of any additional assets that may be impacted by the disruption or destruction of TCA designated items. Once assets have been vetted by Joint Staff for verification of mission impact, appropriate defense critical

25 assets are nominated, reviewed, and approved or denied. The resulting critical asset list fos the asis fo othe DCIP-related activities for the coming year such as vulnerability assessment planning and remediation and mitigation prioritization suissios fo the DoD [34]. Unlike NCIPP, DCIP prioritizes CI assets independently from identifying them.

2.1.5.2.3 European Programme on Critical Infrastructure Protection

EPCIP provides systematic, network-based guidelines for European Member

States to identify European CI assets. States have the option to use these guidelines or implement their own programs. To begin identifying European CI, EPCIP recommends a four-step process that begins with evaluation of assets against sectoral criteria. Each infrastructure sector has its own defined set of criteria that can include multiple properties such as capacity, distance from other infrastructures, or specification of certain assets that must be included. Any assets that meet sectoral criteria are then evaluated against Directive CS/2008/ hih defies CI as a asset, sste, o part thereof located in the Member States which is essential for the maintenance of vital societal functions, health, safety, security, economic or social well-being of people, and the disruption or destruction of which would have significant impact in a Member

“tate as a esult of the failue to aitai those futios [9]. National thresholds or cross-cutting criteria are used to evaluate consequences in relation to the CI definition.

At this point, alternative back-up resources and time to recover are also considered in determining whether or not an asset meets the CI definition. If the asset meets these

26 criteria it advances to Step Three and is evaluated based on the cross-border impact of the asset on other Member States. Lastly, the asset is evaluated against cross-cutting criteria to finalize the critical asset list. Cross-cutting criteria includes potential number of casualties, degree of economic loss, and impact on public morale. This step requires deelopet of a easoale ost ase seaio to suppot oseuee estiates.

Finally, an asset may only be designated as European CI if it meets the criteria at all four steps and is approved as such by the Member State where the asset is located. If the

Member State disagrees with the CI designation, then the asset will be disregarded, even if it meets all criteria.

2.1.5.2.4 Critical Accessibility Recoverability Vulnerability Espyability Redundancy

CARVER2 is an unsystematic approach to CI asset identification. The method is applied across US infrastructure sectors by operators, government, and private industry to fulfill non-tehial eeds of CI aalsis ostl fo a poli ake poit of ie

[35]. CARVER2 scores an asset based on the six criteria that comprise its acronym: criticality, accessibility, recoverability, vulnerability, espyability, and redundancy. As with NCIPP, the scoring feature is used not only to identify an asset as CI, but also to prioritize it. Unlike aforementioned methods, CARVER2 is employed on an individual basis, not systematically within any particular infrastructure sector.

In addition to the CIP methods reviewed, methods from other disciplines were also reviewed for insight and including methods used in Information Technology,

Banking and Finance, Environmental Engineering, and Chemical Engineering. More information on these reviews can be found in Appendix A.

Limitations of Current Identification Methods

2.2.1 General Identification Challenges

From this review, we found that while several methodologies exist today, there is no apparent way to validate and verify that the current programs are assessing the right assets against the right iteia. Additioall, o oe ethod has poe to eet all of the requirements of an effective CI asset identification methodology.

Example Scope Approach Criteria Types Methodology NCIPP Unspecified Unspecified Consequence Criteria DCIP Systematic Function-based Multi-Criteria (Consequence, Fault Tolerance) EPCIP Systematic Network-based Multi-Criteria (Consequence and Spatial) CARVER2 Unsystematic Logic-based Multi-Criteria (Consequence, Recoverability, Redundancy, Vulnerability, Interdependency, Espyability, Accessibility) Table 6: Methodology Classifications

2.2.1.1 Completeness

Generally, it is a combination of selected scope and approach that determine the

oeage of assets o opleteess. The challenge is in determining which combination of assessment characteristics has the capability to provide a complete

28 assesset pool. Fo a sopig pespetie, its lea that a ussteati appoah would not be proven as complete when being implemented as a stand-alone program.

This does not mean that an unsystematic program, such as CARVER2, could not be applied in a systematic manner, but as a generalization, unsystematic methods fails to

eet the opleteess iteia. “steati ealuatio is iitiall iplied i the other approaches, but customized program elements such as requirement to include threat scenarios and exclusion of certain asset types as seen in the NCIPP implementation can

ede a ealuatio ioplete. Its ipotat to udestad ho those customizations impact completeness. Similarly, the EPCIP has limited CI asset identification focus on the energy and transportation sectors [36] while struggling to overcome political disagreements on identification criteria for additional sectors. This temporary limitation of scope hinders completeness of the program.

In selecting the approach for assessment completeness, distinguishing whether function, network, or logic-based approaches is most effective is also a challenge. In the case of NCIPP, various sectors implement their own approaches to identifying assets further complicating attempts to measure completeness across sectors. A function- based approach, as seen in the DCIP example, focuses on assets that support critical functions and can reduce assessment effort required by narrowing assessment scope. It is possible that the function-based approach could overlook assets that dot fit the function or mission as defined, but these may be considered application errors, not systemic failures of the basic approach. Network-based approaches can also be an effective way to approach CI asset identification dependent on the objectives. This is

29 especially helpful in identifying dependencies and interdependencies between infrastructures. Some cons to this approach are the complexity and deep understanding of network analysis required to execute such methodology.

2.2.1.2 Reproducible and Documented

Consistency of results is paramount if risk comparisons are to be made between assets across different sectors. The evaluation component (criteria and application method) is the main variable relevant to ensuring reproducibility. These components should be objective. Yet, methods that incorporate consequence criteria and require scenario justifications for example, introduce a wide range of subjectivity into their assessments and their results may vary accordingly. In addition, the NCIPP nominating poess has ee desied soe “tate offiials as odeatel diffiult o e diffiult [2]. NCIPP results over the years have varied between sectors and users of the method, suggesting a lack of reproducibility and comparability. For example, at one point, one user of the methodology included the entire subway system as a single asset in their evaluation, while another user included each subway station as an asset [37].

Similarly, reviews of the DoD and EPCIP processes revealed that inconsistent criteria and subjective guidelines limited the effectiveness of the program [38] [36]. Conversely,

CARVER2 appears to do a fair job at keeping the methodology intuitive enough to reduce misinterpretation.

Proper documentation also plays a key role in promoting reproducibility. There is no systemic reason why any of the methods cannot be documented effectively. The

30 challenge is determining which combinations of components should be assembled and documented.

2.2.1.3 Defensible

In order to be considered defensible, the methodology should utilize state-of- the-art techniques to identify and apply appropriate criteria that align with the national

CI definition, meets the four NIPP requirements, contributes to identification of dependencies and interdependencies, and ultimately results in a reputable list of critical assets.

Aothe a of osideig defesiilit is to thik of alidatio ad

eifiatio. I othe ods, ae e doig the ight thig, ad ae e doig it ight? I order to answer this questio, e should defie the ight thig. I the otet of CI protection, one answer is that we want to keep CI from causing catastrophic damage either through its subversion, disruption, or destruction. From this perspective,

atastophi daage is the pia oe, ad the ight thig is peetig it fo happeig. Coe oe atastophi daage is a oe aout consequences. This would seem to affirm the appropriateness of applying consequence criteria to asset lists. One problem with the sole consequence-criteria application, though, is tig to distiguish etee eto ad iti. This pole aifested itself i NCIPP hee, aodig to a GAO epot, oseuee iteia ee uale to account for the fact that individual animals could be the entry point for a scenario— such as malicious contamination with an agent like foot-and-mouth disease—which may

ause atastophi effets [25]. While a single sick cow will not trip a consequence threshold, its potetial to ifet all attle ill. The o is ol a eto, ut it a

ake iti a sigifiat potio of the Liestok “useto. This dilea is ot limited to the Food and Agriculture Sector. Was it the buildings or the airplanes that were responsible for the 3,000 lives lost and $40 billion in damages on 9/11? The Twin

Toes did ot ollapse o thei o aod. The aiaft as the eto that aused the to ollapse, ad the Ti Toes ee the iti. B the sae toke, passenger jets o thei o aod ill ot eate atastophi daage. The ust also e iti to soe eto. A oseuee-criteria methodology, as seen in the NCIPP, appears incapable by itself of accounting for additional factors beyond the consequence threshold. The other programs apply consequence criteria, in combination with other criteria points. A challenge remains understanding which combination of criteria is fit to identifying the right assets. From this analysis, it seems that only a systematic multi-

iteia ethod that aouts fo oseuees ight e defesile.

2.2.2 NCIPP Specific Challenges

Wee estalished, though the NIPP, that the methodology should be defensible, complete, reproducible, and documented [39, p. 34]. In other words, the

DHS needs validation that they are indeed looking at the right assets and that all potentially critical assets are considered. The current process has limitations that inhibit attainment of these requirements. The main challenges can be associated with the

32 programs limited definition of an asset, impact of the transition to a solely consequence-based methodology, and program maintenance.

To egi, hile the NIPP euies a oplete assesset, o osideatio of all assets, the NCIPP defies a asset as a sigle failit ith a fied loatio that futios as a sigle etit. B defiitio, oile assets do ot appea to e considered. As was seen during the tragic events of 9/11, it is possible for mobile assets such as passenger jets to be exploited and used to inflict catastrophic damage. In addition, since cyber assets can include fixed, mobile, or logical assets, they can also be overlooked when using this definition alone. Understanding where these two asset types fit into the current identification methodology is not clear.

Secondly, the NCIPP is a consequence based process. In practical implementation, a consequence based approach could be limited by the requirement to include threat scenarios and appears incapable by itself of accounting for additional factors beyond the consequence threshold [40]. For example, even after changing identification criteria to only consider consequence, exceptions have since been introduced for specific sectors to try and address the limitation of the thresholds. The

GAO describes how the consequence-based criteria were unable to account for individual animals as an entry point for malicious contamination such as foot-and-mouth disease [25, p. 15]. This challenge is not limited to the farm sector as there are other industries with similar individual entry points such as the Transportation sector. Instead of continuing a cycle of introducing specialized processes and criteria to accommodate

33 each individual sector, it may help to instead consider how the overarching process can be improved to become more inclusive and succinct by design.

Chages ade to iteia to aoodate speial iustaes ould also hamper effectiveness of the program. For example, the addition of the Threats to

Infrastructure Initiative to the NCIPP blurs the intention of the resulting NCIPP list and can reduce usefulness of data. If there is an active threat to a bridge that is not considered nationally critical, is this included in the list based on this program? If so, the list is no longer a list of critical infrastructure assets, but begins to accommodate any non-critical at-risk infrastructure and defeats the purpose of the list. The purpose of the

NCIPP list is to identify assets that could result in debilitating impact if disrupted.

Eistee of a theat to ifastutue doest dietl suppot the easueet of impact of disruption. In actuality, if there is a threat to critical infrastructure it is considered later in the NIPP RMF using the formula Risk = f (Consequence x Vulnerability x Threat) [39, p. 32]. For this reason, the critical asset identification process should not consider threat or probably of occurrence. Those steps are to be performed after critical assets have been identified.

Lastly, program maintenance seems to be another challenge, specifically around incentivization. While responsibility for critical infrastructure identification and risk assessment lies within the federal government, a considerable portion of CI assets is owned by private sector entities. Therefore, participation of private sector, state, and local governments is crucial. Motivating these critical participants to continue contributing can pose long-term challenges. For example, continuous changes 34 implemented to improve the effectiveness of the program have been well received by most Sector-specific Agencies (SSAs), but most state officials consider the consequence based approach difficult to use and in some cases have contemplated ceasing further participation in the program. In a 2013 data call, only 13 of 56 states and territorial partners participated [25, p. 31]. Reasons for reduced participation may include, but are not limited to opinion of negative cost-benefit value of participation, the perceptions of program effectiveness and sustainability, and the voluntary nature of the program.

Since the program is voluntary, improving ease of use, advertising benefits, and other forms of incentivization are key to enabling continued participation.

Some participants, especially private sector entities, also have concerns over the privacy of data submitted. These concerns can cause groups to shy away from collaborating with other parties to share and understand the CI asset environment. For example, private sector parties may be reluctant to submit information that could present competitive risks in the wrong hands [41, p. 19]. In addition, some sectors must deal with the legal implications of submitting information regarding regulated assets.

The Protected Critical Infrastructure Information Program (PCII) was established to alleviate these concerns and encourage voluntary information sharing, but recent hackings of government databases undermine the credibility of PCII assurances.

Solution Exploration

2.3.1 Problem Statement

The identification of critical infrastructure assets is a core component of national

35 risk management and security. Several nations, such as Europe, US, Canada, Australia,

Japan, etc. [10] have attempted to formally define and identify these critical infrastructure assets. The approaches adopted by each varies and it is not yet clear which methods are most applicable given each nations core concerns. For example, while their definitions are very similar, the U.S. program has considerable variances in program execution when compared the program executed in Europe [42], [2]. Nevertheless, these programs are often used to prioritize protection efforts and allocate risk mitigation resources to critical infrastructure assets. The purpose of this research, is to aid these nations in developing critical asset identification programs that align with the needs and objectives of each nation. The system is based on multi-criteria decision making (MCDM) methodology. We recognize that each nation may have unique characteristics and as such allow nations to tailor programs to meet their needs, while still providing logical decision trail.

This research establishes a methodology that provides consideration of all asset types, not just fixed assets. In addition, in conformance with the NIPP, the method aims to provide a methodology that can be considered complete, reproducible, and defensible.

The aviation sector is used as a baseline and case study for this research.

A key source of motivation to properly identify and prioritize critical infrastructure assets comes from the need for organizations to determine in which areas further risk management resources should be allocated. Critical asset identification is a fundamental step in risk management and sets the foundation for quality analysis and useful results. We cannot optimally conduct risk assessment on critical infrastructure if

36 we do not get this step right. As shown in Figure 2, the DH“s NIPP is o eeptio. I order to determine which assets are critical and which ones are not, we must first understand criticality.

Figure 2: National Infrastructure Protection Plan Risk Management Framework [4]

2.3.2 Understanding Criticality

2.3.2.1 Criticality Definition

In search of a solution, we first sought a deeper understanding of the word

itial. The Merriam-Webster [43] provides 3 thought-provoking definitions for the term. I health, its desied as an illness or condition involving danger of death. In siee, its desied as soethig of suffiiet size to sustai a hai eatio. Its also generally described as a state, measurement, or point in which some quality or property suffers a definite change. Based on these definitions we draw the following conclusions:

Criticality is a question of loss and consequence. In other words, if an asset is degraded or lost, will what’s defied as deilitatig ipat e realized? Criticality is also a question of chain reaction. If one asset is degraded or lost, how does it impact other

37 critical assets? Lastly, criticality is a question of measurement. I other words, what’s the tipping point? At what point does the loss of an asset change, or in this case detrimentally reduce, the desired quality of its provided function?

These are all questions regarding measurement and assessment of impact. This is important to note when understanding the difference, as well as the relationship between criticality assessment and risk assessment. These lines are often blurred. While risk assessment typically considers the product of vulnerabilities, threats, and impact, criticality assessment focuses solely on impact and does not include vulnerability or threat analysis. Impact is the common connecting element between the two assessments and one can even be seen as subset of the other. [44]

2.3.2.2 Identification vs. Prioritization

Identification and prioritization of critical assets is often confused and done interchangeably and is worth discussion to avoid confusion. After all, to some degree identifying critical assets can be seen as a form of prioritizing assets. To gain a better understanding of criticality we separate these components into complementary sequential components rather than steps that can be done in parallel. To simplify, we look at identification phase as a yes or no question: Is the asset critical to other assets or to organizational mission? The question of prioritization refers to: How critical is the asset in consideration of vulnerability, threat, likelihood, etc. in comparison to other assets?

For the purpose of this research we focus on the former and simply attempt to establish a method for answering these questions across sectors inclusive of all asset types. As such, we view criticality as a decision. In the end, there are 2 final choices as show in Figure 3. The deciding factors leading to the selection of a final choice will be the breadth of these research.

Critical Critical to Other to Assets? Mission?

Special Cases?

Criticality Decision YES or NO

Figure 3: High-level Identification Decision Factors

2.3.2.3 Fundamentals of Criticality Assessment

There are numerous approaches to criticality assessment depending on the objective of the assessment and often times the industry the assessment is applied to.

For example, in project management, tasks are assessed for their criticality to the project schedule. In reliability engineering, criticality may be assessed in considerations of the Mean Time to Failure (MTTF) and Mean Time to Repair (MTTR). In business, stakeholders may focus on the criticality in relation to revenue or brand. As such,

39 criticality assessment for critical infrastructure should be tailored to the objectives of

DHS or national security goals.

However, independent of the industry or objective, there are some basic elements of criticality assessment, previously outlined in Table 4, that can be used as a baseline to get started. No matter the sector or industry, we must set the assessment scope, establish criteria and metrics, and then determine how these elements will be applied to assets. Existing theories and methodologies used to do so are discussed in the next section.

2.3.2.4 Theory and Methodology

Approaches to criticality assessment are usually based on one of the following premises in the context of CIP:

1.) Criticality is determined by an assets position in the system/network and its relation to other assets. [45] [46] [42] This approach may consider an asset critical based on its connections and/or points of failure.

2.) Criticality is determined based on an assets ability to meet pre-determined selection criteria meant to evaluate the assets role in fulfilling a specific function. [46]

[42] [25] Criteria often include measurements such as potential loss of life, economic impact, descriptive characteristics, etc. On this same token, criticality can further be considered based on degree of change that the degradation or loss of an asset inflicts on the quality of the provided function.

Metzger concludes that the former aligns more with emergency management goals, while the latter is more applicable to national security efforts [46]. He argues that the criteria-based approach allows non-technical and non-networked assets to be considered. Similarly, ENISA [47] reports that the network approach ignores critical services while criteria-based approaches can be aligned with the critical services outlined in most national definitions of CI. Complexity is cited as a challenge for both networked and non-network approaches due to the sophistication required to identify dependencies and interdependencies and challenges associated with developing appropriate criteria for the assessment.

2.3.2.4.1 System and Network Theory

Network theory is commonly applied to critical asset identification efforts. [42]

[48] [49] Specific program and application examples include Athena, CI^3, DEW, IIM,

LUND, and HAZOP [50]. In addition to traditional networking analysis and simulation methods, new methods are emerging such as social network theory for infrastructure prioritization [51]. A simple noteworthy example of the application of network theory is seen in the work of Ted G. Lewis [52]. The approach uses the measurement of network liks to deteie the itialit of odes ude the aguet that it takes a etok to fight a etok. “peifiall, he argues that due to the complexity of critical infrastructure, network analysis is the best way to identify critical assets. He also notes that emphasis should be placed on protecting hubs (critical nodes), not spokes (links).

This would result in the allocation of 80% of protection investments to be made on 20%

41 of the country. To identify critical hubs, Lewis uses scale-free network theory in which the number of links is counted for each node and then ranked from highest to lowest.

Nodes with the highest number of links are considered most critical. The research acknowledges that in some cases links can also be considered critical. The total number of links is calculated as follows where N is the number of nodes:

N x (N-1)/2

Cascade network theory is further applied to analyze the impact of loss of a critical node or link on other nodes and links, often referred to as interdependency analysis. Lewis calculates this as probability that fault will spread = rate faults are incurred/ rate faults are cured. If ratio is greater than 1, the cascade continues and if less than 1 the cascade fades.

While this provides a simplified network approach to identifying critical assets, it does not fully align with the NIPP and the definition of critical infrastructure. Specifically, there is a gap between how the identification of critical nodes and links translates into measurements of debilitating impact specified in the definition of CI such as economic security or public health. Using this approach alone simply evaluates how critical an asset is based on its position in the network, not its impact on attributes requested by the formal definition of CI and the NIPP. For example, it is possible that an asset can have a high number of nodes, but not impact economic security or public health. While it is also possile that a ode ith a lo ue of liks a still podue hats considered debilitating impact.

Because CI often includes multiple interdependent assets and systems, it is often described as a system of systems. West Churchman [53] notes two approaches to system analysis: structural and functional analysis. Structural analysis entails identifying boundaries between the system and the environment, identifying elements of the system, and then identifying channels of communication. Functional analysis focuses on identifying the systems objectives or intended functions usually through analysis of iputs ad outputs. Its oth othig that ee he titled sstes theo, ost of these approaches still build on network theory to perform system analysis. In some cases, the terms are used interchangeably.

Systems theory approaches are often used to identify and analyze dependencies and interdependencies and understand the system as a whole, thus contributing to identification of components that are critical to the overall system [46] [48] [42] [52]

[50]. Furthermore, CI can change in reaction to variables such as economic environment, demographic changes, thus creating the label: complex adaptive system (CAS). CAS are described as populations of interacting agents and the theory is applied in several research papers to understand the impact of topology on system robustness during disruption or failure [48] [54] [55] [45] [42]. In consideration of these complexities,

Rinaldi and his research partners [45] offer a simplified way to describe interdependencies. They describe links between CI as bidirectional relationships in which the state of one infrastructure influences or correlates to the state of another.

These relationships can be categorized using the 6 dimensions of infrastructure interdependency shown in Figure 4. The approach defines an infrastructure agent as an

43 entity with a location whether physical or abstract, but does not explicitly provide a way to identify critical infrastructure assets. It does provide a means for understanding interdependencies thus contributing to the identification of critical assets. One thing to

ote is that hile Leiss etok foula appeas to fous o oe tpe of failue

(cascading), the complex adaptive system approach categorizes three types of network failures: cascading failure, escalating failure, and common cause failure.

Coupling and Response Infrastructure Type of Failure Environment Interdependences State of Operation Behavior Characteristics • Common Cause • Inflexible • Business • Cyber • Normal • Organizational • Cascading • Adaptive • Public Policy • Physical • Repair/Restoration • Operational • Escalating • Linear/Complex • Security • Logical • Stressed/Disrupted • Temporal • Loose/Tight • Health/Safety • Geographical • Spatial • Economic • Legal/Regulatory • Technical • Social/Political

Figure 4: Dimensions for describing infrastructure dependencies [45]

The European Joint Research Commission takes this research a step further to outline a more detailed process for analyzing CI as a Complex Adaptive System (CAS) applying both system and network theory. The research outlines two approaches: systemic and spatial. As the titles imply, the systemic approach represents the organization of the system and relation of components. The spatial approach relies heavily on network theory to represent the system from a territorial perspective. For example, when considering functions, the systemic approach outlines the quantity or quality of service delivered. In the spatial approach, function is considered the geographical areas where the service is delivered. While approaching identification from

44 both of these angles provides a more thorough assessment, this still does not fully align with the definitions of critical infrastructure as set forth in the Patriot Act, NIPP, etc.

2.3.2.4.2 Criteria-based Methodology

In practice, most CI identification methods determine criticality based on predetermined criticality criteria. In this case, identification efforts rely on the degree to which an evaluated asset meets a defined set of criteria. The details of the approach can vary based on the purpose of the identification efforts as described in the next paragraph, but basic steps in the process include establishment of an asset baseline, identification of criticality criteria, and selection of an application method. Example common criteria categories for CI identification include impacts on public morale, the economy, and loss of life [9]. Some applications use the approach to both identify CI assets and prioritize them based on the magnitude of the consequences. In the context of asset identification, it is important to note that the probability that the consequence will occur is not considered at this stage. In most risk assessment programs, the probability that the consequence will occur is considered later when risk is calculated, after the CI asset is identified [9].

The aforementioned review of criteria-based methods revealed that this approach is highly customizable and implementations vary across programs. From this review, we found that while several methodologies exist today, there is little guidance available to help decision makers ensure they use a methodology that meets their objectives. By combining the strengths of existing methodologies to provide a

45 systematic solution, there is an opportunity to address the current gaps and challenges associated with critical infrastructure asset identification efforts.

2.3.3 Emerging Research

The use of decision theory in the critical infrastructure discipline is growing [3].

Decision theory can be organized in three categories: 1.) Certain 2.) Uncertain 3.) Under

Conflict. Under each circumstance, different decision theory sub-disciplines can be applied. Multi-criteria decision making theory (MCDM) is one sub-discipline with existing approaches that can be used when information is certain or uncertain. This is typically the case for CI asset identification. MCDM provides a multi-level, multi-attribute or multi-objective framework for scoring and ranking alternative decision options against multiple criteria [3].

According to a report [3] on analysis done between 2003 and 2013, applications of MCDM research to CI challenges quadrupled during this timeframe. Furthermore, there was also an increase in application of the theory to transportation sector challenges. The report cites that of 300 existing research papers, 35% of CI MCDM papers were applied in transportation. Generally, MCDM systematically enables the combination and application of multiple and often conflicting criteria subject to different levels or uncertainty to support decision making. Where decisions are traditionally subjective in nature, MCDM allow decisions to incorporate engineering judgment along with expert opinion in order to provide more objective outcomes.

Game theory is another decision related philosophy that has been applied to critical infrastructure problems [56] [57]. Because game theory is concerned with

iteatios of agets hose deisios affet eah othe [58] it seems most applicable to dependency, interdependency, and cascading effect analysis. This type of analysis is just a fraction of the decision components required for a comprehensive critical asset identification methodology. For the challenge at hand, a theory applicable to all components of CI identification must be used. A multi-criteria decision theory appears to be more applicable to the critical infrastructure identification problem in that it allows the customization of identification components to meet specific needs. A major challenge we seek to address is understanding which combination of multi-criteria can be applied to identify the right critical assets and MCDM appears to be a proven methodology that can be applied to address this challenge.

2.3.4 Applying MCDM to CI Problems

In Figure 3, we established that at the first stage, determining criticality can be seen as a yes or no question, or a decision. Will degradation or loss of an asset produce the impact we strive to avoid? If the answer is yes, then the asset can be categorized as critical and further evaluated. The challenge is that there are many factors that can contribute to the decision. What level of degradation produces the unacceptable impact? Exactly what impact do we strive to avoid and how can it be measured? These are just a few questions and information points that contribute to the main question: Is the asset critical? Viewing criticality as a decision and applying MCDM allows us to

47 simplify identification efforts by breaking the decision down into a logical decision making system.

2.3.5 Existing MCDM Applications to CI Asset Identification

While in practice, multiple criteria are considered in the evaluation of critical assets, these methods often lack scientific support and theoretical foundation. We aim to understand which methodology can help identify assets that are critical in accordance with the formal definition of CI. Halim and Mohamed [59] apply Analytic Hierarchy

Process (AHP) to identify critical level of assets in the Malaysian water sector. The paper does a fair job at showcasing how AHP can be applied to infrastructure identification, though it aims to solve a problem that is somewhat different from the one addressed through this research. Furthermore, it bases criticality analysis on probability of failure and consequence of failure. As mentioned in previous sections, in this context of CIP, probability of failure should not be considered during the initial identification process.

We must first focus on identification, before prioritization. Prioritization comes as a result of risk analysis.

Another application of MCDM to critical infrastructure identification proposes a methodology for using the Measuring Attractiveness by a Categorical Based Evaluation

Technique (MACBETH) multi-criteria decision tool. MACBETH is often described as very similar or a subset of AHP [60] [61]. The research highlights the limitations of independently implementing network-based and criteria-based programs and suggests a compensatory method that builds on Canadian and Portuguese identification

48 programs to address challenges. The proposed approach is limited in its ability to prove assessment completeness and also does not provide the basis for the selection of the multiple criteria used for the assessment. In addition, MACBETHs accuracy has been questioned after comparisons between AHP and MACBETH results proved that AHP provided a more precise evaluation of alternatives [61].

3 CHAPTER III

RESEARCH METHODOLOGY

This research aggregates the existing wealth of knowledge in this discipline to design and validate a decision methodology for establishing a CI asset identification program. When customized appropriately, MCDM has the ability to provide transparency, analytic rigor, and auditability of the decisions made using the model [62].

Its idel used i a aiet of idusties [3] and is considered reputable. Thus, by breaking the process down into a logical decision system, developing a systematic process for arriving at criticality decisions and validating the method with industry experts, we provide a solution that can be considered defensible. In addition, the simplified and logical nature of MCDM can support reproducibility. Lastly, the highly customizable nature of MCDM methodology and depth of research included in its development will allow programs to be tailored to precisely meet the objectives of the assessment and provide a method that can be considered complete.

Required Tasks

A major deliverable of the research includes a decision system for developing a

CI asset identification methodology, with application of MCDM to compare alternative

50 methods. The resulting tool is called CIAid. In order to achieve this, the following steps were required:

a. Select MCDM approach

b. Build identification framework

c. Form knowledgebase

d. Relationship analysis

e. Build model and database

f. Complete testing and expert validation

g. Complete aviation case study and dissertation report

Step 1: Select MCDM approach

The first task required selection of a multi-criteria decision making (MCDM) approach. There are numerous existing MCDM methods (e.g. AHP, TOPSIS, ANP,

ELECTRE) which required evaluation in order to select the best method for comparing decision layers and criteria based on objectives. We established that the selected method should be able to accommodate multiple objectives and hierarchical criteria, and that the application of the method should result in a est alteatie desigatio.

Based on the requirements, existing MCDM methodologies were reviewed and proven evaluation and selection methods applied to determine the best option. Through this analysis, it was determined that TOPSIS would be the best MCDM method to apply to the problem. The detailed selection steps are included in Chapter 4.

Step 2: Build identification framework

The second task was to build the identification framework. During preliminary research, it was determined that a common framework should be established to organize the decision system in a way that suits critical infrastructure identification in a scalable way. A high-level framework was created to meet this need, shown in Figure 1, and includes 3 main layers: scope, identification approach, and evaluation method.

These categories acted as a starting standard to the decision system for CI identification methodology. The framework was expanded by breaking objectives down into lower level attributes. Those attribute components were then broken down into a lower level of sub-components, forming a criteria map. An introduction to the expanded framework is shown Figure 5.

Step 3: Form knowledgebase

The third task was to build a knowledgebase comprised of existing and emerging resources. During this phase, we gathered data on assessment objectives and existing attributes at each level of the framework (scope, approach, and evaluation method).

Potential objectives were drawn from CI asset identification programs in the U.S.,

European Union, Australia, Canada, and other countries where information was available. Additional objectives were drawn from policy, existing programs, and methodologies. Using this information, each component of the high-level framework was broken down into further sub-components. The knowledgebase is fully expanded in

Chapter 5.

Figure 5: CIAid Framework Map

Step 4: Relationship Analysis

The fourth task, relationship aalsis, folloed losel. Its ipotat to understand relationships between attributes and objectives as this becomes the basis for development of a methodology. Analysis was performed on the identified objectives and criteria to document common links and map relationships between framework components. A final and complete decision tree was then created to illustrate the

53 relationships between objectives and all other methodology components using MS Visio

Software. These diagrams are details in Chapter 6.

Step 5: Build Model and Database

The fifth undertaking entailed the design and development of a database and model that represents the documented objectives, and their mappings to criteria and applications methods. The purpose of this is to evaluate the different alternatives available to decision makers and select the one that provides the closest fulfillment of objective requirements. The database was established in Microsoft Excel, with additional macros and Visual Basics elements added to create the model. The second component of the model includes a user-driven evaluation process developed to compare alternate methodologies, using TOPSIS, and provide the user with the most viable alternate solution in consideration of their own constraints. These steps are also detailed in Chapter 6.

Step 6: Testing and Validation

The sixth task involved verification, validation, and evaluation of the research and tool with security industry experts, adjustment, re-evaluation until satisfactory results were achieved. This process was split into two core tasks: an online survey targeting a broad audience, and in person interviews targeting a more limited audience.

Step 7: Complete Aviation Case Study

Finally, a deeper analysis was conducted on the aviation sector and how critical assets are currently identified in this area. The case study provides background on the sector, discusses fundamental risk and emerging cyber threats, and shares how these problems relate to critical infrastructure protection. The report then offers ways to establish sector specific aviation criteria and includes the results of a validation survey conducted with aviation industry experts. A dataset was also developed and criticality criteria applied to produce an example critical asset list.

This sector is chosen because of its diverse representation of asset types and methodology limitations discovered during preliminary project research that may exclude key asset types [63]. Additionally, the aviation sector remains an attractive target for malicious intenders aiming to disrupt critical infrastructure and spread terror globally.

Success Criteria

To evaluate the resulting solution, we align with the methodological requirements outlined in the NIPP. We ensure that the methodology development tool supports completeness, reproducibility, documentation, and defensibility.

Requirement Description Interpretation of Need Completeness Coplete eas the ethodolog Consideration of is ssteati i eaiig eah relevant asset types and every relevant system and system component. components

Reproducibility ‘epoduiilit eas that esults Intuitive process and are consistent, simple, and precise. objective scoring Documented Douetatio is a eod of Documented hat ifoatio is used ad ho it dissertation report is synthesized to generate a risk and user guidelines estiate. Defensible Defesiilit eas that the Mission alignment, ethodolog akes use of the criteria and system professional disciplines relevant to validation the analysis, as well as being free from significant errors or oissios. Table 7: Explanation of Success Criteria

Validation Scoring

The methodology was also validated by security experts with the scoring scheme detailed in Table 8. The results of the validation process are included in Chapter 7.

Score Action Required 1/5 pass Develop improved research plan and refine methodology 2/5 pass Develop improved research plan and refine methodology 3/5 pass Changes highly recommended, unless valid reason to do otherwise 4/5 pass Review comments and make optional changes where necessary 5/5 pass Changes optional Table 8: Validation Scoring Scheme

Key Assumptions and Limitations

In this research, we assume that where multiple decision makers are involved, the group must first reach consensus on program objectives prior to using this decision system. The tool does not dynamically accept input from multiple decision makers.

There is an opportunity to explore this option in future research as the selected MCDM methodology could support this. We also assume that the decision maker has access to national CI definitions and policies, and also understands the data gathering capabilities of the organization in order to make informed selections in the system. We initially assumed that when comparing alternative methodologies, a method that exceeds what is currently recommended does not count against the overall score of the methodology.

While one could argue that minimum expectations should be met in order to minimize unnecessary costs and consider this to be a negative mark on the overall score, we decided to give this neutral impact on the score. During testing, we reconsidered this stance because this assumption diminished the value of the entire tool and the output.

We instead address this challenge by tweaking the TOPSIS methodology to our advantage. This is further explained in Chapter 6. One limitation of this research is that the criteria currently included in the tool focuses on aviation criteria. There are several other sectors that will require additional research in order to be incorporated into the body of work. Additionally, within the aviation case study portion, we were unable to secure a publicly available dataset for one of the four sub-criteria categories, air traffic management (ATM). To make up for this, we rely on one sub-criteria dataset, airports, for two categories. Doing so still provides value because the measurement of the two categories are closely related.

4 CHAPTER IV

INTRODUCTION AND SELECTION OF MCDM METHODOLOGY FOR APPLICATION TO CRITICAL INFRASTRUCTURE ASSET IDENTIFICATION

Designation of critical infrastructure assets is often considered a cumbersome and ople task. To siplif its identification, the process can be viewed as a logical decision in which objectives and criteria are used to distinguish those assets whose loss could have signification impact on a nations overall well-being. Numerous MCDM methods were considered during the quest to find a supporting theory equipped to help build this logical decision system and solve the challenge at hand.

Decision theory is growing in popularity in the critical infrastructure discipline.

Specifically, MCDM applications have increased by nearly four times over the last decade. MCDM provides a multi-level, multi-attribute or multi-objective framework for scoring and ranking alternative decision options against multiple criteria [3].

When applying MCDM to problem, a preliminary step in the process is the selection of the methodology that best suits the problem or decision at hand. There are numerous existing MCDM methodologies to be considered (e.g. AHP, TOPSIS, ANP,

ELECTRE). An evaluation was done to determine which approach is best for comparing

58 alternatives against criteria. There are several key requirements that the selected method must be able to accommodate. Based on the requirements, existing MCDM methodologies were reviewed and a customized evaluation method applied to determine the best option. A brief introduction to MCDM and the problem MCDM is being applied to is provided. We then discuss the decision factors and selection process that led to the selection of TOPSIS as the MCDM approach applied to the critical infrastructure asset identification problem.

Background

4.1.1 Introduction to MCDM

MCDM provides a multi-level, multi-attribute or multi-objective framework for scoring and ranking alternative decision options against multiple criteria. University of

Manchester researchers Dr. Xu and Dr. Yang report that MCDM problems typically share the following traits:

Complexity Multiple attributes Hierarchical criteria levels Hundreds of sub-criteria Conflict Conflict among criteria Hybrid Nature Incommensurable units Qualitative and quantitative attributes Deterministic and probabilistic attributes Uncertainty Subjective judgments Incomplete information Variable Output Assessment may not be conclusive Table 9: MCDM Challenge Characteristics [64]

Many of these traits are present in the current critical infrastructure asset identification problem. Using the appropriate MCDM methodology, we can address

59 these challenges in identifying appropriate criteria and establishing a methodology for identification. The general MCDM process includes [65]:

1. Identify the objective of the decision

2. Identify criteria

3. Identify alternatives

4. Select weighing method

5. Select aggregation method

6. Make decision

According to Dr. Yu and Dr. Yang [66], two types of MCDM methods exist: non- compensatory and compensatory methods. Non-compensatory methods do not permit trade-offs between criteria. In other words, each attribute should be considered individually [64]. On the other hand, compensatory methods do permit trade-offs. Yang and Xu further categorize compensatory methods as scoring, compromising, concordance, or evidence reasoning approaches. Scoring MCDM methods, such as

Analytical Hierarchy Process (AHP), translate attributes into preference scales or calculated weighted scores and then analyze alternatives using a decision matrix. AHP, for example, employs pairwise comparisons in the decision matrix to evaluate alteaties. Copoisig ethods ai to fid the et est solutio i opaiso to the ideal solution. An example of this is Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). Weighted scores are developed and incorporated into the decision matrix for the problem. The variance between the options and the ideal solution are compared to determine which option has the closest score to the ideal 60 solution [64]. The Linear Assignment Model [67] is an example of a concordance method. In this case, there is linear correlation between the rank of the attributes and the rank of the overall alternative. Lastly, the evidential reasoning approach employs an algorithm that combines decision theory and Dempster-Shafer theory of evidence combination rule to introduce belief degrees to MCDM. The approach uniquely extends the traditional decision matrix to include a degree of belief for each attribute [64].

4.1.2 MCDM Applied to CI Asset Identification

The application of MCDM to critical infrastructure problems continues to grow.

According to a report [2] on analysis done between 2003 and 2013, applications of

MCDM research to CI challenges quadrupled. Furthermore, there was also an increase in application of the theory to transportation sector challenges. The report cites that of

300 existing research papers, 35% of CI MCDM papers were applied in transportation.

Generally, MCDM systematically enables the combination and application of multiple and often conflicting criteria subject to different levels or uncertainty to support decision making. Where decisions are traditionally subjective in nature, MCDM allows decisions to incorporate engineering judgment along with expert opinion in order to provide more objective outcomes. As shown in Figure 6, AHP is one of the most popular

MCDM methods. In a review of MCDM methods applied to infrastructure management problems, 24% were AHP topping all other individual methods by more than 3 times [3].

Other methods included are Weighted Sum Model (WSM), Elimination Et Choix

Traduisant la REalite: Elimination and Choice Expressing Reality (ELECTRE), Preference

Ranking Organization METhod for Enrichment and Evaluations (PROMETHEE), and

Compromise Programming (CP).

24%

48%

6% 5% 2% 5%

WSM AHP ELECTRE PROMETHEE CP TOPSIS OTHER COMBINED

Figure 6: Types of MCDM applied to Infrastructure Management [64]

MCDM also integrates well with the specific task at hand. Researchers from

Central Queensland University describe the steps of criticality assessment as 1.) asset identification, 2.) Criteria 3.) Weighted Scoring, 4.) Scoring Guidelines, and 5.)

Application. This is very much in line with the process of MCDM and even directly align to steps 2-5 included in the previous section.

Selection Methodology

4.2.1 Existing Selection Methods

There are existing frameworks that have been established to aid in the selection of an approach.

Resource Authors Paper Summary MCDM Tehiue E. Kornyshova, C. Salinesi Compares 7 existing Selection Approaches: selection methods State of the Art covering outranking methods, AHP, MAUT, weighting methods, fuzzy methods, multi-objective, and others. Tetatie Guidelies to A. Guitouni Outlines common Help Choosing an attributes for a wealth of Appropriate MCDA MDCM methods enabling Method selection based on those attributes. Fist Look at MCDM: P. Mota, A. Campos, R. Provides a selection Choosing a Decision Neves-Silva method based on four Method facets to the analysis: problem, action, criteria, and usage A Epet “ste fo K. Eldrandaly, A. Ahmed, Describes background and Choosing the Suitable N. AbdelAziz process used to develop a MCDM Method for Solving tool used to select MCDM a Spatial Decision methodology for spatial Pole problems. Table 10: MCDM Selection Methodologies

For this research, the strengths of the selection methods gathered from resources listed in Table 10, were combined to create a selection aid for the CI asset identification problem. The process is mainly inspired by the work of Eldrandaly, et al.

[68] and [69] whose work has been verified and validated by domain experts and is highly cited respectively. Eldrandalys eseah suggests usig the folloig groupings to select an approach: characteristics of decision problem, characteristics of decision

63 maker, and characteristics of solution technique.

Suitable MCDM Technique

Decision Problem Decision Maker Solution Technique Characteristics Characteristics Characteristics

# of Decision Restriveness of Size and Complexity Accuracy Total Solution Time Ease of Use Makers Assumptions

Decision Maker Decision Making Decision Type Learning Curve Interaction Time

Uncertainty of Desired Input Solution Time Cognitive Burden Problem

Desired Output

Figure 7: Expert Recommended MCDM Selection Process [68]

4.2.2 Customized Selection Method

Based o guidae poided though Eldadals eseah ad the othe selection frameworks, the original graph was restructured to reflect a more optimal selection framework tailored to fit this research.

Complexity of Problem Data Type Problem Type Data Availability Desired Input Preference Timing Decision Problem Characteritics Preference Elicitation

Compensation Attribute Relationships Criteria Interaction

Desired Output Problematic

# of Decision Makers Selection Decision Maker Characteristics

Capability CI Asset Asset CI Identification MCDM Existing Papers Successful Applicaitons Key Limitations Solution Technique Characteristics Software Availability Ease of Use Total Solution Time

Figure 8: Customized MCDM Selection Method for Critical Infrastructure Problem

4.2.2.1 Decision Problem Characteristics

Decision problem characteristics consist of complexity of problem, problem type, desired input, attribute relationships, and desired output. Complexity of the problem is measured by the number of objectives, criteria, alternatives, etc. [68]. The problem type can be considered multi-objective or multi-attribute. In the multi-objective problems, there is usually a high number of alternative solutions, and objectives and constraints 65 are functionally related to the decision variables [70]. In multi-attribute problems, there is usually a small number of alternative solutions, and the attributes and objectives are organized in such a way that attributes are represented as decision alternatives [68]. We also consider whether the method is deterministic or non-deterministic. Desired input is defined by 4 categories. The first is the criteria data type, which can either be qualitative or quantitative. The second is data availability. In other words, is the data required to make the decision complete or incomplete. Then, preference timing can be described as prior articulation, progressive articulation, or posterior articulation [68]. In prior articulation, the objectives and preferences of the decision maker are gathered prior to the start of the decision-making process and used to select the best solution. In progressive articulation, the objective and preference process is interactive. The process undergoes several iterations of consideration of alternatives, updated preferences, and so on until a reasonable decision is made. In posterior articulation, all possible alternatives are identified upfront. Afterward, preference information is elicited from stakeholders to determine the best decision. Lastly, preference elicitation describes the method used to gather preferences from the user and can be categorized as trade-off method, pairwise comparisons, or direct rating from the decision maker.

The next problem characteristic is the attribute relationship. This is judged by criteria compensation and criteria interaction. Criteria interaction pertains to relationship between attributes and can be considered independent, cooperative, or conflicting. Additionally, criteria compensation can either be considered compensatory

66 and non-compensatory. Compensatory methods accommodate trade-offs between criteria, while non-compensatory methods do not.

Lastly, the desired output can be described as ranking, sorting, choice, or description. These definitions are self-explanatory.

4.2.2.2 Decision Making Characteristics

The next components to be considered during the selection process are regarding decision maker characteristics. This is defined by the number of decision makers and capability of decision maker. The number of decision makers is categorized as either a group decision or individual decision. The decision maker capability is desied the deisio akes apait to lea ad use the deisio poess.

4.2.2.3 Solution Technique Characteristics

The solution technique characteristics consist of key limitations, ease of use, software availability, existing research papers, and total solution time. Limitations are described as underlying assumptions that limit the effectiveness or accuracy of the solution. Ease of use is defined by the cognitive burden placed on the decision maker and the learning curve [68]. Software availability refers to accessibility of existing tools that can be used to execute methodology. The total solution time is described by the time required to reach a decision. Lastly, existence of research papers is described by the volume and quality of search results of the method on popular research engines such as Science Direct, Google Scholar, and IEEE.

Selection Process

The CI asset identification problem was broken down into required attributes using the MCDM selection methodology. Then, using the process of elimination, alternatives were evaluated and removed until the most suitable choice remained.

4.3.1 CI Characteristics Formulation

4.3.1.1 CI Decision Problem Characteristics

MCDM can accommodate anywhere from tens to thousands of layers of criteria and sub-criteria. We make the assumption that the solution will not need to accommodate more than 50 objectives and 100 criteria categories. This assumption is made because of the effort required to gather and maintain criteria. It is unrealistic for users to gather large amounts of data for every single asset, as such criteria options should be kept at a bare minimum. The problem type can be described as multi- objective and deterministic. The objectives of the CI asset identification efforts are what inform the selection of attributes and the decision.

The next category focuses on input, namely data type and availability, preference timing, and preference elicitation. In CI asset identification, data objectives are not always complete or finalized. There may be information regarding objectives or assets that is not available or must be estimated. In addition, criteria type can be qualitative, quantitative, or a combination of both making it necessary for the selected

MCDM to support both types.

The preference timing most closely associated with the CI asset identification problem is prior articulation. The assessment process is determined by objectives.

Objectives are not a result of which alternatives are available eliminating posterior articulation as an approach. Progressive articulation could be of value, but adds a great deal of complexity to the decision process. Given the inherently complex nature of identification, we eliminate this approach to avoid over complication. This is not to say that future researchers may not want to explore this approach as an option in the future. When considering decision maker preference elicitation, tradeoff and pairwise based options were eliminated because the ultimate goal is not to make tradeoffs or exclude certain steps. Preference elicitation should be direct. Comparing and picking one alternative at each level may not be necessary. We want to include all of the criteria points that meet needs. Straight forward/direct rating is the best approach for this in alignment with clear objective-based solutions.

The next problem characteristic is the attribute relationship. Once a methodology recommendation is made, the stakeholder may need to adjust the method to more closely align with the organizations capabilities. With this in mind, compensatory methods are preferred in order to account for these adjustments. Criteria is not conflicting at the methodology development level. It can be considered independent and sometimes cooperative.

Lastly, the desied output fo the ethod is the hoie of the appropriate grouping of methodology attributes. The main end goal is not to sort, rank, or describe

69 the methodology options. The goal is to select an appropriate combination. This is not to say that a ranking and sorting methods may not suffice if necessary. In this case, the result could then be that the highest ranked alteatie, is the fial hoie.

4.3.1.2 CI Decision Maker Characteristics

MCDM variants can accommodate different types of decision makers. The CI asset identification problem could benefit from the accommodation of group decisions.

Numerous stakeholders may need to contribute to the identification of assets at various level. Ideally, to maintain simplicity, the stakeholders would agree on their objectives prior to developing the methodology leaving individual decision methods as a more desirable option. In addition, the decision makers using the tools may include government staff with mid-level expertise on asset identification. As such, the methodology must not be so advanced that the average employee cannot grasp the identification process.

4.3.1.3 CI Solution Technique

It is important that the methodology be considered accurate. This starts with existing successful application of the methodology to decision problems. Total solution time should not be more than 1 hour. This is the time in which the user would go through the process to select an identification methodology using the tool, provided all relevant decision data has already been collected upfront. The purpose of considering limitations in the selection process is to ensure that they can be accounted for and

70 addressed in the selected methodology. The ease of use should be reasonable. This requirement is not a great concern as it has more to do with the user friendliness of the interface developed for the final solution. Another way to view this is in making sure that the selected methodology allows intuitive categorization and grouping to limit the amount of interactions required from the decision maker. Lastly, software availability is a plus, but not a requirement. If methodology is sound, but software does not exist, the software can be developed.

4.3.2 MCDM Candidates

Thirty-one MCDM methodologies were gathered and considered through the review and analysis of existing MCDM research reports [64] [65] [71] [50] [69]. Some popular example MCDM types that were evaluated are shown in Table 11. A full list can be found in Appendix B.

Method Title Description AHP Analytical Hierarchy Process ANP Analytical Network Process ELECTRE Elimination Et Choix Traduisant la REalite (Elimination and Choice Expressing Reality) PROMETHEE Preference Ranking Organization METhod for Enrichment and Evaluations TOPSIS Technique for Order of Preference by Similarity to Ideal Solution MAVT/MAUT Multi-attribute value/utility theory WSM Weighted Sum Model Table 11: Examples of MCDM Types Evaluated

4.3.3 Evaluation of Alternatives

Using the process of elimination, each identified method, including the examples listed in Table 11, was evaluated against the selection methodology detailed in Figure 8.

Specific criteria are further outlined in Table 12. Requirements for each category were outlined in a decision-making table. The last contenders were then evaluated against additional criteria such as availability of research applications and papers. All non- compensatory methods were eliminated from consideration, eliminating six of the choices. An additional thirteen were eliminated for not meeting the problematic criteria, leaving twelve methods for consideration. Seven were then eliminated for not meeting the preference elicitation criteria, and an additional removed for being non- deterministic in nature. This left four final contenders for consideration: TOPSIS, WSM,

Compromise Programming (CP), and Evaluation of Mixed Data (EVAMIX) method. To come to a final selection, we then applied remaining requirements from Figure 8, such as availability of existing research papers and applications.

Table 12: MCDM Selection Choices for Evaluation Categories

EVAMIX was eliminated because there are not many existing resources explaining its theoretical bases and process. It is not discussed in recent MCDM state of the art papers and does not appear to have published software tools available.

WSM is a common basic MCDM that has been applied in numerous papers [3].

TOPSIS was preferable, because of the distance function. It can inform how tradeoffs intentionally made by the decision maker can impact the resulting solutions ability to meet all objectives and to what degree. This is because the shift in distance from ideal to negative solution can be measured and analyzed.

Between TOPSIS and CP, TOPSIS also had a greater record of successful applications and tools available. Generally, a goal programming category MCDM solution is needed so that we can define objectives and pick the combination closest to the ideal set of objectives making the distance function another preference between these two options.

Note that the MCDM was chosen in relation to developing the methodology, not to developing the list of assets. In both cases, we are not outranking assets or processes, we are picking all that meet criteria and contribute to the objectives of asset identification.

Selected Method: TOPSIS

Based on the evaluation of thirty-one MCDM methods detailed in the previous section, it was concluded that Technique for Order Preference by Similarity to Ideal

Solution (TOPSIS) is the best option. TOPSIS meets all requirements outlined during the requirements phase. The method supports compensatory and deterministic criteria, has the ability to support preference elicitation through direct rating from the user, uses a prior preference approach to decision making and enables the output of a choice in the end of the assessment.

TOPSIS is a form of MCDM first developed by Hwang and Yoon in 1981 [71].

Using this method, the chosen alternative is the option closest to the positive ideal solution (PIS) and farthest from the negative ideal solution (NIS). As required during our evaluation, it is a compensatory method which allows trade-offs between criteria.

Alternatives are compared by assigning weights to criteria, normalizing the scores of each, and then calculating the geometric distance between each alternative and the PIS and NIS. The PIS is the best score in each criterion, while the NIS is the worst score.

4.4.1 Methodology Steps

The basic steps in the TOPSIS process are outline in this section, followed by a hypothetical example that further explains each step in Section 4.4.2.

Step 1

The first step is to create the evaluation matrix that outlines the alternatives (a) and the criteria (c).

() ×

Step 2

The second step in the process requires that the matrix be normalized.

� = () ×

= � √∑=

= ,, … , | = ,, … , Step 3

The third step is to calculate the weighted normalized decision matrix.

= � ∑=

�

= ,, … , ℎ ∑ = = Step 4

The fourth step is to calculate the positive ideal solution and negative ideal solution

� . (� )

��

� − + (� ) = {[�i�( | =,,…,)|∈� ] , [�ax( | =,,…,)|∈� ]}

= {�| = ,,…,}

� − + � = {[�ax( | =,,…,)|∈� ] , [�i�( | =,,…,)|∈� ]}

= {�| = ,,…,} 75

is associated with negative criteria

�− is associated with positive criteria

�+ Step 5

The fifth step is the calculate the distance between each alternative and the NIS

� or the and then calculate the distance between each alternative and the PIS �

� � or the . (� )

�

� � = √∑( − �) , = ,, … , =

� � = √∑( − �) , = ,, … , = Step 6

The sixth step is to calculate the similarity of the distance measure to the PIS. This is denoted as

�

� = � /(� + �), ≤ � ≤, =,,…, = 0 if the alternative has the most negative condition

� =1 if the alternative has the most positive condition

� Step 7

The seventh step is to rank the alternatives by . In other words, . 76 � � = ,,…,

4.4.2 Simplified Methodology Example

To better acquaint readers with the TOPSIS methodology, we provide a fictitious example representing the basic elements of the method. In this example, lets sa a use would like to decide on which of 2 aviation criteria categories should be used to identify critical infrastructure assets. The options are aircraft capacity and aircraft type. Each option can be evaluated against certain criteria. In this case, we use data availability, accuracy, and time to gather data. This constructs the matrix shown in Table 13 and is represented as , where (a) are the alternative options shows as columns, and

represents the data point at each intersection where the columns and rows meet. The decision maker enters scores or rating data for each alternative.

Aircraft Aircraft Type Capacity Data 6 8 Availability Accuracy 5 8 Time to Gather 30 mins 30 mins Data Table 13: TOPSIS Example Step 1 - Matrix

The second step is to normalize the matrix so that all values fall between 1 and 0 which results in the below normalized matrix. As an example, we calculate the normalized value for data availability under the aircraft capacity alternative. The calculation is as follows:

or = .6 6 � 6 √∑= 77

Aircraft Aircraft Capacity Type Data Availability 0.6 0.8 Accuracy 0.52999894 0.847998 Time to Gather Data 0.707106781 0.707107 Table 14: TOPSIS Example Step 2 - Normalized Matrix

The third step is to apply the appropriate weights to establish the weighted decision matrix. It is up to the decision maker(s) to determine the weight of each criteria and its importance. In this case, we determine that data availability has a weight of 50%, accuracy has a weight of 30%, and time to gather data has a weight of 20%. The weights must sum to 100%. As an example, we calculate the weighted normalized value for data availability under the aircraft capacity alternative as (0.6 x 0.5) = 0.3.

Aircraft Aircraft Weight Capacity Type Applied Data 0.3 0.4 0.5 Availability Accuracy 0.158999682 0.254399491 0.3 Time to 0.141421356 0.141421356 0.2 Gather Data Table 15: TOPSIS Example Step 3 - Weighted Normalized Matrix

The fourth step is to deter mine the positive ideal and negative ideal. This is done by selecting the minimum and maximum value of each criteria category.

NIS PIS (minimum) (maximum) Data 0.3 0.4 Availability Accuracy 0.158999682 0.254399491 Time to 0.141421356 0.141421356 Gather Data Table 16: TOPSIS Example Step 4: PIS and NIS Determination

The distance between each alternative and the PIS and the distance between each alternative and the NIS is then calculated in the fifth step. As an example, we will walk through the calculations for the data availability criteria under the aircraft capacity alternative for the PIS distance. In this case, the PIS is 0.4, and the normalized alternative score is 0.3. The example calculation is as follows:

Or = .1 � � � Once, all calculations = √ are∑= complete,( − )the, distance = ,, … matrix , is√ complete. − . as shown in Table 17.

Aircraft Aircraft Capacity Capacity Aircraft Type Aircraft Type distance to distance to distance to distance to PIS NIS PIS NIS Data 0.1 0 0 0.1 Availability Accuracy 0.095399809 0 0 0.095399809 Time to 0 0 0 0 Gather Data Table 17: TOPSIS Example Step 5 - Distance Matrix Establishment

In the sixth step, the similarity to the worse condition is calculated by dividing the NIS distance, by the sum of the NIS distance and PIS distance for each alternative.

Using aircraft capacity and data availability as an example, the calculation is as follows:

� = � /(� + �) ⁄ + . = Aircraft Aircraft Type Capacity Data Availability 0 1 Accuracy 0 1 Time to Gather Data N/a N/a Table 18: TOPSIS Example Step 6 - Similarity to NIS Calculation

When interpreting the outcome at this stage, it helps to understand the following: = 0 if the alternative has the most negative condition

� =1 if the alternative has the most positive condition =Not Applicable� if there is a match across all alternatives, neutralizing the comparison � The seventh and final step of the process is the sum the scores of each alternative and rank them in order of highest score. In this case, aircraft type has the higher score and is considered the preferred alternative with a total of two points.

Original User Rating Final TOPSIS Score

Aircraft Aircraft Aircraft Aircraft Capacity Type Capacity Type Data Availability 6 8 0 1 Accuracy 5 8 0 1 Time to Gather 30 mins 30 mins N/a (cancelled N/a Data due to match) (cancelled due to match) Table 19: TOPSIS Example Step 7 - Best Alternative

4.4.3 Strengths and Weaknesses

TOPSIS has been successfully applied to decision problems in various domains, iludig itial ifastutue. Its ailit to peset the et est alteatie he an ideal solution is not available sets it apart from other MCDM solutions. In addition, the use of distance calculations provides a selection process that is logical and objective.

The method also accommodates trade-offs ad oets the ak eesal liitatio seen in the popular AHP method [72].

One characteristic of TOPSIS that may introduce a weakness when applied to the

CI identification problem is the assumption that criteria are monotonically increasing or decreasing. The solicitation of criteria ratings in our methodology is qualitative to avoid subjectivity in responses and improve reproducibility. To mitigate this weakness, we assign numerical values for each response in the back-end of the tool using a monotonically increasing rating scale of 1-5. This is further explained in Chapter 6.

In summary, a key step to successful application of MCDM to a problem, is to first select the most suitable MCDM method. The appropriate method varies based on the objectives and characteristics of the problem. This chapter described the process for leveraging existing selection methods to create a custom selection method that supports the problem at hand, defined the MCDM problem, and carried out a process of elimination evaluation to determine which MCDM approach should be applied. Given the complex nature of CI asset identification, and specific characteristics calling for

81 specific requirements, such as compensatory and deterministic approaches, we arrive at the conclusion that TOPSIS is the most viable approach to apply to the CI challenge.

5 CHAPTER V

DEVELOPMENT OF A CRITICAL INFRASTRUCTURE ASSET IDENTIFICATION OBJECTIVE-BASED FRAMEWORK

The objective framework represents the culmination of existing research and new contributions gathered and analyzed over several years. The framework is comprised of an objective list, an attribute list, and a series of relationships. The objective list was developed through a review of critical infrastructure definitions from 7 countries across 6 continents, existing program documentation, and policies. Using these resources along with additional research performed by researchers across the globe, an inventory of iteia optios o attiute list as deeloped. I total, over 50 objective options and 80 criteria options were identified. Please note that the terms

iteia ad attiute ae used itehagealy throughout this report.

Once all objectives and attributes were established, analysis was then conducted to construct a mapping of which attributes contributed to the realization of each objective. The result of this process was an objective framework and database that serves as the basis for the decision system.

The purpose of the framework is the guide the development of an asset identification program based on the program objectives. Before delving into the details of each framework component, we summarize the process it supports, shown in Figure

9 and further explained in Table 20. In essence, the objective framework and system resulting from this research and meant to support this identification process.

Identify Determine Select Criticality Objectives and Baseline Asset Criteria Set Scope List

Conduct Obtain Critical Criticality Asset List Assessment

Figure 9: Identification Process

Task Description Associated Framework Components Step 1: Identify During this step, stake holders Scope Objectives and Set consider identification goals relating Political Scope to all components of the framework Accommodation in order to establish an assessment CI Definition (Asset scope and process. This includes, but type, Impact Type, is not limited to determining which Impact Measurement) asset types (e.g. mobile, cyber, etc.) Stakeholder are in scope of the assessment, Sector Identification relevant sectors to be included, etc. Step 2: Determine A recommended approach, (e.g. Baseline Asset List network-based, function-based, logic-based) selected based on objectives gathered from the prior phase, is then used to gather an initial baseline asset list to be evaluated for criticality Step 3: Select Using the objectives established Criticality Criteria during Step 1, appropriate assessment criteria categories and points can be gathered. At this point, the stakeholder should also consider the evaluation details, such as qualification ratio, which is further explained in subsequent sections. Step 4: Conduct The established criteria list and Evaluation Method Criticality Assessment evaluation method are applied to And Criteria the baseline asset list established in Step 2 in order to filter out assets that meet the criticality criteria. Step 5: Obtain Critical This assessment process will result in All (This is a Asset List a list of assets that are critical in culmination of all accordance with the objectives of prior steps and the stakeholder. This list can then completion of feed the risk assessment process, framework the next step in the RMF. Using this components) process, detailed risk assessment can be limited to only critical assets. Table 20: Framework Components to Support Process

The remainder of this chapter details each of the framework components that support the identification process.

Input Sources and Data Gathering

A starting point for setting direction of the assessment and gathering objectives i the deisio sste is the atios foal defiitio of itial ifastutue. Table 21 includes examples of definitions outlined by various countries, with representation from

6 of the 7 continents. These definitions can be analyzed to inform identification efforts.

Nation Asset Focus Concern Impact or Function

United States Systems and assets Incapacity or Destruction debilitating impact

Europe (Member Asset, System, Functions Disruption or destruction; significant States) Essential impact

Australia Physical facilities, supply destroyed, degraded or significantly chains, information rendered unavailable for an impact technologies and extended period communication networks

Canada processes, systems, Essential function facilities, technologies, maintenance networks, assets, services

Kenya Assets Essential function maintenance

Japan Businesses, services, Extremely difficult to be significant functions substituted by others if its impacts function is suspended, deteriorated or become unavailable

Brazil installations, services or destroyed, disrupted, or debilitating assets incapacitated impact Table 21: Objectives Adapted from Global CI Definitions [10]

We use the EU as an example of this definition translation. The EU definition of

itial ifastutue is a asset, sste, o pat thereof located in the Member States which is essential for the maintenance of vital societal functions, health, safety, security, economic or social well-being of people, and the disruption or destruction of which would have significant impact in a Member State as a result of the failure to maintain those futios. [9] The definition requires attention to individual assets, as well as systems and critical functions. The selected methodology should be able to accommodate each of these asset tpes. I additio, the oe is disuptio o destutio that a sigifiatl ipat itial futioal aeas. This iplies the need to incorporate consequence and impact measurements into the methodology, as well as understanding of which functions are critical. By analyzing these definitions, we are able to establish a common list of objectives that are applicable to various countries.

Stakeholders using the resulting decision tool should also use their own definitions, policies, and strategy documents to determine their own program needs.

In addition, identification of criteria categories can be partially derived from the critical infrastructure definitions. Table 22 includes a list of criteria focus areas derived from the definitions for the US, Europe, Australia, Canada, Kenya, Japan, and Brazil. Some appear duplicative, but may have different intents based on the nation. A full listing of the objectives and criteria identified during this phase are detailed in Appendix C.

 National Economic Security  Health  National Public Health and  Safety Safety  Security

 Combination of Criteria  Government function

 Vital societal functions  Society  Economic well-being  Economy  Social well-being  Social lives  National Defense  National Security Table 22: Example Criteria Categories Adapted from Global CI Definitions [10]

Objective Profiles

A descriptive profile was created for each objective. The profile includes a description of the objective and the criteria associated with each. Each unique pair

(objective-criteria) was then assigned a unique identifier and added to the framework database. The profiles are organized in six core objective areas, shown in Figure 10.

Scope

Political Accomodation

CI Definition Asset Type Core Impact Type Objective Impact Measurement Areas Stakeholder

Sector Identification

Evaluation Method

Figure 10: Core Objective Areas in Framework

5.2.1 Scope

The first portion of the framework focuses on understanding the scope. The objective at this stage can either be to conduct a complete or ad-hoc assessment. A complete assessment considers all asset types, while an ad-hoc assessment only covers a partial assessment of a predetermined set of assets. Where the objective is to perform a complete assessment, a function-based approach is recommended though this can also be supplemented by leveraging a network-based approach as a secondary means of identifying assets. Function-based approaches, also referred to as mission-based approaches, begin the identification process by first identifying the functions that are critical to the mission of the organization. Assets that support those functions are then identified and evaluated against other defined criteria [63]. This approach is the primary recommendation because of its greater ability to align with the formal definition of critical infrastructure. Network-based approaches identify all nodes and relationships in the system and use that system mapping as a basis for the evaluation and are more applicable to other objectives such as understanding dependencies or emergency response planning. Alternatively, if an ad-hoc assessment is determined to be the objective, the logic-based approach becomes the primary recommendation while network and function-based approaches may still supplement the assessment. In logic- based approaches, assets are selected based on specific instruction of the assessor, such as named assets, assets within a specific geographical area, or a specific asset type [63].

Figure 11 shows a graphical representation of the scope mapping which has been built into the logic of the CIAid tool.

Figure 11: Scoping Logic Diagram

5.2.2 Political Accommodation

The need to address political concerns in critical infrastructure asset identification has added further complications to the identification process. To address this concern, we enable the decision maker to include this as a clear objective choice upfront. In the past, political concerns have been addressed by adding new threat based identification programs or omitting critical sectors from analysis all together, making

90 programs inconsistent in their missions. By calling out the willingness or unwillingness of accommodating political concerns on the front end of the assessment, the decision maker can commit to building a methodology for or against the notion. Some stakeholders may wish to address political disputes in asset identification by creating an option to name specific assets and/or classes, while some may not. For those that do wish to accommodate political concerns, it is recommended that the logic-based approach be incorporated into the methodology in combination with the optional consideration of key monumental assets. Using the logic-based approach an asset can be specifically named by a public-sector leader with the authority to do so, even when the asset may not meet criticality criteria. For example, key monumental assets sometimes fail to meet criticality criteria, but are still justified by some as critical. The

U.“. defies a ke asset as, idiidual tagets hose attak—in the worst-case scenarios—could result in not only large-scale human casualties and property destruction, but also profound damage to national prestige, morale, and confidence

[73]. This enables one way for the decision maker to consciously consider political concerns and consider assets that may not meet originally defined criteria. Figure 12 shows a graphical representation of the political accommodation mapping which has been built into the logic of the CIAid tool.

Figure 12: Political Accommodation Logic Diagram

5.2.3 CI Definition

The CI definition can be broken down into three main areas: Asset Focus, Impact

Type, and Impact Measurement. Asset focus refer to the type of assets to be considered in the assessment. Impact type refers to the consequence perspectives to be considered and parameters that may have been derived from the formal definition. Impact

92 measurement refers to the dimensions of impact specifically called out in the definition or through other means.

5.2.3.1 Asset Focus

The asset types to be considered can be mobile, fixed, systems, processes, people, services, and/or businesses. In this context, a mobile asset is defined as an asset that does not have a fixed location and can change locations regularly. When the mobile asset type is selected, a function-based approach is recommended with the additional use of sector specific criteria. The function-based approach is recommended because it has the ability to account for mobile asset with greater simplicity when compared to the network-based approach which tends to be applied to fixed assets in the critical infrastructure asset discipline. Sector-specific criteria are defined as special criteria identified for application to a specific critical infrastructure sector or function. Its incorporation is recommended because for each sector there maybe criteria specifically available to help identify mobile assets. An example of this is given using aviation sector case study, specifically discussing aircraft. The capacity of the aircraft, for example, may be a criterion point that can be used to identify critical aircraft in the transportation sector. This is further detailed in Chapter 8.

A fixed asset is defined as an asset with a fixed geographical location. When the fixed asset type is selected, a function-based approach is recommended though as seen in previous instances, it can also be supplemented by leveraging a network-based

93 approach in conjunction with the functional approach. The function-based approach is the primary recommendation because of its ability to align with the CI definition.

A system is defined as two or more connected assets. Where inclusion of systems is designated as an objective the network-based approach is recommended.

Network and system theory have proven to complement each other, making a network- based approached preferable to a function or logic-based approach when aiming to understand systems and the relationships between asset nodes.

Processes, people, services, and businesses are handled similarly in mapping. A process is defined as the nature in which assets are used to deliver functions and steps involved in that delivery. People are defined as human assets that manage the processes. Services are defined as the summation of what assets, systems, processes, and/or people exist to support and provide. Businesses are defined as organizations and entities that provide services and products. If any of the four are selected a function- based approach is recommended. Processes, services, and the people or entities who support them can clearly be translated into functions. A network-based approach would not account for these objectives. Figure 13 shows a graphical representation of the asset scope mapping which has been built into the logic of the CIAid tool.

Figure 13: Asset Focus Logic Diagram

5.2.3.2 Impact Type

The impact type outlined by the stakeholder can be one of destruction or disruption, critical availability, and/or lack of substitutes and alternatives. Destruction or disruption is defined as the loss of an assets ability to fulfill its intended function. When this objective is selected, the development of a worst-case scenario and/or consequence scenario are recommended to showcase the impact that the disruption of the asset can have on the nation. In this context, a worst-case scenario refers to development of a realistic scenario that represents the worst imaginable case in which the infrastructure asset may be degraded or lost, while a consequence scenario refers to

95 development of a realistic scenario that represents an example of the consequences that maybe realized if an infrastructure asset is degraded or lost.

Critical availability is defined as the level of availability required to sustain a function. In addition to recommending development of a worst case and/or consequence scenario for similar reasons outlined above, selecting this objective also recommends incorporating a function based approach as understanding critical function will assist in understanding the level at which functions must be sustained. The stakeholde a oe thooughl oside ho ipotat a assets pefoae is to the associated function.

Lack of substitutes and alternatives refers to lack of backups or alternate means to maintain a function when an asset is destroyed or disrupted. When this is selected, there is simply a flagged reminder that any back up or fail-over functionality should be documented and considered during the identification process. Figure 14 shows a graphical representation of the impact type mapping which has been built into the logic of the CIAid tool.

Figure 14: Impact Type Logic Diagram

5.2.3.3 Impact Measurement

The criteria derived from the CI definition will typically fall into one or more of the following categories:

 Economic Security  Cascading Impacts  Health and Safety  Public Confidence  Dependencies  Social Well-being  National Security  Asset Location  Sector Specific Criteria

 Business Impact  Environmental Impact These categories and the sub-criteria under each were gathered and summed up from numerous programs documents and research reports. Critical resources in this stage included but were not limited to EU EPCIP documents [8] [9], Bouchons vulnerability management research [42], and Fetekes [74] criteria research. Figure 15 and Figure 16 show graphical representations of criteria and sub-criteria mapping which has been built into the logic of the CIAid tool.

Figure 15: Impact Measurement Logic Diagram

Impact measurement diagram component otiued…

Figure 16: Impact Measurement Logic Diagram continued

Economic Security

The first of the categories is economic security. Economic security is defined as the ability of individuals, households or communities to financially cover their essential needs sustainably and with dignity [75]. When this objective is selected, methods of

99 measurement include cost of degraded critical services, cost of restoration of critical services, loss of productivity, and/or percentage of GDP lost.

The cost of degraded services refers to the cost associated with reduction in ability to provide critical function, service, or goods. The EPCIP guidelines further explain that this a e easued ased o the ipat of ifastutue failue o the dynamics of national economies (macro perspective), rather than on individual actors

(micro perspective). In other words, a distinction is made between losses to private actors (often called private or financial losses) and losses to society as a whole (often called social or economic losses). Within the context of evaluating the economic criteria private losses shall not be taken into account, since these losses do not necessarily affet the eoo as a hole. [9] Next the cost of restoration refers to the cost associated with temporarily or fully restoring services to their expected and acceptable performance level. Loss of productivity refers to loss of ability to optimally provide critical function, service, or goods. Lastly, the percentage of GDP lost refers to fraction of the current annual national gross domestic product lost.

Cascading Impacts

Cascading Impacts are defined as a sequence of events in which each produces the circumstances necessary for the initiation or impact of the next [76]. When cascading impacts are in scope of the assessment, the criteria choices recommended include long duration of disruption, impact of geographically widened area, impact on concentrated and specialized industry or service, and critical nodes or points of failure in

100 the impacted network. Long duration of disruption refers to disruption of normal function that exceeds timeframe deemed acceptable by the nation. Impact of geographically widened area refers to disruption of normal function that exceeds a defined geographical area. This could mean impact that crosses country or state borders to impact multiple areas. Impact on concentrated and specialized industry or service is described as disruption of normal function in a specific industry or specific geographical area containing high concentration of assets. Finally, critical nodes or points of failure in the impacted network is defined as disruption of normal function in a core asset that acts as a critical point in a network and can result in a single point of failure for other connected nodes.

Health and Safety

Health and Safety is defined as protecting and improving the health of families and communities through promotion of healthy lifestyles, research for disease and injury prevention and detection and control of infectious diseases. Public Safety refers to the welfare and protection of the general public. The primary goal is prevention and protection of the public from dangers affecting safety such as crimes or disasters [77]

[78] [79]. When health and safety are selected as objectives criteria choices include number of total fatalities, number of prompt fatalities, number of injuries, and physical suffering.

When measuring health and safety impacts, the number of total fatalities is defined as the number of lives lost as a result of impact, including long-term or indirect

101 causalities, while number of prompt fatalities is defined as total lives immediately lost as a direct result of impact. Number of injuries is defined as the total number of people injured as a result of impact. According to EPCIP documentation, an injured person could be seen as a person requiring more than 24 hours of hospitalization [9].

Measurement of physical suffering can be approached in several ways, including but not limited to lack of water, food, heat or energy, sanitary conditions, housing or lodging, and personal security. This can be defined as the number of people impacted in any of these areas due to the disruption or loss of a service or asset.

Public Confidence

Public Confidence is defined as trust bestowed by citizens based on deostatios ad epetatios of: Thei goeets ailit to poide fo thei common defense and economic security and behave consistent with the interests of society; and their critical infrastrutues ailit to poide poduts ad seies at epeted leels ad to ehae osistet ith thei ustoes est iteests [80].

When this objective is selected criteria choices include possibility of rioting, possibility of mass panic or fear, and possibility of stocking up. Possibility of rioting is defined as the likelihood that rioting will occur and number of people impacted by such riots.

Possibility of mass panic or fear is defined as the likelihood of pervasive mass panic or fear and number of people impacted by such panic. Possibility of stocking up refers to the likelihood of requirement to stock up on goods and number of people impacted by such requirement.

102

Dependencies

Dependencies are described as the nature in which one infrastructure asset relies on and/or support another. When this objective is selected the criteria to be considered include physical, cyber, geographical, and logical dependencies [45]. Rinaldi, et al., describe the dependencies as follows:

 Physical dependencies are defined as two infrastructures that are physically

interdependent if the state of each is dependent on the material output of the

other.

 Cyber dependencies occur if the state of an infrastructure depends on information

transmitted through the information infrastructure.

 Geographical dependencies occur when a local environmental event can create state

changes in all of them.

 Logical dependencies occur when two infrastructures are logically interdependent if

the state of each depends on the state of the other via a mechanism that is not a

physical, cyber or geographic connection.

Social Well-being

Social Well-being is defined by United States Institute of Peace (USIP) as an end state in which basic human needs are met and people are able to coexist peacefully in communities with opportunities for advancement. This end state is characterized by equal access to and delivery of basic needs services (water, food, shelter, and health services), the provision of primary and secondary education, the return or resettlement 103 of those displaced by violent conflict, and the restoration of social fabric and community life [81]. When this objective is selected, criteria choices include infringement of freedom to travel, infringement of freedom to leave accommodations, inability to communicate, separation from family or social networks, separation from information resources, unavailability of funds or payment systems, and mass evacuation length.

Infringement of travel is defined as the number of people whose freedom to travel, whether locally or internationally, is impacted by disruption of critical function or asset. Infringement of freedom to leave accommodations refers to the number of people unable to leave accommodations, such as home, office, or another establishment as a result of disruption of critical function or asset. Inability to communicate refers to the number of people unable to exchange information and interact normally as a result of disruption of critical function or asset. Separation from family, social networks refers to the number of people separated from family, loved ones, and/or social networks as a result of disruption. Separation from information resources refers to the number of people separated from key data, information technology and/or electronic records as a result of disruption of a critical function or asset. Unavailability of funds or payment systems refers to the number of people unable to access funds or payment systems such as ATMs, banks, financial assets, etc. as a result of disruption. Lastly, mass evacuation length refers to a length of evacuation exceeding an acceptable timeframe determined by the decision maker.

104

National Security

National Security is described as a collective term for the defense and foreign relations of a country, protection of the interests of a country [82]. If this objective is selected, criteria considerations added include loss of government function and loss of national defense. Loss of government function refers to the disruption of public sector systemic function that results in national impact, while loss of national defenses refers to disruption of atios ability to deploy forces and/or defend a nation from threats.

Asset Location

Asset Location refers to the geographical location of the asset. This can be defined at various levels, such as county, state, country, geographical coordinates, etc.

When this objective is selected, associated attributes include geographical proximity to other assets and concentration of assets.

Geographical proximity to other assets is described as the distance between infrastructure assets determined by nation to be close enough to experience cascading impacts, usually measured in miles. Concentration of assets refers to the volume of infrastructure assets in a defined geographical area

Sector Specific Criteria

Sector Specific Criteria refers to special criteria identified for application to a specific critical infrastructure sector or function. When this objective is selected, specific criteria are recommended based on the sectors in scope of the assessment. Examples

105 include capacity or dimensional criteria. For the purpose of this research, aviation sector criteria are used. The criteria recommended here includes aircraft, airport, air traffic management (ATM), and airline criteria. More details of these criteria categories and their establishment can be found in the Chapter 8.

Business Impact

Business Impact is defined as the impact the loss of the asset has on major businesses. Suggested criteria where understanding business impact is an objective include brand damage, competitive loss, isolated economic loss to the business, loss of projected sales, and loss of business assets. These criteria are meant to help stakeholders measure the potential impact of a lost asset on the business.

Brand damage is described as the negative impact an event, such as a lost asset or function has on the reputability and public confidence in an entity. Competitive loss refers to the reduction in ability to effectively compete in the current marketplace.

Isolated economic loss to the business refers to the financial loss directly resulting from an event, such as the loss of an asset or function. Loss of projected sales refers to the financial loss in terms of projected future sales or income no longer realized as a result of an event, such as the loss of an asset or function. Lastly, loss of business assets refers to the direct loss of asset to the business, and can be measured beyond monetary criteria.

106

Environmental Impact

Environmental Impact is the possible adverse effects caused by a development, industrial, or infrastructural or by the release of a substance in the environment [83].

When this objective is considered in scope of the assessment, the criteria recommended include loss of land and number of people displaced by loss of land. Loss of land is described as the economic value associated with the loss of a defined geographical area.

Per EPCIP [9], the value can be determined by the possible contribution of the use of this land to the national income. Number of people displaced can be defined by the economic effect that the displacement of people has on the national economy. This can include the cost incurred by the nation to relocate the displaced persons (such as shelter, transport, food etc.) and its impact on the national economy.

5.2.4 Stakeholder

For the purpose of this research the main stakeholders considered include infrastructure owners and operators, and public sector decision makers as shown in

Figure 17. Infrastructure owners and operators are described as individuals or entities that own and/or operate infrastructure. Majority of operators are private sector entities. For example, in aviation, commercial airlines operate a large share of the infrastructure. An important point to note is that the main concern of the stakeholder is most likely reliability of the service delivery and profitability. The criticality of assets from their point of view relates to loss of quality, competitiveness, and reliability of the service delivered [42]. When this stakeholder profile is selected, business relevant 107 criteria are recommended and include measurement of brand damage, competitive loss, isolated economic loss to business, loss of projected sales, and business assets.

It was previously mentioned that most CI are owned by the private sector, even if responsibility of protection is owned by the public sector. By incorporating criteria relevant to the private sector, we provide an assessment effort that is relevant to both parties and can better showcase the benefits of participation. Some private sector benefits include understanding of direct impacts to the business, possible funding, better understanding of asset environment and potential risks.

Public sector decision makers are described as individuals responsible for the identification and protection of critical infrastructure assets usually employed or back by the government/public sector. Majority of the methodology, by default already caters to the use of public sector decision makers. As a point of awareness, public sector decision makers should consider ways to incentivize the private sector participation. Examples include providing tax breaks for organizations that participate in identification efforts, awareness of assistance options once critical infrastructure is identified, or more extremely, disclosure to the public that infrastructure owner opted in or out of participation.

108

Figure 17: Stakeholder Logic Diagram

5.2.5 Sector Identification

The sectors named in the database were derived from the current list of US critical infrastructure sectors as it considers common sectors considered critical to several countries. While all sectors are listed and described here to showcase the applicability of the research in various areas, for this research, the focus in outlining criteria is on the aviation sector, a sub-sector of the transportation sector.

Note that these are definitions adapted from the descriptions use by the US

Department of Homeland Security.

109

5.2.5.1 Lifeline Sectors

The lifeline sectors include Communications, Emergency Services, Energy,

Transportation, and Water Systems. The Communications sector is the provider of voice services and additional connectivity through terrestrial, satellite, and wireless transmission systems. The transmission of these services has become interconnected; satellite, wireless, and wireline providers depend on each other to carry and terminate their traffic and companies routinely share facilities and technology to ensure interoperability [84].

The Emergency Services sector is comprised of a system of prevention, preparedness, response, and recovery elements that represent the nation's first line of defense in the prevention and mitigation of risk from both intentional and unintentional manmade incidents, as well as from natural disasters [84]. The Energy sector is divided into three interrelated segments: electricity, oil, and natural gas. The reliance of virtually all industries on electric power and fuels means that all sectors have some dependence on this sector [84].

The Transportation Systems sector can be organized in the following categories:

1.) Aviation includes aircraft, air traffic management (ATM) systems, airports, heliports, and landing strips. This includes commercial aviation services at civil and joint-use military airports, heliports, and sea plane bases. In addition, the aviation mode includes commercial and recreational aircraft (manned and unmanned) and a wide-variety of support services, such as aircraft repair stations, fueling facilities, navigation aids, and

110 flight schools. 2.) Highway and Motor Carrier encompasses roadways, bridges, and tunnels. Vehicles include trucks, including those carrying hazardous materials; other commercial vehicles, including commercial motor coaches and school buses; vehicle and driver licensing systems; traffic management systems; and cyber systems used for operational management. 3.) Maritime Transportation System consists of miles of coastline, ports, waterways, and intermodal landside connections that allow the various modes of transportation to move people and goods to, from, and on the water. 4.) Mass

Transit and Passenger Rail includes terminals, operational systems, and supporting infrastructure for passenger services by transit buses, trolleybuses, monorail, heavy rail—also known as subways or metros—light rail, passenger rail, and vanpool/rideshare. 5.) Pipeline Systems consist of miles of pipelines spanning the country. Above-ground assets, such as compressor stations and pumping stations, are also included. 6.) Freight Rail consists of carriers, railroads, freight cars, and locomotives.

7.) Postal and Shipping includes large integrated carriers, regional and local courier services, mail services, mail management firms, and chartered and delivery services

[84]. The waste and water systems sector includes public drinking water systems and wastewater treatment systems used to provide safe drinking water [84].

5.2.5.2 Other Critical Sectors

Other sectors that can be considered critical include Chemical, Commercial

Facilities, Critical Manufacturing, Dams, Defense Industrial Base, Financial Services, Food and Agriculture, Government Facilities, Healthcare and Public Health, Information

111

Technology, Nuclear Reactors, Materials, and Waste. The Chemical sector can be divided into five main segments, based on the end product produced and include basic chemicals, specialty chemicals, agricultural chemicals, pharmaceuticals, and consumer products [84].

The Commercial Facilities sector includes a diverse range of sites that draw large crowds of people for shopping, business, entertainment, or lodging. Facilities within the sector operate on the principle of open public access, meaning that the general public can move freely without the deterrent of highly visible security barriers. The majority of these facilities are privately owned and operated, with minimal interaction with the federal government and other regulatory entities [84]. The Critical Manufacturing sector identified several industries to serve as the core of the sector: 1.) Primary Metals

Manufacturing (e.g. iron and steel mills and ferro alloy manufacturing, alumina and aluminum Production and processing, nonferrous metal production and processing 2.)

Machinery manufacturing (e.g. engine and turbine manufacturing, power transmission equipment manufacturing, earth moving, mining, agricultural, and construction equipment manufacturing 3.) Electrical equipment, appliance, and component manufacturing, electric motor manufacturing, transformer manufacturing, and generator manufacturing 3.) Transportation Equipment Manufacturing (e.g. Vehicles and commercial ships manufacturing, aerospace products and parts manufacturing, locomotives, railroad and transit cars, and rail track equipment manufacturing. Products made by these manufacturing industries are essential to many other critical infrastructure sectors [84]. 112

The Dams sector delivers critical water retention and control services, including hydroelectric power generation, municipal and industrial water supplies, agricultural irrigation, sediment and flood control, river navigation for inland bulk shipping, industrial waste management, and recreation [84]. The Defense Industrial Base sector is summarized as an industrial complex that enables research and development, as well as design, production, delivery, and maintenance of military weapons systems, subsystems, and components or parts, to meet national military requirements [84]. The

Financial Services sector includes depository institutions, providers of investment products, insurance companies, other credit and financing organizations, and the providers of the critical financial utilities and services that support these functions [84].

The Food and Agriculture Sector includes farms, restaurants, registered food manufacturing, processing, and storage facilities [84]. The Government Facilities sector includes a wide variety of buildings, that are owned or leased by federal, state, local, and tribal governments. Many government facilities are open to the public for business activities, commercial transactions, or recreational activities while others that are not open to the public contain highly sensitive information, materials, processes, and equipment. These facilities include general-use office buildings and special-use military installations, embassies, courthouses, national laboratories, and structures that may house critical equipment, systems, networks, and functions. In addition to physical structures, the sector includes cyber elements that contribute to the protection of sector assets (e.g., access control systems and closed-circuit television systems) as well as individuals who perform essential functions or possess tactical, operational, or 113 strategic knowledge [84]. The Healthcare and Public Health sector protects all sectors of the economy from hazards such as terrorism, infectious disease outbreaks, and natural disasters [84].

In the Information Technology sector, businesses, governments, academia, and private citizens are increasingly dependent upon Information Technology functions.

These virtual and distributed functions produce and provide hardware, software, and information technology systems and services, and—in collaboration with the

Communications Sector—the Internet [84]. Nuclear power can contribute significantly to national electrical generation. This sector is usually comprised of: nuclear power plants, non-power nuclear reactors used for research, testing, and training, manufacturers of nuclear reactors or components, radioactive materials used primarily in medical, industrial, and academic settings, nuclear fuel cycle facilities, decommissioned nuclear power reactors, transportation, storage, and disposal of nuclear and radioactive waste [84].

5.2.6 Evaluation Method

The evaluation method is also a component of the decision process. There are three areas requiring consideration: qualification ratio, identification levelling, and treatment of vectors and victims.

The qualification ratio is defined as the percentage of all relevant criteria that needs to be met in order to qualify an asset as critical. Some programs, require that

100% of criteria be met, while others may accept 75% or 50%. This is up to the decision 114 maker to decide. Furthermore, the decision maker may want to incorporate tradeoffs of supplementary assessments. This means that where an asset may not meet the percentage of criteria necessary, there is still the functionality of combined qualifying criteria to meet the critical threshold. For example, consider that the criteria being applied to an asset to determine criticality include loss of greater than $1 million and a fatalit out geate tha . No lets sta its ee deteied that the loss of a asset would result in a loss of $500k and a fatality count of 2,000. With supplementary assessments, the stakeholder may say that hile the gie asset doest eet oth criteria categories, the death toll is high enough alone to qualify the asset as critical.

Identification leveling or thresholds can be used to establish tiers within criteria categories. This can be useful for entities that would like to distinguish highly critical assets, from assets that are critical albeit at a lower level. From example, an entity may consider an airport that transports 70 million passengers per year to be a Tier 1 critical airport, while designating an airport that transports 30 million passengers per year as

Tier 2 critical. Both airports may be seen as critical based on the entities objectives, but one is of greater criticality.

Lastly, it is important to be aware of the treatment of assets as vectors vs. victims, especially in identification assessment that require worst case or consequence scenarios be developed. This concept is often applicable to mobile assets. Evaluating an asset as an attack vector may provide different outlook when compared to evaluating the asset as a victim. Both views are important to consider in the evaluation phase. For

115 eaple, the loss of a aiaft i a isolated iti seaio ould geeall iflit less damage than an aircraft used as a eto of attak.

Conclusion

The framework covers six core objective areas that can guide the organization of the identification methodology. By setting the scope of the assessment, considering how to handle political concerns, analyzing the CI definition, identifying relevant stakeholders, outlining the appropriate sectors, and establishing the evaluation method, an assessment methodology that meets objectives can be derived. Based on the objectives selected, the discussed assessment methods and approaches should be considered for use.

116

6 CHAPTER VI

SYSTEM ENGINEERING FOR USER-FRIENDLY CIAID TOOL

The purpose of the Critical Infrastructure Asset Identification Decision (CIAid) tool, is to aid nations in developing critical asset identification programs through an objective based decision system. This essentially translates the framework detailed in the preceding chapter, into a tangible system that provides recommendations on the scope, approach, and criteria that can be used to identify critical infrastructure assets.

The system is based on multi-criteria decision making (MCDM) methodology, and specifically leverages TOPSIS MCDM to select the best alternative method where user constraints exist. We recognize that each nation may have unique characteristics and capabilities, and as such allow nations to tailor programs to meet their needs, while understanding the deviation from the recommended methodology. The section begins with an introduction to the system engineering process, user profiles, and use cases for the system. A deeper dive into the development of the database, system functions, and graphical user interface is then covered.

117

System Engineering Process

System engineering is described as the function of engineering complex systems

[85]. The system lifecycle includes three phases: concept development, engineering development, and post-development as seen in Figure 18.

Post Development •Production Engineering •Operations and Support Development Concept •Advanced Development Development •Engineering Design •Integration and •Needs Analysis Evaluation •Concept Exploration •Concept Definition

Figure 18: System Life Cycle Model [85]

The concept development phase starts off with need analysis in order to confirm that there is indeed a need for the system as well as determining exactly what those needs are. Needs are then transformed into system requirements. The next steps in the concept development phase are concept exploration and concept definition. During these steps the engineer brainstorms potential techniques for meeting the system needs and then conducts analysis to select the preferred concept. Next, the engineering development phase includes advanced development, engineering design, and integration and evaluation. During the advanced development phase, development risks are identified and system specification outlined. In the engineering design phase, 118 component build and testing is done followed by integration of all components and system testing in the next phase. Finally, in the post development phase the system transitions to production where it begins operation and ongoing system support [85].

System Overview

Before the system development process is described, a full process flow of the system is shown in Figure 19 to provide upfront context around each of the sub-sections to follow.

Figure 19: End to End CIAid Process

119

System Lifecycle: Concept Development

6.3.1 Needs Analysis

Requirements were gathered through the literature review and analysis of current challenges associated with CI asset identification detailed in Chapters 2 and 5.

Through this process, the following conclusions were drawn to develop the concept:

Users should have the ability to incorporate objectives that are relevant to their efforts and disregard those that are not. The system should be able to recommend a methodology based on the objectives supplied by the user with minimal additional burden on the user. The recommendation should tell the user which scoping approach to take and which criteria categories should be used. The user should then be able to review the objectives, consider their own capabilities and limitations, and edit the recommendations to further reflect their reality. The user should be able to see how this new customized solution compares to their current process, as well as the methodology originally recommended by the system. If several alternatives are compared they should be ranked and presented to the user.

6.3.2 User Analysis

The anticipated users of the system will be public sector organizations, researchers, and private sector companies operating critical infrastructure assets.

120

 Public Sector Organizations: The system can be used by public sector entities to

establish new identification programs, evaluate their current programs, and/or

compare and select a method that best meets their objectives and means.

 Researchers: The system can be used by researchers to evaluate existing programs,

expand the system to include other sectors and criteria, and to gain understanding

of ways to currently identify critical infrastructure assets.

 Private Sector Organizations: The system can be used by private sector organizations

operating critical infrastructure to understand what role they may play in identifying

those assets and find some common incentives to help with the process. The

organization should be able to not only consider the impact of a lost asset on the

nation, but also outline the direct impact to the business as well.

System Lifecycle: Engineering Development

Using the output from the concept development phase and the new objective framework, a database and a functional definition of the system were established to form the CIAid application.

6.4.1 Database Development

The objective framework introduced in Chapter 5 serves as the foundation of the mapping process. Using the framework, a database was created in Microsoft Excel and serves as the basis of the system.

121

To create the database, we first created an initial list of objectives and created a unique identifier for each objective, resulting in a database containing 50 records. In parallel, we created a database of all unique attributes and assigned a unique identifier for each, resulting in 80 additional records. Using the framework mapping logic, we merged these lists to reflect each objective and all of the attributes associated with each objective. The result was the database of 99 objective-attribute relationships. The full database can be found in Appendix D. The database columns are as follows:

 Combined ID1

 Associated Objective ID

 Attribute ID

 Attribute Title

An example of the database elements is shown in Table 23. In this case the objective is 3.ef, or the code of the social well-being criteria category. Each row represents a unique attribute associated with this objective and each has a unique ID.

The objective code and attribute ID are combined to produce an amalgamated ID for each objective-attiute elatioship sho i the Coied ID olu.

1 Note that the Combined ID, pairs the Associate Objective ID and Attribute ID, to form a unique relationship ID.

122

Combined Associated Attribute Title ID Objective ID 3.ef_F.a 3.ef F.a Infringement of freedom to travel 3.ef_F.b 3.ef F.b Infringement of freedom to leave accommodations 3.ef_F.c 3.ef F.c Inability to communicate 3.ef_F.d 3.ef F.d Separation from family, social networks 3.ef_F.e 3.ef F.e Separation from information resources 3.ef_F.f 3.ef F.f Unavailability of funds or payment systems 3.ef_F.g 3.ef F.g Mass evacuation length Table 23: Example Database Listing for Individual Objective

This database supports all areas of the system process described in the next section. For example, when the user input is collected, it is translated into a list of objective codes. During the mapping process, all attribute IDs associated with the objective codes are collected. This list is then used to develop the recommended methodology presented to the user. This is all sourced from the main objective framework database.

6.4.2 System Development

The futioal defiitio is desied i the fo of iput, poess, output.

Using this functional definition, the engineering requirements were outlined and the system developed using Microsoft Excel, macros, and Visual Basic.

OUTPUT FUNCTION INPUT (Recommended Critical (Methodology Mapping (User Objectives) Asset Identification and TOPSIS Application) Methodology)

Figure 20: CIAid Functional Definition

123

6.4.2.1 Input

The input for this system is derived from the user through a series of questions.

Ultimately, the decision factors and common objective options for recommending a methodology include identification of assessment scope, desire to accommodate political concerns, asset types, impacts and areas of concern, stakeholder type, and sector scope. Each question presents multiples choices described in Table 24. As the user selects the objectives, the objective ID associated with each objective is collected.

Question Choices Rules 1 What is the targeted scope of your Complete Assessment Select one assessment? Ad-hoc Assessment 2 Do you wish to accommodate Yes Select one political concerns? No 3 Which asset types are of concern? Mobile Assets Select all System Assets that apply Fixed Assets Processes Services Businesses 4 Which impact types are of Destruction or Disruption Select all concern? Critical Availability that apply Lack of Alternatives 5 Which consequences should be Economic Security Select all measured? Cascading Impacts that apply Health and Safety Public Confidence Dependencies Social Well-being National Security Asset Location Sector Specific Criteria Business Impact Environmental Impact 6 Who is the main stakeholder? Infrastructure Owners and Operators Select one Public Sector Decision Makers 7 Which sectors are in scope? Transportation Systems – Aviation Select all *This used for the current model, other that apply sectors should be incorporated in future* Table 24: Objective Solicitation Question

124

6.4.2.2 Function and Output

Once the user objectives are gathered, the mapping process is applied in the background to present a recommended methodology. If the user wishes to customize the recommended method, they can do so before being presented a final recommendation. These steps are referred to as methodology mapping and then alternative evaluation using the TOPSIS application, and are further described in this section.

6.4.2.2.1 CIAid: Methodology Mapping

All objective IDs collected from the input phase are populated into one

Ojetie List. Fo the framework, we know that each objective is associated with several supporting criteria points. These criteria points have each been coded and included in an object framework database. Refer to Figure 11 to Figure 17 in Chapter 6 for diagrams detailing the mapping and logic for each question.

Using the logic explained in these diagrams, a list of recommended attributes is presented to the user. We understand that the user may wish to customize the methodology due to preference, limitations, or other reasons. For example, a nations critical infrastructure definition may require identification of asset that if lost could produce environmental impact. The user may review the recommended criteria and determine that the data required to measure environmental impact may not be available currently. This is a constraint of the user. The user can choose to remove that 125 criteria category from the recommendation and evaluate this reduced method as an alternative. When the user desires to take this approach, the TOPSIS method is applied to evaluate and select an alternative. This option can also be used to compare the users current program to whatever is recommended by the CIAid tool.

6.4.2.2.2 CIAid: TOPSIS Application for Alternative Comparison

The TOPSIS multi-criteria process is applied when the user would like to compare alternatives to the recommended methodology. We first walk through a dry version of the methodology and then provide an example using the CIAid tool. The steps of the

TOPSIS methodology are followed as directed by the original methodology, with some additional enhancements incorporated to address potential user driven errors described in Step 6 of Section 6.4.2.2.3. Column represents the methodology

recommended by the tool. Upon initiating( |the = comparison ,,…,) function, this option is static once submitted and should not be changed during the customization process. There are seven steps in the evaluation process outlined below.

Step 1

The first step is to create the evaluation matrix that outlines the alternatives (a) and the criteria (c).

() × In this case is where the criteria and alternatives in each cell of the matrix

intersect. In the tool, this matrix is populated through user input of the alternatives and

126 associated criteria. The user is able to select Yes or No to include or remove recommended methodology criteria (c) for up to three alternatives (a). These alternatives can be new alternatives constructed by the user, as well as the current program in place at the organization. Once all alternatives are entered, these new criteria selections are compared against the originally recommended criteria. There are

4 potential results to this comparison.

If a value of 5 is assigned. In Microsoft Excel, this is sho as a YE“-

YE“ elatioship. = This eas that the iteio is iluded i oth the oigiall recommended methodology and in the alternative being evaluated. If the

value of 3 is assigned. In Excel, this is sho as a NO-YE“ elatioship. This< eas that the criterion is not included in the originally recommended methodology, but is included in the alternative being evaluated. If the value of 1 is assigned. In Excel, this is

sho as a YE“-NO elatioship. I this > ase, the iteio is iluded i the recommended methodology, but is no longer included in the alternative being

osideed. A atig of NO-NO is disegadig as this eas it as ot recommended to be included in the methodology and is not being considered as an alternative making its inclusion in the evaluation unnecessary. The scoring scheme is summarized in Table 25.

This automatic rating scale of 0-5 was chosen to maintain simplicity and also translate qualitative responses into quantitative analysis data. The result is a matrix of values that continue through the TOPSIS process.

127

Copariso Descriptio Coditio Value Match OR YE“-YE“ 5

Ecess OR NO-YE“ 3 = Deficiec OR YE“-NO < Null NO-NO Table 25: TOPSIS > Scoring Scheme

Step 2

The second step in the process requires that the matrix be normalized. The matrix is normalized using this formula in the back end of the tool to result in normalized matrix with all values ranging between 0 and 1.

� = () ×

= � √∑=

= ,, … , | = ,, … , Step 3

The third step is to calculate the weighted normalized decision matrix. Currently, the criteria weights are evenly distributed as weights are already automatically incorporated into construction of matrix in Step 1. Future research may entail allowing users to assign their own weights to each criterion. For now, to maintain simplicity and a reasonable learning curve in the tools initial introduction, this feature is not yet included.

128

= � ∑=

�

= ,, … , ℎ ∑ = =

Step 4

The fourth step is to calculate the positive ideal solution and negative ideal

� solution . (� )

��

(��) = { [�ax(| =,,…,)|∈�+]} = {�| = ,,…,}

�� = { [�i�(| =,,…,)|∈�+]} = {�| = ,,…,} In the CIAid tool, there currently are not any criteria considered to be negative.

As such, the positive min and max formulas have been incorporated into the tool.

Step 5

The fifth step is the calculate the distance between each alternative and the

NIS or the and then calculate the distance between each alternative and the

� � PIS � or the .

(��) �

� � = √∑( − �) , = ,, … , =

129

� � = √∑( − �) , = ,, … , = Step 6

The sixth step is to calculate the similarity of the distance measure to the PIS.

This is denoted as

�

� = � /(� + �), ≤ � ≤, =,,…, = 0 if the alternative has the most negative condition

� =1 if the alternative has the most positive condition

�

Step 7

The seventh step is to sort the alternatives by . In other words,

� � . The output of the process is a best alternative methodology. Once the = alternative,, … , is agreed upon the user can leverage a user guide to further develop the program and process.

6.4.2.2.3 TOPSIS Use Case

Scenario: In this example, we will be determining the best alternative when comparing four alternatives including the NCIPP program, the solution recommended by the CIAid tool, and 2 user-generated alternatives (A and B) based on constraints. Using

130 the current definition of critical infrastructure in the US, objectives are selected and the

TOPSIS method applied.

Objective Definition

In the U.S., critical infrastructure is described as any assets, systems, and networks, whether physical or virtual, so vital to the United States that their incapacitation or destruction would have a debilitating effect on security, national economic security, national public health or safety, or any combination thereof [1].

Using this definition, we conclude that a complete assessment is desired and that mobile assets, fixed assets, and systems should be considered. We also select economic security, cascading impacts, health and safety, dependencies, and national security as categories consequence measurements. Since the case study focuses specifically on aviation, that is the sector selected for the scope. In real world applications, additional sectors would be considered.

Step 1

The first step is to create the evaluation matrix that outlines the alternatives and criteria ( ). In this example, the alternatives are: CIAid, Alternative A,

Alternative( B,) and × NCIPP. Each of the objectives are translated into the categories listed as rows in Table 26.

To populate the matrix, a methodology is first established and recommended by

CIAid based on the mappings of the objective framework. Alternative A and B are

131 hypothetical examples of what may be added by a user and would reflect alternative programs the user is considering adopting. NCIPP reflects the current program used in the United States.

132

CIAid ( Alternative Alternative B NCIPP A ( ) ( ) ( ) Network-based YES (5) � NO (1) YES (5) YES (5) Function-based YES (5) NO (1) � YES (5) � NO (1)� Logic-based NO (1) YES (3) YES (3) YES (3) Economic Security YES (5) YES (5) YES (5) YES (5) Cascading Impacts YES (5) YES (5) NO (1) YES (5) Health and Safety YES (5) YES (5) YES (5) YES (5) Public Confidence NO (1) NO (1) YES (3) NO (1) Dependencies YES (5) NO (1) YES (5) NO (1) Social Well-being NO (1) NO (1) NO (1) NO (1) National Security YES (5) YES (5) YES (5) YES (5) Asset Location NO (1) NO (1) YES (3) YES (3) Sector Specific Criteria- Transportation Systems - Aviation YES (5) NO (1) NO (1) NO (1) Business Impact NO (1) NO (1) NO (1) NO (1) Environmental Impact NO (1) NO (1) NO (1) NO (1) Evaluation Options YES (5) NO (1) NO (1) YES (5) Table 26: TOPSIS Example - User Ratings

A comparison of each alternative is done against the CIAid recommendation.

Based on the comparison the ratings and weights are automatically assigned as described in the following paragraph. This logic is hardcoded into the tool to increase simplicity and objectivity in user responses. There are four possible outcomes from this comparison:

 Match: If the CIAid recommendation matches the alternative selected by the user

( ) a value of 5 is assigned. In the backend of the tool the result is coded as

133

a YE“-YE“ elatioship. This eas that the iteio is iluded i oth the

originally recommended methodology and in the alternative being evaluated.

 Excess: If the criterion is not included in the originally recommended methodology,

but is included in the alternative being evaluated or ( ) the value of 3 is

assigned. In the tool this is coded as a NO-YE“ elatioship. <

 Deficiency: If the criterion is included in the recommended methodology, but is no

longer included in the alternative being considered ( ) the value of 1 is

assigned. In the tool this is coded as a YE“-NO elatioship. >

 Null: A atig of NO-NO is disegaded as this eas it as ot eoeded to

be included in the methodology and is not being considered as an alternative making

its inclusion in the evaluation unnecessary. To complete the matrix, these spaces are

assigned a value of 0.

Step 2-3

The second and third steps in the process require that the matrix be normalized

( ) and weighted ( ). The matrix is normalized using

this� =formula ( ) in ×the back end of the �tool = to ( result) ×in normalized matrix with all values ranging between 0 and 1.

= � √∑= Currently, the criteria weights are evenly distributed because weights are already automatically incorporated in the first step. Future research may entail allowing

134 users to assign their own weights to each criterion. For now, to maintain simplicity and a reasonable learning curve in the tools initial introduction, this feature is not yet included. As such, steps 2 and 3 are combined making ( ) equivalent to

( ). � = ( ) ×

� = () × CIAid AltA AltB NCIPP Network-based 0.939557535 0.037582301 0.939557535 0.939557535 Function-based 0.939557535 0.037582301 0.939557535 0.037582301 Logic-based 0.037582301 0.338240713 0.338240713 0.338240713 Economic Security 0.939557535 0.939557535 0.939557535 0.939557535 Cascading Impacts 0.939557535 0.939557535 0.037582301 0.939557535 Health and Safety 0.939557535 0.939557535 0.939557535 0.939557535 Public Confidence 0.037582301 0.037582301 0.338240713 0.037582301 Dependencies 0.939557535 0.037582301 0.939557535 0.037582301 Social Well-being 0.037582301 0.037582301 0.037582301 0.037582301 National Security 0.939557535 0.939557535 0.939557535 0.939557535 Asset Location 0.037582301 0.037582301 0.338240713 0.338240713 Sector Specific 0.939557535 0.037582301 0.037582301 0.037582301 Criteria- Transportation- Aviation Business Impact 0.037582301 0.037582301 0.037582301 0.037582301 Environmental Impact 0.037582301 0.037582301 0.037582301 0.037582301 Evaluation Options 0.939557535 0.037582301 0.037582301 0.939557535 Table 27: TOPSIS Example- Normalized and Weighted Matrix

135

Step 4

The fourth step is to calculate the positive ideal solution and negative ideal solution

� . The minimum and maximum are taken from each criteria(� ) category to establish the

� criteria� level PIS and NIS.

(��) = { [�ax(| =,,…,)|∈�+]} = {�| = ,,…,}

�� = { [�i�(| =,,…,)|∈�+]} = {�| = ,,…,} NIS PIS Network-based 0.037582301 0.939557535 �� (��) Function-based 0.037582301 0.939557535 Logic-based 0.037582301 0.338240713 Economic Security 0.939557535 0.939557535 Cascading Impacts 0.037582301 0.939557535 Health and Safety 0.939557535 0.939557535 Public Confidence 0.037582301 0.338240713 Dependencies 0.037582301 0.939557535 Social Well-being 0.037582301 0.037582301 National Security 0.939557535 0.939557535 Asset Location 0.037582301 0.338240713 Sector Specific 0.037582301 0.939557535 Criteria- Transportation Systems - Aviation Business Impact 0.037582301 0.037582301 Environmental 0.037582301 0.037582301 Impact Evaluation Options 0.037582301 0.939557535 Table 28: TOPSIS Example - NIS and PIS

136

Step 5

The fifth step is the calculate the distance between each alternative and the

NIS or the and then calculate the distance between each alternative and the

� � PIS � or the .

(��) �

� � = √∑( − �) , = ,, … , =

Table 29: TOPSIS Examples - Distance Calculations

137

Step 6

The sixth step is to calculate the similarity of the distance measure to the NIS.

This is denoted as 2. Results of each alternative are summed together to produce a

� final score.

� = � /(� + �), ≤ � ≤, =,,…,

Using the traditional TOPSIS method we found a limitation in the methodologies ability to account for users who may boost their scores by including multiple criteria not recommended by the CIAid tool. To address this issue, we add an additional calculation.

For each criterion, the scores are compared against the recommendation from the CIAid tool. Where the criteria were recommended, but not included in alternative, .5 points are deducted from the overall score to denote this deficiency. The criteria are not required and should not receive the full point granted by TOPSIS. Additionally, to ensure thee is a oplete ueial ati epeseted at all ties, a Not Appliale ratings are replaced with 1. With this step, each alternative increases by an equal number of points. It does not change the distance calculation, but simply evens out the matrix.

2 = 0 if the alternative has the most negative condition

�=1 if the alternative has the most positive condition �=Not Applicable if there is a match across all alternatives, neutralizing the comparison �

138

CIAid Alt A Alt B NCIPP

Network-based 1 � 0 � 1 � 1 � Function- 1 0 1 0 based Logic-based 0 1 1 1 Economic Not Applicable Not Applicable Not Applicable Not Applicable Security Cascading 1 1 0 1 Impacts Health and Not Applicable Not Applicable Not Applicable Not Applicable Safety Public 0 0 1 0 Confidence Dependencies 1 0 1 0 Social Well- Not Applicable Not Applicable Not Applicable Not Applicable being National Not Applicable Not Applicable Not Applicable Not Applicable Security Asset Location 0 0 1 1 Sector Specific Criteria- Transportation 1 0 0 0 Systems - Aviation Business Not Applicable Not Applicable Not Applicable Not Applicable Impact Environmental Not Applicable Not Applicable Not Applicable Not Applicable Impact Evaluation 1 0 0 1 Options Sum 6 2 6 5 Table 30:TOPSIS Example - Distance Similarity Calculation �

139

CIAid Score Alternative A Score Alternative B Score NCIPP Score TOPSIS Offset TOPSIS Offset TOPSIS Offset TOPSIS Offset 1 0 1 0 1 0 1 0 0 -0.5 1 0 0 -0.5 0 0 1 0 1 0 1 0 N/A 1 N/A 1 N/A 1 N/A 1 1 0 1 0 0 -0.5 1 0 N/A 1 N/A 1 N/A 1 N/A 1 0 0 0 0 1 0 0 0 1 0 0 -0.5 1 0 0 -0.5 N/A 1 N/A 1 N/A 1 N/A 1 N/A 1 N/A 1 N/A 1 N/A 1 0 0 0 0 1 0 1 0 1 0 0 -0.5 0 -0.5 0 -0.5 N/A 1 N/A 1 N/A 1 N/A 1 N/A 1 N/A 1 N/A 1 N/A 1 1 0 0 -0.5 0 -0.5 1 0 6 6 2 4 6 4.5 5 4.5 12 6 10.5 9.5 Table 31:TOPSIS Example - Distance Calculation with Custom Offset

Step 7

The seventh step is to sort the alternatives by . In other words,

� � . =

,, … , In this example, the sorting results are as follows:

1. CIAid = 12 2. Alternative B = 10.5 3. NCIPP = 9.5 4. Alternative A = 6

140

The user can conclude that the best alternative to the recommended solution is

Alternative B. Additionally, we find that the current program is 2.5 points below the

CIAid recommendation.

6.4.3 CIAid Graphical User Interface

To improve ease of use and ensure that the tool is user-friendly, the model was augmented with a simple graphical user interface (GUI) and a user guide included in

Appendix E. Decision makers are able to walk through a series of tabs presenting user friendly information while the more complex objective framework and TOPSIS application run in the background. The database, model, and GUI were built in Microsoft

Excel, with integration of Visual Basic and Excel Macros. The user-facing tabs are explained in this section.

Tab 1: Home Page

The system starts with an introductory page that describes the tool, the benefits of use, and potential users, followed by high-level instructions on how to use the tool.

Instructions:

1. Review objective descriptions and select preferences

2. Review methodology recommendations

3. Implement recommended methodology or choose to customize and compare

alternative methods

4. If choosing to customize, input alternatives and minimum acceptable criteria

141

5. Select alternative with greatest score and implement

Figure 21: CIAid Homepage

A START button then leads into the next page of the tool. The macro code is shown in Figure 22.

Figure 22:Homepage Macro

142

Tab 2: User Objective Solicitation (Model)

A series of seven questions is posed to the user, each with choices available to either select one or multiple responses. As the user submits choices, the objective ID list is being populated in hidden fields between COLUMN C-F.

Figure 23: CIAid User Objective Solicitation

143

Figure 24: CIAid User Objective Solicitation cont.

Once selections are made, the user hits the SUBMIT OBJECTIVES button. This initiates a macro that first filters the database against the objective list, creates a list of unique attributes, and transfers the list so that it is presented in Tab 3. The macros enabling this process are shown in Figure 25 and Figure 26.

Figure 25: CIAid Objective Solicitation Macro 1 144

Figure 26:CIAid Objective Solicitation Macro 2

Tab 3: Recommendations

The recommended attributes are presented to the user in the form of a categorized list. The user can then choose to use the CLICK HERE TO CUSTOMIZE button to customize the methodology.

Figure 27: CIAid Recommendations GUI 145

The macro code for this button is shown in Figure 28.

Figure 28: Recommendations Macro

Tab 4: User Selections

The user has the option to add up to three additional choices for comparison, oe of hih a e the uet poga i use the use o the uses ogaizatio.

This is done by selecting YES or NO to include or omit certain criteria categories.

Figure 29: CIAid User Alternative Selections

146

Once the additional alternatives are entered, the user selects COMPARE

ALTERNATIVES to initiate the comparison process. The macro code for this button is shown in Figure 30. At this point, Steps 2 through 7 of the TOPSIS calculation process occur in the background to normalize the matrix, calculate the PIS and NIS, and determine the required distance measurements.

Figure 30: Alternative Comparison Macro

Tab 5: Output

Finally, the user is presented with a sorted display of all alternatives in the form of a scatter plot as well as a table showing the scores and best alternatives.

Figure 31: CIAid Tool Output

147

6.4.4 Conclusion

The objective framework will require maintenance and updates as the industry evolves. In addition, new research should continuously be incorporated. With this in mind, its recommended that the objective framework be revisited every 6-12 months so that new objectives and attributes can be evaluated and added as necessary. Prior to adding to the database, all new records should be vetted by industry experts and analysis done to define the relationships between the objectives and attributes.

Additionally, the tool should be integrated into an informational website that may also require ongoing maintenance.

Alleviating current challenges associated with critical infrastructure asset identification, a fundamental step in the risk management process, will strengthen risk analysis and improve security. The CIAid system is a starting point to addressing these challenges and provides a simplified way to construct and compare methodologies.

148

7 CHAPTER VII

CIAID VALIDATION, VERIFICATION, AND EVALUATION This body of research has undergone a process of validation, verification, and evaluation through expert input and feedback analysis3. We first define what we mean by each of these terms, and then explain the process for gathering and analyzing the feedback to meet these requirements. The verification, validation, and evaluation of the tool is organized in two parts: 1.) an online survey 2.) in-person interviews.

 Verification (VE): Confirmation that the system was designed and built appropriately

 Validation (VA): Confirmation that the right system was built

 Evaluation (EV): Confirmation that the system is useful

Aviation Criteria Expert Verification

To verify the aviation criteria developed and propose recommendations, a 14- question survey was crafted soliciting input on the criteria categories and ways to utilize them to identify critical infrastructure assets. Prior to launch, the survey was vetted by

3 Note: The IRB process was not required because the validation interviews were conducted to gather feedback on the tool, not the individual participant. All five interviewees provided written consent to include their input in the research report. 149 four aviation and security industry experts and feedback incorporated to improve the survey.

The survey was sent to the Aviation-ISAC community. The Aviation-ISAC is a security information sharing organization comprised of 22 member organizations including airlines, airports, air cargo, manufacturers, industry associations, and more.

Additionally, the survey was shared with critical infrastructure experts via LinkedIn. 63 responses were collected with responders across 7 countries. This feedback was used to generate a report of recommendations and measurement options included in Chapter 8.

The detailed survey questions are included

The key takeaways from the survey are summarized here, and a full detailed report of results can be found in Appendix G.

 Majority of respondents agreed that aircraft type, operational airport and air traffic

management criteria can contribute positively to identification of critical assets, but

there is still much research required in this area

 Freeform comments revealed additional opportunities such as adding e-enabled

aircraft criteria and back office airport network infrastructure to the identification

requirements

 69% of respondents believe that participation of airlines in critical asset

identification efforts should be regulated

150

 31% of respondents disagree with the regulation of airline participation and

alternatively suggest focusing on incentivization, establishing an overarching

international organization responsible for aviation asset security, and more

CIAid Verification and Validation Interviews

To verify, validate, and evaluate this research and the CIAid tool, one on one interviews were completed with 5 industry experts to solicit feedback on 3 core areas:

The Objective Framework, The CIAid TOPSIS Model, The CIAid Tool and GUI. The expert list was carefully generated to include a mix of domestic and international respondents, with at least 7 years of experience in critical infrastructure security and some experience in aviation.

151

7.2.1 Interview Agenda

Time and Topic Interviewer Talking Points Questions Asked to Interviewee

10 mins –  Explain questionnaire and 1. On a scale of 1 to 5, how confident Research review process are you in your ability to provide Background  Purpose of research feedback on this research?  Research tasks completed and progress  Major findings and results

20 mins –  Explain framework 2. On scale of 1 to 5, rate accuracy in Objective development process mapping logic of each of the 7 Framework  Walk-through each mapping diagrams. (VE) flow chart and solicit 3. What can be done to improve the interactive feedback for each objective framework? (VE/VA) (total 7 mappings) 4. Based on review of aviation criteria  Walk-through aviation and input from 60+ industry experts, criteria recommendations rate how confident you are that and verification survey these criteria are valid. (VA) results

20 mins – CIAid  Explain simple example of 5. On scale of 1 to 5, rate theoretical TOPSIS Model how TOPSIS works soundness of applying TOPSIS to this and GUI Tool  Walk through a comparison problem. (VE) Walkthrough scenario in the CIAid tool 6. On scale of 1 to 5, rate confidence in tools ability to aid national security organizations in establishing identification methodology. (VA) 7. Rate likelihood that you would recommend use of this tool to your local national security organizations. (EV) 8. Rate the graphical user interface on appearance. (EV) 9. Rate tool on ease of use. (EV)

10 mins – Q&A  Answer any additional 10. Are there any recommendations for questions improvement? (VE)

 Send user deck and allow 2-3 11. Using Pass or Fail, rate overall body days to complete all feedback of work and contribution of and provide final response dissertation research to the critical form infrastructure protection industry. (VE, VA, EV) 12. Using Pass or Fail, rate overall body of work and contribution of dissertation research to the aviation industry. (VE, VA, EV) Table 32: Validation Interview Plan

152

7.2.2 Interview Candidate Qualifications:

 7+ years of experience in critical infrastructure security; Some aviation

experience required

 At least 1 international participant; At least 4 domestic participants

7.2.3 Selected Expert Interview Candidates:

1. Senior Security Engineer and Consultant at Illiam Consulting – Chicago, IL

2. Executive Director of the Aviation ISAC – Washington, DC.

3. Project Manager Centralized Services and Cyber Security at EUROCONTROL –

Belgium

4. Senior Security Analyst at United Airlines – Chicago, IL

5. Director of IT Governance, Risk and Compliance at United Airlines – Chicago, IL

7.2.4 Interview Results

All expert interview candidates had 7-20 years of experience in security and aviation.

Each expert offered valuable insight and feedback on the research, and rated the overall body of work and contribution to the critical infrastructure protection industry with a

PA““ soe. The results of the interviews are summarized in Table 33. This meets the validation requirements initially outlined in Table 8.

All experts rated the objective framework diagrams at four or above on a 5-point scale and would recommend the use of this tool in their respective national security

153 organizations. The experts also recommended continued expansion of the overall body of work and advancements in the design and simplicity of the tool.

Expert Expert Expert Expert Expert Average 1 2 3 4 5 Grade Expertise Confidence 4 4 5 4 4 4.2 Scoping Logic Diagram 5 5 4 4 5 4.6 Political Logic Diagram 4 5 4 5 4 4.4 Asset Type Logic 4 5 4 4 Diagram 4 4.2 Impact Type Logic 4 5 4 5 Diagram 4 4.4 Impact Measure Logic 5 5 4 4 Diagram 5 4.6 Stakeholder Logic 5 5 4 5 5 4.8 Aviation Criteria Study 5 4 5 4 5 4.6 Use of TOPSIS 4 5 N/A 4 5 4.5 Relevance and 5 4 5 4 applicability of tool 4 4.4 Likelihood to 5 5 5 4 recommend 5 4.8 GUI 4 5 4 4 4 4.2 Tool Simplicity 3 3 4 5 5 4 Overall Score PASS PASS PASS PASS PASS PASS

Table 33: Expert Validation Interview Scores

Success Criteria Evaluation

We sought to develop a method that is complete, reproducible, defensible, and documented. To meet completeness criteria, an in-depth analysis of existing policies, programs, methodologies, international critical infrastructure definitions spanning six continents, and research was conducted to gather and develop the objective framework which serves as the basis of this research. This process is detailed in Chapter 5. To meet

154 the defensibility criteria, the research was validated and affirmed by aviation security experts as detailed throughout this chapter. To meet documentation criteria, the development process and tool details are documented in this dissertation report. To meet reproducibility criteria, the tool was configured with static ratings based on the uses ojeties and a simple, user friend question based GUI developed to ensure results are consistent. The logic diagrams introduced in Chapter 5 are ingrained in the tools functionality, making the output deterministic and repeatable.

155

8 CHAPTER VIII

AVIATION CASE STUDY The apparent oversight of mobile assets in the NCIPP process is becoming increasingly acute with respect to the aviation industry. Certainly, 9/11 justified the enormous investment made in airport physical security, but within the same span of years, aircraft have become increasingly more vulnerable to cyber-attack. Technological enhancements, which were made to improve air traffic management and the flight experience for customers, also introduce new cyber risks. For example, an increase in aircraft network connectivity, which is used for passenger Wi-Fi, aircraft maintenance functions, and NextGen Air Traffic Management [86] [87] leave the aviation sector theoretically vulnerable to cyber-attack.

As a result of this body of research, we conclude that sector specific criteria should be established to address the increasing risks to aviation assets, especially mobile assets. This case study provides background on the aviation sector, discusses fundamental risk and emerging cyber threats, and shares how these problems relate to critical infrastructure protection. We then share recommended sector specific aviation criteria and use this sector to walk through an example of what including sector specific criteria would look like as recommended by the CIAid tool. The example is limited to

156 illustration of aviation specific criteria with primary focus on economic and safety impacts. Future research should be conducted to expand criteria options to cover other impact types.

Background

While security in the aviation industry has long been a topic of discussion amongst policy makers, government officials, research groups, citizens, and private industries, the heaviest emphasis is typically placed on physical security concerns and safety. The new threats surfacing due to the growing reliance on technology and connectivity make security in the aviation industry challenging for stakeholders to manage in an industry that is not historically accustomed to cyber threats [88, p. 4].

8.1.1 A Valuable System of Systems

As of 2013, the national aviation network is comprised of 690 air traffic control facilities, 19,800 general aviation airports, 11,000 air navigation facilities, and 13,000 flight procedures [89, p. 129] [90]. The system also includes 617,128 pilots, 222, 520 general aviation aircraft, and 7,185 air carriers. Additionally, it supports the transportation of over $562 billion in cargo, 728 million passengers annually and is reported to have generated 10 million jobs through related goods and services in 2009

[90]. Globally, the aviation sector is estimated to have a global economic impact of 2.7 trillion and supports 56.6 million jobs worldwide [91] [86].

157

As such, major disruption in this industry can lead to significant impact on the national and global economy. For example, 9/11 is estimated have caused roughly $50 billion in direct costs and $5 trillion in indirect, long-term costs [92] [93]. Another much smaller, but considerable example is seen in reports that the cost of airport and

ogestio delas aloe i ielded a $ illio lo to the atioal eoo. Its estimated that this cost will reach $63 billion within the next 25 years [90]. Lastly, significant disruption in this industry could also have significant local and global impact on the physical and emotional wellbeing of mass populations. For example, the 9/11 attack damaged public confidence in the safety of the aviation sector resulting in a significant reduction in air travel afterward.

Table 34 provides a further breakdown of key functions and services provided by the aviation sector. While this list is not all-inclusive, it provides some context behind the statistical data outlined in this section. Note that aviation also supports functions in other sectors outside of aviation. For example, oil and mineral or agricultural support are provided by the aviation sector.

158

Core Functional Area Sub-functions Emergency Preparedness and  Aeromedical flights Response  Law enforcement/National Security/Border Security  Emergency Response  Aerial Fire Fighting Support  Emergency Diversionary Airport  Disaster Relief and Search and Rescue  Critical Federal Functions Critical Community Access  Remote Population/Island Access  Air Taxi/Charter Services  Essential Scheduled Air Service Cargo Other Aviation Specific  Charter Passenger Services Functions  Aircraft/Avionics Manufacturing/Maintenance  Aircraft Storage  Aerospace Engineering Research  Self-Piloted Business Flights  Corporate  Flight Instruction  Personal Flying Commercial, Industrial, and  Agricultural Support Economic Activities  Aerial Surveying and Observation  Low-orbit Space Launch and Landing  Oil and Mineral Exploration/Survey  Utility/Pipeline Control and Inspection  Business Executive Flight Service  Manufacturing and Distribution  Express Delivery Service Air Cargo Destination and Special  Tourism and Access to Special Events Events  Intermodal Connections (rail/ship)  Special Aeronautical (skydiving/airshows) Table 34: Type of Aeronautical functions serving public interest [94, p. 2]

8.1.2 Key Stakeholders

Key stakeholders in U.S. aviation security include agencies such as Federal

Aviation Administration (FAA), Transportation Security Agency (TSA), and Department of

Homeland Security (DHS). DHS is responsible for developing and implementing a cross- sector risk management approach across all sectors, including the transportation sector and each of its sub-sectors. TSA is responsible for supporting counterterrorism by

159 overseeing security of domestic airports, airport operators, and both domestic and foreign aircraft operators and also contributes to crisis management in the sector. The

FAA is a national civil aviation authority responsible for regulatory management of the national aviation system [89, p. 131]. Other stakeholders include Airlines for America

(A4A), US Department of Transportation (DoT), Joint Coordination Group (JCG),

International Civil Aviation Organization (ICAO), Aviation-ISAC, commercial airlines, original equipment manufacturers, airports, and regulatory bodies. Each of these groups help maintain and contribute to the effort to improve security in civil aviation.

Key Cyber Issues in Aviation

Some notable concerns regarding aviation risk include: the perception of safety vs. security, aging technology and introduction of new technology, and complexity and interdependency of systems [88, p. 4].

8.2.1 Security vs. Safety

Emphasis in aviation security is typically placed on physical security concerns and promoting safety of its passengers and cargo. The greater purpose of investing in civil aviation security is to promote safety and protect the nation from any harm brought on, whether intentionally or unintentionally, by any individuals with authorized or unauthorized access to airspace systems and assets that can impact this sector.

Traditionally, cyber threats were associated with exploiting confidentiality, availability, on integrity of information systems for financial gain, competitive advantage, etc. In the

160 case of aviation, cyber threats can have an additional association with safety. Aircraft are becoming more comparable to a complex information system and can be targeted not only for financial gain, but to negatively impact safety and potentially cause a loss of life. The reliance of critical aviation components on cyber technology, creates the potential for such links. This example is not limited to aviation. In the medical industry for example, pace maker technology is evolving to include wireless connectivity. In theory, these devices can be hacked and manipulated to potentially impact safety of the patient using the device. These examples raise questions such as: What impact can cyber security have on safety? Are there any acceptable trade-offs between safety and security? Can a cyber-security breach potentially result in a loss of life? This topic is further explored and discussed in a paper published in conference proceedings for the

International Conference on Information Systems Security and Privacy (ICISSP) [95].

8.2.2 Aging Technology, New Technology, and Interdependence of Systems

In its beginning stages, aviation systems were supported by highly customized software and systems proprietary to the sector. This technology has aged and is in need of a refresh. As new technology is introduced, its desiged to itegate ith other systems and enable compatibility with other popular information technologies. With the new shift from critical software systems being housed on proprietary systems and using popieta tools, to the use of oeial off-the-shelf teholog these critical systems are also susceptible to the same attacks other commercial systems are exposed to [86]. For example, where the hardware and software technology is publicly available,

161 adversaries have the ability to look for vulnerabilities in these publicly available systems and develop exploits. Furthermore, there is a wealth of knowledge available on the internet that quickly exposes these vulnerabilities for the masses to view. This puts aviation systems into the arms race of critical patching and updating, a practice that should be done in any critical system proprietary or not, but is of even greater concern under this circumstance. Due to regulatory requirements for testing and approving configurations for aviation systems, patches may not be applied immediately leaving an even greater window for potential exploitation. For instance, critical information systems on the aircraft, such as the avionics systems located in the cockpit, are separated and protected from other connected systems using firewalls. Firewalls are a common layer of security used in many systems. Since it is software, it can theoretically be hacked or bypassed just like any other software [96, p. 18] unless the proper controls are in place to prevent such exploitation.

8.2.3 Examples of Cyber Threats in Aviation

In an effort to manage aviation risks, The Aviation Mode Transportation Sector

Specific Plan [89] outlines a plan to provide a secure, resilient network of aviation system components. The plan notes two major originators of threats (terrorists and criminals) and categorizes aviation threats into 3 categories:

1. Aircraft as a target and/or weapon

2. ATS infrastructure as a target

3. Hostile exploitation of cargo 162

The assets included in these categories are comprised of a combination of fixed, mobile, and cyber assets. Nevertheless, the NCIPP chooses to focus its efforts on fixed assets. Some examples of aviation threats that span beyond the coverage of fixed assets include: 1.) Exploiting protocol weaknesses or compromised systems to spoof GPS navigation data or flight plans 2.) Launching denial of service through flooding or ARP poisoning to disrupt communication between critical systems 3.) Injecting malicious content into critical software/databases [87, p. 14], to name a few.

To expand on the examples above, we use injection attacks and spoofing examples:

8.2.3.1 Injection Attacks

Injection attacks can be done by adding malicious code to software systems and databases. For example, if a backdoor is available on a production server housing a critical aviation application, without proper controls in place, malicious code may be injected to impact operations. Another related issue that comes up in this space is the insider threat. If a user with legitimate access to this system is able to change the code and commit this same act whether intentionally or unintentionally, the change may go undetected and aviation security compromised if otols aet i plae to aage this risk through detection and prevention. These are examples that physical airport (fixed asset) security and the limited program definition can overlook.

163

8.2.3.2 Spoofing

Spoofing can occur when data is modified to look as though it is from a trustworthy source. Systems that may be susceptible to spoofing include flight planning systems, GPS navigation data, etc. One spoofing concern is that ghost aircraft can be injected in air traffic management systems. Data communicated using the ADS-B protocol is unauthenticated and unencrypted. It can be seen by anyone with equipment available in general markets. If the data can be read and manipulated this can enable attackers to note patterns in communication in contribution to future attack plans. For example, where data patterns are noted, it may be possible for malicious intenders to mimic legitimate traffic and launch a denial of service attach against either the aircraft or ground system. Without proper risk management controls in place to detect and peet these attaks, soues a e ale to oad itial sstes ith tash data and deny legitimate users access to resources.

Table 35 includes a sample of additional theoretical risks [86] [97] [98] [96].

These are not exhaustive examples as there are many threats to aviation cyber security.

164

Area of Risk Description Implication General Systems Use of commercial off the shelf systems and Systems are susceptible to the network technology same vulnerabilities most other commercial IT systems General Systems Insider threats Authorized and unauthorized users may be able to access and manipulate code; Changes may go undetected Airport Security Screening System Manipulation System integrity may be targeted Infrastructure to manipulate system performance Airport Remote maintenance of AIT systems and System integrity may be targeted Infrastructure other systems to manipulate system performance Airline Operations Maintenance System Manipulation or Aircraft maintenance systems Infection may be infected or manipulated to prevent critical alerts Airline Operations Protection of card holder data during in-flight System vulnerabilities may be purchase as often required by Payment Card exploited to gain access to Industry Data Security Standard (PCI-DSS); sensitive data Personal identifiable information (PII) collected from passengers Air Traffic Ghost aircraft injections, ghost aircraft Air traffic management Management flooding, ATS system flooding, virtual communication may be trajectory modification, ADS-B hacking manipulated or spoofed. Aircraft System Retrofitting new services and technology to Integration of new technology existing aircraft given life span of systems and old systems, or lack thereof introduces a variety of security risks that are not well understood and documented Aircraft System In-flight entertainment system hacking In-flight entertainment and other onboard systems are segmented by firewall software which can theoretically be bypassed and used to hop to other more critical aviation systems if proper controls are lacking Aircraft System Security implications of mobile devices used Common vulnerabilities to download engine diagnostics; Call home associated with remote access functions; Remote control of maintenance may be exploited to gain functions unauthorized access to critical aviation systems Aircraft System GPS spoofing, flight plan spoofing, ACARS Flight plan and/or GPS hacking coordinates may be manipulated to increase chances of collision Table 35: Sample Theoretical Cyber Threats Applicable to Aviation Sector

165

8.2.4 Perceived Reality of Cyber Security Threats in Aviation

Many theoretical demos as well as real world events have been documented involving cyber threats to aviation, [99], [100], [101], [102], [103] , but there have also been conflicting views amongst aviation experts on the viability of successful cyber- attacks in aviation. Some have been labelled false and exaggerated claims, while some appear to raise valid concerns in the sector.

The International Civil Aviation Organization (ICAO) released a working paper in

2014 reporting on risk assessment of cyber-attack against the air traffic management system [104]. The report discloses that threats such as the disruption of aircraft separation data feeds could potentially increase the risk of aircraft collision, though marginally. But also states that, The ATC sste has a iteal heks ad alaes that make it very unlikely that a hacker can seriously compromise controlled traffic in

otolled aispae. Most of the lais…hae ee ade i igoae of these sste

heks. Neetheless, the epot oludes that hile the isk of a suessful attack on

Air Traffic Management (ATM) is low, the overall impact of a cyber-attack on the air taffi aageet sste ould e ategoized as high. The lo isk atig is partially attributed to the advanced level of skill, access, and knowledge required to carry out such attacks. One could argue that this explanation may not be as applicable for an insider threat, in which case expertise and knowledge of systems is much more advanced. In addition, because the impact of an attack on aviation would be considered high, coupled with the fact that new vulnerabilities and threat actors continue to rise,

166 this is not something to be taken lightly or ignored under the premise that successful attack likelihood was considered low by ICAO.

Illuminating a Fundamental Gap

8.3.1 The “ector’s Current Approach to Aviation Security

The Aviation Sector takes on a layered approach to security comprised of 20 programs shown in Figure 32. These programs were designed in order to manage risks to the aviation system. While these programs appear to focus on preventing hostile people or objects from entering aircraft or destroying infrastructure, they may not account as strongly for cyber threats. The plan does include a sub-section on cybersecurity in which the objectives are to understand risks to cyber infrastructure, share information in contribution to risk management and decision making in the sector, and develop mitigation strategies to address growing threats. An example of this is noted in the participation of TSA and FAA in the Cross-Sector Cyber Security Working

Group (CSCSWG), a group comprised of public and private sector security experts, to share information on risk management across all sectors [89, p. 143].

167

Figure 32: Layered Approach to Aviation Security [89, p. 143]

8.3.2 The Critical Infrastructure Identification Gap

That aircraft themselves are being overlooked by the NCIPP appears to be further confirmed by the Taxonomy of Aviation Assets stipulated in the 2010

Transportation Sector Specific Plan, listed in Table 36 [105, p. 105]. The current NCIPP preoccupation with fixed assets may represent a significant shortcoming in current infrastructure protection strategy. No amount of physical airport security can prevent a cyber-attack on aircraft avionics. A coordinated cyber hijacking could result in a catastrophe far greater than 9/11.

Aviation Conveyances (Aircraft) and Airports Air Traffic Control and Navigation Facilities Space Transportation Facilities Aviation Sector Command Control Communication Coordination Facilities Other Aviation Facilities Table 36: Aviation Sector Assets

The challenge with protecting mobile assets is not limited to aviation. Other assets such as those listed in Table 37, present similar opportunity for weaponization. It

168 would seem unwise to exclude them from any accounting of critical infrastructure so they may be included as part of the national risk strategy. While we cannot protect every single asset, we can improve the way that we identify critical assets for more complete risk assessment and better prioritization.

Sector Asset Maritime Water Vessels Aviation Passenger & Freight Aircraft Railroad Freight Trains Carrying Toxic or Volatile Compounds Through Urban Areas Road Vehicles Carrying Toxic or Volatile Compounds Through Urban Areas Mass Transit Vehicles Carrying Thousands of Passengers Pipeline LNG Tankers (Liquefied Natural Gas) Table 37: Transportation Sector Mobile Assets [105, pp. 105-119]

Where originally aviation systems did not feature wireless connectivity and its inherent threats, some aviation systems, including aircraft, can now be considered a system of connected systems in itself. Numerous reports cite that cyber security research is either suggested or already underway [86], [106], [88], [87] but this research is in its beginning stages. As new technology is introduced, the list of threats and enablers will continue to grow. They are enabled by technology and connectivity, and can be activated using remote connections from aircraft to ground sites, network connections between aircraft systems and vulnerable equipment, and interference with technology services. Understanding these threats will provide a means for prioritizing and determining which system components require elevated levels of protection and which systems have the potential to impact safety from a cyber-security perspective.

169

Using CIAid and Sector Specific Criteria to Lessen the Gap

The first step in addressing these challenges is to understand what is critical in the aviation sector. For an organization using the CIAid tool, the process would start with selection of objectives in the tool. The following hypothetical objectives could be selected by an aviation sector user in accordance with the guiding questions outlined in

Table 24:

 Component 1: Scope: Complete

 Component 2: Political Accommodation: No

 Component 3: Asset Types: Fixed and mobile

 Component 4: Impact Type: Destruction

 Component 5: Impact Measurement: Economic Security, Public Health

and Safety, Public Confidence, Cascading Impacts

 Component 6: Stakeholder: Public Sector

 Component 7: Sector: Transportation – Aviation

Based on these objectives, the CIAid tool, introduced in Chapter 6, would recommend that a combination of network and function based methods be used to identify the initial scope and asset list for the sector, comprised of both mobile and fixed asset types. It is specifically recommended that the function-based identification approach be used to assist with identifying mobile assets. While using a function-based approach does have the potential to accommodate mobile assets, the network-based approach can be used to map out the asset nodes and connections supporting those 170 critical functions for potentially a more complete scoping. Table 34 provides an example of aviation functions that could serve as a starting point to the initial function criticality aalsis. Based o the additioal oe of destutio, identified by the user during component four, the tool would recommend creating consequence or worst case scenarios for the assets.

Next, to address the impact measurement, sector-specific criteria are recommended to further account for mobile assets, such as aircraft, in addition to the other criteria objectives outlined by the user. Examples of sector specific criteria for aviation are detailed in the remainder of this chapter. In order to accomplish this, we first went through a brainstorming session, analysis, and expert validation process to establish potential aviation criteria. We then established a dataset for analysis and concluded with an example list of critical aviation sector assets.

At component six, which focuses on the stakeholder, because the focus is the public-sector perspective for this case, the user would be advised with tips for engaging and incentivizing the private sector to participate. Example include, providing tax breaks for airlines who participate in identification efforts, awareness of assistance options once critical infrastructure is identified, or more extremely, warning of disclosure to the flying public that infrastructure owner opted out of participation. At component seven, which focuses on sector scope, aviation is selected meaning that sector-specific criteria should be included as discussed above.

171

Identification criteria development represents the largest component of the framework. As such, we focus a considerable portion of the case study providing examples of sector specific criteria and how they can be applied in the sector.

8.4.1 Methodology for Development of Aviation Sector Specific Criteria

To begin, core aviation processes were outlined and major asset categories that support those processes were identified. From the preliminary research, we found that the majority of aviation functions and assets can be summed up into four main categories: Airports, Aircraft, Aviation Operators or Airlines, and Air Traffic Management

Systems (ATM). The document review included but was not limited to a review of

Federation Aviation Administration (FAA) documentation [94], Transportation Sector

Specific Plan Taxonomy and the Aviation Annex [105] [89], and research documentation from organizations such as Honeywell and AIAA [87] [98].

Keeping in mind data availability and the need to maintain simplicity, we brainstormed potential criteria points for each category summarized in Table 38. To determine the best options and remove any ineffective criteria, an expert validation survey was launched soliciting feedback from aviation sector and critical infrastructure protection experts. Input from over sixty industry experts across seven countries was reviewed and analyzed to conclude on the best criteria categories. We then gathered and analyzed data on one of the selected criteria points from each category, each bolded in Table 38, in order to provide an example of how this research can be applied

172 in the industry today. Next, we incorporated these criteria categories into the overall methodology and system previously presented in Chapter 6.

Category Brainstormed Criteria Options Airports  Number of annual enplanements  Number of commercial flights operated  Pounds of cargo weight processed Aircraft  Aircraft seating capacity  Aircraft type Air Traffic  Number of flights managed annually Management  Number of destinations served through System (ATM) flights managed by facility  Management of a major hub by air traffic control system Airline  National share of passenger traffic Operators  Percentage of aviation industry revenue  Aircraft type operated Table 38: Brainstormed Aviation Criteria Options

The purpose of the aviation criteria survey was to gain input from aviation industry experts around the globe to contribute to determination of which methods were seen as reasonable in determining criticality of assets. The survey was comprised of 14 questions outlined in Appendix F.

8.4.2 Expert Survey Results

Over sixty aviation and security industry experts from around the world participated in an aviation criteria survey. 50% of the respondents had at least 10 years of experience in the security industry over 5 years of experience in the aviation industry.

The feedback includes responses from 7 executive/C-level resources, 23 management level resources, 23 technical experts, and 7 intermediate level resources. While majority

173 of the responses where from the U.S., participants also came from Belgium, Germany,

Canada, Singapore, India, and United Kingdom, as shown in Figure 33. Within the U.S. responses were also nationally dispersed with respondents from Illinois, Texas, and D.C., and New Jersey to name a few. All respondents confirmed at least a medium level familiarity with the concepts of Critical Infrastructure Protection, National Security, and

Critical Asset Identification.

Figure 33: Geographical Distribution of Responses

The survey focused on the four core aviation asset categories: Aircraft, Air Traffic

Management, Airports, and Airlines. Using a scale of 1 – 5, where 1 is the lowest rating and 5 is the highest, participants rated each on its ability to help distinguish critical assets. Based on the results, we found that some recommended means for measurement were more favored than others. Through additional free form comments,

174 we also gained insight on additional measurement types that should be considered. The average rating for each criteria category is shown in Figure 34.

Figure 34: Expert Criteria Ratings

Respondents were also able to recommend new ways of thinking and measuring through the freeform comments fields, and provided suggestions in each area of measurement. For example, for aircraft criticality, a few experts suggested that whether the aircraft is e-enabled or not should also play a role in determining criticality. For airport, experts suggested adding criteria to distinguish airports that heavily contributed to cargo distribution and military operations, or included approach patterns near other critical infrastructures.

We also asked whether participation of airlines in critical asset identification efforts should be required by law. Currently, participation of all infrastructure owners is voluntary which provides a challenge when it comes to incentivization and sustaining 175 participation. We learned that majority of the respondents agreed with the notion of regulating participation. 69% of respondents believe that participation of airlines in critical asset identification efforts should be regulated. 31% of respondents disagree with the regulation of airline participation and alternatively suggest focusing on incentivization, establishing an overarching international organization responsible for aviation asset security, and more.

Figure 35: Regulation of Airline Participation in Identification Efforts

A full report of the survey results can be found in Appendix G.

8.4.3 Aviation Sector Criteria Recommendations

The journey towards more defined examples of aviation criteria was organized in four parts: 1.) Analysis of aircraft data 2.) Analysis of airport data 3.) Analysis of airline data 4.) Conclusions drawn about air traffic management. In each of these areas, we chose to do a proof of concept on one or more of the sub-criteria categories identified. 176

For aircraft analysis, we looked at the capacity and aircraft type. For airport analysis, we looked at the number of annual enplanements. For airline, analysis we looked at operating revenue. For ATM, we leverage flight volume data gathered during the airport analysis. These selections were made based on a combination of the rating received from industry experts and availability of public data.

8.4.3.1 Analysis of Aircraft Data

A RAND terrorism database [107] comprised of 40129 records spanning 4 decades of events was analyzed to determine which aircrafts were involved in the deadliest attacks. The database was searched for records where aircraft was a target or attak eto. Keods seahed i the desiptio of iidets ilude ailie,

aiaft, jet, aipot, aiaft, ai, et. This etued oughl eods.

These records were further filtered on incidents that resulted in loss of life of 2 or more people, or an injury count of 10 or more. This narrowed the list down to about 131 records meeting loss of life criteria and an additional 33 meeting the injury criteria, for a total of 164 records to be analyzed. The other 916 records did not meet the loss of life/injury criteria and were excluded from the analysis. Of the 164, only 52 directly related to aircraft, eliminating 112 records. In the 112 records, 61 occurred at or near an airport, 12 at an airline office, and 39 were not applicable. By not applicable, we mean that they did not fit the criteria or any of the categories mentioned above. For example, situations in which terrorists negotiated planes to transport their hostages or in order to

etu to thei o safe teitoies duig egotiatio ee eoed fo the out

177 and as it is not the aircraft that was implicated in the injuries/death toll. Also, note that loss of life count may include terrorists killed by raids, etc.

Aviation Terrorism Data Keyword Search (1080 of 40129 records)

Did not meet analysis criteria Airport/Airline Related Aircraft Related (916 records) (112 records) (52 records)

Figure 36: RAND Database Filtering

Using the remaining records, several online aircraft incident databases were reviewed to document the aircraft types involved in the attacks. For each aircraft type, the aircraft capacity was documented to understand aircraft types most commonly targeted for the deadliest attacks. The aircraft capacity data was gathered from a variety of sources, with the main source being a FAA database of aircraft registrations. Averages were taken on the capacity of each aircraft type. Where the type was not found in this database, Google was used to find additional information about aircrafts on manufacturer sites, aircraft data repositories, etc. These resources were only used when data could not be found in the FAA database. The findings are summarized in Figure 37 and Table 39.

178

Average of Average of Sum of Count of Sum of Normalized Fatalities Normalized Aircraft Injuries Capacity Capacity Type Boeing 767 275 1438 275 2 2309 Boeing 757 216 116.5 432 2 76 Boeing 747 474 89.14285714 3318 7 167 DC-10 319 86 638 2 0 Boeing 720 143 82 143 1 0 Boeing 737 241 78.66666667 1446 6 65 Boeing-707 194.6666667 78.33333333 584 3 18 DC-8 157 73 157 1 0 TU-154 154 66.33333333 462 3 0 Viscount 748D 51 59 51 1 Antonov 24RV 44 55 44 1 Convair-990A 108 47 108 1 0 Antonov AN-26 50 44.66666667 150 3 0 Fokker 27 46.5 29 93 2 16 Tupolev 134 84 27 84 1 DC-9 132 26 132 1 0 Sud Aviation SE- 93 25 93 1 0 210 Caravelle III Lockheed C-130 50.5 22 101 2 0 Hercules Embraer-110 21 21 21 1 0 Boeing 727 140 15 420 3 21 Lockheed L-1011 302 15 604 2 40 Cessna 404 8 13 8 1 Falcon 50 9 12 9 1 Ilyshin ii-76 7 11 7 1 0 Airbus A-300 355 10 1065 3 30 BAC-111 99 10 99 1 22 Military - capacity not 10 0 1 0 Ecudorian Air found Force DC-7 78 8 78 1 0 not found capacity not 7.428571429 0 7 359 found Boeing-720B 143 6 143 1 11 Airbus 310 355 4 355 1 0 DC-3 32 3 32 1 0 Medecines Sans capacity not 3 0 1 0 Frontiers relief- found aid aircraft Tupolew TU-156 capacity not 2 0 1 0 found Table 39: Aircraft Types Involved in Terrorism Incidents

179

AIRCRAFT DATA ANALYSIS

Average of Normalized Capacity Average of Fatalities

1800 1600 1400 1200 1000 800 600 400 200

DC-9

DC-3

DC-7

DC-8

DC-10

TU-154

BAC-111

Falcon 50 Falcon

Fokker27

Airbus310

Boeing727

Boeing747

Boeing757

Boeing737

Cessna 404 Cessna

Ilyshin ii-76 Ilyshin

Boeing-707

Boeing767

Boeing720

Tupolev 134 Tupolev

Boeing-720B

AirbusA-300

Embraer-110

Convair-990A

Antonov 24RV Antonov

Viscount 748D Viscount

Antonov AN-26 Antonov

Lokheed C-… Lokheed “ud Aiatio “E-… “ud Lockheed L-1011 Lockheed Figure 37: Average Fatalities and Capacity per Aircraft Type

From this analysis, we find that some of the deadliest attacks have occurred in aircraft with capacity between 45 – 475 seats. This can serve as a basis in developing aircraft classes. For example, one might conclude that a tier-one aircraft class can be any aircraft with a seating capacity above 180 as majority of the deadly attacks fit this category. A tier-two category may entail aircraft with a capacity of 45 – 180 seats. Note that the Boeing 767 presents an anomaly in the data trend, but is due to the 9/11 incident whose death toll included both aircraft passengers and mass lives lost on the ground through the collapse of the towers, the Pentagon strike, and more.

8.4.3.2 Analysis of Airport Data

A Federal Aviation Administration (FAA) National Plan of Integrated Airport

Systems (NPISAS) database [108] comprised of 3345 airports was analyzed to determine which airports accounted for the majority of passenger traffic. From this analysis, we

180 found that 50% of enplanements are covered by 16 major airports and 80% of enplanements are covered by 43 airports. (27 airports in addition to the 16 aforementioned). We performed a secondary analysis on a database [109] of 13,000 airports based on the number of commercial operations. The results of this analysis provided results very similar to that of the NPISAS with 80% of the commercial operations covered by 50 airports. Using this data, one may conclude that airports with over 15 million enplanements can be considered a tier-one class, and 4-15 million enplanements be considered a tier-two class. Using this data analysis, we can begin to develop and recommend asset classes in the future.

181

Airport Aircraft Enplaned Percentage Hartsfield - Jackson Atlanta International 45,798,928 6.25% Chicago O'Hare International 32,171,795 4.39% Los Angeles International 31,326,268 4.28% Dallas/Fort Worth International 28,022,904 3.83% Denver International 25,799,841 3.52% John F Kennedy International 24,520,981 3.35% San Francisco International 21,284,236 2.91% Charlotte/Douglas International 20,033,816 2.74% McCarran International 19,959,651 2.73% Phoenix Sky Harbor International 19,560,870 2.67% George Bush Intercontinental/Houston 19,039,000 2.60% Miami International 18,987,488 2.59% Orlando International 17,159,427 2.34% Newark Liberty International 17,055,993 2.33% Seattle-Tacoma International 16,121,123 2.20% Minneapolis-St Paul International/Wold-Chamberlain 15,943,878 2.18% Detroit Metropolitan Wayne County 15,599,879 2.13% Philadelphia International 14,589,337 1.99% General Edward Lawrence Logan International 14,293,695 1.95% La Guardia 12,818,717 1.75% Fort Lauderdale/Hollywood International 11,445,103 1.56% Baltimore/Washington International Thurgood 11,186,444 Marshall 1.53% Washington Dulles International 10,816,216 1.48% Salt Lake City International 9,579,840 1.31% Ronald Reagan Washington National 9,462,231 1.29% Chicago Midway International 9,436,387 1.29% Honolulu International 9,225,848 1.26% San Diego International 8,686,621 1.19% Tampa International 8,218,487 1.12% Portland International 7,142,620 0.98% Lambert-St Louis International 6,208,750 0.85% William P Hobby 5,043,737 0.69% Metropolitan Oakland International 4,926,683 0.67% Kansas City International 4,866,850 0.66% Nashville International 4,797,102 0.65% Austin-Bergstrom International 4,606,252 0.63% Raleigh-Durham International 4,490,374 0.61% John Wayne Airport-Orange County 4,381,172 0.60% Sacramento International 4,357,899 0.59% Cleveland-Hopkins International 4,346,941 0.59% Louis Armstrong New Orleans International 4,293,624 0.59% Luis Munoz Marin International 4,204,478 0.57% Norman Y. Mineta San Jose International 4,077,654 0.56% Table 40: Example Tier One Critical Airports Based on Enplanements 182

Figure 38: Annual Airport Enplanement Trends

8.4.3.3 Analysis of Airline Data

As a starting point, we find that the aviation industry directly contributed $664 billion to the global economy in 2014 [91]. One way to estimate the criticality of an airline, is by the role it plays in economic security. By taking an airlines annual revenue and determining its share in the aviation contribution, we can use this number to rank airlines and determine which ones may be considered critical to economic security. For example, through an analysis of data gathered from a Bureau of Transportation

Statistics 2014 Airline Finance Database [110], we find that almost 25% of the aviation direct contribution comes from 10 carriers. A criterion could be that any airline controlling more than 2% of the direct contribution is considered critical to economic security.

183

Figure 39: Aviation Global GDP Impact in 2014 [91]

Airline Operating Percentage of Revenue Aviation Direct (millions) Contribution Delta 40,427 6.09% United 38,901 5.86% American 27,140 4.09% Southwest* 18,605 2.80% US Airways 15,750 2.37% JetBlue 5,817 0.88% Alaska 5,363 0.81% Hawaiian 2,311 0.35% Spirit 1,932 0.29% SkyWest 1,889 0.28% Total 158,134 23.82% Table 41: Airline Operating Revenue as Percentage of Aviation GDP [110]

8.4.3.4 Analysis of Air Traffic Management Data

Though we were unable to find a publicly available database for ATM, we find that this section can leverage the research done on the airport database. Air traffic facilities that are supporting the greatest hubs can be a means to classifying ATM infrastructure. For example, a system supporting a small private airport may not be as critical on a national level as a majo it aipot suh as Chiagos O‘D o Houstos

184

IAH. Using Table 40, we can begin to identify critical ATM assets by identifying those associated with the major airports.

Building an Aviation Asset Dataset

Using the data gathered during the analysis phase, an aviation asset dataset comprised of 3,434 records was compiled. The dataset currently includes 34 aircraft classes, 3,345 airports, and 55 airlines. A publicly available and more useful dataset on air traffic management systems has not yet been located.

8.5.1 Data Collection

The aircraft data was gathered as a result of the analysis done on the RAND aviation incident database, with the main criteria point currently included in the database for each aircraft class being the aircraft capacity. The airport data was gathered as a result of analysis done on the NPISAS airport database. The main criteria point currently included in the database is annual count of enplanements. Lastly, the airline data was gathered as result of analysis on the Bureau of Transportation Statistics database. The main criteria point currently included in the database is annual operating revenue of each airline. A small sample of the airline dataset is shown in Table 42 and expanded in Appendix H. Please reach out to the author for access to the full dataset.

185

Criteria Asset Title Data Data Description Data Source Category Collected Airline Southwest Airlines Co.: 18605 2014 Operating U.S. WN Revenue (Millions) Transportation Statistics Bureau Airline Hawaiian Airlines Inc.: 2311 2014 Operating U.S. HA Revenue (Millions) Transportation Statistics Bureau Airline Delta Air Lines Inc.: DL 40426 2014 Operating U.S. Revenue (Millions) Transportation Statistics Bureau Airline American Airlines Inc.: 27140 2014 Operating U.S. AA Revenue (Millions) Transportation Statistics Bureau Airline United Parcel Service: 5X 5814 2014 Operating U.S. Revenue (Millions) Transportation Statistics Bureau Airline Alaska Airlines Inc.: AS 5363 2014 Operating U.S. Revenue (Millions) Transportation Statistics Bureau Airline United Air Lines Inc.: UA 38900 2014 Operating U.S. Revenue (Millions) Transportation Statistics Bureau Airline Atlas Air Inc.: 5Y 1634 2014 Operating U.S. Revenue (Millions) Transportation Statistics Bureau Airline Spirit Air Lines: NK 1931 2014 Operating U.S. Revenue (Millions) Transportation Statistics Bureau Airline Sun Country Airlines 450 2014 Operating U.S. d/b/a MN Airlines: SY Revenue (Millions) Transportation Statistics Bureau Airline Frontier Airlines Inc.: F9 1574 2014 Operating U.S. Revenue (Millions) Transportation Statistics Bureau Airline Virgin America: VX 1489 2014 Operating U.S. Revenue (Millions) Transportation Statistics Bureau Table 42: Sample Aviation Dataset- Airlines

In the future, researchers can continue building this dataset for experimental and research purposes. In the real world, the data will likely require an annual update to account for new assets and dynamic data.

186

8.5.2 Data Maintenance

This research scratches the surface on aviation asset analysis. As the research continues to mature over time, this database will likely grow significantly and require greater effort for analysis. While the current data is housed in an excel spreadsheet, as the volume grows, it may be useful to consider how big data analytics research and tools can be incorporated into database maintenance. Big data analytics is a growing discipline in which large amounts of data can be analyzed to find hidden trends, correlations, and provide valuable insight. Using data analytics, as time progresses, we have an opportunity to draw new insights on the asset criticality results. This may inform future criteria category development and continuous transformation towards more effective asset identification. For the time being, the dataset is still relatively small and can be managed in a traditional database.

8.5.3 Real World Data Collection4

An aviation asset database comprised of current, real data would be very helpful not only in asset identification, but also in risk assessment and security operations. Four areas would need to be addressed to support this effort: data sources, data collection, data maintenance and data connection to the next steps in the risk management process. An example roadmap is described here to illustrate a potential way to apply these recommendations and expand this effort further.

4 This is an important component of future research required and is further emphasized in the concluding chapter. 187

The Department of Homeland Security, Office of Infrastructure Protection and

Sector Specific Agencies should collaborate with aviation industry stakeholders to collect and maintain the data. The data sources should be agreed upon by this collaborative team and can include data from public and private sector resources depending on the sector and asset category in question. The collaborative group can then combine the data from these sources, normalize it, and add it to the aviation database. The aviation database would then contain information that can be fed to risk assessment processes as necessary.

In relation to aircraft data, DHS IP can engage original equipment manufacturers such as Boeing, Airbus, Embraer, and others to collect aircraft data. Example data points that may be collected can include aircraft capacity, engine type, and whether or not the aircraft is e-enabled.

In relation to airport and air traffic management data, DHS IP can engage the

FAA and TSA to help collect relevant data. Example data points that may be collected for airport support include the airport location, annual number of enplanements, economic contribution of airport to national economy, or amount of cargo processed annually. For air traffic management, example data points include counts on airports supported by the facility/system, number of flights managed, or number of destinations supported by function.

In relation to airline data, DHS IP can engage major airlines, aviation information sharing groups such as Aviation ISAC, and aviation alliance groups to contribute airline

188 related data. At a basic level, for identification efforts, example data points that may be collected include airline operating revenue, share of national traffic, or number of passengers transported annually etc.

To further support risk assessment and operations after critical assets have been identified, additional data points may also be collected and added to the database such as active threat scores, likelihood of asset loss based on consequence scenarios possibly developed during the identification phase, etc. Dependency information and the like could be beneficial to include in assistance of both risk assessment and operations, particularly emergency response management. This database can then connect to risk management tools and support those processes as well. Admittedly, this connection effort is not a simple task. To achieve this aspiration, research should be done to determine what data points are required for risk assessment prior to data collection and the identification process. Those data points can then be incorporated during the initial population of the database.

Example Critical Aviation Asset List

Using the dataset and criteria discussed throughout this research we establish a sample critical aviation asset list. To do this, we took the 3,434-record database and ran it against the example criteria developed through analysis. Through this process, we found that 79 of the 3,434 were considered critical with 32 being Tier 1 critical and 47 designated as Tier 2.

189

The criteria the database was run against is shown in Table 43. This is example criteria chosen based on data availability and the ratings received from the expert survey. Categories were used where publicly available data could be found of the analysis. In the future, a deeper analysis of which criteria best apply should be established with the help of the aviation community.

Criteria Tier 1 Min Tier 2 Min Tier 2 Max Aircraft Capacity 180 40 179 Airport Enplanements 15000000 4000000 15000000 Operating Revenue (Millions) 15000 2000 Table 43:Aviation Criteria Used for Developing Example Aviation CI List

The resulting example Tier 1 critical aviation asset list is shown in Table 44. An additional

Tier 2 example listing is included in Appendix I.

190

Criteria Asset Title Data Collected Data Description Criticality Decision Category Aircraft Class Boeing 767 275 Aircraft Capacity Tier1Critical Aircraft Class Boeing 757 216 Aircraft Capacity Tier1Critical Aircraft Class Boeing 747 474 Aircraft Capacity Tier1Critical Aircraft Class DC-10 319 Aircraft Capacity Tier1Critical Aircraft Class Boeing 737 241 Aircraft Capacity Tier1Critical Aircraft Class Boeing-707 194.6666667 Aircraft Capacity Tier1Critical Aircraft Class Lockheed L-1011 302 Aircraft Capacity Tier1Critical Aircraft Class Airbus A-300 355 Aircraft Capacity Tier1Critical Aircraft Class Airbus 310 355 Aircraft Capacity Tier1Critical Airport ATL 45798928 Airport Tier1Critical Enplanements Airport ORD 32171795 Airport Tier1Critical Enplanements Airport LAX 31326268 Airport Tier1Critical Enplanements Airport DFW 28022904 Airport Tier1Critical Enplanements Airport DEN 25799841 Airport Tier1Critical Enplanements Airport JFK 24520981 Airport Tier1Critical Enplanements Airport SFO 21284236 Airport Tier1Critical Enplanements Airport CLT 20033816 Airport Tier1Critical Enplanements Airport LAS 19959651 Airport Tier1Critical Enplanements Airport PHX 19560870 Airport Tier1Critical Enplanements Airport IAH 19039000 Airport Tier1Critical Enplanements Airport MIA 18987488 Airport Tier1Critical Enplanements Airport MCO 17159427 Airport Tier1Critical Enplanements Table 44:Example Tier-1 Aviation CI Asset List

191

Table 44 otiued…

Criteria Asset Title Data Collected Data Description Criticality Decision Category Airport EWR 17055993 Airport Tier1Critical Enplanements Airport SEA 16121123 Airport Tier1Critical Enplanements Airport MSP 15943878 Airport Tier1Critical Enplanements Airport DTW 15599879 Airport Tier1Critical Enplanements Airline Southwest 18605 Operating Tier1Critical Airlines Co.: WN Revenue (Millions) Airline Delta Air Lines 40426 Operating Tier1Critical Inc.: DL Revenue (Millions) Airline American Airlines 27140 Operating Tier1Critical Inc.: AA Revenue (Millions) Airline United Air Lines 38900 Operating Tier1Critical Inc.: UA Revenue (Millions) Airline Federal Express 26523 Operating Tier1Critical Corporation: FX Revenue (Millions) Airline US Airways Inc.: 15750 Operating Tier1Critical US (Merged with Revenue America West (Millions) 9/05. Reporting for both starting 10/07.)

Key Conclusions

The evolving threat landscape calls for a new way of identifying critical aviation assets, one that accounts for mobile and cyber assets. The results of the expert survey suggest that aircraft classes based on capacity and aircraft type can be used as criteria points in identifying critical aircraft. Airport and ATM criteria can be developed based on volume of annual passenger traffic, amongst other criteria. Airline criteria can be

192 developed by conducting analysis of contributed revenue against the aviation industry direct contribution to the national economy.

Based on these criteria and analysis we classify airports in Atlanta, Chicago, Los

Angeles, Dallas, Denver, New York, and San Francisco as some of the most critical airports, and are therefore supported by some of the most critical ATM systems. We also conclude that aircraft classes can be established based on the aircraft type. For example, Boeing 747, Airbus A-300, Airbus 310, and comparable aircraft could be considered Tier 1 critical, given the large count of fatalities resulting from incidents involving these aircraft. We conclude that Delta, United, American, Southwest, etc. can be considered critical aviation corporations based on their contribution to economy.

Note that these are not all encompassing conclusions. Additional analysis and conclusions can be drawn from additional research in the aviation sector. This report serves as a foundational starting point to continually researching and optimizing criteria in this sector.

In practice, establishment of a common, usable aviation asset database would require input and collaboration from numerous aviation stakeholders including, but not limited to FAA, TSA, original equipment manufacturers, and airlines. If done appropriately, this database can not only contribute to critical asset identification, but also to subsequent steps in the risk management process.

193

9 CHAPTER IX

RESEARCH SUMMARY AND CONCLUSION

Contributions to the CIP and Aviation Industries

Upon initiating this research, we sought to find a way to more effectively identify critical infrastructure assets. After first attempting to use multi-criteria decision theory to technically identify critical assets, we recognized that there was a larger issue at hand; the need to first select the appropriate methodology for identification in a logical and defensible manner. We recognized that each nation may have unique characteristics and objectives requiring different methodology components, and as such needed to allow nations to tailor programs to meet their needs, while still providing the logical decision trail. CIAid solves this problem by providing an objective-based decision system that can aid stakeholders in developing a critical asset identification program that aligns with their own needs and objectives. We also aimed to use the aviation sector as a case study in developing an example critical asset list. This was accomplished through the aviation sector case study which included criteria analysis with aviation sector experts, development of an aviation dataset, and an example tiered asset list.

194

Through the completion of tasks described throughout this chapter, we achieved our ultimate mission to find a more effective way to identify critical infrastructure assets.

The major contributions of this dissertation are:

Delivery of an Objective Framework

 Core Deliverable: Development and validation of an objective framework for

identification methods

 Literature review of existing programs, policies, and methodologies

 Identification of gaps existing between policy and execution of methodology

Development of the CIAid Tool

 Core Deliverable: Development of a database and model representative of the

objective framework, including a TOPSIS comparative component

 Selection of an MCDM approach for application to a CI challenge

 Delivery of a user friend tool for easy access to the model and database (CIAid)

Completion of the Aviation Case Study

 Core Deliverable: An aviation asset dataset and example of a critical asset list

 Identification of aviation sector gaps

 An analysis of potential aviation sector specific criteria

Publications and Conference Presentations

 Four accepted publications and four additional currently under review

195

 Two completed international conference presentations

9.1.1 Delivery of an Objective Framework

In approaching the literature review and gap analysis, we took a different approach in that we developed a three-level identification classification method used to categorize existing approaches. This scheme became the foundation of the later developed objective framework. Using this new classification method, we were able to understand how each methodology faired up against recommended methodology criteria outlined by the NIPP. An introduction to the challenge at hand was presented in

February 2016 at the European Project Space Symposium in Rome, Italy. A more detailed and advanced gap analysis as well as recommendations for a way forward were presented at the International Conference on Critical Infrastructure Protection in the

Washington, D.C area in March 2016. Finally, a summary of the literature review and historical background on identification was published in the International Journal of

Critical Infrastructures.

In tandem, an expanded seven-layer identification objective framework was developed to map identification objectives with assessment attributes. This effort combined the work of numerous researchers, publish and private sector programs, policies, and incorporates new ideas to tie historically disbanded methodology components into a cohesive and holistic picture. The expanded framework is currently comprised of roughly 50 objectives, 80 attributes, and 100 relationships. There are a few unique ideas that were incorporated into the framework. The first is that the 196 infrastructure operators/ private sector organizations needs and interests are considered. While critical infrastructure protection is the responsibility of the public sector, most are owned by the private sector. By incorporating criteria relevant to the private sector, we incorporate information relevant to both parties and can better showcase the benefits of participation. Additionally, an objective was added to address political concerns. In the past, political concerns have been addressed by adding new threat based identification programs or omitting critical sectors from analysis all together, making programs inconsistent with their missions. By calling out the willingness or unwillingness of accommodating political concerns on the front end of the assessment, the decision maker can commit to building a methodology for or against the notion. Finally, a goal of this research was to ensure mobile assets are considered.

We call this out as a potential objective and then provide a way to account for mobile assets throughout the methodology recommendation process, but particularly in using a function-based methodology and sector specific criteria. By doing so, we are able to identify critical functions and the assets supporting those functions regardless of the asset is fixed or mobile.

9.1.2 Development of the CIAid Tool

In order to develop the tool, a MCDM approach first had to be selection for application. Existing MCDM selection frameworks were reviewed and then customized to meet the needs of this research. In addition to selecting TOPSIS as the method to be

197 applied, the result is also a new way to select a MCDM method for use on critical infrastructure problems.

This research applies MCDM, specifically the TOPSIS methodology, to evaluate alternative methodologies when the method recommended by the framework is not feasible for immediate implementation. The tool enables comparison of the method suggested through the objective framework, methods developed based on the decision

akes o apailities, ad opaiso of the deisio akes uet idetifiatio program. A comparable framework and tool does not exist today.

To encourage use of the framework/model and reduce the learning curve required, a user friendly graphical user interface was developed that incorporates both deliverables. This system is called the Critical Infrastructure Asset Identification (CIAid) tool and is one of the main deliverables of this research. Instead of manually going through the list of 50 objectives to understand which ones are applicable and then following the mapping to understand which assessment attributes support those objectives, the CIAid solicits the uses ojeties though simple questions with choices presented for each. Additionally, instead of having the user manually follow the

TOPSIS calculation process to evaluate their alternatives, through CIAid the user is able to select their alternatives in the tool. At the click of a button, the calculation is done in the back end and the best alternative is presented to the user.

198

9.1.3 Aviation Case Study

This research provides a unique aviation case study that discusses challenges in identifying critical assets and provides insight into a way forward by taking the CIAid tool a step further. To develop aviation sector specific criteria, aviation experts around the globe participated in a survey on new aviation asset criteria categories. The results garnered interest in the aviation community and serve as the basis for new aviation criteria recommendations for identifying critical aircraft, airports, air traffic management systems, and airlines. The case study includes delivery of an example aviation asset database that can inform the development of a current real world database. The dataset used in this research will be made available for others to use in furthering this research. The case study also provides an example of what it would look like to select coverage of the aviation sector as an objective in the CIAid tool.

9.1.4 Publications and Conference Presentations

Four papers have been accepted for publication in relation to this research, three of which were presented at international security conferences in Washington, DC and Rome, Italy:

199

Title Journal Author(s) Understanding the Impacts Conference Proceedings Christine Izuakor of Cyber Security Risks on from 2nd International Safety Conference on Information Systems Security and Privacy Critical Infrastructure Asset Journal of Critical Christine Izuakor and Identification: Seemingly Infrastructure Richard White Simple, but Frustratingly Elusive Critical Infrastructure Asset Journal of Critical Christine Izuakor and Identification: Policy, Infrastructure Protection Richard White Methodology, and Gap Analysis Critical Infrastructure Asset INSTICC European Project Christine Izuakor Identification: An Space 2016 Objective-based Decision Table 45: Accepted Research Publications

Four additional papers to be submitted for review:

Title Journal Author(s) Critical Infrastructure Asset IEEE – Security and Privacy Christine Izuakor Identification: Selecting an Journal Appropriate MCDM Approach Program Decisions CIAid: An Objective-based Journal of Critical Christine Izuakor Framework for Identifying Infrastructure Protection Critical Infrastructure CIAid: Using MCDM to IEEE – Security and Privacy Christine Izuakor Select a Critical Journal Infrastructure Asset Identification Methodology Critical Infrastructure Asset National Journal of Christine Izuakor Identification: An Aviation Homeland Security Case Study Table 46: Papers Currently Under Review for Publication

200

9.1.5 Lessons Learned

When initially considering ways to address the challenge of identifying critical infrastructure, we had several ideas that have evolved over the course of this research.

The first was that specialized criteria should not be used for identification because it limits cross-sector comparison abilities. We also viewed special criteria as somewhat unorganized process that added confusion to the identification of assets. We have since found that it is not the addition of specialized criteria that as the pole. Its the lak of systematic application or holistic consideration of criteria across sectors. Specialized criteria may be necessary in many cases. Instead, we recommended consideration of establishing sector specific criteria for each sector in relations to organizational objectives.

The second lesson was that we initially planned to apply MCDM methodology to the actual technical identification of critical assets. As the research progressed, we realized that MCDM really provided the most value in comparing alternative identification methods, not the comparison of the assets themselves. This is not to say that futue eseah shouldt e doe to try identifying specific assets using MCDM.

Though much progress was made, this research barely scratches the surface on the opportunity for growth in the critical infrastructure asset identification discipline.

201

Future Research

This section details future research opportunities that can strengthen this body of work and also shares a call to action for critical infrastructure protection stakeholders including, but not limited to, the Department of Homeland Security and comparable organizations in other countries.

9.2.1 DHS, or Government Agency, Call to Action

Failing to address the gaps and opportunities discussed throughout this report could result in or continue to propagate the misallocation of scarce security resources and funding, limit our awareness of vulnerabilities and threats, and lessen our overall national security confidence. Through the future research recommended in the following sections, specifically working with various stakeholders to expand this framework, investing in stronger model and software development, organizing efforts to develop security specific criteria (see Section 8.5.3), and dedicating to increasing awareness and adoption, we believe that the CIP stakeholders can improve identification efforts, thus improving the overall risk management process.

9.2.2 Continuous Framework Expansion and Maintenance

The objective framework should be continuously expanded and maintained to ensure its relevance and usefulness. As the threat landscape continues to evolve and researchers design new methods for identification, they should be incorporated into the tool. As such we recommend that the objective framework be revisited at least every 6-

202

12 months so that new objectives and attributes can be evaluated and added as necessary. Any new records should be vetted by industry experts and analysis completed to determine the relationships between the objectives and attributes.

Additionally, with these changes the database and any knowledge documents will need to be updated as well.

There is also an opportunity to more deeply explore the components of the framework. For example, through this research we find that it makes more sense to recommend a function-based approach for alignment to the formal definition of critical infrastructure. The next challenge is then ensuring that the decision maker is aware of how to effectively identify critical functions. Additional research can be done to provide detailed guidance on how identification of critical functions and assets supporting them can be accomplished. The same applies for network-based and logic-based approaches.

Deeper guidance should be developed and shared to help decision makers execute the recommended methodology. Another example is in the business impact criteria. This criterion was included to help infrastructure operators better understand the criticality of their own assets and possibly provide an incentive to participate. After the methodology is adopted and socialized in the private sector, feedback should be gathered to validate the positive or negative impact of including this criterion. Does it give infrastructure operators more incentive to participate in identification?

203

9.2.3 Sector Specific Criteria Development

The initial aviation case study presented through this research serves as a foundational starting point to continually researching and optimizing criteria in this sector. Through this research we have identified criteria categories and provided example criteria points based on statistical analysis. Decision makers in this space should contribute by deciding which criteria points/thresholds are acceptable for identification of their critical assets.

Additionally, while the aviation sector is used here, sector specific criteria is necessary in many other sectors. This research method can be applied to other sectors to establish criteria categories as well. This will take significant effort and can be incorporated into a long-term strategy. A great starting point is to get feedback from leaders in sector specific agencies and alike organizations to understand the existing state of identification and if any criteria currently exists. If criteria already exist, they can potentially serve as a basis for fine tuning and improving the criteria used in that sector.

If no criteria exist, the sector can follow a similar process as described in the aviation case study to begin gathering criteria.

9.2.4 Aviation Dataset and Database

An example aviation database was developed through this research for testing and analysis purposes. To ensure other researchers are able to leverage the research and data only publicly available databases were used. The database includes aircraft,

204 airport, and airline data. As additional data becomes publicly available, it should be added to the dataset.

In practice, establishment of a real aviation asset database could prove to be valuable to both the aviation industry and the critical infrastructure protection industry.

Not only would it aid in identification and management of critical assets, but it could also feed subsequent steps in the risk management process such as risk assessment and mitigation. In the U.S., establishment of the database would require input and collaboration from numerous aviation stakeholders including, but not limited to FAA,

TSA, original equipment manufacturers, and airlines. In addition, a resource from DHS IP would likely need to lead the effort and facilitate collaboration for each sector, including transportation and aviation. For instance, in the example database, the aircraft data was pulled together through analysis of information from various sources and the process for doing so was somewhat time consuming. It may not be practical to continue with this data collection approach in practice. Alternatively, original equipment manufacturers such as Boeing, Airbus, etc. can be engaged to provide their input on aircraft classes and provide a list of aircraft and configurations. This information should be fairly easy for the OEMs to gather and provide in comparison to manual data gather by an individual resource outside of the manufacturing industry. Details of this recommendation can be found in the aviation case study on Page 122.

205

As the dataset grows, both in the research world and in the real world, there is also an opportunity to leverage big data analytics concepts and technology to improve the asset analysis process.

9.2.5 Model Advancement

In the future, researchers may want to explore ways to further improve the model. One way is to organize a study around how other MCDM approaches and new

MCDM research can be incorporated into the model. For example, one could try building the same model using AHP, ELECTRE, PROMETHEE, and other techniques to determine if the recommendations are consistent or if other methods should be further explored. Another way is to test out the introduction of user solicited weights for each category. Currently, they are hardcoded into the backend of the tool to maintain simplicity and reproducibility of the recommendations. In the future, researchers may explore ways to allow the user to incorporate their own weight factors.

Along this same theme, users may eventually want to account for additional objectives or attributes that are not currently included in the CIAid tool. Instead of waiting for the next round of updates to the tool, users may want to update the attributes and criteria themselves. Future research can try to fulfill this need either at the software level by allowing decision makes to create profiles and add their own customer criteria categories, or by openig the odel up to od aiteae where researchers are able to either update the model themselves or submit requests to have the tool changes made. 206

9.2.6 Software Development

The current model was developed using Microsoft Excel. There is a future opportunity to expand the capabilities of the tool either with Excel or a more advanced platform. The process can be made more seamless by incorporating the sector specific criteria from each industry into the database thus enabling all sectors to use the same tool.

9.2.7 Long-term Vision

9.2.7.1 Optimization and Automation

A longer-term goal of this program is to create a seamless, end to end identification process. Currently, a decision maker is able to use CIAid to develop an idetifiatio ethodolog ad selet a et est alteatie if the ae uale to accept the recommended methodology. In the future, the decision makers should be able to move forward with immediately inputting their criteria thresholds, importing their asset databases using an import template that matches the model, and running the program to filter out all assets that match the criteria. This would provide an end to end experience in the tool where a user starts out with objectives and an asset database, and ends with a critical asset list. This is demonstrated in this research through the aviation case study, but is done so manually. Future research provides an opportunity for optimization and automation.

207

9.2.7.2 Adoption Plan

Identification of critical infrastructure assets has proven to be challenging in even the most advanced countries. This tool provides the foundation for organizing and executing a methodology and strategy for identification that is scalable. Our greater vision is that DHS and similar organizations will leverage this research and commit to supporting the future research required. The first next steps required for adoption are to gather sector specific criteria across sectors and incorporate into the tool. To do this, a template criteria development plan can be established and distributed. The plan should specify stakeholders and resources that should be engaged in the collaborative effort. The next steps can then be to build an optimized web application, conduct a pilot and develop additional use cases, and create a communication and public relations strategy for socializing the tool and its contribution to critical infrastructure protection.

Those responsible for national security will use the plan to then introduce the research and tool to their organizations.

Conclusion

Effective risk management is contingent upon the identification of critical assets, making this topic a key contributor to national security. Yet, the U.S. and other countries have struggled with establishing appropriate critical infrastructure asset identification programs and the resulting critical infrastructure lists. Without this information, we are unable to optimally invest scarce resources into our most critical assets faced with the greatest vulnerability and risk. Given the abundance of assessment options available 208 and varying objectives across programs, it is our belief that critical asset identification should be viewed as an objective-based decision process, where decision makers can establish tailored methodology for identifying assets based on specific objectives.

This research amalgamated the existing wealth of siloed resources within the discipline to create a holistic decision system. MCDM methodology has proven to be an effective decision theory within the infrastructure management space, and was applied to further tailor identification methods to meet reality of stakeholders. Though our planned solution to addressing the challenge at hand has evolved over time, the resulting system is a fundamental first step to addressing the original problem. It provides a more effective method of identifying critical infrastructure assets by first estalishig a ethod that eets the uses ojeties.

While much progress was made towards improving the identification of CI assets, this research is only the beginning to a long road of opportunities for improvement. This research can be expanded to cover additional sectors and, additionally, can be streamlined by adding more advanced functionality to this inaugural model. Further this body of work and this discipline will afford us the opportunity to improve national securit esuig that e a ofidetl idetif hats itial to our nations and manage risks accordingly.

209

10 REFERENCES

[1] United States Government, "UNITING AND STRENGTHENING AMERICA BY PROVIDING APPROPRIATE TOOLS REQUIRED TO INTERCEPT AND OBSTRUCT TERRORISM (USA PATRIOT ACT) ACT OF 2001, PUBLIC LAW 107–56," 2001. [Online]. Available: http://www.gpo.gov/fdsys/pkg/PLAW-107publ56/pdf/PLAW- 107p. [Accessed 5 January 2015].

[2] S. Caldwell, J. Mortin, A. Curry, C. Bausell, M. Nichols-Blake, A. Ehlow, K. Davis, M. Fejfar, E. Hauswirth, M. Karpman, T. Lombardi and J. Sam, "Critical Infrastructure Protection: DHS List of Priority Assets Needs to be Validated and Reported to Congress," Government Accountability Office, Washington, DC, 2013.

[3] G. Kabir, R. Sadiq and S. Tesfamariam, "A review of multi-criteria decision-making methods for infrastructure management," Structure and Infrastructure Engineering: Maintenance, Management, Life-Cycle Design and Performance, vol. 10, no. 9, pp. 1176-1210, 2014.

[4] US Department of Homeland Security, "NIPP 2013: Partnering for Critical Infrastructure Security and Resilience," Department of Homeland Security, Washington, DC, 2013.

[5] J. D. Moteff, "Critical Infrastructures: Background, Policy, and Implementation," Congressional Research Service, Washington, DC, 2015.

[6] United States Governement, "Homeland Security Act of 2002," Public Law, Vols. 107-296, 2002.

[7] The White House, "Presidential Policy Directive #21: Critical Infrastructure Security and Resilience," The White House, Washington, DC, 2013.

210

[8] European Union, "Council Directive 2008/114/EC on the identification and designation of European critical infrastructures and the assessment of the need to improve their protection," Official Journal of the European Union, vol. 345/75, 2008.

[9] S. Bouchon, C. D. Mauro, C. Logtmeijer, J.-P. Nordvik, R. Pride, B. Schupp and M. Thornton, "Non-binding Guidelines: For application of the Council Directive on the identification and designation of European Critical Infrastructureand the assessment of the need to improve their protection," Luxembourg: Office for Official Publications of the European Communities, Italy, 2008.

[10] "CIPedia," [Online]. Available: cipedia.eu.

[11] Australian Government, "Critical Infrastructure Relience Strategy: Policy Statement," Commonwealth of Australia, Australia, 2015.

[12] Canadian Governments, "National Strategy for Critical Infrastructure," Her Majesty the Queen in Right of Canada, Canada, 2009.

[13] The White House, "Presidential Decision Directive #63," The White House, Washington, DC, 1998.

[14] National Commission on Terrorist Attacks Upon the United States, "The 9/11 Commission Report," Government Printing Office, Washington, DC, 2004.

[15] The White House, "Executive Order 13231: Critical Infrastructure Protection in the Information Age," The White House, Washington, DC, 2001.

[16] The White House, "Homeland Security Presidential Directive #7: Critical Infrastructure Identification, Prioritization, and Protection," The White House, Washington, DC, 2001.

[17] The White House, "The National Strategy for the Physical Protection of Critical Infrastructures and Key Assets," The White House, Washington, DC, 2003.

[18] J. D. Moteff, "Critical Infrastructures: Background, Policy, and Implementation," Congressional Research Service, Washington, DC, 2011.

[19] US Department of Homeland Security, "National Infrastructure Protection Plan," Department of Homeland Security, Washington, DC, 2006.

211

[20] US Department of Homeland Security, "Progress in Developing the National Asset Database," Department of Homeland Security Office of Inspector General, Washington, DC, 2006.

[21] J. Moteff, "Critical Infrastructure: The National Asset Database," Congressional Research Service, Washington, DC, 2007.

[22] Public Law 110-53, "Implementing Recommendations of the 9/11 Commission Act of 2007," US Government Printing Office, Washington, DC, 2007.

[23] US Department of Homeland Security, "Efforts to Identify Critical Infrastructure Assets and Systems," Deparment of Homeland Security Office of Inspector General, Washington, DC, 2009.

[24] US Department of Homeland Security, "IT Program Assessment; NPPD - Critical Infrastructure Technology and Architecture," Department of Homeland Security, Washington, DC, 2011.

[25] US Government Accountability Office, "Critical Infrastructure Protection: DHS List of Priority Assets Needs to be Validated and Reported to Congress," Government Accountability Office, Washington, DC, 2013.

[26] Department of Homeland Security, "National Infrastructure Protection Plan Partnering to enhance protection and resiliency," 2009.

[27] S. Musman, A. Temin, M. Tanner, D. Fox and B. Pridemore, "Evaluating the Impact of Cyber Attacks on Missions," The MITRE Corportation, McLean, VA, 2010.

[28] IEEE, "IEEE Xplore Digital Library," [Online]. Available: http://ieeexplore.ieee.org/Xplore/home.jsp. [Accessed 1 1 2016].

[29] NPS Center for Homeland Defense and Security, "Homeland Security Affairs," [Online]. Available: https://www.hsaj.org/.

[30] Elsevier, "ScienceDirect," [Online]. Available: http://www.sciencedirect.com/. [Accessed 1 1 2016].

[31] Informa Group, "Taylor and Francis Online," [Online]. Available: http://tandfonline.com/. [Accessed 1 1 2016].

[32] Google, "Google," [Online]. Available: www.google.com. [Accessed 1 1 2014].

212

[33] Google, "Google Scholar," [Online]. Available: https://scholar.google.com/. [Accessed 1 1 2016].

[34] Department of Defense, "Defense Critical Infrastructure Protection (DCIP): DoD Mission-Based Critical Asset Identification Process (CAIP)," Department of Defense Manual, vol. 1, no. 3020.45, 2008.

[35] G. Giannopoulos, R. Filippini and M. Schimmer, "Risk assessment methodologies for Critical Infrastructure Protection. Part 1: A State of the art," European Commision, 2012.

[36] European Commission, "Commission staff working document on the review of the European Programme for Critical Infrastructure Protection (EPCIP)," 2012.

[37] J. Mottef, "Critical Infrastructure: The National Asset Database," CRS Report for Congress, vol. RL33648, 2007.

[38] D. D'Agostino, "Defense Critical Infrastructure: Actions Needed to Improve the Cosiste, ‘eliailit, ad Usefuless of DoDs Tie Task Critical Asset List," United States Government Accountability Office, 2009.

[39] US Department of Homeland Security, "National Infrastructure Protection Plan," 2009.

[40] C. Izuakor and R. White, "Critical Infrastructure Asset Identification: Policies, Methodologies, and Gap Analysis," Under Review for Publication, 2015.

[41] DHS Office of Inspector General, "Progress in Developing the National Asset Database," 2006.

[42] S. Bouchon, "The Vulnerability of Interdependent Critical Infrastructure Systems," Office of Official Publications for the European Commission, Italy, 2006.

[43] Merriam-Webster (An Encyclopedia Britannica Company), "Critical," [Online]. Available: http://www.merriam-webster.com/dictionary/critical. [Accessed 30 September 2015].

[44] M. Theoharidou, P. Kotzanikolaou and D. Grtiza, "Risk-based Criticality Analysis," in Critical Infrastructure Protection III, Hanover, Springer Berlin Heidelberg, 2009, pp. 35-49.

213

[45] S. Rinaldi, J. Peerenboom and T. Kelly, "Identifying, Understanding, and Analyzing Critical Infrastructure Dependencies," IEEE Control Systems Magazine, pp. 11-25, 2001.

[46] J. Metzger, "The Concept of Critical Infrastructure Protection," in Business and Security: Public-private Sector Relationships In a New security Environement, New York, Oxford University Press, 2004, pp. 197-209.

[47] R. Mattoili and D. C. Levy-Bencheton, "Methodologies for the Identification of Critical Infrastructure Assets and Services," ENISA, Heraklion, Greece, 2014.

[48] T. Brown, "Multiple Modeling Approaches and Insights for Critical Infrastructure Protection," in NATO Science for Peace and Security Series - D: Information and Communication Security - Computational Models of Risks to Infrastructure, vol. 13, Albuquerque, New Mexico, 2007, pp. 23-35.

[49] I. Eugeld, W. Kroger, G. Sasavini, M. Schlapfer and E. Zio, "The role of network thoery and object-oriented modeling within a framework for vulnerability analysis of critical infrastructures," Reliability Engineering and System Safety, vol. 94, pp. 954-963, 2008.

[50] J. Yusta, G. Correa and R. Lacal-Arantegui, "Methodologies and applications for critical infrastructrue protection: State-of-the-art," Energy Policy, vol. 39, pp. 6100-6119, 2011.

[51] X. Liu, W. Zhang, Z. Baber and C. Chai, "Application of social network theory to prioritizing Oil & Gas industries protection in a networked critical infrastructure system," Journal of Loss Prevention in the Process Industry, vol. 24, pp. 688-694, 2011.

[52] T. G. Lewis, Critical Infrastructure Protection In Homeland Security: Defending A Networked Nation, New Jersey: John Wiley & Sons, Inc., 2006.

[53] W. Churchman, The Systems Approach, New York: Dell Publishing Co., Inc., 1969.

[54] W. Chunlei, F. Lan and D. Yiqi, "National Critical Infrastructure Modeling and Analysis Based on Complex System Theory," in IEEE International Conference on Instrumentation, Measurement, Computer, Communication, and Control, 2011.

[55] T. Kron and G. Thomas, "Society as a Self-Organized Critical System," Cybernetics and Human Knowing, vol. 16, no. 1-2, pp. 65-82.

214

[56] R. Fekolkin, "CAS and Game Theory in Critical Infrastructure: Stuxnet attack against Iran's nuclear facility," Luleå University of Technology , 2015.

[57] D. R. White, "Towards a Unified Homeland Security Strategy: An Asset Vulnerability Model," Journal of Homeland Security Affairs, 2014.

[58] W. T. F. Encyclopedia, "Game Theory," [Online]. Available: https://en.wikipedia.org/wiki/Game_theory. [Accessed 9 October 2015].

[59] M. H. Halim and A. H. Mohamed, "Identification of critical level of assets by using analytic hierarchy process for water assets management," International Journal of Technical Research and Applications, vol. 2, no. 3, pp. 54-58, 2014.

[60] V. A. P. Salomon and J. A. B. Montevechi, "A COMPILATION OF COMPARISONS ON THE ANALYTIC HIERARCHY PROCESS AND OTHERS MULTIPLE CRITERIA DECISION MAKING METHODS: SOME CASES DEVELOPED IN BRAZIL," in ISAHP, Berne, Switerzerland, 2001.

[61] V. A. P. Salomon, "AN EXAMPLE ON THE UNRELIABILITY OF MACBETH APPLICATIONS," in International Conference on Production Research - ICPR Americas's, 2008.

[62] D. J. Dunning, Q. E. Ross and M. W. Merkhofer, "Multiattibute utility analysis; best technology available; adverse environemental impact; Clean Water Act," Environmental Science and Policy, vol. 3, no. 1, pp. -14, 2000.

[63] C. Izuakor and R. White, "Critical Infrastructure Asset Identification: Policies, Methodologies, and Gap Analysis," Journal of Critical Infrastructure Protection, 2016.

[64] D. L. Xu and D. J.-B. Yang, "Introduction to Multi-Criteria Decision Making and the Evidential Reasoning Approach," University of Manchester Institute of Science and Technology, Manchester, 2001.

[65] M. Majumder, "Multi-criteria Decision Making," in Impact of Urbanization on Water Shortage in Face of Climate Aberrations , Springer-Verlag Singapur, 2015, pp. 35-47.

[66] C. Hwang and K. Yoon, "Multiple attribute decision making in lecture notes in economics and mathematical systems," Springer-Verlag, Berlin, 1981.

215

[67] Department for Communities and Local Govenment: London, "Multi-criteria Analysis: a manual," Communities and Local Government Publications, Wetherby, 2009.

[68] K. Eldrandaly, A. H. Ahmed and N. A. Aziz, "An expert system for choosing the suitable mcdm method for solving a spatial decision problem," in 9th International Conference on Production Engineering, Design and Control, Alexandia, Egypt, 2009.

[69] A. Guitouni and J.-M. Martel, "Tentative guidelines to help choosing an appropriate MCDA method," European Journal of Operational Research, no. 109, pp. 501-521, 1998.

[70] M. Mollaghasemi and J. Pet-Edwards, Making Multi-Objective Decisions, Institute of Electrical & Electronics Engineering, 1997.

[71] C.-L. Hwang and K. Yoon, Multiple Attribute Decision Making: Methods and Applications, New York: Springer-Verlag, 1981.

[72] W. P. Fix, "Applications and Modelling Using Multi-Attribute Decision Making to Rank Terrorist Threats," Journal of Socialomics, vol. 5, no. 2, pp. 1-12, 2016.

[73] US Department of Homeland Security, The National Strategy for the Physical Protection of Critical Infrastructures and Key Assets, Washington, 2003.

[74] A. Fekete, "Common criteria for the assessment of critical infrastructures," Internaltion Journal of Disaster Risk Science, vol. 2, no. 1, pp. 15-24, 2011.

[75] International Committee of the Red Cross, "Introduction to Economic Security," Red Cross, [Online]. Available: www.icrc.org. [Accessed 2 August 2016].

[76] M. Allaby, A Dictionary of Ecology, Oxford University Press, 2004.

[77] CDC Foundation, "What is Public Health?," 2016. [Online]. Available: www.cdcfoundation.org. [Accessed 1 September 2016].

[78] Oxford Dictionaries, "Health and Safety," 2016. [Online]. Available: www.oxfordictionaries.com. [Accessed 1 September 2016].

[79] US Legal, "Public Safety Law and Legal Definition," 2016. [Online]. Available: definitions.uslegal.com. [Accessed 1 September 2016].

216

[80] Wikia, "IT Law Wiki: Public Confidence," 2016. [Online]. Available: itlaw.wikia.com. [Accessed 1 September 2016].

[81] United States Institute of Peace, "Socaial Well-being," 2016. [Online]. Available: www.usip.org. [Accessed 1 September 2016].

[82] Dictionary, "National Security," 2016. [Online]. Available: dictionary.com. [Accessed 1 September 2016].

[83] Business Dictionary, "Environmental Impact," [Online]. Available: www.businessdictionary.com. [Accessed 1 September 2016].

[84] US Department of Homeland Security, "Critical Infrastrucuture Sectors," [Online]. Available: https://www.dhs.gov/critical-infrastructure-sectors. [Accessed 1 September 2016].

[85] A. Kossiakoff, W. Sweet, S. Seymour and S. Biemer, Systems Engineering Principles and Practice, New Jersey: Wiley, 2011.

[86] American Institue of Aueronautics and Astronautics, "A Framework for Aviation CyberSecurity," 2013.

[87] Dr. Daniel P. Johnson, Honeywell Aerospace and Advanced Technology, "Civil Aviation and CyberSecurity," 2013.

[88] UK Centre for the Protection of National Infrastructure, "Cyber Security in Civil Aviation," 2012.

[89] Department of Homeland Security, "Transportation Systems Sector-Specific Plan, Annex A: Aviation," 2010.

[90] American Society of Civil Engineers, "2013 Report Card for America's Infrastructure," 2013. [Online]. Available: http://www.infrastructurereportcard.org/aviation/. [Accessed 20 September 2015].

[91] Air Transport Action Group, "Aviation Benefits Beyond Borders: Value to the economy," [Online]. Available: http://aviationbenefits.org/economic- growth/value-to-the-economy/. [Accessed 1 September 2016].

[92] S. Carter and A. Cox, "One 9/11 Tally: 3.3 Trillion," The New York Times, 8 September 2011. [Online]. Available:

217

http://www.nytimes.com/interactive/2011/09/08/us/sept-11-reckoning/cost- graphic.html?_r=1&. [Accessed 20 September 2015].

[93] Institute for Analysis of Global Security, "How much did the September 11 terrorist attack cost America?," [Online]. Available: http://www.iags.org/costof911.html. [Accessed 20 September 2015].

[94] U.S. Department of Transportation Federal Aviation Adminstration, "General Aviation Airports: A National Asset," Washington, DC, 2010.

[95] C. Izuakor, "Understanding The Impacts of Cyber Security Risks on Safety," in International Conference on Information System Security and Privacy, Rome, 2016.

[96] US Government Accountability Office, "Air Traffic Control: FAA Needs a More Comprehensive Approach to Address Cybersecurity As Agency Transitions to NextGen," 2015.

[97] A. Costin and A. Francillon, "Ghost in the Air (Traffic): On insecurity of ADS-B protocol and practical attacks on ADS-B devices," Black Hat, Sophia-Anipolis, 2012.

[98] International Civil Aviation Organization, "Initial Report on Risk Assessement of Cyber-Attack - Air Traffic Management," Montreal, 2014.

[99] Wikipedia, "Spanair Flight 5022," [Online]. Available: https://en.wikipedia.org/wiki/Spanair_Flight_5022. [Accessed 20 September 2015].

[100] D. Storm, "Hacker uses an Android to remotely attack and hijack an airplane," Computer World, 10 April 2013. [Online]. Available: http://www.computerworld.com/article/2475081/cybercrime-hacking/hacker- uses-an-android-to-remotely-attack-and-hijack-an-airplane.html. [Accessed 20 September 2015].

[101] M. Soperus, "Conficker Work Shuts Down French and UK Air Forces," MaximumPC, 10 February 2009. [Online]. Available: http://www.maximumpc.com/conficker-worm-shuts-down-french-and-uk-air- forces/. [Accessed 20 September 2015].

218

[102] K. Zetter, "Is it possible for passengers to hack commercial aircraft?," Wired, 28 May 2015. [Online]. Available: http://www.wired.com/2015/05/possible- passengers-hack-commercial-aircraft/. [Accessed 20 September 2015].

[103] A. Costin and A. Francillon, "Ghost in the Air (Traffic): On insecurity of ADS-B protocol and practical attacks on ADS-B devices," Sophia-Anipolis.

[104] International Civil Aviation Organization, "Initial Report on Risk Assessement of Cyber-Attack - Air Traffic Management," Montreal, 2014.

[105] Department of Homeland Security, "Transportation Systems Sector Specific Plan," 2010.

[106] D. D. P. Johnson, "Civil Aviation and CyberSecurity," Honeywell, 2013.

[107] RAND National Security Research Division, "RAND Database of Worldwide Terrorism Incidents," [Online]. Available: http://www.rand.org/nsrd/projects/terrorism-incidents.html. [Accessed 1 January 2016].

[108] Federal Aviation Administration, "National Plan of Integrated Airport Systems Report," [Online]. Available: http://www.faa.gov/airports/planning_capacity/npias/reports/. [Accessed 1 January 2016].

[109] Federal Aviation Adminstration, "Aeronautical Information Services - National Flight Data Center," [Online]. Available: https://nfdc.faa.gov. [Accessed 1 September 2016].

[110] United States Depoart of Transportation: Bureau of Transportation Statistics, "2014 Airline Financial Data," [Online]. Available: http://www.rita.dot.gov/bts/press_releases/bts022_15. [Accessed 1 September 2016].

[111] National Institute of Standards and Technology, "Guide for Conducting Risk Assessments," NIST, Gathersburg, 2012.

[112] PCI Security Standards Council, "PCI DSS Risk Assessment Guidelines," 2012.

[113] G. Park, A. Roberts, A. Curatolo, S. Spry, S. Marsh and D. Pannell, "Significant Asset Identification Guide," 2010.

219

[114] Canadian Society for Chemical Engineering, Risk Assessment - Recommended Practicies for Municipalities and Industry, Ottawa, 2004.

[115] Merriam-Webster , "Security; Safety," [Online]. Available: http://www.merriam- webster.com/dictionary/security.

[116] E. Albrechtsen, "Security vs Safety," Norwegian University of Science and Technology, 2003.

[117] U.S. Department of Labor, "VI. Risk Assessment," 10 August 1992. [Online]. Available: https://www.osha.gov/.

[118] Canadian Centre for Occupational Health and Safety, "Risk Assessment," 6 November 2015. [Online]. Available: http://www.ccohs.ca/oshanswers/hsprograms/risk_assessment.html.

[119] TrendMicro, "Understanding Targeted Attacks: Goals and Motives," 22 October 2015. [Online]. Available: http://www.trendmicro.com/vinfo/us/security/news/cyber- attacks/understanding-targeted-attacks-goals-and-motives.

[120] K. Zetter, "Feds Say That Banned Researcher Commandeered A Plane," 15 May 2015. [Online]. Available: http://www.wired.com/2015/05/feds-say-banned- researcher-commandeered-plane/.

[121] City University London, "SESAMO - Security and Safety Modelling," 13 December 2015. [Online]. Available: https://www.city.ac.uk/centre-for-software- reliability/research/research-projects/sesamo-project.

[122] M. Theoharidou, P. Kotzanikolaou and D. Gritzalis, "A multi-layer Criticality Assessment methodology based on interdependencies," Computers & Security, vol. 29, no. 6, pp. 643-658, 2010.

[123] G. Stergiopoulos, P. Kotzanikolsou, M. Theochardiou and D. Grizalis, "Risk Mitigation Strategies for Critical Infrastructures Based on Graph Centrality Analysis," Journal of Critical Infrastructure Protection, vol. 10, pp. 34-44, 2015.

[124] P. Katina and P. Hester, "Systemic determination of infrastructure criticality," International Journal of Critical Infrastructures, vol. 3, no. 3, pp. 211-225, 2013.

[125] K. Brown, Critical Path: A Brief History of Critical Infrastructure Protection in the United States, Fairfax, Virginia: Spectrum Publishing Group, Inc, 2006.

220

[126] J. Birchmeier, "Systematic assessment of the degree of criticality of infrastructures," Risk, Reliability and Societal Safety – Aven & Vinnem (eds), pp. 859-864, 2007.

[127] Department of Homeland Security, "National Strategy for Homeland Security," 2002.

[128] American Society of Civil Engineers, "Infrastructure Report Card," 2013. [Online]. Available: www.infrastructurereportcard.org. [Accessed 25 July 2015].

[129] Project Management Institute, A guide to the Project Management Body of Knowledge (PMBOK Guide), Project Management Institute, 2013.

[130] D. Anderson, P. Kelcher and P. Smith, "Towards an Assessment Tool for the Strategic Management of Asset Criticality," in Engineering Asset Management, Queensland, Australia, 2006.

[131] R. Aliev, W. Pedrycz, V. Kreinovich and O. Huseynov, "The General Theory of Decision," Information Sciences, vol. 327, pp. 125-148, 2015.

[132] S. Musman, A. Temin, M. Tanner, D. Fox and B. Pridemore, "Evaluating the Impact of Cyber Attacks on Missions," The MITRE Corportation, McLean, VA, 2010.

221

11 APPENDIX

Appendix A: Critical Asset Identification in Other Industries

In other disciplines, outside of critical infrastructure protection, there are varying uses of asset identification including but not limited to failure analysis, investment strategy development, and risk management. The following asset identification methods from other disciplines were reviewed based on their rank in search results, relevance in respective fields, and applicability to this research: NIST 800-30 NIST is a popular organization in the information technology industry known especially for its production of industry standards and guidelines. The NIST Risk Management Framework [111] is used by risk management professionals in various fields and targets assessment of information technology. As previously mentioned, similar trends are seen in other disciplines outlining the first step i isk assesset as idetifiatio of itial assets. NI“T is o eeptio. I the NI“T ‘MF], the fist of isk assesset atiities outlined was System Characterization. While the superseding version has been consolidated into 6 steps, the first step still remains characterization. The purpose of this step is to gain understanding of the asset environment. The system and operational environments are documented and classified into groups such as hardware, software, information, etc. Outputs of the step include characterization of IT systems, a picture of IT system environment, and delineation of system boundaries. Asset information is gathered through automated scanning tools, interviews, documentation review, and questionnaires. This information is used to identify threats and continue through the remaining RMF steps. The framework outlines three analysis risk assessment approaches: 1) Threat-oriented 2) Asset/impact-oriented 3) vulnerability-oriented. Each of the approaches requires the same steps. The difference is the sequence in which the stops are completed. The asset/impact-oriented approach is most in sync with the NIPP RMF in that the NIPP requires a complete, consequence based approach to match the formal definition of CI.

222

As described by the NIST documentation, this approach begins with the identification of undesired consequences and impacts based on mission or business impact analysis. Critical assets associated with those consequences are then identified followed by threat identification. The NIST guidelines acknowledge that the differences in starting points of risk assessment and associated critical asset identification can potentially bias esults oittig soe isks. Its suggested that the effetieess of the assesset can be improved by incorporating multiple approaches. PCI DSS Risk Assessment Guidelines The Payment Card Industry Data Security Standard (PCI DSS) Risk Assessment Guidelines [112] were published by the Payment Card Industry Security Standards Council to assist organizations that accept and process credit card data with meeting the PCI audit requirements. The guidelines outline asset identification as a component of the risk identification process. In relation to PCI, an asset is defined as anything of value to an organization and is particularly involved in processing, storing, transmitting, and protecting, card holder data. The details for how these assets are identified are left to the organization. One popular approach used in organizations is a function-based approach. Process, storing, transmitting, and protecting are critical functions in the context of the audit. Once these functions are agreed upon, the organization documents how the card holder data flows through each of these functions, also referred to as modes, and documents the involved technology, people, and processes along the way. The resulting list outlines the scope of the environment and the relevant assets. The requirement suggests that each asset have an identified owner responsible for its protection and that asset value may also be assigned based on the assets importance and criticality. The process documentation also notes that it may be helpful to structure assets into groups and sub-groups. Four critical success factors are listed for what is considered a proper assessment program. At the top of this list is asset identification. Identification plays a critical role in the identification of risks. Other criteria also include proactive approach, keeping it simple, ad taiig. I ode to aitai sipliit, its suggested that the ue of categories used for measurement should be limited to avoid unhelpful complexity. Standard definitions of values used for measurement are also key to enabling consistency and limiting subjectivity of evaluation. The guideline notes that results should be able to be interpreted consistently across users. Users of the process and results should be trained in order to have common understanding of the nature and use of the process. Lastly, as suggested by most risk management programs, the process should be proactive rather than reactive.

223

INNFER The Investment Framework for Environment Resources (INFFER) is an Australian tool used to prioritize environmental investments and has been recognized on numerous occasions for the advancements in environmental research. Within the framework, significant asset identification is the first of seven steps and requires development of a list of significant natural assets in relevant regions [113]. The second step involves filteig the step list usig a siplified set of iteia fo pioitizatio pio to conducting detailed assessment. The identification step includes 3 approaches to asset identification that can be used in tandem: 1.) Asset nominations from the community 2.) Significant asset mapping performed by technical specialists 3.) Amalgamation of assets from existing information sources such as national and state inventories. The framework provides suggested asset categorizations relevant to environmental assets and requires that all significant assets identified be capable of SMART (Specific, Measurable, Achievable, Relevant, and Time- bound) goal specification. Criteria is specified for each asset category in order to determine the assets level of significance. Significance is defined by ecological, social, and economic values. Asset profiles are created for each asset in order to document those significance values in addition to other information such as location, description, condition of asset, active threats, etc. The process can be considered a systematic multi-criteria method and starts with the generation of a landscape asset map as a baseline. Next, spatial location of the assets is identified using the aforementioned asset categories and respective criteria. In a separate effort, specifications of top 10-20 highest-value assets are requested from regional and state experts for each asset category also using step oes baseline map. In the fourth step, the lists from these two initiatives are combined and compared. The resulting list then moves on to the next step in the INFFER process and is filtered for assets requiring detailed assessment.

Risk Assessment – Recommended Practices for Municipalities and Industry The Canadian Society for Chemical Engineering published recommended practices for risk assessment in the chemical industry [114]. The methodology was developed by the Risk Assessment Expert Committee which consisted of 16 experts from diverse disciplines of engineering. The methodology uses a consequence based approach to identify hazards. The program requires development of a system description which osists of, udestadig the opoets of the faility and how it operates, establishing an inventory of hazardous substances used, transported, and manufactured, and being familiar with the surrounding area that might be affected by hazardous events in the facility, in terms of population, land-use, and cliate, et.[] I other words, what we refer to as the inventory of critical infrastructure asset in CIP 224 effort is known as an inventory of hazardous assets in this discipline. Examples of the inventory items include chemical plants, refineries, and transportation corridors in which dangerous chemicals are transported. The process is based on identification of hazardous events that lead to unwanted consequences and deliverables of the step include development of scenarios representing frequency and consequence of event. Once the scenario is identified, the undesirable consequences are identified followed by identification of the materials, systems, processes, and facility characteristics that can cause those consequences. This resulting list can be considered the inventory of hazardous assets. The documentation notes, that finer analysis results in development of a greater quantity of scenarios which can improve comprehensiveness of evaluation, but also translates to increased cost.

225

Appendix B: MCDM Evaluation List

Weighted Sum Model (WSM) Lexicographic Method Conjunctive Method Disjunctive Method Maximin Method Fuzzy Weighted Sum Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) Multi-attribute Value Theory (MAVT) Utility Additive (UTA) Simple Multi-attribute Rating Technique (SMART) Multi-attribute Utility Theory (MAUT) Analytic Hierarchy Process (AHP) Evaluation of Mixed Data (EVAMIX) Fuzzy Maximin Elimination and Choice Reflecting Reality I (ELECTRE I) Elimination and Choice Reflecting Reality II (ELECTTE II) Elimination and Choice Reflecting Reality III (ELECTRE III) Elimination and Choice Reflecting Reality IV (ELECTRE IV) Elimination and Choice Reflecting Reality IS (ELECTRE IS) Elimination and Choice Reflecting Reality TRI (ELECTRE TRI) Preference Ranking Organization Method for Enrichment of Evaluations I (PROMETHEE I) Preference Ranking Organization Method for Enrichment of Evaluations II (PROMETHEE II) MELCHIOR ORESTE REGIME NAIADE QUALIFLEX Fuzzy conjunctive/disjunctive method Martel and Zara Method VIseKriterijumska Optiizacija I Kompromisno Resenje (VIKOR) Compromise Programming (CP)

226

Appendix C: Objective and Attribute Code IDs

Unique ID Objective Title 1.a Scope - Complete assessment 1.b Scope - Ad-hoc assessment 2.a Accommodate political agenda 2.b Do not need to accommodate political agenda 3.aa Asset focus - Mobile 3.ab Asset focus - Fixed 3.ac Asset focus - Systems 3.ad Asset focus - Processes 3.ae Asset focus - People 3.af Asset focus - Services 3.ag Asset focus - Businesses 3.ca Concern - Destruction or disruption 3.cb Concern - Critical availability 3.cc Concern - Lack of substitutes and alternatives 3.ea Criteria - Economic Security 3.eb Criteria - Cascading Impacts 3.ec Criteria - Health and Safety 3.ed Criteria - Public Confidence 3.ee Criteria - Dependencies 3.ef Criteria - Social Well-being 3.eg Criteria - National Security 3.eh Criteria - Asset Location 3.ei Criteria - Sector Specific 3.ej Criteria - Business Impact 3.ex Criteria - Environmental Impact 4.a Infrastructure owners and operators 4.b Public sector decision makers 5.a Chemical 5.b Commercial Facilities 5.c Communications 5.d Critical Manufacturing 5.e Dams 5.f Defense Industrial Base 5.g Emergency Services 5.h Energy

227

5.i Financial Services 5.j Food and Agriculture 5.k Government Facilities 5.l Healthcare and Public Health 5.m Information Technology 5.n Nuclear Reactors, Materials, and Waste 5.o Transportation Systems 5.oa Transportation Systems - Aviation 5.p Sector - Water and Wastewater Systems 6.a Network-based 6.b Function-based 6.c Logic-based

Unique ID Attribute Title L.a Network-based L.b Function-based L.c Logic-based M.e Key monumental assets I.a Sector Specific Criteria I.aa Capacity I.ab Dimensions J.a Brand damage J.b Competitive loss J.c Isolated economic loss to business J.ca Loss of projected sales J.cb Loss of business assets M.c Develop worst case scenario M.d Develop consequence example M.b Consideration of alternatives and temp options A.a Cost of degraded critical services A.b Cost of restoration of critical services A.c Loss of productivity A.d % of GDP lost B.a Long duration of disruption B.b Impact of geographically widened area B.c Impact on concentrated and specialized industry of service B.d Critical notes/failure points in network impacted C.a Number of total fatalities C.b Number of prompt fatalities

228

C.c Number of injuries C.d Physical Suffering C.da Physical Suffering - Lack of water C.db Physical Suffering - Lack of food C.dc Physical Suffering - Lack of heat/energy C.dd Physical Suffering - Lack of sanitary conditions C.de Physical Suffering - Lack of housing and lodging C.df Physical Suffering - Lack of personal security D.a Possibility of rioting D.b Possibility of mass panic or fear D.c Possibility of stocking up E.a Physical E.b Cyber E.c Geographical E.d Logical F.a Infringement of freedom to travel F.b Infringement of freedom to leave accommodations F.c Inability to communicate F.d Separation from family, social networks F.e Separation from information resources F.f Unavailability of funds or payment systems F.g Mass evacuation length G.a Loss of government function G.b Loss of national defense H.a Geographical proximity to other assets H.b Concentration of assets X.a Loss of land X.b Number of people displaced I.aa Sector Specific Criteria- Chemical I.ab Sector Specific Criteria- Commercial Facilities I.ac Sector Specific Criteria- Communications I.ad Sector Specific Criteria- Critical Manufacturing I.ae Sector Specific Criteria- Dams I.af Sector Specific Criteria- Defense Industrial Base I.ag Sector Specific Criteria- Emergency Services I.ah Sector Specific Criteria- Energy I.ai Sector Specific Criteria- Financial Services I.aj Sector Specific Criteria- Food and Agriculture I.ak Sector Specific Criteria- Government Facilities I.al Sector Specific Criteria- Healthcare and Public Health I.am Sector Specific Criteria- Information Technology 229

I.an Sector Specific Criteria- Nuclear Reactors, Materials, and Waste I.ao Sector Specific Criteria- Transportation Systems I.ao1 Sector Specific Criteria-Transportation Systems - Aviation I.ao1a Sector Specific Criteria-Transportation Systems - Aviation - Airports I.ao1b Sector Specific Criteria-Transportation Systems - Aviation - Aircraft I.ao1c Sector Specific Criteria-Transportation Systems - Aviation - ATM I.ao1d Sector Specific Criteria-Transportation Systems - Aviation - Airlines I.ap Sector Specific Criteria-Sector - Water and Wastewater Systems N.a Government named assets

230

Appendix D: Expanded CIAid Objective Framework

ID Objective Description Source 1.a Scope - Consideration of all asset types Complete assessment 1.b Scope - Ad-hoc A partial assessment of a assessment predetermined set of assets 2.a Accommodate Some stakeholders may wish to political agenda address political disputes in asset identification by creating an option to name specific assets and/or classes 2.b Do not need to Some stakeholders may not wish to accommodate address political disputes in asset political agenda identification by creating an option to name specific assets and/or classes 3.aa Asset focus - Assets that do not have a fixed Mobile location and usually have frequent change in location 3.ab Asset focus - Assets with a fixed facility and a fixed Fixed location 3.ac Asset focus - A cluster of two of more connected Systems assets 3.ad Asset focus - The nature in which assets are used Processes to deliver functions and the steps involved in that delivery 3.ae Asset focus - Human assets that manage the People processes 3.af Asset focus - Summation of what assets, systems, Services processes, and people exist to support and provide. 3.ag Asset focus - Businesses are organizations and Businesses entities that provide services 3.ca Concern - Loss of an assets ability to fulfill its Destruction or intended function disruption 3.cb Concern - Critical The level of availability required to availability sustain a function

231

3.cc Concern - Lack of Lack of alternative means to substitutes and maintain function when asset is alternatives destroyed or disrupted 3.ea Criteria - The ability of individuals, households https://www.icrc.o Economic or communities to cover their rg/en/document/in Security essential needs sustainably and with troduction- dignity economic-security 3.eb Criteria - A sequence of events in which each http://www.encycl Cascading produces the circumstances opedia.com/doc/1 Impacts necessary for the initiation of the O14- next. cascadeeffect.html 3.ec Criteria - Health Public health is the science of http://www.cdcfounda and Safety protecting and improving the health tion.org/content/what- public-health of families and communities through

promotion of healthy lifestyles,

research for disease and injury http://www.oxforddicti prevention and detection and onaries.com/us/definiti control of infectious diseases. Public on/american_english/h Safety refers to the welfare and ealth-and-safety protection of the general public. The primary goal is prevention and http://definitions.u protection of the public from slegal.com/p/public dangers affecting safety such as -safety/ crimes or disasters. 3.ed Criteria - Public Trust bestowed by citizens based on http://itlaw.wikia.c Confidence demonstrations and expectations of: om/wiki/Public_co Thei goeets ailit to poide nfidence for their common defense and economic security and behave consistent with the interests of society; and their critical ifastutues ailit to poide products and services at expected levels and to behave consistent with thei ustoes est iteests. 3.ee Criteria - The nature in which one Dependencies infrastructure asset relies on and/or support another 3.ef Criteria - Social Social well-being is an end state in http://www.usip.or Well-being which basic human needs are met g/guiding- and people are able to coexist principles- peacefully in communities with stabilization-and- opportunities for advancement. This reconstruction-the- 232

end state is characterized by equal web-version/10- access to and delivery of basic needs social-well-being services (water, food, shelter, and health services), the provision of primary and secondary education, the return or resettlement of those displaced by violent conflict, and the restoration of social fabric and community life.

3.eg Criteria - A collective term for the defense and http://www.diction National Security foreign relations of a country, ary.com/browse/na protection of the interests of a tional-security country 3.eh Criteria - Asset Geographical location of the asset. Location This can be defined at various level, such as county, state, country, coordinates, etc. 3.ej Criteria - The financial impact the loss of the Business Impact asset has on the business 3.ex Criteria - Possible adverse effects caused by a Read more: Environmental development, industrial, or http://www.busine Impact infrastructural project or by the ssdictionary.com/d release of a substance in the efinition/environm environment. ental- impact.html#ixzz4G YocXtZr 4.a Infrastructure Individuals or entities that own Europe JRC owners and and/or operation infrastructure. vulnerability of operators Majority of operators are private interdependent sector entities. E.g. in aviation, systems commercial airlines operate a large share of the infrastructure.

The ai oe of the oes of infrastructures, most often private ones, is the reliability of the service delivery. The criticality of their infrastructure lays in the potential loss of quality, competitiveness, reliability of the service delivered. Criteria are therefore for them business continuity, infrastructure 233

reliability and service opetitieess. 4.b Public sector Individuals responsible for the decision makers identification and protection of critical infrastructure assets usually employed or back by the government/public sector 5.a Chemical The sector can be divided into five https://www.dhs.g main segments, based on the end ov/chemical-sector product produced: Basic chemicals, Specialty chemicals, Agricultural chemicals, Pharmaceuticals, Consumer products 5.b Commercial The Commercial Facilities Sector https://www.dhs.g Facilities includes a diverse range of sites that ov/commercial- draw large crowds of people for facilities-sector shopping, business, entertainment, or lodging. Facilities within the sector operate on the principle of open public access, meaning that the general public can move freely without the deterrent of highly visible security barriers. The majority of these facilities are privately owned and operated, with minimal interaction with the federal government and other regulatory entities. 5.c Provider of voice services and https://www.dhs.g Communications additional connectivity through ov/communication terrestrial, satellite, and wireless s-sector transmission systems. The transmission of these services has become interconnected; satellite, wireless, and wireline providers depend on each other to carry and terminate their traffic and companies routinely share facilities and technology to ensure interoperability. 5.d Critical The Critical Manufacturing Sector https://www.dhs.g Manufacturing identified several industries to serve ov/critical- as the core of the sector: 1.) Primary 234

Metals Manufacturing (e.g. iron and manufacturing- steel mills and ferro alloy sector manufacturing, alumina and aluminum Production and processing, nonferrous metal production and processing 2.) Machinery manufacturing (e.g. engine and turbine manufacturing, power transmission equipment manufacturing, earth moving, mining, agricultural, and construction equipment manufacturing 3.) Electrical equipment, appliance, and component manufacturing, electric motor manufacturing, transformer manufacturing, and generator manufacturing 3.) Transportation Equipment Manufacturing (e.g. Vehicles and commercial ships manufacturing, aerospace products and parts manufacturing, locomotives, railroad and transit cars, and rail track equipment manufacturing. Products made by these manufacturing industries are essential to many other critical infrastructure sectors. 5.e Dams Delivers critical water retention and https://www.dhs.gov/d control services, including ams-sector hydroelectric power generation, municipal and industrial water supplies, agricultural irrigation, sediment and flood control, river navigation for inland bulk shipping, industrial waste management, and recreation. 5.f Defense Industrial complex that enables https://www.dhs.g Industrial Base research and development, as well ov/defense- as design, production, delivery, and industrial-base- maintenance of military weapons sector systems, subsystems, and

235

components or parts, to meet national military requirements. 5.g Emergency A system of prevention, https://www.dhs.g Services preparedness, response, and ov/emergency- recovery elements that represent the services-sector nation's first line of defense in the prevention and mitigation of risk from both intentional and unintentional manmade incidents, as well as from natural disasters. 5.h Energy The energy infrastructure is divided https://www.dhs.g into three interrelated segments: ov/energy-sector electricity, oil, and natural gas. The reliance of virtually all industries on electric power and fuels means that all sectors have some dependence on this sector. 5.i Financial Includes depository institutions, https://www.dhs.g Services providers of investment products, ov/financial- insurance companies, other credit services-sector and financing organizations, and the providers of the critical financial utilities and services that support these functions. 5.j Food and Farms, restaurants, registered food https://www.dhs.g Agriculture manufacturing, processing, and ov/food-and- storage facilities. agriculture-sector 5.k Government Includes a wide variety of buildings, https://www.dhs.g Facilities that are owned or leased by federal, ov/government- state, local, and tribal governments. facilities-sector Many government facilities are open to the public for business activities, commercial transactions, or recreational activities while others that are not open to the public contain highly sensitive information, materials, processes, and equipment. These facilities include general-use office buildings and special-use military installations, embassies, courthouses, national laboratories, and structures that may house critical equipment, systems, 236

networks, and functions. In addition to physical structures, the sector includes cyber elements that contribute to the protection of sector assets (e.g., access control systems and closed-circuit television systems) as well as individuals who perform essential functions or possess tactical, operational, or strategic knowledge. 5.l Healthcare and Protects all sectors of the economy https://www.dhs.g Public Health from hazards such as terrorism, ov/healthcare- infectious disease outbreaks, and public-health- natural disasters. sector 5.m Information Businesses, governments, academia, https://www.dhs.g Technology and private citizens are increasingly ov/information- dependent upon Information technology-sector Technology functions. These virtual and distributed functions produce and provide hardware, software, and information technology systems and services, and—in collaboration with the Communications Sector—the Internet. 5.n Nuclear Nuclear power can contribute https://www.dhs.g Reactors, significantly to national electrical ov/nuclear- Materials, and generation. This sector is usually reactors-materials- Waste comprised of: nuclear power plants, and-waste-sector non-power nuclear reactors used for research, testing, and training, manufacturers of nuclear reactors or components, radioactive materials used primarily in medical, industrial, and academic settings, nuclear fuel cycle facilities, decommissioned nuclear power reactors, transportation, storage, and disposal of nuclear and radioactive waste. 5.o Transportation Transportation can be organized in https://www.dhs.g Systems the following categories: 1.) Aviation ov/transportation- includes aircraft, air traffic control systems-sector systems, airports, heliports, and landing strips. This includes 237 commercial aviation services at civil and joint-use military airports, heliports, and sea plane bases. In addition, the aviation mode includes commercial and recreational aircraft (manned and unmanned) and a wide-variety of support services, such as aircraft repair stations, fueling facilities, navigation aids, and flight schools. 2.) Highway and Motor Carrier encompasses roadways, bridges, and tunnels. Vehicles include trucks, including those carrying hazardous materials; other commercial vehicles, including commercial motor coaches and school buses; vehicle and driver licensing systems; traffic management systems; and cyber systems used for operational management. 3.) Maritime Transportation System consists of miles of coastline, ports, waterways, and intermodal landside connections that allow the various modes of transportation to move people and goods to, from, and on the water. 4.) Mass Transit and Passenger Rail includes terminals, operational systems, and supporting infrastructure for passenger services by transit buses, trolleybuses, monorail, heavy rail—also known as subways or metros—light rail, passenger rail, and vanpool/rideshare. 5.) Pipeline Systems consist of miles of pipelines spanning the country. Above-ground assets, such as compressor stations and pumping stations, are also included. 6.) Freight Rail consists of carriers, railroads, freight cars, and locomotives. 7.) Postal and Shipping

238

includes large integrated carriers, regional and local courier services, mail services, mail management firms, and chartered and delivery services. 5.oa Transportation See 5.o Systems - Aviation 5.ob Transportation See 5.o Systems - Railway 5.oc Transportation See 5.o Systems - Motor Vehicle 5.od Transportation See 5.o Systems - Maritime 5.p Sector - Water Includes public drinking water https://www.dhs.gov/ and Wastewater systems and wastewater treatment water-and-wastewater- systems-sector Systems systems used to provide safe drinking water ATTRIBUTE DEFINITIONS L.a Network-based Network-based approaches identify CIP conference all nodes and relationships in the paper system and use that system mapping as a basis for the evaluation. L.b Function-based Function-based approaches, also CIP conference referred to as mission-based paper approaches, begin the identification process by first identifying the functions that are critical to the mission of the organization. Assets that support those functions are then identified and evaluated against other defined criteria. L.c Logic-based In logic-based approaches, assets are CIP conference seleted ased o est judget paper of the assessor. M.e Key monumental Key assets and high profile events https://www.dhs.gov/x assets are individual targets library/assets/Physical_ Strategy.pdf whose attack—in the worst-case scenarios—could

239

result in not only large-scale human casualties and property destruction, but also profound damage to national prestige, morale, and confidence. I.a Sector Specific Special criteria identified for Criteria application to a specific critical infrastructure sector or function J.a Brand damage The negative impact an event, such as a lost asset or function has on the reputability and public confidence in an entity J.b Competitive loss Reduced ability to effectively compete in the current marketplace J.c Isolated Financial loss directly resulting from economic loss to an event, such as the loss of an asset business or function J.ca Loss of projected Financial loss in terms of projected sales future sales or income no longer realized as a result of an event, such as the loss of an asset or function J.cb Loss of business Direct loss of asset to the business assets M.c Develop worst Evaluator is required to develop a case scenario realistic scenario that represents the worst imaginable case in which the infrastructure asset may be degraded or lost M.d Develop Evaluator is required to develop a consequence realistic scenario that represents an example example of the consequences that maybe realized if an infrastructure asset is degraded or lost M.b Consideration of Calculation takes into account EPCIP non-binding alternatives and whether alternatives or temporary guidelines temp options solutions may be found, including the additional costs these incur. A.a Cost of degraded Cost associated with reduced of EPCIP non-binding critical services ability to provide critical function, guidelines service, or goods

240

The eooi iteia is ealuated based on the impact of infrastructure failure on the dynamics of national economies (macro perspective), rather than on individual actors (micro perspective). In other words, a distinction is made between losses to private actors (often called private or financial losses) and losses to society as a whole (often called social or economic losses). Within the context of evaluating the economic criteria private losses shall not be taken into account, since these losses do not necessarily affect the eoo as a hole. A.b Cost of Cost associated with temporarily or restoration of fully restoring services to their critical services expected and acceptable performance level A.c Loss of Loss of ability to provide critical productivity function, service, or goods A.d % of GDP lost Percentage of national gross domestic product lost B.a Long duration of Disruption of normal function that disruption exceeds timeframe deemed acceptable by the nation B.b Impact of Disruption of normal function that geographically exceeds defined geographical area. widened area This could mean impact that crosses country or state borders to impact multiple areas. B.c Impact on Disruption of normal function in concentrated geographical area containing high and specialized concentration of assets industry of service B.d Critical Disruption of normal function in a notes/failure core asset that acts as a critical point points in in a network and can result in a network single point of failure for other impacted connected nodes.

241

C.a Number of total Total number of lives lost as a result fatalities of impact C.b Number of Total number of lives immediately prompt fatalities lost as a result of impact C.c Number of Total number of people injured as a EPCIP non-binding injuries result of impact; An injured person guidelines could be defined as a person requiring more than 24 hours of hospitalization. C.d Physical Suffering C.da Physical Total number of people impacted by Suffering - Lack lack of water caused by asset of water disruption or loss C.db Physical Total number of people impacted by Suffering - Lack lack of food caused by asset of food disruption or loss C.dc Physical Total number of people impacted by Suffering - Lack lack of heat/energy caused by asset of heat/energy disruption or loss C.dd Physical Total number of people impacted by Suffering - Lack lack of sanitary conditions caused by of sanitary asset disruption or loss conditions C.de Physical Total number of people impacted by Suffering - Lack lack of housing and lodging caused of housing and by asset disruption or loss lodging C.df Physical Total number of people impacted by Suffering - Lack lack of personal security caused by of personal asset disruption or loss security D.a Possibility of Likelihood of rioting and number of rioting people impacted by such riots D.b Possibility of Likelihood of mass panic or fear and mass panic or number of people impacted fear D.c Possibility of Likelihood of requirement to stock stocking up up on goods and number of people impacted by such requirement E.a Physical two infrastructures are physically Gheorghe and interdependent if Schlapfer, 2004

242

the state of each is dependent on the material output of the other E.b Cyber if the state of an infrastructure Gheorghe and depends on Schlapfer, 2004 information transmitted through the information infrastructure E.c Geographical if a local environmental event can Gheorghe and create state Schlapfer, 2004 changes in all of them E.d Logical two infrastructures are logically Gheorghe and interdependent if Schlapfer, 2004 the state of each depends on the state of the other via a mechanism that is not a physical, cyber or geographic connection F.a Infringement of Number of people whose freedom to freedom to travel is impacted by disruption of travel critical function or asset. F.b Infringement of Number of people unable to leave freedom to leave accommodations as a result of accommodation disruption of critical function or s asset. F.c Inability to Number of people unable to communicate communicate as a result of disruption of critical function or asset. F.d Separation from Number of people separated from family, social family and/or social networks as a networks result of disruption of critical function or asset. F.e Separation from Number of people separated from information information sources as a result of resources disruption of critical function or asset. F.f Unavailability of Number of people unable to access funds or funds or payment systems as a result payment of disruption of critical function or systems asset. F.g Mass evacuation Length of evacuation exceeding (x) length amount of people that is considered beyond an acceptable timeframe by the nation

243

G.a Loss of Disruption of a public sector function government that results in national impact function G.b Loss of national Disruption of nations ability to defense deploy forces and/or defend nation from threats H.a Geographical Distance between infrastructure proximity to assets determined by nation to be other assets close enough to experience cascading impacts, usually measured in miles H.b Concentration of Volume of infrastructure assets in a assets defined geographical area X.a Loss of land the economic value associated with EPCIP non-binding the loss of a geographical area. The guidelines value can be determined by the possible contribution of the use of this land to the national income X.b Number of the economic effect of the EPCIP non-binding people displaced displacement of people has on the guidelines national economy. The can include the cost incurred by the nation to relocate the displaced persons (such as shelter, transport, food etc) and its impact on the national economy I.ao1 Sector Specific Aircraft Class Criteria- Capacity, Type Transportation Systems - Aviation I.ao1 Sector Specific Airport a Criteria- Number of flights per day / Annual Transportation Passenger Traffic Systems - Aviation I.ao1 Sector Specific Airline b Criteria- % of national air traffic | % of global Transportation air traffic Systems - Aviation I.ao1 Sector Specific Air Traffic Control c Criteria- # of flights managed Transportation 244

Systems - Aviation M.a Qualification Percentage of all criteria that needs Ratio to be met in order to qualify an asset as critical. (e.g. 100%, 75%, etc.) N.a Government Asset specifically named by a public named assets sector leader with the authority to do so, even when the asset may not meet criticality criteria

245

Appendix E: CIAid User Guide

CIAid User Guide

By: Christine Izuakor

246

1 Table of Contents Tool Description ...... 248

Example Tool Use Cases ...... 249

Using the Tool ...... 250

Interpreting the Results ...... 265

Additional Methodology Considerations ...... 272

Program Adoption and Incentivization ...... 273

Summary ...... 274

247

2 Tool Description

The purpose of the CIAid tool is the guide the development of an asset identification program based on the program objectives. The overall identification process includes five steps shown throughout this section. The CIAid tool addresses the first three steps, with the last two managed by the user beyond the tool.

Identify Determine Select Criticality Objectives and Baseline Asset Criteria Set Scope List

Conduct Obtain Critical Criticality Asset List Assessment

248

Task Description Role Step 1: Identify During this step, stake holders consider Completed Objectives and Set identification goals relating to all components within CIAid Scope of the framework in order to establish an Tool assessment scope and process. This includes, but is not limited to determining which asset types (e.g. mobile, cyber, etc.) are in scope of the assessment, relevant sectors to be included, etc. Step 2: Determine A recommended approach, (e.g. network- Baseline Asset List based, function-based, logic-based) selected based on objectives gathered from the prior phase, is then used to gather an initial baseline asset list to be evaluated for criticality Step 3: Select Using the objectives established during Step 1, Criticality Criteria appropriate assessment criteria categories and points can be gathered. At this point, the stakeholder should also consider the evaluation details, such as qualification ratio, which is further explained in subsequent sections. Step 4: Conduct The established criteria list and evaluation Completed Criticality method are applied to the baseline asset list by user Assessment established in Step 2 in order to filter out outside of assets that meet the criticality criteria. CIAid tool Step 5: Obtain Critical This assessment process will result in a list of Completed Asset List assets that are critical in accordance with the by user objectives of the stakeholder. This list can outside of then feed the risk assessment process, the CIAid tool next step in the RMF. Using this process, detailed risk assessment can be limited to only critical assets. 3 Example Tool Use Cases The anticipated users of the system will be public sector organizations, researchers, and private sector companies operating critical infrastructure assets.

249