A Meta-Study on the Key Issues for Organisations to Implement Cyber

INSTITUTE OF SECURITY AND GLOBAL AFFAIRS FACULTY OF GOVERNANCE AND GLOBAL AFFAIRS – LEIDEN UNIVERSITY

MASTER THESIS CRISIS AND SECURITY MANAGEMENT

Cyber resilience in organisations A meta-study on the key issues for organisations to draft a cyber resilience policy

Program: Master Crisis and Security Management Student: Meret Keeris Student number: 2106183 Date of submission: 13 January 2019 Subject: Cyber resilience in organisations Word count: 19954 Thesis Supervisor: Dr. E. de Busser Second Reader: Dr.ir. V. Niculescu-Dinca Cyber resilience in organisations Master Thesis Meret Keeris

Contents 1. Introduction ...... 3 2. Context ...... 5 2.1 Current frameworks ...... 5 2.2 Need for research ...... 6 3. Methods ...... 7 3.1 Literature review ...... 7 3.2 Case studies ...... 8 4. Introduction to the topic ...... 10 4.1 Definitions and bordering concepts ...... 10 4.2 Cyber security versus cyber resilience ...... 11 4.3 The need for cyber resilience ...... 12 4.4 Cyber resilience of European organisations ...... 13 5. Literature analysis ...... 17 5.1 Factor 1: Understanding the organisation ...... 17 5.1.1 Organisational identity: culture, awareness, behaviour ...... 17 5.1.2 Asset identification ...... 19 5.2 Factor 2: Effective management ...... 20 5.2.1 Risk assessments ...... 20 5.2.2 Financial resources ...... 21 5.2.3 Deciding on measures ...... 22 5.2.4 Roles and Responsibilities ...... 23 5.2.5 Response ...... 24 5.3 Factor 3: The cyber chain ...... 26 5.3.1 Interdependencies ...... 26 5.3.2 Cooperation ...... 27 5.3.3 Ecosystem ...... 28 5.4 Factor 4: Regulation ...... 29 5.4.1 Purpose of regulation ...... 29 5.4.2 Regulation on the protection of critical infrastructures ...... 30 5.4.3 Regulation on data protection...... 32 6. Case studies ...... 37 6.1 Case study 1: The DigiNotar incident ...... 37 6.1.1 The hack explained ...... 37 6.1.2 Technical security of DigiNotar’s systems ...... 39

Cyber resilience in organisations Master Thesis Meret Keeris

6.1.3 Awareness, assets and risks ...... 39 6.1.4 Roles and responsibilities ...... 40 6.1.5 Response ...... 41 6.1.6 The cyber chain ...... 41 6.1.7 Regulation ...... 42 6.1.8 Recommendations of the Dutch Safety Board ...... 42 6.1.9 Response of the Dutch governmental crisis structure ...... 44 6.2 Case study 2: The WannaCry attack at the NHS England ...... 45 6.2.1 The attack explained ...... 45 6.2.2 Technical security of NHS’s systems ...... 46 6.2.3 Awareness, assets and risks ...... 46 6.2.4 Roles and responsibilities ...... 46 6.2.5 Response ...... 47 6.2.6 The cyber chain ...... 48 6.2.7 Recommendations of the NAO ...... 48 7. Discussion and Conclusions ...... 49 7.1 Discussion of results ...... 49 7.2 Conclusions ...... 54 7.3 Limitations of the research ...... 55 7.4 Future research options ...... 55 Bibliography ...... 57

Cyber resilience in organisations Master Thesis Meret Keeris

1. Introduction Yearly 1.5 million cyber-attacks translate to 4000 attacks every day, 170 per hour and almost three every minute (Deloitte, 2016), associated with an average cost of $3.86 million U.S. dollars for each successful attack resulting in a breach, where the average time to identify a breach is 196 days (IBM, 2018). These numbers demonstrate the hardship in securing cyberspace. High pace developments in technology constantly throw up new challenges (Linkov, 2013). Organisations can therefore no longer deny the need for cyber security and are urged to implement security measures. However, the rising number and complexity of cyber- attacks makes preventing cyber incidents a difficult – if not impossible – endeavour. Rather than aiming to prevent cyber incidents from happening, the focus has shifted towards cyber resilience, which helps to bounce back, minimise the impact and ensure process continuity (Conklin, 2017). Furthermore, cyber resilience is not a merely technical issue, but encompasses organisational, business, financial and legal aspects (Van der Meulen, 2015). Technology is the accessory, and for organisations to improve cyber resilience there is need for a comprehensive approach that considers the multiple dimensions and aspects of the issue. Current cyber resilience frameworks offer clear overviews of five main steps in the process towards cyber resilience, but often lack the wider organisational context and a connection to practice (Bagheri, 2017) (Knapp, 2009). This thesis presents a comprehensive overview of the issues that are currently left out, underexposed, or sprawled across the literature on cyber resilience. It adopts a helicopter view to the literature and combines strategic, organisational and legal perspectives to increase the understanding of organisations and academics on the non-technical aspects of cyber resilience. This multidisciplinary approach to cyber resilience answers the question: “Which factors should organisations minimally consider when drafting a cyber resilience policy?”

This thesis is organised as follows. It starts by shaping the context of this thesis by discussing related work and the need for this research in chapter 2. After discussing the methods of this research in chapter 3, in chapter 4 it provides an overview of the relevant concepts and theory. Chapter 4 also discusses the implementation problems of organisations with regard to cyber resilience and the issues that arise with regard to the need for cyber resilience. From there, the thesis dives into the literature in chapter 5, with the aim of decomposing this cyber resilience problem into a limited number of key factors. These factors are the corner stones for identifying the flaws in organisations and can be used to ensure cyber resilience. The usefulness of the discovered factors is illustrated by performing two case studies in chapter 6: the DigiNotar hack

Cyber resilience in organisations Master Thesis Meret Keeris of 2011 and the WannaCry ransomware attack of 2017. Altogether, in chapter 7, this research offers a synthesis of knowledge on cyber resilience and adds to the understanding of the organisational side of cyber resilience by presenting the conclusions of this research.

Cyber resilience in organisations Master Thesis Meret Keeris

2. Context

2.1 Current frameworks Research into existing cyber resilience and security frameworks shows that the four abilities that denote resilience – plan or prepare, absorb, recover and adapt – form the basis for most frameworks on cyber resilience (Linkov, 2013). The frameworks are accordingly structured by five steps: identify, prevent, detect, respond and recover. Conklin & Shoemaker (2017) even work with seven principles of the cyber resilience process: classify, risk, rank, design or deploy, test, recover and evolve. In general, the steps allow for an adequate process of preparation, monitoring, early detection of disruptions and rapid anticipation, thereby ensuring process continuity and quick recovery. Such frameworks are established to act as a basis to improve cyber resilience and security practices of organisations. It helps to assess organisational cybersecurity operations and to identify gaps in the strategy or process (Keys, 2016).

Furthermore, NOREA - a professional association of Information Technology auditors in the Netherlands - analysed and assessed several existing cyber security standards and frameworks on their ability to guide in improving cyber resilience. The research covered the ISF-Cyber Resilience Framework, the SANS-Critical Controls for Effective Cyber Defense, the ISACA- Cybercrime Audit/Assurance Program, the PAS 555-Cyber security risk, Governance and management, the NIST-Cybersecurity Framework, and the ISO27032-Guidelines for cybersecurity. NOREA assessed these frameworks on the following seven elements: (1) organisation and governance, (2) behaviour and culture, (3) focus on value chain and stakeholders or on risks, (4) insight into use of technologies and technology landscape, (5) law and regulation, (6) detection, and (7) reaction. A score on value was given for each category and indicates the extent to which the relevant standard or framework provides handles (measures or guidance) to deal with cyber risks (NOREA, 2015).

An analysis of the scored values points out that in particular (2) behaviour and culture and (3) focus on value chain/stakeholders or on risks scored low in most of the frameworks. Also (1) organisation and governance, (5) law and regulation and (7) reaction were considered insufficiently addressed to provide good guidance (NOREA, 2015). More elaborately, NOREA’s assessment of current cyber resilience frameworks leads to the following findings: Current approaches to cyber resilience declare that cyber resilience is not merely an IT issue but should be integrated in the entire organisation. There is a need for measures to both the IT infrastructures as for the organisation’s culture, awareness and behaviour. To determine the

Cyber resilience in organisations Master Thesis Meret Keeris necessary measures, the frameworks indicate that asset management and risk analyses are important and that responsibilities, authorities, resources should be assigned specifically. The frameworks, however, do not provide further specification on this. Also, even though most of the frameworks acknowledge the importance of looking beyond the perimeters of the organisation, most still focus on internal risks rather than on the organisation’s environment. The last issue that is underexposed in current cyber resilience frameworks is the role and impact of regulation and legislation (NOREA, 2015). NOREA’s assessment was helpful to identify which issues are relevant, which are relevant but underexposed, and which are still neglected.

2.2 Need for research The literature on cyber resilience in organisations points out the need for more research on resilience approaches. Bagheri and Ridle’s review (2017) of current literature on the organisational aspects of cyber resilience concluded that most of the academic research into cyber resilience focuses on the technical aspect, rather than organisational aspects. Organisations tend to be over-reliant on technologies to be resilient, but cyber resilience demands for a combined contribution of people, process and technology (The Stationary Office, 2015). If organisations are not aware of the organisational factors that influence cyber resilience, it will be problematic to ensure a sufficient level of protection of their cyberspace (Bagheri, 2017). Therefore, they argue more research should be done on organisational aspects of cyber resilience and best practices should be shared. This will help to better depict the larger organisational context of cyber resilience. Also information security management expert Knapp (2009) argues there are gaps in the literature on cyber and information security policies. He argues for research that combines academic and practitioner sources, to bring theory and practice closer together. The resulting knowledge will offer the guidance organisations need to establish cyber resilience practices and policies (Knapp, 2009).

This research uses existing frameworks and knowledge as a basis to fill the indicated gaps. The aim is to provide an overview of the most important issues to consider when determining the cyber resilience policy. The next section explains the methods for systematically answering that question and building the overview.

Cyber resilience in organisations Master Thesis Meret Keeris

3. Methods This research examines a broad range of literature to explore the most important non-technical issues for cyber resilience in organisations. The main contributions can be summarised as follows. This thesis conducted a multidisciplinary examination on a broad range of literature to establish a gap: the problem of cyber resilience in organisations consists of much more than technical aspects. After establishing this gap, the multidisciplinary approach enabled to look at the problem of cyber resilience from new angles. This led to the identification of four factors which should be integrated into the cyber resilience policy of an organisation to improve its cyber resilience.

3.1 Literature review This research was conducted by desk research. The larger part consists of an extensive literature review to identify the main factors that form the basis of this research. Within the main factors several subcategories on relevant issues have been established. Structured this way, it should provide a comprehensive but comprehendible approach to cyber resilience. Since this research adopts a helicopter view, the initial analysis into cyber resilience in organisations started broad by consulting a large variety of sources on cyber resilience. It was noted that certain issues gain more attention than others. Consequently, a more in-depth examination of these issues confirmed their relevance and led to the synthesis of ‘sub-issues’ into four main factors: (1) understanding the organisation, (2) effective management, (3) the cyber chain and (4) regulation. Subsequently, the literature review was targeted at these factors to further confirm the relevance of these factors and understand their implications.

Cyber resilience in organisations Master Thesis Meret Keeris

To ensure a broad scope, the concept of ‘organisations’ has been left generic on purpose. Hence, this meta-study provides handles and knowledge for organisations to better address the identified issues in cyber resilience, while leaving room for individual interpretation and implementation by the specific organisation. While most parts of this research are generally applicable to all kinds and origins of organisations, there is a slight focus on European organisations. This can be noticed in particular in several numbers and examples, and in chapter 5.4 Regulation, which mainly discusses European legislation. Also the case studies are European. The motivation for a European focus is that current literature on cyber resilience in European organisations is rather limited. Moreover, it allows for a slight demarcation of the topic, while keeping it generic. Combined with recent European regulations and cyber security initiatives it offers an even more relevant research focus.

The literature review consulted a mixture of academic and practitioner sources such as scholarly articles, policy papers, government papers, company reports, organisational publications and media publications. By combining scholarly literature with practitioner knowledge, this thesis is able to present enriched insights into cyber resilience strategies for organisations. The academic sources include scholarly articles written in the field of organisation science, crisis and security studies, computer science, and closely related fields. The practitioner knowledge has mostly been derived from publications of consultancies, research institutions and government or policy papers. Consultancies are positioned close to the practice of cyber resilience. They experience the challenges faced by organisations, and how organisations put measures into practice. Furthermore, consultancies generally conduct field research and therefore possess actual and practical information. Taking into account the position and interests of consultancies, their reports offer valuable information to map the current practices and situation of cyber resilience in organisations. Furthermore, publications from research institutions can perhaps best be categorised roughly in between academic and practitioner sources, and therefore conveniently supplement both. Finally, government or policy papers include cyber security assessments and national cyber security reports. Also reports from the cyber security councils that consult both governments and organisations form part of the literature. Incorporating such diversity of sources and disciplines allows for the helicopter view this research aims for.

3.2 Case studies The two case studies that are be discussed are the DigiNotar hack in the Netherlands in 2011, and the WannaCry crypto-ransomware attack at the National Health Service (NHS) in the

Cyber resilience in organisations Master Thesis Meret Keeris

United Kingdom in 2017. The case studies serve an illustrative purpose to the research. They help to validate the findings of this research by demonstrating the identified factors in practice. For the case study on the DigiNotar incident, the official investigation report of the Dutch Safety Board on the hack is used. The Dutch Safety Board investigated how governmental organisations on administrative level performed in ensuring digital security. For the case study on the WannaCry cyber-attack at the NHS in the UK, the official investigation report of the National Audit Office (NAO) of the UK is used. The NAO investigated the impact of the WannaCry attack on the NHS England specifically and focused on the NHS’s response to the attack.

The cases are analysed from the perspective of the Dutch government authorities and the National Health Service respectively, since those actors have been harmed most from the cyber incidents and are therefore considered the organisations that need to be cyber resilient. The relationship between the two cases is that in both cases the organisations are not considered the most vulnerable for cyber incidents at first sight. The identification of assets, however, proves the organisations actually are vulnerable. In the DigiNotar case the Dutch government never contemplated the possibility that digital certificates – essentially the mean to protect data – could be compromised. When this actually happened, the government could not ensure the protection of their assets: the authenticity of communication and data of citizens, businesses and government institutions. For the NHS the continuity of health services and the protection of patient data are crucial. Both of which assets were seriously threatened by the WannaCry attack.

Furthermore, the selected cases cover two of the most occurring types of cyber threats in Europe: data breaches (DigiNotar) and malwares (WannaCry) (EESC, 2018) and hence are representative. Moreover, the cases are two of the few cyber incidents upon which the investigation has been completed and the investigation reports have been published and are publicly available. Furthermore, the DigiNotar case is selected because it is a European cyber incident and is considered the turning point in awareness on digital security in the Netherlands. The WannaCry attack on the NHS England is selected because it equally concerns a European case and is more recent than the DigiNotar case, which could offer different insights.

Cyber resilience in organisations Master Thesis Meret Keeris

4. Introduction to the topic In this section, an introduction to cyber resilience and its concepts is provided, where the identification of the relationship between cyber resilience and cyber security is central.

4.1 Definitions and bordering concepts To get a better idea of what cyber resilience entails, it is good to start with the concept of cyber security. The comprehensive and complex character of cyber security makes it difficult to establish one consistent definition (McSweeney, 2018). Although the development of a uniform definition of cyber security is outside the scope of this thesis, it is important to be aware of the different interpretations of cyber security.

This research adopts the definition of the International Telecommunication Union (ITU), which is the specialised agency for information and communication technologies (ICTs) of the United Nations. The ITU develops technical standards, guidelines and international networks to improve the benefits of ICT systems and interconnectedness worldwide. ITU defines cyber security as follows: “Cyber security is the collection of tools, policies, security concepts, security safeguards, guidelines, risk management approaches, actions, training, best practices, assurance and technologies that can be used to protect the cyber environment and organisation and user’s assets” (ITU, 2018) (p.2). This definition has been adopted by different actors – academics, the private sector, governments - and for multiple purposes – in research, business strategies and governmental policies.

The definition is comprehensive and resonates several relevant aspects for this research. First, the definition encompasses a variety of cyber security tools, measures and approaches. It shows the various ways in which cyber security can be implemented or increased, and that this is beyond technologies solely. Furthermore, it explains that the aim of cyber security is the protection of both the cyber environment as organisation and its user’s assets. The aim to protect the cyber environment indicates an interest to combine efforts and responsibilities among actors that participate in cyberspace. Since cyberspace knows no borders, and therefore mutually affects participants, it demands a collective approach. Another relevant aspect that the definition touches upon is the organisation and its user’s assets. The ITU describes that the “organisation and user’s assets include connected computing devices, personnel, infrastructure, applications, services, telecommunications systems, and the totality of transmitted and/or stored information in the cyber environment” (ITU, 2018) (p.2). Assets are resources, strengths and other things of value that an organisation owns, benefits from, or uses to generate value or income (CFI,

Cyber resilience in organisations Master Thesis Meret Keeris

2018). Assets can be non-technological or non-digital such as knowledge, buildings, inventory and trademarks (CFI, 2018), but increasingly either become digital or are linked to or protected by digital means and technologies (National Research Council, 2014). Cyber security is therefore ever more important for the protection of assets against threats that can breach the confidentiality, integrity, availability, authenticity and non-repudiation of systems and assets (ITU, 2018) (p.6).

Cyber security is often used interchangeable with information security or data security. Perspectives differ however which concept is the more comprehensive one (Von Solms, 2013) (Schatz, 2017). Information security is the security of information or data against unauthorised access, use, disruption, disruption and modification. Information is generally considered secure when the confidentiality, integrity and availability are preserved (Von Solms, 2013). Information – or data – is not necessarily stored in cyberspace but can also be non-electronic. The protection of information – both electronic and non-electronic – is mostly facilitated by IT infrastructures, which are the infrastructures that process, store and communicate the information. Although the majority of cyber incidents concern breaches to the security of data (EESC, 2018), to say cyber security is synonymous to information security misrepresents their scopes (Schatz, 2017). Cyber security is concerned with the protection of systems and networks against attacks, threats and failures. This also includes the protection of information, but in line with the International Organisation for Standardisation (ISO/IEC27032:2012) (Von Solms, 2013), this research considers cyber security to be more comprehensive than information security. Hence, next to information security, cyber security includes computer security, ICT security, network security and infrastructure protection (ISO, 2012).

4.2 Cyber security versus cyber resilience Cyberspace is difficult to secure due to the high pace of technological developments and sophistication of cyber-attacks (Linkov, 2013). Organisations are not only vulnerable to threats with malicious intent, but also to technical failures and unintended acts of harm. Moreover, the architecture of cyberspace makes discovering the cause of harm troublesome. Aiming for full security is therefore not a realistic objective. The aim to prevent threats needs to be balanced with the aim to ensure flexible and rebounding systems, also known as resilience (De Bruijn, 2017). Safeguarding operational continuity is not only a more realistic endeavour than full security, for most organisations it is also more important. Process interruption can have serious consequences to organisational assets such as business operations, revenues, reputation and quality of services (Linkov, 2013) (Van der Meulen, 2015).

Cyber resilience in organisations Master Thesis Meret Keeris

The definition of resilience is twofold and is defined as (1) the ability to prepare for and adapt to changing circumstances, and (2) the ability to bounce back or recover after a breach/crisis (Björck, 2015). Cyber resilience is generally defined as the ability to prepare for, withstand, rapidly recover and learn from deliberate attacks or accidental events in cyberspace (Linkov, 2013) (KPMG, 2017). Although a large part of the literature does not explicitly differentiate between cyber security and cyber resilience, there are some important differences between cyber security measures and cyber resilience measures (Rajamäki, 2017). Some practical examples of cyber security measures are firewalls, anti-virus software, controlled application access, passwords, updates and patches, and the security of portable devices (Ponemon Institute, 2018). Examples of cyber resilience measures are surveillance, back-up systems, segmentation, security breach detection tools, response teams and incident simulations (Ponemon Institute, 2018). While security aims at prevention, resilience focuses on continuity and recovery. Cyber security measures and controls alone therefore do not provide the same stability and capacity to cope with cyber threats as cyber resilience practices can do (Bagheri, 2017). In this research, cyber resilience includes the prevention of cyber incidents and therefore considers cyber security to be a component of cyber resilience. Based on above knowledge, it defines cyber resilience as the ability to prevent, prepare, detect, respond and recover from cyber incidents of both intentional as unintentional nature.

4.3 The need for cyber resilience The attention to cyber resilience has increased significantly over the past decade. Technological developments create new possibilities, but also increase vulnerabilities and dependencies of organisations, infrastructures and societies. Each rely on well-functioning systems, with the least amount of interruptions (Clemente, 2013). In particular for sectors such as water, health,

Cyber resilience in organisations Master Thesis Meret Keeris transport, banks and energy the stakes are high, since these sectors bear responsibility to provide essential elements of the daily lives of many people (National Research Council, 2014). A considerable part of the literature is therefore focused on essential infrastructures. However, also non-essential organisations need to be cyber resilient, since interconnected organisations mutually affect each other’s security position (Clemente, 2013). The separation between critical and ‘normal’ organisations therefore becomes meaningless by the interconnectivity that cyberspace creates. The complexity makes it difficult to identify and distinct the industries and assets that need (more) protection, and sometimes leads to a denial of responsibilities. However, refusing to participate in cyberspace is not possible, as even the most trivial tasks are now in some way connected to or dependent on cyberspace (Keys, 2016). The risk to damage and disruption will likely increase as vulnerabilities increase, attacks continue to take place and the connectivity will grow. Although there might be different interests and incentives involved in improved cyber resilience, it is important that all involved actors contribute to this objective (Clemente, 2013). This research therefore aims to encompass all kinds of organisation.

4.4 Cyber resilience of European organisations An increasing number of organisations understand the need to be as secure and resilient as possible (Linkov, 2013) (Van der Meulen, 2015). Organisations have significantly improved their ability to prevent and detect cyber threats, and investments start to pay off and lead to better performance during cyber incidents (Ponemon Institute, 2018) (Accenture Security, 2018).

However, many organisations are still seriously challenged by the cyber threat. The European Economic and Social Committee (EESC) studied the state of cyber resilience of European organisations. They examined the level of engagement and awareness, and the threats and challenges they face. The study aimed to find the bottlenecks organisations face in implementing cyber resilience practices, approaching the issue from an internal company and an external policy perspective. The study presents relevant observations on the current cyber resilience situation in European Union (EU).

In 2017, 80% of European companies experienced at least one cyber incident of different extents. Moreover, despite improvements nearly 70% of companies have no or only basic understanding of their cyber risk exposure and only 30% has ever estimated the potential financial loss by a cyber-attack (EESC, 2018).

Cyber resilience in organisations Master Thesis Meret Keeris

One of the main issues is the lack of awareness of the gravity of the issue. Organisations tend to underestimate the issue and therefore abstain from taking action. Organisations indicate that cyber incidents remain complex and difficult to comprehend. These gaps in insight can be tackled by actionable threat intelligence and increased security monitoring. The current reactive approach leads to late detection and response, thereby unduly increasing the impact of the incident. While insufficient cyber resilience and awareness hampers business improvement, it also has adverse effects on the European economy as a whole (EESC, 2018).

The three most emerging cyber threats to European organisations are (1) malware and phishing, (2) Distributed Denial of Services (DDoS), and (3) data breaches. The three cyber threats operate in in different ways, but are all able to cause considerable physical, operational and reputational damage. Malware and phishing incidents are generally not the most costly type in terms of damage compared to other types of attacks, but their frequent occurrences make the total costs the highest. DDoS are mostly aimed at large organisations or economics, but make small business vulnerable too. Finally, although the number of data breaches has decreased, the sizes of breaches are increasing (EESC, 2018).

Data breaches are mostly faced by the finance, retail, healthcare, information technology and business services sectors. The telecommunication, finance and energy sector have significantly improved their cyber security position by developing advanced defence mechanisms. Inertia in the healthcare sector, contrarily, makes it a vulnerable sector with little awareness and preparation to cyber threats. Another group that is in a vulnerable position are European small and medium-sized enterprises (SMEs). SMEs account for 99,8% of European enterprises but demonstrate below average maturity levels of security. They generally lack the awareness, preparation and budget to implement security measures. The costs of cyber incidents are also disproportionately high for SMEs (EESC, 2018).

Applicable to the larger part of organisations, the EESC (2018) identifies two important hurdles for improving cyber security: the uneven development of digitalisation and cyber investments and the growing lack of ICT personnel and cyber experts. Also the Ponemon Institute (2018) states that in particular a lack of skilled personnel, response plans and investments in new cyber security technologies, such as machine learning and artificial intelligence, throws up barriers for further improvement.

A lack of insight into applications and data assets makes effective cyber resilience efforts hard to realise. Organisations have difficulties to understand and estimate the value of assets and the

Cyber resilience in organisations Master Thesis Meret Keeris required measures. Furthermore, the budget for cyber security decreases while there is actually a need for more financial resources to stand up against new threats (Ponemon Institute, 2018) (Accenture Security, 2018). Moreover, budgets and expenditures are further pressured by new regulations such as by the General Data Protection Regulation that require compliance (KPMG, 2017). Finally, a lack of cooperation and trust between organisations and sectors hinders effective and cost-efficient measures and strategies (EESC, 2018).

From a public policy perspective, the EESC study explains that gaps exist in the security positions of EU Member States. There are clear differences in awareness, knowledge and capacity and consequently in strategies, policies and competences. Divergent levels of security and protection weakens the overall cyber environment of Europe. A disintegrated regulatory situation does not stimulate cross-border cooperation, intelligence sharing and trust building between organisations. New EU regulations and directives should improve effective transnational cooperation by increasing the homogeneity of security and privacy regulations (EESC, 2018).

Until now, EU regulations and directives addressed issues on data protection and privacy and the security of information networks and essential infrastructures. Two main examples of regulations are the Directive on Security of Network and Information Systems (NIS Directive) and the General Data Protection Regulation (GDPR), of which the former is focused on incident notification and response and the latter on data protection and privacy (EESC, 2018). The NIS Directive provides the objectives on the cyber resilience of vital infrastructures and digital service providers in the EU. However, since the Directive leaves room for Member States to determine which sectors they consider vital, the EESC warns that this flexibility might cause inconsistencies within the EU and makes it harder to establish a strong common response (EESC, 2018). Furthermore, the GDPR has been established to improve data protection. The EESC concludes, however, that many organisations are still unfamiliar with the regulation, its implementation and implications. A third of the organisations do not have the technologies available to implement the GDPR, and even more organisations state to lack the knowledge to determine which data should be protected. As a result, more than 85% of the organisations fears non-compliance and the consequent negative impact this will have on their business. Thus, for organisations to start bearing fruit effectively from the regulations, more time is needed for organisations to internalise the provisions (EESC, 2018).

Finally, the EESC concludes that there is a lack of trust in the digital world; between Member States, between public and private organisations and mutually between organisations. The lack 15

Cyber resilience in organisations Master Thesis Meret Keeris of trust between actors and organisations increases the perception of risks, which demotivates the adoption of new technologies and results in a reluctance to share information. So, although intelligence sharing could benefit the resilience of all without undue costly measures, the exchange of information and experience remains minimal (EESC, 2018).

Cyber resilience in organisations Master Thesis Meret Keeris

5. Literature analysis The examination of the literature points out the issues that play an important role in cyber resilience in organisations. This knowledge has been structured and categorised into four main factors. This generated the following structure: (1) understanding the organisation, (2) effective management, (3) the chain and (4) regulation. Each factor will be discussed below.

5.1 Factor 1: Understanding the organisation To draft an effective cyber resilience policy it is essential to understand the organisation. This means a good understanding of the organisation’s identity, culture, awareness and assets. The understanding of the organisation and the identification of key assets help to direct the policy.

5.1.1 Organisational identity: culture, awareness, behaviour Organisational science explains that the organisational identity answers the question “Who are we as an organisation?” (Yeatman, 2015). Organisational identities are generally constructed of inner and outer layers. The inner layers are the core attributes, values and notions and tells how things are done and can be considered the organisation’s culture. The outer layers concern the more concrete features and tell what is done, such as the processes, activities and the sector in which the organisation is active (Lin, 2004). The identity helps to determine what is essential to the organisation: the key assets. These assets need to be identified first to subsequently assess the risks the organisation faces (ENISA, 2017). Altogether, this knowledge forms the first part that is required for drafting an effective cyber resilience policy.

Cyber resilience is not merely an IT issue but is installed throughout the organisation and facilitates a processes, methods and structures. It involves departments and levels organisation- wide and requires cyber resilience to be an integrated part of the organisation (Conklin, 2017) (The Stationary Office, 2015). Integrating cyber resilience starts with raising awareness on the severity of the issue. This awareness can be raised by transparency and training on cyber threats to the organisation (KPMG, 2017). To further integrate cyber resilience and to actually change behaviour, organisations need to understand the culture (Knapp, 2009) (ENISA, 2017). The culture constitutes the beliefs, values and perspectives that influence people’s interests,

Cyber resilience in organisations Master Thesis Meret Keeris approaches and behaviour. The organisational culture can largely explain what the organisation values most, how it reacts to certain situations and how decisions are made (Schein, 1997). Practitioners argue for a risk culture and sustainable risk-return thinking to be more vigilant and resilient. A risk culture can ensure the necessary behaviour for a cyber resilient organisation. Understanding the current culture helps to identify the steps needed for full integration of the core values necessary for a cyber resilient culture (Deloitte, 2017) (ENISA, 2017).

The importance of organisational culture, employee engagement and behaviour is confirmed by literature on the human factor in cyber resilience. Research states that employees are generally a high-risk group and that more than half of the businesses state employees are the weakest link in cyber security (Boyce, 2011). Most of the cyber incidents can be attributed to human error, both intentionally as unintentionally. Internal risk analyses can help to find out the cause and whether, for example, an employee has been compromised unknowingly. Especially SMEs fear the inadequate use of IT by employees, more than larger enterprises do. Large organisations often have stricter policies in place and provide more training, while smaller organisations are more flexible towards employees (PWC, 2018) (Kaspersky, 2017).

Research also shows that human performance is essential for the optimal and effective functioning of technologies. Based on the psychology of security, West (2008) argues that people commonly think they are less vulnerable and less likely to be harmed than others. Moreover, since the cyber threat is not a visual threat it is generally associated with abstract consequences. People generally do not understand the security consequences of inadequate security and therefore lack motivation to adequately implement cyber security measures (Boyce, 2011). Clear task performance requirements, transparency, and demonstrated commitment to a risk-based culture are important ingredients to stimulate awareness and behaviour needed to enhance cyber resilience (Boyce, 2011) (PWC, 2015).

To get a full picture of the organisational identity, the outer layers i.e. the processes, activities and the sector have to be understood. Determining this part of the organisational identity can be rather complex, because the transformation of organisations towards digital entities blurs their identity and makes activities harder to separate (Abraham, 2016). This becomes clear when an organisation provides multiple services or is active in different sectors. Take for example Eurotransplant, a donor organ allocation service that cooperates with hospitals, laboratories and transplant centres in eight European countries. It facilitates donor organ allocations using databases of waiting lists, donor data and medical records. This means their

Cyber resilience in organisations Master Thesis Meret Keeris main aim or service of donor organ allocation is a health service, but is almost entirely enabled by databases and IT infrastructures (Eurotransplant, n.d.). With such business models categorising the organisation and its activities can be hard. Figuring this part of the organisational identity is essential to determine priorities in cyber resilience.

5.1.2 Asset identification A first step to setting priorities is the identification of assets. Assets relevant for cyber resilience can be both digital assets as well as assets that are facilitated by digital means. Strategic organisational studies show that organisational assets are the tangible and intangible building blocks that allow for organisations to achieve strategic objectives (Frigo, 2014). Although such studies are often focused on objectives such as increased returns, market shares and competitive advantage, it explains that it is crucial for organisations to identify the vital assets to determine a strategy. To identify the critical assets, all departments should be involved, because the department itself can best determine the value and the maximum acceptable time of disruption or takedown of an asset (The Stationary Office, 2015). What seems challenging for organisations in mapping the assets, is to correctly assess the value of assets (Evans, 2014). In many organisations, in particular the value of information assets is underestimated. As a result, information asset management is neglected or inadequate. Well-managed and employed information assets can considerably add to the organisation’s value, for example in the form of intellectual property (The National Archives, 2017). Contrarily, poorly managed or safeguarded information can give rise to risks, because the information becomes vulnerable (Evans, 2014). Improved storage and protection structures increase the value of the assets while mitigating the vulnerability (The National Archives, 2017).

Furthermore, asset prioritisation could be done on the basis of criticality. The key assets of an organisation, also called the ‘crown jewels’, are crucial to the existence of the organisation and therefore bear a high level of criticality (Anuar, 2011). This can for example be a process, service or technology, or assets such as personal data or intellectual property (Capgemini Consulting, 2016). Since the key assets are essential for the organisation’s existence, they should be protected at high costs but still remain functional and utilisable (Clemente, 2013).

This section explained why and how a good understanding of the organisation is essential to set priorities and direct a policy. More knowledge and insight into the organisation, enables more tailored strategies. The next step is to find out the essential factor needed to effectively implement cyber resilience: effective management. This is discussed in the next section.

Cyber resilience in organisations Master Thesis Meret Keeris

5.2 Factor 2: Effective management The previous section discussed the organisational identity and assets. This section will focus on what is essential to translate this knowledge into effective management.

5.2.1 Risk assessments Once an organisation has identified its assets, the risks to these assets should be assessed to determine the appropriate measures to mitigate the risks (Park, 2011). For most organisations managing risks related to cyber are more problematic than managing other more conventional kinds of risks. They do not have the same knowledge, expertise and tools at their disposal as with ‘traditional’ risks (World Economic Forum, 2017).

A risk concerns the potential loss, damage or corruption of the asset in case a threat exploits an asset’s vulnerability. Vulnerabilities are the weaknesses of assets that make threats possible and threats are the potential happening of harm to assets (Park, 2011). Cyber risks can be of malicious nature, but as explained before also technical failures and inadequate employee behaviour give rise to risks. Moreover, risks to assets can also arise from organisational changes. For example, in case the asset is information and an organisation plans to implement an organisational change, such as a relocation of the office, there is a risk that information could go lost. Once the organisation is aware of this ‘natural’ risk, it can implement measures to mitigate the risk, which in this case could be external data storage.

Research indicates that the greater part of successful cyber-attacks exploits known vulnerabilities (Agrafiotis, 2018). These attacks are possible because organisations’ misunderstanding of risks causes slow provision of cyber security measures (Agrafiotis, 2018). Both academics and practitioners point out the benefit of conducting risk assessments. Risk assessments point out where the vulnerabilities are and how likely it is that those vulnerabilities will be exploited by threats. It is an effective way for organisations to obtain insight into the current state of cyber resilience, to map the threat landscape, and to identify the areas that need improvement (Anuar, 2011) (Deloitte, 2017) (Ponemon Institute, 2018). Not all areas can be improved, and planning for all worst-case scenarios in not feasible. Hence a trade-off has to be made between the relevance (i.e. how frequently or likely is an event to occur) and the severity of the scenario (Adams et al., 2015). The organisation can act by implementing risk mitigation measures (McGill, 2007). Moreover, it allows for a budget-proportionate spending on cyber risks (Agrafiotis, 2018) (Ponemon Institute, 2018) (Van der Meulen, 2015).

Cyber resilience in organisations Master Thesis Meret Keeris

The Risk Index Model for security incident prioritisation of Anuar et al. (2011) offers a comprehensive tool for assessing risks. It helps to identify which incidents are crucial and which are trivial. The model is comprehensive, but the two main decision factors form the basis of many risk assessment tools. These are: (1) the consequence of the event based on the impact on the asset, and (2) the likelihood of an event, based on the likelihood of threat and vulnerability. The first decision factor is indicated by the following indicators: criticality, maintainability, replaceability, and dependability and control. By means of these rating issues the impact of a cyber incident on the asset can be assessed. For example, criticality assesses the importance of the asset and is generally based on the confidentiality, availability and the integrity of the asset. Generally, there is a trade-off between the criticality and the replaceability of an asset. The easier to replace the asset, the lower the criticality. To accurately assess the value of an asset and the impact of a potential threat, the level of each of indicators should be determined. The second decision factor - the likelihood of an incident - uses the following indicators: severity, exploitability, sensitivity, similarity and frequency. Generally, the higher these indicators of a potential incident score, the higher the risk. Again, for the most accurate picture each indicator should be assessed (Anuar, 2011).

5.2.2 Financial resources For many organisations a lack of financial resources hampers further enhancement of their cyber resilience strategy. Budgetary limitations require the organisation to choose set priorities on which to spend their scarce resources (Ponemon Institute, 2018). Insufficient budgets also withhold organisations from investing in tools and technologies such as artificial intelligence (AI) and machine learning that can effectively help both to make better decisions as to increase resilience (Ponemon Institute, 2018). Especially SMEs struggle to implement resilience measures due to a lack of financial resources. The EU aims to support these organisations in investments by offering of cyber security funds and subsidies, of which most SMEs are not aware (EESC, 2018).

In general, organisations understand the continuity of primary processes has priority, but are often still hesitant to invest in security measures. This reluctance is often explained by the difficulty to quantify the business value of improved security. This makes it hard to assess the incentive of improved protection by measures. Moreover, negative and positive externalities that come with cyberspace being a public good, withheld organisations to invest. When there is lack of a clear business case or the benefits do not outweigh the costs, organisations are inclined to invest less (Van der Meulen, 2015) (NCTV, 2018).

Cyber resilience in organisations Master Thesis Meret Keeris

Although asset identification and risk assessment are useful to define the focus of measures and efforts, in practice organisations hardly ever possess all the necessary information to make optimal decisions (Van der Meulen, 2015). To determine expenditures, the increase and amount of expenditures should be in proportion to the potential impact of a threat and to the value of protected assets (Gordon, 2003). Section 5.2.3 Deciding on measures, discusses potential tools for achieving this.

Most of the time, the eventual costs of a cyber breach are much higher than the initial investment that could have mitigated or prevented it (Capgemini Consulting, 2016). To better interpret the potential impact of an incident, an organisation can assess the potential damage indicated by the Key Performance Indicators (KPI’s). The KPI’s are the measurable values to evaluate the effectiveness and success of an organisation in achieving its key business objectives (Capgemini Consulting, 2016). Furthermore, Gordon et al. (2003) argue that security measures are often reactive rather than proactive, meaning that they are implemented after their necessity has been pointed out by an actual breach. They argue that penetration tests and mock breaches have the same effect as actual breaches and are therefore important drivers for organisations to invest in security measures (Gordon, 2003).

Finally, sometimes organisation’s security investments are motivated solely by compliance requirements. To meet with regulatory demands organisations feel urged to invest and do so mostly in security infrastructure, instead of using a risk-based approach and invest where it is needed (PWC, 2018). The matter of regulation and compliance will be further discussed in the section 5.4.

5.2.3 Deciding on measures Once the risks and resources are mapped, the next step is to determine what will be done with the results in terms of measures, which can differ per organisation. An important concept explaining this difference is risk appetite or risk acceptance. Risk appetite is the level of risk that an organisation is prepared to take in order to realise, for example, strategic objectives. This is confirmed by the concept of economics of information security, which states that there is often a trade-off between the efficiency and the security of systems (Gordon, 2003) Organisations can therefore face friction between increased security and the flow of business operations (Duane, 2018). When conflicts between resilience measures and business functions arise, a cost-benefit analysis can be helpful. A cost-benefit analysis will point out the issues that bear acceptable risk levels in proportion to the costs of a measure and the value of the asset. A cost-benefit analysis is based on three considerations: (1) the probability of an incident or

Cyber resilience in organisations Master Thesis Meret Keeris successful attack in absence of a security measure, (2) the consequences of an incident or successful attack and (3) the degree to which the security measure would reduce risks by decreasing the probability or consequence of a successful attack (Adams et al., 2015). An organisation will have to determine whether the risk that arises when a security measure will be neglected is acceptable or not (ENISA, 2017). In case a risk to a non-essential asset is known, but the potential damage does not outweigh the costs of the mitigation measure, an organisation might choose to take the risk for granted (The National Archives, 2017). Since strategic objectives differ per organisation, risk appetite will influence how organisations interpret and implement the results of a risk analysis (Van der Meulen, 2015).

Although risk assessments and cost-benefit analyses offer useful tools, many organisations do not use their potential. Organisations generally only conduct risk analysis primitively, due to a lack of integration between measures (Capgemini Consulting, 2016). Coordination and feedback between departments increases the utility and accuracy of a risk assessment (The Stationary Office, 2015) (Capgemini Consulting, 2016).

5.2.4 Roles and Responsibilities Since cyber resilience involves the entire organisation, it is key that the roles and responsibilities between departments and levels are transparent and aligned (Conklin, 2017). High levels of coordination help personnel to take responsibility and thereby prevents unexecuted tasks that could lead to risks. Moreover, it helps to detect anomalies in the organisation’s processes and increases the effectiveness and efficiency of incident responses (de Crespigny, 2012).

The difficulty with cyber resilience is that its comprehensive nature demands for a comprehensive approach. Many organisations still tend to approach cyber resilience as a technological issue instead of a business-wide issue. Therefore, technologists within organisations have been taking the lead but depend largely on existing technology risk management frameworks. These technologists, however, lack the strategic business perspective, and the business managers lack the technical skills (PWC, 2015). An organisation therefore needs to merge these competences. One option is to combine positions and establish the cyber expertise in the form of joint cooperation. The EESC argues that this delivers better levels of preparedness, especially in SMEs (EESC, 2018). The bottleneck with this approach is, however, that it requires high levels of coordination and communication to be effective (KPMG, 2017). The Ponemon Institute argues that cyber resilience is most effective when the responsibility is in the hands of one appointed person: a high-level security leader with

Cyber resilience in organisations Master Thesis Meret Keeris organisation-wide responsibility (Ponemon Institute, 2018). Practitioners also emphasises the need for an appointed cyber security manager, rather than dispersed efforts or tasks added to an existing function (Deloitte, 2017) (KPMG, 2017). Either way of fulfilling the function, the combination of technical and strategic business knowledge is an important ingredient for good cyber incident management.

Strategic assignment of tasks is best be done at an early stage. Organisations need to be far- sighted to foresee the need of positions. Especially with the lack of skilled personnel, organisations are required to pay extra attention to roles and responsibilities. Possibly, large enterprises could choose to attract professionals from a broad or to relocate operations. Smaller enterprises could be supported by sector cooperation and public-private partnerships in tackling the need by sharing practices and intelligence (EESC, 2018).

5.2.5 Response The final issue in effective management is response. Research shows that organisations often have obsolete or ‘off-the-shelf’ incident response plans in place. Each organisation resides in a different and change-prone cyber environment, determined by a unique combination of assets, threats and risk appetite that is tailored to the organisation and its objectives. Such plans are therefore not able to answer to today’s cyber threat realities and are usually ineffective (KPMG, 2017). Another common issue is that response plans are drafted, but not tested. Many response plans have been constructed in the past, but have neither been revised nor validated in a test case. Consequently, during an actual cyber incident information on tools, people and procedures proves incorrect or outdated and leads to an ineffective incident response with undue impact (KPMG, 2017). Organisations should therefore have clear, up-to-date and tested response plan in place.

During a response it is essential that as much information on the threat is gathered. This threat intelligence will help to determine the necessary response. Furthermore, an effective response is largely dependent on high levels of coordination. Clear coordination of tasks and between departments will help to avoid undue delays. For optimal incident handling, the detection and response processes need frequent practice, revision and adjustment (Carnegie Mellon, 2015). To ensure effective response to cyber incidents, an organisation could decide to employ a Computer Security Incident Response Team (CSIRT) (GvIB, 2006). The term CSIRT is mainly used in Europe instead of the protected term CERT – Computer Emergency Response Team, which is used in the United States is registered by the CERT Coordination Center (CERT /CC) (ENISA, 2006). A CSIRT generally has three tasks: (1) to detect cyber threats in time, (2) to

Cyber resilience in organisations Master Thesis Meret Keeris investigate the magnitude of the threat, and (3) to ensure that the threat reduces or disappears (Ministery of Defense). The number of and need for CSIRTs has increased as a response to past cyber incidents. Cyber incidents require an immediate and coordinated response by trained cyber security experts. CSIRT operate alongside conventional IT operators and focus mostly on operating systems, network protocols and vulnerability management tools. Important in implementing a CSIRT in the organisations, is a clear coordination of tasks with the IT department. This will prevent undue unclear situations and response delays in case of a cyber incident (GvIB, 2006).

There are different kinds of CSIRT’s. An organisation CSIRT works for one client and ensures that the security of the internal organisation is in good order. These functions can also be established throughout the organisation without naming it a CSIRT, but should be coordinated well. Furthermore, sectors can join to set up a joint CSIRT. This stimulates sharing threat and protection intelligence with other organisations in the sector, and improves sector resilience. These CSIRTs are also a good solution for SMEs that are not able to establish a CSIRT individually (GvIB, 2006). Finally, national CSIRTs coordinate the CSIRT activities for an entire country. In the Netherlands, for example, this is the National Cyber Security Center. They act as advisers and stimulate information security in the Netherlands and facilitate CSIRT’s to protect critical infrastructures (NCSC). The EU’s NIS Directive provides for the requirements for CSIRTs in Member States. Via ENISA – the European Union Agency for Network and Information Security - these efforts are combined in a pan-European collaboration of CSIRT Networks to build and advance cyber resilience capabilities (ENISA). The CSIRT Community represents business, sectors and Member States and stimulates cooperation, information exchange and trust building. If a European organisation wishes to improve its cyber resilience by cooperating in a CSIRT, it can consult the European CSIRT Inventory of ENISA. This inventory comprises all actual European CSIRTs and can be filtered for the specific purpose of the organisation that wishes to engage in the network (ENISA, n.d.). Effective coordination between CSIRTs, both public and private, contributes to a more powerful response to cyber incidents (EESC, 2018). The next chapter will further elaborate on the advantages of cooperation.

Cyber resilience in organisations Master Thesis Meret Keeris

5.3 Factor 3: The cyber chain Once an organisation obtained a good understanding of its internal structures and management, the aspects beyond the organisation’s perimeters should be investigated to gain insight into the chain of connected organisations in which it is situated.

5.3.1 Interdependencies The interconnectivity and interdependencies that cyberspace creates make organisational perimeters become fluid and blurry. Actors of different natures - public, private, non-profit – increasingly exceed organisational borders to enhance effectiveness and efficiency of services through cooperation (The Stationary Office, 2015). Consequently, organisations’ borders merge with those of other stakeholders in the chain and establishes a structure of interconnected organisations (Clemente, 2013). Generally, organisations do not devote sufficient effort and attention to the extended chain of partners (Accenture Security, 2018). The increased mutual dependency of organisations within a chain also duplicates vulnerabilities, since most cyber incidents initiate with third parties, such as business partner, contractors and suppliers (Deloitte, 2017). It is therefore important that organisations also conduct external risk assessments. Interconnected organisations need to understand that the disruption of one organisation in the cyber chain can, and will most likely affects others as well. Improving the resilience of the chain demands for a collective approach and starts with chain-wide awareness on the importance of a cyber resilient chain and the need for joint effort to achieve this (Clemente, 2013) (Capgemini Consulting, 2016). Stakes and interests will intertwine and requires involved parties to take responsibility and contribute to this aim (NCTV, 2018) (World Economic Forum, 2017).

Generally, however, organisations are not aware of the consequences this growing digital connection brings along for their own cyber resilience requirements (Capgemini, 2017). The lack of insight into this new joint agenda is mostly due to the complexity of the interconnected landscape. It is hardly possible for organisations to overlook the entire chain and map a complete overview of connections, dependencies, shared risks and vulnerabilities (NCTV, 2018) (Clemente, 2013). Only a small number of companies and governments claim to have good overview on their chain. Most organisations solely focus on their own resilience. Reasons for not sharing intelligence are a lack of resources in terms of staff, few perceived benefits and high costs (Ponemon Institute, 2018).

Interdependencies hamper asset identification and risk management because there is less oversight and control. Also an accurate impact assessment of the joint impact of a disruption in 26

Cyber resilience in organisations Master Thesis Meret Keeris the chain is hardly feasible. Organisations could therefore benefit from shared asset and threat intelligence. Moreover, early notifications of third-party incidents will give an organisation a better chance to handle the incident efficiently and minimise the impact (Deloitte, 2017).

Furthermore, interdependencies also raise questions on responsibility and accountability since those issues neither end at the perimeters of the organisation anymore (Capgemini, 2013). An organisation should aim to ensure due diligence of all involved parties and could best work with contractually determined provisions, for example on incident communication (Deloitte, 2017). These issues are even more important when there are critical infrastructures involved with regard to stricter requirements and regulations than non-critical organisations (Van der Meulen, 2015). Chain and sector specific knowledge is therefore needed to understand the specific security requirements of related organisations policy (Knapp, 2009).

5.3.2 Cooperation The cyber resilience of one organisation largely depends on the cyber resilience of the other players in the chain. Through cooperation with the related stakeholders and partners an organisation can considerably improve its resilience (Ponemon Institute, 2018). By mobilizing and expressing common interests and shared chain responsibility the cooperation in the chain can be improved. Accordingly, the priorities and tasks of the chain in its entirety can be determined (Capgemini, 2017). An examination of the practices of organisation that are cyber resilient show that they are more likely to share information on data breaches (Ponemon Institute, 2018). Best practices include strong relationships with stakeholders and partners. The relationships facilitate information and initiative sharing as well as early notifications in case of an incident. There is a need for collaboration, exchange of intelligence and know-how and joint efforts (NCSC, 2018). Shared efforts will decrease weak links that lead to higher risks and hence improve the overall resilience of the cyber chain (The Kosciuszko Institute, 2017).

Dutch examples of partnerships The Netherlands offers several good examples of initiatives on information sharing and partnerships with the cyber chain. Two examples of Dutch initiatives are “FERM” of the Rotterdam Port (Rotterdam Port, n.d.) and “Cyssec” of Schiphol Group (Spot Schiphol, 2017). To raise awareness about cyber security and resilience in the port of Rotterdam, the FERM was created. Schiphol Group states that together with all companies that are connected to the airport, they form the 'Schiphol ecosystem'. Both partnerships are aimed to raise awareness, share knowledge and to increase economic opportunities while strengthening their cyber chain (Capgemini, 2017).

Cyber resilience in organisations Master Thesis Meret Keeris

Furthermore, also the National Cyber Security Council (NCSC) of the Netherlands argues that a lot of resilience can be won by an integrated approach between chain partners. To enable this, good relationships and trust are needed (Capgemini, 2017). In November 2018, the Dutch Cyber Security Council launched four new manuals that help organisations to start cyber security partnerships. The first three are aimed at the (1) sector, (2) region and (3) chain of organisations. The fourth manual is meant for incident response teams that already actively cooperate to further enhance this cooperation. The manuals are aimed to improve the cyber resilience of the Netherlands through cooperation and information sharing (NCSC, 2018).

The manuals are a response to the conclusion of the Dutch Cyber Security Council of July 2017, which stated that there was a severe lack of cyber security information in the business community. Until now the organisations that actively share information and cooperate for better resilience, are mostly governments and organisations that are part of a critical infrastructures. The Council concluded there was a need for shared cyber security information among non-vital infrastructures. Non-critical infrastructures often lack the connection to a structure within which information on threats, measures and perspectives can be shared. Their cyber resilience is therefore often insufficient and makes them an easy target for cyber-incidents.

Furthermore, vital infrastructures are to a large extent dependent on suppliers from the non- vital part of the business sector. These suppliers therefore also affect the cyber resilience of the entire chain. The realisation that collaboration within digital chains is necessary for overall resilience is currently growing, but still needs to be shaped to an important extent (NCSC, 2017).

5.3.3 Ecosystem In addition to the cyber resilience of the chain and the need for cooperation, literature also discusses the value of the digital ecosystem of organisations (Weill, 2015) (World Government Summit). The cyber ecosystem can be regarded as the extended environment of the chain, including a larger variety of participants and cooperation on a larger scale. The focus in a cyber resilience ecosystem is still on strengthening cyber resilience by information sharing and cooperation, but the ecosystem combines a broader variety of participants, such as governments, private companies, research institutions and individuals. The mixture of academia, industry and governments allows for consolidating knowledge, capacities, and resources to achieve an overall higher level of cyber resilience that benefit all parties (Weill, 2015) (World Government Summit).

Cyber resilience in organisations Master Thesis Meret Keeris

5.4 Factor 4: Regulation

Since cyberspace is a public space, there is need for a certain degree of guidance to ensure its secure existence and use. Mostly, this guidance takes the form of regulation and demand for compliance. If such regulations are understood and implemented adequately, regulation can improve resilience and make its benefits prevail over the burden of compliance.

5.4.1 Purpose of regulation By some, legislation is by some considered an undue regulatory burden for organisations. With regard to cyber resilience, they argue that the private sector should independently determine the necessary level and measures of security. The main argument against regulation is that it will hinder innovation (Lewis, 2009) or that it is a way to increase governmental control over information security issues (Knapp, 2009). Cyberspace challenges policy- and decision makers to create policies that effectively improve resilience without taking away economic growth, innovation or freedoms (Clemente, 2013). Moreover, organisations are driven by different motivations to meet with regulatory requirements. Some introduce new security measures and protocols on a frequent basis merely to ensure compliance and avoid fines. This, however, raises an undue focus on compliance rather than that the potential positive effects on resilience are understood (Panda Security, 2017) (Van der Meulen, 2015) (KPMG, 2017).

International strategic studies argue that regulation does not have to be an impediment to innovation when it does not strictly require or forbid certain technologies or activities (Lewis, 2009). The level of prescription of regulation determines the available space for innovation. Moreover, complete absence of regulation is neither expected to facilitate innovation nor does it protect the public against insecure products and services. To illustrate how regulation can contribute to both intentions without imposing burdensome obligations, the EU Certification Framework poses a good example. As part of the EU Cyber Security Package established in 2017, the European Commission proposed an EU-wide certification framework for information and communication technology (ICT) products, services and processes. The certification framework of rules, technical requirements, standards and procedures will guarantee the security of products and services. This should raise trust and confidence and contributes to smoother cross-border business operations because the certification will be valid in all Member States (European Commission, 2018). The certification will be voluntary, but it can provide organisations with a competitive advantage over other market players. Moreover, such certification framework will contribute to the development of ‘cybersecurity by design’. This means that products and services are initially build with a secure design, instead of secured by

Cyber resilience in organisations Master Thesis Meret Keeris security measures over time and helps to avoid future security vulnerabilities (European Commission, 2017).

Furthermore, digital networks build the backbone of society and economy and therefore require some form of governance. To ensure an adequate level of cyber resilience the public sector, the private sector and individuals need to contribute. Since cyberspace is a public good, governments play an important role in facilitating the handles that support organisations in building cyber resilience. Governments aim to increase cyber resilience by providing tools and standards that will offer organisations guidance to achieve this goal (Adams et al., 2015). Besides guiding frameworks that help to build resilience, also legal and regulatory provisions where organisations can lawfully function have been established.

Regulations and policies can, however, not meet the specific needs of all. The number of different actors and interests is simply too large to be able to provide specific provisions and requirements for each case (Clemente, 2013). Decisionmakers cannot include all specific circumstances to draft tailor-made regulations or policies addressing organisation-specific issues. Finding the optimal solution that effectively addresses the issues and achieves only the benefits of increased security measures is hardly possible. Therefore regulation should rather be considered a means to a desired end. Generally, it is significant that regulation is focused on the purpose of ensuring safe and reliable networks and services to the public. Moreover, when regulation is set out in terms of goals and outcomes, it can provide useful guidance for organisations, while describing responsibilities and stimulating processes for compliance (Lewis, 2009). By implementing the available standards and regulation organisations should be able to improve their cyber resilience. The periodical assessment of compliance with policies, industry standards and regulations should ensure a minimum level of protection and provide a basis for further enhancement (Lewis, 2009) (Deloitte, 2017). The subsequent sections focus on the latter, regulations.

5.4.2 Regulation on the protection of critical infrastructures Current regulations related to cyber resilience focus in particular on data protection and the protection of critical infrastructures. In the EU, these matters are regulated primarily by the General Data Protection Regulation and the Directive on Security of Network and Information Systems (NIS Directive) respectively, which both fully came into effect in 2018. The regulations will be discussed to learn about their implications on organisations with regard to cyber resilience. The GDPR is discussed more elaborately than the NIS Directive, because it is

Cyber resilience in organisations Master Thesis Meret Keeris applicable to a much broader range of organisations and as a Regulation it is binding in its entirety. The NIS Directive will be discussed first.

The NIS Directive The NIS Directive is part of the Cyber Security Package with which the EU aims to strengthen the cyber resilience of the Union. The NIS Directive provides the guidelines and regulations on cyber resilience for critical infrastructures and digital service providers. The interconnectivity of European critical infrastructures has strengthened the digital single market, but also considerably increases the possibility of a cross-border European crisis (Pursiainen, 2009). Organisations classified as critical infrastructures are generally more vulnerable to cyber threats due to the value of their services and operations. Sustaining their operability and availability of services is of high importance for society and the economy (Adams et al., 2015) (HSD, 2015). Since many of the critical infrastructures and digital service providers are in private hands, governments and private sectors need to cooperate to reduce the risk of disruption of services (Pursiainen, 2009).

The NIS Directive requires Member States to ensure a certain set of cyber resilience capabilities in case the security of the networks and services of vital infrastructures and digital service providers are disturbed (European Commission, 2018). The European Commission defines a critical infrastructure as “an asset or system which is essential for the maintenance of vital societal functions. The damage to it, its destruction or disruption […] may have a significant negative impact for the security of the EU and the well-being of its citizens” (European Commission, Last update: 2018). Generally this includes sectors like energy, banking, drinking water, public safety and health (European Commission, 2016). However, since it concerns a Directive, it only sets out the objectives it aims for. It does not provide for specific technologies or products for the protection of services, but measures should be appropriate, minimise the incident impact, ensure service continuity and compliance (IT Governance, 2018). EU Member States are required to fully and correctly incorporate the Directive’s provisions into national law, but can do so in their own way. Moreover, it can differ per MS which organisations within those sectors are considered ‘essential’ or ‘critical’, as each Member State is required to provide its own list of critical infrastructures within their country (European Commission, 2018). Due to the considerable degree of flexibility for MS in the implementation of the NIS Directive, it is difficult to draw concrete conclusions on the implications of the regulation for organisations. An organisation should therefore be well-aware of their criticality status in the relevant MS,

Cyber resilience in organisations Master Thesis Meret Keeris since that is what imposes a certain set of responsibilities and requirements with regard to their cyber resilience (Adams et al., 2015).

5.4.3 Regulation on data protection Besides the protection and resilience of critical infrastructures, data protection is a major issue covered by regulations. This section focusses on the GDPR, which entered into force in May 2018. The GDPR is the uniform framework of the EU on data protection regulation (European Commission, 2018).

The large amounts of available data combined with new technologies allow organisations to gather and share data with minimum effort. With data as the currency of a growing digital economic market personal data developed into a highly valued asset (KPMG, 2017). The European Union’s Data Protection Directive of 1995 did no longer cover these developments and required revision. The new GDPR should therefore provide the necessary protection in the new cyber landscape (European Commission, n.d.).

The GDPR The GDPR preoccupies European organisations to take measures for increased cyber security. The fines for lacking compliance to the GDPR are high – up to 4% of global revenues for the preceding fiscal year - and therefore constitute a strong argument for organisations to be compliant (ENISA, 2017) (Van der Meulen, 2015). The Regulation significantly increases the standards for data protection and privacy and is considered the most important change in data privacy regulation in twenty years (European Commission, n.d.). The regulation is equally applicable to all 28 Member States, including all government organisations, private businesses, and other organisation that store and process personal data. The regulation is also applicable to organisations that are located outside of the European Union but process the personal data of EU citizens. Set out in ten key issues, the GDPR roughly entails the following (EP & CoEU, 2016) (IT Governance, 2018).

Data protection. (1) The main principles are accountability and transparency and means compliance should demonstrate: a governance structure of roles and responsibilities, a detailed documentation of data processing activities, data protection policies and procedures, data protection impact assessments, appropriate measures to secure personal data, provision of staff awareness training and in some certain cases, the appointment of a Data Protection Officer. (2) The GDPR provides for the mandatory appointment of a Data Protection Officer (DPO) in case the processing is a) conducted by a public authority or body, b) the data processing involves

Cyber resilience in organisations Master Thesis Meret Keeris high risk and large-scale processing, or c) when it concerns special categories of data on a large scale. The tasks of the DPO include informing and advising the organisation on its obligations, monitoring and imposing compliance, and cooperating with the data protection authorities and acting as a point of contact. The Data Protection Officer should be in the position to perform its tasks and duties in an independent manner (EP & CoEU, 2016) (IT Governance, 2018).

Data processing and storage. (3) The GDPR sets out six data processing principles: data processing should be lawful, fair and transparent and should ensure appropriate security. Personal data should only be collected for specific legitimate purposes, relevant and limited to what is necessary, and kept up to date. Also, the data storage period should only be as long as is necessary. (4) This personal data processing can only take place if: a) the subject has given consent, b) it is necessary for contractual obligations or to comply with legal obligations or c) to protect the subject’s essential interests. It can also be processed for tasks in the public interest or for legitimate interests of the organisation. (5) The conditions for valid consent are stricter and demand that consent has been given freely, informed and specific, the request must be intelligible and clear, inaction is no consent, consent can be withdrawn, and the organisation must be able to prove the consent. (6) To implement the data processing principles adequately, data processors and controllers need appropriate safeguards into the processing and the design of new systems, processes and technologies. Moreover, a data protection impact assessment should be integrated in this “privacy by design” standard. The data protection impact assessment should evaluate the origin, nature, particularity and severity of a risk and should be taken into account when determining the appropriate measures.

Notifying the data subjects. (7) Data controllers must ensure a privacy notice at the moment of direct data collection, or in case of indirect data collection without undue delay within one month. Also the processing and intent should be clear and transparent. (8) In case of a breach of personal data the data processor needs to report the breaches to the data controller; the data controller needs to report the breaches the supervisory authority with 72 hours after the discover if there is a risk to the subject’s rights and freedoms; and the data subjects must be notified without undue delay if this same risk is high. A personal data breach means: “a breach of security leading to the accidental or unlawful destruction, loss, alteration, unauthorised disclosure of, or access to, personal data transmitted, stored or otherwise processed”.

International communication. (9) The GDPR also provides for protection of personal data that is transferred to international organisations or countries outside the EU. Such transfer is only allowed when a) the EU has determined that the country has an adequate level of 33

Cyber resilience in organisations Master Thesis Meret Keeris protection, b) there are model contracts or binding corporate rules in place, or c) there is compliance with an approved certification mechanism, for example the EU-US Privacy Shield.

Rights for the data subject. Finally, (10) the privacy rights of individuals have been enhanced, which include amongst others: the right to be informed, the right of access, the right to erasure, the right to restrict processing, and the right to object.

Risk mitigation by compliance Compliance with the GDPR aims at providing organisations with a good base of cyber resilience. The following section will further elaborate upon two articles that are considered the most important to ensure an adequate risk-based approach. These are Article 32 “Security of Processing” and Article 35 “Data Protection Impact Assessment”.

Article 32 “Security of Processing” Section 2 Security of Personal Data, Article 32 “Security of processing” of the GDPR sets out that: “Taking into account the state of the art, the costs of implementation and the nature, scope, context and purposes of processing as well as the risk of varying likelihood and severity for the rights and freedoms of natural persons, the controller and the processor shall implement appropriate technical and organisational measures to ensure a level of security appropriate to the risk […] ” (EP & CoEU, 2016).

Accordingly it sets out what is understood as ‘appropriate’. This includes (a) the pseudonymisation and encryption of personal data, (b) the continuous assurance of the confidentiality, integrity, availability and resilience of processing systems and services, (c) the timely restoring of availability and access to personal data in case of a physical or technical incident and (d) to have procedures in place for regular testing, assessing and evaluating the effectiveness of technical and organisational measures for ensuring the security of the processing (EP & CoEU, 2016).

The article subsequently provides that the levels of security should be determined based on the potential risks of data processing, and requires a code of conduct or another kind of mechanism that ensures and shows compliance with the requirements of the GDPR (EP & CoEU, 2016) (p.52). Furthermore, the data processor or controller should ensure that any natural person that has access to personal data, does not process the data except in case this is instructed by the controller or Union or Member State law (EP & CoEU, 2016).

Cyber resilience in organisations Master Thesis Meret Keeris

In other words, Article 32 of the GDPR determines that organisations which process data should take the necessary measures to ensure the required levels of protection. Moreover, it requires organisations to implement a risk mitigation mechanism that acts as an effective plan when the required level of data protection cannot be ensured. Based on the data’s threats, vulnerabilities and the likelihood that a vulnerability will be exploited, directed security controls should be implemented (KPMG, 2017).

Article 35 “Data Protection Impact Assessment” Another article of the GDPR that explicitly focuses on risk mitigation measures is Article 35, which introduces the concept of a Data Protection Impact Assessment (DPIA). The European Data Protection Board (EDPB – previously “Article 29 Working Party”) provides recommendations, guidelines, and best practices to encourage adequate implementation of the GDPR.

On Article 35 the EDPB drafted guidelines on the Data Protection Impact Assessment (DPIA) as required by the GDPR in case processing is “likely to result in a high risk”. The EDPB helps organisations to determine when processing meets these requirements and explains that DPIAs can help data controllers to show appropriate measures have been taken to be compliant with the GDPR. It argues that it is “a process for building and demonstrating compliance”. A DPIA helps with managing risks to the rights and freedoms of natural persons that could arise from processing the persons’ personal data. By describing the data processes and assessing the necessity and proportionality of the measures to mitigate the risks, DPIAs are a useful tool for data controllers to control risks and take accountability (Article 29 Working Party, 2017).

Article 35 requires a DPIA in case ‘the processing is likely to result in a high risk to the rights and freedoms of natural persons’ and describes a DPIA roughly as follows. The controller should provide (a) a systematic description of expected processing operations and the purposes and interest, (b) an assessment of the necessity and proportionality of the processing operations with regard to its purposes, (c) an assessments of risk to the rights and freedoms of data subjects, and (d) the envisaged measures to address these risks, such as security measures, safeguards, and mechanisms to guarantee protected personal data and compliance to the GDPR. In doing so, the controller must take into account the rights and interests of data subjects and other concerned persons (Article 29 Working Party, 2017) (p.4).

A DPIA can help to determine which processes are risky and which measures would be appropriate, also for those processes which do not require a DPIA. Similar processes may be

Cyber resilience in organisations Master Thesis Meret Keeris assessed and addressed by a single DPIA. Data controllers can also make use of a reference DPIA, which implies a shared or publicly accessible DPIA for processes that are similar. In such case, a justification for a single DPIA must be provided and the measures described in the DPIA must be implemented. Furthermore, a DPIA can also be used for assessing the risk and impact of a technology product on data protection (Article 29 Working Party, 2017).

The GDPR sets out the data protection standards which data processor or controllers are required to meet, but similar to the NIS Directive, it does not demand for specific technologies or measures to facilitate the security of data. Organisations are required to establish their own policies to ensure compliance and the protection of data (European Commission, 2018) (KPMG, 2017). However, as explained before, the larger part of European organisations fear non-compliance because they do not understand the implications and required measures of the GDPR. Since the fines for non-compliance are high, organisations need to understand the risks of data processing. The highlighted Articles 32 and 35 are helpful to set out the point of considerations for organisations to ensure a risk-based approach towards data protection. Measures should be appropriate, minimise the incident impact, ensure service continuity and compliance and are therefore a significant building block for cyber resilience.

Cyber resilience in organisations Master Thesis Meret Keeris

6. Case studies In this thesis, two case studies illustrate and validate the relevance of the identified factors. The case studies cover two instances of actual cyber incidents: the DigiNotar incident in the Netherlands in 2011 and the WannaCry crypto-ransomware attack at the National Health Service (NHS) in the United Kingdom in 2017. The analysis focusses on the presence of the four identified factors to analyse their relevance in practice. Here, only the present identified factors are discussed. The discussion in chapter 7, sets out the degree of presence of all four factors.

6.1 Case study 1: The DigiNotar incident The Dutch Safety Board (hereafter also ‘the Board’) conducted a thorough investigation of the DigiNotar incident and therein focused on the administrative-organisational issues of the case. They investigated how governmental organisations performed on administrative level performed in ensuring the digital security of citizens and how the incident was dealt with (The Dutch Safety Board, 2012). The official investigation report of the Dutch Safety Board is used to examine the DigiNotar incident (The Dutch Safety Board, 2012).

Note: The report uses the term ‘digital security’ rather than ‘cyber security’ but implies the same. This analysis uses the term ‘digital security’ to stay closest to the findings of the investigation.

6.1.1 The hack explained DigiNotar B.V. (hereafter DigiNotar) was a company that delivered digital certificates used to protect electronic data flows. It provided amongst others certificates to the Dutch government to protect electronic data flows between governmental institutions, and between governmental institutions and both citizens and businesses. Citizens rely on governmental efforts to ensure protection of digital personal information and communication with government services against any kind of harm. Hence, the digital certificates are an important instrument in facilitating protection.

Cyber resilience in organisations Master Thesis Meret Keeris

A digital certificate is an electronic document that verifies the identity of the person or institution which makes a public key available for encrypting electronic data traffic. The authenticity of a digital certificate is guaranteed by the certificate authority, who can provide the signature of the private key that corresponds to the public key. Accordingly, corresponding parties know the sent information originates from the verified sender. The certificate authority is commonly a trusted third party, in this case DigiNotar.

In June and July 2011, a hacker penetrated the computer systems of DigiNotar. Via the servers that were connected to the internet, the hacker succeeded to access the shielded parts of the business network. Accordingly, he managed to gain administrative rights and to generate false digital certificates. Moreover, the hacker deployed the keys used for digital signatures to prove the supposedly authenticity of the certificates. Over a period of weeks, the hacker was able to generate at least 531 certificates, of which a small portion has been confirmed to be distributed. On 29 August the CERT-Bund – the CERT of Germany – notifies the Dutch Govcert on a received notification of a log-in warning on an untrustworthy DigiNotar certificate by a Gmail- user in Iran. Intensive attempts to abuse these false certificates to intercept communication of Iranian Gmail users have been confirmed. The hacker was probably active in DigiNotar’s systems for the duration of more than one month. Starting off with a thorough exploration of the systems to get familiar, to accordingly conduct its activities in different phases.

There is no evidence of successful abuse of the certificates in the Netherlands. The eventual damage of the DigiNotar hack remained limited by effective handling of the institutions in the government’s crisis structure. They decided to inform citizens and organisations, ask Microsoft to postpone a planned update, and to subsume the operational management of DigiNotar. However, as a result of this hack, data flows between government institutions and citizens and businesses could have been intercepted. The confidentiality of certificates and the communication could not be assured, rising risks such as misuse of information, breaches of

Cyber resilience in organisations Master Thesis Meret Keeris privacy, identity fraud and financial damage. Moreover, the trust in the Dutch government would have been seriously damaged and have severe economic and societal consequences. The hack therefore opened many eyes on the risks in digital security and the need for adequate management (The Dutch Safety Board, 2012).

6.1.2 Technical security of DigiNotar’s systems The technical state of security of DigiNotar has been investigated by an ICT security specialist and concluded the following. Firstly, DigiNotar’s response to early signals of a potential targeted breach were minimal. The signals were not exceptional and did not raise concern, so some general security measures were taken without further investigation. Furthermore, although the systems of DigiNotar were configured in such way that the hacker was able to manipulate and erase most of his traces, it has been concluded that the hacker used vulnerabilities of outdated parts in the software which were not updated. Furthermore, the security sensitive parts in which the issuance of certificates took place were not separated from the servers that were connected to the internet. By building tunnels in between different segments of the network the hacker was able to allow data transfer between these parts. In addition, the hacker managed to unravel the password of an administrator, whose log-in provided access to the servers containing the certificate service software. Finally, within this shielded part also the secret keys to sign certificates were hosted. The hacker gained access and used the secret keys.

With regard to technical security of DigiNotar, the Board concluded that DigiNotar’s business network was not adequately secured. The identified issues in security were obsolete software, absence of a firewall and the use of randomized passwords. The Board thereby questions how it is possible that these vulnerabilities and deficiencies were unknown. One of the explanations is the acquisition of DigiNotar by the American company called Vasco, which caused disagreement on the responsibility of daily practices, among which security. Also, the acquisition shifted interests from operational to commercial results. The Board considered this a potential cause of decreased attention to security.

6.1.3 Awareness, assets and risks The government institutions did not account for the possibility that DigiNotar and hence their digital certificates could be compromised. The provision, availability and use of digital certificates takes place in a system of agreements, parties and technologies and is referred to as a “public key infrastructure” (PKI). The involved parties at the PKI-government had insufficient insight into the factors that could threaten the certificates’ confidentiality. There

Cyber resilience in organisations Master Thesis Meret Keeris was a lack of understanding and awareness on the risks to the confidentiality of the digital certificates and the consequent harm to the security of assets: protected data flows. Went unforeseen.

Also on other levels of governmental institutions this insight in digital security risks proved defective. The Board states that risk awareness was mostly present with the IT department and only scarcely with the administrative departments. The risks that were managed and assessed systematically, were the risks of handling municipal personal data records. The protection of these databases needed to be compliant to stricter regulation and therefore probably raised more awareness.

In their investigation, the Board also examined other government institutions and concluded that within the more executive organisations the level of operational involvement with digital security was higher. These institutions have higher levels of digital security of data flows and are more aware of the risks. The most likely explanation is that for the more operational organisations ensuring undisrupted digital data flows and business continuity is crucial and received more attention.

6.1.4 Roles and responsibilities The investigation also pointed out bottlenecks in matters on tasks and responsibilities. The Safety Board argues that digital security concerns should be an integral part of business operations and that managers should take responsibility for ensuring this security. The issue, however, was that the managerial part of cyber security was left to the persons operationally charged with it. The higher levels of management were minimally engaged with the cyber security of their organisation. They were not educated and skilled to fulfil this task, or to even determine which information was required to do so. This lack of this knowledge led to defective managerial capacity and as a result merely large incidents reached the managing board. Moreover, when the potential breach was reported to the board, they lacked the insight and knowledge on the functioning and risks of falsified digital certificates and therefore took distance from further developments concerning the breach.

Another issue is that especially in public organisation there is a tendency to delegate certain tasks to a separate organisation at distance from the relevant processes. Consequently, some tasks disappear in a blurry landscape of roles and responsibilities. Which, in the case of security tasks, can have serious consequences. The Board however emphasises that this form of horizontal responsibility is not a bad thing by definition. Combined capacities and joint

Cyber resilience in organisations Master Thesis Meret Keeris implementation can instead considerably improve efficiency and quality. The structures between actors should however be clear, so that each takes its responsibility.

Furthermore, open norms in the contractual appointments with the other parties allowed for own interpretation on reporting obligations. This, together with the assumption that governmental certificates were not breached, led DigiNotar to decide not to notify the other involved parties of a potential breach. The Board argues that despite the juridically right reporting choices of DigiNotar, suchlike situation demanded for a notification towards potentially affected parties. This would have allowed for earlier detection of security risks and to take appropriate measures to constrain damage.

6.1.5 Response The consequence of the unknown risks were insufficient preventative measures. Moreover, the incident response measure that was in place - withdrawing the certificates and shift to another provider - proved inadequate and risky. Implementing the incident measure could have led to widespread disruption of vital systems. The Public Key Infrastructure should ensure that a provider of certificates who is deemed unreliable is removed from the system. Moreover, the certificates the provider has distributed up to that point will become inoperable. Parties that use the unreliable and unusable certificates then need to shift to other providers. However, in the case of DigiNotar such intervention would not be possible without severe complications, such as wide-ranging economic and social disruption. There were no realistic back-up measures that could absorb processes and ensure operational continuity, once it turned out that also the PKI- certificates could have been compromised.

6.1.6 The cyber chain Due to full reliance on the quality of services of other parties, the government institutions were not much concerned with the security of the digital security. Even though DigiNotar was responsible for sensitive security services that could impact the vital interest of the government, the government institutions had limited insight into the actual business operations of DigiNotar. Government institutions assumed the certificate services were in good order. Both the client and the regulatory authority fully relied on the company’s services and the assessment by external auditors. Initially, neither internal nor external audits, such as periodic examinations, supervision and maintenance efforts, on accreditations and certification of DigiNotar revealed any irregularities. There were no observations that led to concern. However, the Board concluded that the external auditors and administrators Logius and OPTA (currently ACM) performed insufficient in guaranteeing the certainty the government relied upon to safeguard

Cyber resilience in organisations Master Thesis Meret Keeris the confidentiality of the certificates. By delegating auditing functions on certificate services to external parties, the government lost control over the means to verify the reliability of certificate authorities.

6.1.7 Regulation An examination of the applicable regulations at times of the DigiNotar incident provides more insight into the legal ground upon which the organisations acted at that time. In 2011, the year of the DigiNotar incident, Directive 95/46/EC of 24 October 1995 “on the protection of individuals with regard to the processing of personal data and on the free movement of such data” was applicable. Directive 95/46/EC – or in the Netherlands the Data Protection Act - has expired and has been repealed by the GDPR, which is more extensive and explicit than Directive 95/46/EC (EUR-Lex). The main requirement of Directive 95/46/EC was that personal data flows should be processed with the utmost diligence. The Directive focused on the general obligation to notify to the supervisory authorities of the processing of personal data (EP & CoEU, 1995). This general obligation did not, however, contribute much to the improvement of personal data protection. The Data Protection Act required appropriate technical and organisational measures to ensure an adequate level of protection against unlawful processing of personal data, thereby taking into account the risks of personal data processing. It did not, however, prescribe specific methods or organisational structures to meet the requirements. Moreover, since a Directive leaves room for individual application by Member States, it resulted in a fragmented implementation of data protection across the Union.

The Dutch Safety Board concludes that the applicable regulation included openly formulated provisions. The provisions provided for the end objectives, but left open which measures the organisation should take to reach those. The Board acknowledges that open norms are understandable, especially in a domain with constantly evolving technologies. However, it also argued that provisions with considerable room for own implementation requires high levels of responsibility of organisations to adequately fulfil the norms. The DigiNotar case demonstrates how the free room for implementation of regulatory provisions failed to provide adequate protection of personal data. The Board concluded that the government had an important role to play in creating the circumstances in which organisations would be stimulated to compliance.

6.1.8 Recommendations of the Dutch Safety Board The Board concluded the following three recommendations to the governmental institutions and ministers: (1) ascertain that managers and administrators take responsibility for managing digital security, (2) set out the conditions and requirements for government organisations to

Cyber resilience in organisations Master Thesis Meret Keeris systematically manage their digital security, and (3) develop a more secure issuance and use of digital certificates. The following issues can contribute to this.

Awareness and approach The DigiNotar case raised the awareness of government authorities that unforeseen risks will always exist in digital operations. The Board stresses that organisations should be aware that, equal to other kinds of security, full security is not possible. Instead, attention should be devoted to damage control and recovery. Moreover, past cyber incidents made the institutions realise that cyber security is an organisation-wide issue, rather than an ICT issue, and demands for more than technical solutions.

Risk management Furthermore, the Board explains that controlling cyber risks requires active cyber security management. This implies that the risks embedded in digitalisation should be mapped and assessed. An organisation should constantly assess the strengths and weaknesses/vulnerabilities of processes, and estimate whether the benefits are worth the risk, and determine its risk appetite accordingly. This information enables the drafting of appropriate measures to mitigate the risks.

Also, government institutions that are involved in electronic data traffic should ensure recovery mechanisms have been drafted beforehand, to ensure quick and effective recovery of processes in case of a disruption. This will not only minimise financial damage, but also any loss of trust of citizens in the government.

Chain and cooperation The Board furthermore concludes instead of focusing on the attribution of blame, it is more fruitful to open the dialogue for improved security initiatives while respecting mutual interests. Each involved party should take responsibility to contribute to the security of the total chain. The cooperation of involved parties needs to change to one based on incident notification, information sharing and learning from experience.

Regulation Finally, the Board argues for stricter supervision on data protection regulation compliance so that security standards are ensured. Compliance with regulation can provide the basis of a policy with concrete objectives, measures and a time frame.

The measures taken aimed to improve digital resilience and include measures on business continuity (such as stand-by certificates), improved communication, stricter supervision,

Cyber resilience in organisations Master Thesis Meret Keeris clearer task distribution and a refinement of the requirements for the ‘PKIgovernment’ (The Dutch Safety Board, 2012).

6.1.9 Response of the Dutch governmental crisis structure The investigation of the Dutch Safety Board has been conducted in conjunction with an investigation of the Dutch ‘Inspection Justice and Security’. The latter investigation focused on the adequacy of response of the Dutch governmental crisis structure to the DigiNotar crisis (Inspectie Justitie en Veiligheid, 2012). Although this case study focuses on the cyber resilience of government institutions on administrative level, the DigiNotar hack was, as mentioned before, put on the right track mostly due to adequate performance of the governmental crisis teams. It can therefore be relevant to briefly touch upon and learn from their response. This national crisis structure exists of multiple governmental security and crisis organisations, such as National Crisis Centre, Ministerial Commission on Crisis Management, Interdepartmental Commission on Crisis Management, the Advisory Team, and the National Information Centre. In case of a cyber crisis, certain cyber specific institutions such as the Cyber Security Council complement the generic structure. When on September 2 it became clear that the PKI- certificates could also have been compromised, the governmental crisis structure was deployed. The national crisis structure during the DigiNotar crisis was largely organised as described in the prepared response plan. Despite several deviations from the actual plan, the response was not negatively affected. Although in the beginning there is no complete picture of the threat, the crisis teams were scaled up quickly. This allowed for quick control of the (impending) crisis. The provision of information was efficient due to short lines and trust between the involved parties. This allowed for good interaction and resolute but informed decision-making. The Inspection report concludes that the crisis structure functioned adequately and effectively during the DigiNotar crisis (Inspectie Justitie en Veiligheid, 2012).

Cyber resilience in organisations Master Thesis Meret Keeris

6.2 Case study 2: The WannaCry attack at the NHS England The National Audit Office (NAO) of the UK is an independent organisation that scrutinises the UK government to improve public services (NAO, 2018). The NAO investigated the impact of the WannaCry attack on the NHS England in specific and examined the response to the attack of the NHS in its entirety. The findings of this investigation have been published in a report by the Comptroller and the Auditor General of the Department of Health. This official investigation report is used to examine the WannaCry cyber-attack at the NHS (NAO, 2018).

6.2.1 The attack explained On Friday 12 May 2017 a crypto-ransomware named “WannaCry” attacked numerous organisations in over 150 countries. WannaCry is an aggressive type of ransomware that is able to spread itself rapidly. It sophisticatedly exploits known vulnerabilities such as Eternal Blue in Windows driven computers. The ransomware encrypts files and demands to pay ransom to receive the encrypted files back. WannaCry seemed, however, not designed to receive ransom and is therefore expected to be created for large-scale disruption of operations (IBM Security, 2018). The ransomware also hit multiple critical infrastructures, among which the National Health Service in England.

WannaCry was the largest cyber-attack that the NHS has faced and seriously affected its services for about a week. It affected more than 80 of the 236 trusts in England, of which 34 trusts were directly infected and 46 trusts were disrupted without direct infection. The disruptions that the NHS staff experienced was a lock-out of devices. As a result, staff was not or hardly able to access and update patient information, to transfer test results, or to transfer or discharge patients. Moreover, medical equipment and devices were locked or isolated on purpose to prevent this. Especially departments such as radiology or pathology that are dependent on their equipment experienced severe disruptions of operations. The trusts that were not directly infected suffered disruptions due to several reasons. First of all, to prevent being hit, trusts undertook action such as shutting down or isolating devices on own their initiative in absence of timely central guidance. Furthermore, due to shared systems the trusts’ access to electronic data records was inhibited by the lock down of connected systems. Also, as a result of the attack trusts were disconnected from the main broadband network that connects all NHS sites in England.

Consequently, several processes had to be absorbed by manual tasks, numerous appointments had to be cancelled and several patients required diversion to other departments and trusts. Moreover, the communication with outside authorities and other trusts was extremely limited. 45

Cyber resilience in organisations Master Thesis Meret Keeris

The NHS did not report any direct damage to patients or patient’s personal data. There are no calculations available on the total financial costs of the disruption and recovery of services. The WannaCry attack could be stopped by a cyber security researcher who discovered the kill- switch of the ransomware.

6.2.2 Technical security of NHS’s systems The National Audit Office offers several technical explanations why the NHS was affected by the WannaCry ransomware. First of all, although the NAO acknowledges that full security is impossible, they argue that the NHS England failed to adequately implement security measures. All infected NHS organisations used unpatched or unsupported Windows operating systems. In March 2017 a patch for Windows 7 had been released by Microsoft which the NHS Digital stressed to install. Also, some trusts still used Windows XP operation systems even though this version no longer received patches. NHS England and NHS Digital states the equipment in the trusts is commonly managed by system vendors whose support was often inadequate. Isolating these devices from the rest would make it safer. Moreover, the organisations failed to install a firewall that could have protected them against infection.

NHS organisations were demanded to implement plans to move away from Windows XP. However, despite the Department of Health and Cabinet Office’s offer to assist with security for old equipment, a considerable part of devices was still using Windows XP in May 2017. After the WannaCry attack Microsoft distributed the necessary patch for Windows XP.

6.2.3 Awareness, assets and risks There was limited insight by the Department of Health and related bodies in the digital assets and the state of preparation of local trusts to cyber-attacks. Prior to the WannaCry attack NHS Digital, the party that evaluates the digital security of trusts, was conducting assessments on the level of cybersecurity. Of the 88 out of 236 trusts that were assessed by 12 May all had failed. Moreover, the trusts that actually implemented the provided advice and guidance already had rather mature cyber-security packages in place, while the most vulnerable trusts neglected any security action. Furthermore, the NAO states that NHS Digital concluded that trusts both failed to calculate cyber security as a risk to patient results and overestimated their readiness in managing a cyber-attack. NHS Digital explains this was due to a lack of understanding of cyber risks rather than conscious disregard of cyber security measures.

6.2.4 Roles and responsibilities The responsibility for cyber resilience in the health sector is shared by national and local institutions. The national responsibility for cyber resilience in the health sector lies with the

Cyber resilience in organisations Master Thesis Meret Keeris

Department of Health. Therein, it oversees the cyber resilience and incident responses. The Department, however, delegates responsibilities for actually managing cyber resilience to the local health service providers. These are supported by supervising and regulating authorities. Although these latter authorities and the Department can demand trusts to improve security, the bodies are generally not engaging in IT or cyber issues. NHS Digital, on the other hand, actively provides support, signals and evaluations, but cannot require a local trust to implement measures to improve resilience. Furthermore, NHS England is co-responsible for embedding cyber security standards in the health sector and should ensure that adequate incident response plans are in place with trusts. During a cyber incident, NHS England cooperates with NHS Improvement to communicate to the healthcare sector. NHS Improvement also helps to establish incident response plans and ensures that recommended cyber resilience measures are implemented.

6.2.5 Response In July 2016 the Department had been warned by the National Data Guardian and the Care Quality Commission on cyber threats and the need for preparation and response plans. Moreover, some of England’s trusts had been hit by a cyber-attack in the year before. It was proposed that all health organisations would prove measures to improve cyber resilience. These measures however seemed to remain absent.

The Department had established a cyber incident response plan including roles and responsibilities for national and local organisations prior to the WannaCry attack. It failed, however, to test the plan at local level and therefore the plan was unclear for local NHS trusts. When WannaCry hit the local trusts and the NHS England declared the situation a national major incident in which it subsumed the lead, the trusts were not familiar with the response plan and caused delayed responses. Another consequence of the unclear response plans was that local organisations reported to different authorities and also the communication to patients and local institutions came from different sources. Moreover, improvised channels for communication such as WhatsApp had to be deployed, since email correspondence was no longer available due to precautionary measures.

Part of the Emergency Preparedness, Resilience and Response processes (EPRR) was the CareCERT alert which would help to direct assistance effectively to the most needy trust. Compliance to this CareCERT alert had been required by the Department prior to the attack, but there was no formal assessment to verify compliance. Accordingly, on the day of the attack it was uncertain whether the trusts would apply the alert. The alert turned out to be used and

Cyber resilience in organisations Master Thesis Meret Keeris proved useful in helping the EPRR to prioritize and direct assistance to the trusts. The trusts could be supported by advice and technical support by the NHS Digital. Moreover, although NHS England did not have any stand-by IT teams in place, staff voluntarily responded and joint to solve issues.

6.2.6 The cyber chain NHS trusts mutually affected each other via their networks and systems. The trusts that employed an above average number of employees were mostly affected. Possible explanations that were given were that WannaCry exploits the Windows system that shares files, which meant that large Windows networks, i.e. large trusts, were most affected. Furthermore, in case of trust mergers, integrating different version of Windows operating systems is challenging. There was little awareness of the implications and vulnerabilities that arose from these interconnected systems.

6.2.7 Recommendations of the NAO The NAO concluded that the following measures are implemented or needed to improve the cyber resilience of the NHS.

Awareness and approach First of all, the organisation-wide cyber threat should be taken seriously. This means more insight into the NHS’ digital assets, improved understanding of cyber risks and implementing measures to improve cyber resilience and minimise the potential impact. NHS England is more actively concerned with providing trainings to boards and local levels to raise awareness on cyber threats and the need for cyber resilience.

Supervision Moreover, there is increased and stricter supervision on the implementation of security and alert measures. It should be ensured that organisations implement software patches and anti-virus updates and conduct CareCERT alerts. NHS Digital further developed its services to now include a hotline for incident responses, sharing best practices and on-site assessments.

Preparation and response A comprehensive response plan should be established ant tested which clearly sets out the roles and responsibilities of both local and national bodies and the Department. Also, during an incident essential communication channels should be safeguarded when normal communication means are unavailable. Finally, in response to the WannaCry attack NHS England and NHS Improvement redirected IT budgets to reinforce cyber resilience in the main trauma centres.

Cyber resilience in organisations Master Thesis Meret Keeris

7. Discussion and Conclusions 7.1 Discussion of results The current literature indicates there is a need for knowledge on the wider organisational context of cyber resilience and a merger of academic and practitioner knowledge. Present frameworks offer a clear overview of five main steps in the process towards cyber resilience. However, issues such as organisational culture, the chain of stakeholders and the role of regulation are discussed only limited. To be able to respond to the cyber threat, organisations need a better understanding of the implications of cyber resilience on the organisation.

This research aimed to fill this gap in knowledge and adopted a multidisciplinary approach to answer the question “Which factors should organisations minimally consider when drafting a cyber resilience policy?”

The findings of the literature analysis and the findings of the case studies are combined below to chart the answer to the research question. After the discussion section, the final answer to the research question is discussed.

Factor 1: Understanding the organisation Cyber resilience is an IT-exceeding issue that demands for an integrated organisation-wide approach. This, however, is a challenge for many organisations. Engaging the entire organisation and moving it towards a cyber resilient enterprise means that the strategy should be build bottom-up. The main bottleneck in building from the bottom is a lack of awareness. Organisations and employees often underestimate the cyber threat, likely due to its abstract nature. There is a need for awareness on cyber risks, threats, vulnerabilities and a need to take preparation seriously. From there, cyber responsible behaviour can be integrated. It requires a shift of organisational culture to establish the springboard from which behaviour can be changed and cyber resilience can be further enhanced. Developing this springboard requires identification of what is most valuable to the organisation: the key assets. The insight into this fundament of the organisation helps to determine the needs and priorities. Understanding the organisation means an organisation should be able to answer the questions “what is our organisational identity?” and “what needs to be protected?”, thereby explaining the culture, awareness and key assets of the organisation.

Cyber resilience in organisations Master Thesis Meret Keeris

Illustration from the case studies

The case studies demonstrate the importance of awareness and asset identification the clearest. In the DigiNotar case, the uncritical attitude and dependence of the Dutch government on the security of digital certificates was remarkable and risky. Prior to the incident there was a lack of awareness on potential cyber threats and a limited insight into assets. The hack was a wake- up call on the digital security of the Dutch government. It learnt that protected data flows are an important asset to the government and need protection against cyber threats. In the case of the WannaCry attack on the NHS, it is noteworthy that although the NHS was aware of cyber threats to their assets – the continuity of services and the protection of patient data – they still failed to better secure their assets. Most likely the NHS lacked understanding of the gravity of cyber threats and the potential impact of future cyber threats on their assets.

Neither of the case studies explicitly illustrate organisational identity in its entirety as factor of influence on the cyber resilience. However, both do demonstrate the importance of awareness and certain behaviour to be resilient. A possible explanation is that awareness and behaviour are more concrete and therefore easier to assess and understand.

Factor 2: Effective management Furthermore, effective management is essential. A large part of this comes down to risk management. Risk management implies more than identifying which risks the organisation faces but requires strategic courses of action to demarcate the threat landscape and focus on the most important threats. The first step to do so, is to ensure that risks to the identified assets are understood. Considering the probability and the potential loss caused by an attack, weighted against the degree to which security measures decrease this probability or loss, can help deciding on measures to be implemented. Depending on the risk appetite of an organisation, one may choose to take certain risks for granted. Organisations, should, however, always ensure a minimum level of protection of the key assets.

Financial resources have been identified as one of the main bottlenecks for cyber resilience. Generally, the costs of the damage caused by a cyber incident is considerably higher than the cost of the investment needed to prevent it. However, due to the complexity of cyber resilience, organisations struggle to spend resources correctly or even abstain from any expenditures on measures at all. Investments should be smart and demand for a strategic approach. This starts with understanding the organisation’s context and the priorities that arise from it. Clear insight into assets and risks will help to determine the focus of expenditures and measures.

Cyber resilience in organisations Master Thesis Meret Keeris

Most of the literature argues for the involvement of all departments to determine the most important risks and consequent measures. There are, however, differences in opinion on whether the actual responsibility for cyber resilience should lie with the cyber security manager, the entire management board or with a group of joint positions. Matters of considerations are: the size of the organisation, communication channels, and available resources in terms of budget, people and time. It is however, generally agreed that the person or persons in charge should be able to approach cyber resilience from a strategic perspective, and therefore requires more than technical IT skills. Whichever way the responsibility is configured, the key to effective management is, a very clear, transparent, and aligned division of tasks.

This clear division of task also benefits the organisation’s response once a cyber incident occurs. Organisations need effective incident responses to minimise the impact and prevent undue losses. Whether an organisation deploys an internal or external response team, an effective response requires a tailored response plan that is up to date, tested, and clearly sets out the coordination of tasks and procedures.

Illustration from the case studies

In both case studies, risk assessments would have helped to identify the vulnerabilities that allowed the incidents to happen. Risks could have been mitigated by implementing several basic technical security measures. Although it can only be speculated how the incident would have evolved in case of stronger passwords, updated software and a firewall in place, it is conceivable that the incidents could have been prevented, or at least be hampered considerably. In the NHS case, the weak state of cyber security was known, but further steps to ensure measures were implemented were not taken. In both cases, the misaligned structure of roles and responsibilities and limited supervision were the causes of unforeseen risks.

Moreover, the DigiNotar incident demonstrates that in absence of an integral response plan it becomes very challenging to foresee and effectively manage incidents. It caused considerable delay in notification of the involved government institutions. The incident response of the NHS was forced to improvise because the response plan was not practiced. The performance of the crisis structure in the DigiNotar case demonstrates the importance of threat intelligence, response plans, and adequate cooperation.

Finally, the cases do not explicitly discuss financial resources or expenditures. Only the NHS case mentions that cyber security budgets will be redirected and increased for the main trauma centres. It is therefore hard to draw conclusions on how resources played a role in the two cases.

Cyber resilience in organisations Master Thesis Meret Keeris

Factor 3: The cyber chain Cyber resilience demands for an approach that goes beyond the perimeters of the organisations. Understanding the influential interdependencies and other parties in the cyber chain helps to foresee organisational risks and opportunities. As stakes and interests become interrelated, organisations are required to cooperate, and if necessary, to contractually appoint issues on tasks and responsibilities. It is important that each party in the chain takes its responsibility, on which others can rely, such that they can focus on theirs. Cooperation implies sharing threat intelligence, best practices, building trust and establishing joint response plans. Taking this to a higher level, cooperation could also take place within the larger cyber ecosystem. The ecosystem is the extended environment of the organisation and can be of high value when knowledge, capacities, and resources are combined.

Illustration from the case studies

In the DigiNotar case, there was little consideration of interdependencies for the security of digital certificates. Although trust has been mentioned as the basis for effective cooperation, the DigiNotar case demonstrates how reliance on the performance of other parties makes organisations less alert. Trust should therefore be rightly balanced with vigilance. This way it stimulates cooperation but avoids the rise of vulnerabilities. In the case of NHS, the role of the cyber chain proved important, as connected trusts were affected via the interconnected networks of the NHS. Despite these negative effects, the chain also proved to be beneficial in the recovery after the attack, due to the cooperation of the related parties.

Furthermore, the cases do not discuss how the wider ecosystem (i.e. academia, governments, private sector, and individuals) was involved. One could, however, argue that the organisations would have been better prepared when there was more cooperation on such scale. Such cooperation could have provided the organisations with the knowledge needed to better understand the cyber threats they faced.

Factor 4: Regulation Cyberspace challenges governments to provide regulations that effectively improve security, without imposing undue obligations to organisations. Although some consider regulation on cyber security an impediment for innovation and independency, it is necessary to ensure a minimum level of protection of this public good. Moreover, compliance can considerably contribute to the organisation’s cyber resilience by stimulating a risk-based approach towards cyber threats. European regulation on cyber security mostly focuses on two matters: the

Cyber resilience in organisations Master Thesis Meret Keeris protection of critical infrastructures and the protection of personal data. Within the EU, the cyber resilience of critical infrastructures is regulated by the NIS Directive. It concerns a Directive and therefore primarily sets out the end objectives on improved cyber resilience, instead of requiring specific measures.

Furthermore, the GDPR of May 2018 places considerable pressure on European organisations to implement measures that ensure adequate protection of personal data. The Regulation repealed Directive 95/64/EC and significantly increased the standards for data protection and privacy. The GDPR is binding, more explicit and more extensive in scope than Directive 95/64/EC. Under the GDPR, personal data implies considerably more (e.g. IP addresses, geolocation and biometric data) than the rather limited scope definition of the Directive (e.g. photos, email addresses, phone numbers, and names). Furthermore, the GDPR sets out more explicitly the requirements and responsibilities of both data controllers and processors, where the Directive did not include data processors to be accountable. It furthermore stimulates protection and privacy by design, so that security measures are embedded in the operations. Finally, the GDPR requires impact assessments for large-scale processing of personal data and the appointment a data protection officer in case high risks are involved (EP & CoEU, 1995) (EP & CoEU, 2016) (Beaumont, 2018). The GDPR demands measures that are appropriate, minimise the incident impact, and ensure service continuity and compliance. Understanding the implications and value of the GDPR can offer organisations important handles to determine its policy and enhance cyber resilience.

Illustration from the case studies

At the time of the WannaCry attack on the NHS in 2017, the NIS Directive was not yet in force. However, now that in the UK the NIS Directive applies amongst others to the health sector (Department for Transport, 2018), it is interesting to consider how the NIS Directive could have made a difference to the impact of the WannaCry attack on the NHS. Critical infrastructures such as the NHS would have been forced to have appropriate measures in place, to minimise the incident impact, and to ensure service continuity and compliance. Perhaps that would have pressured the NHS to implement measures that restricted or prevented WannaCry to impact their services.

Furthermore, the DigiNotar hack was mainly possible because there was little awareness on the digital vulnerability of data assets and the potential impact of a breach. The differences between Directive 95/64/EC and the GDPR demonstrate well how risks in personal data processing were

Cyber resilience in organisations Master Thesis Meret Keeris underestimated and underexposed at the time of the DigiNotar incident. The DigiNotar hack illustrates that regulation could have helped to raise awareness on digital risks and stimulate organisations to take adequate security measures.

Although some of the sub-issues of the four main factors appeared to a lesser degree in the case studies than others, the case studies validated the importance of all factors that were identified from the literature.

7.2 Conclusions For drafting a cyber resilience policy, an organisation should minimally consider (1) a good understanding of the organisation, (2) effective management, (3) the cyber chain, and finally (4) regulation. It has become clear that although the factors are separated under different headings, most are interrelated. The factors mutually affect each other, and the framework can therefore best be interpreted in its entirety. The findings of this research demonstrate that the multidisciplinary approach adopted in this thesis is more than appropriate.

Nevertheless, each factor in itself invites for further examination. Academic research on cyber resilience is still in its early stages and likewise, this research merely scratched the surface of a complex subject. The findings of this research raise questions on the need of what could perhaps be called the “governance of cyber resilience” and how responsibilities shift or diffuse when the cyber threat continues to grow. It could, for example, be argued that a more prominent role of the government in controlling cyber space would be justified, since cyber incidents of organisations continue to impact citizens, society and economy for which governments largely bear responsibility. The versatility of cyber resilience leaves considerable room for future research. Some ideas for future research in line with this thesis are discussed below.

Cyber resilience in organisations Master Thesis Meret Keeris

7.3 Limitations of the research This thesis did not conduct research with actual organisations due to feasibility issues. To further enhance insight into the relevant factors for cyber resilience in organisations, future research could therefore include field research within organisations. This will facilitate access to documents and information such as policy papers, organisational strategies and budgets. Moreover, conducting interviews would be useful to find out more about the culture of an organisations and will perhaps lead to new uncovered issues.

In the search of case studies suitable for this research, it became clear that the availability of investigatory reports on cyber incidents is limited. Cyber incidents were mostly covered by news articles. Some were further investigated by cyber security practitioners or governments, but the focus was mostly on the technical aspects of the attack or on the wider implications for the need for cyber resilience in general. Limited material was available on how an organisation effectively dealt with a cyber incident or which factors proved essential in their response and resilience. The sensitivity of the topic and alike information most likely play a role in this. Although the case studies served an illustrative purpose, it would be interesting to examine more cyber incidents to further enhance the validity of this research’ findings.

7.4 Future research options The case studies in this research are both focused on governmental organisations. Although this should not be problematic due to the generic approach of the research, it would be interesting to learn more about private organisations. An interesting example would be the case of container logistics company A.P. Møller-Mærsk (Mærsk) and the NotPetya attack in June 2017. It is considered an interesting case first and foremost, because of the extremely large financial damages the company suffered as collateral damage of a cyber-attack. And secondly, with Mærsk’s nature as a logistics company, it would demonstrate the importance of other organisations in the chain well. Regarding both the negative impact an attack on the chain can have, as well as the cooperation Mærsk received in the recovery from the cyber-attack (A.P. Møller-Mærsk A/S, 2018) (Chirgwin, 2018). Furthermore, future case studies should also cover SMEs. As discussed before, SMEs is the largest group of organisations in Europe, but are generally least mature in cyber resilience. Commonly, SMEs have the most limited resources available, are mostly affected by regulations, have the least possibilities to rely on information sharing and cooperation and accordingly suffer disproportionately. Examining a case on an SME would offer SMEs more insight and understanding on what the cyber threat means for them and how to respond to the threat adequately.

Cyber resilience in organisations Master Thesis Meret Keeris

To conclude, this research identified four main factors for cyber resilience by starting from the basics of cyber resilience. Future research can take this as a starting point, and dive deeper into one of the identified factors. Finally, although the number of cyber incidents decreases, the impact is growing. More attention and research into the recovery element of resilience will therefore be needed.

Cyber resilience in organisations Master Thesis Meret Keeris

Bibliography A.P. Møller-Mærsk A/S. (2018, February). Annual Report 2017. Retrieved from Mærsk - Media: https://www.maersk.com/-/media/ml/about/sustainability/20180209-a-p- moller-maersk-annual-report.pdf Abraham, P. (2016). Why organisations need to speak about identity. Retrieved from The People Development Network - Change: https://peopledevelopmentmagazine.com/2016/11/20/organisations-identity/ Accenture Security. (2018). Gaining ground on the cyber attacker - 2018 State of Cyber Resilience. Retrieved from Accenture Security: https://www.accenture.com/t20180416T134038Z__w__/us-en/_acnmedia/PDF- 76/Accenture-2018-state-of-cyber-resilience.pdf#zoom=50 Adams et al., S. B. (2015, November P. 18-19). The governance of cybersecurity: A comparative quick scan of approaches in Canada, Estonia, Germany, the Netherlands and the UK. Retrieved from Tilburg University - Research Portal: https://research.tilburguniversity.edu/en/publications/the-governance-of-cybersecurity- a-comparative-quick-scan-of-appro Agrafiotis, I. N. (2018). A taxonomy of cyber-harms: Defining the impacts of cyber-attacks and understanding how they propagate. Retrieved from Journal of Cybersecurity - Oxford University Press: https://academic.oup.com/cybersecurity/article/4/1/tyy006/5133288 Anuar, B. F. (2011). A risk index model for security incident prioritisation . Retrieved from Edith Cowan University; Research Online - Australian Information Security Management Conference : https://ro.ecu.edu.au/ism/108/ Article 29 Working Party. (2017, October). Guidelines on Data Protection Impact Assessment (DPIA) and determining whether processing is “likely to result in a high risk” for the purposes of Regulation 2016/679 . Retrieved from Article 29 Data Protection Working Party: https://ec.europa.eu/newsroom/document.cfm?doc_id=44137 Bagheri, S. a. (2017). Organisational Cyber Resilience: Research opportunities. Retrieved from Organisational Cyber Resilience - Australasian Conference on Information Systems: https://www.acis2017.org/wp- content/uploads/2017/11/ACIS2017_paper_238_FULL.pdf Beaumont, S. (2018, January). The Data Protection Directive versus the GDPR: Understanding key changes. Retrieved from Synopsys - Software Security Blog: https://www.synopsys.com/blogs/software-security/dpd-vs-gdpr-key-changes/ Björck, F. e. (2015). Cyber Resilience – Fundamentals for a Definition. Retrieved from New Contributions in Information Systems and Technologies, pp 311-316: https://link.springer.com/chapter/10.1007/978-3-319-16486-1_31 Boyce, M. D.-R. (2011). Human Performance in Cybersecurity: A Research Agenda . Retrieved from Sage Publications - Journals: https://journals.sagepub.com/doi/pdf/10.1177/1071181311551233

Cyber resilience in organisations Master Thesis Meret Keeris

Capgemini. (2013). Cyber Resilience in het jaarverslag? Retrieved from Trends in veiligheid - Capgemini Nederland: https://www.trendsinveiligheid.nl/wp- content/uploads/2018/05/1B-004.13a-Trends-in-Veiligheid-2013_vFINAL_Cyber_- resilience_140313.pdf Capgemini. (2017, November). Trends in Cybersecurity 2017-2018. Retrieved from Capgemini Nederland - bronnen: https://www.capgemini.com/nl-nl/wp- content/uploads/sites/7/2017/11/trends-in-cybersecurity-report-2017-2018.pdf Capgemini Consulting. (2016, January). Insights into Cybersecurity 2015-16. Retrieved from Capgemini Consulting - Resources - Cybersecurity: https://www.capgemini.com/wp- content/uploads/2017/07/report_inisghts_in_cybersecurity_2015.pdf Carnegie Mellon. (2015, February). Computer Security Incident Response Plan. Retrieved from Carnegie Mellon University - Information Security Office: https://www.cmu.edu/iso/governance/procedures/docs/incidentresponseplan1.0.pdf CFI. (2018). Types of assets - Classifying assets based on convertibility, physical existence and usage. Retrieved from Corporate Finance Institute: https://corporatefinanceinstitute.com/resources/knowledge/accounting/types-of-assets/ Chirgwin, R. (2018, January). IT 'heroes' saved Maersk from NotPetya with ten-day reinstallation bliz. Retrieved from Cyber Peace - Security: https://cyber-peace.org/wp- content/uploads/2018/01/IT-heroes-saved-Maersk-from-NotPetya-with-ten-day- reinstallation-bliz-%E2%80%A2-The-Register.pdf Clemente, D. (2013, February). Cyber Security and Global Interdependence: What Is Critical? Retrieved from Chatham House - The Royal Institute of International Affairs: https://www.chathamhouse.org/sites/default/files/public/Research/International%20Se curity/0213pr_cyber.pdf Conklin, W. S. (2017, March). Cyber-Resilience: Seven Steps for Institutional Survival. Retrieved from The EDP Audit, Control, and Security Newsletter - Volume 55 Issue 2: https://www.tandfonline.com/doi/full/10.1080/07366981.2017.1289026 De Bruijn, H. J. (2017, January). Building Cybersecurity Awareness: The need for evidence- based framing strategies. Retrieved from Elsevier - Science Direct, Government Information Quarterly Vol. 34, Issue 1: https://www-sciencedirect- com.ezproxy.leidenuniv.nl:2443/science/article/pii/S0740624X17300540 de Crespigny, M. (2012, April). Building cyber-resilience to tackle threats. Retrieved from Elsevier Network Security - Information Security Forum: https://ac-els-cdn- com.ezproxy.leidenuniv.nl:2443/S1353485812700247/1-s2.0-S1353485812700247- main.pdf?_tid=48a835bf-17ba-4c12-86be- fba5e8cc7724&acdnat=1545141534_2a5bd1cdb31ddd004a2917b684ec4c6a Deloitte. (2016). Cyber crisis management: Readiness, response, and recovery. Retrieved from Deloitte - Strategic and Reputation Risk: https://www2.deloitte.com/content/dam/Deloitte/global/Documents/Risk/gx-cm- cyber-pov.pdf

Cyber resilience in organisations Master Thesis Meret Keeris

Deloitte. (2017). Assessing cyber risk - Critical questions for the Board and C-suite. Retrieved from Deloitte - Perspectives: https://www2.deloitte.com/content/dam/Deloitte/global/Documents/Risk/gx-ra- assessing-cyber-risk.pdf Deloitte Forward. (2018, November). Podcast Cases 1.1: PHGR Goes, hackers en het MKB. Retrieved from Deloitte Forward - Podcasts: https://www.deloitteforward.nl/podcasts/podcast-cases-1-1-phgr-goes-hackers-en-het- mkb/ Department for Transport. (2018, December). Implementation of the NIS Directive. Retrieved from UK Government: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachme nt_data/file/765786/implementation-of-the-nis-directive-dft-guidance.pdf Duane, M. B. (2018, April). When the going gets tough, the tough get going: overcoming the cyber risk appetite challenge. Retrieved from Marsh & McLennan Companies: Oliver Wyman - Publications: http://www.mmc.com/content/dam/oliver- wyman/v2/publications/2018/april/Oliver-Wyman-Overcoming-The-Cyber-Risk- Appetite-Challenge.pdf EESC. (2018, March). Cybersecurity: Ensuring awareness and resilience of the private sector across Europe in face of mounting cyber risks. Retrieved from The European Economic and Social Committee - Study on Cybersecurity - The Hague Security Delta -: https://www.thehaguesecuritydelta.com/media/com_hsd/report/191/document/qe- 01-18-515-en-n.pdf ENISA. (2006). Een stapsgewijze aanpak voor het samenstellen van een CSIRT. Retrieved from Europees Agentschap voor Netwerk- en Informatiebeveiliging (ENISA) - Resultaat WP2006/5.1(CERT-D1/D2): https://www.enisa.europa.eu/publications/csirt- setting-up-guide-in-dutch/at_download/fullReport ENISA. (2017, November). Cyber Security Culture in organisations. Retrieved from European Union Agency For Network and Information Security: https://www.enisa.europa.eu/publications/cyber-security-culture-in- organisations/at_download/fullReport ENISA. (n.d.). CSIRT Inventory. Retrieved from CSIRTS in Europe : https://www.enisa.europa.eu/topics/csirts-in-europe/csirt-inventory ENISA. (n.d.). CSIRTS in Europe. Retrieved from European Union Agency for Network and Informaton Security: https://www.enisa.europa.eu/topics/csirts-in-europe EP & CoEU. (1995, October). Directive 95/46/EC of the European Parliament and the Council of the European Union. Retrieved from EUR-Lex Europa - Official Journal of the European Communities: https://eur-lex.europa.eu/legal- content/EN/TXT/PDF/?uri=CELEX:31995L0046&from=EN EP & CoEU. (2016, April). Regulation (EU) 2016/679 of the European Parliament and of the Council of the European Union. Retrieved from EUR-Lex Europa - Official Journal of

Cyber resilience in organisations Master Thesis Meret Keeris

the European Union: https://eur-lex.europa.eu/legal- content/EN/TXT/PDF/?uri=CELEX:32016R0679 EUR-Lex. (n.d.). Document 31995L0046. Retrieved from Eur-Lex Access to European Union Law: https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:31995L0046 European Commission. (2016, July). Factsheet Directive on Security of Network and Information Systems. Retrieved from European Commission - Press Release Database: http://europa.eu/rapid/press-release_MEMO-16-2422_en.htm European Commission. (2017, September). State of the Union 2017: The Commission scales up its response to cyber-attacks. Retrieved from European Commission - Press Release Database: http://europa.eu/rapid/press-release_MEMO-17-3194_en.htm European Commission. (2018, May). Questions and Answers: Directive on Security of Network and Information. Retrieved from European Commission - Fact Sheet: http://europa.eu/rapid/press-release_MEMO-18-3651_en.pdf. European Commission. (n.d.). 2018 reform of EU data protection rules. Retrieved from Commission and its priorities: https://ec.europa.eu/commission/priorities/justice-and- fundamental-rights/data-protection/2018-reform-eu-data-protection-rules_en European Commission. (2018, August). The EU cybersecurity certification framework. Retrieved from European Commission - Digital Single Market - Policies: https://ec.europa.eu/digital-single-market/en/eu-cybersecurity-certification-framework European Commission. (2018). The GDPR: new opportunities, new obligations. Retrieved from European Commission - Justice and Consumers - Publications Office of the European Union: https://ec.europa.eu/commission/sites/beta-political/files/data- protection-factsheet-sme-obligations_en.pdf European Commission. (n.d.). EU law and its application. Retrieved from European Commission - Applying EU Law: https://ec.europa.eu/info/law/law-making- process/applying-eu-law_en European Commission. (Last update: 2018). Critical infrastructure. Retrieved from European Commission - Migration and Home Affairs: https://ec.europa.eu/home-affairs/what- we-do/policies/crisis-and-terrorism/critical-infrastructure_en European Commisson. (2018, August). Digital Single Market - Policy - The Directive on security of network and information systems (NIS Directive). Retrieved from European Commission Europe: https://ec.europa.eu/digital-single-market/en/network-and- information-security-nis-directive European Parliament. (2018, December). Legislative train schedule - Connected Digital Single Market - Cyber security package. Retrieved from European Parliament : http://www.europarl.europa.eu/legislative-train/theme-connected-digital-single- market/file-cyber-security-package Eurotransplant. (n.d.). Fit for the Future. Retrieved from Eurotransplant: https://www.eurotransplant.org/cms/index.php?page=fitfuture

Cyber resilience in organisations Master Thesis Meret Keeris

Evans, N. P. (2014). Responsibility and Accountability for Information Asset Management (IAM) in Organisations. Retrieved from The Electronic Journal Information Systems Evaluation - Volume 17 Issue 1 : http://www.ejise.com/issue/download.html?idArticle=943 Frigo, M. H. (2014, February). Understanding Your Organization’s Genuine Assets. Retrieved from SFMagazine Strategic Management - Strategic Finance: https://sfmagazine.com/wp-content/uploads/sfarchive/2014/02/STRATEGIC- MANAGEMENT-Understanding-Your-Organizations-Genuine-Assets.pdf Gordon, A. L. (2003). Information security expenditures and real options: a wait-and-see approach. Retrieved from SSRN - Computer Security Journal, Vol. XIX, No. 2: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1375460 GvIB. (2006, January). GvIB Expertbrief - CERT in de organisatie. Retrieved from Platform voor Informatie Beveiliging - Genootschap van Informatie Beveiligers (GvIB): https://www.pvib.nl/kenniscentrum/documenten/expertbrief-cert-in-de- organisatie/downloaden HSD. (2015). Securing Critical Infrastructures in the Netherlands - Towards a National Testbed. Retrieved from The Hague Security Delta: https://www.thehaguesecuritydelta.com/media/com_hsd/report/53/document/Securing -Critical-Infrastructures-in-the-Netherlands.pdf IBM. (2018). IBM Security - 2018 Cost of a Data Breach Study by Ponemon. Retrieved from IBM Security - Cost of a Data Breach Study: https://www.ibm.com/security/data- breach?ce=ISM0484&ct=SWG&cmp=IBMSocial&cm=h&cr=Security&ccy=US&cm _mc_uid=81779339829815455949034&cm_mc_sid_50200000=31908941545594903 476&cm_mc_sid_52640000=11474021545594903482 IBM Security. (2018, March). IBM X-Force Threat Intelligence Index 2018 - Notable security events of 2017, and a look ahead. Retrieved from IBM Security Solutions IBM X- Force Threat Intelligence Index 2018: https://public.dhe.ibm.com/common/ssi/ecm/77/en/77014377usen/security-ibm- security-solutions-wg-research-report-77014377usen-20180404.pdf Inspectie Justitie en Veiligheid. (2012). Evaluatie van de rijkscrisisorganisatie tijden de DigiNotar-crisis. Retrieved from Ministerie van Justitie en Veiligheid - Inspectie Justitie en Veiligheid: https://zoek.officielebekendmakingen.nl/blg-177937.pdf ISO. (2012). ISO/IEC 27032:2012. Retrieved from Organisation Internationale de Normalisation (ISO): https://www.iso.org/obp/ui/#iso:std:iso-iec:27032:ed-1:v1:en IT Governance. (2018). The Directive on security of network and information systems (NIS Directive/NIS Regulations). Retrieved from IT Governance - NIS Directive: https://www.itgovernance.co.uk/nis-directive IT Governance. (2018). The EU General Data Protection Regulation (GDPR). Retrieved from ITGovernance, Data Protection - GDPR: https://www.itgovernance.co.uk/data- protection-dpa-and-eu-data-protection-regulation

Cyber resilience in organisations Master Thesis Meret Keeris

ITU. (2018). Recommendation ITU-T X.1205 (04/2008) - SERIES X: DATA NETWORKS, OPEN SYSTEM - Overview of cybersecurity . Retrieved from International Telecommunications Union (ITU) - Telecommunicatino Standardization Sector of ITU: https://ccdcoe.org/sites/default/files/documents/ITU-080418- RecomOverviewOfCS.pdf Kaspersky. (2017). The Human Factor in IT Security: How Employees are Making Businesses Vulnerable from Within. Retrieved from Kaspersky Lab, Daily - Blogs: https://www.kaspersky.com/blog/the-human-factor-in-it-security/ Keys, B. C. (2016, April). A framework for assessing cyber resilience - A Report for the World Economic Forum. Retrieved from Edward J. Bloustein School of Planning and Public Policy : http://bloustein.rutgers.edu/wp- content/uploads/2016/05/2016_WEF.pdf Knapp, K. M. (2009, October). Information security policy: An organizational-level process model. Retrieved from Science Direct, Elsevier - Computers & Security, Vol. 28 Issue 7: https://www.sciencedirect.com/science/article/pii/S0167404809000765#bib48 KPMG. (2017). Cyber Resilience - Protecting your business. Retrieved from KPMG: https://assets.kpmg.com/content/dam/kpmg/ie/pdf/2017/09/ie-cyber-resilience-2.pdf Lewis, A. J. (2009, March). Innovation and Cybersecurity Regulation . Retrieved from Center for Strategic and International Studies: https://csis-prod.s3.amazonaws.com/s3fs- public/legacy_files/files/media/csis/pubs/090327_lewis_innovation_cybersecurity.pdf Lin, Y. (2004). Organization Identity and Its Implication on Organization Development. Retrieved from ERIC - the Academy of Human Resource Development International Conference: https://eric.ed.gov/?id=ED492427 Linkov, I. E. (2013, November). Resilience metrics for cyber systems. Retrieved from SpringerLink - Environment Systems and Decisions: https://link.springer.com/content/pdf/10.1007%2Fs10669-013-9485-y.pdf McGill, W. A. (2007). Risk Analysis for Critical Asset Protection. Retrieved from Wiley Online Library - Risk Analysis, Vol. 27, No. 5,: https://onlinelibrary-wiley- com.ezproxy.leidenuniv.nl:2443/doi/epdf/10.1111/j.1539-6924.2007.00955.x McSweeney, K. (2018, January). Motivating cybersecurity compliance in critical infrastructure industries: a grounded theory study. Retrieved from ProQuest Dissertations Publishing, Capella Universit: https://search-proquest- com.ezproxy.leidenuniv.nl:2443/docview/2011012472?pq-origsite=primo Ministery of Defense. (n.d.). Defensie Computer Emergency Response Team. Retrieved from Rijksoverheid - Cyber security: https://www.defensie.nl/onderwerpen/cybersecurity/defcert Musaraj, K. (2014, December). Future cybersecurity threats and needs. Retrieved from Project Liris - Thales Group: https://projet.liris.cnrs.fr/cyber/WS3Presentations/Musaraj.pdf

Cyber resilience in organisations Master Thesis Meret Keeris

NAO. (2018, April). Investigation: WannaCry cyber attack and the NHS. Retrieved from National Audit Office - Investigation: https://www.nao.org.uk/wp- content/uploads/2017/10/Investigation-WannaCry-cyber-attack-and-the-NHS.pdf National Research Council. (2014). At the Nexus of Cybersecurity and Public Policy: Some Basic Concepts and Issues - Why care about cybersecurity? Retrieved from The National Academics of Sciences, Engineering and Medicine: https://www.nap.edu/read/18749/chapter/3#9 NCSC. (2017). Naar een landelijk dekkend stelsel van. Retrieved from Cyber Security Council - Advies informatieuitwisseling: https://www.cybersecurityraad.nl/binaries/CSR_Advies_Informatieuitwisseling_NED _tcm107-314535.pdf NCSC. (2018, November). Nieuwe handreikingen voor het starten van cybersecurity- samenwerkingsverbanden. Retrieved from Cyber Security Council - Actueel: https://www.cybersecurityraad.nl/010_Actueel/nieuwe-handreikingen-voor- cybersecurity-samenwerkingsverbanden.aspx NCSC. (n.d.). Incident Response. Retrieved from Nationaal Cyber Security Centrum: https://www.ncsc.nl/incident-response/24-uurshulp.html NCTV. (2018). Cyber Security Assessment Netherlands. Retrieved from Nationaal Coordinator for Security and Counterterrorism: https://english.nctv.nl/binaries/CSBN2018_EN_web_tcm32-346655.pdf NOREA. (2015, August). Cyber Security Assessment (NOREA-CSA) Analyse Cyber Security Standaarden en Frameworks . Retrieved from NOREA: https://www.norea.nl/download/?id=2291 Panda Security. (2017). Cyber Resilience: The Key to Business Security. Retrieved from Panda Security Summit - The European Cybersecurity Hub: https://www.pandasecurity.com/mediacenter/src/uploads/2018/05/Cyber-Resilience- Report-EN.pdf Park, K.-Y. Y.-G. (2011). Security Requirements Prioritization Based on Threat Modeling and Valuation Graph . Retrieved from SpringerLink - International Conference on Hybrid Information Technology: https://link.springer.com/chapter/10.1007/978-3-642- 24106-2_19 Ponemon Institute. (2018, March). The Third Annual Study on the Cyber Resilient Organization . Retrieved from Resilient Systems - Global - Ponemon Institute & IBM Resilient: https://info.resilientsystems.com/hubfs/IBM_Resilient_Branded_Content/White_Paper s/2018_Cyber_Resilient_Organisation_Study.pdf Pursiainen, C. (2009). The Challenges for European Critical Infrastructure Protection. Retrieved from Taylor & Francis - Journal of European Integration, Vol. 31: https://www.tandfonline.com/doi/abs/10.1080/07036330903199846 PWC. (2015, December). Enhancing business resilience: Transforming cyber risk management through the role of the Chief Risk Officer (CRO). Retrieved from PWC

Cyber resilience in organisations Master Thesis Meret Keeris

Financial Services: https://www.pwc.com/gx/en/financial-services/pdf/fs-enhancing- business-resilience.pdf PWC. (2018). Building a cyber resilient financial institution. Retrieved from PriceWaterhouseCoopers and the Asian Institute of Chartered Bankers: https://www.pwc.com/my/en/assets/publications/2018/aicb-pwc-publication.pdf Rajamäki, J. P. (2017, June). Towards the cyber security paradigm of ehealth: Resilience and design aspects. Retrieved from Applied Mathematics and Computer Science - The American Institute of Physics: https://aip.scitation.org/doi/pdf/10.1063/1.4981969?class=pdf Rotterdam Port. (n.d.). FERM Rotterdam Port Cyber Resilience. Retrieved from Havenbedrijf Rotterdam: https://ferm-rotterdam.nl/ Schatz, D. B. (2017, June). Towards a More Representative Definition of Cyber Security . Retrieved from The Journal of Digital Forensics, Security and Law - Vol. 12, Nr. 2: https://commons.erau.edu/jdfsl/vol12/iss2/8/?utm_source=commons.erau.edu%2Fjdfsl %2Fvol12%2Fiss2%2F8&utm_medium=PDF&utm_campaign=PDFCoverPages Schein, E. H. (1997). Organisational Culture & Leadership. Retrieved from Educational Impact: http://www.educationalimpact.com/resources/usl2/pdf/usl2_3_organisational_culture. pdf Spot Schiphol. (2017, July). Cyssec: Samen Schiphol Secure. Retrieved from Spot Schiphol Community: https://spotschiphol.nl/nl/news/cyssec-samen-schiphol-secure The Dutch Safety Board. (2012, June). Het DigiNotarincident: Waaromd digitale veiligheid de bestuurstafel weinig bereikt. Retrieved from Onderzoeksraad voor de Veiligheid: https://www.onderzoeksraad.nl/nl/page/1730/het-diginotarincident The Kosciuszko Institute. (2017). European Cybersecurity Market. Retrieved from Cybersechub: https://cybersechub.eu/files/ECM_regional_small.pdf The National Archives. (2017). Identifying Information Assets and Business Requirements . Retrieved from National Archives: http://www.nationalarchives.gov.uk/documents/information-management/identify- information-assets.pdf The Stationary Office. (2015, June). Cyber Resilience Best Practices. Retrieved from The Stationary Office: https://www.tsoshop.co.uk/gempdf/RESILIA_Cyber_Resilience_Best_Practices.pdf Van der Meulen, N. (2015, August). Investing in cybersecurity. Retrieved from RAND Europe - Commissioned by the Research and Documentation Centre (Wetenschappelijk Onderzoek- en Documentatiecentrum, WODC) : https://www.wodc.nl/binaries/2551-full-text_tcm28-73946.pdf Von Solms, R. V. (2013). From information security to cyber security. Retrieved from Elsevier - Science Direct - Computers and Security 38: https://profsandhu.com/cs5323_s18/Solms-Niekerk-2013.pdf

Cyber resilience in organisations Master Thesis Meret Keeris

Weill, P. W. (2015). Thriving in an Increasingly Digital Ecosystem. Retrieved from MITSloan Management Review - Digital Business: Strategy - Vol. 56, No.4: http://ilp.mit.edu/media/news_articles/smr/2015/56417.pdf West, R. (2008, April). The Psychology of Security. Retrieved from Research Gate: https://www.researchgate.net/publication/220426390_The_Psychology_of_Security World Economic Forum. (2017, January). Advancing Cyber Resilience: Principles and Tools for Boards. Retrieved from WeForum - Whitepapers: Future of Digital Economy and Society Systems Initiative: https://www.weforum.org/whitepapers/advancing-cyber- resilience-principles-and-tools-for-boards World Government Summit. (n.d.). Cyber Resilience in the Digital Age. Retrieved from World Government Summit - Publications: https://www.worldgovernmentsummit.org/api/publications/document?id=24717dc4- e97c-6578-b2f8-ff0000a7ddb6 Yeatman, C. (2015, November). Organisational Identity: From “why are we” to “who are we”. Retrieved from Worldsview Academy: https://worldsviewacademy.com/organisational-identity-from-why-are-we-to-who-are- we/