Cyber-Attacks As an Instrument of Terrorism? – Motives – Skills – Likelihood

Home , Chaos Computer Club, Petya (malware)

Cyber-Attacks as an Instrument of Terrorism? – Motives – Skills – Likelihood – Prevention

DRAFT VERSION

Hendrik Hoffmann / Kai Masser

German University of Administrative Sciences Speyer / German Research Institute for Public Administration Speyer 05/08/2018

Paper to be presented at the 2018 IIAS (International Institute of Administrative Sciences) Congress, 25-29 June, Tunis, Tunisia:

Strive, Adapt, Maintain: Resilience of Governance Systems

Abstract “Cyber-Attacks” are mostly “Cyber-Crime”. Most attacks are “financially driven”. However, the most notorious ”cyber-attacks” like Stuxnet, WannaCry and Petya/NonPetya are linked to state or state- partisan activities. Seemingly, there is a growing market for “Cyber-Defense”. However, the cyber- attack arena is very much reminiscent to piracy of the 16th and 17th century with the ingredients money, power and a legal void.

In the second part of the paper, we investigate technical aspects of the attacks introduced in the first part. First, we look at attack vectors with a critical examination of how to minimize the attack surface. We then suggest defensive approaches which lead to best practices. We conclude with a consideration of the cyber inherent issue of unattributeability in conjunction with false flag operations.

Table of Contents

Abstract ...... 1 A Introduction: The Pirates of the Cyberspace ...... 3 A.1 Crime as a Key Driver for Cyber-Attacks ...... 3 A.2 Cyber-Attacks: A growing Threat ...... 4 A.3 A Threat to Whom, by Who and How can it do Harm? ...... 7 A.4 Criminals – the State – and Unconfirmed Information ...... 9 A.5 Summing-Up: Cyber-Piracy - A Frame of Reference ...... 13 B Offense and Defense ...... 13 B.2 Attack vectors: Technical and Human Failure ...... 14 B.2 Internet of Things: Leaving the Backdoor Open...... 15 B.3 Defense: Mitigation is Good - Avoidance is Better ...... 16 B.4 Open Source – Documentation, strategy and emergency planning - Backups ...... 16 B.5 Staff: Encryption – Physical Access ...... 17 B.6 Honeypots and Oracles ...... 18 B.7 Best Practice ...... 18 C To Sum-Up ...... 18 C.1 Parties in the Background ...... 18 C.2 Lessons Learned: Piracy is a Commercial Business, sometimes used by States to Attack Enemies, but rather Unsuitable for Terrorists ...... 20 C.3 Outlook ...... 21 References ...... 22 Print ...... 22 Online (all accessed from January until May 2018) ...... 22

An Introduction: The Pirates of the Cyberspace An alert observer of the press, especially online media, must conclude that the number of cyber- attacks is increasing steadily and heavily1. The website HACKMAGEDDON claims to monitor cyber- attacks on a daily base. According to the statistics of “hackmageddon.com” (2018) the number of cyber-attacks is rather stable:

Figure 1: Number of Cyber-Attacks reported to “hackmaddegon.com” (2018), 2014-2017, own calculations, absolute figures

According to the data gathered by hackmageddon.com, there is a rather stable figure around 900 to 1.000 attacks per year with a trend to rise until the year 2016, but then drops again to under 900 in the year 2017.

A.1 Crime as a Key Driver for Cyber-Attacks Moreover, according to the data of hackmageddon.com crime is the foremost and dominating motivation for Cyber-Attacks:

1 E.g. the website „thebestvpn“ (2018) introduces an article about „Cyber Security Statistics” with the sentence: “Cyber-attacks are growing in prominence every day – from influencing major elections to crippling businesses overnight, the role cyber warfare plays in our daily lives should not be underestimated.” Unfortunately, the site is offering no statistical data aka evidence about the “growing” of the number or impact of cyber-attacks. 3

Crime Espionage Hacktivism Warfare

100

77 72 75 67

25 21 15 14 9 10 5 3 4 2 0 2017 2016 2015

Figure 2: Motivation behind Cyber-Attacks according to “hackmaddegon.com” (2018), 2015-2017, own calculations, percentage

While hacktivism seems to play a lesser role from 2015 (21%) to 2017 (5%), espionage (2015: 10% / 2017: 15%) and particularly crime seems to be on the rise (2015: 67% / 2017: 77%). According to hackmageddon.com., crime is the overwhelming motivation for cyber-attacks.

With regard to the techniques used for cyber-attacks, Malware2/POS3 are mainly on the rise:

Unkown Account Hijacking Targeted Attack Malware/POS

40 33 30 30 24 22 23 20 16 16 15 12 11 10 8 9 10 9 10 6

0 2017 2016 2015 2014

Figure 3: Most frequent Techniques behind Cyber-Attacks according to “hackmaddegon.com” (2018), 2015-2017, own calculations, percentage

A.2 Cyber-Attacks: A growing Threat If you listen to the media, “the number of devastating cyberattacks is surging” (CNBC 2017). The quoted CNBC article bases its alarming conclusion upon an increase in data breaches (“in the first six months of 2017”). The question rises, whether data breaches and cyberattacks can be can equated?

2 A malicious program that can execute itself and spreads by infecting other programs or files. 3 POS: Point of Sales. 4

Moreover, six months of observation is a rather short period and, additionally, the data comes from “Gemalto” (Google Search 2018). Gemalto is a company which offers its services via google as “the world leader in Digital Security”. However, companies providing IT / Digital / Cyber Security are delivering almost all data about the development and the threats especially about cyberattacks.4 Our first data source “hackmaddegon.com” already has a link to an IT security company. However, asking IT-Cyber security companies about increasing threats of cyber-attacks and hence a need for increased defense measures, is like ask the hairdresser whether you need a haircut.

If we look at the revenues of companies selling cyber security compared to their spending for sales & marketing (S&D) there seems to be a correlation that the less money a cyber security company is making the more – proportionally - the company spends on S&D (Marketing).

5000 75

4000

50 3000

2000 25

1000

0 0 Symantec Palo Alto Checkpoint Fortinet FireEye Proofpoint

Revenue (million USD) % Sales & Marketing

Figure 4: Marketing spending (S&D) of selected public cyber security selling companies, compared to their total revenue, Source: Own inquiries according to latest business reports

This is not necessarily a sign nor a statistical proof that cyber-security companies are mostly selling “hot air”. On the other hand, a vast investment in S&D is a precondition to sell any kind of “hot air”.

However, cybersecurity seems to be a growing business. The ICT5 market review and forecast of TIA6 (2018) shows that cybersecurity spending in the U.S. increased from 2009 from about 27 billion USD to about 63 billion USD in 2017. Its share of the GDP7 increased in the same period from about 0,18% (2009/10) to nearly 0,38% (2017).

4 See for example the websites of FireEye, IBM or Symantec (all 2018). 5 ICT: Information and communication technology. 6 TIA: The Telecommunications Industry Association. 7 GDP: Gross domestic product. 5

100 1

75 0,75

50 0,5

25 0,25

0 0 2009 2010 2011 2012 2013 2014 2015 2016 2017

billions, USD percent of GDP

Figure 5: Cybersecurity Spending in the U.S., percent of GDP and USD billions, 2009-2017, Source: TIA (2018)

Does the increase in spending for cyber security seemingly indicates a rising need for it? However, what role the cyber security companies are playing? Are they just meeting the rising demands of governments, administrations and private companies; or do they might increase the market by over exaggerated-large thread scenarios; very much like some doctors supposedly sell unnecessary surgery and medication?

The US Government in 2017 banned all security software of the Russian IT-security company Kaspersky Lab (Wired 2018). It was assumed that the Russian security software contained access opportunities which allow Russian Hackers to attack Western Governments and Companies. Moreover, it is not to be dismissed cyber security companies organize cyber-attacks themselves in order to get more orders. Like glazier`s workshops, in days of yore, might have payed bullies to brake windows in the neighborhood.

Some of the biggest breaches / hacks in 2017 (Wired 2017):

1. Shadow Brokers Mysterious hacker group “Shadow Brokers” claimed to breach elite spy-tools of the NSA. The Shadow Brokers offered samples of stolen data and offered an auction of data not even stolen yet (contract work?) Moreover, the group published significant tools – allegedly to be developed by the NSA – like “EternalBlue”, which allows infecting standard software tools (e.g. based on Windows) and was used by hackers in two high-profile ransomware attacks in 2017. The leaks of the group provoke the question to what extend it is justifiable to use bugs in standard (commercial) software for intelligence purposes and not to inform the public about these security issues. This practice endangers billions of software users only balanced by a very vague benefit in security.

2. WannaCry Ransomware. Spread around the world, affecting hundreds of thousands of public as well as private targets. WannaCry crippled even critical infrastructure, e.g. hospitals in the UK. Even 6

if the malware had significant flaws easy to defend against, the damage done is significant but not impressive: “In total, WannaCry netted almost 52 bitcoins, or about $130,000—not much for such viral ransomware” (Wired 2017). The North Korean Government was accounted for developing and introducing WannaCry with “moderate confidence”. Because of the small revenue a project gone awry.

3. Petya/NotPetya/Nyetya/Goldeneye “More advanced than WannaCry”, but still had some flaws, “like an ineffective and inefficient payment system” (Wired 2017). It infected networks in many countries for instance Russia, the oil giant Rosneft. “Researchers suspect that the ransomware actually masked a targeted cyberattack against Ukraine. The ransomware hit Ukrainian infrastructure particularly hard, disrupting utilities like power companies, airports, public transit, and the central bank, just the latest in a series of cyber assaults against the country” (Wired 2017)

4. Wikileaks CIA Vault 7 In March 2017 Wikileaks published more than 8.500 documents allegedly stolen from the CIA. The material contained information about how to use vulnerabilities in iOS and Android, bugs in Windows as well as information about how to turn smart TVs into listening devices. Additional material delivered information about using Wi-Fi signals to track devices locations and surveilling Mac’s. “WikiLeaks claims that Vault 7 reveals "the majority of [the CIA] hacking arsenal including malware, viruses, trojans, weaponized 'zero day' exploits, malware remote control systems and associated documentation."” (Wired 2017)

5. 198 Million Voter Records Exposed A publicly accessible database of 198 million US voters was uncovered in the beginning of the year 2017. The accessibility of the data was caused by the misconfiguration of a server, which isn’t a malicious hack in itself but is a too-common cybersecurity risk.

6. Macron Campaign Hack Hackers dumped a 9GB trove of leaked e-Mails of the now French President Emmanuelle Macron just before the presidential runoff in May 2017. The attack by no means as serious as the emails interfere the presidential campaign of Hillary Clinton. “Allegedly” researchers thought to have found evidence that Russian-government-linked hacker group “Fancy Bear” were behind the attack against the Macron campaign. Was it an attempt to destabilize democracy supported by the Russian government as seen before in the US? (Wired 2017)

In chapter A.1 we argued that crime is a key driver for cyber-attacks. The preceding examples (1.-6.) on the other hand indicate political motives behind many of (the most spectacular cases with corresponding attention by the media).

A.3 A Threat to Whom, by Who and How can it do Harm? Firstly, state agencies like the NSA are accused of using spying tools (isn’t espionage the task of agencies like the NSA, not only in the USA?). However, as long as there are nation states, there will be intelligence services. As long as the importance of the internet and social media for information

7 and communication is growing, intelligence services will increase their efforts to control the internet. (The most famous counterstrike of “the net” against state espionage and control might be the “torproject” (2018)8.

Moreover, states or state agencies alleged to develop ransomware like WannaCry or NonPetya (and many others) that attacked numerous targets as well as critical public infrastructure like hospitals. Usually and obviously, the accusations follow existing conventional (non-digital aka9 analogue) conflicts between countries:

 North Korea accused of “cyber-attacks” especially by South Korea and the USA with growing capabilities (New York Times 2017) (not confirmed). British Newspaper “The Guardian” (2018a) headlined “North Korea is a bigger cyber-attack threat than Russia, says expert”. “The Guardian” has a specialized section “cyberwar” (sic!). It is hardly surprising that the quoted specialist is head of a security firm: “Head of security firm says highly skilled DPRK hackers may attack US financial sector to deter military action against the regime”. The author of the article resumes that “North Korea has been implicated in a number of major cyber-attacks over the past few years, primarily against South Korea”. But when it comes down to hard facts the author states that: “the “Lazarus group”, an elite North Korean hacking unit, is believed to have created and deployed the WannaCry ransomworm.” Has the “believe” of someone already become a hard fact like an empirical proof or juridical evidence?  Israel seems to be a famous target for cyber-attacks (from all over the world) (Jerusalem Post 2018) – On the other hand, the infamous “Stuxnet Worm” (the world’s first “cyber or digital weapon”) was allegedly designed by Israel and the USA to attack critical infrastructure supporting facilities like power plants, water treatment facilities, gas lines) (lifewire 2018 (sic!)). Iran and especially the Iranian Nuclear Facilities should be the foremost targets. The origin of the worm is a subject of speculations: “However, the designers of the worm are still unknown” (Stanford.edu 2015).  Russia is a large and nice country with very friendly people but sometimes issues with its neighbors. Hence there are concerns about cyber-attacks: o Estonia: “The cyber-attacks that befell Estonia in 2007 is a case much discussed and underrated at the same time. Many tend to ignore the eloquent fact that this incident represents the first time when an entire country’s information defense systems and resources were put to the test.” (INFOSEC Institute 2013). The before quoted sources subsequently argues whether the, mainly DDOS/DOS attack against Estonia had been a test for using cyber-attacks in upcoming conventional wars? Blocking or disturbing the whole or even most of the critical means of communication systems of a country might by a good preparation for a “conventional” attack. o Ukraine: “Russia was behind a devastating cyber-attack on Ukraine’s banks, government and power grid, the Foreign Office has said. .. Cyber experts at the National Cyber Security Centre say the GRU Russian military intelligence agency was almost certainly responsible for the NotPetya attack in June 2017. .. The Foreign

8 It is important to note here that the “tor browser” is the necessary tool to get access to the so called “dark net” as well as the alleged “deep web” (Darknet 2018). 9 Aka: also known as. 8

Office last night said the attack had masqueraded as criminal ransomware, but had in fact been a deliberate attack on the Ukrainian state.” (Telegraph 2018) Sean Townsend, Spokesmen of the Ukrainian Cyber Alliance is speaking frankly: “The Ukrainian Cyber Alliance is engaged not only in protecting national data resources but also in attacking the enemy’s network”. (Townsend 2018: 26) Moreover, Mr. Townsend refers that the “cyber-defense” strategy of the Ukraine and all other government and administrative bodies, more or less, does not exist. So, was the incident of NotPetya primarily and foremost a hostile attack or the result of the Ukrainian inability to defend itself? However, the possibility that a cyber-attack turns against itself (as a kind of friendly fire) seems not to be impossible; especially if the defense is weak. o Georgia: Alongside the Russian-Georgian War in 2008, alleged (!) cyber-attacks on Georgia happened: “The (alleged) Russian attack upon Georgia's military and government networks was highly successful. It seems that 54 web sites in Georgia related to communications, finance, and the government were attacked by rogue elements within Russia..” (Small Wars Journal 2011). Like in the Ukraine, even critical infrastructure IT systems (communication, finance and government), in Georgia appeared to be “exceptionally vulnerable”. However, it was concluded, it might have been the first case in history of a coordinated cyberspace domain attack synchronized with major combat actions in the other warfighting domains (consisting of Land, Air, Sea, and Space). “…three weeks before the shooting war between Georgia and Russia began, online attackers started assaulting Georgia's websites. Since then, researchers have tried to find out who masterminded the network strikes - military electronic warriors, patriotic hackers, cyber-crooks - without finding anything definitive." (Small Wars Journal 2011)10. In 2011, the governmental CERT (“Computer Emergency Response Team”) of the Republic of Georgia announced the discovery of a cyber-attack incident, which seems to be cyber-espionage (“Malicious Software was Collecting Sensitive, Confidential Information about Georgian and American Security Documents”). (DEA; undated). Again, the attack linked to “Russian Official Security Agencies”. Again, the accusation sounds plausible, but there is no evidence allowing a 100% proof. Even if attacks had been investigated, for example by the “Data Exchange Agency” (Ministry of Justice of Georgia) in 2008 (DEA 2011) the delivered evidence is not reliable and convincing. Russia might be responsible for the cyber-attacks against Georgia (Ukraine and Estonia and others) but, there is no firm evidence. It is just as conceivable, attacks of Georgian (or Ukrainian, Estonian etc.) hackers against Russia gone wrong and ended as “friendly fire”, because of the insufficient security measures in the country itself.

A.4 Criminals – the State – and Unconfirmed Information The foremost problem of accusations about state or even any responsibility for cyber-attacks is that evidence is “almost certain”. “Almost certain” means that there is no proof or evidence is subject to intelligence and hence, publication prohibited. However, this kind of evidence is point- as well as

10 Original quote: Noah Shachtman “Top Georgian Official: Moscow Cyber Attacked Us – We Just Can‟t Prove It” Wired Magazine, (11 March 2009) found at: http://www.wired.com/dangerroom/2009/03/georgia-blames/ 9 useless. Especially in the intelligence business there is always “almost certain” information. We have proof, but delivering and verifying it would endanger our sources. Everything intelligence sources deliver can be true or false.

The Website “turbofuture” (2018) delivers some interesting (unconfirmed!) insights about hackers in general and “the ten most powerful hacker groups”:

 Hackers in general: Hackers are a diverse group o There are young, prank hackers like Lizard Squad. Lizard Squad infamously used a DDOS attack to take down Playstation and Xbox networks during Christmas. o Some hackers are older and often work as "computer security consultants" who advise companies on how to protect themselves. o Some hackers are in it for monetary gain, while others are in it for the lulz and the power. o More and more there are state-sponsored groups that have Hollywood-style capabilities. The best known example of this is Stuxnet. Since a virus like Stuxnet is much harder to pull off than a DDOS attack, most of the groups on this (upcoming) list are state-sponsored? (Unconfirmed)

 “Top 10 Hacker Groups” o Bureau 121 – North Korea “According to defectors, military hackers live extravagant lives in North Korea. Top students are handpicked from straight out of their "University of Automation" school. The primary wing of this hacking group is known as Bureau 121. It comprises about 1,800 people that work around the world (because internet infrastructure in NK is pretty terrible).” o Chaos Computer Club (CCC) – Germany Chaos Computer Club (CCC) is probably only one of two groups on this list with any sort of moral code. It also is probably the oldest. CCC has made a number of hacks where they first consulted legal experts to make sure that what they were doing was legal. Although they almost permanently reside in or around legal grey-area, this willingness to operate within legal bounds has allowed their survival. Not only have they survived, but they've been accepted, recognized, and sometimes glorified by the press. Since it is a large disorganized association of people with exceptional computer security technical knowledge, not everyone has always behaved according to law. o Morpho Morpho, a.k.a. Wild Neutron, is a well-funded group that has executed dozens of high profile hacks since 2011 on tech, pharmaceutical, and investment companies. They're likely not state-sponsored because their hacks usually steal insider information for monetary gain. Morpho is particularly interesting because they are likely a sophisticated small group. Some of their signatures include multi-platform malware, well-documented code, bitcoins to pay hosting providers, and multi-staged command and control networks with encrypted virtual machines. They are English-speaking and are very good at covering their tracks.

10 o Syrian Electronic Army The Syrian Electronic Army (SEA) is a hacker group with Syrian sympathies as well as connections to Iran and Hezbollah. They've show a wide array of attack capabilities. The SEA is unique because of its varied tone and style. For example, it tweeted from AP's account that Obama had been injured in explosions at the White House. This one simple tweet sparked a dramatic temporary fall in the DOW Jones Index. On the lighter side, they've tweeted from BBC Weather that "Saudi weather station down due to head on-collision with camel". Their familiarity with English colloquialism and humor raises questions about the SEA's identity, but the NYT has stated that the SEA is probably Iranian. o Anonymous Anonymous is probably the most recognizable hacker group to at least the American general public.. Most of their hacks historically have been of the liberal hacktivist variety, although others have been extremely serious or extremely light-hearted in nature. Anonymous is an idea, and it is an idea with unprecedented staying power. o Tarh Andishan/Ajax Understandably, Iran was not pleased with Stuxnet. Iran decided it was best to aggressively upgrade their cyber capabilities. They did this in at least two ways: create an independent state-sponsored group, Tarh Andishan, and consult and hire existing Iranian hackivist groups (like Ajax). Ajax was better known for website defacement, but after Stuxnet it's likely they were consulted for patriotic espionage (pioneered by the Chinese). Ajax is most famous for "Operation Saffron Rose" in which they attempted to gain information on U.S. defense industry officials with advanced phishing attacks. o Dragonfly Another likely state-sponsored group, this time out of Eastern Europe and Russia, is Dragonfly. Dragonfly is likely state-sponsored due to its targets: electric grids, energy industry, and other control systems in U.S. and Europe. They're designated as an APT (Advanced Persistent Threat). Their most common attacks are spear-phishing and watering hole attacks. This is not unusual for APT groups. They've also demonstrated capabilities to embed trojans in legitimate software for industrial control systems. This is very reminiscent Stuxnet. o APT28 They're Russian, and might share funding sources with Dragonfly (although I don't know so I didn't group them together). All of their targets are targets that the Russian government is interested in, they speak Russian, and they've been traced back to a government sponsor in Moscow. APT28 uses pretty well known hacking methods, and uses them successfully and often. They've hacked NATO, Polish government websites, Georgia ministries, and OSCE. They're unique in that they've been caught framing the Cyber Caliphate (ISIS) for their attacks. Just like other organizations on this list, they operate in areas with no extradition treaty to the U.S. - so they are immune to legal repercussions. o Elderwood Group and 20 other Chinese APTs Elderwood Group, Axiom, Unit 61398, Comment Crew, Putter Panda, Hidden Lynx, and many more. China pioneered the state-sponsored hacking group, and they've

continued to perfect the practice. Often it is difficult to tell whether the Chinese government is pulling the strings, funding, or even has affiliation with a group.

To sum-up, reports and stories about cyber-attacks are mostly not based upon hard facts, but speculations.

British newspaper “The Guardian” (2018b) cites the head of the UK’s National Cyber Security Centre (NCSC) Cirian Martin in January 2018 that a major cyber-attack on the UK is not a matter of “if” but “when”. His main concerns are about disastrous disruptions of critical infrastructure, e.g. energy supply and the financial sector, and elections (a so-called C1 attack). There is a growing threat and the greatest risk of attacks allegedly (!) comes in particular from Russia, “both on the battlefield and on civilian services”. Instead of proven evidence, the article only offers speculation: “Most comparable western countries have experienced what we would consider a category one attack so we have been fortunate in avoiding that to date,..”. However, the article presented no examples, besides “the hacking of the US Democratic party in the run-up to the 2016 White House election and an attack on a French television station in 2015, both blamed on Russia”. (However, are these C1 attacks, which risk lives or destroy the stability of a political system?). But it is not only Russia to blame: “What we have seen over the past year or so is a shift in North Korean attack motivation from what you might call statecraft – disrupting infrastructure – through to trying to get money through attacks on banks but also the deployment of ransomware, albeit in a way that didn’t pan out in the way the attackers wanted to.” Moreover, China and Iran are additional suspects, initially for cyber espionage. Who knows? Maybe espionage is only the preparation for attacks against critical infrastructure. Everything in subjunctive, the threat as well as the attacker.

Another UK Newspaper, “The Independent” (2018), came up with a similar story. The alarming message is, the Kremlin did not only attack Ukraine but that there is “new era of warfare” against Britain. Although the Kremlin denies being behind attacks like NonPetya, “Gavin Williamson said the UK and its allies must be “primed and ready” to tackle intensifying online threats to energy, infrastructure, finance and public services. … "Russia is ripping up the rule book by undermining democracy, wrecking livelihoods by targeting critical infrastructure, and weaponising information.”” (ibid). Furthermore, the article accuses masquerading state (military and/or intelligence) action as criminal enterprises. Russia’s neighboring countries (like Ukraine, Georgia and the Baltic States) strongly suggest the Kremlin as the originator of cyber-attacks. Keir Giles, an expert on Russian security at Chatham House concludes: “the country’s [Russia] actions were based on a “permanent mindset of conflict” with the West.” (ibid). In the following, the article accuses Russian-linked hackers known as the “sandworm-group” allegedly behind NonPetya and numerous other cyber- attacks against Ukraine, based upon information of the cyber security firm “FireEye”. (Is there anything like a comprehensive list of all the alleged Russian or Russian linked hacker groups?) However, the presented evidence to support that Russians or the Russian government is behind the attacks is extremely poor. More or less the article says, NonPetya has its base in earlier malware but ransomware elements had been included. “These prior attacks share features, including distribution through a compromised software provider and a wiper masquerading as ransomware, with the June 2017 Petya attack supporting the case of a link between Sandworm and Petya.” (ibid). There is no proof for the link between sandworm and Petya nor is there any evidence given that sandworm is a Russian state activity. “Russia denied responsibility for the NotPetya attack, pointing out that Russian

12 firms were among those whose systems were affected. … “We categorically dismiss such accusations - we consider them unsubstantiated and groundless,” said Kremlin spokesman Dmitry Peskov.” (ibid).

CNN reported on March 18 2018, the Department of Homeland Security (DHS) blames Russia for multi-stage power grid cyberattacks. (CNN 2018) The attack initially targeted less shielded small commercial third-party networks with malware. “Russia has attempted to attack targets that include "energy, nuclear, commercial facilities, water, aviation, and critical manufacturing sectors" since March 2016” (ibid). Was the attack successful? “Gaining access to the networks that are tied to various aspects of US infrastructure is extremely difficult, said Vikram Thakur, of Symantec Security Response. … "Usually the bar for flipping the switch is extremely high" for the attacker” (ibid).

In February 2018, the detection of a hacking attack against the German government IT-network was leaked. Seemingly, the attack lasted many months (years?) but security authorities assured little or no damage at all damage was incurred. Head of the German Federal Intelligence Agency Hans-Georg Maaßen declared that Russia was behind the attack. As is customary in these cases, Maaßen concludes, a 100% proof of the perpetrator is not possible. It cannot be ruled out, attackers only planted evidence to blame Russia as the instigator of the attack. (ZEIT 2018).

A.5 Summing-Up: Cyber-Piracy - A Frame of Reference In conclusion, the field of cyber-attacks and cyber-security is characterized predominantly by the following patterns:

1. There is little empirical and assured knowledge on the subject. There is a lot of speculation and insinuations, especially against countries not being “close friends” 2. The existence of the field is possible due to the existence of a legal vacuum together with a space not or only hardly controllable by state forces (netzpolitik 2017). However, freedom always raises the question “cui bono”, who is benefiting? If there is no state control, usually other instances from the civil society like the mafia fulfill the task. Furthermore, when it comes to intelligence, all states prefer areas with a legal vacuum and blaming other states for doing the same thing 3. Linkage between criminal actions / actors and the state / official bodies. Using criminals (hacker groups) as mercenary soldiers or contracting public private partnerships (PPP). Problem: Who finally defines the “rule of law”, the criminals or the state? 4. Romantic glorification of the free and lawless live of hackers compared to the bourgeois-live of the ordinary people

All that is included in cyber-attacks (preceding bullet point 1.-4.) is well known from the beginning of mankind, and it has a name: Piracy.

Hence, we have to check the nature and character of piracy in the following chapter in order to try to understand how attackers will behave and subsequently how to protect.

B Offense and Defense “Although offense and defense are opposites, offensive and defensive aren't always. Defensive can mean anxiously challenging all criticism. Offensive can mean not just attacking someone or something, but belching, insulting people, or otherwise not respecting common standards of behavior.” (Vocabulary 2018). 13

B.1 Attack vectors: Technical and Human Failure The four key areas of interest for attackers are denial of service (DOS), theft, to wreak havoc and finally to use the attacked system as a platform for further attacks. As detailed in part one of this report, the intentions for these attacks vary widely, ranging from leisure activities over conventional crime to state sponsored terrorism. There are commonalities in the execution of all these attacks, in the following section we take a look at the attack vectors of target systems and discuss possibilities to avoid some attacks.

It has to be stated that there is no perfectly secure IT system. The absence of errors in an IT system cannot be proven. Whilst testing can show the presence of errors, their absence cannot be assumed as a consequence of no errors being found (Dijkstra 1969). Wherever IT systems are used, one has to expect that the systems can misbehave. To further this, the perceived misbehavior might be intended by the developer. To establish a taxonomy of attack vectors with a focus on public administration we stay clear of technological subtleties as hardware and software defects but instead take a more practice oriented approach. All four listed attack vectors share properties in regards to criticality and commonness, but the mitigation concepts vary drastically. Server parks and IT infrastructure in public administration tend to run much longer than in commercial environments. Whilst this has obvious implications for metrics as reliability, there are even more dramatic consequences for IT security. Starting with outdated and unpatched operating systems over vulnerable application software on servers to no longer supported infrastructure systems as firewalls or other security appliances. Personal computers and laptops are affected by the same issues as servers and infrastructure, but as the common interface to staff bear additional risks. Amongst other risks, unrestricted internet access and the use of USB devices comes to mind. In the private sector it would be unthinkable within larger organizations to find working USB slots on desktop computers or laptops, they are either disabled or intentionally dysfunctional. Likewise, common measures to restrict internet access in companies are rarely seen in German public administration. We conducted a relatively small survey to validate these claims and whilst the sample size remains small, the unanimous results give some credibility to these observations. A relatively recent trend is to bring your own device(s) (BYOD) to work. Whilst this might not be seen as undesirable in times of reduced spending within public administration, from a security perspective it another key issue. The official equipment is in most cases managed by a central IT department and usually common security practices are in place. These might not be deemed industry standard but typically include at least a permission management and where possible regular security updates. The own devices which cover a wide range of portable devices11 are outside any security scope. IT departments are often unaware of their presence, their capability and usage. Even relative plain malware might be able to spread through a network when an infected device gets connected to the infrastructure. A common de- nominator in the so far introduced attack vectors are a lack of funding and humans. Human operators run outdated infrastructure despite better knowledge, employees are trying to check the content of an USB memory stick they found on their way to work, employees connect their smartphones to their office computers to transfer photos. The USB device might have been placed where it was found by adversaries with the intent to place malware on a workstation, the smartphone might be infected with malware waiting to get transferred to a computer. Physical attacks can be very effective in the context of denial of service, but furthermore, physical break-ins can be the prelude for cyber-attacks. Rather than stealing material which would usually attract

11 Smartphones, tablets, laptops and even WiFi range extenders 14 attention, equipment is planted by the attackers. This can be an embedded system connected to unmonitored network ports or even network equipment which is exchanged for rigged systems. With able detractors, the professionalism can reach levels where serial numbers of exchanged equipment are matching.

B.2 Internet of Things: Leaving the Backdoor Open Infrastructure, independent of the particular private or public ownership, can be an attractive target mainly for denial of service or chaos attacks. Criminal theft might be considered as well, but should be seen here in the context of the theft causing disruption and damage to the infrastructure making it effectively a DOS or chaos attack. The internet of things is a not any more so recent tendency to connect sensors, actuators and entire control systems to the internet. What seems to be harmless if maybe unnecessary for a household central heating, a fridge or CCTV can become dangerous if applied to public infrastructure. A typical small electricity producing facility might be a combined household heat and power unit or an on the roof photovoltaic installation with an output of some KW, a medium facility a larger solar or wind power station with an output of some MW and a large facility a typical power station (water, gas, oil, coal or nuclear) on the national grid. In the example of power generation these facilities are already interconnected by the national and international grid and as such can communicate indirectly. A shortage of supply will be indicated by a subtle drop in the network AC frequency, an oversupply will cause the frequency to rise above the standard 50Hz. Subsequently the generation of power will be adjusted accordingly. From a naive perspective, no further communication is necessary to accommodate the basic functionality of the grid. However, the internet of things allows more. Stakeholders of wind-parks can monitor the function and output of their installation, maintenance firms can detect and predict failures and act accordingly whilst owners might adjust the power output to react to spot market prices for electricity. These possibilities require access to the relevant systems and in many cases this access is realized through public networks. Public networks allow theoretical access for anyone, in practice this means that unauthorized access is limited to skilled and financially enabled adversaries in the best case and almost not limited at all in the worst case.

An ubiquitous issue are the traditional life spans of investment goods. Whilst some power stations or industrial system run for several decades, IT equipment has a practical life expectancy of a fraction of that. One might believe the Internet of Things (or Industry 4.0) where the traditional concepts of industry and IT are merging has to undergo changes to accommodate the differing life cycles. In practice there are indications that the longer lifespan of the traditional components sets the benchmark and in turn the IT components age without security relevant updates from a certain point in time. This is not just negligence from the side of the operators, but the consequence of unavailability once the product reached a certain age. Many million IoT-devices become obsolete from a security point of view every year on the basis that their (often Linux-based) operating system is not updated any longer by the vendor. Not strictly to be seen in the IoT-context but of security relevance are millions of home and SMB routers which typically receive vendor support for two years and are operated for many years after that.

It needs to be stressed that standardized safety integrity levels (SIL) take common operation into consideration. The probability of failure per hour for a continuously running system12 defined in IEC EN 61508 as SIL4 is 108 − 109. In many cases this is implemented by multiple, independent and

12 E.g. a train control system. 15 redundant systems. However and here comes the caveat, sabotage is not part of this consideration. The reliability of multiple sensors is calculated by likelihoods for failure, not considering the possibility that someone feeds malicious values into the system. The latter becomes even more problematic if the attacker has knowledge of the implementation of the SIL measures. Wrong data can be fabricated in ways to bypass certain fail-safe mechanisms. From power stations over train control systems to various industrial systems, SIL4 specified systems are very robust during normal operation. In case of sabotage or (unexpected) natural disasters13 SIL are not making any statement towards robustness.

B.3 Defense: Mitigation is Good - Avoidance is Better In the following sections, we are going to introduce and explain some concepts to prevent and mitigate attacks. Depending on the environments, some of these concepts might already be implemented and some might be hard to adapt to specific requirements. We are not giving specific technical advice or administrative guidelines, but highlight and outline the areas which should be considered.

Conclusive defense of IT systems is not possible, but there are guidelines which make attacks less likely and if they happen less successful. The foremost consideration should be if a system is required and if so, if full connectivity cannot be avoided. Attackers of all the sorts discussed in this paper will act under certain economic or HR constraints, aiming for maximized results with minimized effort. This makes one centralized large database a far more rewarding target than multiple smaller ones. If these databases are following a multiple concept approach, not a one breach, breaches all, there is a substantial gain in security.

B.4 Open Source – Documentation, strategy and emergency planning - Backups Open source software is not inherently safer than closed source software, the constant community checks of the source code is an argument contradicted by prominent examples as “heartbleed” (Limer 2014), where the insecure function was (possibly deliberately) added to the package and nobody realized this. However, open source systems reduce the dependence upon a vendor, a party introducing back-doors into open source has a higher risk of being exposed and open source leads to more diverse and heterogeneous environments. Closed source monocultures bear similar risks as their biological equivalents, even if hardened to withstand certain threats, there is a higher likelihood that a certain problem will affect the entire organization. Cryptographic systems bear many hazards during implementation, even well designed and established cryptographic systems can be made very vulnerable by poor implementation. In a closed source system the quality is entirely in the hand of the vendor. Given that the majority of strong cryptographic algorithms is public, published in academic papers, the temptation for closed source vendors is big to monetize them without full understanding which can lead to faulty implementations.

The bad practice of security by obscurity is also a substantially smaller issue for open source than closed source systems. An inherently secure system should be secure by design, not as a result of an adversary not knowing internals. Once these internals become public, and experience shows that typically at some stage they will, the security falls apart. The widely used public key cryptography is a

13 E.g.the Fukushima Daiichi nuclear disaster. 16 good example where the security level is solely dependent on the key length and not on the obscurity of the underlying algorithms.

A well-documented IT environment is paramount to re-establish functionality in a secure and timely manner in cases of malfunction. There is a wide range of causes for malfunctions, starting from simple hard- or software defects, over staff shortages to natural disasters and sabotage. All of these malfunctions have in common that the appropriate counter measures are easier, better and more swiftly applied if a risk assessment accompanied the planning and possible failure scenarios and the subsequent responses are documented.

Conventional backups are obviously mandatory in any IT environment. As such, they are not a defense mechanism although if done fine grained and long lasting, they might assist in forensic examinations to establish the time a system was compromised. As a mitigating approach and in defense of denial of service attacks alternative backup systems are of importance. The more the alternative backup system differs from the original system, the higher the likelihood that a DOS attack will come to an end after the system switch.

B.5 Staff: Encryption – Physical Access An attacker will usually look for the weakest link in a system and often, this is not technology. Many attacks are driven by social engineering, trying to extract passwords directly from people within the attacked organization. Whilst approaches like bribery or blackmail might be hard to counter, very often the breaking points are less sophisticated. These could be emails containing malware or phone calls asking for passwords. Even at a low entry point of the organization it could be enough for an attacker to get the needed access from inside the organization. This can only be countered by the hiring of high quality staff and especially where the former is impossible in breadth, by constant training and raising of awareness.

A lesser discussed 'cyber' attack is old-school theft and recycling negligence. Most vulnerable to theft are obviously handheld devices and laptops, but also desktops and servers can be stolen. These devices can then either provide an entry point into the organizations systems or the device itself can contain the desired information. Recycling negligence affects all device categories, especially including servers and infrastructure equipment. Decommissioned systems often become available through various legal and illegal channels. It would be desirable to prevent this, but realistically this goal is not likely to be fully achieved. As a precaution systems should be encrypted and protected. If the protection is achieved by passwords, smartcards or biometric sensors are a secondary concern as long as the respective quality standards are taken into consideration. As a result, off-line attacks have a limited impact.

Physical intrusion is another starting point for many attacks, here the attacker gains access to the premises not with the intention to remove/steal items, but with the intend to leave a device allowing access to internal network at a later stage. These devices can be well hidden in cable conduits, they can replace similar infrastructural equipment in obvious but inconspicuous places and so forth. Careful planning, protective measures and event logging are methods to prevent these direct attacks.

In large IT deployments, systems should be encapsulated as far as possible and with increasing sensitivity of stored data, there should be increased fencing (Onion-Principle). This is obvious for the hardware side of the infrastructure, but should be especially considered with regards to permissions. The concept should always follow the principle of data avoidance and data minimization, in analogy 17 these should be applied in the context of permission management. Only what is absolutely necessary should be stored or be accessible.

B.6 Honeypots and Oracles Whilst not strictly a defense mechanism, honeypots serve this purpose to a certain degree as well by occupying attackers and binding their resources. A honeypot is supposed to draw the attackers attention to the system whilst neither containing sensible, valuable information nor allowing the attacker to infiltrate the network any further. Apart from occupying the attacker, honeypots can serve more objectives. In an ideal scenario, the attacker might not become aware of the nature of the system and leave in the belief to have achieved the mission targets. Depending on the quality of the honeypot and the abilities of the adversary, often the honeypot will be debunked for what it is. Even in these cases there will be gains for the defending side mainly by being able to monitor the system closely and thus learning about the strategy of the attackers, their goals and in some cases even their origins. Where the honeypot wasn't revealed, there is even the possibility to become an oracle. Here the attackers might reveal themselves by attempting to use information which they can have only gained whilst compromising the honeypot. We will later discuss the attributablity of attacks and the possibility of predominant opponents who might give the misapprehension of falling for a honeypot.

B.7 Best Practice A common cause for insecure systems is not so much the lack of knowledge how to implement a more secure system, but often limited budgets in combination with demands for inherently insecure features to provide a more convenient user experience.

There are examples however which can be taken as best practice and in most states around the globe, the flight traffic control authorities4 appear to take this role. There is an obvious high demand for IT-based infrastructure combined with requirements for robustness, reliability and resilience. To postulate a general rule, if the guidelines of air traffic control services are followed, most IT infrastructures would be considerably less vulnerable than before. The predominant principle followed within these guidelines is safety over convenience.

C To Sum-Up According to the German football coach Jupp Heynkes, the offense is are winning games, the defense is winning championships (goalimpact.com 2013). As in the vastness of the internet it is nearly impossible to take cyber-pirates into custody, defense seems to be the method of choice.

C.1 Parties in the Background The World Economic Forum's (WEF 2018) 'The Global Risks Report 'considers 'Cyber Attacks', strictly distinct from 'Data fraud or theft' and 'Terrorism', as high likelihood and high impact events endangering the state of the world. The likelihood of disruptive cyber-attacks is ranked third, with the likelihood of data fraud or theft in fourth position. Only natural disasters and extreme weather events are seen as more likely disruptive events to the world economy. In comparison 'Asset bubbles in a major economy' are only ranked in tenth place. The intensity of the impact of cyber-attacks is seen as the sixth most severe, directly behind water crises and ahead of food crises. In short, cyber issues are seen as a major threat to the world economy and awareness of this seems extended. Not all companies in the cyber security area are public, but the following table 1 (also, see figure 4) gives 18 an overview of some large public cyber security companies, their last reported revenues and the percentage of revenue spent on sales & marketing. This is obviously strictly differentiated from research & development spending.

Figure 6: Risk Assessment Graph (Likelihood and Impact of Different Attack Strategies), The Global Risks Report 2018 (World Economic Forum)

company revenue (million USD) sales & marketing Symantec 4,019 36.3% Palo Alto 1,761 52.2% Checkpoint 1,741 24.1% Fortinet 1,275 49.1% FireEye 714 61.5% Proofpoint 375 53.6% Table 1: Marketing spendings of selected public cyber security firms.

It has to be noted that, due to the venture capital nature of some of these companies, some reported a loss and hence the cost of revenue including operating expenses exceeded the total revenue. Still, FireEye for example reported a R&D to S&M ratio of 0.63, for every 63 Cent spent on research and development, 1 Dollar was spent on marketing. One might consider that it is hard to define definitive metrics for security, that threats are relatively abstract and that the main players in the market consider it paramount to create a certain atmosphere of fear. Thus creating a market could be considered easier than to create measurable security by investing in R&D. This does not mean that threats are non-existent, but often it might be beneficial to analyze threats in more detail, considering risk avoidance rather than risk mitigation. It is safer to run a system with the most recent system vendor released patches than to try to achieve security by third party products. A well maintained car is certainly considered to run safer than a less well maintained one with an additional anchor chain attached to it.

C.2 Lessons Learned: Piracy is a Commercial Business, sometimes used by States to Attack Enemies, but rather Unsuitable for Terrorists To “classical” terrorists cyber-attacks appear to be not attractive. It is not so much about technical skills and organizational efforts. As pirate (as false-flag) actions need to be “anonymous”, cyber- attacks cannot be claimed by a certain terrorist group. Therefore, “classical” attacks with firearms, explosives or easy to do attacks with SUVs or trucks seem more suitable to terrorists. On the other hand, cyber-crime like hijacking data sources or announcing DOS-attacks might be an attractive way for terrorists financing their activities. (The conjunction of terrorism and pure crime is nothing new. In Germany, the so-called “Red Army Fraction” (RAF) from the beginning financed political murder and terror attacks with thievery (zeitklicks 2013). Hence, the “RAF-Oldtimers”, not covered by the state pension insurance, apparently are still in the same business (murder and thievery). (Faz.Net 2016).)

However, from the point of view of piracy the main threat is that states use cyber-attacks and preferably cyber-pirates as unofficial partisan troops. Cyber-attacks, with the help of partisan pirate troops are possible in three ways:

1. Assistance for “regular troops” to support warfare activities. Mostly, the attacking country / government / force attempt to destroy (critical) infrastructure, like energy supply, traffic and communication. (See chapter A.3 for the examples of Georgia and Ukraine). However, concerning cyber-attacks, it is rather hard to figure out who was the attacker, because attacks or counter-attacks might have been gone wrong. Cyber-attacks or counter-attacks have a high probability to come back to the attacker due to the vastness and uncontrollability of the internet. 2. To attack critical infrastructure by cyber-attacks, there are two main options: a. The defense system is weak, like it is reported e.g. for Ukraine (chapter A.3) b. Infiltration of spies into the system (espionage) 3. Destabilization of the political and social system of the enemy. The enemy, seemingly uses the internet to destabilize the western society, by affecting the most important means of the free world: Elections. a. In chapter A.2 we already mentioned the “Macron Campaign Hack”. According to the former head of FBI James Comey Russia wanted to influence the U.S. presidential election in 2016. Again there is no hard evidence, neither that Russians tried to

attack the election nor the Russian government was involved. (Heise 2017). However, violating the trust in the orderliness and fairness of elections is a very promising way to destabilize a western society. b. Moreover, any kind of disturbance of the financial sector, communication and the media, traffic, health care and many other areas might help to weaken a country. Hence, it is not impossible that the support of hackers is part of the intelligence activities of a government.

C.3 Outlook However, the lack of hard evidence about attacks and attackers together with the high impact and likelihood of cyber-attacks motivates us to develop this paper further. What we already could figure out is that the theory and practice of piracy, e.g. “false flag operations”, is a good frame of reference to describe and analyze cyber-attacks. To be continued…

References

Print Dijkstra, E., 1969, Software engineering techniques, in: Report on a conference sponsored by the NATO Science Committee, Rome, Italy, 27th to 31st October 1969 (R. B. Buxton, J.N., ed.)

Townsend (2018), Townsend, S., When there are to many cooks. Prospects for Ukraine’s cyberspace and recommendations for those who want to protect it, in: The Ukrainian Week, #12 (118), December 2017

Online (all accessed from January until May 2018) CERT.GOV.GE (undated), http://dea.gov.ge/uploads/CERT%20DOCS/Cyber%20Espionage.pdf

CNBC (2017), https://www.cnbc.com/2017/09/20/cyberattacks-are-surging-and-more-data-records- are-stolen.html

CNN (2018), https://edition.cnn.com/2018/03/15/politics/dhs-fbi-russia-power-grid/index.html

Darknet ( 2018), https://www.comparitech.com/blog/vpn-privacy/how-to-access-the-deep-web-and- darknet/

DEA (CERT.GOV.GE , undated), http://dea.gov.ge/uploads/CERT%20DOCS/Cyber%20Espionage.pdf

DEA (2011), http://dea.gov.ge/uploads/GITI%202011/GITI2011_3.pdf

Faz.Net (2016), http://www.faz.net/aktuell/politik/inland/ex-raf-mitglieder-kehren-mit- geldtransporter-ueberfall-zurueck-14022955.html

FireEye (2018), https://www.fireeye.com/

Gemalto (Google Search 2018), https://www.google.de/search?q=Gemalto&rlz=1C1SVED_enDE441DE447&oq=Gemalto&aqs=chrom e..69i57j0l5.5731j0j7&sourceid=chrome&ie=UTF-8 goalimpact.com (2013), http://www.goalimpact.com/blog//2013/02/der-sturm-gewinnt-spiele-die- abwehr.html

Guardian (2018a), https://www.theguardian.com/technology/2018/feb/26/north-korea-cyber- attack-threat-russia

Guardian (2018b), https://www.theguardian.com/technology/2018/jan/22/cyber-attack-on-uk- matter-of-when-not-if-says-security-chief-ciaran-martin hackmageddon.com (2018), https://www.hackmageddon.com/category/security/cyber-attacks- statistics/

Heise (20217), https://www.heise.de/newsticker/meldung/Ex-FBI-Chef-Russland-wollte-mit-Hacks- die-US-Wahl-beeinflussen-3739216.html

IBM (2018), (https://www.ibm.com/security/data-breach/threat-intelligence) 22

Independend (2018), https://www.independent.co.uk/news/uk/home-news/russia-cyber-attacks- notpetya-gavin-williamson-defence-secretary-putin-hacking-ransomware-a8212801.html

INFOSEC Institute (2013), http://resources.infosecinstitute.com/estonia-to-black-out-an-entire- country-part-one/#gref

Jerusalem Post (2018), http://www.jpost.com/Arab-Israeli-Conflict/Major-Israeli-websites-targeted- in-large-anti-Israel-cyberattack-547834 lifewire (2018), https://www.lifewire.com/stuxnet-worm-computer-virus-153570

Limer, E. (2014), How heartbleed works: The code behind the internet's security nightmare, found at: https://gizmodo.com/how-heartbleed-works-the-code-behind-the-internets-se-15611341209 netzpolitik (2017), https://netzpolitik.org/2017/freedom-on-the-net-report-2017-freiheit-im- internet-verschlechtert-sich-weiter/.

New York Times (2017), https://www.nytimes.com/2017/10/15/world/asia/north-korea-hacking- cyber-sony.html

Small Wars Journal (2011), http://smallwarsjournal.com/blog/journal/docs-temp/639-hollis.pdf

Symantec (2018), https://www.symantec.com/

Stanford.edu (2015), http://large.stanford.edu/courses/2015/ph241/holloway1/

Telegraph (2018), https://www.telegraph.co.uk/news/2018/02/15/russia-behind-malicious-cyber- attack-ukraine-foreign-office/

TheBestVPN (2018) https://thebestvpn.com/cyber-security-statistics-2018/

TIA (2018), TIA’s 2010-2017 ICT Market Review and Forecast, found at: http://publications.atlanticcouncil.org/cyberrisks// torproject (2018), https://www.torproject.org/projects/torbrowser.html turbofuture (2018), https://turbofuture.com/internet/Most-Powerful-Active-Hacking-Groups.

Vocabulary (2018), https://www.vocabulary.com/dictionary/offensive

Wired (2009), http://www.wired.com/dangerroom/2009/03/georgia-blames/

Wired (2017), https://www.wired.com/story/2017-biggest-hacks-so-far/

Wired (2018), https://www.wired.com/story/us-kaspersky-ban-evidence/

World Economic Forum (WEF) (2018), http://reports.weforum.org/global-risks-2018/

ZEIT (2018), https://www.zeit.de/politik/2018-04/hackerangriff-bundesregierung-russland- verfassungsschutz-hans-georg-maassen zeitklicks (2013), http://www.zeitklicks.de/top- menu/zeitstrahl/navigation/topnav/jahr/1970/bankueberfaelle-erste-taten-der-raf/ 23