Gregory Pope, CSQE

LLNL-PRES-722772 This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC Source: Cisco (Smart Cities)

The Internet of things (IoT) is the internetworking of physical devices, vehicles (also referred to as "connected devices" and "smart devices"), buildings and other items—embedded with electronics, software, sensors, actuators, and network connectivity that enable these objects to collect and exchange data.[1][2][3] 1 Brown, Eric (13 September 2016). "Who Needs the Internet of Things?". Linux.com. Retrieved 23 October 2016. 2 Eric (20 September 2016). "21 Open Source Projects for IoT". Linux.com. Retrieved 23 October 2016. 3 "Internet of Things Global Standards Initiative". ITU. Retrieved 26 June 2015

2 Lawrence Livermore National Laboratory LLNL-PRES-722772 3 Lawrence Livermore National Laboratory LLNL-PRES-722772 The Internet Of Things Heat Map, 2016 Where IoT Will Have The Biggest Impact On Digital Business by Michele Pelino and Frank E. Gillett January 14, 2016

4 Lawrence Livermore National Laboratory LLNL-PRES-722772 “Internet of Things” security is hilariously broken and getting worse

Shodan search engine is only the latest reminder of why we need to fix IoT security.1

J.M. Porup

1 J. M. Porup, https://arstechnica.com/security/2016/01/how-to-search-the-internet-of-things-for-photos-of-sleeping-babies/

5 Lawrence Livermore National Laboratory LLNL-PRES-722772 . During software development identify: • Hazards Accidents (Death, Injury) • Risks Loss (Time, Money, Trust) • Vulnerabilities Cyber Attack

6 Lawrence Livermore National Laboratory LLNL-PRES-722772 Root Cause Analysis (RCA or the Five Why’s) Developed by Sakichi Toyoda, RCA was first used during the development of Toyota’s manufacturing processes in 1958

https://chapters.theiia.org/pittsburgh/Events/Documents/Root%20Cause%20Analysis%20Presentation.ppt

7 Lawrence Livermore National Laboratory LLNL-PRES-722772 Failure Modes and Effect Analysis (FMEA) was developed by reliability engineers in the late 1950s to study problems that might arise from malfunctions of military systems.

8 Lawrence Livermore National Laboratory LLNL-PRES-722772 Fault Tree Analysis (FTA) was originally developed in 1962 at Bell Laboratories by H.A. Watson, under a U.S. Air Force Ballistics Systems Division contract to evaluate the Minuteman I Intercontinental Ballistic Missile (ICBM) Launch Control System.

Bill Vesely NASA HQ

9 Lawrence Livermore National Laboratory LLNL-PRES-722772 IBM 7094 32,768 words of memory 36 bit word Floating Point double precision 300MB Disk Cost: 2.9 Million Safety Hazard – Dropping the 25 inch disk onto your foot.

10 Lawrence Livermore National Laboratory LLNL-PRES-722772 100 Million lines of code

14 Million lines of code

12 Million lines of code

2 Billion lines of code ://wwhttpw.informationisbeautiful.net/visualizations/million-lines-of-code/

11 Lawrence Livermore National Laboratory LLNL-PRES-722772 One cannot rely solely on hazard analysis methods developed for hardware when performing hazard analysis for software.

Nancy Leveson, MIT Aeronautics and Aerospace Professor

1995 2011

12 Lawrence Livermore National Laboratory LLNL-PRES-722772 Hardware wears out

http://www.onlineclassnotes.com/2013/01/software-doesnt-wear- Software wears in out-explain-this.html

13 Lawrence Livermore National Laboratory LLNL-PRES-722772 1. Three Mile Island 2. Vincennes (Iran Air Flight 655) 3. Cali (American Airlines Flight 965) 4. Ariane 501

5. Mars Climate Orbiter (MCO) In no case did the software fail 6. Mars Polar Lander (MPL) to execute. Usual problem is requirements and interactions 7. Titan/Centaur/Milstar between components. 8. SOHO (Solar Heliospheric Observatory) 9. Therac 25 10. ATT Phone System 11.Kegworth British Midlands 12.Asiana Flight 214

14 Lawrence Livermore National Laboratory LLNL-PRES-722772 Step 1: Step 3: Identify Model Potential The Accidents System Step 5: Plan Hazard/ Risk Step 2: Step 4: Mitigation Identify Create Strategies Potential The Hazards/ Context Risks Tables

15 Lawrence Livermore National Laboratory LLNL-PRES-722772 High Level Requirements:

. Doors shall open at station when train is completely stopped and aligned with platform.

. Doors shall remain closed when train is moving.

. Doors shall remain open when someone is in the doorway.

. Doors shall open in an emergency.

16 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Doors shall remain open when someone is in the doorway.

. Doorway: A volume of space extending from the top edge of the door frame to the floor of the car and 6 inches to either side.

. Someone: A person or object that is detected within the doorway volume.

17 Lawrence Livermore National Laboratory LLNL-PRES-722772 . A1 - Passenger falls out of moving train . A2 - Passengers not able to escape train during emergency . A3 - Passenger steps off stopped train not at platform . A4 – Doors close on passenger in doorway

18 Lawrence Livermore National Laboratory LLNL-PRES-722772 . H-1 Doors open when the train is moving (A-1) . H-2 Doors close while person in the doorway (A-4) . H-3 Doors stuck closed during emergency (A-2) . H-4 Doors stuck open (A-1, A-3) . H-5 Doors open when not aligned to platform (A-3)

19 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Diagram the system as a control structure

Automated Control Model of Controller Algorithm Process

Actuator Sensor

Controlled Process

20 Lawrence Livermore National Laboratory LLNL-PRES-722772 Control Commands Status Other Inputs: -Train motion Commands: -Train position -Open door, stop opening door -Emergency Indicator -Close door, stop closing door Control Model of Algorithm Process Feedback: -Door position -Door clear

Door Door Actuator Sensor

Physical Door Mechanical Force Mechanical Position

21 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Identify Hazards that can cause accidents using guide phrases: 1. A safety control action is not provided or is not followed. 2. An unsafe control action is provided. 3. A safety control action is provided too late or out of sequence. 4. A safety control action is stopped too soon or applied too long.

22 Lawrence Livermore National Laboratory LLNL-PRES-722772 Control Train Train Hazard/ Action Motion Position Emer. Door State Risk Open not Person not provided Stopped Aligned No in doorway No Prefatory STPA Open not Person not context provided Stopped Not AlignedNo in doorway No tables Open not Person in provided Stopped Aligned No doorway Yes (H-2) Open not Person in provided Stopped Not AlignedNo doorway No Open not provided Stopped N/A Yes N/A Yes (H-3) Open not provided Moving N/A N/A N/A No

23 Lawrence Livermore National Laboratory LLNL-PRES-722772 Hazard/ Control Train Train Hazard/ Risk Hazard/ Action Motion Position Emer. Risk Any Early Risk Late Prefatory STPA Open Moving N/A No Yes (H-1) Yes (H-1) Yes (H-1) context tables Open Moving N/A Yes Yes (H-1) Yes (H-1) Yes (H-1)

Open Stopped N/A Yes No No Yes (H-3) Not Open Stopped Aligned No Yes (H-5) Yes (H-5) Yes (H-5)

Open Stopped Aligned No No No No

24 Lawrence Livermore National Laboratory LLNL-PRES-722772 Prefatory STPA Analysis of context tables

25 Lawrence Livermore National Laboratory LLNL-PRES-722772 Open ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidOpe Case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Stopped: T T T T T T T T T T T T T T T T F F F F F F F F F F F F F F F F

Aligned: T T T T T T T T F F F F F F F F T T T T T T T T F F F F F F F F

Emergency: F F F F T T T T F F F F T T T T F F F F T T T T F F F F T T T T

Doorway Obstructed F F T T F F T T F F T T F F T T F F T T F F T T F F T T F F T T

Doorway Closed T F T F T T T F T F T F T F T F T F T F T F T F T F T F T F T F

Hazard (Me) N N N N N N N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Hazard (Chart) N N N N N N N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Hazard (Equation) N N N N N N N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

Open Provided

26 Lawrence Livermore National Laboratory LLNL-PRES-722772 Open ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidOpe Case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 Open NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpen Provided NotOpe P Case 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 Stopped: T T T T T T T T T T T T T T T T F F F F F F F F F F F F F F F F Stopped: T T T T T T T T T T T T T T T T F F F F F F F F F F F F F F F F Aligned: T T T T T T T T F F F F F F F F T T T T T T T T F F F F F F F F Aligned: T T T T T T T T F F F F F F F F T T T T T T T T F F F F F F F F Emergency: F F F F T T T T F F F F T T T T F F F F T T T T F F F F T T T T Emergency: F F F F T T T T F F F F T T T T F F F F T T T T F F F F T T T T Doorway Obstructed F F T T F F T T F F T T F F T T F F T T F F T T F F T T F F T T Doorway Obstructed F F T T F F T T F F T T F F T T F F T T F F T T F F T T F F T T Doorway Closed T F T F T T T F T F T F T F T F T F T F T F T F T F T F T F T F Doorway Closed T F T F T F T F T F T F T F T F T F T F T F T F T F T F T F T F Hazard (Me) N N N N N N N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Hazard (Me) Y Y Y Y Y Y Y Y N N N N Y Y Y Y N N N N N N N N N N N N N N N N Hazard (Chart) N N N N N N N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Hazard (Chart) Y Y Y Y Y Y Y Y N N N N Y Y Y Y N N N N N N N N N N N N N N N N Hazard (Equation) N N N N N N N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Hazard (Equation) Y Y Y Y Y Y Y Y N N N N Y Y Y Y N N N N N N N N N N N N N N N N Open Provided Open Not Provided

Open ProvidedOpen Provided OpenToo Late ProvidedOpen Too or Out LateProvidedOpen Too of or Sequence LateProvidedOutOpen Too ofor LateSequenceProvidedOutOpen Too ofor LateSequenceProvidedOutOpen Too ofor LateSequenceProvidedOutOpen Too ofor LateSequenceProvidedOutOpen Too ofor LateSequenceProvidedOutOpen Too ofor ProvidedLateSequenceOpenOut Too ofor Provided OpenLateSequenceOut Too ofor ProvidedOpenLate SequenceOut Too orof ProvidedOpenLate OutSequence Too orof ProvidedOpenLate OutSequence Too orof ProvidedLate OpenOutSequence Too orof ProvidedLate OpenOutSequence Too orof LateProvided OpenOutSequence Too orof Provided LateOpenOutSequence Too ofor Provided OpenLateSequenceOut Too orof ProvidedOpenLate OutSequence Too orof ProvidedOpenLate OutSequence Too orof ProvidedOpenLate OutSequence Too orof ProvidedOpenLate OutSequence Too orof ProvidedOpenLate OutSequence Too orof ProvidedOpenLate OutSequence Too orof ProvidedOpenLate OutSequence Too orof ProvidedOpenLate OutSequence Too orof ProvidedOpenLate OutSequence Too orProvidedof OpenLate OutSequen Too orProof OpLat T OS Open StoppedOpen Stopped OpenToo Soon Stopped OpenToo or Soon AppliedStopped OpenToo or Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppliedStopped Too OpenToo or Long Soon AppStop Too OpTo Case 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128

Stopped: T T T T T T T T T T T T T T T T F F F F F F F F F F F F F F F F T T T T T T T T T T T T T T T F F F F F F F F F F F F F F F F

Aligned: T T T T T T T T F F F F F F F F T T T T T T T T F F F F F F F F T T T T T T T F F F F F F F F T T T T T T T T F F F F F F F F

Emergency: F F F F T T T T F F F F T T T T F F F F T T T T F F F F T T T T F F F T T T T F F F F T T T T F F F F T T T T F F F F T T T T

Doorway Obstructed F F T T F F T T F F T T F F T T F F T T F F T T F F T T F F T T F T T F F T T F F T T F F T T F F T T F F T T F F T T F F T T

Doorway Closed T F T F T F T F T F T F T F T F T F T F T F T F T F T F T F T F F T F T F T F T F T F T F T F T F T F T F T F T F T F T F T F

Hazard (Me) Y Y Y Y Y Y Y Y N N N N Y Y Y Y N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Hazard (Chart) Y Y Y Y Y Y Y Y N N N N Y Y Y Y N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Hazard (Equation) Y Y Y Y Y Y Y Y N N N N Y Y Y Y N N N N N N N N N N N N N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y

Open Provided Too Late or Out of Sequence Open Stopped Too Soon or Applied Too Long

27 Lawrence Livermore National Laboratory LLNL-PRES-722772 Open Provided When: (Stopped = "T" And Aligned = "T") Or (Stopped = "T" And Aligned = "T" And Obstructed = "T") Or (Stopped = "T" And Alarm = "T")

And / Or Chart Or Stopped F T Aligned F And Emergency F Obstructed Closed Use null (del) or blank for d Open Not Provided When: Stopped = "F" Or (Stopped = "T" And Aligned = "F" And Alarm = "F") Or (Stopped = "T" And Closed = "F")

And / Or Chart Or Stopped T T T Aligned T F T And Alarm T Obstructed T Closed Use null (del) or blank for don

28 Lawrence Livermore National Laboratory LLNL-PRES-722772 Open Provided Too Late of Out of Sequence: Stopped = "F" Or (Stopped = "T" And Aligned = "F" And Alarm = "F")

And / Or Chart Or Stopped T T T Aligned T T F And Alarm T Obstructed T Closed Use null (del) or blank for don't care

Open Stopped Too Soon or Applied Too Long: Stopped = "F" Or (Stopped = "T" And Aligned = "F" And Alarm = "F")

And / Or Chart Or Stopped T F Aligned And Alarm Obstructed Closed

29 Lawrence Livermore National Laboratory LLNL-PRES-722772 Open ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen ProvidedOpen Provided Case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 And / Or Chart Stopped: T T T T T T T T T T T T T T T T F F F F F F F F F F F F F F F F Or Stopped F T Aligned: T T T T T T T T F F F F F F F F T T T T T T T T F F F F F F F F Aligned F And Emergency F Emergency: F F F F T T T T F F F F T T T T F F F F T T T T F F F F T T T T Obstructed Closed Doorway Obstructed F F T T F F T T F F T T F F T T F F T T F F T T F F T T F F T T Use null (del) or blank for d

Doorway Closed T F T F T T T F T F T F T F T F T F T F T F T F T F T F T F T F Hazards Hazard (Me) N N N N N N N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y 20 Hazomatic Hazard (Chart) N N N N N N N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y 20 Hazard (Equation) N N N N N N N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y 20 Clear

30 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Number of Combinations = 128 • s = number of binary states • g = number of guide phrases • Combinations = 2s x g = 25 x 4 = 128 . Number of Potential Hazards Combinations: me and/or equation Open Provided = 20 20 20 Open Not Provided = 12 12 12 Open Too Late or Out of Sequence = 12 12 12 Open Stopped Too Soon Applied Too Long = 32 32 32 Total 76 76 76

. Resulting Revised Requirements = 4 . Manual was 7 hazards and 0 Revised Requirements

31 Lawrence Livermore National Laboratory LLNL-PRES-722772 Revised High Level Requirements:

. Doors shall open at station when train is completely stopped and aligned with platform.

. Doors shall remain closed when train is moving

. Doors shall remain open when someone is in the doorway, the train shall not move until the doors close.

. The train shall remain stopped and a door open command issued if an object is detected in the doorway when doors are closed

. An emergency shall stop the train, Doors shall open in an emergency after the train has stopped

32 Lawrence Livermore National Laboratory LLNL-PRES-722772 Home Wi-Fi Router CO Detector

Thermostat Smart Phone

Dash Button Cell Alarm System Tower

Internet Service Camera Camera Provider Vehicles Front Door Driveway Motion Detection

33 Lawrence Livermore National Laboratory LLNL-PRES-722772 Thermostat Internet Home Wi-Fi Router Service Cell Provider Tower

Lower Utility Bills Energy Preservation

34 Lawrence Livermore National Laboratory LLNL-PRES-722772 Thermostat Internet Home Wi-Fi Router Service Cell Provider Tower

Lower Utility Bills Energy Preservation

Motion Detection

35 Lawrence Livermore National Laboratory LLNL-PRES-722772 Thermostat Internet Home Wi-Fi Router Service Cell Provider Tower

Lower Utility Bills Energy Preservation

Alarm System

36 Lawrence Livermore National Laboratory LLNL-PRES-722772 Home Wi-Fi Router Camera Internet Driveway Service Cell Provider Tower

Safety Win Arguments With Teens

Motion Detection

37 Lawrence Livermore National Laboratory LLNL-PRES-722772 Thermostat Internet Home Wi-Fi Router Service Cell Provider Tower

Safety CO Detector

38 Lawrence Livermore National Laboratory LLNL-PRES-722772 Internet Home Wi-Fi Router Service Cell Provider Tower

Security

House Sounds Home Lighting

39 Lawrence Livermore National Laboratory LLNL-PRES-722772 Camera Home Wi-Fi Router Internet Front Door Service Cell Provider Tower

Safety Security

40 Lawrence Livermore National Laboratory LLNL-PRES-722772 https://www.amazon.com/Dash-Buttons/b?ie=UTF8&node=10667898011

Energy Preservation

41 Lawrence Livermore National Laboratory LLNL-PRES-722772 Safety Security Energy Preservation

42 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Literature searches cite 6-10 concerns about vulnerabilities. . Have not found a formal hazard or vulnerability analysis on the IoT. . Can STPA identify the same 6-10 Hazards and Vulnerabilities? . Can STPA find additional Hazards and Vulnerabilities?

43 Lawrence Livermore National Laboratory LLNL-PRES-722772 44 Lawrence Livermore National Laboratory LLNL-PRES-722772 45 Lawrence Livermore National Laboratory LLNL-PRES-722772 Development Subsystem

Mobile Phone Subsystem

External Communications Subsystem

46 Lawrence Livermore National Laboratory LLNL-PRES-722772 Software Requirements

Actuator Sensor

Software Elicitation

47 Lawrence Livermore National Laboratory LLNL-PRES-722772 STPA Guide Phrases

A resource or action required for Software correct operation is not provided or is Requirements not followed.

An incorrect resource or action is Actuator Sensor provided that leads to a hazard/risk.

A potentially correct resource or action Software is provided too late, or out of sequence. Elicitation

A correct resource or control action is stopped too soon or applied too long.

48 Lawrence Livermore National Laboratory LLNL-PRES-722772 Each Box Links to component Development Level STPA Analysis Subsystem

Update

Mobile Phone Subsystem

External Communications Subsystem

49 Lawrence Livermore National Laboratory LLNL-PRES-722772 Physical Model Risks Comb. Average Development Risks 110 94592 9.2 Communications Risk 64 6080 6.4 Mobile Risks 27 1424 5.4 Total 201 102096 7.0 Assume “or” of all three #### Assume “and” of all s = number of binary states g = number of guide phrases Combinations = 2s x g = 2012 x 4 = ####

Age of the Universe in Plank time 8E+60 Plank time = 10E-43 seconds

Estimated number of electrons in the universe =10E+88

50 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Representative STPA which represents all the analyzed components of the physical model:

Development Requirements Incorrect for IoT Weak Security Design Difficult User Interface Software Defects Acuator Software Vulnerabilities Sensor Tainted Input Possible Logic Bomb in Code Back Door in Code Physical Inadequate Testing Device Untrusted Supply Chain Updates Not Used

51 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Representative STPA which represents all the analyzed components of the physical model:

Internet for Unclear Error Messages IoT Confusing Documentation Weak Encryption Open Ports Acuator Insufficient Platform Testing Sensor Weak Negative Testing Inadequate Security Testing Updates No Longer Supported Physical Updates Contain Malware Device Browser Vulnerabilities

52 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Representative STPA which represents all the analyzed components of the physical model:

Cellular for IoT Inadequate Encryption Backbone Vulnerabilities O/S Vulnerabilities Faux Cell Tower Acuator Sensor Juice Jacking Directed Phishing Unsecured Hot Spots Physical Browser Vulnerabilities Device Smart Phone Not Updated

53 Lawrence Livermore National Laboratory LLNL-PRES-722772 ?

54 Lawrence Livermore National Laboratory LLNL-PRES-722772 Good News: “Apple Home Kit” Emerging “Works With Nest” Standards https://workswith.nest.com/products http://www.apple.com/shop/accessories/all-accessories/homekit http://www.digitaltrends.com/home/everything-that-works-with-apple-homekit/ 1985 Home Sweet Home

Climate Surveillance Alarm Vacuum Oven Washer Control Cameras System Cleaner

Entertain- Door Toaster Dryer Lights Hot Tub ment Center Controller

Pet Door Children Refrigerator Door Bell Utilities Game Center Controller Monitors

Outdoor Phone Fire/Smoke Vehicle Freezer Pet Feeding Sprinkler System Alarms Monitors System Home IT Engineer 2022

55 Lawrence Livermore National Laboratory LLNL-PRES-722772 Development Subsystem

Mobile Phone Subsystem

External Communications Subsystem

56 Lawrence Livermore National Laboratory LLNL-PRES-722772 https://www.wired.com/2017/02/russians-engineer-brilliant-slot-machine-cheat-casinos-no-fix/

2 Min. Upload to Video St. Petersburg

Aristocrat MKV MK6 Source Code

App .25 Download Aristocrat MKV MK6 Backplane Board

57 Lawrence Livermore National Laboratory LLNL-PRES-722772 http://www.express.co.uk/news/world/600771/ISIS-Islamic-State-remote-controlled-cars- bomb-attacks-Syria-Iraq-drones-Britain

Wireless Carjacking https://www.wired.com/2015/07/hackers -remotely-kill-jeep-highway/#slide-2

58 Lawrence Livermore National Laboratory LLNL-PRES-722772 Video Cameras Wide Spread Internet Outages

https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massive-internet-outage/

59 Lawrence Livermore National Laboratory LLNL-PRES-722772 In January a fully booked four-star hotel in Austria, Romantik Seehotel Jaegerwirt, had its locks hacked.

The hackers demanded the equivalent of 1,500 Euros in bitcoin in exchange for restoring the keys’ functionality http://www.welivesecurity.com/2017/01/30/austrian-hotel-experiences-ransomware-things-attack/

60 Lawrence Livermore National Laboratory LLNL-PRES-722772 61 Lawrence Livermore National Laboratory LLNL-PRES-722772 166,667 ovens x 3,000 watts per oven OR =

500 Megawatts

Wide Spread Power Disruption 454,545 toasters x 1,100 watts per toaster

62 Lawrence Livermore National Laboratory LLNL-PRES-722772 Assure No One Home

Disable Alarms Open Doors

Self Parking Get Away Car

63 Lawrence Livermore National Laboratory LLNL-PRES-722772 Increase Premium Send Ambulance Telltale heart: Pacemaker data leads to arson, fraud charges http://www.foxnews.com/us/2017/02/08/police-use-data-on-mans-pacemaker-to-charge-him-with-ohio-arson.html

64 Lawrence Livermore National Laboratory LLNL-PRES-722772 auto

65 Lawrence Livermore National Laboratory LLNL-PRES-722772 66 Lawrence Livermore National Laboratory LLNL-PRES-722772 …and how did you want to pay for your anesthesia today Martha?

67 Lawrence Livermore National Laboratory LLNL-PRES-722772 Driving Violations for April 2017

Speeding: 23 Stop Signs: 12 Traffic Lights: 3 Acceleration: 8 Turn G Force: 3

Rate Increase = 15%

Driving Violations for April 2017

Speeding: 23 Stop Signs: 12 Traffic Lights: 3 Acceleration: 8 Turn G Force: 3

Rate Increase = 15%

68 Lawrence Livermore National Laboratory LLNL-PRES-722772 69 Lawrence Livermore National Laboratory LLNL-PRES-722772 70 Lawrence Livermore National Laboratory LLNL-PRES-722772 Donald Reifer, “Industry Software Cost, Quality, and Productivity Benchmarks”, DoD Software Tech News, July 2004

71 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Assumption is defect rate after one year, not at release as in 2004 table.

. .55 defects per ksloc for Mission Critical codes . What should defect rate be for IoT and Mobile software then? Reifer Consultants http://reifer.com/

72 Lawrence Livermore National Laboratory LLNL-PRES-722772 IoT Software Error Rates 12

10

8 Web Business 2026

6

4 Errors KESLOC Errors 2

0 Defect Rates will approach 2004 2016 2028 but never reach zero. Web Business Telecommunications Linear (Web Business) Linear (Telecommunications)

Telecommunications 2021

73 Lawrence Livermore National Laboratory LLNL-PRES-722772 1 1 2 Numbers Linked to IoT Tools for each component Development Subsystem 3 4 5 6 7 8

23 24 11 9

Mobile Phone Subsystem 26 12 10 25

External 13 14 15 16 17 Communications Subsystem

18 19 20 21 22

74 Lawrence Livermore National Laboratory LLNL-PRES-722772 1. Do not use inexpensive household routers for IoT applications as the software is buggy. 2. Use strong (AES or better) encryption everywhere 3. Replace default factory user names and passwords with strong passwords, change periodically http://www.howtogeek.com/195430/how-to-create-a-strong- password-and-remember-it/ 4. Assure firmware and software updated to latest versions. 5. Assure media and updates are from trusted sources 6. Don’t use public recharge stations

75 Lawrence Livermore National Laboratory LLNL-PRES-722772 7. Alarm on IoT devices disabled 8. Wait until IoT industry matures 9. Don’t allow remote help desks without confirming source. 10. Static and Dynamic code analysis on Source and Binary codes 11. Assure IoT applications are security tested and support security with updates 12. Segment networks (IoT, Personal, Business) 13. Purchase new device every three year

76 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Literature search found 6-10 security vulnerabilities

. STPA found201 security vulnerabilities, hazards, and risk types. . Surrogate STPA rolled up to 30 issues 10

201 Literature STPA

77 Lawrence Livermore National Laboratory LLNL-PRES-722772 1. STPA facilitated me to think of hazards and risks I could not think of intuitively 2. STPA identified risks and hazards not identified in the literature search 3. STPA helped me research further important topics 4. STPA helped me ask better questions and enroll experts

78 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Prefatory STPA requires human analysis to identify hazards or vulnerabilities when requirements are stated. . Exhaustive STPA (examining all combinations) can be useful for small numbers of variables or when requirements are not stated. . Exhaustive STPA may not be practical for analysis of a larger number if variables, decomposition and a surrogate approach may be helpful.

79 Lawrence Livermore National Laboratory LLNL-PRES-722772 . STPA useful addition to other techniques, literature search, and expert consultation, for IoT and Mobile hazard, risk, and security analysis

80 Lawrence Livermore National Laboratory LLNL-PRES-722772 81 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Requirements . Design • Rational Requisite Pro • IBM Architect Designer • Objectiver • Structure Chart • CaseComplete • Data Flow Diagram • RMTrac • Doxygen • Optimal Trace • Visustin • Analyst Pro • McCabe IQ • DOORS • Design By Contract • GMARC • Assertions • Jira

82 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Editors . Compilers • Sublime Text • Intel • Notepad ++ • Vim • Gnu • Atom • Microsoft • Emacs • Visual C++ • Eclipse • Code::Blocks • Clang • GNAT Programming • CUDA Studio • ROSE Custom Tool • Code Lite • NetBeans • Qt Creator

83 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Build Tools . Libraries • Cmake • Boost • Autoconf • OpenMP • Jenkins • Bamboo • OpenGL • Ant • VTK • CruiseControl • and many more • Git • Subversion • Mecurial • Perforce • and many more

84 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Debuggers . Software Testing • Code View • Cppunit • Valgrind • Junit • MS Visual Studio • Google Test • Intel Debugger • Test Complete • Parasoft Insure++ • HP Functional Tester • IDAPro • Selenium • ROSE Custom Tool • Squish • and many more • ATS • and many more

85 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Developed Code . IoT Device • Purify • Vector Software • Intel Inspector XE • HiTex • Insure++ • eggPlant • Lcov,Gcov • Xilinx SDK • Cobertura • Peta Linux Tools • Klocworks • IDAPro • Parasoft • ROSE Custom Tool • Coverity • ROSE Custom Tool

86 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Electronic . Mechanical • Spark • AutoCad • PulseForge 3300 • Autodesk CAM • Project Wire • IronCAD • TDK4PE • SolidWorks • PCB Web Designer • and many more • NanoTech AMD 3D • and many more

87 Lawrence Livermore National Laboratory LLNL-PRES-722772 . HMI . Peripherals • Active X • LabView • Lab Windows • FastSend • Tk/Tcl • PacketCheck • Motif • Intel Admin Tools • Visual Basic/C++ • NDT • PowerBuilder • and many more

88 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Software Aps . Utilities • Norton • CrashPlan • McAfee • COMODO Back Up • TrendMicro • AOMEI Backupper • Malwarebytes • Cobian Backup • Bitdefender • and many more • MS Office • ROSE Custom Tool

89 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Operating Systems . Cable or Wi-Fi • Windows • Vastar • Linux • Fluke — Embedded Linux • Ideal Networks — Android • Unix BSD • AirCheck G2 • Mac OS X • Wi-Fi Pineapple — iOS • Wi-Spy DBx • Commercial Unix • Wi-Fi Analyzer • Solaris/OpenSolaris • Yellowjacket-BANG • Custom OS

90 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Routers . Firewalls • Angry IP Scanner • Nessus • Ping Plotter • Nmap • Blast • Netcat • TCP View • Tcpdump • PRTG Network • Wireshark Monitor • CommView • HP Load Runner • and many more • and many more

91 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Internet . Update Patches • ZenMap • Tripwire CCM • Ipplan • CodeDx • Tcpdump • Corporate Software • Nmap NSE Scripts Inspector • Dig • Open VAS • Acunetix • Retina CS • John the Ripper • MBSA • and many more • Nexpose • and many more

92 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Cellular Network . Smart Phone O/S • Speedtest • iOS • Network Signal Info • Android Pro • Windows Mobile • wiggle.net • BlackBerry OS • LG webOS

93 Lawrence Livermore National Laboratory LLNL-PRES-722772 . Smart Phone Aps . Browsers • McAfee • Chrome • Norton • Firefox • Trend • Safari • Panda • Opera • Webroot • Skyfire • iPhone N/A • UC Browser • BeFE

94 Lawrence Livermore National Laboratory LLNL-PRES-722772 95 Lawrence Livermore National Laboratory LLNL-PRES-722772 Software Requirements Elicitation/Software Requirements Requirements Stakeholder Sample Wrong Requirement Not Speciific Requirement Not Measurable Actuator Sensor Requirement Not Attainable Requirement Not Realizable Requirement Not Time Bounded Requirements Requirement Not Complete Elicitation Requirement Not Concise Approved Before Understood Jumping to Design https://www.linkedin.com/pulse/top-8-mistakes-requirements-elicitation-aaron-whittenberger-cbap

96 Lawrence Livermore National Laboratory LLNL-PRES-722772 Software Software Requirements/Software Design Design Missing Functional Requirements Missing Non-Functional Requirements Missing Security Requirements Actuator Sensor Missing Safety Requirements Exception Cases Not Handled Unclear Software Unprioritized Requirements Incomplete Unconsumable Unreflective of business goals http://www.seilevel.com/requirements/5-reasons-software-projects-fail-hint-its-often-due-to-incomplete-incorrect-requirements

97 Lawrence Livermore National Laboratory LLNL-PRES-722772 Editor Software Design/Editor Lacks Intuitiveness Weak Exception Handling Human Machine Interface not well Defined Actuator Sensor Requirement Creep Weak Scalability Weak Coupling Software Design Weak Cohesion Sparse Commenting Lack of Meaningful Naming http://www.pcworld.com/article/146282/article.html

98 Lawrence Livermore National Laboratory LLNL-PRES-722772 Editor/Compiler Uninitialized Variable Compiler Un Released Memory Array overflow Buffer Overflow Actuator Sensor Tainted Input Null pointer Stale Pointer Unreachable Code Editor Back Door Logic Bomb Missing Requirements Misunderstood Requirements Syntax Error

99 Lawrence Livermore National Laboratory LLNL-PRES-722772 Compiler/Build Tools Build Compiler Warnings Ignored Tools Wrong Test Baseline Inadequate Resources Missing or Wrong Includes Actuator Sensor Wrong Options Set Non-Standard Features Used Exceptions Not Caught Compiler Unclear Error Messages Errors Ignored Warnings Ignored Compiler Not Updated

100 Lawrence Livermore National Laboratory LLNL-PRES-722772 Libraries Build Tools/Libraries Wrong Library Version Misnamed Library Library Vulnerability Actuator Sensor Library Not Updated Library Back Door Untrusted Supply Chain Build Tools Library Not Updated

https://msdn.microsoft.com/en-us/library/8x5x43k7.aspx

101 Lawrence Livermore National Laboratory LLNL-PRES-722772 Debuggers Libraries/Debuggers Heisenberg Effect Diagnostics Residual Actuator Sensor Intermittent Problem Residual Debug Code Unrealistic Simulators Libraries (Built Code)

102 Lawrence Livermore National Laboratory LLNL-PRES-722772 Software Debuggers/Software Testing Testing Low Coverage No/Few Security Tests Rushed Testing Actuator Sensor No/Few Requirements Traceability Redundant Testing Debuggers No/Few Performance Tests (Debugged Weak Platfoirm Testing Code)

103 Lawrence Livermore National Laboratory LLNL-PRES-722772 Tested Software Software Testing/IoT Code Developed Residual Bugs Slow Performance Actuator Sensor Race Condition Weak Error Recovery Weak On Line Help IoT Code Weak Documentation Developed

104 Lawrence Livermore National Laboratory LLNL-PRES-722772 IoT Code IoT Code Developed/IoT Device Developed HW/SW Compatibility Operating System Vulnerability Actuator Sensor Unsupported Operating System Version Slow Performance Race Conditions Confusing Install Procedure IoT Device Missing Resources Tainted Install Media

105 Lawrence Livermore National Laboratory LLNL-PRES-722772 Mechanical IoT Device/Mechanical Design Design Insufficient Air Flow Confusing Labeling Harness Rub Actuator Sensor Poor Serviceability Fan Cooling Needed EMI/EMC Shielding Insufficient Vibration Resistance IoT Device Insufficient Shock Resistance Insufficient Environmental Resistance Manufacturing Defect Supply Chain Defect

106 Lawrence Livermore National Laboratory LLNL-PRES-722772 IoT Device/ Electrical Design Inadequate Power Supply Electronic Inadequate Component Reliability Design Inadequate Surge Protection Inadequate Noise Suppression Actuator Sensor Counterfeit Parts Slow Processing Lack of Fault Detection Inadequate Shock Protection IoT Device Inadequate Static Discharge Protection Version Control Issues Weak Serviceability Weak or No Upgrade Ability

107 Lawrence Livermore National Laboratory LLNL-PRES-722772 IoT/Device HMI Device HMI Confusing User Interface Weak Exception Handling Unclear Error Messages Actuator Sensor Unlike Incumbent Lack of Documentation Confusing Documentation IoT Device Secret Codes Remain Unix Jan 19, 2038 Bug /32 bit https://en.wikipedia.org/wiki/Year_2038_problem

108 Lawrence Livermore National Laboratory LLNL-PRES-722772 IoT/Peripherals Peripherals Current Drivers Not Available Encryption Not Available Inadequate Installation Guide Actuator Sensor No Guest Device Controls Open Unused I/O Ports Driver Not Updated IoT Device No Longer Supported Buggy Software

109 Lawrence Livermore National Laboratory LLNL-PRES-722772 IoT/Other SW Apps Other File Names Not Unique Software Applications Versions Not Compatible Supply Chain Not Secure Actuator Sensor Trojans Present Back Door Present Not Enough Memory IoT Device Rogue Applications

110 Lawrence Livermore National Laboratory LLNL-PRES-722772 Utilities IoT/Utilities Wrong Version Known Vulnerabilities in Version Version Not Updated Actuator Sensor Wrong Numerical Precision Wrong Data Format Counterfeit Item IoT Device No Upgrade Path Browser Vulnerabilities

111 Lawrence Livermore National Laboratory LLNL-PRES-722772 Operating IoT/Utilities Systems Wrong Version Known Vulnerabilities in Version Version Not Updated Actuator Sensor Wrong Numerical Precision Wrong Data Format Counterfeit Item No Upgrade Path IoT Device Browser Vulnerabilities

112 Lawrence Livermore National Laboratory LLNL-PRES-722772 Peripherals Peripheral/ IO Cable or Wi-Fi Not Encrypted Weak Encryption Network Not Password Protected Actuator Sensor Open I/O Ports Driver Not Updated I/O Trap Door in Driver Cable or Wi-Fi

113 Lawrence Livermore National Laboratory LLNL-PRES-722772 I/O I/O Cable or Wi-Fi / LAN Routers Cable or No / Weak Encryption Wi-Fi Buggy Router Software Router Firmware Not Updated Actuator Sensor Router Updates No Longer Available Back Door Left In Router Updates Contain Virus LAN Default User and Password Routers Ports Left Open (7547) Services Exposed TR-64 -69

114 Lawrence Livermore National Laboratory LLNL-PRES-722772 LAN LAN Routers/Firewall Routers Ports Left Open Wrong Security Policy Firewall No Longer Supported Actuator Sensor Firewall Contains Back Door Capacity Insufficient VPN Not Available Firewalls Not Updated

115 Lawrence Livermore National Laboratory LLNL-PRES-722772 Firewall Firewall/Internet Capacity Insufficient VPN Not Available No Port Scanning Detectors Actuator Sensor Required Port Blocked

Internet

116 Lawrence Livermore National Laboratory LLNL-PRES-722772 Update Internet/Update Patched Patches Patch Infected With Virus Patch Has Vulnerability Device No Longer Supported Actuator Sensor Updates Not Done

Internet

117 Lawrence Livermore National Laboratory LLNL-PRES-722772 I/O Connection I/O Connection Cable or Wi-Fi/Cellular Network Cable or Not Encrypted Wi-Fi Inadequate Encryption A5/1, A5/2 Actuator Sensor Backbone Vulnerabilities SS7 Backdoors Public Unsecured Wi-Fi Cellular Operating System Vulnerabilities Network O/S Not Upgraded

118 Lawrence Livermore National Laboratory LLNL-PRES-722772 Cellular Cellular/Smart Phones Juice Jacking Faux Cell Tower Actuator Sensor

Smart Phones

119 Lawrence Livermore National Laboratory LLNL-PRES-722772 Smart Smart Phones/Browser Phone O/S Lack of Frame Busting, Tap Jacking Vulnerable Mobile Site Actuator Sensor Phishing Counterfeit Address Bar Unsecured Hot Spots Unintelligible Error Message Browsers

120 Lawrence Livermore National Laboratory LLNL-PRES-722772 Browsers/Other Apps Broswer Not Downloaded From Trusted Source Malicious Apps Actuator Sensor Apps Not Updated App No Longer Supported Apps Not Updated Other Smart Phone Apps

121 Lawrence Livermore National Laboratory LLNL-PRES-722772 IoT Code Smart Phone/Updates Developed Counterfeit Updates Trap Door in Update Actuator Sensor Discontinued Updates Slow Updates Unencrypted Updates Smart Out Of Date Firmware Phones Device Not Updated

122 Lawrence Livermore National Laboratory LLNL-PRES-722772