FEATURE

A Reference Architecture for Secure Medical Devices Steven Harp, Todd Carpenter, and John Hatcliff

Abstract and users cannot assume that medical Steven Harp, is a distinguished We propose a reference architecture aimed at devices will operate in a benign security engineer at Adventium Labs in supporting the safety and security of medical environment. Any device that is capable of Minneapolis, MN. Email: steven. devices. The ISOSCELES (Intrinsically connecting to a network or physically [email protected] Secure, Open, and Safe Cyber-Physically exposes any sort of data port is potentially at Enabled, Life-Critical Essential Services) risk. How should we think about and Todd Carpenter, is chief engineer architecture is justified by a collection of design manage risk in this context? at Adventium Labs in Minneapolis, principles that leverage recent advances in This question has been explored exten- MN. Email: todd.carpenter@ software component isolation based on sively.3,4 Special publications from the adventiumlabs.com hypervisor and other separation technologies. National Institute of Standards and Technol- The instantiation of the architecture for ogy (e.g., 800-39) provide a conceptual risk John Hatcliff, PhD, is a particular medical devices is supported by a management framework.5,6 AAMI TIR577 distinguished professor at Kansas development process based on Architecture describes how security risk management State University in Manhattan, KS. Analysis and Design Language. The architec- can be integrated with safety risk manage- Email: [email protected] ture models support safety and security ment (e.g., as addressed in ISO 14971 and analysis as part of a broader risk management specifically for medical devices in IEC framework. The models also can be used to 80001). The UL 2900 series of standards derive skeletons of the device software and to provides cybersecurity requirements for configure the platform’s separation policies and medical devices.8 an extensive set of services. We are developing In brief, a threat source initiates a threat prototypes of the architecture and example event, which may exploit a device vulnera- medical device instantiations on low-cost bility, causing an adverse impact. The boards that can be used in product solutions. impact might affect the mission of the The prototype and supporting development device, the end user, or the organization and assurance artifacts are being released that created or operated the device. In this under an open-source license. framework, risk is based on the likelihood of a threat event occurring and amplified by A reference architecture is a domain-spe- the potential loss (adverse impact) should cific design template for the structure of a the event occur. class of systems as a set of constituent parts Threat sources have capabilities, intents, with special roles and communications and often preferences for particular targets. patterns. The reference architecture Automated threats, such as malware, can be discussed in this article addresses the class indiscriminate in their target selection and of small bedside medical devices (e.g., do not care whether they are exploiting a infusion pumps, electrocardiographs, personal system or a hospital. Threats ventilators). The architecture also aims to targeting particular organizations also are support interoperability interfaces through well documented.9–11 The intent of the threat compatibility with the ASTM F2761 Inte- source is frequently financial, as exemplified grated Clinical Environment (ICE)1 or the by the recent spate of ransomware (e.g., IEEE 11073 service-oriented device connec- WannaCry).12 In addition to direct extortion, tivity standards.2 Any such architecture the medical information that is present in must address a wide range of requirements, some devices can have indirect value, such including safety, cost, maintainability, and as protected health information or personally performance. It also must address an identifiable information being leveraged to unpleasant modern reality: Manufacturers steal funds or illegally acquire drugs.

www.aami.org/bit 311 FEATURE

A special type of imact needs to be instantiate it can reduce the attack surface. considered for medical devices: The lives Security controls in the device can further and health of patients may depend on them. reduce an attacker’s ability to exploit certain Security and safety are intertwined in such outstanding vulnerabilities and minimize systems, and failure to manage security the impact of successful exploitation of risks may pose safety risks.7 Beyond being others. an end target of an attack, a medical device The remainder of this article presents may represent a stepping stone for a larger principles driving the design and instantia- campaign. Exploitation of a vulnerable tion of Intrinsically Secure, Open, and Safe device may permit access to other clinical or Cyber-Physically Enabled, Life-Critical financial systems by allowing the attacker to Essential Services (ISOSCELES)—a safe and leverage trust information or to extract secure architecture for medical devices. credentials that allow the device to connect to these other systems. An example of a Architecture successful attack of this sort is documented The ISOSCELES reference architecture is in the MEDJACK report on blood-gas-ana- specified with tiered requirements: lyzers attacked in 2015–16.9 1. Platform core requirements for hard- A noteworthy long-term risk is when ware, system software, and services malware authors or researchers automate supporting the medical application. the attacks. This could result in multiple, 2. Platform design requirements derived nearly simultaneous malfunctions, similar from the platform core requirements to what occurred with the WannaCry attack and refined to a specific design. in May 2017. With automation, botnet 3. Device design requirements specific to a controllers could use the devices to attack particular medical device being devel- other devices on healthcare facility net- oped (e.g., an infusion pump). works. Decades ago, the attackers were The first two platform tiers drive the people who actively attacked your computer. reference architecture. These 134 require- Unfortunately, automated botnets—mil- ments are intended to be suitable for a lions of already-compromised machines broad family of potential medical devices. trying to grow their reach—are now the Figure 1 illustrates various components of prevalent attackers. a hypothetical medical device design based Recent Department of Homeland Security on this reference architecture. The exact advisories documented hard-coded pass- medical applications and services needed words and credentials in medical devices.13,14 may vary according to the device. A hard- Vulnerabilities such as these make it easier ware layer (bottom of figure) contains for malware (e.g., botnets, ransomware) to processors and peripheral devices that gain a foothold on the device. support security mechanisms. A low-level By itself, a medical device architecture separation layer of software uses these cannot address all aspects of risk. Security mechanisms to isolate all other software analyst Sergey Lozhkin noted that although components from each other, except for vulnerable devices were revealed in penetra- specifically allowed interactions. The tion testing, “the problem is not only one of ISOSCELES platform services satisfy typical weak protection of medical equipment, it needs of the top-layer medical applications has a much wider scope. The whole IT that provide medical diagnostics and infrastructure of modern hospitals is not therapies. The design of a specific device properly organized and protected, and the might incorporate only the required ele- problem persists worldwide."10 ments needed for that class of device. Until the day when software can be automatically and completely validated, we Architectural Principles must assume that some vulnerabilities The ISOSCELES reference architecture remain even after formal design and attempts to promote basic principles that extensive testing. However, a suitable provide security and safety. These principles reference architecture and tools to help and their motivations are described below.

312 Biomedical Instrumentation & Technology September/October 2018 FEATURE

Use Strong Time and Space Separation Many modern systems provide a basic security perimeter but lack internal security barriers, making it easy for attackers to access arbitrary information within the device, and even to control the device, after they have breached the perimeter. Strong barriers between software components help contain faults and attacks from propagating to safety-critical components. For example, if an infusion pump requires a network connection (e.g., to maintain current drug libraries), the network stack and code associated with retrieving the drug library should be wholly separate from the software that interprets the contents of the Figure 1. The layered ISOSCELES (Intrinsically Secure, Open, and Safe Cyber-Physically Enabled, drug library. In turn, that function also Life-Critical Essential Services) architecture. Abbreviations used: CPU, central processing unit; HMI, should be entirely separate from the function human-machine interface; HSM, hardware security module; ICE, integrated clinical environment; that controls the rate at which the drug is MMU, memory management unit; OEM, original equipment manufacturer; RAM, random-access memory; TPM, trusted platform module. pumped. In safety architectures, each unit of separation is called a partition.15 Separation spatial and temporal separation. It is the one kernels and some real-time operating program on the device that has direct ability systems (OSs) provide strong separation. to control the protection facilities provided Historically, application software on medical by the hardware. It is intentionally as small devices has been built either on minimal and simple as possible to increase its OSs that fail to provide strong separation or trustworthiness. without the benefit of any OS at all. Key hardware features that support this Spatial separation refers to barriers role are hardware memory management between the memory regions and addressa- units (MMUs) and input/output MMUs ble peripherals assigned to different (IOMMUs). While standard on desktop components. A component operating in one systems, these features historically have not partition should not be able to directly been present on inexpensive or low-power reference the memory or input/output (IO) microprocessors. For example, ARM micro- devices assigned exclusively to another controllers prior to the ARMv3 family lacked component. Failure to do so would allow a integral MMUs, and Intel Atom processors defective or compromised component to prior to 2016 lacked IOMMUs. These classes affect the missions of other components. of processors are present in many fielded Temporal separation refers to the inability medical devices, providing opportunities for of one component to affect or measure the attacks that begin in vulnerable processes to time intervals during which another compo- escape and tamper with the resources of nent accesses a resource. For example, a other processes. However these features defective or compromised component have recently appeared in affordable embed- should not be able to dominate a central ded CPUs in both ARM and x86 processor processing unit (CPU) and thereby keep architectures. another component from performing its An MMU is programmable hardware that mission in a timely manner. Programs controls the physical memory that a given running in one partition should be oblivious process may access while providing the to the operation and resource usage of process with the appearance of a well-or- programs in other partitions, except as dered “virtual address space.” It also can explicitly intended. mark a section of memory as read-only. In ISOSCELES addresses these requirements some cases, it may be able to prevent the by specifying a low-level software kernel processor from executing instructions in dedicated to establishing and maintaining certain areas of memory. If a process

www.aami.org/bit 313 FEATURE

attempts to address memory outside of its the ability to access resources such as partition or in ways not permitted by the memory, arbitrary processor instructions, programmed policy, then access is blocked and peripheral devices. Unfortunately, and the CPU will trap to a handler to deal embedded systems based on commodity OSs with the fault. (e.g., Linux, Windows) are all too often Recent Meltdown and Spectre attacks have implemented with “root” access, which demonstrated several flaws in super-scalar means that as soon as the perimeter is processors that violate this protection; the breached, the attacker can easily access processors specified by ISOSCELES, such as anything. the ARM A53, do not currently exhibit these For our reference architecture, the hard- issues. The IOMMU performs a similar ware layer assists in this goal in ways beyond function for peripheral devices. Failure to the separation mechanisms described above. isolate the IO space opens a door for defec- Modern microprocessors offer multiple tive or compromised software components operation modes, including a supervisory to breach their partitions and interfere with mode with higher privilege and a user mode other components and external devices. For with lower privilege. The supervisory mode example, a process designed to check for is allowed to execute all of the processor network updates might be compromised by instruction set and is permitted to program a maliciously crafted packet, and injected the MMU and IOMMU. The user mode code could modify the GPIO (general-pur- cannot change the memory management pose input/output) pins to turn on a pump and is blocked from using privileged instruc- or change the power state of the device. tions, such as controlling interrupt handling Prototypes of ISOSCELES have used both or processor faults. the seL4 microkernel and the Xen hypervisor ISOSCELES stipulates that only the for separation.16,17 A microkernel is like an separation layer operates in the most OS kernel but is devoid of the usual panoply privileged mode. All other software compo- of device drivers, network stacks, and file nents run in an unprivileged mode and rely system drivers. It specializes in strongly on the separation layer to delegate to them separating memory and processor time into only those special privileges needed for their partitions, along with minimal support for missions. Although something like this interpartition communication (IPC). The “kernel-versus-userland” distinction exists in seL4 microkernel is distinguished by a general purpose OSs, systems based on formal proof of its correctness on certain microkernels extend this demotion to processor architectures.18 Hypervisors offer components such as device drivers, file the additional ability to provide each parti- system drivers, and protocol stacks. These tion an emulated environment, giving the often are relatively complex components, component the illusion that it is running on and their failure should not jeopardize other its own hardware. This permits partitions to parts of the overall system. For example, the run general-purpose OSs (e.g., Linux, compromise of a network device driver in a Berkeley Software Distribution, Microsoft monolithic kernel would threaten every Windows). So-called type 1 hypervisors (e.g., aspect of the system’s operation. (The driver Xen) support both low-level separation and would run with full privileges in the kernel the emulation capabilities. memory space.) In a microkernel, the device driver is just another user-level component, Minimize Privilege and compromise can only affect other Going hand-in-hand with separation, the components that interact with it. principle of minimal privilege, or least privilege, is a cornerstone of security. In the Minimize Complexity context of the reference architecture, it The trustworthiness of software tends to means that any given component should be correlate inversely with complexity; the more granted only the privileges required to complex code is, the less trustworthy it is. perform its mission. Privileges here include Many factors influence this, including the

314 Biomedical Instrumentation & Technology September/October 2018 FEATURE

number of attack surfaces, the difficulty of new channels to other components. These correctly developing code, and the effort channels are specified and checked in the required to verify and validate desired design’s AADL model, then translated to the properties. The reference architecture enforcement mechanism in the microkernel encourages the construction of components or hypervisor. that are as small and simple as possible. External communications are further Many application components can per- regulated by an integral firewall that is form their primary functions without all of independent of the internal communicating the accouterments of typical applications component. A default-deny firewall policy running in a general purpose OS. Example ensures that the device only communicates “extra baggage” includes general purpose C with a “whitelist” of approved parties. COTS and other standard software libraries, OSs often ship with open policies to ease arbitrary software languages, and full access integration and development overheads; to all system calls. The Linux 2.6 kernel has these policies must be explicitly tightened 338 system calls.19 The Standard C library down. adds 500 kB (and often much more) binary The framework encourages communica- code.20 The presence of this code in the tions patterns that limit the degree of trust address space of a component provides an required. In many cases, the relationships attack surface that facilitates remote code between components can be ordered by an execution using techniques such as evaluation of their relative trustworthiness or return-oriented programming.21 by their relative criticality. For example, some In contrast, ISOSCELES specifies a components are measurably more complex minimal component runtime environment than others or contain third-party code that includes only essential IPC primitives. libraries that cannot be fully evaluated. Components can leverage the architecture’s Components that communicate directly with service layer using IPC and avoid including external networks might be less trustworthy, service layer code in their own memory as they could be compromised by external partitions. Using this methodology, compo- attack. Safety-related components should be nents (in the microkernel realization of treated as more critical than other opera- ISOSCELES) can be shrunk from hundreds tional components and should not directly to tens of kilobytes and only expose a very depend on those external-facing compo- narrow IPC attack surface. The Architecture nents.15 The reference architecture allows Analysis and Design Language (AADL) designers to place less trustworthy compo- modeling tools (see leverage models of nents in a subordinate role to more correctness section below) help specify and trustworthy peers. construct appropriately minimal designs. Designs may select communications mechanisms on a per-channel basis to Manage Trust Relations provide the desired separation, including: Even in the simplest systems, a device’s • Synchronous interpartition remote software components must communicate procedure calls. with each other. Modern medical devices also • Message queues. must communicate with external systems. • Shared memory marked as read-only to This interaction carries inherent risk, as one some components. party must trust another to provide valid and • Asynchronous event signaling. timely messages. The safety and liveness of critical processes depends on this trust being Leverage Common Services warranted. The service layer shown in Figure 1 contains The reference architecture addresses this several components that support common challenge in several ways. First, it restricts medical device needs. A medical device internal communications by policy to based on the reference architecture would predefined channels. A defective or compro- include implementations of these services; mised component is incapable of opening device designers simply select those

www.aami.org/bit 315 FEATURE

required, avoiding the need to fully develop Leverage Cryptography for them in house. Confidentiality and Integrity • A logging service can direct device log Medical devices require information protec- messages to a local persistent store and/ tion to preserve confidentiality and integrity. or over the network to a remote logging This is vital to several tasks for the medical host. device: • The time service provides current • Verifying software before executing it synchronized time, as well as precision • Transferring sensitive data to or from delayed signals, to trigger timed actions. persistent storage • A storage service manages the persistent • Validating software updates for the storage devices on the device, which may device include multiple encrypted logical • Communicating with external services partitions. (e.g., pharmacies) • The cyber-physical abstraction layer • Communicating with peer devices in an component isolates the medical applica- ICE tions from directly interfacing with Cryptography plays a central role in medical sensors and actuators. This adds satisfying these requirements. Any compo- flexibility by allowing devices to map nent in a design may include a cryptographic logical sensor or actuator roles to differ- library to satisfy its information protection ent physical devices, thereby reducing needs. This includes standard services (e.g., the verification and validation impacts if storage, update, ICE) and device-specific a sensor is updated. It also can support components (e.g., a drug library component networked devices. For example, a future that must synchronize with an external infusion pump might leverage a blood pharmacy). The reference architecture calls oxygen sensor that resides on a different for a cryptographic key management service medical device in the same ICE.22 (CKMS), which provides a mechanism to • A firewall service controls access to data control cryptographic materials such as keys, networks connected to the device. shared secrets, and certificates. This enforces Internal components that need network a policy that controls which operations may access do so through this internal be performed by which components on any firewall, which exposes a single external given cryptographic datum. The CKMS may address. in turn use a hardware security module, if • An update service manages secure device available (e.g. a trusted platform module). firmware upgrades in the field, which Such devices are designed to be tamper may occur over the network or through resistant, and keys stored within them are an attached storage device. protected even from attackers who attempt • Device control and device configuration to disassemble a device. services support the operating life cycle and the configuration of the device and Leverage Models of Correctness its components. While system testing is the common • The authentication and authorization approach for verifying the correctness of an services control the access that users are implementation, analytical proofs of correct- granted to particular device capabilities. ness are even more valuable. Software Users may be granted roles (e.g., clini- engineering has made progress in cian, administrator) that, through policy, machine-generated proofs from formal dictate the operations they may perform models of systems. When the system on the device. implementation is generated from these • A user interface service provides display models, in part or in whole, developers can and human input devices to allow device reap additional advantages of reduced users to monitor and control it. engineering effort and a correct design. • The key service controls cryptographic The ISOSCELES reference architecture is operations, as discussed below. represented in an AADL model.23,24 This language has well-defined semantics for

316 Biomedical Instrumentation & Technology September/October 2018 FEATURE hardware and software components of a Intel Atom, and AMD G-series–based SoCs system, with attributes that support reason- running the seL4 or NOVA microkernels ing about system-level characteristics of a with Genode application framework (Figure design. Figure 2 shows example communica- 3).26 tions flows for part of a design. These flow ISOSCELES requirements, designs, annotations specify the allowed information example implementations, and associated flows between particular components development tools will be made available as operating in different partitions (e.g. from open source products. Medical device firms diagnostics to interoperability services), then may use these artifacts as a starting point to out to network services (e.g., electronic health records). Flows can be checked at design time against a security policy that might forbid information flowing directly from unregulated components, such as the interoperability services flowing to the class III therapy services. In addition, tools convert these modeled flows into the runtime communications tables, which provides the tie between the model that was analyzed and the actual implementation. The models also support temporal analy- sis. The model may specify tolerances on Figure 2. AADL models of interaction between components in an ISOSCELES (Intrinsically Secure, latencies between events or the amount of Open, and Safe Cyber-Physically Enabled, Life-Critical Essential Services) design. Abbreviation jitter tolerated in a cyclic schedule. Tools can used: OEM, original equipment manufacturer. examine models and evaluate whether the process schedule will meet all of the require- ments, accounting for factors such as interpartition transition overhead. The tools also can generate the runtime schedules. AADL also supports modeling features that permit reasoning about faults and fault management strategies.25 Formalizing the assumptions about failure modes and effects analysis means that the design’s hazards can be assessed against fault behavior and propagation in both qualitative and quantita- tive ways.

Conclusion The ISOSCELES medical device reference architecture currently is under development. When completed, it will provide FDA-re- quired documentation and an exemplar device to demonstrate the use of the archi- tecture. One prototype is based on the Xen hypervisor16 and an embedded computer using an ARM CPU (dual-core Cortex-A7) system-on-chip (SoC). It uses a commodity Figure 3. A laboratory prototype of a patient-controlled analgesia (PCA) pump demonstrating compute module and interfaces with the ISOSCELES (Intrinsically Secure, Open, and Safe Cyber-Physically Enabled, Life-Critical Essential electromechanical components of a decom- Services) partitioning on the seL4 separation kernel. The ISOSCELES-based PCA pump software runs on the small Intel Atom embedded development board. That board is connected to modified missioned patient-controlled analgesia surplus pump hardware to demonstrate functionality. Abbreviation used: HDMI, high-definition pump. Another prototype runs on ARM, multimedia interface.

www.aami.org/bit 317 FEATURE

develop their proprietary designs. Original References equipment manufacturers (OEMs) will need 1. ASTM F2761. Medical Devices and Medical Sys- to select a hardware and separation tems—Essential Safety Requirements for Equip- approach based on their unique product ment Comprising the Patient-Centric Integrated needs. Next, if not already available, OEMs Clinical Environment (ICE)—Part 1: General should port the chosen separation approach Requirements and Conceptual Model. West Con- to their hardware and particular interface shohocken, PA: ASTM International; 2009. devices. They can use the ISOSCELES platform services on top of that separation 2. IEEE 11073-10207-2017. IEEE Health informat- layer and refine features, such as the specific ics—Point-of-care medical device communication key management and authentication Part 10207: Domain Information and Service approaches, to satisfy their market needs. Model for Service-Oriented Point-of-Care Medical ISOSCELES was developed specifically to Device Communication. Piscataway, NJ: IEEE; target ICEs with interoperable devices, with 2017. strong user and machine-to-machine authentication and configurable networking 3. Almohri H, Cheng L, Yao D, Alemzadeh H. and key management. The current form of On Threat Modeling and Mitigation of Medical ISOSCELES targets external devices with Cyber-Physical Systems. In: The Second IEEE/ modern processors, communications, and ACM International Conference on Connected power supplies and is not intended for Health: Applications, Systems and Engineering extremely power-constrained environments. Technologies (CHASE), July 17–19, 2017, Philadel- Although the architecture principles remain phia. 114–9. important in those environments, the power constraints driven by the ever-shrinking 4. Martins G, Bhatia S, Koutsoukos X, et al. To- device volumes will challenge current wards a Systematic Threat Modeling Approach technologies due to increased memory and for Cyber-physical Systems. In: 2015 Resilience computation requirements. Week (RWS). Piscataway, NJ: IEEE; 2015:1–6.

Acknowledgments 5. National Institute of Standards and Technology. This material is based on research spon- Managing Information Security Risk: Organi- sored by the Department of Homeland zation, Mission, and Information System View. Security (DHS) Science and Technology Special Publication 800-39. Gaithersburg, MD: Directorate; Homeland Security Advanced National Institute of Standards and Technology; Research Projects Agency (HSARPA), Cyber 2011. Security Division (DHS S&T/HSARPA/ CDS) Broad Area Announcement number 6. National Institute of Standards and Technology. HSHQDC-14-R-B0005; the Government of Guide for Conducting Risk Assessments. Special Israel; and the National Cyber Bureau in the Publication 800-30. Gaithersburg, MD: National Government of Israel via contract number Institute of Standards and Technology; 2012. D16PC00057. The views and conclusions contained herein are those of the authors 7. AAMI TIR57. Principles for medical device and should not be interpreted as necessarily information security—risk management. Arling- representing the official policies or endorse- ton, VA: Association for the Advancement of ments, either expressed or implied, of the Medical Instrumentation; 2016. DHS, U.S. government, Government of Israel, or National Cyber Bureau in the 8. UL 2900. Software Cybersecurity for Network-Con- Government of Israel. nectable Products. Washington, DC: UL; 2017. Several artifacts are being developed in collaboration with Food and Drug Adminis- 9. TrapX. Attackers Target Blood Gas Analyzers. tration (FDA) engineers through the Available at: https://trapx.com/wp-content/up- National Science Foundation/FDA Schol- loads/2017/08/Case_Study_TrapX_Healthcare_ ar-in-Residence program, which is providing MEDJACK_BGA.pdf. Accessed Aug. 2, 2018. partial support for this work.

318 Biomedical Instrumentation & Technology September/October 2018 FEATURE

10. Lozhkin S. Hospitals are under attack in 2016. 20. Eta Labs. Comparison of C/POSIX standard Available at: https://securelist.com/blog/re- library implementations for Linux. Available at: search/74249/hospitals-are-under-attack-in-2016. www.etalabs.net/compare_libcs.html. Accessed Accessed Aug. 2, 2018. Aug. 2, 2018.

11. Brook C. Sergey Lozhkin on How He Hacked 21. Shacham H. The geometry of innocent flesh on His Hospital. Available at: https://threatpost. the bone: return-into-libc without function calls com/sergey-lozhkin-on-how-he-hacked-his-hospi- (on the x86). In: Proceedings of the 14th ACM Con- tal/116314. Accessed Aug. 2, 2018. ference on Computer and Communications Security, Oct. 29 through Nov. 2, 2007, Alexandria, VA. New 12. Fox-Brewster T. Medical Devices Hit by Ran- York: ACM; 2007:552–61. somware for the First Time in US Hospitals. Available at: www.forbes.com/sites/thomasbrew- 22. King A, Arney D, Lee I, et al. Prototyping closed ster/2017/05/17/wannacry-ransomware-hit-re- loop physiologic control with the medical device al-medical-devices. Accessed Aug. 2, 2018. coordination framework. In: Proceedings of the 32nd International Conference on Software Engi- 13. Department of Homeland Security. Advisory neering, May 1–8, 2010, Cape Town, South Africa. (ICSMA-17-250-02A): Smiths Medical Medfusion New York: ACM; 2010:1–11. 4000 Wireless Syringe Infusion Pump Vulnerabil- ities (Update A). Available at: https://ics-cert.us- 23. AS5506. Architecture Analysis & Design Language cert.gov/advisories/ICSMA-17-250-02A. Accessed (AADL). Warrendale, PA: SAE International; Aug. 17, 2018. 2004.

14. Department of Homeland Security. Advisory 24. Software Engineering Institute. Open Source (ICSMA-18-037-02): GE Medical Devices Vulnera- AADL Tool Environment (OSATE). Available at: bility. https://ics-cert.us-cert.gov/advisories/ICS- http://osate.org. Accessed Aug. 17, 2018. MA-18-037-02. Accessed Aug. 17, 2018. 25. SAE International. SAE Architecture Analysis 15. Larson BR, Jones P, Zhang Y, Hatcliff J. Princi- and Design Language (AADL) Annex Volume ples and Benefits of Explicitly Designed Medical 1: Annex A: ARINC653 Annex, Annex C: Code Device Safety Architecture. Biomed Instrum Generation Annex, Annex E: Error Model Annex Technol. 2017;51(5):380–9. AS5506/1A. Available at: www.sae.org/standards/ content/as5506/1a. Accessed Aug 17, 2018. 16. Xen Project. Homepage. Available at: www.xen- project.org. Accessed Aug. 2, 2018. 26. Genode Labs. Documentation of the Genode OS Framework. Available at: https://genode.org/ 17. Carpenter T, Hatcliff J, Vasserman EY. A refer- documentation/index. Accessed Aug. 2, 2018. ence separation architecture for mixed-criticality medical and iot devices. In: Proceedings of the 1st ACM Workshop on the Internet of Safe Things, Delft, Netherlands, Nov. 6–8, 2017. New York: ACM; 2017.

18. Heiser G, Elphinstone K. L4 Microkernels: The Lessons from 20 Years of Research and Deploy- ment. ACM Transactions on Computer Systems. 2016:34(1):1–30.

19. Linux Syscall Reference. Available at: http:// syscalls.kernelgrok.com. Accessed Aug. 2, 2018.

www.aami.org/bit 319