12 an Exhaustive Survey on Security Concerns and Solutions at Different

An Exhaustive Survey on Security Concerns and Solutions at Different Components of Virtualization

RAJENDRA PATIL and CHIRAG MODI, National Institute of Technology Goa, India

Virtualization is a key enabler of various modern computing technologies. However, it brings additional vulnerabilities that can be exploited to affect the availability, integrity, and confidentiality of the underlying resources and services. The dynamic and shared nature of the virtualization poses additional challenges to the traditional security solutions. This article explores the vulnerabilities, threats, and attacks relevant to virtualization. We analyze the existing security solutions and identify the research gaps that can help the research community to develop a secured virtualization platform for current and future computing technologies. CCS Concepts: • Security and privacy → System security; Operating system security; Virtualization and security; Network security; Denial-of-service attacks; Additional Key Words and Phrases: Virtualization, hypervisor, virtual machine, vulnerability, security ACM Reference format: Rajendra Patil and Chirag Modi. 2019. An Exhaustive Survey on Security Concerns and Solutions at Different Components of Virtualization. ACM Comput. Surv. 52, 1, Article 12 (February 2019), 38 pages. https://doi.org/10.1145/3287306 12

1 INTRODUCTION Virtualization is an abstraction of hardware and software resources allowing heterogeneous archi- tectures to run on the same hardware [9]. The major components of the virtualization are virtual machines (VMs), hypervisors, and virtual networks. VM is a software system similar to the physical system allowing us to run an operating system and other applications. Hypervisor offers an “abstraction of physical resources” like CPU, memory, network, and storage. It allows us to run multiple operating systems at a time on the same physical resources. It can be installed directly on hardware (Type I) or as a part of host OS (Type II). A virtual network enables communication among the VMs through virtual switch (vSwitch). In past decade, virtualization-based applications have increased by a tremendous amount. How- ever, the current implementations of the virtualization introduce many vulnerabilities and security challenges. In the National Vulnerability Database (NVD), several vulnerabilities related to well- known hypervisors such as XEN, VMware, and Qemu have been recorded. The successful exploitation of these vulnerabilities leads to many attacks, which can affect the confidentiality, integrity,

This work is a part of the project titled “Designing out-of-VM Monitoring based Virtual Machine Introspection Frame- work for Securing Virtual Environment of Cloud Computing (ECR/2017/001221)” with funding support from Science and Engineering Research Board, Department of Science and Technology, Government of India. Authors’ address: R. Patil and C. Modi, National Institute of Technology Goa, Ponda, Goa 403401, India, emails: {rajendrapatil, cnmodi}@nitgoa.ac.in. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) mustbe honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. 0360-0300/2019/02-ART12 $15.00 https://doi.org/10.1145/3287306

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:2 R. Patil and C. Modi

Table 1. Key Terms

Term Description Control VM It is a VM with higher privileges, e.g., Dom0 in Xen Rouge VM VM with abusive administrator Co-hosted VM VM which has shared resources with other VMs on the same physical server Hypervisor Management It manages the hypervisor and VM building functionalities Interface (HMI) Launching channel It is a communication channel from the origin of the VM image to the destination hypervisor where image is to be deployed Hypervisor Layer This layer enables the VMs to run on a physical server VM Management Layer It manages the Virtual Machines VM (OS) Kernel Layer It builds the Virtual Machine Virtual Machine Layer It contains all VMs on a single physical server VM Network Layer It makes VMs to communicate with each other through virtual switch Trusted Computing Base It is a combination of hardware and software components that defines the trusted (TCB) computing environment Hardware VM (HVM) It require virtualization extensions from the host CPU and thus, using Qemu (Quick emulator), it emulates the physical hardware Paravirtualized VM (PVM) It can run efficiently without hardware emulation. or availability (CIA) of the hypervisor or its underlying VMs. For instance, critical VENOM vulnerability (CVE-2015-3456) in the open-source Qemu hypervisor allows an attacker to break out a VM, execute code on a host machine, and access all the other VMs on the host. This leads to hundreds or thousands of virtualization products being vulnerable to VENOM. The Foreshadow vulnerability (CVE-2018-3646) in XenServer allows an attacker to create a speculative side channel and steal data in VM RAM from other non-trusted VMs on the same physical server. Kaspersky Lab has analyzed that the cost of recovering from a security incident doubles when the attack affects virtual infrastructure. Thus, the virtualization environment needs to be secured. This article explores various vulnerabilities, security threats, and attacks related to virtualization and classify them according to the core components of the virtualization infrastructure. We investigate the existing security approaches and techniques for each component and explore the research gaps. By considering these research gaps, we provide possible security research directives that will help the research community to address the needs of virtualization security and to design a secured and trusted virtualization platform for future computing technologies. Table 1 presents different key terms that we have used in this article. In following, Section 2 discusses the vulnerabilities, threats, and attacks to virtualization. Sec- tion 3 analyzes the existing security solutions and challenges to the virtualization. Section 4 discusses the research gaps in the existing solutions and provides future research directives for offer- ing better security in virtualization. Finally, Section 5 concludes our findings.

2 SECURITY CONCERNS AT VIRTUALIZATION AsshowninFigure1, the virtualization environment includes the physical servers and other related components such as the management server, the VM repository, and networks. A physical server is virtualized using the hypervisor, which provides the abstractions of the physical resources. It has five components: VM, hypervisor, virtual network, host OS (control VM inthe case of a Type 1 hypervisor), and underlying hardware. A hypervisor runs multiple VMs on the same hardware. A VM or applications running on VMs can be accessed by an end user through the Internet. A management server is used to handle the management-related tasks of the virtualization. The VM repository stores the VM images, which are managed through the interaction

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:3

Fig. 1. Components of virtualization and attack surface. with the management server. A VM owner can launch a VM image to a hypervisor through the interaction with the management server. The network component presents typically three different networks: virtual network, internal network, and external network. VMs are connected over vSwitch, which forms the virtual network. The internal network is responsible for connecting the physical servers, storage servers, and management server. The external network connects the virtual environment to the external users through the Internet. The security threats to the virtualization are due to the vulnerabilities existing in the different components and operations involved in the virtual environment. Figure 2 shows the taxonomy of virtualization-related attacks. An attack surface is mapped from Figure 1 to Figure 2 to discuss the possible attacks from a particular source to a victim.

2.1 Security Challenges to Hypervisor Hypervisor enables the virtualization, and it has different components. For example, the Xen (Type 1) hypervisor contains a hypervisor core, management interface/privileged domain (Dom0), Qemu, and guest VMs. the KVM (Type 2) hypervisor interacts with the host OS, which is an additional component of KVM. The hypervisor core controls the execution of VMs by providing hardware abstraction. It performs CPU scheduling and memory partitioning for the VMs running on the hardware device. The hypervisor management interface carries all management actions of a VM. In a type 1 hypervisor, it is a control VM (Dom0), while in a type 2 hypervisor, it is a trusted process of the host OS. Dom0 is a privileged VM that has special rights to access I/O resources. Xen supports two types of VM: paravirtualized VM (PVM) and hardware VM (HVM). For guests running as HVMs, Xen uses Qemu, which runs in Dom0 to provide device emulation. For hypervisor versions after Xen 3.3, Qemu is considered as a large attack surface. Thus, it is moved outside of Dom0 and placed in stub domain for each guest. Unlike Xen, KVM relies entirely on Qemu for device emulation. It has been observed that an attacker can target each of these components. A compromised hypervisor can provide the control of all the underlying VMs. Thus, the most general way to attack a virtual environment is to gain access of the hypervisor.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:4 R. Patil and C. Modi

Fig. 2. Taxonomy of virtualization-related attacks.

2.1.1 Vulnerabilities. Hypervisor-related attacks are possible due to many vulnerabilities like uncontrolled flexibility to create VMs, misconfiguration, bugs and poor design, weak control over privileged and management interfaces, uncontrolled resource allocation to VM, and so on. In past 2 years (Jan 2016–Dec 2017), around 90 Xen hypervisor-related vulnerabilities have been recorded in NVD. We have classified Xen-related vulnerabilities by considering the source and target, as showninTable2. The root cause of Xen vulnerabilities is given in Table 3. In the past two years (Jan 2016–Dec 2017), around 174 Qemu+KVM–related vulnerabilities have been recorded in NVD. By considering guest VM and remote attacker as the sources and hypervisor, host OS, and guest VM applications as targets, we have classified these vulnerabilities, as shown in Table 4.Theroot cause of Qemu+KVM vulnerabilities is given in Table 5. Here, we have considered four types of vulnerabilities: Denial of Service (DoS), Gain Privilege (GP), Gain Information (GI), and Code Execution (CE).

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:5

Table 2. Source- and Targetwise Distribution of Xen Vulnerabilities

Source Target Type Vulnerabilities recorded in NVD Guest Hyper- Dos CVE-2017-17566, CVE-2017-17565, CVE-2017-17564, CVE-2017-17563, CVE-2017-17045, User, visor CVE-2017-17044, CVE-2017-15597, CVE-2017-15596, CVE-2017-15595, CVE-2017-15594, Guest CVE-2017-15593, CVE-2017-15592, CVE-2017-15591, CVE-2017-15590, CVE-2017-14431, Kernel, CVE-2017-12136, CVE-2017-12135, CVE-2017-12134, CVE-2017-10923, CVE-2017-10922, Qemu CVE-2017-10921, CVE-2017-10920, CVE-2017-10919, CVE-2017-10917, CVE-2017-10914, Stub- CVE-2016-10025, CVE-2016-10024, CVE-2016-9818, CVE-2016-9817, CVE-2016-9816, domain CVE-2016-9815, CVE-2016-9385 CVE-2016-9383, CVE-2016-7154, CVE-2016-7094, CVE-2016-6259, CVE-2016-3960, CVE-2016-2270, CVE-2016-1571, CVE-2016-1570, CVE-2015-8615, CVE-2015-8552, CVE-2015-8551, CVE-2015-8550 GP CVE-2017-17566, CVE-2017-17564, CVE-2017-17563, CVE-2017-17045, CVE-2017-15595 CVE-2017-15594, CVE-2017-15592, CVE-2017-15590, CVE-2017-12137, CVE-2017-12136, CVE-2017-12135, CVE-2017-12134, CVE-2017-10914, CVE-2017-10913, CVE-2016-7093, CVE-2016-7092, CVE-2016-6258, CVE-2016-4962, CVE-2016-4480, CVE-2016-3960, CVE-2016-3157, CVE-2016-1570 CVE-2015-8554, CVE-2015-8550 GI CVE-2017-17046, CVE-2017-17045, CVE-2017-15597, CVE-2017-15589, CVE-2017-12855, CVE-2017-12135, CVE-2017-12134, CVE-2017-10917, CVE-2017-10916, CVE-2017-10914, CVE-2017-10913, CVE-2017-10911, CVE-2017-7995, CVE-2016-9932, CVE-2016-9384, CVE-2016-9383, CVE-2016-7154, CVE-2016-3159, CVE-2016-3157, CVE-2016-1570, CVE-2015-8553 CE CVE-2017-15588, CVE-2017-8905, CVE-2017-8904 CVE-2017-8903, CVE-2016-9383, CVE-2016-7154 Dom0, Dos/ CVE-2016-9603, CVE-2017-2615, CVE-2017-2620, CVE-2016-5403, CVE-2016-9381, Tool- GP/GI CVE-2016-9637, CVE-2015-8554, CVE-2015-7504, CVE-2017-14317, CVE-2016-4963, stack CVE-2016-4962, CVE-2016-5242 Guest Dos/ CVE-2017-8905, CVE-2017-10911, CVE-2017-10916, CVE-2016-6259, CVE-2016-3158, VM GP/GI CVE-2017-12134, CVE-2017-12855, CVE-2017-15589, CVE-2017-17046 Guest Guest Dos/ CVE-2016-2271, CVE-2016-3157, CVE-2016-3961, CVE-2016-4480, CVE-2016-3712, User OS GP/GI CVE-2016-7777, CVE-2016-9386, CVE-2016-9382, CVE-2016-9377, CVE-2016-9378 CVE-2016-10013 DoS: Denial of Service, GP: Gain Privileges, GI: Gain Information, CE: Code Execution

2.1.2 Threats. The vulnerabilities in a hypervisor can be exploited through external interfaces of the hypervisor resulting into some common threats: insertion of malware or rootkits, management interface compromise, uncontrolled growth of VMs, unauthorized access to hypervisor resources, denial of service through resource starvation by VM, network threats, launching rouge VM, and so on. A threat model to hypervisor is shown in Figure 3. A hypervisor interacts with the following different components: hypervisor code, hypervisor management interface/control VM/insider, VMs, networks, and VM user. A hypervisor code may be infected with the rootkits, which can violate the integrity of the hypervisor. A control VM may be exposed to different interfaces such as drivers, insider, and remote access. A VM, malicious insider, or a remote attacker can compromise the control VM. It can lead to taking control over management functionalities of the hypervisor. A hypervisor runs the VMs, which may be exposed to the Internet. A running VM can violate the confidentiality, integrity, as well as availability of the hypervisor. Since hypervisor implements a VM migration functionality, a malicious hypervisor in the network may compromise its migration process. A VM user may be malicious and can launch rouge VM on the hypervisor.

2.1.3 Attacks. Since the hypervisor is the core component in virtualization, the attacks related to it have a serious impact on virtualization security.

(1) Hyperjacking through VM-based rootkit (VMBR). This aims to violate the integrity of the hypervisor and is typically launched against type 2 hypervisors. Here an attacker attempts

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:6 R. Patil and C. Modi

Table 3. Root Cause of Xen Vulnerabilities

Root cause Vulnerabilities recorded in NVD Insufficient validation of CVE-2015-8615, CVE-2016-4962, CVE-2014-3672, CVE-2016-6259, CVE-2016-9384, guest input CVE-2016-9932, CVE-2016-9815, CVE-2016-9816, CVE-2016-9817, CVE-2016-9818 Insufficient bug check CVE-2016-1570, CVE-2016-4963, CVE-2016-5403, CVE-2016-7094, CVE-2016-9386, by hypervisor CVE-2016-9377, CVE-2016-9378, CVE-2016-10025, CVE-2017-10913, CVE-2017-10914, CVE-2017-10919, CVE-2017-14316, CVE-2017-14318, CVE-2017-14319, CVE-2017-15591, CVE-2017-17565 Incorrect error handling CVE-2017-10918 , CVE-2017-17044, CVE-2017-17045, CVE-2017-17563, CVE-2017-17564 Page fault CVE-2016-3961, CVE-2016-4480, CVE-2017-8903 , CVE-2017-8904, CVE-2017-10912, CVE-2017-10920,CVE-2017-10921,CVE-2017-10922, CVE-2017-15595, CVE-2017-15588, CVE-2017-15593, CVE-2017-17566 Shadow memory flaws CVE-2016-3960, CVE-2017-10915, CVE-2017-15592 PCI device config. error CVE-2015-8554, CVE-2017-15590 Bug in domain building CVE-2016-5242, CVE-2017-14317 Hypervisor memory CVE-2016-3712, CVE-2016-7093, CVE-2016-7154, CVE-2016-9385, CVE-2016-9383, corruption CVE-2016-9381, CVE-2016-10024, CVE-2016-9603, CVE-2017-8905, CVE-2017-12855, CVE-2017-15597 Unauthorized memory CVE-2017-2615, CVE-2017-2620, CVE-2017-7228, CVE-2017-10911, access CVE-2017-10918, CVE-2017-10923, CVE-2017-17046 Code emulation CVE-2015-7504, CVE-2016-7777, CVE-2016-9379,CVE-2016-9380, CVE-2016-10013 Data remanence CVE-2015-8555, CVE-2017-15589 Hardware fault CVE-2016-1571, CVE-2016-2271, CVE-2016-3158,CVE-2016-3159 Privilege escalation CVE-2016-3157, CVE-2016-6258, CVE-2016-7092, CVE-2016-9382, CVE-2016-9637, CVE-2017-12137, CVE-2017-15594 Other CVE-2017-10916, CVE-2017-10917, CVE-2017-12135, CVE-2017-12136, CVE-2017-12134, CVE-2017-15596

to get control over a hypervisor. Through hyperjacking, an attacker can inject a VMBR between hardware and the OS. It can run as a thin rogue hypervisor under the legitimate infrastructure (hypervisor or OS). Such a rogue hypervisor can take complete control of the host OS or hypervisor to control all the interactions between legitimate hypervisor and hardware. For example, SubVirt [55] installs a hypervisor below the host OS and controls the VM. Blue Pill [32] exploits hardware extensions in the virtualization enabled CPUs and runs an infected system into a VM. (2) Attacks from the compromised management interface/Malicious insider. In Xen HVM mode and in KVM, Qemu flaws affect Xen security. A privileged user with the guest VMcan target vulnerabilities (e.g., heap overflow) in the device model process (Qemu) running in Dom0 and can gain privileges of Dom0 (CVE-2016-9603) and (CVE-2017-2615). Thus, the recent research [30, 97] states that the Dom0 cannot be considered in the hypervisor TCB. Thus, different attacks are possible to the hypervisor from the Dom0. A. Compromising CIA of DomUs. A VM user always trust the VM provider and thus the hypervisor. If Dom0 is compromised, then it can compromise the CIA of the DomUs. B. Direct Memory Access (DMA) attack. This aims to violate the integrity of hypervisor. Since Dom0 can communicate with hardware devices directly, it can access the main memory of hypervisor through DMA transfer. If the physical memory address of the hypervisor is known to Dom0, then it can read and write the data structures of the hypervisor. In addition, it can overwrite parts of the main memory to insert the rootkit.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:7

Table 4. Source- and Targetwise Distribution of Qemu+KVM Vulnerabilities

Source Target Type Vulnerabilities recorded in NVD Guest Hyper- Dos CVE-2017-17381, CVE-2017-15289, CVE-2017-13711, CVE-2017-13673, CVE-2017-13672, User, visor CVE-2017-12809, CVE-2017-11434, CVE-2017-10806, CVE-2017-10664, CVE-2017-9524, Guest CVE-2017-9503, CVE-2017-9375, CVE-2017-7718, CVE-2017-6058, CVE-2017-5987, Kernel CVE-2017-5973, CVE-2017-5957, CVE-2017-5931, CVE-2017-5667, CVE-2016-10029, CVE-2016-10028, CVE-2016-9922, CVE-2015-8613, CVE-2016-9104, CVE-2016-8669, CVE-2016-8668, CVE-2016-8667, CVE-2016-8578, CVE-2016-8576, CVE-2016-7909, CVE-2016-7908, CVE-2016-7907, CVE-2016-7423, CVE-2016-7422, CVE-2016-7421, CVE-2016-7170, CVE-2016-7157, CVE-2016-7156, CVE-2016-7155, CVE-2016-6888, CVE-2016-6835, CVE-2016-6834, CVE-2016-6833, CVE-2016-6490, CVE-2016-6351, CVE-2016-5403, CVE-2016-5338, CVE-2016-5238, CVE-2016-5126, CVE-2016-5107, CVE-2016-5106, CVE-2016-4964, CVE-2016-4952, CVE-2016-4454, CVE-2016-4453, CVE-2016-4441, CVE-2016-4439, CVE-2016-4002, CVE-2016-4001, CVE-2016-3712, CVE-2016-2858, CVE-2016-2857, CVE-2016-2841, CVE-2016-2538, CVE-2016-2392, CVE-2016-2391, CVE-2016-1714, CVE-2016-1568 GP CVE-2017-8284, CVE-2016-9381 GI CVE-2017-15038, CVE-2016-9908, CVE-2016-9845, CVE-2016-9103, CVE-2016-6836, CVE-2016-5337, CVE-2016-5105, CVE-2016-4454, CVE-2016-4020 CE CVE-2017-14167, CVE-2017-7980, CVE-2017-5931, CVE-2017-5667, CVE-2015-7504, CVE-2014-0145, CVE-2016-7161, CVE-2016-6351, CVE-2016-5338, CVE-2016-5126, CVE-2016-4439, CVE-2016-4002, CVE-2016-3710, CVE-2016-1714, CVE-2016-1568 Host Dos/ CVE-2017-9374, CVE-2017-9373, CVE-2017-9330, CVE-2017-9310, CVE-2017-9060, GP/ GI CVE-2017-8379, CVE-2017-8309, CVE-2017-8112, CVE-2017-8086, CVE-2017-7980, CVE-2017-7377, CVE-2017-6505, CVE-2017-5857, CVE-2017-5856, CVE-2017-5579, CVE-2017-5578, CVE-2017-5552, CVE-2017-5526, CVE-2017-5525, CVE-2016-10155, CVE-2015-8568, CVE-2015-8567, CVE-2016-9916, CVE-2016-9915, CVE-2016-9914, CVE-2016-9913, CVE-2016-9106, CVE-2016-9105, CVE-2016-9102, CVE-2016-9101, CVE-2016-8910, CVE-2016-8909, CVE-2016-8577, CVE-2016-7995, CVE-2016-7994, CVE-2016-7466, CVE-2016-5403, CVE-2016-4037, CVE-2015-8558, CVE-2015-1779 Guest Guest Dos/ CVE-2017-11334, CVE-2017-5898, CVE-2015-8619, CVE-2015-8504, CVE-2015-8345, User VM GP/ GI CVE-2015-7504, CVE-2014-0146, CVE-2014-0145, CVE-2014-0143, CVE-2014-0142, CVE-2015-8818, CVE-2015-7512, CVE-2015-5158 DoS: Denial of Service, GP: Gain Privileges, GI: Gain Information, CE: Code Execution

C. VM sprawl attack. This aims to violate the availability of the hypervisor. A hypervisor offers the flexibility to create new VMs. However, such flexibility carries a potential risk, like VM sprawl. A malicious insider with an administrative access to management interface can create large numbers of VMs. Thus, at a particular time, large numbers of VMs are very difficult to monitor and manage. (3) Attacks from the VM. Here we assume that a VM image launched to the hypervisor is secure. However, the risk to the hypervisor remains due to the insecure interface or malicious user at the running VM. Table 2 and Table 4 show most of the hypervisor vulnerabilities that are exploited from the guest VM. D. Code execution on Hypervisor. A malicious VM user can modify arbitrary memory (heap memory) of hypervisor and perform arbitrary code execution. E. VM (to Hypervisor) escape. It takes an advantage of isolation failure between the hypervisor and VMs. Here a rouge VM completely bypasses the access control function of the hypervisor and gain privileges of the hypervisor. Thus, a rouge VM can control other VMs and the hypervisor itself. F. Hypervisor crash (DoS). A VM may corrupt the hypervisor memory and cause the hypervisor to crash. G. Hypervisor information leakage. A VM can arbitrarily access the hypervisor memory, which results in information leakage of the hypervisor and other VMs.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:8 R. Patil and C. Modi

Table 5. Root Cause of Qemu+KVM Vulnerabilities

Root cause Vulnerabilities recorded in NVD Out-of-bounds read CVE-2017-13672, CVE-2017-11434, CVE-2017-7718, CVE-2016-10029, access CVE-2016-10028, CVE-2016-9104, CVE-2016-8668, CVE-2016-5107, CVE-2016-4454, CVE-2016-3712 Out-of-bounds array CVE-2017-16845, CVE-2017-11334, CVE-2017-6058, CVE-2016-4952 Divide-by-zero error CVE-2017-17381,CVE-2016-9922,CVE-2016-8669,CVE-2016-8667 Out-of-bounds write CVE-2017-15289, CVE-2015-8619, CVE-2016-7423, CVE-2016-7170, CVE-2016-6351, access CVE-2016-5238, CVE-2016-5106, CVE-2016-4441, CVE-2016-4439, CVE-2016-1714 Remote VNC client CVE-2017-15124 , CVE-2017-7980 Illegal access to host CVE-2017-15038, CVE-2017-5931, CVE-2017-5667, CVE-2016-9103, CVE-2016-7161, heap memory CVE-2016-4020, CVE-2016-2857 NULL pointer CVE-2017-12809, CVE-2017-9503, CVE-2014-0146, CVE-2016-8578, dereference CVE-2016-7422,CVE-2016-2392, CVE-2016-2391, CVE-2016-1922 Infinite loop CVE-2017-9330, CVE-2017-9310, CVE-2017-6505, CVE-2017-5987, CVE-2017-5973, CVE-2015-8345, CVE-2016-9776, CVE-2016-8576, CVE-2016-7909, CVE-2016-7908, CVE-2016-7421, CVE-2016-7156, CVE-2016-7155, CVE-2016-6834, CVE-2016-6490, CVE-2016-4964, CVE-2016-4453, CVE-2016-4037, CVE-2016-2841, CVE-2015-8558 Memory consumption CVE-2017-9374, CVE-2017-9373, CVE-2017-9060, CVE-2017-8379, CVE-2017-8309, CVE-2017-8086, CVE-2017-7377, CVE-2017-5857, CVE-2017-5856, CVE-2017-5579, CVE-2017-5578, CVE-2017-5552, CVE-2017-5526, CVE-2017-5525, CVE-2016-10155, CVE-2015-8568, CVE-2015-8567, CVE-2016-9916, CVE-2016-9915, CVE-2016-9914, CVE-2016-9913, CVE-2016-9106, CVE-2016-9105, CVE-2016-9102, CVE-2016-9101, CVE-2016-8577, CVE-2016-7995, CVE-2016-7994, CVE-2016-7466, CVE-2016-5403 CPU consumption CVE-2017-8112, CVE-2016-8910, CVE-2016-8909, CVE-2016-4964, CVE-2016-4037 Qemu Device memory CVE-2016-9912, CVE-2016-9912, CVE-2016-9908, CVE-2016-9907,CVE-2016- leakage 9846,CVE-2016-9845,CVE-2016-2538,CVE-2015-8743,CVE-2015-8701 Qemu emulator DoS CVE-2016-2198, CVE-2016-2197, CVE-2016-1981,CVE-2016-1568, CVE-2015-8745

Fig. 3. Threat model for hypervisor.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:9

H. Resource starvation. This aims to violate an availability of the hypervisor. The uncontrolled resource allocation allows a VM to consume the resources of the physical server, which results into a denial of service to other VMs. It makes physical resources unavailable to other VMs by flooding a large number of requests. I. Attack on control VM. As control VM shares the resources with DomUs, it can be attacked from the DomU. It aims to take control of the hypervisor. J. Attack on other VMs (VM hopping). It is a process of jumping from one VM to another VM on the same physical server by exploiting hypervisor vulnerabilities. The arbitrary access to hypervisor resources (e.g., memory) may result into attacks like information leakage, traffic analysis, modification of the configuration file, denial of service, and soon,onother VMs. The authors of Reference [49] have demonstrated the VM hopping by maliciously gaining an access to different VMs. (4) Attacks from the malicious hypervisors (networks). The migration module in hypervisor implements the VM migration functionalities. The vulnerabilities like buffer overflow, stack overflow, and so on, in the migration module or bugs in the authentication and access control mechanisms can be exploited through the network to compromise the control plane of a migration process and to perform following attacks. Migrating a compromised VM. For dynamic load balancing, a live migration is automat- ically initiated. An attacker at malicious hypervisor may falsely advertise available resources and influence the control plane to migrate the compromised VM to thetrusted hypervisor. (5) Launching rouge VM. A malicious VM user may launch a malware-infected VM image on a trusted hypervisor. It introduces the new source of an attack in the virtual environment when underlying infrastructure is secured.

2.2 Security Challenges to VM A VM is a potential target to gain an unauthorized access to the hypervisor and other VMs. At particular time, a VM can be in one of the three states: inactive, moving (migrating to other host machine), and running state. In all these states, it is vulnerable to different types of attacks. NVD has recorded following vulnerabilities related to VM: “Xen 3.2.x through 4.4.x does not properly clean memory pages that are recovered from guests, which allows local guest OS users to obtain sensitive information via unspecified vectors (CVE-2014-4021).” “Apache CloudStack before 4.5.2 does not properly preserve VNC passwords when migrating KVM virtual machines, which allows remote attackers to gain access by connecting to the VNC server (CVE-2015-3252).” “BIG-IP and BIG-IQ AWS, Azure, or Verizon cloud instances do not properly regenerate certificates and keys when deploying VM image, which makes multiple instances to share the same certificates and keys. It causes the disruption of services and/or an information leak (CVE-2016-2084).”

2.2.1 Security Challenges to VM in Running State. In virtualization, a VM in the running state is a potential target since it has interfaces that are exposed to an attacker. Vulnerabilities. VM in running state includes vulnerabilities like poor isolation between VM and hypervisor, poor access control over management interface, uncontrolled VM rollback, default state of new VMs, poor isolation for shared resources, network vulnerabilities, and so on. Threats. The possible threats are rootkit insertion in a VM, illegal access from the hypervisor management interface or a malicious insider, network threats, threats from the rouge VM, isolation failure among the VMs. A threat model to the VM running state is shown in Figure 4. A running VM interacts with different components: Hypervisor, Hypervisor management interface, VM kernel code, Co-hosted VMs and Networks. A hypervisor may be exposed to perform different attacks on

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:10 R. Patil and C. Modi

Fig. 4. Threat model for VM running state. the VMs. A control VM may be exposed to different interfaces such as drivers, insider and a remote access interface. A VM, malicious insider or a remote attacker can compromise the control VM. A VM instance may be created from the rootkit infected VM image that can tamper the kernel of a VM. A VM kernel can be compromised,which can tamper with the execution of a trusted application. A VM co-hosted with other VMs may be exposed to the Internet. A co-hosted VM can perform different attacks on other VMs that share the underlying resources. A VM network traffic may be exposed to a network attacker. Attacks on a VM in running state. (6) Attacks from the compromised hypervisor. Here we consider both type 1 and type 2 hypervisor. In case of type 1 hypervisor, we assume that the control VM is in the TCB of the hypervisor. The VMs running on a compromised hypervisor are under the security risk. Here CIA of the VMs can be compromised. (7) Attacks from the compromised management interface/malicious insider. The trusted computer base (TCB) of type 1 hypervisor includes not only the hypervisor but also the control VM. Control VM can control the access to the physical resources by other VMs (Do- mUs) and can control all these DomUs. Thus, we assume that the hypervisor is trusted; however, the code instantiating the functionality of the Dom0 contains bugs. In addition, in case of both the hypervisors, the remote access of management interface can be compromised through the web application attacks (CSS and SQL injection). It exposes the VM management actions to a remote attacker. For example, “CSS vulnerability (CVE-2012- 5050) in VMware vCenter Operations before 5.0.x allows remote attackers to inject arbitrary web script via unspecified vectors and take control of vCenter.” In addition, management interface can be directly accessed by a malicious insider [52, 84]. The successful compromise of a management interface can lead to following attacks on the VMs [56]. K. VM privacy violation. Since Dom0 has read access to the memory of a running VM, it is possible to extract sensitive information in clear text through memory snapshots. A malicious Dom0 allows us to get passwords, cryptography keys, files, and other confidential data [84]. It uses a cold boot attack on the memory snapshot of the target VM. ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:11

The permissive access by Dom0 to the VM memory allows a malicious insider to capture the information in regular intervals to read the data exchanged between VMs [85]. L. VM integrity violation. Since Dom0 has write access to the memory of a running VM, it can inject arbitrary code in a VM. M. VM network traffic sniffing/spoofing. Dom0 can intercept all network traffic before it gets to DomU. Here network traffic of a particular VM can be sniffed to read thecom- munication with the paired destination or to perform MITM attacks. N. VM rollback. The rollback operation allows us to restore a VM to the previous state. It can disable the security measures that were previously enabled. In a VM rollback attack, a compromised hypervisor management interface executes a VM from the previous snapshot [129]. An attacker can use a brute-force technique to find the login password of a VM. O. VM relocation. With the evolution of integrity protected hypervisors, the features like memory snapshots and disk accesses are disabled. However, for the essence of load balancing, virtualization provides a functionality where VM can be relocated to another server. The attacker can find a malicious server to relocate the VM, where it is possible to take memory snapshot of a VM [95]. P. VM cloning. As VM is a software, it can be backed up, copied, and moved to other servers. A VM owner is unaware of the number of running instances of a VM. A malicious hypervisor management interface can copy the VM and start it on a compromised hypervisor. It can have complete access to the VM. (8) Kernel-level attack. A VM instance created through an infected VM image allows viruses, rootkits, and other malware to do substantial damage to a VM. Q. Kernel rootkits. It may get an administrative privileges and can tamper the VM kernel. Here, we see the severity of the kernel rootkits. —Kernel code modification. An attacker can modify the OS kernel to insert rootkits. The goal of such modification is to run the rootkits at an administrator level. —Kernel data structure modification. A rootkit can use memory corruption vulnerability to corrupt the data. It modifies the pointers and data directly in the kernel data structures. Direct Kernel Object Modification (DKOM) attack16 [ ] changes the accounting table information in the kernel and completely bypasses the object manager. Direct Ker- nel Structure Manipulation (DKSM) attack [8] exposes the security code hosted in a VM by changing the syntax of the kernel structure. —Kernel control-flow modification. A return-oriented rootkit [22] can bypass the code integrity protection and compromises the kernel code execution (control flow) of the running processes. Control-flow hijacking attack exploits a memory corruption bug to execute a malicious code in address space of a valid user process. An attack like binary code reuse redirects the control-flow to the existing executable code. R. Iago attack. In rootkit-based attack, it is assumed that the kernel is trusted, and rootkits may tamper its execution. However, commodity OS kernels are frequently prone to compromise and, thus, the reverse is also true, where kernel is untrusted and may tamper the execution of a trusted application. In an Iago attack [23], a malicious kernel makes an application to act against its interests by communicating with it. Applications interact with the kernel through the system call interface. Legacy applications generally do not check return values from the kernel. Thus, kernel can return crafted values (e.g., new memory mappings) to corrupt application behavior. S. Hypervisor to application tampering. The applications can be tampered by an attacker having an access to the hypervisor.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:12 R. Patil and C. Modi

(9) Illegal access from the co-hosted VMs. A VM isolation failure can lead to the unexpected side channel and covert channel among the VMs. The vulnerabilities exposed due to the shared resources allow an attacker to co-locate a VM with the target the VMs and then to build a side/covert channel with target VMs. The co-hosted VMs are also vulnerable to the RowHammer attack [94]. The repeated access to a memory can cause bit flips in adjacent rows that may belong to another VM. T. VM side-channel attack. A side channel allows us to extract confidential information of the target VM. It uses the shared resources such as the CPU load, cache, memory, and so on, to extract the resource consumption status, cryptography keys, and other sensitive information of a target VM. Here an attacker does not need to share the address space with the target VM; rather, he or she shares hardware caches to access its memory. To enable side-channel attacks, it is required to detect the co-location of the target VM. The authors of Reference [83] have identified whether a particular VM is likely to reside on then same physical server. In addition to co-location detection, L2 cache-based side-channel attack is demonstrated, which causes information leakage across VMs on Amazon EC2. Similarly, HomeAlone [139] have used Prime+Probe technique on L2 cache to detect co-location on Xen. The authors of Reference [140] have extracted the ElGamal secret key that is used for GNU Privacy Guard decryption performed in another VM by monitoring the L1 cache timing. The authors of Reference [121] have targeted a cache timing attack to extract AES keys of a VM running on an ARM Cortex-A8 processor. The authors of Reference [39] have demonstrated a side-channel attack to recover AES keys in Xen and VMWare. L1 and L2 cache-based attacks are extremely difficult to perform, as these caches are private to the cores and thus, an attacker needs to exploit the vulnerability of the hypervisor scheduler to perform cross-VM attacks. In addition, attacker VM and target VM should co-reside on the same core. However, as the “last-level cache (LLC)” is shared among all cores, LLC-based side-channel attacks can be performed when the victim and attacker are not executing on the same core. Flush+Reload [134] observes the information leak through shared pages in Intel X86 processors and extracts GNU Privacy Guard private keys across multiple processor cores and across VMs. The authors of Reference [11] have used a combination of Flush+Reload and lattice techniques. Flush+Reload technique obtains the information and the lattice techniques to reconstruct secret keys for 256 bit elliptic curves. The Flush+Reload technique is employed in the VMware to recover the AES keys of “OpenSSL 1.0.1” running inside a target VM [47]. It takes an advantage of memory deduplication in the VMware. The authors [46] have exploited access time variations on the L3 cache. Here, huge pages are exploited to work across VM boundaries without requiring the memory deduplication. It requires only machine co-location in a multi-core scenario. It is a variation of L1 cache-based Prime+Probe attack and works like L3 cache-based Flush+Reload attack. The authors of Reference [45] have monitored L3 cache to achieve co-location. Here Prime+Probe technique is used to extract noisy data from a “Amazon EC2” VM. It processes the noisy data to obtain 2048-bit RSA encryption key. Prime+Probe attack translates the virtual addresses to physical addresses. It checks an access to these addresses through another VM to leak the VM information [135]. A Prime+Probe attack against the L3 cache on GnuPG can be used to extract the key of GnuPG crypto software [62]. A Flush+Flush attack [41] uses the execution time of the flush instruction. Since it does not make any memory accesses, there are nocache misses. The constant cache flush reduces the number of cache hits. Therefore, this attack cannot be detected using cache hits and misses. A shared memory-based side-channel

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:13

attack exploits the vulnerabilities of the balloon driver, which is used by the hypervisors to balance physical memory among the VMs [119]. U. VM covert-channel attack. It transfers confidential information, while violating the specified security policies. The authors of Reference71 [ ] have presented a CPU load- based covert-channel attack. The authors of Reference [133] have used L2 cache to build a covert channel to steal the private keys. The authors of Reference [124] have introduced a shared memory-based covert channel in the Xen. The authors of Reference [130]have presented the memory deduplication-based covert-channel attack. Such covert channels are faster than the L2 cache-based covert channels. The authors of Reference [127]have presented a memory bus-based high bandwidth covert channel on the virtualized x86 platform. The authors of Reference [64] have introduced a LLC-based covert channel using Prime+Probe technique. V. RowHammer attack. The authors of Reference [94] have exploited the RowHammer bug in the DRAM devices to gain kernel privileges of same VM. The Rowhammer attack can be performed across the VMs by exploiting the memory de-duplication and flipping memory bits in a controlled79 way[ ]. A malicious VM exploits bit flips induced by RowHammer attack to crack memory isolation enforced by virtualization [131]. It exploits the Xen paravirtualized memory isolation to access arbitrary physical memory of the target VM. In addition, it can exploit bit flips in random memory locations. Itcan create bit flips in cryptographic keys that are stored in memory [12]. For this, it requires only user-level privileges for kernel and memory location of the key. It combines timing Prime + Probe attack and row-buffer collision to induce bit flip faults in a 1,024-bit RSA key on modern processors. The authors of Reference [75] have performed side-channel and covert-channel attacks across processors using RowHammer. The authors of Ref- erence [27] have performed a memory ambush attack similar to a RowHammer attack, which places the hammerable kernel buffers physically adjacent to the target objects (VM memory). (10) Classical network attacks. The newly joined VM typically starts with the open ports, default services, and active protocols, which allows an attacker to perform port scanning, ARP spoofing, IP spoofing, network sniffing, DDoS, andsoon.

2.2.2 Security Challenges to VM in Moving (or Migrating) State. A live migration of VM is an important feature for load balancing, hardware maintenance, and so on. However, it is susceptible to severe security risks such as a network sniffing, malicious code injection. An attacker may place himself in the migration transit path and can perform a MITM attack.

(11) MITM attack. The migration traffic can be evesdropped, which results into the leakage of current status of the running applications and sensitive information. In addition, it is possible to modify VM contents and to inject rootkits in VM code.

2.2.3 Security Challenges to a VM in Inactive State. A VM image in the repository, during launching or its data on a physical device, is considered as an inactive VM. It includes configuration files to create the VM instances. Vulnerabilities. A VM image in the virtual environment has many vulnerabilities like weak access control, insecure launching channel, untrusted hypervisor, and so on. The presence of data on a physical device after VM release is considered as an additional vulnerability. Threats. The most common threats to VM inactive state are the uncontrolled upload, creation, modification, and usage of VM images. Some others include unauthorized access to alaunching channel and a physical device, deployment of the image to an untrusted hypervisor, and so on. A

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:14 R. Patil and C. Modi

Fig. 5. Threat model for VM inactive state. malicious modification to VM image and storing images with malicious code are serious threats. A threat model to VM inactive state is shown in Figure 5. VM inactive state can be considered through four different states: image in the repository, launching image, running image, and image removal in the virtual environment. The VM images in the repository can be exposed to the external users due to the weak access control and authentication. The VM image launching channel may be exposed to the network, where an attacker can perform a MITM attack. A VM image may be launched to an untrusted hypervisor. Once VM image is removed from the virtualization platform, its hardware can be given to another VM. Consider a new user of VM is malicious, who can violate the confidentiality of the removed VM data. In addition, the outdated software andknown vulnerabilities of image can be exploited. As per VM image state, different attacks are possible.

(12) Attacks on VM image contents —Exploiting outdated software and OS vulnerabilities. Once a VM is created from the image, the outdated software and known OS vulnerabilities can be exploited by an attacker through other VMs or virtual networks. For example, “Directory traversal vulnerability in the TS WebProxy in latest Microsoft OSes like Windows 7 SP1, Windows 8, Windows Server 2012 Gold and R2, etc., allows remote attackers to gain privileges (CVE-2015- 0016).” (13) Attacks on a VM image in repository —Leakage of sensitive information. A user with a valid account at repository can access all images in the repository. He may create a VM instance from the VM image and may recover the passwords and cryptography keys of the previous user, which results into information leakage [42, 120]. —Modification of the VM image. The legitimate VM image can be infected through the insertion of rootkits or malware [40, 74]. A VM instantiated through such an image can introduce malware in virtual environment. It can be leveraged to analyze the VM activi- ties and data, which results in a VM privacy breach. —Malware infected image upload. An attacker can upload a VM image with malware to a public repository due to the lack of proper access control [40, 74, 120].

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:15

—VM image sprawl. As creating a new VM image is just a code, multiple images can be created in the repository through cloning, version, archives, and so on. Here an attacker can create such multiple copies to make the maintenance and security very difficult. This problem is known as VM image sprawl [80]. (14) MITM attack on VM image. A MITM attack is possible on a VM image during its launching from repository to destination hypervisor [74]. (15) Attack on VM image at destination hypervisor. A VM image of a user may be launched on an untrusted hypervisor. The malicious hypervisor may falsely advertise the available resources and make the VM owner deploy an image on it [74]. Since the VM owner cannot determine the trustworthiness of the destination hypervisor, his/her VM instance may run at an unsecured platform. (16) VM data remanence attack. Data remanence is the physical representation of digital data that remains on the physical device even after removal of it. If the same hardware is allocated to an attacker, then VM data remanence allows an attacker to reconstruct the sensitive information of the previous VM. It is possible that data may present on both VM memory and the hard disk [2].

2.3 Security Challenges at Virtual Network Network virtualization allows us to create and run multiple independent virtual networks over the shared infrastructure. In certain situations, the virtual network needs to be reconfigured dynamically. For example, when a VM migrates among the physical servers, its IP needs to be changed and corresponding VLANs need to be reconfigured. This makes virtual networks very complex. A software-defined network (SDN) makes such network reconfigurations easy where it separates the control functions of the network switch and places it in the centralized control plane [48]. It helps to control the data plane of network switches and routers in a software-defined manner. However, the sharing of network infrastructure increases the vulnerabilities in DNS servers, DHCP, IP, ARP protocols, and so on. In addition, there are many vulnerabilities such as vSwitch software bugs, open ports, insecure network channels, and so on. (17) Network attacks. The exploitation of network vulnerabilities leads to different traditional attacks such as Denial of Service (DoS), port scanning, sniffing, IP/MAC spoofing, and so on.

3 EXISTING SOLUTIONS TO VIRTUALIZATION SECURITY There have been many works to date for virtualization security. We classify the existing solutions according to the security of each component in virtualization.

3.1 Hypervisor Security Hypervisor has an interface to its code, management VM, hosted VMs, networks, and VM user (least trusted interface for VM provider). As shown in the threat model of hypervisor (see Figure 3), hypervisor can be attacked from all these interfaces. While protecting hypervisor from particular interface, it is assumed that inner interfaces are trusted and can be used to develop a security solution. Hypervisor code is protected using hardware or a nested hypervisor.

3.1.1 Securing Hypervisor from Rootkits through Integrity Checking. HyperGuard [86]uses SMM of the x86 processor to inspect the integrity of the hypervisor. SMM protects the integrity measurement code using hardware. Similarly, HyperCheck [111] prevents rootkits that target Xen hypervisor integrity. It outsources the analysis of integrity state. In addition, it can monitor the

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:16 R. Patil and C. Modi code and data of the control VM and other guest VMs. In HyperGuard [86] and HyperCheck [111], the measurement agent in the SMM can be tampered. To protect tampering of the measurement agent, HyperSentry [6] uses a software-based integrity measurement agent with an access to the contextual information (correct CPU state) of hypervisor. The agent runs in the hypervisor context and isolated from the hypervisor by adopting a TCB that is composed of hardware and a software component. However, it finds the traces of an attack that has been happened. HyperGuard [86], HyperCheck [111], and HyperSentry [6] are SMM dependent, which can be attacked by the SMM rootkits. In addition, as such a solution provides static hypervisor code integrity, it can be bypassed by the return-oriented rootkits. HyperSafe [116] verifies the control- flow integrity of type 1 hypervisor execution. It includes two techniques: Non-bypassable memory lockdown and Restricted pointer indexing. Non-bypassable memory lockdown provides hypervisor code integrity by locking down memory pages of hypervisor code and static data. The restricted pointer indexing converts the control data into pointer indexes that strictly follow the hypervisor control flow graph. It protects the control data for control-flow integrity by extending thecode integrity. The authors of Reference [145] have measured a checksum of the runtime hypervisor by performing CPU microcode modification. MGUARD [63] prevents any illegal modifications of the hypervisor code. The advanced memory buffer of “FB-DIMM” can be used to integrate apro- grammable guard. The programmable guard continuously monitors the memory traffic between CPU and physical memory. HyperGuard [86], HyperCheck [111], HyperSentry [6], HyperSafe [116], and MGUARD [63]focus on hypervisor control data and cannot protect hypervisor non-control data. HyperVerify [33], a VM-assisted architecture, monitors hypervisor non-control data using a control VM. The hardware state of hypervisor is captured in control VM using programmable device driver. It translates the low-level hardware state into the high-level hypervisor context using a memory analysis library. As per our observation, hypervisor should have an integrity checking mechanism during its deployment and runtime.

3.1.2 Securing Hypervisor from Compromised Management Interface/Malicious Insider. Hyper- visor needs to be protected from the management interface, which can be either compromised by a VM or accessed by a malicious insider/remote attacker. In first case, the research reveals a new tread of disaggregating the management interface of hypervisor. It aims to provide a separate management interface to each VM; however, the execution environment may remain the same. In the second case, researchers use the logging and auditing mechanisms. Disaggregating the management interface. The authors of Reference [68] have moved the VM- building functionality into a separate privileged VM that runs along with Dom0. Nova [103], a thin hypervisor, moves the most hypervisor functionality to the user level. It uses the principle of least privilege to reduce a TCB. Xoar [30] modifies the original Xen design by breaking the control VM into multiple unprivileged service VMs. It modifies the Xen hypervisor without compromising its functionality. It breaks the control VM into multiple service VMs with least privilege. Self-service computing (SSC) [17] splits Dom0 into a systemwide Dom0 and a VM-specific Dom0. It allows access to the VM-specific Dom0 and, thus, hypervisor functionality without breaking the isolation between VMs. HyprBIOS [141] shifts the control VM from the hypervisor TCB. The reduced TCB is further decomposed into two layers. The slave layer is a original hypervisor and runs the VMs; however, it passes the crucial VMEXIT handling to the master layer at SMM. The master layer checks the access control violation by VMEXIT. Privilege separation virtual machine (PSVM) security model [136] allocates a separate Dom0 to each VM. The authors of References [17, 30, 68, 103, 136, 141] have secured hypervisor functionalities, leaving the hypervisor still

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:17 vulnerable through network drivers, and so on. Nexen [97] decomposes Xen into multiple domains such as “a privileged security monitor,” “a shared service domain,” and multiple VM specific Xen slices. Each VM slice is made up of a duplicated Xen code and data. A VM with compromised slice cannot directly access data of other VMs, and, thus, it cannot affect rest of the VMs or hypervisor. Logging and auditing. Although TCB reduction approaches reduce the risk of accessing the hypervisor functionalities by a VM, they cannot protect them from the malicious insider. TVGuarder [58], a trace-enabled protection, detects malicious access to hypervisor functionalities by an insider. It observes the traces of VM operations and allows only legitimate VM operations. It can protect the VM sprawl attack on the hypervisor.

3.1.3 Securing Hypervisor from Rouge VMs. Here, a primary objective is to restrict any illegal access to hardware or hypervisor code from a VM. It aims at either provisioning strong isolation to hypervisor resources from VMs or an isolated execution environment for a VM. sHype [89] adds hooks in the hypervisor to enforce information flow constraints between VMs. It aims to reduce the VM interface to shared resources through the controlled resource sharing between VMs. No- Hype [104] provides a multi-tenant architecture to ensure isolation between VMs. It aims to build an isolated execution of VM by providing fixed cores, pre-allocated memory to a VM. Similarly to NoHype, TrustOSV [113] builds an isolated execution environment for VM using hardware. Unlike NoHype, it minimizes the interaction of management OS with the VMs. Min-V [70]usesadelu- sional boot mechanism to restrict the boot time initialization code or earlier OS code. It aims to reduce the interfaces between the hypervisor TCB and VMs. It disables non-critical virtual devices offered to the VMs. In case of type 2 hypervisor, it is focused on providing an isolated hypervisor environment for each VM. Thus, even if hypervisor is compromised through a VM, it will not affect other VMs. SplitVisor [73] has two layers: SplitVisor and GuestVisor. SplitVisor resides in a trusted computing base and provides isolation between VMs. GuestVisor offers virtual resources to the VMs. Hyper- Lock [118] provides hypervisor isolation using separate “address space and restricted instruction set.” In addition, it creates a separate shadow hypervisor for each VM so that compromised hypervisor can affect only the corresponding VM. DeHype123 [ ] reduces the exposed attack surface of a hosted hypervisor by deprivileging it to user mode. It decouples the hypervisor code from the host OS and makes the hypervisor as a user-level library. It allows concurrent execution of multiple hypervisors on the same physical machine. Similarly, in the case of Xen, Nexen [97]decomposesthe Xen to provide a separate execution environment for each VM.

3.1.4 Securing Hypervisor from Networks (Other Hypervisors). Here it is necessary to secure a hypervisor when it communicates with other hypervisors for VM migration. The authors of Reference [112] have verified the destination hypervisor before it initiates the VM migration. A remote attestation protocol [18] attests the destination hypervisor where VM is to be migrated. The authors of Reference [109] have performed a property-based attestation of destination hypervisor. The authors of Reference [3] have attempted to guarantee the VM migration to a trustworthy hypervisor.

3.1.5 Securing Hypervisor from Malicious VM User. It aims to secure hypervisor from an untrusted launch of a VM image. Cloud Verifier (CV)92 [ ] protects the integrity of VM and host. Here, the integrity of CV is verified by cloud users, and then attestation requests are sent to the hoston which a VM instance is launched. Host checks the integrity of a VM image. The launching of a VM is decided based on VM image integrity. Tenant attested trusted cloud service [81] attests the VM image when it is received by a hypervisor. It uses Integrity Verification and Report Service (IVRS)

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:18 R. Patil and C. Modi

Table 6. Summary of Hypervisor Security Solutions

Approach Summary Limitations Static and dynamic HyperGuard [86] protects the hypervisor HyperGuard [86] secures only hypervisor integrity measurement code. HyperCheck [111]protectsthe code. HyperCheck [111] needs hardware of hypervisor integrity of hypervisor. HyperSentry[6] changes. HyperSentry [6] cannot prevent performs in-depth integrity measurement. the attack. HyperSafe [116]doesnot HyperSafe [116] protects hypervisor protect hypervisor non-control data. control flow integrity. Technique [145] Technique [145] needs hardware changes. measures the checksum of the runtime MGUARD [63] detects attacks based on hypervisor. MGUARD [63] monitors traffic only traces in physical memory. between CPU and physical memory. HyperVerify [33] faces overhead to HyperVerify [33] monitors hypervisor translate low-level hardware state into hardware state from Control VM. hypervisor context. Disaggregating the Disaggregating Xen [68] separates a In Reference [68], still attacks are possible Management Interface; VM-building functionality from Dom0. through network drivers. Nova[103] and Logging and Nova[103] moves the hypervisor cannot reduce the TCB completely. SSC auditing functionality to user level. Xoar [30]breaks [17] is still vulnerable through VM to the control VM into unprivileged service hypervisor interaction. HypeBIOS [141] VMs. SSC [17] splits Dom0 into a faces performance overhead for identifying systemwide Dom0 and a VM-specific crucial VMEXIT. PSVM[136]faces Dom0. HypeBIOS [141] shifts Control VM overhead of maintaining multiple Dom0s. from TCB. PSVM [136] separates Dom0 Xoar [30]andNexen[97] needs privileges. Nexen [97] provides multiple modification to Xen hypervisor. For Dom0. TVGuarder [58]tracesVM Reference [58], it is difficult to identify operations. illegal operations. Minimizing the sHype [89] controls resource sharing sHype [89] cannot prevent external interfaces (e.g. between VMs. NoHype [104] runs VMs modification. In NoHype [104]and Execution natively at the top of physical resources. TrustOSV [113] dynamic resources environment. Virtual TrusrOSV [113] provides micro-hypervisor allocation becomes difficult and still VMs resources, Physical for each VM. Min-V [70] restricts interact with management OS. Min-V [70] resources) between re-engineering of boot time initialization Provides only boot time protection. In hypervisor and VM code or earlier OS code. SplitVisor [73] SplitVisor [73], new functionalities of provides a separate hypervisor to each VM. hypervisor cannot be added dynamically. HyperLock [118] provides shadow HyperLock [118] faces overhead of hypervisor for each VM. DeHype [123] creating and pairing a hypervisor shadow deprivileges the hypervisor execution. to each VM. DeHype [123] faces overhead Nexen [97]providesVMspecificXen of identifying privileged operations. Nexen instance. [97] needs modification to Xen hypervisor. Hypervisor verification Role-based migration using Intel vPro TPM Technique [112] cannot be deployed in before initiating VM hardware [112]. TPM-based remote existing infrastructure. Remote attestation migration attestation [18]. Improved vTPM for [18] does not provide confidentiality and property-based attestation [109]. Trust integrity of data. Property-based token-based VM migration protocol [3]. attestation [109] increases migration time. In Reference [3], it is difficult to define trust level for each VM and hypervisor. Verifying VM image Cloud Verifier92 [ ] and Tenant service [81] Cloud Verifier92 [ ] needs to trust the integrity verify the VM image integrity. middle agent with human interaction. In Tenant service [92], Dom0 can be compromised. at Dom0 to decrypt the VM image and derives the hash to verify its integrity. NIST [20]hasrecom- mended that the continuous monitoring of the VM states and in-out network traffic is necessary. A summary of the hypervisor security solutions is presented in Table 6.

3.2 Virtual Machine Security We discuss the existing security solutions in terms of securing running VM and VM image.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:19

Table 7. A Summary of the Existing Proposals for Securing VM from Compromised Hypervisor

Approach Summary Limitations Tiny hypervisor-based CloudVisor [138] protects VMs from CloudVisor [138] cannot protect the protection compromised hypervisor. TinyChecker [107] security monitor. TinyChecker [107] can recover VM after hypervisor failure. considers only hardware, not the HA-VMSI [144] separates VM isolation vulnerabilities. HA-VMSI [144]is functionalities from hypervisor. dependent on hardware. Hardware-based SICE [7] provides hardware-level isolated and SICE [7] can be tampered by SMM protection for VMs protected VM execution. Hypervisor-secure rootkits. Hypervisor-secure virtualization [105]protectsVMmemory virtualization [105] faces overhead during execution. HyperWall [106] prevents due to dynamic management of access to VM memory. HyperCoffer128 [ ] physical resources and management extends CPU virtualization to prevent functions can be abused. HyperWall memory access. H-SVM [51]provides [106], HyperCoffer128 [ ] and H-SVM hardware-based memory isolation. [51] need hardware modifications.

3.2.1 Security of VM in Running State. As shown in the threat model of the VM running state (refer Figure 4), VM can be attacked from different components of the virtualization. Protecting VMs from the compromised/untrusted hypervisor. A tiny hypervisor-based approaches protect VMs from the compromised hypervisor. CloudVisor [138] separates resource management from security protection. A tiny security monitor provides security of the hosted VMs. Any access to VM contents from the hypervisor can be trapped. TinyChecker [107], a tiny nested hypervisor, protects guest VMs against hypervisor failure. It records all the communication between the VM and hypervisor. When any failure is detected, it provides an inplace recovery to preserve the VM state and data. SICE [7] provides a hardware-level isolated and protected execution environment for the VMs on x86 multi-core architecture. It provides a hardware only TCB for VM. TCB includes the hardware, the BIOS, and the “System Management Mode (SMM).” A hypervisor-secure virtualization mechanism [105] uses the multi-core microprocessors to protect the VMs from the hypervisor. It focuses on VM memory protection during the execution. It stores cryptography keys in VM memory and controls that region of memory from the hypervisor or DMA access. HyperWall [106] implements hypervisor-secure virtualization using hardware to protect VMs from an untrusted hypervisor. Here, the hypervisor freely manages the VM resources such as memory, CPU cores, and so on. However, when VMs are created, the “Confidentiality and Integrity Protection (CIP)” ta- bles prevent the hypervisor access to the memory of the guest VMs. HyperCoffer [128] protects the VM privacy and integrity from the untrusted hypervisor. It extends the processor virtualization to “memory encryption and integrity checking” for securing data communication. It introduces VM- Shim, which declassifies necessary information designated from VM to the hypervisor. H-SVM [51] protects the VMs from an untrusted hypervisor. It provides memory isolation by extending the current hardware support for memory virtualization using nested paging. It decouples memory isolation from memory allocation that results in reducing TCB to only the hardware. SICE [7], a hypervisor-secure virtualization [105], HyperWall [106], HyperCoffer128 [ ], and H- SVM [51] need to extend or modify existing hardware architecture. Similarly to CloudVisor, a hardware-assisted VM isolation called HA-VMSI [144] protects the guest VMs on ARM architecture during runtime. A thin security monitor separates memory isolation of VMs from the hypervisor. It uses hardware with the TrustZone support to protect the security monitor from the hypervisor and VMs. A summary of the existing proposals for securing VM from the compromised hypervisor is given in Table 7.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:20 R. Patil and C. Modi

Table 8. A Summary of the Existing Proposals for Securing VM from Hypervisor Management Interface

Approach Summary Limitations Protecting VM The authors of Reference [56]use The authors of Reference [56]require rsources from hypervisor hardening to protect guest hardware changes to hypervisor. Credo management interface VMs from the control VM. Credo [78] [78] needs hardware changes. A memory provides VM execution independent to access model [85] needs to differentiate the root partition. A memory access between privileged and unprivileged model [85] restricts VM memory access memory access Restricting VM The authors of References [122, 129] The authors of Reference [129]focusonly operations. Log and audit the VM operations. on malicious VM operations. The authors TVGuarder [58] traces insider’s actions. of Reference [122] log the file access only. TVGuarder’s [58] deployment in virtual environment is a challenge.

Protecting VMs from the compromised management interface/Malicious insider. Here, a secured hypervisor is considered. However, its management interface can be compromised by VMs or accessed by malicious insider/remote attacker. The authors of Reference [56] have proposed a secure VM execution architecture to protect the guest VMs from an untrusted management VM. The management VM cannot access any memory of a VM after VM creation. It reduces the TCB of the guest VMs in an untrusted management environment. Here, DomU is protected from the Dom0, while allowing Dom0 to perform VM administrative tasks. Credo [78] protects guest VMs against a malicious administrator and malware in the root partition. It reduces the TCB of VM to a securely launched hypervisor and excludes the privileged partitions. It enables a VM to execute without taking any security dependency on the root partition. The authors of Reference [85] have proposed a memory access model for the management VM. It uses the principle of least privilege for management VM where it cannot read the memory allocated to the VM. Although existing approaches [56, 78, 85] protect the VM memory from the management VM. They cannot restrict VM operations. The authors of Reference [129] prevent VM rollback attack by logging VM operations at a “small trusted computing base.” This log is audited to identify any unauthorized rollback, which constrains the VM operations. At starting of each operation, the VM hash is derived based on memory contents, its registers, and an image disk. The authors of Reference [122] have proposed a logging solution to produce the file-centric logs. It uses file-centric logs as evidence to enhance accountability of a VM. TVGuarder [58], a trace-enabled protection, detects illegal access of a VM. It is a log-based back-tracing mechanism that traces VM operations and allows only legitimate VM operations. A summary of the existing proposals for securing VM from hypervisor management interface isgiveninTable8. Securing VM from kernel-level rootkits. Although protecting kernel code integrity is a primary step to maintain control-flow integrity and data integrity, they limit the further extension ofker- nel for better functionality. The DKOM- and DKSM-based rootkits can be detected based on data structure invariants, but any modification in data structure cannot be prevented. In addition, the return-oriented rootkits do not modify the kernel code but hijack the kernel control flow by modifying the existing kernel hooks. To protect such hooks, “Hooksafe” [117] relocates “kernel hooks” to the “page-aligned memory space” and then regulates accesses through “hardware-based page- level protection.” It protects hooks from rootkits by moving them to a write-protected area. HUKO [132] protects kernels from the untrusted kernel extensions. It allows execution of untrusted kernel extension and observes behavior through mandatory access control policies. It prevents rootkits through checking any violation in data and control flow integrity. Sentry102 [ ] protects the

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:21

“security-critical kernel data” used by the kernel. The kernel memory of guest OS is partitioned into different regions with access control for preventing the malicious modifications. In VM Introspection (VMI) [69], the information delivered to hypervisor or derived at hypervisor is not enough to detect the kernel rootkits. Thus, researchers have focused on executing kernel identical to the target VM kernel for detecting hidden processes. Exterior [37] uses the writable VMI to execute an out-of-VM shell of an entire guest-OS. It uses a dual VM mechanism in which a secure VM (SVM) runs an identical kernel of the guest VM (GVM) and dynamically and transpar- ently redirects the memory state at the hypervisor layer from SVM. Hybridbridge [87]usesdata memorization and fall back at hypervisor. Nitro [76] performs monitoring and analyzes the system calls to check the state of a VM from the hypervisor. NumChecker [114] detects the rootkits that are modifying control-flow. It uses Hardware Performance Counters (HPC) to count the number of hardware events that occur during the system call’s execution. ShadowContext [126] reuses guest OS system calls in a shadowed portion of the out-of-VM inspection program to detect the rootkits. The authors of Reference [100] have collected the traces of HPCs for both malicious and trusted programs, and then machine- learning techniques are applied to detect rootkits. Such approaches do not need any pre-knowledge of the guest OS. Protecting trusted applications from an untrusted kernel. To provide secure execution of the trusted application in the compromised kernel environment, it is necessary to protect application code and data from the kernel. Hardware extensionsbased approaches like TrustVisor [65], Bastion [19], and SecureME [28] attempt to secure the execution of application in a malicious kernel environment. TrustVisor [65] offers code and data integrity as well as isolates the selected portions of an application usingx86 hardware virtualization support. Bastion [19] enhances processor hardware by using a thin hypervisor and generates a security compartment for each application block. SecureME [28] secures an application from hardware attacks by using a secure processor. In addition, it protects the entire address space of an application from an untrusted OS through memory cloaking, permission paging, and system call protection. Approaches like CHAOS [24], Overshadow [25], Virtual Ghost [31], and AppSec [82] attempt to enhance the hypervisor for securing application code and data in a malicious kernel environment. CHAOS [24] isolates CPU context and memory owned by a trusted process from the kernel and other processes. Overshadow [25] presents application resources as an encrypted view to the OS and thus allows the OS to manage these resources without read or modification operations. It uses multi-shadowing to present different views of physical memory. Virtual Ghost [31] combines compiler instrumentation and performs runtime checks on OS code to create ghost memory that the OS cannot read or write. AppSec [82], a hypervisor-based safe execution environment verifies the behavior of the untrusted OS according to the application’s expectation. It uses a safe loader to verify the code integrity of the protected applications and avoids the compilation of applications statically. InkTag [43] uses hypervisor with para-verification to monitor and verify the untrusted OS actions and offers process isolation. AppShield [26] protects the code, data, and runtime integrity of an application. It uses the Extend Page Table (EPT) provided by hardware virtualization extensions to isolate an application from the OS. MiniBox [57] protects an application from a malicious OS and secures a benign OS from a misbehaving application. Iso-X [34] provides isolated components to security-critical pieces of an application to securely execute an application on an untrusted OS. AppGuard [137] uses Intel’s hardware virtualization support to protect an application. It uses a cryptographic approach only in system call-based I/O and paging-related I/O to ensure data privacy and integrity.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:22 R. Patil and C. Modi

Table 9. Summary of the Proposals for Detecting and Preventing Kernel Level Attacks

Approach Summary Limitations Protect kernel hooks Hooksafe [117] protects kernel hooks by Hooksafe [117] requires prior knowledge from control-flow moving them to write protected area. of kernel hooks to be protected. Huko hijacking rootkits Huko [132] protects kernels from [132] requires modification to the guest untrusted kernel extensions. Sentry [102] OS. Sentry [102] requires modification to avoids an illegal access to kernel data. the guest OS. Identical kernel Exterior [37] mimics the VM state in For Exterior [37], it is difficult to analysis secure VM. Hybridbridge [87] performs maintain copies of all VMs. Hybridbridge data memorization and fall back at [87] faces high performance overhead. hypervisor. Nitro [126] runs system call Nitro [126] faces performance overhead. code at “out-of-VM inspection program.” NumChecker [114] cannot determine the NumChecker [100, 114] detect rootkits most significant counters. Technique based on hardware performance counters. [100] needs to be trained for normal applications and malware. Hardware TrustVisor [65] isolates the application TrustVisor [65] cannot protect whole extensions-based block. Bastion [19] generates a security application. Bastion [19] is too restrictive application protection compartment for application block. for protecting user data or an entire SecureMe [28] uses memory cloaking, application. SecureMe [28]requires permission paging and system call modifications to the OS and applications. protection. CHAOS [24] isolates CPU CHAOS [24] needs modification to the context and memory of application. OS and code expansion to the hypervisor. Overshadow [25] presents application Overshadow [25] needs complex resourcesasanencryptedviewtothe encryption and decryption operations. kernel. Virtual Ghost [31] creates ghost Virtual Ghost [31]needstomodifyor memory for an application. AppSec [82] re-compile the OS kernel. AppSec [82] protects the application code. cannot verify the correctness of kernel. Hypervisor-based InkTag [43] monitors the untrusted InkTag [43] needs to modify the source protection of an kernel actions. AppShield [26] uses code of the kernel. AppShield [26] needs application from Iago hardware virtualization extensions to a trusted component in the guest kernel. attack isolate an application. MiniBox [57] MiniBox [57] cannot protect whole protects an application from a malicious application. Iso-X [34] cannot protect kernel. Iso-X [34] isolates the execution whole application. AppGuard [137] adds of critical part of an application. a small amount of performance overhead. AppGuard [137] uses cryptographic approach for system call and paging I/O. Protect applications Haven [10] and Glamdring [59] use Intel In Haven [10], placing all code inside the from kernel/HMI SGX enclave to protect the security enclave increases the TCB. In Glamdring critical application code. [59], security code may access sensitive data.

Haven [10] performs the shielded execution of an application in the presence of untrusted kernel. It leverages Intel SGX hardware to instantiate a secure region of address space known as an enclave. It prevents an access to enclave memory through hypervisor, BIOS, or OS. Glamdring [59] partitions the applications into security sensitive enclave and security-insensitive non-enclave parts. It protects the confidentiality and integrity of application data, even if an attacker hasfull control over the hypervisor. A summary of the kernel-level attack detection and prevention techniques for VM is given in Table 9. Protecting VMs from the rouge co-hosted VM. Many proposals are attempting hard isolation, which dedicates separate hardware for each VM. However, hard isolation leads to reduced effi- ciency. The authors of Reference [77] have prevented cache sharing among the VMs. The authors of Reference [96] have designed a dynamic page coloring for cache isolation during security-critical operations. Page coloring uses several colors for secure processes, and these colors cannot be used

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:23 by other VMs during the execution of security critical operations. The authors of Reference [54] have proposed a page coloring to isolate cache lines in Xen. For each VM, it locks/reserves the cache lines of each CPU core where security-sensitive code and data are stored. The locked lines are not allowed to read by other VMs, and, thus, a VM can load sensitive data into the locked cache lines. The authors of Reference [38] have partitioned the cache memory during boot time. At boot time, a VM belonging to a particular partition is only permitted to use memory of the same partition and, thus, VM is confined to a fixed partition of the shared cache. SAFEPERIMETER [44] secures an access of LLC cache by locking it line by line without assigning any locked cache line to other VMs. The authors of Reference [108] have reduced the risk of resource sharing through better scheduling. Here, the Xen scheduler is modified to limit the frequency in which an attacker can preempt the victim. Nomad [66] identifies a co-residency as a root cause of a side-channel attack and prevents the information leakage by migrating the target VM. Due to migration, the VM co- location becomes difficult for an attacker. CATalyst61 [ ] prevents LLC side-channel attacks using Intel’s performance optimization feature called Cache Allocation Technology (CAT). It partitions the LLC into secure and non-secure partitions. The non-secure partition is hardware managed and the secure partition is software managed. The secure partition uses CAT to enforce the cache- pinning of secure pages. It uses secure partition to store sensitive code and data so that malicious VMs cannot read the sensitive data. The authors of Reference [143] have proposed a software-only defense against attacks like Flush+Reload and Prime+Probe. It uses a copy-on-access mechanism to create a separate copy of physical pages when accessed by other VMs. Thus, the access of VM to its own copy is invisible to an attacker in a Flush+Reload attack. To defeat Prime+Probe attacks, it controls the cacheability of memory pages so that the memory of VM mapped to cache sets becomes invisible to the attacker. The authors of Reference [101] have eliminated LLC-based side-channel attacks using the combination of CAT and Completely-Fair-Scheduler. It dynamically schedules the cache partition that belongs to a separate VM. Observer [60] runs a secure VM that mimics the vulnerable VM and redirects all inbound traffic destined to the vulnerable VM to the secure VM. It differentiates the outbound traffic todetect covert channels. BusMonitor, a hypervisor-based protection system [90], prevents misuse of memory bus operation. It partitions the memory bus operations among VMs to prevent a cross-VM side channel. C2Detector [125] consists of captor and detector. Captor captures hypercalls generated by VMs. Detector at Dom0 analyzes the captured hypercall records and detects the covert channels. ANVIL [5] is designed to detect RowHammer attack. It tracks the locality of DRAM accesses using existing hardware performance counters. It identifies the rows being frequently accessed and then selectively refreshes the nearby rows to prevent hammering. B-CATT [14] extends the system bootloader to disable vulnerable physical memory. It identifies vulnerable rows and prevents RowHammer attack. CATT [13] divides the physical memory into multiple partitions where each partition is separated by at least one DRAM row and owned by a specified domain. Thus, bit-flips induced by one domain do not affect the memory partition of other domains. It is implemented for user and kernel domains. The existing proposals for securing VM from co-hosted VMs are summarized in Table 10.

3.2.2 Security of VM in Inactive State. As per NIST, common recommendations for the VM image security are to use an encrypted and digitally signed images and place the VM image repository outside the hypervisor with the strict access control. Securing image from the outdated software and known vulnerabilities.Mirage[120] periodically executes malware detection tools and discovers the vulnerabilities for patching. The authors of Reference [142] have used Mirage image library to patch multiple images offline. It uses a script rewriting technique to create safe patch scripts. Similarly, authors [35] have considered two

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:24 R. Patil and C. Modi

Table 10. A Summary of the Existing Proposals for Securing VM from the Co-hosted VMs

Attack Summary Limitations Side-channel Techniques such as Cache partitioning Cache partitioning [77] faces overhead on attack [77], Dynamic page coloring [96], Page resource management and requires hardware coloring and locking [54], Boot time changes. Dynamic page coloring [96]requires memory partitioning [38], Reduced frequent swapping pages between colors. Page resource sharing through scheduling coloring and locking [54] cannot detect LLC-based [108], Migrating the target VM [66], Line covert channels. Boot time memory partitioning by line cache locking [44], Dynamic [38] limits the resource utilization. Reduced partitioning of LLC using CAT [61], resource sharing [108] needs modification to the Copy-on-access [143], Dynamic Xen scheduler. Migrating VM [66] is not time partitioning and scheduling of LLC efficient. Line by line cache locking [44] adds partitions [101] protect VM resources overhead on resource management. Dynamic from co-hosted VM. partitioning [61] needs to determine data and instructions that are to be executed in secure partition. Copy-on-access [143] cannot maintain multiple copies of accessed memory region. Scheduling of LLC partitions [101] is not feasible with recent hypervisors. Covert-channel Observer [60] mimics the vulnerable VM For Observer [60], maintaining a secure copy of attack to detect covert channels. BusMonitor every VM is a complex process. BusMonitor [90] [90] partitions bus operations between increases performance overhead. In C2Detector VMs. C2Detector [125] detects OS-level [125], switching between secure VM and covert-channel attack. hypervisor increases the performance overhead. Row- Hammer ANVIL [5] refreshes the adjacent rows. ANVIL [5] induces a worst-case overhead of 8% attack B-CATT [14] disables vulnerable rows. and suffers from false positives. B-CATT [14]does CATT [13] separates physical memory not tackle the memory isolation in physical partitions by at least one DRAM row. memory. CATT [13]needstobeextendedforVM domains. modules: collector and patcher. The collector identifies the outdated software and malware. The patcher applies patches to the identified vulnerabilities. When a VM image is launched tothe hypervisor, the outdated software and VM software vulnerabilities can be identified for further patching [93]. ImageElves [50] identifies the running VMs that need to be patched and creates reliable patches using online updates. It groups the VMs into equivalent classes and applies update to one VM and the update image is generated for other VMs. Securing VM image in repository. “Mirage image format” [80] gives the semantic information that is hidden in the disk-image files. For each image, it maintains a file name for mapping itto the semantic information. Thus, it restricts the creation of multiple images. Mirage [120] protects the images from an unauthorized access using check-in- and check-out-based access control. A tracking mechanism keeps track of all the VM images. An access control mechanism [36] prevents the placement of the poisoned images or modification of images in a repository using a role-based access control mechanism. It applies the authorization rights to the user and logs its repository accesses. “Encrypted Virtual Disk Images in Cloud (EVDIC)” [53] protects the stored VM images using an advanced encryption standard. While loading the image, it is decrypted by interacting with the key distribution server. It protects VM image confidentiality and integrity. Securing VM image from MITM attack and destination hypervisor. A trusted cloud computing platform (TCCP) [91] has two elements: “Trusted Virtual Machine Monitors (TVMM)” and a “Trusted Coordinator (TC).” TC is hosted at an external trusted entity and manages the set of the trusted hypervisors. Each hypervisor runs a TVMM that is verified by TC. A VM user cooperates with TC to launch the VM image to the trusted hypervisor. Similarly, Cloud Verifier92 [ ] verifies the integrity of VM and hypervisor and helps VM user to launch their VM image to a trusted

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:25

Table 11. A Summary of the Existing Proposals to Secure VM Images

VM image location Summary Limitations Destination Mirage [120] periodically executes malware Filters in Mirage [120] cannot scan all hypervisor detection tools. The authors of Reference malwares. In Reference [142], suspended VM [142] patch images offline. The authors of cannot be patched. The authors of Reference Reference [93] patch old software and [93] require manual process. Technique [35] known vulnerabilities. Technique [35] work only on dormant images. In ImageElves patches VM images offline. ImageElves [50] [50], patch-based image classification is very patches VM image through online updates. difficult. Image Mirage image format [80] protects images Mirage image format [80] cannot protect an repository sprawling. Mirage [120] filters poisoned illegal access to VM images. In Mirage [120], images. Access control mechanism [36] access by filters raises the image privacy issue. prevents the placement of poisoned images Access control mechanism [36] cannot or modifying existing images. EVDIC [53] maintain the roles in virtualization hierarchy. uses symmetric key encryption. In EVDIC[53], encryption/decryption cost is high when images are more. Launching TCCP [91] and Cloud Verifier92 [ ] guarantee TCCP [91] is difficult to adopt because ofan channel and the launching of VM to trusted hypervisor. external trusted entity. Cloud Verifier [92] Destination The authors of Reference [4] allow VM user needs regular human interaction. The authors hypervisor. to verify the hypervisor. The authors of of Reference [4] limit the flexibility of load Reference [72] use a TTP to VM image balancing. In Reference [72], trustworthiness of launch. The authors of Reference [74] hypervisor is not consistent. In Reference [74], provide a mobile agent-based secure image key distribution is a great challenge. Tenant launch protocol. Tenant attested service [81] attested service [81] faces performance verifies integrity of Dom0. overhead. Physical Sanitization [21] overwrites VM data. Sanitization [21] is time consuming. In hardware Enxryption [1] encrypts VM data. Enxryption [1], key management is difficult. hypervisor. In TCCP [91] and Cloud Verifier92 [ ], the coordinator needs to be trusted to verify the trustworthiness of the hypervisor. A secure VM launch protocol [4] avoids the middle coordinator and uses the combination of IPsec, an asymmetric key cryptosystem, and trusted computing techniques to enable a VM user to bind a VM to a hypervisor that has been booted into a trustworthy state. Similarly, the authors of Reference [72] have ensured the launch of a VM image to a trusted hypervisor using the trusted third party (TTP). A mobile agent-based secure launch protocol [74] protects the VM images against an MITM attack using end-to-end symmetric key encryption. In addition, the mobile agent is encapsulated with a VM image that verifies the trustworthiness of the hypervisor and calculates the decryption key to decrypt the image. A tenant-attested trusted cloud service [81] allows a VM owner to check whether the Dom0 is trusted or not. Securing VM image from data remanence attack. The authors of Reference [1] have studied different approaches to defend against data remanence attack. It includes sanitization [21], encryption, and so on. In sanitization [21], the hardware assigned to VM can be overwritten with new data. In addition, VM sensitive information can be encrypted and keys can be stored on a VM. Table 11 presents a summary of the existing proposals to secure VM images.

3.3 Virtual Network Security For virtual network security, the security solutions such as Firewall, NIDS, NIPS, DIDS, and so on, can be used. However, these solutions face the problem of dynamic network confugirations, lack of centralized control, and high detection latency. Thus, researchers attempt to focus on SDN- based security for virtual networks. SDN enhances network security through the features such as

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:26 R. Patil and C. Modi centralized control, dynamic network configuration, network traffic control, network programma- bility, and so on. CLOUDWATCHER [98] dynamically changes routing paths of the packets for inspection at pre- installed network security devices. NETSECVISOR [99] uses the existing network security devices and leverages SDN to redirect the network traffic to security devices. NetFuse [115] protects traffic overload in SDN-based data center networks. Network intrusion detection and the countermeasure sElection (NICE) [29] framework deploys NICE agent at each physical server for network traffic analysis. It assesses VM network vulnerabilities to generate an attack graph and then, as per impact of vulnerability, it applies “deep packet inspection (DPI)” and/or virtual network reconfiguration. DCPortalsNg [67] isolates virtual network traffic using SDN. It unpacks the network packet and replaces the source and destination IP addresses. In addition, the MAC address is replaced with host’s MAC address. The packet rewriting guarantees that traffic from a tenant will be out of reach for other tenants. The authors of Reference [110] have used SDN for DDoS attack mitigation. It monitors the global view of network and quickly builds the control structure for fast response to attack. The authors of Reference [88] attempted to find a DDoS attack class and make the controller install the corresponding rule in the OpenFlow switch. FlowTrApp [15] uses network traffic flow rate and duration to detect DDoS attack ranging from low rate to high rate and long lived to short lived attacks.

4 VIRTUALIZATION SECURITY REQUIREMENTS: RESEARCH SCOPE We discuss the research scope in terms of security requirements for each component of the virtualization.

4.1 Hypervisor Security Requirements Although there have been several works for hypervisor security, still there is a scope for further improvements. 4.1.1 Protecting Hypervisor Integrity. Protecting integrity measurement agent. The hypervisor integrity protection mechanisms need a trusted component that can guarantee the security of the integrity measurement agent. Hyper- Guard [86] and HyperCheck [111] use hardware (e.g., SMM), HyperVerify [33] uses a control VM or HyperSentry [6], and MGUARD [63] use a combination of hardware and software as a trusted component. One of the issues with this approach is that these approaches consider only a specific threat model. However, with advances in technology, new threats may emerge, for example, SMM used in HyperGuard [86] and HyperCheck [111] needs to be considered for hardware-based rootkits. In addition, HyperVerify [33] needs to be considered for a control VM compromise. Complete hypervisor introspection. As HyperGuard [86], HyperCheck [111], and HyperSentry [6] provide static hypervisor code integrity, they can be bypassed by the return-oriented rootkits. HyperSafe [116] protects the control flow integrity. The authors of Reference145 [ ] verify only the checksum of hypervisor code. However, they cannot protect hypervisor non-control data. Hy- perVerify [33] protects hypervisor non-control data. It faces overhead when deriving high-level hypervisor context from the low-level hardware state. 4.1.2 Protecting Hypervisor from the Management Interface/Malicious Insider. When securing hypervisor from the management interface, researchers either disaggregate the management interface or restrict access to hypervisor functionalities. Reducing the attack surface through disaggregation. Disaggregated Xen [68], Nova [103], Xoar [30], SSC [17], and PSVM [136] have minimized the interaction between the hypervisor and management interface by taking Dom0 away from the TCB of the hypervisor. However, still they have

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:27 left the large TCB of the hypervisor for VM to VM or VM to hypervisor interaction. Thus, hypervisor remains vulnerable to attacks through other exposed interfaces like network drivers, block drivers, and so on. HyprBIOS [141] have attempted to reduce the VM to VM and VM to hypervisor interaction based on access control violation by VMEXIT. However, identifying the VMEXIT of interest adds the performance overhead. In addition, it has left a surface for DOS attack from VM to VM. Recently, Nexen [97] have attempted to decompose Xen into multiple VM-specific Xen instances for providing an isolated Xen environment to each VM. However, they require new design of the Xen hypervisor. They have not considered the attacks from the malicious insider or attacks during the live migration of the VM. For example, PV guests are typically run in shadow mode for live migration, as well as for features like VM snapshot. In this case, PV guest can crash the hypervisor (CVE-2017-17565). Nexen only controls the memory accesses; however, other hardware resources are shared by VMs and Xen slices. It is again an attack surface that, if compromised, may crash the hypervisor through the DoS attack (CVE-2015-4105). Nexen can be vulnerable to Iago attack where VM kernel is malicious and may attack the VM applications. The shared service domain performs the functionalities that need to cross the Xen slice boundary. In addition, Nexen cannot handle hardware bugs (CVE-2017-15594) (CVE-2017-10916). Restricting illegal VM operations. Although disaggregation helps in deprivileging the management interface, the access to management interface by malicious insider or remote attacker also needs to be considered. Usually VM rollback and VM sprawl attacks are emerged from a poor VM management. TVGuarder [58] allows only genuine VM operations using logging and auditing. However, in a virtual environment where VMs need to be operated frequently, differentiating illegal and genuine VM operations is a challenge. It may face the overhead of the false-positive rate.

4.1.3 Protecting Hypervisor from Rouge VMs. Hypervisor can be secured from VMs by provisioning either a strong isolation to hypervisor resources from VMs or an isolated execution environment for VM. Protecting hypervisor resources from VMs. The approaches like sHype [89], NoHype [104], Min-V [70], and TrustOSV [113] protect hypervisor resources through controlled sharing. How- ever, they limit dynamic resource allocation, which is a vital need of virtualization-based computing technologies. In addition, still the hypervisor shares some resources with the VM through which the guest can crash the hypervisor. Providing isolated execution environment. HyperLock [118], DeHype [123], and SplitVisor [73] attempted to provide a separate hypervisor instance for each VM. Similarly, Nexen [97]decomposes Xen to provide a separate Xen instance for each VM. However, this design is not backward- compatible with commercial hypervisors and, thus, existing computing models need to be migrated with heavy cost. In addition, creating a separate hypervisor instance in a dynamic virtualization environment is an extra overhead, which may degrade the hypervisor performance. Although this design is adopted in future computing models, still it poses security concerns related to the monitoring code that runs in the privileged mode, leaving an additional attack surface.

4.1.4 Verifying Trustworthiness of Destination Hypervisor. The authors of References [3, 18, 109, 112] have used trusted computing technologies (e.g., TPM) to perform attestation of the hypervisor where VM is to be migrated. However, in practice, trusted computing technologies use cryptography, and, thus, distributing and managing keys in dynamic virtual environment is very difficult. The deployment of trusted computing technologies needs TPM-compatible hardware.

4.1.5 Protecting Hypervisor from VM User. Cloud Verifier92 [ ] and Tenant-attested trusted cloud service [81] verify the integrity of a VM image that is to be launched. One of the issues is that

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:28 R. Patil and C. Modi these approaches need to maintain hypervisor or secure VM integrity. They do not consider any compromise of these components from a VM user.

4.2 VM Running State Security Requirements 4.2.1 Protecting VMs from the Compromised Hypervisor. To protect VM from the compromised hypervisor, existing solutions use a trusted tiny hypervisor or hardware-based protection. Protecting a security monitor. CloudVisor [138] and TinyChecker [107] use a tiny nested hypervisor as a security monitor to protect VMs from the hypervisor. However, these approaches do not consider the integrity of a security monitor, which may result in a single point of failure. In addition, they do not defend against physical attacks. HA-VMSI [144] protects a security monitor using a trusted hardware component (TrustZone). However, while using hardware as a trusted component, its security needs to be considered against hardware rootkits and physical attacks. In addition, as it is only implemented on ARM architecture, its feasibility on x86 architecture needs to be checked. In future, it can be implemented on x86 architecture with the Intel SGX support where security monitor can be placed in SGX enclave. Protecting VM resources from hypervisor. The hardware-based approaches like SICE [7]a hypervisor-secure virtualization [105], HyperWall [106], HyperCoffer128 [ ], and H-SVM [51]protect the VM memory from hypervisor during execution. These approaches do not consider any illegal access from a management interface that is involved in performing privileged operations (e.g. VM snapshot, rollback and so on) and may need access to the VM memory. 4.2.2 Protecting VM from Management Interface. Protecting VM resources from management interface. The authors of References [56, 78, 85] protect VM memory access from the management interface. They cannot restrict illegal operations on VM by malicious insider. However, through specific VM operations such as cloning or relocation, itis possible to affect the VM privacy without accessing the VM memory. Restricting illegal VM operations. The authors of References [58, 122, 129] restrict the the illegal VM operations. However, differentiating between privileged and unprivileged VM operations isa challenge. In addition, their deployment should be out of reach to the insider. 4.2.3 Protecting VM Kernel. While protecting the VM, it should be viewed through its two components, kernel and trusted applications. Kernel hook protection.Hooksafe[117], HUKO [132], and Sentry [102] protect the kernel hooks of the original kernel from being hijacked. However, identifying the kernel hooks of the rootkit interest is a challenge at the hypervisor level and causes extra overhead. Complete introspection (Leveraging Inspection). Exterior [37] and Hybridbridge [87] perform cross-verification of VM kernel state to detect rootkits. Although identical kernel state isobserved, the complete view of the VM cannot be available. These approaches perform kernel memory introspection only. However, some rootkits can perform repeated logging to the hypervisor and fill up the disk, causing denial of service to the hypervisor. Thus, the (log) files on the disk also need to be introspected. Protecting security solution (Leveraging Isolation). The rootkit detection and prevention systems like Exterior [37] and Hybridbridge [87] use secure VM as a part of their security solution. How- ever, secure VM has different interfaces through which it can be compromised. In addition, an attacker having an access to secure VM can tamper with the security solution. In-place prevention (Leveraging Interposition).Hooksafe[117], HUKO [132], and Sentry [102] protect the original kernel hooks. However, it is necessary to provide in-place prevention of rootkits. Using writable VMI, the immediate response to rootkits can be given. It includes killing a rootkit, restricting write access to rootkits, and so on.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:29

Minimum overhead. Exterior [37] and Hybridbridge [87] mimic the VM kernel state in secure VM to get the complete view of the VM kernel. These systems are designed by considering single-VM for the scan. However, they face the problem of performance overhead if there are multiple VMs to be scanned for the rootkits. The hardware-based lightweight approaches [100, 114, 126] detect rootkits through system call interception; however, they cannot prevent the rootkits. In addition, these approaches need to be trained for the normal behavior of the kernel and newer rootkits. Early detection. In rootkit detection, the target VM gets affected at least once that may result in affecting the CIA of whole virtual environment. In addition, the rootkit prevention systemsdo not allow unauthorized kernel code executions, which limits the execution of some useful kernel extensions. Thus, in future, early detection and prevention can be performed on the VM immediately when it joins the virtual environment. Here, VM needs to be isolated from an entire virtual environment with the extra cost of delay to start the VM services. 4.2.4 Providing an Isolated Execution Environment for Applications. In a virtual environment, an application needs to be protected from untrusted kernel, hypervisor, and physical access. Hardware-based approaches like TrustVisor [65], Bastion [19], and SecureME [28] and hypervisor- based approaches like CHAOS [24], Overshadow [25], Virtual Ghost [31], and AppSec [82]isolate the memory and CPU state of an application. However, these approaches are vulnerable to Iago attack through system call interface. The approaches like InkTag [43], AppShield [26], MiniBox [57], Iso-X [34], and AppGuard [137] assume a trusted hypervisor and cannot protect applications from an attacker having control over the hypervisor. In addition, they add extra functionalities such as memory access control, isolation, encryption, and so on, to the hypervisor, which increases the size of the hypervisor TCB, resulting in a new attack surface. Haven [10] and Glamdring [59] protect the applications without trusting the hypervisor or any hardware beyond the processor. However, they place the application code in the Intel SGX enclave where code executes at a higher privilege level with access to sensitive data. These approaches focus on security of only single-VM, and thus infeasible to directly apply for securing multi-tenant VMs. 4.2.5 Secure Resource Sharing. Different techniques have been used for restricting illegal access through the shared resources. The authors of Reference [77] use static cache partitioning. The authors of Reference [96] use dynamic page coloring. The authors of Reference [54] use page coloring and locking. The authors of Reference [38] use boot time cache partitioning. SAFEPERIMETER [44] uses line-by-line cache locking. However, these techniques limit resource utilization, which is a vital feature of virtualization-based computing technologies. The authors of Reference [108]use scheduling-based resource sharing to improve resource utilization. However, it needs modification of the original Xen scheduler. Nomad [66] limits co-residency through VM migration. However, it is not efficient to avoid side channel. CATalyst61 [ ] uses CAT to store sensitive data and instructions in a secure partition. However, it needs to determine the instructions and data to be executed in a secure partition. The authors of Reference [143] use a copy-on-access, where it is difficult to maintain multiple copies of the accessed memory regions. The authors of Reference [101]use the combination of CAT and Completely-Fair-Scheduler. However, it is not compatible with all well-known hypervisors since it needs a hypervisor-specific scheduler. To detect covert channels, Observer [60] uses outbound traffic in SVM, BusMonitor90 [ ]partitions the memory bus operations, and C2Detector [125] uses hypercalls. However, these approaches increase the performance overhead on the hypervisor in theh case of multiple target VMs. ANVIL [5] refreshes the adjacent rows, and B-CATT [14] disables the vulnerable rows to prevent a RowHammer attack. However, identifying the rows under attack is a challenge, since some benign applications may access the rows frequently. In addition, it adds extra overhead. CATT

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:30 R. Patil and C. Modi

[13] partitions the physical memory, where each partition is separated by at least one DRAM row. However, it restricts the dynamic memory allocation.

4.3 VM Inactive State Security Requirements Although there have been several approaches proposed for VM image security, there is room to enhance the security of a VM image.

4.3.1 Keeping VM Image Updated. When performing patching of VM images, it is important to determine when it needs to be patched. It is emphasized that to apply patches, the following steps need to be considered [35]: (1) Identify VM to be updated, (2) identify outdated software and known vulnerabilities, and (3) create patches from online updates and apply them to VM. The authors of References [50, 93] have followed the necessary steps to perform online patching of image when it is launched to the hypervisor. However, it is not efficient if there are a higher number of images to be patched at a time. The best practice is to maintain a patch repository along with image repository and keep it updated with recent patches of well-known OSes. When a VM image is created or deployed in the hypervisor, immediately it should be patched with recent patches from the patch repository.

4.3.2 Securing Image in Repository. To combat the threat against a VM image in a repository, either image read can be restricted or the access control policies can be improved. Restrict the image read. Researchers are adopting an encryption-based restriction on images. For instance, the authors of Reference [53] have encrypted images with symmetric key encryption. However, a VM image needs to be decrypted before its use on the hypervisor. This may degrade hypervisor performance. In addition, it cannot limit the attacker to tamper or destroy the images in the repository. Improving access control policies. When improving the access control, it is important to determine what all interfaces have access to in the repository, as the repository has many interfaces other than the image publisher and retriever. The authors of Reference [120] have considered access control for the image publisher and retriever. It does not consider access control over the image filters. The authors of Reference36 [ ] have used role-based access control. However, a repository may have an increasing number of users, and the administrator needs to create corresponding roles. Managing all these roles become complex. In addition, the authors of References [36, 120] do not consider access control over the patching interface, which may risk image privacy. Thus, research needs to be done to improve access control. In future, attribute-based access control can be applied, which is an advanced way of handling authorization.

4.3.3 Securing Image in Transit. To protect a image on channel, researchers have adopted end- to-end security from the repository to the hypervisor. Recently, the authors of Reference [74]have proposed a security protocol where symmetric key encryption is used to protect image confidentiality, and hashing is used to protect image integrity. As the decryption key needs to be derived by the hypervisor, it also faces the problem of hypervisor performance degradation.

4.3.4 Verifying Trustworthiness of Hypervisor. To verify the trustworthiness of the hypervisor, existing solutions use either trusted computing technologies [72, 91, 92], direct verification by a VM user [4], or mobile agent-based hypervisor verification74 [ ]. However, TTP needs to be trusted to perform the actual verification of the target hypervisor. The direct communication between the VM user and the hypervisor can be tampered with. The authors of Reference [74] have faced performance overhead when encrypting and decrypting a VM image.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:31

Table 12. A Summary of the Virtualization Security Approaches

Attack index as per Figure 2 Objective/Security solutions 1 2 3 4 5 6 7 8 9 AB C D E F G H I J KLM NOP Q R S T U V Hyp. static integrity check [6, 60, 63, P P ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ - - - P ✗ ------86, 111] Hyp. runtime integrity check [33, ✓ ✗ ✗ ✓ ✗ ✗ ✗ ✗ ✗ ✗ - - - P ✗ ------116, 145] Disaggregate the HMI [17, 30, 68, 103] ✗ ✓ ✗ P P P P P P ✓ - - ✗ P ✗ ✗ ✗ P - - - Disaggregate the HMI [141] ✗ ✓ ✗ ✓ ✓ P ✓ P ✓ ✓ - - ✗ P ✗ ✗ ✗ P - - - Disaggregate the HMI [136] ✗ ✓ ✗ P P P P P P ✓ - - ✗ P ✗ ✗ ✗ P - - - Disaggregate the HMI [97] P ✓ ✗ ✓ ✓ P ✓ P ✓ ✓ - - P P ✗ ✗ ✗ P - - - Restrict hyp. operations [58] - ✗ ✓ ------✗ ✗ ✓ ✗ ✗ ✗ - - - Restrict access to hyp. resources [70, P P ✗ ✓ ✓ ✓ ✓ ✓ ✓ ✓ - - ✓ P ✗ ------73, 89, 104, 113] Providing isolated hyp. [118, 123] P - ✗ ✓ ✓ ✓ ✓ ✓ ✓ ✓ - - ✓ - ✗ ✗ ✗ P - - - Providing isolated hyp. [97] P ✓ ✗ ✓ ✓ P ✓ P ✓ ✓ - - P P ✗ ✗ ✗ P - - - Securing hyp. from network [3, 18, P - - ✗ ✗ ✗ ✗ ✗ ✗ P ✓ ✗ ------109, 112] Securing hyp. from VM user [81, 92] P P ✗ P P P P P P P P ✓ ------Protect VMs from compromised hyp. - ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ - - ✓ P ✗ ✗ ✗ P - - - [7, 51, 105, 106, 107, 128, 138, 144] Protect VM resources from CHMI - P ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ - - ✗ ✓ ✗ ✗ ✗ P - - - [56, 78, 85] Restrict operations on VM [58, 122, - ✗ ✓ ------P ✗ ✓ - ✗ ✗ - - - 129] Detect/prevent kernel rootkits [37, ------✓ ✗ ✗ - - - 87, 100, 102, 114, 117, 126, 132] Protect applications from untrusted ------✗ ✗ ✗ P P ✗ - - - kernel and hyp. [19, 24, 25, 28, 31, 65, 82] Protect applications from untrusted ------✗ ✗ ✗ P ✓ ✗ - - - kernel and hyp. [26, 34, 43, 57, 137] Protect applications from untrusted ------P P P P ✓ ✓ - - - kernel and hyp. [10, 59] Prevent side-channel attacks [38, 44, ------✗ - - ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✗ 54, 61, 66, 77, 96, 101, 108, 143] Prevent covert-channel attacks [60, ------✗ - - ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✓ ✗ 90, 125] Prevent RowHammer attacks [5, 13, ------✗ - - P P ✗ ✗ ✗ ✗ ✗ ✗ ✓ 14] Hyp.: Hypervisor, ✓: Fulfilled, ✗: Not fulfilled, P: Partially fulfilled, -:Not considered.

4.3.5 Destroying VM Data. The physical destruction of VM data is very difficult in virtualization. In sanitization [21], overwriting the VM hardware is time consuming. Encryption makes VM data invisible; however, data processing in virtualization becomes difficult. The authors of Refer- ence [1] have suggested to combine both techniques to sanitize only hardware area that contains keys. However, identifying the hardware area that contains the keys is an extra overhead. Table 12 presents the summary of the hypervisor and running VM security by considering defense techniques against attacks.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:32 R. Patil and C. Modi

4.4 Virtual Network Security Requirements In addition to the basic virtual network security requirements such as detecting a variety of attacks, minimizing communication overhead, resistance to compromise, and so on, providing SDN security is one of the crucial requirements that is raised with the use of SDN in virtual network security. SDN secures the virtual network through IDS, IPS, DIDS, and so on. However, it reveals a technique to control and operate the virtual network, which can be targeted by an attacker. If communication from the control plane to data planes is not properly secured, it may affect the security of an entire network, and therefore the SDN control plane and related access control mechanisms should be properly designed and implemented.

5 CONCLUSION Virtualization is considered as a base technology in the construct of modern computing technologies. However, due to vulnerabilities existing in current implementations of virtualization platforms, the demand for secure virtualization is increased. We have attempted to show various vulnerabilities, threats, and attacks at different components of virtualization, which inherently pose several security and privacy concerns in modern computing technologies. By investigating the existing security solutions, we have identified the research scope and some future challenges. There is a need of an advanced security model and better access control and cryptoalgorithms that can target different levels of security for each component. Our findings can help the research community address virtualization security challenges and to make virtualization a platform for the delivery of secured and trusted computing technologies.

REFERENCES [1] Khalid Aissaoui, Hafsa Aitidar, Hicham Belhadaoui, and Mounir Rifi. 2017. Survey on data remanence in cloud computing environment. In Proceeedings of the IEEE International Conference on Wireless Technologies, Embedded and Intelligent Systems.1–4. [2] Bushra Albelooshi, Khaled Salah, Thomas Martin, and Ernesto Damiani. 2015. Experimental proof: Data remanence in cloud VMs. In Proceeedings of the International Conference on Cloud Computing. 1017–1020. [3] Mudassar Aslam, Christian Gehrmann, and Mats Björkman. 2012. Security and trust preserving VM migrations in public clouds. In Proceeedings of the 11th IEEE International Conference on Trust, Security and Privacy in Computing and Communications. 869–876. [4] Mudassar Aslam, Christian Gehrmann, Lars Rasmusson, and Mats Björkman. 2012. Securely launching virtual machines on trustworthy platforms in a public cloud. In Proceeedings of the International Conference on Cloud Computing and Services Science. 1–10. [5] Zelalem Birhanu Aweke, Salessawi Ferede Yitbarek, Rui Qiao, Reetuparna Das, Matthew Hicks, Yossi Oren, and Todd Austin. 2016. ANVIL: Software-based protection against next-generation rowhammer attacks. ACM SIGPLAN Not. 51, 4 (2016), 743–755. [6] Ahmed M Azab, Peng Ning, Zhi Wang, Xuxian Jiang, Xiaolan Zhang, and Nathan C Skalsky. 2010. HyperSentry: Enabling stealthy in-context measurement of hypervisor integrity. In Proceeedings of the 17th ACM Conference on Computer and Communications Security. 38–49. [7] Ahmed M. Azab, Peng Ning, and Xiaolan Zhang. 2011. Sice: A hardware-level strongly isolated computing environment for x86 multi-core platforms. In Proceeedings of the 18th ACM Conference on Computer and Communications Security. 375–388. [8] Sina Bahram, Xuxian Jiang, Zhi Wang, Mike Grace, Jinku Li, Deepa Srinivasan, Junghwan Rhee, and Dongyan Xu. 2010. Dksm: Subverting virtual machine introspection for fun and profit. In Proceeedings of the 29th IEEE Symposium on Reliable Distributed Systems. 82–91. [9] Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield. 2003. Xen and the art of virtualization. In Proceeedings of the ACM SIGOPS Operating Systems Review,Vol. 37. 164–177. [10] Andrew Baumann, Marcus Peinado, and Galen Hunt. 2015. Shielding applications from an untrusted cloud with haven. ACM Trans. Comput. Syst. 33, 3 (2015), 266–283.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:33

[11] Naomi Benger, Joop Van de Pol, Nigel P. Smart, and Yuval Yarom. 2014. Ooh aah just a little bit: A small amount of side channel can go a long way. In Proceeedings of the International Workshop on Cryptographic Hardware and Embedded Systems. 75–92. [12] Sarani Bhattacharya and Debdeep Mukhopadhyay. 2016. Curious case of rowhammer: Flipping secret exponent bits using timing analysis. In Proceeedings of the International Conference on Cryptographic Hardware and Embedded Systems. 602–624. [13] Ferdinand Brasser, Lucas Davi, David Gens, Christopher Liebchen, and Ahmad-Reza Sadeghi. 2017. Cannot touch this: Software-only mitigation against rowhammer attacks targeting kernel memory. In Proceedings of the 26th USENIX Security Symposium (Security). 116–130. [14] Franz Ferdinand Brasser, Lucas Davi, David Gens, Christopher Liebchen, and Ahmad-Reza Sadeghi. 2016. Can’t touch this: Practical and generic software-only defenses against rowhammer attacks. Computing Research Repository abs/1611.08396 (2016), 1–15. [15] Chaitanya Buragohain and Nabajyoti Medhi. 2016. FlowTrApp: An SDN based architecture for DDoS attack detection and mitigation in data centers. In Proceeedings of the 3rd International Conference on Signal Processing and Integrated Networks (SPIN’16). 519–524. [16] Jamie Butler. 2004. Dkom (direct kernel object manipulation). Black Hat Windows Security (2004). [17] Shakeel Butt, H. Andrés Lagar-Cavilla, Abhinav Srivastava, and Vinod Ganapathy. 2012. Self-service cloud computing. In Proceeedings of the 2012 ACM Conference on Computer and Communications Security. 253–264. [18] Antonio Celesti, Angelo Salici, Massimo Villari, and Antonio Puliafito. 2011. A remote attestation approach fora secure virtual machine migration in federated cloud environments. In Proceeedings of the 1st International Symposium on Network Cloud Computing and Applications. 99–106. [19] David Champagne. 2010. Scalable Security Architecture for Trusted Software. Princeton University. [20] Ramaswamy Chandramouli. 2014. Security recommendations for hypervisor deployment. Draft NIST Special Pub- lication (2014), 1–37. [21] Suresh N. Chari and Ashish Kundu. 2016. Sanitization of Virtual Machine Images. US Patent App. 15/086,290. [22] Stephen Checkoway, Lucas Davi, Alexandra Dmitrienko, Ahmad-Reza Sadeghi, Hovav Shacham, and Marcel Winandy. 2010. Return-oriented programming without returns. In Proceeedings of the 17th ACM Conference on Com- puter and Communications Security. 559–572. [23] Stephen Checkoway and Hovav Shacham. 2013. Iago attacks: Why the system call API is a bad untrusted RPC interface. SIGARCH Comput. Archit. News 41, 1 (2013), 253–264. [24] Haibo Chen, Fengzhe Zhang, Cheng Chen, Ziye Yang, Rong Chen, Binyu Zang, and Wenbo Mao. 2007. Tamper- resistant execution in an untrusted operating system using a virtual machine monitor. Technical Report FDUPPITR- 2007-08001. Parallel Processing Institute, 1–16. [25] Xiaoxin Chen, Tal Garfinkel, E Christopher Lewis, Pratap Subrahmanyam, Carl A Waldspurger, Dan Boneh, Jeffrey Dwoskin, and Dan RK Ports. 2008. Overshadow: A virtualization-based approach to retrofitting protection in commodity operating systems. ACM Spec. Interest Group Operat. Syst. Rev. 42, 2 (2008), 2–13. [26] Yueqiang Cheng, Xuhua Ding, and R Deng. 2013. Appshield: Protecting applications against untrusted operating system. Singaport Management University Technical Report, SMU-SIS-13 (2013). [27] Yueqiang Cheng, Zhi Zhang, and Surya Nepal. 2018. Still hammerable and exploitable: On the effectiveness of software-only physical kernel isolation. Computing Research Repository abs/1802.07060 (2018), 1–12. [28] Siddhartha Chhabra, Brian Rogers, Yan Solihin, and Milos Prvulovic. 2011. SecureME: A hardware-software approach to full system security. In Proceeedings of the International Conference on Supercomputing. 108–119. [29] Chun-Jen Chung, Pankaj Khatkar, Tianyi Xing, Jeongkeun Lee, and Dijiang Huang. 2013. NICE: Network intrusion detection and countermeasure selection in virtual network systems. IEEE Trans. Depend. Sec. Comput. 10, 4 (2013), 198–211. [30] Patrick Colp, Mihir Nanavati, Jun Zhu, William Aiello, George Coker, Tim Deegan, Peter Loscocco, and Andrew Warfield. 2011. Breaking up is hard to do: Security and functionality in a commodity hypervisor. In Proceeedings of the 23rd ACM Symposium on Operating Systems Principles. 189–202. [31] John Criswell, Nathan Dautenhahn, and Vikram Adve. 2014. Virtual ghost: Protecting applications from hostile operating systems. ACM Spec. Interest Group Comput. Arch. News 42, 1 (2014), 81–96. [32] Anthony Desnos, Éric Filiol, and Ivan Lefou. 2011. Detecting (and creating!) a HVM rootkit (aka BluePill-like). J. Comput. Virol. 7, 1 (2011), 23–49. [33] Baozeng Ding, Yeping He, Yanjun Wu, and Yuqi Lin. 2013. HyperVerify: A VM-assisted architecture for monitoring hypervisor non-control data. In Proceeedings of the IEEE 7th International Conference on Software Security and Reliability-Companion (SERE-C’13). 26–34. [34] Dmitry Evtyushkin, Jesse Elwell, Meltem Ozsoy, Dmitry Ponomarev, Nael Abu Ghazaleh, and Ryan Riley. 2014. Iso-x: A flexible architecture for hardware-managed isolated execution. In Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture. 190–202.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:34 R. Patil and C. Modi

[35] Kang Fan, Dehui Mao, ZhiHui Lu, and Jie Wu. 2013. Ops: Offline patching scheme for the images management ina secure cloud environment. In Proceeedings of the IEEE International Conference on Services Computing. 587–594. [36] Eduardo B. Fernandez, Raul Monge, and Keiko Hashizume. 2013. Two patterns for cloud computing: Secure virtual machine image repository and cloud policy management point. In Proceeedings of the 20th Conference on Pattern Languages of Programs. 1–11. [37] Yangchun Fu and Zhiqiang Lin. 2013. Exterior: Using a dual-vm based external shell for guest-os introspection, configuration, and recovery. ACM Spec. Interest Group Program. Lang. Not. 48, 7 (2013), 97–110. [38] Michael Misiu Godfrey and Mohammad Zulkernine. 2014. Preventing cache-based side-channel attacks in a cloud environment. IEEE Trans. Cloud Comput. 2, 4 (2014), 395–408. [39] Irazoqui Gorka, Inci Mehmet Sinan, Eisenbarth Thomas, and Berk Sunar. 2014. Fine grain cross-VM attacks on xen and VMware. In Proceeedings of the International Conference on Big Data and Cloud Computing. 737–744. [40] Bernd Grobauer, Tobias Walloschek, and Elmar Stocker. 2011. Understanding cloud computing vulnerabilities. IEEE Sec. Priv. 9, 2 (2011), 50–57. [41] Daniel Gruss, Clémentine Maurice, Klaus Wagner, and Stefan Mangard. 2016. Flush+ Flush: A fast and stealthy cache attack. In Detection of Intrusions and Malware, and Vulnerability Assessment. 279–299. [42] Keiko Hashizume, Nobukazu Yoshioka, and Eduardo B. Fernandez. 2013. Three misuse patterns for cloud computing. Security Engineering for Cloud Computing: Approaches and Tools, D. G. Rosado, D. Mellado, E. Fernandez-Medina, and M. Piattini (Eds.). IGI Global, Pennsylvania, United States, 36–53. [43] Owen S. Hofmann, Sangman Kim, Alan M. Dunn, Michael Z. Lee, and Emmett Witchel. 2013. Inktag: Secure applications on an untrusted operating system. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems. 265–278. [44] Zakaria Igarramen and Mustapha Hedabou. 2016. Protecting co-resident VMs from side-channel attack in cloud environment: SAFEPERIMETER system. In Proceeedings of the Mediterranean Conference on Information & Commu- nication Technologies. 539–547. [45] Mehmet Sinan Inci, Berk Gulmezoglu, Gorka Irazoqui, Thomas Eisenbarth, and Berk Sunar. 2015. Seriously, get off my cloud! cross-VM RSA key recovery in a public cloud. IACR Cryptology ePrint Archive. Report 2015/898. 1–15. [46] Gorka Irazoqui, Thomas Eisenbarth, and Berk Sunar. 2015. A shared cache attack that works across cores and defies VM sandboxing and its application to AES. In Proceeedings of the IEEE Symposium on Security and Privacy (SP’15). 591–604. [47] Gorka Irazoqui, Mehmet Sinan Inci, Thomas Eisenbarth, and Berk Sunar. 2014. Wait a minute! a fast, cross-VM attack on AES. In Proceeedings of the International Workshop on Recent Advances in Intrusion Detection. 299–319. [48] Raj Jain and Subharthi Paul. 2013. Network virtualization and software defined networking for cloud computing: A survey. IEEE Commun. Mag. 51, 11 (2013), 24–31. [49] Amarnath Jasti, Payal Shah, Rajeev Nagaraj, and Ravi Pendse. 2010. Security in multi-tenancy cloud. In Proceeedings of the IEEE International Carnahan Conference on Security Technology. 35–41. [50] Deepak Jeswani, Akshat Verma, Praveen Jayachandran, and Kamal Bhattacharya. 2013. Imageelves: Rapid and reliable system updates in the cloud. In Proceeedings of the 33rd IEEE International Conference on Distributed Computing Systems. 390–399. [51] Seongwook Jin, Jeongseob Ahn, Jinho Seol, Sanghoon Cha, Jaehyuk Huh, and Seungryoul Maeng. 2015. H-SVM: Hardware-assisted secure virtual machines under a vulnerable hypervisor. IEEE Trans. Comput. 64, 10 (2015), 2833– 2846. [52] Miltiadis Kandias, Nikos Virvilis, and Dimitris Gritzalis. 2011. The insider threat in cloud computing. In Proceeedings of the International Workshop on Critical Information Infrastructures Security. 93–103. [53] Muhammad Kazim, Rahat Masood, and Muhammad Awais Shibli. 2013. Securing the virtual machine images in cloud computing. In Proceeedings of the 6th International Conference on Security of Information and Networks. 425–428. [54] Taesoo Kim, Marcus Peinado, and Gloria Mainar-Ruiz. 2012. STEALTHMEM: System-level protection against cache- based side-channel attacks in the cloud. In Proceeedings of the USENIX Security Symposium. 189–204. [55] Samuel T. King and Peter M. Chen. 2006. SubVirt: Implementing malware with virtual machines. In Proceeedings of the IEEE Symposium on Security and Privacy. 1–14. [56] Chunxiao Li, Anand Raghunathan, and Niraj K. Jha. 2010. Secure virtual machine execution under an untrusted management OS. In Proceeedings of the 3rd IEEE International Conference on Cloud Computing. 172–179. [57] Yanlin Li, Jonathan M. McCune, James Newsome, Adrian Perrig, Brandon Baker, and Will Drewry. 2014. MiniBox: A two-way sandbox for x86 native code. In Proceeedings of the USENIX Annual Technical Conference. 409–420. [58] Li Lin, Shuang Li, Bo Li, Jing Zhan, and Yong Zhao. 2016. TVGuarder: A trace-enable virtualization protection framework against insider threats for IaaS environments. Int. J. Grid High Perf. Comput. 8, 4 (2016), 1–20. [59] Joshua Lind, Christian Priebe, Divya Muthukumaran, Dan OKeeffe, Pierre-Louis Aublin, Florian Kelbert, Tobias Reiher, David Goltzsche, David Eyers, Rüdiger Kapitza, et al. 2017. Glamdring: Automatic application partitioning for Intel SGX. In Proceeedings of the USENIX Annual Technical Conference. 284–298.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:35

[60] Anyi Liu, Jim Chen, and Li Yang. 2011. Real-time detection of covert channels in highly virtualized environments. In Proceeedings of the International Conference on Critical Infrastructure Protection. 151–164. [61] Fangfei Liu, Qian Ge, Yuval Yarom, Frank Mckeen, Carlos Rozas, Gernot Heiser, and Ruby B. Lee. 2016. Catalyst: De- feating last-level cache side channel attacks in cloud computing. In Proceeedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA’16). 406–418. [62] Fangfei Liu, Yuval Yarom, Qian Ge, Gernot Heiser, and Ruby B Lee. 2015. Last-level cache side-channel attacks are practical. In Proceeedings of the IEEE Symposium on Security and Privacy (SP’15). 605–622. [63] Ziyi Liu, JongHyuk Lee, Junyuan Zeng, Yuanfeng Wen, Zhiqiang Lin, and Weidong Shi. 2013. Cpu transparent protection of os kernel and hypervisor integrity with programmable dram. In Proceeedings of the 40th Annual Inter- national Symposium on Computer Architecture (ISCA’13). 392–403. [64] Clémentine Maurice, Christoph Neumann, Olivier Heen, and Aurélien Francillon. 2015. C5: Cross-cores cache covert channel. In Proceedings of the International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. 46–64. [65] Jonathan M. McCune, Yanlin Li, Ning Qu, Zongwei Zhou, Anupam Datta, Virgil Gligor, and Adrian Perrig. 2010. TrustVisor: Efficient TCB reduction and attestation. In Proceeedings of the IEEE Symposium on Security and Privacy (SP’10). 143–158. [66] Soo-Jin Moon, Vyas Sekar, and Michael K. Reiter. 2015. Nomad: Mitigating arbitrary cloud side channels via provider- assisted migration. In Proceeedings of the ACM Conference on Computer and Communications Security. 1595–1606. [67] Heitor M. B. Moraes, Rogério V. Nunes, and Dorgival Guedes. 2014. DCPortalsNg: Efficient isolation of tenant networks in virtualized datacenters. In Proceeedings of the 13th International Conference on Networks. 230–235. [68] Derek Gordon Murray, Grzegorz Milos, and Steven Hand. 2008. Improving Xen security through disaggregation. In Proceeedings of the 4th ACM International Conference on Virtual Execution Environments. 151–160. [69] Kara Nance, Matt Bishop, and Brian Hay. 2008. Virtual machine introspection: Observation or interference? IEEE Secur. Priv. 6, 5 (2008), 32–37. [70] Anh Nguyen, Himanshu Raj, Shravan Rayanchu, Stefan Saroiu, and Alec Wolman. 2012. Delusional boot: Secur- ing hypervisors without massive re-engineering. In Proceeedings of the 7th ACM European Conference on Computer Systems. 141–154. [71] Keisuke Okamura and Yoshihiro Oyama. 2010. Load-based covert channels between Xen virtual machines. In Pro- ceeedings of the ACM Symposium on Applied Computing. 173–180. [72] Nicolae Paladi, Christian Gehrmann, Mudassar Aslam, and Fredric Morenius. 2012. Trusted launch of virtual machine instances in public iaas environments. In Proceeedings of the International Conference on Information Security and Cryptology. 309–323. [73] Wuqiong Pan, Yulong Zhang, Meng Yu, and Jiwu Jing. 2012. Improving virtualization security by splitting hypervisor into smaller components. In Proceeedings of the Annual Conference on Data and Applications Security and Privacy. 298–313. [74] Anjali Pandey and Shashank Srivastava. 2014. An approach for virtual machine image security. In Proceeedings of the International Conference on Signal Propagation and Computer Technology. 616–623. [75] Peter Pessl, Daniel Gruss, Clémentine Maurice, Michael Schwarz, and Stefan Mangard. 2016. DRAMA: Exploiting DRAM addressing for cross-CPU attacks. In Proceeedings of the USENIX Security Symposium. 565–581. [76] Jonas Pfoh, Christian Schneider, and Claudia Eckert. 2011. Nitro: Hardware-based system call tracing for virtual machines. In Proceeedings of the International Workshop on Security. 96–112. [77] Himanshu Raj, Ripal Nathuji, Abhishek Singh, and Paul England. 2009. Resource management for isolation enhanced cloud services. In Proceeedings of the ACM Workshop on Cloud Computing Security. 77–84. [78] Himanshu Raj, David Robinson, Talha Bin Tariq, Paul England, Stefan Saroiu, and Alec Wolman. 2011. Credo: Trusted computing for guest VMs with a commodity hypervisor. Technical Report MSR-TR2011-130. Microsoft Research (2011), 1–12. [79] Kaveh Razavi, Ben Gras, Erik Bosman, Bart Preneel, Cristiano Giuffrida, and Herbert Bos. 2016. Flip feng shui: Hammering a needle in the software stack. In Proceeedings of the USENIX Security Symposium. 1–18. [80] Darrell Reimer, Arun Thomas, Glenn Ammons, Todd Mummert, Bowen Alpern, and Vasanth Bala. 2008. Opening black boxes: Using semantic information to combat virtual machine image sprawl. In Proceeedings of the 4th ACM International Conference on Virtual Execution Environments. 111–120. [81] Jiangchun Ren, Ling Liu, Da Zhang, Qi Zhang, and Haihe Ba. 2016. Tenants attested trusted cloud service. In Pro- ceeedings of the IEEE 9th International Conference on Cloud Computing (CLOUD’16). 600–607. [82] Jianbao Ren, Yong Qi, Yuehua Dai, Xiaoguang Wang, and Yi Shi. 2015. Appsec: A safe execution environment for security sensitive applications. In Proceedings of the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. 187–199.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:36 R. Patil and C. Modi

[83] Thomas Ristenpart, Eran Tromer, Hovav Shacham, and Stefan Savage. 2009. Hey, you, get off of my cloud: Exploring information leakage in third-party compute clouds. In Proceeedings of the 16th ACM Conference on Computer and Communications Security. 199–212. [84] Francisco Rocha and Miguel Correia. 2011. Lucy in the sky without diamonds: Stealing confidential data in the cloud. In Proceeedings of the IEEE/IFIP 41st International Conference on Dependable Systems and Networks Workshops (DSN-W’11). 129–134. [85] Francisco Rocha, Thomas Gross, and Aad van Moorsel. 2013. Defense-in-depth against malicious insiders in the cloud. In Proceeedings of the IEEE International Conference on Cloud Engineering (IC2E’13). 88–97. [86] Joanna Rutkowska and Rafał Wojtczuk. 2008. Preventing and detecting Xen hypervisor subversions. Blackhat Brief- ings USA (2008). [87] Alireza Saberi, Yangchun Fu, and Zhiqiang Lin. 2014. HYBRID-BRIDGE: Efficiently bridging the semantic gap in virtual machine introspection via decoupled execution and training memoization. In Proceeedings of the 21st Annual Network and Distributed System Security Symposium. 1–15. [88] Rishikesh Sahay, Gregory Blanc, Zonghua Zhang, and Hervé Debar. 2015. Towards autonomic DDoS mitigation using software defined networking. In Proceeedings of the NDSS Workshop on Security of Emerging Networking Tech- nologies (SENT’15).1–7. [89] Reiner Sailer, Enriquillo Valdez, Trent Jaeger, Ronald Perez, Leendert Van Doorn, John Linwood Griffin, and Stefan Berger. 2005. sHype: Secure hypervisor approach to trusted virtualized systems. IBM Research Report RC23511. [90] Brendan Saltaformaggio, Dongyan Xu, and Xiangyu Zhang. 2013. Busmonitor: A hypervisor-based solution for memory bus covert channels. In Proceeedings of the 6th European Workshop on Systems Security.1–6. [91] Nuno Santos, Krishna P. Gummadi, and Rodrigo Rodrigues. 2009. Towards trusted cloud computing. In Proceedings of the 2009 Conference on Hot Topics in Cloud Computing.1–5. [92] Joshua Schiffman, Thomas Moyer, Hayawardh Vijayakumar, Trent Jaeger, and Patrick McDaniel. 2010. Seeding clouds with trust anchors. In Proceeedings of the ACM Workshop on Cloud Computing Security. 43–46. [93] Roland Schwarzkopf, Matthias Schmidt, Christian Strack, Simon Martin, and Bernd Freisleben. 2012. Increasing virtual machine security in cloud environments. J. Cloud Comput. Adv. Syst. Appl. 1, 1 (2012), 1–12. [94] Mark Seaborn and Thomas Dullien. 2015. Exploiting the DRAM rowhammer bug to gain kernel privileges. Black Hat (2015). https://www.blackhat.com/docs/us-15/materials/us-15-Seaborn-Exploiting-The-DRAM-Rowhammer- Bug-To-Gain-Kernel-Privileges.pdf. [95] Saeed Shafieian, Mohammad Zulkernine, and Anwar Haque. 2014. Attacks in public clouds: Can they hinderthe rise of the cloud? In Cloud Computing. 3–22. [96] Jicheng Shi, Xiang Song, Haibo Chen, and Binyu Zang. 2011. Limiting cache-based side-channel in multi-tenant cloud using dynamic page coloring. In Proceeedings of the IEEE/IFIP 41st International Conference on Dependable Systems and Networks Workshops. 194–199. [97] Lei Shi, Yuming Wu, Yubin Xia, Nathan Dautenhahn, Haibo Chen, Binyu Zang, Haibing Guan, and Jinming Li. 2017. Deconstructing xen. In Proceeedings of the Network and Distributed System Security Symposium. 1–15. [98] Seungwon Shin and Guofei Gu. 2012. CloudWatcher: Network security monitoring using OpenFlow in dynamic cloud networks (or: How to provide security monitoring as a service in clouds?). In Proceeedings of the 20th IEEE International Conference on Network Protocols (ICNP’12).1–6. [99] Seungwon Shin, Haopei Wang, and Guofei Gu. 2015. A first step toward network security virtualization: From concept to prototype. IEEE Trans. Inf. Forens. Secur. 10, 10 (2015), 2236–2249. [100] Baljit Singh, Dmitry Evtyushkin, Jesse Elwell, Ryan Riley, and Iliano Cervesato. 2017. On the detection of kernel- level rootkits using hardware performance counters. In Proceeedings of the ACM on Asia Conference on Computer and Communications Security. 483–493. [101] Read Sprabery, Konstantin Evchenko, Abhilash Raj, Rakesh B. Bobba, Sibin Mohan, and Roy H. Campbell. 2018. A novel scheduling framework leveraging hardware cache partitioning for cache-side-channel elimination in clouds. Computing Research Repository abs/1708.09538 (2018), 1–12. [102] Abhinav Srivastava and Jonathon Giffin. 2012. Efficient protection of kernel data structures via object partitioning. In Proceeedings of the 28th Annual Computer Security Applications Conference. 429–438. [103] Udo Steinberg and Bernhard Kauer. 2010. NOVA: A microhypervisor-based secure virtualization architecture. In Proceeedings of the 5th European Conference on Computer Systems. 209–222. [104] Jakub Szefer, Eric Keller, Ruby B Lee, and Jennifer Rexford. 2011. Eliminating the hypervisor attack surface for a more secure cloud. In Proceeedings of the ACM Conference on Computer and Communications Security. 401–412. [105] Jakub Szefer and Ruby B. Lee. 2011. A case for hardware protection of guest vms from compromised hypervisors in cloud computing. In Proceeedings of the 31st International Conference on Distributed Computing Systems Workshops (ICDCSW’11). 248–252.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. An Exhaustive Survey on Security Concerns and Solutions at Virtualization 12:37

[106] Jakub Szefer and Ruby B Lee. 2012. Architectural support for hypervisor-secure virtualization. ACM Spec. Interest Group Program. Lang. Not. 47, 4 (2012), 437–450. [107] Cheng Tan, Yubin Xia, Haibo Chen, and Binyu Zang. 2012. Tinychecker: Transparent protection of vms against hypervisor failures with nested virtualization. In Proceeedings of the IEEE/IFIP 42nd International Conference on De- pendable Systems and Networks Workshops (DSN-W’12).1–6. [108] Venkatanathan Varadarajan, Thomas Ristenpart, and Michael Swift. 2014. Scheduler-based defenses against cross- VM side-channels. In Proceeedings of the 23rd USENIX Security Symposium. 687–702. [109] Xin Wan, XinFang Zhang, Liang Chen, and JianXin Zhu. 2012. An improved vTPM migration protocol based trusted channel. In Proceeedings of the International Conference on Systems and Informatics. 870–875. [110] Bing Wang, Yao Zheng, Wenjing Lou, and Y Thomas Hou. 2015. DDoS attack protection in the era of cloud computing and software-defined networking. Comput. Netw. 81 (2015), 308–319. [111] Jiang Wang, Angelos Stavrou, and Anup Ghosh. 2010. Hypercheck: A hardware-assisted integrity monitor. In Recent Advances in Intrusion Detection. 158–177. [112] Wei Wang, Ya Zhang, Ben Lin, Xiaoxin Wu, and Kai Miao. 2010. Secured and reliable VM migration in personal cloud. In Proceeedings of the International Conference on Computer Engineering and Technology. 705–709. [113] Xiaoguang Wang, Yue-hua Dai, and Jianbao Ren. 2014. TrustOSV: Building trustworthy executing environment with commodity hardware for a safe cloud. Int. J. Grid High Perf. Comput. (IJGHPC’14) 9, 10 (2014), 2303–2314. [114] X. Wang and R. Karri. 2013. NumChecker: Detecting kernel control-flow modifying rootkits by using hardware performance counters. In Proceeedings of the 50th ACM/EDAC/IEEE Design Automation Conference.1–7. [115] Ye Wang, Yueping Zhang, Vishal Singh, Cristian Lumezanu, and Guofei Jiang. 2013. NetFuse: Short-circuiting traffic surges in the cloud. In Proceeedings of the IEEE International Conference on Communications. 3514–3518. [116] Zhi Wang and Xuxian Jiang. 2010. Hypersafe: A lightweight approach to provide lifetime hypervisor control-flow integrity. In Proceeedings of the IEEE Symposium on Security and Privacy. 380–395. [117] Zhi Wang, Xuxian Jiang, Weidong Cui, and Peng Ning. 2009. Countering kernel rootkits with lightweight hook protection. In Proceeedings of the 16th ACM Conference on Computer and Communications Security. 545–554. [118] Zhi Wang, Chiachih Wu, Michael Grace, and Xuxian Jiang. 2012. Isolating commodity hosted hypervisors with hyperlock. In Proceeedings of the 7th ACM European Conference on Computer Systems. 127–140. [119] Ziqi Wang, Rui Yang, Xiao Fu, Xiaojiang Du, and Bin Luo. 2016. A shared memory based cross-VM side channel attacks in IaaS cloud. In Proceeedings of the IEEE Conference on Computer Communications Workshops. 181–186. [120] Jinpeng Wei, Xiaolan Zhang, Glenn Ammons, Vasanth Bala, and Peng Ning. 2009. Managing security of virtual machine images in a cloud environment. In Proceeedings of the ACM Workshop on Cloud Computing Security. 91–96. [121] Michael Weiß, Benedikt Heinz, and Frederic Stumpf. 2012. A cache timing attack on AES in virtualization environments. In Proceedings of the International Conference on Financial Cryptography and Data Security. 314–328. [122] Winai Wongthai, Francisco Rocha, and Aad Van Moorsel. 2013. Logging solutions to mitigate risks associated with threats in infrastructure as a service cloud. In Proceeedings of the International Conference on Cloud Computing and Big Data (CloudCom-Asia’13). 163–170. [123] Chiachih Wu, Zhi Wang, and Xuxian Jiang. 2013. Taming hosted hypervisors with (mostly) deprivileged execution. In Proceeedings of the 20th Network & Distributed System Security Symposium. 1–15. [124] JingZheng Wu, Liping Ding, Yongji Wang, and Wei Han. 2011. Identification and evaluation of sharing memory covert timing channel in Xen virtual machines. In Proceeedings of the IEEE International Conference on Cloud Com- puting (CLOUD’11). 283–291. [125] Jingzheng Wu, Liping Ding, Yanjun Wu, Nasro Min-Allah, Samee U Khan, and Yongji Wang. 2014. C2detector: A covert channel detection framework in cloud computing. Secur. Commun. Netw. 7, 3 (2014), 544–557. [126] Rui Wu, Ping Chen, Peng Liu, and Bing Mao. 2014. System call redirection: A practical approach to meeting real- world virtual machine introspection needs. In Proceeedings of the 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. 574–585. [127] Zhenyu Wu, Zhang Xu, and Haining Wang. 2012. Whispers in the hyper-space: High-speed covert channel attacks in the cloud. In Proceeedings of the USENIX Security Symposium. 159–173. [128] Yubin Xia, Yutao Liu, and Haibo Chen. 2013. Architecture support for guest-transparent VM protection from untrusted hypervisor and physical attacks. In Proceeedings of the 19th IEEE International Symposium on High Perfor- mance Computer Architecture. 246–257. [129] Yubin Xia, Yutao Liu, Haibo Chen, and Binyu Zang. 2012. Defending against vm rollback attack. In Proceeedings of the 42nd IEEE International Conference on Dependable Systems and Networks Workshops.1–5. [130] Jidong Xiao, Zhang Xu, Hai Huang, and Haining Wang. 2012. A covert channel construction in a virtualized environment. In Proceeedings of the ACM Conference on Computer and Communications Security. 1040–1042. [131] Yuan Xiao, Xiaokuan Zhang, Yinqian Zhang, and Radu Teodorescu. 2016. One bit flips, one cloud flops: Cross-VM row hammer attacks and privilege escalation. In Proceeedings of the USENIX Security Symposium. 19–35.

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019. 12:38 R. Patil and C. Modi

[132] Xi Xiong, Donghai Tian, and Peng Liu. 2011. Practical protection of kernel integrity for commodity OS from untrusted extensions. In Proceeedings of the Annual Network and Distributed System Security Symposium. 1–17. [133] Yunjing Xu, Michael Bailey, Farnam Jahanian, Kaustubh Joshi, Matti Hiltunen, and Richard Schlichting. 2011. An exploration of L2 cache covert channels in virtualized environments. In Proceeedings of the 3rd ACM Workshop on Cloud Computing Security Workshop. 29–40. [134] Yuval Yarom and Katrina Falkner. 2014. FLUSH+ RELOAD: A high resolution, low noise, L3 cache side-channel attack. In Proceeedings of the USENIX Security Symposium. 719–732. [135] Younis A. Younis, Kashif Kifayat, Qi Shi, and Bob Askwith. 2015. A new prime and probe cache side-channel attack for cloud computing. In Proceeedings of the IEEE International Conference on Computer and Information Technology (CIT’15). 1718–1724. [136] Cong Yu, Li Xin Li, Kui Wang, and Wen Tao Yu. 2013. Protecting the security and privacy of the virtual machine through privilege separation. In Applied Mechanics and Materials, Vol. 347. 2488–2494. [137] Zili Zha, Min Li, Wanyu Zang, Meng Yu, and Songqing Chen. 2015. AppGuard: A hardware virtualization based approach on protecting user applications from untrusted commodity operating system. In Proceeedings of the Inter- national Conference on Computing, Networking and Communications (ICNC’15). 685–689. [138] Fengzhe Zhang, Jin Chen, Haibo Chen, and Binyu Zang. 2011. CloudVisor: Retrofitting protection of virtual machines in multi-tenant cloud with nested virtualization. In Proceeedings of the 23rd ACM Symposium on Operating Systems Principles. 203–216. [139] Yinqian Zhang, Ari Juels, Alina Oprea, and Michael K Reiter. 2011. Homealone: Co-residency detection in the cloud via side-channel analysis. In Proceeedings of the IEEE Symposium on Security and Privacy (SP’11). 313–328. [140] Yinqian Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. 2012. Cross-VM side channels and their use to extract private keys. In Proceeedings of the ACM Conference on Computer and Communications Security. 305–316. [141] Yulong Zhang, Wuqiong Pan, Qingpei Wang, Kun Bai, and Meng Yu. 2013. Hypebios: Enforcing vm isolation with minimized and decomposed cloud tcb. Technical Report, Virginia Commonwealth University (2013). [142] Wu Zhou, Peng Ning, Xiaolan Zhang, Glenn Ammons, Ruowen Wang, and Vasanth Bala. 2010. Always up-to-date: Scalable offline patching of vm images in a compute cloud.In Proceeedings of the 26th Annual Computer Security Applications Conference. 377–386. [143] Ziqiao Zhou, Michael K. Reiter, and Yinqian Zhang. 2016. A software approach to defeating side channels in last-level caches. In Proceeedings of the ACM Conference on Computer and Communications Security. 871–882. [144] Min Zhu, Bibo Tu, Wei Wei, and Dan Meng. 2017. HA-VMSI: A lightweight virtual machine isolation approach with commodity hardware for ARM. In Proceeedings of the 13th ACM International Conference on Virtual Execution Environments. 242–256. [145] Vincent J. Zimmer and Yasser Rasheed. 2011. Hypervisor runtime integrity support. US Patent 7,962,738.

Received July 2017; revised September 2018; accepted October 2018

ACM Computing Surveys, Vol. 52, No. 1, Article 12. Publication date: February 2019.