5G Multi-Access Edge Computing: Security, Dependability, and Performance Gianfranco Nencioni, Rosario G

©2021 IEEE. This paper is under review at IEEE Communications Surveys & Tutorials. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,1 creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. 5G Multi-access Edge Computing: Security, Dependability, and Performance Gianfranco Nencioni, Rosario G. Garroppo, and Ruxandra F. Olimid

Abstract—The main innovation of the Fifth Generation (5G) smartphones have already been produced and sold by many of mobile networks is the ability to provide novel services with manufacturers. Currently, mMTC is not under deployment new and stricter requirements. One of the technologies that since many network operators have deployed LTE-M and NB- enable the new 5G services is the Multi-access Edge Computing (MEC). MEC is a system composed of multiple devices with IoT in relatively recent times. URRLC is a usage scenario that computing and storage capabilities that are deployed at the edge is more immature and challenging, and it is attracting a lot of of the network, i.e., close to the end users. MEC reduces latency attention from the research community [2]. and enables contextual information and real-time awareness One of the technologies that enable 5G to provide URLLC of the local environment. MEC also allows cloud offloading services is the Multi-Access Edge Computing (MEC). MEC and the reduction of traffic congestion. Performance is not the only requirement that the new 5G services have. New mission- consists in the deployment of storage and computing platforms critical applications also require high security and dependability. at the edge of the (radio) access network. In this way, MEC These three aspects (security, dependability, and performance) is enabling the delivery of services with low latency but can are rarely addressed together. This survey fills this gap and also enable context awareness and task offloading. Moreover, presents 5G MEC by addressing all these three aspects. First, we MEC is also the enabler of the edge intelligence, which is overview the background knowledge on MEC by referring to the current standardization efforts. Second, we individually present anticipated to be one of the main innovations of the Sixth each aspect by introducing the related taxonomy (important for Generation (6G) of mobile networks [3]. the not expert on the aspect), the state of the art, and the MEC is the name given by the European Telecommunica- challenges on 5G MEC. Finally, we discuss the challenges of tions Standards Institute (ETSI) that has an Industry Spec- jointly addressing the three aspects. ification Group (ISG) [4] that is standardizing MEC since Index Terms—5G, MEC, Security, Dependability, Performance. 2014. Before 2017, MEC was standing for "Mobile Edge Computing", but ETSI decided to generalize the standard to other access technologies, not only 4G and 5G, but also fixed- access networks and Wireless Local Area Networks (WLAN). I.INTRODUCTION ETSI MEC is not the only standardization effort on edge The Fifth Generation (5G) of mobile networks is currently computing. Fog computing and cloudlet are the two main under deployment. The main innovation is the provision of alternatives. The cloudlet was proposed in 2009 [5] and can wireless connectivity for various usage scenarios [1]: en- be considered the first effort on edge computing. The cloudlet hanced Mobile Broadband (eMBB), for services with very consists of a micro-cloud close to the mobile device. Fog high data rate requirements (up to 20Gb/s); massive Machine- computing has been firstly proposed by Cisco in 2011. Since Type Communication (mMTC), developed for connecting a 2015, fog computing is promoted and standardized by the huge number of Internet-of-Things (IoT) devices (up to one OpenFog consortium [6]. Fog computing has been introduced million devices/km2); Ultra-Reliable Low-Latency Communi- as an extension of the cloud computing paradigm from the cation (URLLC), for services requiring high reliability and core to the edge of the network. Fog Computing consists arXiv:2107.13374v1 [cs.NI] 28 Jul 2021 very low latency (up to 1ms). eMBB allows improving the of a three-layer architecture where clouds, fog nodes, and services provided by the Fourth Generation (4G) of mobile IoT devices interact. There is also another technology called networks. mMTC enhances the services that are now pro- Mobile Cloud Computing (MCC), but it is not edge computing vided by Low-Power Wide Area Networks, such as Long- since it consists of offloading of tasks from the mobile users Term Evolution MTC (LTE-M) and Narrowband IoT (NB- to the cloud. This paper uses as reference the ETSI MEC, but IoT). URLLC enables innovative advanced services, such as many considerations can also be generalized for the other edge mission-critical applications, industrial automation, and en- computing solutions, and, in our study, works on alternative hanced Vehicular to Everything (V2X) such as platooning or edge computing are also included. remote driving. eMBB services are under deployment, and 5G As already mentioned, URLLC is the most innovative and challenging usage scenario for 5G and the one that requires This work was supported by the Norwegian Research Council through the MEC. To support URLLC, MEC has to cope with high 5G-MODaNeI project (no. 308909). G. Nencioni is with the University of Stavanger, 4021 Stavanger, Norway requirements of ultra reliability, which means security and (e-mail: [email protected]). dependability, and of low latency, which is a performance R. G. Garroppo is with the Univrsity of Pisa, 56126 Pisa, Italy (e-mail: indicator. This is one of the reasons for which security, [email protected]). R. F. Olimid is with the University of Bucharest, 030018 Bucharest, dependability, and performance are three critical aspects of Romania (e-mail: [email protected]). MEC. 2

If MEC security has been to some extent investigated TABLE I during the recent years and several surveys are available, LISTOF ACRONYMS fewer surveys are available on performance and even less on 5G Fifth Generation of mobile networks dependability. To the best knowledge of the authors, this is the BSS Business Support System first work that jointly investigates the security, dependability, CN Core Network and performance of 5G MEC. CFS Customer-Facing Service DNS Domain-Name System This paper has the following contributions: EM Element Management • State of the art and challenges on the security, depend- ETSI European Telecommunications Standards Institute GR Group Report (ETSI) ability, and performance of 5G MEC. Each aspect is GS Group Specification (ETSI) addressed individually by using a similar structure and LCM Life-Cycle Management content organization. MANO Management and Orchestration MEAO MEC Application Orchestrator – First, the taxonomy of the investigated aspect is intro- MEC Multi-access Edge Computing duced. In this way, experts on the other aspects can MEH MEC Host MEO MEC Orchestrator better understand the investigated aspect. MEP MEC Platform – Second, the state of the art is presented. The state MEPM MEC Platform Manager of the art is divided into standardization efforts and MEPM-V MEC Platform Manager - NFV NFV Network Function Virtualization academic publications. NFVI NFV Infrastructure – Finally, the challenges are presented and organized NFVO NFV Orchestrator according to the ETSI MEC architecture. OSS Operation Support System RAN Radio Access Network • Challenges in jointly addressing security, dependability, SDN Software-Defined Networking and performance in 5G MEC. The joint provision of these SST Slice/Service Type three aspects and the related trade-offs are analyzed and VIM Virtualization/Virtualized Infrastructure Manager VM Virtual Machine discussed. VNF Virtualized Network Function The next section introduces the necessary background con- VNFM VNF Manager cepts and definitions of 5G MEC. Sections III, IV, and V present the state of the art and the challenges of 5G MEC related to security, dependability, and performance, respectively. Section VI discusses the challenges and trade-offs of jointly addressing security, dependability, and performance aspects. Finally, Section VII presents the conclusions.

II.BACKGROUND In the ETSI specifications [7], MEC is defined as "system which provides an IT service environment and cloud- computing capabilities at the edge of the access network which contains one or more type of access technology, and in close proximity to its users". Before presenting the state of the art of MEC focusing on the three different aspects, the fundamental concepts of MEC and the related enabling technologies are presented. Table I lists the main acronyms that will be used in the rest Fig. 1. ETSI MEC Reference Architecture [8] of the paper. Most of the MEC acronyms are defined in [7], [8]. Note that what is currently defined as MEC was previously defined as Mobile Edge. For example, MEO was the acronym Machines (VMs) and can offer and consume MEC services. of Mobile Edge Orchestrator. The MEP is a collection of functionalities required to run the MEC applications. The MEP can also host MEC services. More information on MEC applications can be found in [9] A. ETSI MEC Architecture (development) and in [10] (enablement). A MEC system is defined as a collection of MEC Hosts The MEC host-level management is composed of the MEC (MEHs) and MEC management necessary to run MEC applica- Platform Manager (MEPM) and the Virtualization Infrastruc- tions [7]. Figure 1 illustrates the ETSI MEC general reference ture Manager (VIM). The MEPM manages MEC applications’ architecture, divided into two levels: the MEC host level and life cycle, rules, and requirements (e.g., required resources, the MEC system level [8]. latency), and provides element management functions to the 1) MEC host level [8]: A MEH contains a virtualization MEP. The MEP receives Domain-Name System (DNS) records infrastructure that provides computation, storage, and net- from the MEPM and configures the DNS proxy/server. Also, working resources to run MEC applications (MEC Apps) and the MEP receives the traffic rules from the MEPM, MEC a MEC Platform (MEP). MEC applications run as Virtual applications, and services and instructs the data plane. Based 3

TABLE II KEY ELEMENTSINTHE MECARCHITECTURE

Element Functionalities MEH MEP - Offers the necessary environment for the MEC applications. - Hosts MEC services. - Instructs the data plane based on received trafﬁc rules. - Conﬁgures the DNS proxy/server. - Provides access to persistent storage. - Provides timing information.

Virtualization - Provides compute, storage, and network resources Infrastructure for the MEC applications. - Data plane routes trafﬁc among MEC applications, services, DNS proxy/server, and various access Fig. 2. ETSI NFV Reference Architectural Framework [13] networks.

MEC App - Discover, advertise, consume, and offer MEC services. on the traffic rules received from the MEP, the data plane - Perform support procedures related to the life cycle routes the traffic among MEC applications, services, DNS of the application. proxy/server, and various access networks. The VIM manages the allocation and monitoring of the virtual resources MEC Host-level Management MEPM - Manages the life cycle, rules, and requirements of and transmits faults and performance reports to the MEPM. the applications. OpenStack is commonly used to implement VIM [11]. - Prepares the virtualization infrastructure to run a 2) MEC system level [8]: The MEC system-level man- software image. - Provides element management functions to the agement is composed of the MEC Orchestrator (MEO), the MEP. operator’s Operations Support System (OSS), and the user VIM - Manages the allocation and monitoring of the vir- application Life-Cycle Management (LCM) proxy. tual resources. The MEO is the core component that has an overview of the - Prepares the virtualization infrastructure to run a complete MEC system. MEO on-boards the application pack- software image. - Transmits faults and performance reports to the ages (performs integrity and authenticity checks, as well as MEPM. compliance with the operator policies), selects the appropriate MEH(s) for MEC application instantiation based on the related MEC System-level Management rules and requirements and triggers the MEC application MEO - Maintains an overview of the complete MEC sys- instantiation, termination, and relocation (if needed). tem. - On-boards the application packages. The operator’s OSS receives requests from the Customer- - Selects the appropriate MEH(s) for MEC applica- Facing Service (CFS) portal or from the device applications, tion instantiation. via the user application LCM proxy. The CFS portal allows - Triggers MEC application instantiation, termination, and relocation. operators’ third-party customers to select and order MEC applications or receive service level information concerning OSS - Decides on granting requests for instantiation and provisioned applications. A device application is an appli- terminating of applications. cation in a device that can interact with the MEC system. User app - Permits the device applications to request the on- In response to a user request via a device application, an LCM proxy boarding, instantiation, and termination of user applications. user application is instantiated in the MEC system. The user - Informs the device applications about the status of application LCM proxy allows the device applications to the user applications. request the on-boarding, instantiation, and termination of user applications in the MEC system. If the OSS decides to grant a request it transmits it to the MEO for further processing. Table II summarises the functionalities of the main archi- Figure 2 illustrates the ETSI NFV reference architec- tectural elements, as explained in [8]. ture [13]. To provide network services, the NFV is composed of Virtualized Network Functions (VNFs), an underlying NFV Infrastructure (NFVI), and a NFV Management and Orches- B. ETSI MEC-in-NFV Architecture tration (MANO). MEC uses a virtualization platform to run the MEC appli- A VNF is a software implementation of a network function, cation in the MEH. NFV is a virtualization platform where which is decoupled from the hardware resources it uses. the network functions are decoupled from the hardware by The VNFs rely on the NFVI, where the needed virtualized using virtual hardware abstraction. It is, therefore, beneficial resources (computing, storage, and network) are obtained from to reuse the infrastructure and the infrastructure management the hardware resources through the virtualization layer. A VNF of NFV [12]. can be deployed, by case, over one or several VMs [13], where 4

Fig. 4. Integrated MEC Deployment in 5G Network [18]

orchestration of the MEC applications (as VNFs). The other elements remain unaffected. More details about the MEC deployment in an NFV environment can be found in [15].

Fig. 3. MEC-in-NFV Reference Architecture [15] C. 5G and Network Slicing MEC is one of the key technologies of 5G, together with VMs are partitioned on the resources of a hardware host by NFV and Software-Defined Networking (SDN) [17]. In par- software programs called hypervisors. ticular, MEC provides 5G of contextual information and real- The NFV MANO is composed of three main components: time awareness of the local environment. In [18], [19], ETSI the NFV Orchestrator (NFVO), the VNF Manager (VNFM), explains how to deploy and integrate MEC in the 5G system. and the Virtualized Infrastructure Manager (VIM). The NFVO Figure 4 depicts the MEC architecture deployed in a is the highest hierarchical level of the NFV MANO and is 5G network [18]. In the left part of the figure (in dark responsible for the creation and LCM of network services. blue), there is the 5G Service-Based Architecture (SBA), The VNFMs are instead responsible for the LCM of the VNFs, as described by 3GPP in [20], which is composed of the which are locally managed by the Element Management (EM) following network function of the 5G Core Network (CN): systems. A VNFM can serve one or multiple VNFs. Finally, Network Slice Selection Function (NSSF), Network Resource the VIM controls and manages the NFVI resources (e.g., it is Function (NRF), Unified Data Management (UDM), Policy in charge of the inventory of software, computing, storage, and Control Function (PCF), Network Exposure Function (NEF), network resources, increasing resources for VMs, improving Application Function (AF), Authentication Server Function energy efficiency, collection of infrastructure fault operations, (AUSF), Access and Mobility Management Function (AMF), capacity planning, and optimization) [13]. and Session Management Function (SMF). Moreover, the An NFV-based network service is composed of an ordered 5G SBA is also composed of User Equipment (UE), Radio set of VNFs between two end-points, where the traffic is Access Network (RAN), User Plane Function (UPF), and steered through them. This composition to provide an NFV- Data Network (DN). Each function can consume or produce based network service is similar to the one specified by the services. Service Function Chaining (SFC) [14]. In 5G SBA, MEC is seen as a set of AFs. The MEO and the MEP are acting as AFs. The MEH is instead often deployed Figure 3 illustrates the MEC architecture deployed by using as DN. NFV [15]. For continuity and clarity, the architectural changes The NRF contains the registered network functions and are explained separately for each of the two layers. their provided services. The services provided by the MEC 1) MEC host level [15]: On the host side, both the MEC applications are instead in the service register in the MEP. applications and the MEP are deployed as VNFs, while the Some of the services are available only through the NEF, virtual infrastructure is deployed as NFVI. The virtualiza- which acts as a centralized point for service exposure. The tion infrastructure, as the NFVI, can be implemented with AUSF manages the authentication. various virtualization technologies, such as hypervisor-based The PCF handles the policies and the rules. The MEP uses or container-based solutions, but also mixing or/and nesting the PCF services to impact the traffic steering rules. virtualization technologies [16]. On the host management side, The UDM is instead responsible for services related to the MEPM is substituted by the MEC Platform Manager - users and subscriptions. It generates authentication credentials, NFV (MEPM-V) and a VNFM. The MEPM-V has the same handles information related to user identification, manages responsibilities as the MEPM. The VNFM is delegated the access authorization, registers users on AMF and SMF. management of the VNF life cycle. The VIM maintains similar Finally, connected to the network slicing (which will be functionalities. presented more in detail later on), the NSSF manages the 2) MEC system level [15]: In the MEC-in-NFV architec- selection of network slice instances and the allocation of the ture, the MEO is replaced by the MEC Application Orches- necessary AMFs. trator (MEAO) and an NFVO. The MEAO has the same A key role in the integration of 5G and MEC is performed responsibilities as the MEO. However, the MEAO delegates by the UPF, which can be seen as a distributed and config- an NFVO to perform the resource orchestration and the urable data plane from the MEC. 5

Many surveys and reviews have been published in the recent years. Many works focus on MEC (first mobile and then multi- access), referring (at least partially) to the ETSI architecture [52]–[75]. Some works focus exclusively or jointly on fog computing [55], [59], [60], [62], [63], [65]–[67], [71], [73], [76]–[79]. Several works also consider cloudlets [55], [58], [60], [63], [65]–[67], [71], [73]. Other works do not focus in any particular architecture, but consider generic edge computing [80]–[88]. Several works are general surveys [52], [58], [60], [65]– [67], [70], [71], [77], [78], [80]. Some works are focusing on specific perspectives: security [62], [68], [74], [83], [87], security and resilience [59], security and efficiency Fig. 5. 5G-MEC Deployment Scenarios [18] [84], reliability and latency [86], dependability [76], [79], [85]. Other works focus on specific environments: vehicular networks [81], [88], IoT [63], [82]–[84], industrial Internet In [18], four scenarios for the deployment of MEHs in a [69]. Finally, other works focus on specific tasks or parts: 5G network are presented (see also Figure 5): location trade-off [53], orchestration [54], capabilities on 1) MEH and the local UPF collocated with the Base Station; computing, caching, and communication [55], communication 2) MEH collocated with a transmission node, possibly with [56], computation offload [57], service adoption and provision a local UPF; [61], infrastructure [64], optimization [72], tools and applica- 3) MEH and the local UPF collocated with a Network tions [73], integration with network slicing [75]. Aggregation Point (NAP); This work focuses on MEC considering as reference the 4) MEH collocated with the CN functions (i.e. in the same ETSI architecture, but it also considers research on other ar- data centre). chitectures. It focuses on three perspectives (security, depend- ETSI has also investigated the deployment of MEC in other ability, and performance) individually and jointly. This work access technologies, such as 4G [21], WLAN [22], and fixed is up to date and does not focus on any particular environment access [22]. Moreover, ETSI has investigated the deployment or task. All the content related to security, dependability, and in cloud RAN [23]. performance of the above works will be commented in the Initially, the Radiocommunication sector of the Interna- following sections. tional Telecommunication Union (ITU-R) defined three us- Several research projects focus directly or indirectly on age scenarios for 5G and beyond [1]: eMMB, mMTC, and MEC, a good summary can be found in [63], [89]. This URLLC. The use scenarios have been already described in work is part of the 5G-MODaNeI1 project, which focuses on the introduction In 3rd Generation Partnership Project (3GPP), dependability and security in 5G MEC. the use scenarios are called Slice/Service Types (SSTs). As ITU-R, 3GPP SSTs include eMBB and URLLC, but they III.SECURITY do not include mMTC and there are instead Massive In- The research community is active on the security in 5G ternet of Things (MIoT), Vehicle-to-everything (V2X) ser- MEC. Many works highlight the challenges and try to improve vice, and High-performance Machine-Type Communication the security in 5G MEC. After a brief introduction of the (HMTC) [20]. security taxonomy for the readers that are not experts on the The ability of 5G to provide services in very different topic, we summarize the current research activity, focusing use scenarios is enabled by network slicing. Network slicing on security-oriented surveys of MEC, and discuss the security allows the flexible and efficient creation of specialized end-to- challenges. end logical networks (network slices) on top of shared network infrastructure. In order to properly operate, the network slices A. Taxonomy have to be isolated. The network slice isolation needs to be valid with respect to security, dependability, and performance. Security targets some objectives, for homogeneity further More details on network slice isolation can be found in [24]. called security attributes (also known as security requirements In this paper, network slicing is not further described. A more or security properties) that are guarded against adversaries (or detailed description (together with a security overview) can be attackers). Adversaries are malicious parties that intentionally found in [25]. In [26], ETSI has defined how MEC supports mount attacks with the aim to break one or more security network slicing. attributes and thus gain illegitimate advantages. Attacks are security issues that are possible because of vulnerabilities that D. Related Works reside in the system and can be mitigated or defended against by enforcing a variety of countermeasures. Figure 6 sketches In the previous subsections, we have extensively referred to the taxonomy related to security. For more details, the Na- documents produced by the ETSI MEC group. Table III shows tional Institute of Standards and Technology (NIST) glossary the ETSI specifications, reports, and white papers that have content in one of the three perspectives that are investigated. 1https://5g-modanei.ux.uis.no 6

TABLE III ETSISPECIFICATIONS,REPORTS, AND WHITE PAPERS

No. & Ref. Name Security Dependability Performance GS MEC 002 [12] Phase 2: Use Cases and Requirements GS MEC 003 [8] Framework and Reference Architecture GS MEC-IEG 006 [27] Market Acceleration; MEC Metrics Best Practice and Guidelines GS MEC 009 [28] General principles, patterns and common aspects of MEC Service APIs GS MEC 010-1 [29] Mobile Edge Management; Part 1: System, host and platform management GS MEC 010-2 [30] MEC Management; Part 2: Application lifecycle, rules and requirements management GS MEC 011 [10] Edge Platform Application Enablement GS MEC 012 [31] Radio Network Information API GS MEC 013 [32] Location API GS MEC 014 [33] UE Identity API GS MEC 015 [34] Traffic Management APIs GS MEC 016 [35] Device application interface GS MEC 021 [36] Application Mobility Service API GS MEC 024 [26] Support for network slicing GS MEC 026 [37] Support for regulatory requirements GS MEC 029 [38] Fixed Access Information API GS MEC 030 [39] V2X Information Service API GS MEC 031 [19] MEC 5G Integration GS MEC-DEC 032-1 [40] API Conformance Test Specification; Part 1: Test Requirements and Implementation Conformance Statement (ICS) GS MEC-DEC 032-2 [41] API Conformance Test Specification; Part 2: Test Purposes (TP) GR MEC 017 [15] Deployment of Mobile Edge Computing in an NFV environment GR MEC 018 [42] End to End Mobility Aspects GR MEC 022 [43] Study on MEC Support for V2X Use Cases GR MEC-DEC 025 [44] MEC Testing Framework GR MEC 027 [16] Study on MEC support for alternative virtualization technologies GR MEC 035 [45] Study on Inter-MEC systems and MEC-Cloud systems coordination White paper No. 20 [9] Developing Software for Multi-Access Edge Computing White paper No. 23 [23] Cloud RAN and MEC: A Perfect Pairing White paper No. 24 [21] MEC Deployments in 4G and Evolution Towards 5G White paper No. 28 [18] MEC in 5G networks White paper No. 30 [46] MEC in an Enterprise Setting: A Solution Outline White paper No. 32 [47] Network Transformation; (Orchestration, Network and Service Man- agement Framework) White paper No. 34 [48] Artificial Intelligence and future directions for ETSI White paper No. 36 [49] Harmonizing standards for edge computing - A synergized architecture leveraging ETSI ISG MEC and 3GPP specifications White paper No. 39 [50] Enhanced DNS Support towards Distributed MEC Environment White paper No. 46 [51] MEC security: Status of standards support and future evolutions

containing terms and deﬁnitions related to cybersecurity is available at [90]. Similarly, for a more in-depth description of related notions, the reader may refer to [91].

1) Attributes: Table IV lists the commonly accepted security attributes, as defined in [90]–[92]. Confidentiality, In- tegrity, and Availability, known as the CIA Triad, are fundamental requirements. Confidentiality is related to privacy and guarantees the privacy of data. Privacy includes other aspects too, such as the privacy of identity (in relation to anonymity and unlinkability) or the privacy of location. Data integrity can be perceived as data authentication because it guarantees that the data has not been altered in any way, so it is authentic. For the purpose of this paper, both integrity (or authentication of data) and authentication of entities are valuable requirements. Moreover, mutual authentication is a strong form of Fig. 6. Security Taxonomy entity authentication that requests that all the communicating parties authenticate to each other. Other traditional and general accepted requirements exist, such as non-repudiation, which prevents the denial of previous actions. However, we leave 7

TABLE IV (e.g., Denial of Service - DoS, Distributed DoS - DDoS). SECURITY ATTRIBUTES [90]–[92] With respect to the implications for the target, active attacks can be roughly classified into three types: deteriorate - par- Attribute Description tially or fully damage on the functionality of a target, corrupt - take control over the functionality of a target by either leaking Confidentiality (C) Keep the information secret, except from the authorized parties. sensitive information or behaving in a specific, desired way, Integrity (I) Ensure that the information has not been altered or impersonate - fake an entity to gain an advantage of its in an undetected manner by unauthorized par- functionality in the system. The attacks that deteriorate the ties. Availability (A) Assure that the legitimate parties can access a good functionality usually aim to damage the availability by service (or a resource) when they need to. consuming resources in excess, so they are strongly related Authentication (Au) Attest the identity of a party (entity authentica- to performance and dependability too. Of course, disruption tion) or the source of information (data origin authentication). can also be caused by corrupting a target (by either resetting, Privacy (P) Maintain concealed different aspects, such as stopping, or change its normal functionality). Moreover, an the data (privacy of data, or confidentiality), the adversary can mount complex attacks that are combinations identity (privacy of identity or anonymity), the of several attacks (of different types), and the adversary can geographical location (privacy of location), etc. be either a single entity or a coalition of entities that collide to mount the attack together. Of course, both adversaries and attacks can be referred to in this outside of the very succinct taxonomy presented, as it is both classifications (e.g., an active / insider adversary mounts not of particular interest to our work. For more discussion an active / insider attack). The reader that is unfamiliar with on attributes, refer to [91], [92]. These requirements are the attacks’ exemplification can further refer to [90], [91]. enforced by security functions, such as e.g., access control 3) Countermeasures: The countermeasures, also known as that protects access to resources against unauthorized parties safeguards [90], can be reactive or proactive and come in a or authorization that officially grants to a party the right to variety of means. As traditionally categorized in the McCum- be or to do something. Based on previous work [93], [94], ber cube [95], the safeguards can be enforced in technology, Khan et al. accept visibility and centralized policy as two policies and practices, and the human factor. Technology can additional security parameters [89]. We omit them here for be implemented in either software or hardware, and can consist several reasons, one being that they are not directly related to of cryptographic primitives and protocols (e.g., encryption to the security and privacy of the end beneficiaries, but they can protect confidentiality), firewalls, Intrusion Detection Systems be seen more as functions that help to achieve these. (IDS), isolation techniques, and many others. Policies and 2) Issues: In essence, security issues are caused by adver- practices consist of rules, regulations, and best practices that saries that mounts attacks. The attack surface is wide, and specify the expected behavior of involved entities. Examples many classifications of adversaries and attacks exist in the include authorization policies, incident response procedures, literature, based on different criteria. Table V defines the most and recovery procedures. Finally, the human factor includes common types of adversaries and attacks that will be referred education, training, and awareness of users to obey the security to in the paper. policies and make use of technology mechanisms in correspon- Concerning the position of the adversaries with respect to dence to the security goals, make people aware of possible the target, they can be classified into outsiders and insiders. An consequences, and become responsible for their acts [95]. internal adversary has a clear advantage over an external adversary (e.g., has physical access to the network or resources, B. State of the Art is authorized to access some data, resources, or services). This makes internal adversaries more powerful than outsiders and 1) Standards, regulations, and white papers: The ETSI insider attacks usually easier to mount than outsider attacks. specifications that refer to MEC security aspects (listed in Further, attacks can be classified into passive and ac- Table III) are used as references for presenting the state- tive [91], [92]. Passive attacks are simpler to mount, cheaper, of-the-art security requirements and regulations for MEC, and more efficient in resource consumption but less powerful presented in Table VI. These include general requirements than active attacks. They are more difficult to detect because [12], API-related aspects [10], [28], [31]–[35], [38], [39], the adversary does not actively interfere and thus changes network slicing [38], MEC integration within 5G [19], end- nothing. Passive attacks include eavesdropping and traffic to-end mobility aspects [42]. analysis. They directly threaten confidentiality but can also White papers by ETSI (also listed in Table III) but even precede active attacks by gathering the necessary informa- other organizations and industry companies [96], [97] refer to tion to mount these. On the other hand, active attacks can the security aspects of MEC within 5G too. directly target any security property by modifying, deleting, 2) Academic publications: Table VII lists recent surveys and injecting messages (or in general, data in any form: on security and privacy for MEC. Papers that refer to general storage, computation, or transmission) or altering in any way MEC security aspects are also considered. Not many papers the functionality of the target. Besides confidentiality, active are fully dedicated to MEC but consider it together with other attacks can also directly damage integrity and authenticity technologies such as cloud or fog (sometimes only marginally (e.g., replay attacks, Man-in-the Middle Attacks) or availability such as in e.g., [67]). As mentioned in some of these (e.g., 8

TABLE V MAIN TYPESOF ADVERSARIESAND ATTACKS

Type Main Description Examples Target Adversary Outsider (all) The adversary is an outsider, unrelated to the target. An adversary outside the organization, a malicious competitor Insider (all) The adversary is an insider, somehow related the target A malicious employee, a compromised (e.g., the adversary has some insight knowledge, can MEC app access some services / resources). Attack Passive (C,P) The adversary is passive, he/she can listen but not actively Eavesdropping, data capture, (ofﬂine) interfere with the system. (crypt)analysis on collected data. Active (all) The adversary actively interferes by modifying, deleting, injecting messages (or in general, data in any form: storage, computation, or transmission), or altering in any way the input / output or functioning of the system. - Deteriorate (A) The active adversary damages the functionality of the Service deprecation, Denial-of-Service victim in a black-box fashion, either partially or fully. (DoS), partial or total physical damage - Corrupt (all) The active adversary compromises the victim, controlling Malware, credential theft, active moni- it either partially or fully. toring - Impersonate (all) The adversary impersonates an entity to gain advantage. Man-in-the-Middle (MitM), replay attacks, rogue / fake entities

[62], [89]), some specialized work on the security for MEC has the weakest spot (the adversaries tend to attack weak spots). been performed. However, the number of papers referring to The MEHs with poor security can easily become points of particular aspects regarding general edge technologies (and, attacks [83]. to some extent, applicable to MEC too) or general security b) Location privacy: Location tracking enabled by MEC issues that are not MEC specific is large and thus out of the can be seen as both a feature and a risk. Unauthorized access to goal of this paper. For example, a large number of papers the Location Application Programming Interface (API) for the are dedicated to defences against (D)DoS in MEC (e.g., [98]– MEC Location Service can leak sensitive information about [101]) or usage of ML for MEC (e.g., [102], [103]). Security the localization and tracking of users in time [32], similar in 5G network slicing has been analyzed in [25], and some to unauthorized access to the Radio Network Information in aspects are relevant for isolation in MEC too. mobile networks (e.g., access to identifications, which might damage the privacy of the UE) [31]. MEHs (and, in conse- C. Challenges quence, end-users too) are thus directly exposed to location In the following, we present security challenges for MEC privacy risks [84]. To mitigate such risks, an important role is by categorizing them in MEC host level, MEC system level, in the security of the APIs and the generated location reports and general challenges. or processed data. On the other hand, the MEC localization 1) MEC host level: service can be beneficial when GPS coverage is unavailable a) Physical security: The MEHs are located at the or for emergencies (e.g., for healthcare applications where edge of the network, closer to the user and in open en- monitoring devices can send signals requiring assistance to vironments. The physical location of the MEHs becomes the closest MEC platform in case of emergency) [107]. thus insecure [85], [89], host-level devices being even more c) Local defences: Due to their local character, attacks vulnerable to physical attacks than system-level equipment at the host level influence a geographical limited area, in the that is normally placed in a more physically secured area. proximity of end users [62]. This gives MEC capabilities to Moreover, the tendency to deploy many MEHs to cover an enforce security mechanisms and limit attacks in the local area raises problems with respect to a good physical security network segment [63]. MEC is suitable to deploy a defense level [68]. This increases the risk of unauthorized physical perimeter, one example being against (D)DoS attacks when the access and hence physical deterioration or corruption of the adversary only targets a smaller volume of traffic, and the edge devices, with direct consequences against the availability (e.g., can alert the core network about the source of danger, resulting DoS attacks) and the confidentiality (e.g., data leakage by both in overall better availability [101]. The local character of MEC passive and active attacks) [96]. The equipment might lack can also enhance privacy protection by preventing data to the hardware protection of commodity servers [62], but as a arrive at centralized servers, and thus avoiding a centralized form of protection, the MEC devices should implement anti- point of trust. An example from [63] consists in the processing theft and anti-damage measures [96]. Surely, tamper resistance of images with car plates at the edge and identification of is a general good security strategy to prevent the reading of the plate number only to forward to central processing (this confidential data (e.g., cryptographic keys) and thus should prevents for example location leakage). At the same time, it is be adopted in case of MEC equipment too [63], [85]. In this believed that local data exchange (as contrary to e.g., sending scenario, the well-known principle of the weakest link holds: the data over the internet) reduces the exposure of data [84], the security of the overall system is given by the security of but of course can raise security risks when the number of nodes 9

TABLE VI SECURITY REQUIREMENTSINTHE MECARCHITECTURE

Element Security Requirements General - The MEC system shall provide a secure environment for running services for the involved actors, in particular the user, the network operator, the operators’ third parties, and the platform vendor [12]. - The MEC system shall prevent illegal access from dishonest terminals, secured communication is necessary between the radio nodes and the MEC service [12]. - The MEC system should securely collect and store logs, including information about charging [12]. - MEC services might require end-to-end security mechanisms [12]. - APIs must be secured, including general aspects such as controlling the frequency of the API calls, unexposure of sensitive information via the API, the authentication of a client to RESTful MEC Service API is performed using OAuth 2.0, and APIs shall support HTTP over TLS 1.2 [28], [31], [33], [35]. - The MEC system discovery, including security (authentication/authorization, system topology hid- ing/encryption), charging, identity management, and monitoring aspects, is an essential prerequisite to form a MEC federation [45].

MEH MEP - The MEP shall only provide a MEC app with the information the MEC app is authorized for [12]. - The MEP shall provide a secure environment for providing and consuming MEC services when necessary [12]. - A MEP should securely communicate to a MEP that might belong to different MEC systems [8]. - If the MEP is not dedicated to a single NSI, then the MEP shall support different set of services and functionalities in distinct NSIs; in particular, the MEP needs to share an infrastructure that allows authentication and authorization at NSI level and assure isolation of data and services between NSIs [26]. - The MEP discovery is provided by means of the MEC systems exchanging information about their MEPs, i.e., their identities, a list of their shared services, as well as authorization and access policies [45].

Virtualization - Virtualization should not introduce any new security threat; in particular, hypervisors should not introduce Infrastructure new vulnerabilities, patch management, state-of-the-art protections and secure boot mechanisms should be in place [13]. - A proper isolation of VNFs [13] and VMs [16] should be realized, the interfaces between the NFV components should be secured [13].

MEC App - The application instance must satisfy the necessary security constraints [12]. - The MEC applications should have access to a persistent storage space [12]. - The MEC application should only have access to information for which it is authorized, should manage the access control and integrity of the user content, and should be authorized to consume or provide MEC services [12]. - Upon authentication, the MEC applications should be able to communicate securely regardless if they are placed in the same MEH or different MEHs [12], even located in different MEC systems (in case of multi-operatior scenario) [8], [12]. - Operator trusted MEC application (seen as an extension of the MEP functionality) have advanced privileges of securely communication with the MEP [12].

MEC Host-level Management - MEPM - The MEC management should verify the authenticity and integrity of a MEC application [12], [30].

VIM - N/A (see Virtualization Infrastructure)

MEC System-level Management MEO - The MEO checks the integrity and authenticity of the application packages [8], [30]. - The MEO authorizes the requests of the OSS (e.g., application instantiation and termination, fetch on-boarded application package, query application package information) [30]. - MEAO adapts the orchestration operations based on the available NSIs and their requirements, including security requirements too [26].

OSS - N/A

User app LCM - N/A proxy

is high, because of high trafﬁc and positioning at the edge of with escalated privileges [62], [63], [109]. If a VM is running the network [70]. [108] discusses some security improvements on multiple hosts, then a simple DoS attack can damage all that MEC can bring in the IoT scenario. hosts simultaneously [109]. As a protection against the DoS attacks, the VMs should be limited in resource consumption, d) Virtualization security: Malicious virtual machines and the resource consumption should be balanced among can try to exploit their hosts [62]. Attacks such as VMs hosts. With respect to the privacy of data, the user data manipulation might include a malicious insider with enough is stored at MEC host level, so it might get leaked [68]. privileges to access and damage a VM or a malicious VM 10

TABLE VII ACADEMIC SURVEYS RELATED TO MECSECURITY

Ref. Aspect MEC Main contribution Relevance to MEC security only [104] 5G security No Gives an overview of 5G security challenges and solu- Mentions briefly some MEC security challenges, consid- tions, refering to other technologies (e.g., SDN, NFV, ered together with mobile clouds. cloud). [62] Edge general No Analyses the security threats, challenges, and mecha- Discusses MEC from the perspectives of security, de- nisms in all edge paradigms. pendability, and performance but keeps the discussion decoupled from the architecture. [83] MEC/IoT Yes Considers MEC privacy issues in the context of hetero- Focuses on big data privacy issues in MEC with respect security geneous IoT. to data aggregation and data mining, and considers ML privacy preserving techniques. [63] MEC/IoT Yes Overviews the MEC technology for the realisation of IoT Reviews papers and discusses problems, challenges, and security applications. possible solutions for IoT security, privacy, and trust in relation to MEC. [65] MEC general Yes Presents a survey on different aspects related to MEC Presents some security mechanisms and privacy issues (e.g., computation, storage, energy efficiency, research related to MEC keeping the discussion quite general. infrastructure), also including security and privacy. [59] Edge security No Reviews some security and resilience aspects in MEC Generally discusses security requirements and challenges and resilience and fog in relation to cloud technologies. in MEC together with fog and in comparison with cloud. [60] Edge general No Presents an overview of potential, trends, and challenges Refers only briefly to security and privacy. of edge computing. [105] Edge security No Surveys the data security and privacy-preserving in edge Analysis the data security and privacy challenges and computing. countermeasures. [106] Edge security No Discusses network threats in fog and edge computing. Indicates threats, challenges and trends with respect to network security and awareness, with no focus on MEC particularities. [82] Edge / IoT No Surveys how edge computing improves the performance Analysis the security attributes and proposes a framework of IoT networks, but also considers security issues in for security evaluation for edge computing-based IoT. edge computing. [85] Edge dependabil- No Explores dependability and deployment challenges in Presents new challenges in physical security and scalable ity edge computing. Considers dependability in a wider authentication, considering both centralized and decen- meaning that includes security. tralized security mechanisms. [89] 5G security No Gives a security and privacy perspective on 5G, looking Discusses MEC security and presents some MEC threats into several enabling technologies (e.g., cloud, fog, MEC, and recommendations, but without special focus on MEC SDN, NFV, slicing). and mostly together with cloud related security issues. [68] MEC security Yes Discusses MEC from a security perspective. Identifies seven security threat vectors and discusses possible solutions and approaches to secure MEC by design. [66] Edge general No Surveys edge computing, identifies requirements and Gives low importance to MEC security, which is very discusses open challenges. briefly discussed. [84] Edge/IoT No Discusses security and privacy together with efficiency in Discusses general security threats, secure data agregation, security data communication and processing computation for IoT and secure data deduplication at the edge, with no focus at the edge. on MEC, but basing the findings on fog nodes. [70] MEC general Yes Surveys MEC fundamentals, discussing its integration Discusses MEC security very briefly but it points to many within the 5G network and relation to similar UAV related papers (e.g., in the topic of MEC for IoT, security communication, IoT, machine learning and others. and privacy in the context of MEC-enabled IoT in V2X, smart cities, healthcare). [87] Edge security No Aims to provide a systematic review of security and Surveys a significant number of papers, but suffers from privacy requirements in edge computing, including a some clear shortcomings with respect to content and taxonomy of attacks and performance evaluation. presentation. [74] MEC security Yes Analyses the security and privacy of MEC. Discusses threat vectors, vulnerabilities that lead to the identified threat vectors, and proposes solutions to over- come these. The paper can be perceived as an extension of [68].

Moreover, possible alteration of data requires adequate backup e) Constrained resources: Computational expensive se- and recovery possibilities, in strong relation with dependability curity mechanisms, including the usage of heavy cryptography, prevention. Virtualization attacks can damage the orchestration can be a problem. For example, the edge devices might have at the host level, and a compromised VIM can lead to the limited connectivity and resources, which impose restrictions disruption of MEC services [68]. Service manipulation is on the security protocols that can be deployed and facilitate another example of corruption, with important consequences attacks against availability [97]. This might conclude in restric- such as DoS or data leakage attacks [62]. If a host is corrupted tions in the deployment of high secure protocols, for example (not necessary by virtualization attacks but in general), the for authentication [62]. Usage of public-key cryptography adversary might interfere at several levels (e.g., apps, services, and Public-Key Infrastructure (PKI) in particular might be resource consumption) and run a wide set of possible attacks. an issue because of high computational costs and management [85]. Lightweight cryptography should be considered. 11

Data deduplication mechanisms at the edge (in the sense of subject to masquerading for adversaries that pretend to have detecting and discarding copies of data, or even prevent re- legitimate access [68]. A significant number of requests from computations) would increase the performance on limited de- the OSS to the MEO might (in absence of proper security vices. But realizing this while maintaining security is normally mechanisms) damage the functionality of the MEO. possible via (Fully) Homomorphic Encryption ((F)HE), which 3) General challenges: by itself requires very high computational costs [84]. The a) Trust models and relations: Contrary to other tech- European Agency for Cybersecurity (ENISA) itself identifies nologies at the edge (e.g., edge cloud, fog), in MEC there is a the complexity of the implementation of security solutions in lower number of owners that need to cooperate [62]. However, 5G (caused of mixing technologies such as cloud, fog, and it is of critical importance to clarify the trust models and edge), as well as the efficient cryptography solutions (because relations between the entities involved (users, platforms, slices, of nodes constrained in resources) to be key topics in research apps, etc.) [96]. In particular, trust needs to be considered and innovation of 5G security [110]. in relation to mobility and network functions performed in 2) MEC system level: the MEC and integration to the 5G standard too [96]. Trust a) Global defences: The management and orchestration models have been considered for different edge technologies of the diverse security mechanisms is a complex issue, and [62], [113]. A flexible trust manager has been proposed as a enabling security mechanisms independently on multiple en- solution to incorporate within MEC [63]. Other trust schemes tities does not necessarily mean that the complete system is have been proposed for MEC (e.g., [114]). Nevertheless, secured [62], [70]. A balance between local (decentralized) general protection mechanisms such as mutual authentication and global (centralized) defense mechanisms, between respon- and access control mechanisms at all levels should be set in sibility and autonomy has to be considered [62], [85]. A global place. A good prevention is to minimize the data transmitted monitoring that allows an overview of the MEC system should and stored on low reputation entities [62]. The ownership of be set in place, and everything should be auditable [62]. To personal data must be assigned and clearly decided between provide privacy, end-to-end encryption becomes a necessity different roles (e.g., stakeholders, MNO, third parties) [89]. whenever applicable [89]. To protect end users’ data, techniques such as watermarking, b) MEO security: MEO is exposed to virtualization visual cryptography, and biometrics were considered in the attacks [68]. A compromised MEO could have a critical impact literature [89]. on the functionality of the overall MEC system. Examples Nevertheless, security models (not only trust models) in include the termination of MEC critical applications, on- accordance with the MEC requirements need to be developed boarding of malicious application packages, an unbalanced and applied for security analysis. usage of the MEHs in terms of resources, and others. Hypervi- b) Network security: The heterogeneous nature of edge sor introspection methods need to be applied, for Linux-based and its dynamic character introduces risks [66], [70], [89], platforms Security Enhanced Linux (SELinux) might be of [97]. Security is prone to risks at the interconnection with other use [68]. More on virtualization defenses will be discussed technologies, mostly within the 5G context. In particular, MEC in III-C3, the subsection dedicated to general challenges. should access the internet and establish connectivity to other However, in the rapidity of change in technology and attacks MEC domains via the internet [68]. Naturally, security should nowadays, it is considered suitable to approach by software be considered for data in all forms (storage, computation, and programmable solutions than rigid hardware (to allow updates, transmission) and approached by appropriate cryptographical moving targets, dynamic attacks, etc.) [101]. Security and primitives and security technologies (including general ap- privacy challenges in softwarization and virtualization are proaches such as VPN communication, access control func- not specific to MEC, as flexible solutions are required to tions and policies, firewalls [68]). Lightweight encryption secure 5G in general [89]. Solutions to prevent virtualization can be used for performance enhancements whenever conve- problems and security frameworks within the MEC-in-NFV nient [68]. However, there is still place for improvement with architecture have been considered [68], [111]. These include respect to lightweight cryptography needed for solutions such Trusted Platform Manager (TPM) to attest and validate MEC as MEC. MEC is exposed to attacks on the communication Apps and VNFs, as well as requests from the CFS portal [68]. channels, mostly on the wireless channels close to the end- Auto-configurable security mechanisms are proposed, as well user [68]. A specific risk lies in the communication between as methods to secure VNF in NFV environments [68], [112]. the edge and the core [89], [97]. Data has to be encrypted in Ideally, the idea of a security orchestrator based on soft- transit (e.g., IPSec, TLS) [96], and end-to-end encryption is a warization (SDN / VNF) would replace the need for manual way to prevent data leakage [84]. Adaptive security protocols configuration that is no longer feasible under the current could be used to enhance the security of communication circumstances [89]. But how such a security orchestrator channels [68]. Software-Defined Virtual Private Local Area should be built and integrated with the architecture of MEC Network (Soft-VPLS) can help in securing the communication is still an open question. between MEC components [68]. Soft-VPLS allows different c) Interconnection security: At MEC system level, OSS traffic categories (e.g., MEC service requests, user data, con- is exposed to threats outside the MEC system by the com- trol statistics) to be routed via distinct tunnels, aiming to munication with the CSF Portal and device applications via enhance both end-to-end security and overall communication the LCM proxy. This opens up for security risks, for example, performance [68]. Proper isolation of network traffic, data, the CSF Portal is prone to (D)DoS attacks [68]. OSS can be services, slices, etc. is required [62], [96]. The use of gateways 12 at strategic points in the network is considered a good practice [104], and firewalls are now implemented as NFs [107]. Techniques to provide physical layer security might turn out beneficial because of performance aspects [70], [115], [116]. More on communication security is discussed in [89]. c) Monitoring and detection: Mitigation techniques in network security include monitoring and logging, abnormal traffic analysis, malware and intrusion detection. In MEC, this should be performed at all levels. The collaboration between the edge nodes can be useful in this respect [96]. Some work on access control and Intrusion Detection Systems (IDS) for MEC has been pointed out before [62], [117]. This can prevent or mitigate attacks such as DoS, malicious actions, rogue entities [62]. Artificial Intelligence (AI), in particular Machine Learning (ML), could be successfully applied for intrusion and anomaly detection [68], [107]. Deep Learning (DL) was specifically considered for the detection of attacks, and Reinforcement Learning (RL) was proposed for edge caching security [70], [102], [103]. Federated Learning (FL) suits MEC because it allows the training data to be kept locally and privately among collaborating nodes. Thus, advances in Fig. 7. Dependability Taxonomy using FL for constraint devices might be useful [70], [118], [119]. More references about ML in IDS and against DDoS attacks can be found in [89], also in correlation with SDN Usage of the appropriate technologies and primitives for solutions. The study of ML (and AI in general) for the MEC security is always a challenge. For example, solutions nowa- security is a valid research direction [63]. days tend to adopt blockchain-based technology, including d) Virtualization security: General virtualization and in MEC. However, many times the blockchain technology is softwarization issues need to be considered in MEC too. This in fact not needed [124] and more suitable solutions (e.g., include security issues of both SDN and NVF [62], and in which introduce less complexity) are available. While ETSI particular security aspects related to network slicing [25], has a dedicated group on Permissioned Distributed Ledgers [89]. SDN/NFV-based frameworks or approaches to provide (PDL) [125], it remains open if blockchain is indeed a useful security in relation to MEC and IoT have been considered [63], technology for MEC. Quantum security mechanisms have to [111]. Software-Defined Privacy (SDP), a solution currently in be considered [89]. Cryptographical primitives currently used place for enforcing the security of Internet as a Service(IaaS) in 5G and MEC (mainly public-key primitives) are known cloud customers, might be extended to provide privacy protec- to be vulnerable to quantum attacks. Although nowadays tion in MEC too [63]. Virtual Machine Introspection (VMI) quantum attacks are still in their infancy, quantum-resistant and hypervisor machine introspection should monitor the cryptography is an active research field that aims to provide activities in terms of resource utilization to prevent deprivation security against quantum adversaries. and DoS attacks [68]. These should be run at both host and Mobility-related security challenges are of importance in system level [68]. Examples of VMI include LibVMI [68], MEC too. More about these will be discussed together with [120]. Artificial intelligence can be used to achieve better VMI other aspects, in Section VI. solutions [68], [121]. e) Standardization and awareness: The necessity for IV. DEPENDABILITY standardized or universal security measures to ensure the If security is a well-known term, dependability is a less security of the overall MEC system is considered to be an open common term, whose meaning is sometimes ignored or con- problem [63]. At the same time, the human factor remains fused. In this paper, we define dependability as in [126]. a risk, so it is important to raise awareness of the users Dependability is the ability to deliver a service that can that need to understand and apply security policies [62]. A justifiably be trusted. significant category is given by the application developers [63], and special focus should be put on the integration mechanisms. f) Other security challenges: Many other security chal- A. Taxonomy lenges exist in MEC (e.g., see [89] for some specific Backhaul Figure 7 represents the taxonomy related to dependability. threats). We next refer briefly to some of these. The attributes are the various ways to evaluate the depend- Privacy of identity is known to be a challenge in mobile ability of a system. The issues are the causes that may lead to networks (including 5G), and it remains a challenge in MEC a lack of dependability. The countermeasures are the methods too [89], [122]. More on Personally Identifier Information (PII) to enhance the dependability of a system. protection in the general context of General Data Protection Alternative definitions and taxonomies can be found in Regulation (GDPR) can be found in [123]. literature [127], [128], where the term threat is used instead 13

TABLE VIII TABLE IX DEPENDABILITY ATTRIBUTES MAIN TYPESOF FAULTS AND FAILURES

Attribute Description Type Description Availability (A) Readiness for a correct service. Faults Reliability (R) Continuity of a correct service. Development fault Fault occurring during development. Survivability (S) Capability to continue to deliver a correct service Physical fault Fault that affects hardware. in the presence of failures or accidents. Interaction fault External fault. Safety (Sf) Absence of catastrophic consequences on the user(s) and the environment. Failures Maintainability (M) Ability to undergo modifications and repairs. Content failure Deviation of the content of the information delivered. Timing failure Deviation of the arrival time or duration of the information delivered. Erratic failure The service is delivered (not halted), but is erratic. of issue and the term mean is used instead of counter- Inconsistent failure Some or all system users perceive differently measure. Moreover, some works define a joint dependability incorrect service. Catastrophic failure The cost of harmful consequences is orders of and security taxonomy [126], [128]. Instead, performance, or magnitude, or even incommensurately, higher than performability, is sometimes seen as one of the attributes of the benefit provided by correct service delivery. dependability [128]. For sake of clarity, we keep separated security, dependability, and performance, and we will jointly discuss them in Section VI. number of faults; 1) Attributes: The attributes consist in metrics that are able - Fault tolerance is carried out via error detection and to characterize and measure specific properties of a system. system recovery in order to avoid failures; The attributes can be applied to the MEC as a system, but - Fault removal can be performed during the system de- also to MEC-based services. The main attributes are listed velopment (via verification, validation, and testing) and in Table VIII. The description of the attributes is based on during the system use (via corrective or preventive main- previous definitions [126] [128]. The most know attributes tenance); are availability and reliability. A system is available when - Fault forecasting is conducted by evaluating the system it is ready to deliver a service that complies with the service behavior (it can be both qualitative and quantitative). specifications. The simplest way to compute the availability of a system is the ratio between the expected uptime of the system and the aggregate time (sum of expected values of B. State of the Art up and down times). A system is reliable when it is able to 1) Standards, regulations, and white papers: The ETSI continuously deliver a service that complies with the service specifications that refer to MEC dependability aspects (listed specifications. The reliability can be computed based on the in Table III) are used as references for presenting the state-of- mean time to failure or the mean time between failures. the-art dependability requirements and regulations for MEC, The survivability and safety are sometime not listed among listed in Table X. the attributes. The maintainability is associated to recovery Availability and reliability are mentioned as non-functional mechanisms and can be measured as the mean time to repair. metrics in [27]. Resiliency and high availability is mentioned 2) Issues: The dependability issues, fault and failure as in the white paper [9]. listed in Figure 7, in the daily language are used inter- In many ETSI specifications, service availability is men- changeably and mean that something that is not working. In tioned but not always with the same meaning in the depend- dependability taxonomy, they are not only independent causes ability: availability tracking API in [30]; together with appli- of lack of dependability, but they represent a cause-effect cation availability in [10]; testing the related query in [44]; sequence and they can be defined as follows [126]: testing API query in [40], [41]; together with 5G [19]; in - Fault is the adjudged or hypothesized cause of an error; connection with the coordination of inter-MEC systems and - Error is the part of the system state that is liable to lead MEC-cloud systems [45]. to a failure; The service continuity in mobility is extensively addressed - Failure is the deviation of the delivered service from the in [42], [43]. It is also mentioned in [35], [41], [45] and in correct service (i.e., the service is not anymore compliant several white papers [18], [21], [46], [49]. with the specification). The fault management is briefly addressed in [8] and more An example of the cause-effect sequence is the following: in detail in [29]. Fault management in NFV implementation an external fault flips a bit in the memory causing an error is addressed in [15]. Testing of MEH fault management in that manifests as a failure when that partition of the memory presented in [44]. API test for MEH fault management in is accessed. Table IX lists the main types of faults and discussed in [40]. Fault management is also addressed in the failures [126]. white papers [47] and [48]. 3) Countermeasures: In order to improve the dependability The network dependability is addressed in [34] and also in of a system, various categories of methods are used [126]: the white papers [11], [46]. - Fault prevention uses development methodologies, for Several specifications refers to URLLC and mission-critical both software and hardware, in order to reduce the application, for example in [26], [42] and the white papers 14

[18], [21]. The specifications highlight the importance of MEC increasing the system pressure can be investigated [132]. Note in mission-critical low-latency applications, such as Industrial that the VM can be migrated to a different host or within the IoT and Self-Driving Cars. These applications require commu- same host. Regarding NFV, a problem that can be addressed nication with very high reliability and availability, as well as is the VNF placement in a MEC-NFV environment in order to very low end-to-end latency going down to millisecond level. maximize the availability [133]. The most critical part of this V2X communication is maybe the most relevant use case works is to identify and include in the evaluation the necessary in MEC. It is initially mentioned in [12], then more in detail kinds of failures, e.g., the failures of the network, VMs, or in [43]. The V2X communication is important because it has physical machines. Moreover, considering 5G network slicing, strict requirements in all three aspects. In particular, the ETSI the protection of the network slices can be considered in the specifications mention the reliability and availability (from VNF placement [134]. both security and dependability perspectives) together with 2) MEC system level: latency and throughput. Given the high mobility of the V2X a) Deployment and failover mechanisms: As already users, one of the main topics to investigate is the handover mentioned, a manner to make a MEC system resilient to between the MEHs, which has an impact on both service the failure of MEH is to use failover mechanisms. Failover continuity and service availability [12]. Specifically, the pre- mechanisms that need a proper deployment of the MEHs, i.e., dictive handover is investigated to meet the dependability and the MEHs need to be close enough in order for a user to be performance requirements [43]. In [39], service continuity is able to reach multiple MEHs [76]. mentioned in the context of having multiple operators. The For this reason, the deployment of MEHs can be performed white paper [129] highlights the high reliability and security by maximizing the failover capability [135] or by considering requirements in the V2X communication in 5G-MEC. a 1+1 protection, where the users are able to connect to two 2) Academic publications: Table XI lists recent surveys MEHs (one is active and the other one is for backup) [136]. that address dependability aspects in MEC. As for security, Proactive failover mechanisms can be considered in order not many papers are fully dedicated to MEC but consider it to reduce the impact of the failure [137]. In case a user is together with other technologies such as cloud or fog. A survey not able to reach another MEHs, other users can be used as a on dependability on MEC does not exists. A few papers are relay in order to reach active MEHs [138]. mainly focused on dependability aspects [76], [79], [85] and b) Resource allocation: Another manner to improve the others have parts dedicated to dependability aspects [66], [67], dependability of MEC is by a proper allocation of resources, [69] or shared with other aspects [56], [60], [78], [86]. The usually computing and storage, in the different MEHs. There other papers just mention it or use it as requirement [56], [69], are already several works that address this aspect [139]–[146]. property [58], benefit [71], or challenge [66], [70]. Many of these works refer to the term task and they are talking about offloading. The term task is used to refer to an application or procedure, instead of offloading is referring C. Challenges to the movement of the execution of a task from the mobile In the following, we present dependability challenges for device to a MEH. MEC by categorizing them on MEC host level, MEC system Most of the current works have as target the energy effi- level, and general challenges. ciency and consider the dependability metrics as requirements. 1) MEC host level: The main difference between current works is the depend- a) Physical dependability: The MEH is practically con- ability metrics they are considering. Some works consider the sisting of a computer or a small server. For this reason, failure probability of MEHs to develop a k-out-of-n allocation traditional techniques for making dependable a computer or schemes, where the tasks are distributed among several MEH server can be considered. These techniques would act on both and the task is correctly executed if at least k out of n hardware and software. MEHs are not failed [139], [140]. One work considers as Given the distributed nature of the MEC system, which is reliability the probability of the delay bound violation, which composed of multiple MEHs, the alternative is to consider a is actually more similar to the performability [141]. Another MEH expendable and provide fault tolerance by migrating to a work considers an offloading failure probability, which is new MEH in case of failure [76]. In this case, careful deploy- connected to the error probability in the transmission between ment of MEHs and efficient failover mechanisms are needed. the user and the access where the MEH is located [146]. Deployment and failover mechanisms will be presented more Another work considers also the execution reliability [142]. in detail later. Finally, some works model the MEC system with queues and b) Virtualization dependability: An important character- define the reliability as outage probability. There is an outage istic of a MEH is virtualization. A MEH uses virtualization when a queue length exceeds a predefined threshold [143]– technologies, such as containers or VMs, which are managed [145]. by a VIM, such as Kubernets or Openstack [16]. Moreover, Furthermore, there are works that address the task allocation as already mentioned, the MEC architecture can be integrated together with the user-host association [145] or consider with a virtualization architecture such as NFV [15]. that different users may have different dependability require- The virtualization in the MEH needs to be considered in ments [147]. order to have a dependable MEC. For example, the live VM c) MEO dependability: The MEO is a critical element migration in Openstack while injecting network failures and because it is a single point of failure of the MEC system. A 15

TABLE X DEPENDABILITY REQUIREMENTSINTHE MECARCHITECTURE

Element Dependability Requirements General - Additional tools are needed to generate workload and challenge the service in terms of service scalability, availability, and reliability [27]. - For alarm management, the following 3GPP-defined Integration Reference Points are used: ETSI TS 132 111-2 [130] and ETSI TS 132 332 [131] [29]. - The Multi-access Traffic Steering can be used by MEC apps and MEP for seamlessly steering/splitting/duplicating application data traffic across multiple access network connections [34].

MEH MEP - The MEP shall allow MEC services to announce their availability [12]. - The MEP interacts with the MEC apps via the reference point Mp1, which provides functionalities, such as service availability and session state relocation support procedures [8], [10]. - The MEP may use available radio and core network information to optimize the mobility procedures required to support service continuity [36].

Virtualization - N/A Infrastructure MEC App - Some MEC applications expect to continue serving the UE after a location change of the UE in the mobile network. In order to provide continuity of the service, the connectivity between the device application and the MEC application needs to be maintained [8]. - Each MEC service instance that has previously registered in MEP and is conﬁgured for heartbeat sends heartbeat message to the MEP periodically in order to show that the MEC service instance is still operational [10].

MEC Host-level Management MEPM - The MEPM also receives Virtualised resources fault reports and performance measurements from the VIM for further processing [8]. - In the NFV variant, the MEPM-V does not receive Virtualised resources fault reports and performance measurements directly from the VIM, but these are routed via the VNFM [8], [15].

VIM - The VIM collects and reports the performance and fault information about the virtualised resources [8].

MEC System-level Management MEO - The MEO maintains an overall view of the MEC system based on deployed MEHs, available resources, available MEC services, and topology [8]. - The MEO interacts with the MEPM via the reference point Mm3 to keep track of available MEC services [8].

OSS - The OSS interacts with the MEPM via the reference point Mm2 for the fault management [8], [15]. - In the NFV variant, the OSS interacts with the NFVO for the fault management [15].

User app LCM - N/A proxy

dependable MEO is important because if it fails the whole 3) General challenges: MEC system became unavailable. Moreover, the MEO is also a) Dependability modelling: A first way of evaluating important to manage failure of the other MEC elements, in the dependability of a new system is to realize dependabil- order to have a fault-tolerant MEC system. For this reason, ity models. For example, this approach has been used to the design of the MEO and its functionality must be addressed evaluate the availability of SDN [151] and NFV [152]. A carefully by considering the state-of-the-art technology [47] dependability model can allow to evaluate the impact of the and techniques of Artificial Intelligence [48]. The challenges elements composing the MEC system and identify the critical for the MEO are similar to the challenges for the NFVO, the issues. Advanced models can also consider the dependability same best practice should be followed [148]. For eliminating correlation bet. The more common models techniques are the single point of failure, the MEO can be designed as derived by the Petri Nets, such as the Stochastic Activity logically centralized but physically distributed, as for the SDN Networks and Stochastic Rewards Nets (SRN), and can be controllers [149]. In this case, the coordination among the implemented by using tools such as Möbius2 or SHARPE3. different MEOs becomes critical. For example, SRN has been used to model the availability of d) Consistency: Consistency is defined as the property an edge system [153]. of multiple elements to have the same information and vision b) Network dependability: The network connectivity is of the system. The lack of consistency is a problem for a an important element in the MEC system [34]. It includes distributed implementation of the MEO, but also for naturally the access network to which the users are connected, the distributed elements as the MEH. For example, the consistency regarding the agreement in the event of a failure needs to be 2https://www.mobius.illinois.edu/ addressed [150]. 3https://sharpe.pratt.duke.edu/ 16

TABLE XI ACADEMIC SURVEYS RELATED TO MECDEPENDABILITY

Ref. Aspect MEC Main contribution Relevance to MEC dependability only [76] Fog reliability No Addresses the reliability in fog computing. Presents the reliability challenges by combining the reliability requirements of cloud computing and networks of sensors and actuators. [54] MEC general Yes Surveys the use cases and the enabling technologies. Mentions the five-nine availability. Briefly discusses the Focuses on orchestration and related challenges. reliability aspect in deploying MEC services and suggests dubbed checkpoints or, for improving the scalability, the replication of MEC service instances. Discuses also the resiliency as service enhancement. [78] Fog general No Surveys taxonomy, existing works, challenges, and future Identifies the reliability of fog computing as poorly dis- directions of fog computing. cussed topic and suggests to better investigate the consistency of fog nodes and availability of high-performance services. [56] MEC communi- No Focuses on joint radio-and-computational resource man- Diffusely mentions reliability in various contexts (link, cation agement transmission, server). Focuses more in detail on mobility- aware fault-tolerant MEC. Mentions resiliency and high availability of MEC system as requirement. [58] Edge general No Provides an overview on the sate of the art and the future Inserts fault tolerance as property in comparing the research directions for edge computing. existing frameworks in edge computing. Mentions the availability of resources by stating that it is mostly dependent upon server capacity and wireless access medium for ensuring constant service delivery. [59] Edge security No Reviews some security and resilience aspects in MEC Generally discusses resilience requirements and chal- and resilience and fog in relation to cloud technologies. lenges in MEC together with fog and in comparison with cloud. [82] Edge/IoT No Surveys how edge computing improves the performance Addresses the dependability with respect to the storage of IoT networks by focusing on the recovery policy. [60] Edge general No Presents an overview of potential, trends, and challenges Mentions fault tolerance (together with Quality of Ser- of edge computing. vice) as one of the challenges on edge computing. Dis- cusses about the need of proactive fault tolerance and automatic recovery. [63] MEC/IoT Yes Overviews the MEC technology for the realisation of IoT Diffusely mentions reliability in various contexts. security applications. [66] Edge general No Surveys edge computing, identifies requirements and Presents the provision of low-cost fault tolerant deploy- discusses open challenges. ment models as open challenge. [67] Edge general No Provides a tutorial on edge computing and related taxon- Considers the RAS (Reliability Availability Survivabil- omy and surveys the state-of-the-art efforts. ity) as objective and the resilient fog system design as challenge. [69] MEC Industrial Yes Surveys the existing works on MEc for Industrial Inter- Discusses the high reliability as typical industrial require- Internet net. ment. [70] MEC general Yes Surveys MEC fundamentals, discussing its integration Considers the reliability as a challenge in MEC and within the 5G network and relation to similar UAV diffusely discuses it. communication, IoT, machine learning and others. [85] Edge dependabil- No Explores dependability and deployment challenges in Presents resiliency challenges, which includes new fail- ity edge computing. Considers dependability in a wider ures modes, network impact, limited fail-over options, meaning that includes security. multi-tenancy support, and interoperability. [79] Fog dependabil- No Surveys the research efforts on fault tolerance and de- Presents redundancy models and fault management so- ity pendability in fog computing. lutions. Discusses the dependability challenges and open problems. [71] Edge general Yes Surveys different aspects of edge computing from an Mentions the improvement of reliability as a benefit of architectural point of view. edge computing. [86] Wireless edge No Discusses the feasibility and potential of providing edge Overviews the challenges and the enablers for realizing computing services with latency and reliability guaran- hugh reliability in wireless edge computing. tees. [72] MEC general Yes Surveys the MEC and focuses on the optimization of the Diffusely mentions reliability, availability, and resilience. MEC resources [88] Vehicular Edge No Surveys the state of the art of vehicular edge computing. Diffusely mentions reliability. Computing

connection between the access and the MEHs, the connections c) Service continuity and user mobility: One property between the MEHs, and the connection between the MEHs and that is carefully addressed in the ETSI speciﬁcation is the the MEO. Unreliable network connectivity can have a huge service continuity [42], [43]. The importance is also due to the impact on the dependability of the MEC system [11], [46]. nature of MEC, or more precisely MEC with mobile access Reliable network connectivity can be provided via physical networks. redundancy and dedicated protocols. ETSI deﬁnes the service continuity as "the perception of the out-of-service time and can be measured by the latency 17

Figure 8 represents the taxonomy related to the performance. The attributes are the various metrics to evaluate the performance of a system. The issues are the causes that may lead to a lack of performance. The countermeasures are the methods to address the performance issues. The attributes are presented first from a general perspective, then according to what has been defined by ETSI [27]. The issues and the countermeasures are related to a networking and computing context (which MEC belongs to), because having a general perspective would have been resulted in a too broad introduction. 1) Attributes: The performance of a system can be eval- uated by means of metrics. There are different well-known metrics that can be divided into two classes. A first class contains general metrics focused on only one aspect, such as data transport service. The following well-known metrics belong to this class:

Fig. 8. Performance Taxonomy - Throughput is the number of bits or messages successfully delivered per unit time. - Latency is the delay introduced for completing a service, between the terminated instance and the resumed instance e.g., delivering a block of data between two points of the of the same service, maintaining the instance state" [42]. system. The causes of a lack of continuity are application software - Jitter allows to quantify the latency variation. failure, malfunction of MEC system, loss of connectivity due - Loss rate refers to the amount of bits or messages per to network failure, and user’s voluntary or involuntary action. unit of time that are lost during the service. Depending on the scenario and the application, ETSI de- All these metrics can be measured at different layers of the fines different levels of service continuity: no continuity, low protocol stack, depending on the particular service considered continuity, soft continuity, and hard continuity [42]. in the performance evaluation. For example, measuring the d) Failure management: MEC might introduce more latency at the network layer allows to quantify the delay complex failure modes [79], [85]. This situation is a common introduced only by the network. This kind of metric is not problem in modern complex ICT systems [154]. Moreover, sufficient to quantify the performance of a Voice-over-IP faults and failures in MEC (as generally in computing plat- (VoIP) service because it does not take into account the latency forms) are hard to detect [76]. added by the application during the elaboration of the received The fault management is diffusely addressed in the ETSI bits [157]. The second class is composed of metrics that aim specifications [29]. Anyway, there are aspects that need to be to summarize in a value the performance of a complex service properly addressed: uncontrolled error propagation by defining that requires the interaction of a set of components. Most of error-containment regions; recovery of faulty components [79]; the metrics belonging to this class are related to the concept extreme event control [86]. of Quality of Service (QoS) or Quality of Experience (QoE). e) Dependable architecture: Beyond the ETSI archi- These metrics can be classified as follows [157]: tecture, alternative edge computing architectures have been - Subjective metrics, which require the involvement of proposed in order to improve dependability. One architecture humans for quantifying the experimented performance of aims to deliver failure resistant and efficient applications [155]. the service. The quantification is performed using some Another work proposes a dependable edge computing archi- reference scale, such as that defined in the Mean-Opinion- tecture customized for smart construction [156]. Score (MOS) procedure and its evolution [158]. - Objective metrics, which allow to quantify the perfor- V. PERFORMANCE mance by using machine-executable algorithms, such as PSNR (Peak Signal-to-Noise Ratio used for quantifying The performance is usually the main target of a new the video signal quality). technology, where security and dependability are aspects that need to be guaranteed. For this reason, the research community Starting from this general definition, the ETSI docu- has focused on performance on MEC and many works are ment [27] has defined a set of metrics specifically designed on the topic although that the number of surveys on MEC for the MEC systems. Given the high flexibility of MEC performance is limited. architecture, the MEC metrics have been defined with the following two goals: - Evaluate the performance increase given by a MEC A. Taxonomy solution with respect to a non-MEC one; Given the wide nature of the performance, this subsection - Compare the performance of different MEH locations presents the performance taxonomy specific for MEC. within the network in order to select the most suitable 18

by the users, such as non-user data volume exchange or energy efficiency at network side. Indeed, these performance parameters impact the costs necessary to the network operator to manage the service. On the other hand, some metrics, such as delay to process API request and number of failed API requests, negatively impact the user experience. Indeed, a high delay in processing API request can increase the set-up time of a new service request and/or the service processing time. For both classes, the metrics need to be adapted to the particular MEC use case. In particular, the actual assessment of these metrics can depend on the particular service and/or application. For example, the latency in localization (time to fix the position) is different from latency in content delivery. In both cases, one could measure all the statistics over the above metrics. In fact, all metrics are in principle time-variable, and could be measured in a defined time interval and described Fig. 9. Functional Metrics by a profile over time or summarized through the following values: - Maximum value; - Mean and minimum value; - Standard deviation; - Value of a given percentile. 2) Issues: Poor system design is one of the most important performance issue. The system design deals with the definition of the resources (and their location) necessary for supporting the user services. For example, having no (or insufficient) resources in the area where user requires URLLC services leads to support the service in a remote server, where the simple propagation delay is higher than the maximum accepted delay. The remoteness of resource location for video-on-demand services implies the involvement of many network resources with consequently a consumption increase of communication resources and energy. Resources congestion can impact the different performance Fig. 10. Non-Functional Metrics parameters, depending on the kind of congested resources. For example, the communication resources congestion increases MEH for the considered use case. the latency for data transferring, the lack of storage resources can add data loss, whereas insufficient computation resource Taking into account these goals, ETSI defined two adds delay in processing data. classes [27], which we will further present by summering the Service deployment can manifest problems at network layer content of the ETSI document: and service layer due to configuration errors or bad archi- - Functional metrics: they quantify MEC performance im- tecture (hardware and software) choices, which have a deep pacting on user perception (often called Key Performance impact on performance. Indicators, KPIs). The set of KPIs is shown in Figure 9. User mobility adds variability to the features of the con- - Non-functional metrics: they are related to the perfor- nection with the resources providing the service. The user mance of the service in terms of deployment and man- movement can degrade the data rate available at the access agement. Figure 10 describes the set of non-functional network and increase the latency for achieving the service metrics. location. In some cases, in the new user location there are no As shown in Figure 9, the functional metrics consider resource to support the service, leading to service interruption. the performance by both taking into account single aspects 3) Countermeasures: To solve the presented issues, there of the service (i.e., latency, jitter, loss rate, throughput, and are some general approaches. Poor system design issues can energy efficiency) and referring to metrics that summarize the be coped with a preventive analysis of the amount of services interaction of multiple components forming a complex service and related features, in terms of offered load and service (i.e., all the metrics under the umbrella indicated as quality). requirements. This information is then used as input in models The non-functional metrics in Figure 10 are strictly related developed for system design. The models differ from the main to the aspects that are important for the network operator, target of the design. The most common target is to reduce i.e., deployment and management aspects. In some cases, the capital expenditures (CAPEX) and the operating expenses these aspects are not related to the performance perceived (OPEX) by maximizing the number services with satisfied 19

TABLE XII SUMMARY OF PERFORMANCE ISSUESAND COUNTERMEASURES

Issue Description Countermeasure

System design - Poor networking, memory and computation resources - Forecasting of the necessary resources for each class of negatively impact latency, loss rate, throughput and quality. services, deﬁned by taking into account the performance - Bad resources location can negatively effects the perfor- requirements and the offered load. mance also in the case of resource overprovisioning (but - Usage of models for system design starting from the badly located). forecasted service requests and requirements.

Resources congestion - The resources congestion can deeply impact the latency, - Reactive congestion control techniques aim to detect the throughput, loss rate, and quality at different layers. congestion status of the system and then react by using strategies aimed at reducing the time for which the system is in this state, in order to reduce the negative effects of a congestion event on the user service. - Proactive congestion control techniques aim to prevent the congestion state of the system resources.

Service deployment - Service deployment errors and problems: at commu- - Usage of management and monitoring tools at different nication layer (e.g., IP address plan or routing protocols layers helps to detect and solve issues related to deployment conﬁguration errors, and radio channel selections) and errors. at service layer (e.g., software and hardware architecture - These performance issues can be avoided by proactively selected for the service deployment). evaluating the performance of alternative hardware - The network softwarization and, in general, the service and software architectures, e.g. service implementation virtualization add further issues related to the used virtu- in dedicated hardware, used CPU and OS, connections of alization technology: these technologies differ in terms of different components of a complex services, used virtual- required physical resource (e.g., memory required by the ization technique (e.g., hypervisor-based, containers, and image and CPU utilization), and offered performance (e.g., Unikernel) in the case of service implemented in general- latency and throughput). purpose hardware.

User mobility - High variability of the edge resource utilization can lead - Definition of techniques for managing user mobility at to temporary congestion, degradation of latency and, in network layer, such as the handover management defined some cases, loss of data. in mobile networks, at service layer, such as live service - The user may move away from the location of the migration permitted by agile virtualization technology. resources supporting the requested services, degrading the - In both cases, the forecasting of the user movements performance and, in some cases, interrupting the service can allow to proactively reserve resources and start all when there is no (or insufficient) resource in the new operations related to the connection handover and/or ser- position. vice migration, reducing a lot all the issues related to user mobility.

requirements. Recently, some models aim to reduce energy distributed cloud. consumption. The issues related to user mobility is well-known in the To cope with the resources congestion issues, there are mobile networks, where different techniques have been de- two big classes of congestion control approaches: reactive signed to maintain the network performance during the user and proactive. The TCP congestion control is an example of movements. Similar issues need to be considered at service reactive approaches, whereas admission control of new service layer in the case of critical services, where the time necessary requests is an example of proactive congestion control. This to achieve the resources offering the services is higher than control consists in rejecting new service requests when their the minimum latency requirements. In this case, the service admission degrades the performance of the already accepted migration is necessary. In all cases, the forecasting of the user services. The congestion is prevented analysing the available movements can allow to proactively reserve resources and start resources and estimating the required ones by the new re- all operations related to the connection handover and/or service quested service. migration, reducing a lot all the issues related to user mobility. The usage of management and the monitoring tools at dif- A summary of performance issues and countermeasures is ferent layers represents an important countermeasure against shown in Table XII. service deployment problems. They help to detect and solve issues related to device, network and service conﬁguration. The analysis of alternative hardware and software architectures is B. State of the Art necessary to prevent performance issues related to bad choices 1) Standards, regulations, and white papers: Table XIII on deployment aspects, such as service implementation in ded- shows the ETSI speciﬁcations that represent the state-of-the- icated hardware, connection among the different components art of performance requirements and regulations for MEC. of a complex service, and virtualization technology used for This list has been obtained analysing the documents shown service (or of one component) implementation in a central or in Table III). 20

Several API-focused standards [34], [36] describe some the shared container kernel, Kata Containers have recently performance requirements to implement in some MEC el- been proposed. Kata Containers act and perform like classic ements. The standards that define the performance metrics containers but provide stronger workload isolation by using is [27]. A lot of use cases with the related performance hardware virtualization technology as a second layer of de- requirements are described in [12]. Whereas studies for the fense [166]. Unikernels are lightweight machine images that MEC support of some important services and technology enclose the application and require only OS libraries and aspects, such as network slicing [26], NFV [15], mobility [42], dependencies. It works in a single address space and has a and alternative virtualization technologies [16], indicate the much smaller attack surface due to its minimal nature. Each performance required to the MEC elements. An important of these technologies faces a trade-off between isolation and document describes the MEC 5G integration [19]. agility. The studies presented in [163], [164], [167] show the The white papers listed in Table III mainly discuss the performance in terms of the following parameters: problems and solutions related to the MEC deployment with - Image size of the service, which impacts the amount of 4G, 5G, and cloud RAN (CRAN) [21] [18] [23]. In these storage required to host the application; documents, some performance issues are presented, but no - CPU and memory utilization of the service, which has a explicit performance requirements are clearly defined for the great impact on the number of services that a physical MEC elements. The enhanced DNS can have a key role for server can run simultaneously; the performance of MEC services because it enriches the list - Throughput of service requests; of deployment options suitable to support the distributed MEC - Delay in responding to user requests, which can drasti- environment, in terms of providing the connectivity between cally degrade the user’s satisfaction from the service (a devices and application instances [50]. website in the experimental analysis). 2) Academic publications: We can find the discussion of For example, the results of [163] show a boot time of the performance issues (and, in some cases, related solutions) of container (0.623 s) lower than that of VM (32 s), whereas the MEC deployment in many of the technical surveys dedicated service throughput of the implementation of a HTTP server to MEC. Among these, in Table XIV, we summarize the most with VM (130 req./s) is higher than that of Container (143.4 updated and important from the performance perspective. req./s) Kata Containers and unikernels are in their development C. Challenges phase, but they provide serious competition to the ecosystem of containers and VMs. Indeed, they offer a lightweight solu- In the following, we present performance challenges for tion for the deployment and migration of virtualized services, MEC by categorizing them on MEC host level and MEC improving the performance in terms of latency, throughput, system level. and quality. These improvements are more evident in pres- 1) MEC host level: ence of user mobility, where the agility and the lightweight a) Virtualization performance: The MEC architecture al- of the virtualized services are key factors. However, Kata lows not only to improve the performance of network functions Containers are not mature yet for using them in production implemented in the form of VNF, close to the users, but also environments managing data-intensive workloads, as shown to be prepared to host third-party services, creating a new by the experimental results presented in [164]. Indeed, this market for the operator [162]. The virtualization technologies paper confirms the results shown in [163]: Kata Containers can deeply impact the performance of the MEC system, are not really efficient in terms of memory consumption and as shown by some works (such as the most recent [163], speed, while still being a good deployment choice in security- [164] and reference therein), which presented an experimental sensitive multi-tenant environments. comparison of virtualization technologies for implementing b) Performance isolation: As shown in [168] the per- VNF and edge services. This well-known result spurs the study formance isolation can be a very critical issue to taking of new virtualization technologies and deployment paradigms into account when VM shares the physical resources. With in order to improve performance and resource efficiency. From some initial experiments to understand the co-existential or container-based virtualization to micro virtual machines, new neighbor-dependent behavior of VMs, the authors inferred virtualization solutions claim to offer performance close to that the problem of performance isolation can be improved bare metal, with quick deployment and startup times. by reducing the resource contention amongst the VMs on the Recent works analyzed the performance of multiple virtu- same physical host. Performance isolation can be drawn to the alization technologies, including VMs, containers, unikernels, lowest level abstraction of shared resources, like CPU, mem- and Kata Containers [163].VMs have traditionally been the ory, network, and disk. For example, the disk is continuously primary technology for VNF deployment, creating an isolated being used by multiple processes waiting in I/O queue. Thus, and secure environment with high associated overhead. To I/O scheduler will play a vital role in resource contention, reduce the memory clutter of VMs, containers represent an as studied in [169] [170]. In [171], the authors present a interesting alternative solution, as they pack the application systematic overview of existing isolation techniques in nodes and its dependencies into a light and agile entity that can be and networks, especially in RAN and CN of 5G systems. run on any platform. However, due to the underlying shared 2) MEC system level: kernel, containers face security problems in a multi-tenant a) MEC deployment: The network operator selects the environment [165]. To address the security issues related to number of MEHs and their location analyzing different tech- 21

TABLE XIII PERFORMANCE REQUIREMENTSINTHE MECARCHITECTURE

Element Performance Requirements General - A large set of use cases is described with the goal of deriving useful requirements. However, some requirements are defined by design constraints and do not originate from use cases [12]. - A number of performance metrics which can be used to demonstrate the benefits of deploying services and applications on a MEH is presented in [27]. - The MEC 5G integration implies several key issues. The issues and their related solution proposals are discussed in [18], [19]. - The current QoS-related information in RNI API is not sufficient in order to allow necessary prediction regarding the QoS performance (e.g. latency, throughput, reliability). Therefore, potential enhancements on RNI API for the prediction should be considered including both relevant measurements in RAN or processed results for the prediction [43].

MEH MEP - The MEP on a MEH provides a framework for delivering MEC services and platform essential functionality to MEC applications running on the MEH [12].

Virtualization - Multiple Alternative Virtualization Technologies (AVTs) need to be supported in the same MEH to satisfy the Infrastructure performance requirements of heterogeneous MEC services [16]. The different AVT solutions have an impact on MEC framework and on MEC management APIs [16].

MEC App - When MEC is deployed in an NFV environment, the most simple possibility to realize the Data Plane needs the support of a physical network function or VNF or combination thereof, and its connection to the network service that contains the MEC app VNFs [15].

MEC Host-level Management MEPM - The MEPM also receives Virtualized resources fault reports and performance measurements from the VIM for further processing [8]. - In the NFV variant, the MEPM-V does not receive virtualized resources fault reports and performance measurements directly from the VIM, but these are routed via the VNFM [8], [15]. - The MEPM shall be able to collect and expose performance data regarding the virtualisation environment of the MEH related to a specific MEC app and a specific MEC app instance [12]. These data can be used to verify how well the Service-Level Agreements (SLAs) are met. - Different MEC applications running in parallel on the same MEH may require specific static/dynamic up/down bandwidth resources, including bandwidth size and bandwidth priority. To this aim, optional traffic management services are described in [34]. - The MEPM should be able to provide different sets of features/services in distinct Network Slice Instances (NSIs) [26]. - The MEPM should be able to provide the same feature/service differently in distinct NSIs [26]. - The MEPM collects usage and performance data per NSI [26]. - The MEPM-V can access the performance management and fault information of virtualized resources related to a particular ME app VNF instance that is lifecycle-managed by the VNFM [15].

VIM - The VIM collects and reports the performance and fault information about the virtualised resources [8].

MEC System-level Management MEO - The MEC system management should support management of MEHs, including suspend, resume, conﬁgure, add and remove, by an authorized third-party [12]. - The functional requirements related to application mobility are reported in [36]. Examples of functional requirements are the maintaining of the connectivity between a UE and an application instance when the UE performs a handover to another cell associated/not associated with the same MEH, the support of two instances of a MEC application running on different MEHs to communicate with each other, or the use of radio/core network information for optimizing the mobility procedures required to support service continuity. - The MEO is able to trigger the application relocation owing to external relocation request, unsatisﬁed performance requirements, load balancing and disaster recovery [42].

MEAO - The MEC system management shall be able to expose up to date performance data of the application to the authorized third-parties such as application developers and application providers [12]. - When MEC is deployed in an NFV environment, for performance enhancements, it can make sense to re-use the SFC functionality provided by the underlying NFVI for traffic routing. Differently from the simple solution based on the MEC App VNFs, in such a deployment, the Data Plane is obtained using the MEAO without a dedicated MEC App. The MEAO will need to translate the traffic rules into a Network Forwarding Path (NFP) and send it to the NFVO. The MEP will not control the traffic redirection directly, but will pass requests to activate / deactivate / update for traffic rules to the MEPM-V which will forward them to the MEAO. When receiving such a request, the MEAO will request the NFVO to update the NFP accordingly [15].

OSS - N/A

User app LCM - N/A proxy 22

TABLE XIV ACADEMIC SURVEYS RELATED TO MECPERFORMANCE

Ref. Aspect MEC Main contribution Relevance to MEC performance only [54] MEC general Yes Surveys the use cases and the enabling technologies. Discusses the role of MEO in orchestrating resource Focuses on orchestration and related challenges. allocation and service placement for assuring efficient network utilization and QoE. Provides a qualitative comparison of different orchestrator deployment options. [56] MEC communi- No Focuses on joint radio-and-computational resource man- Provides a summary of future research directions for cation agement. the joint radio-and-computational resource management referring to the performance issues related to the deployment of MEC systems, the Mobility management for MEC and the Green MEC. [58] Edge general No Provides an overview on the sate of the art and the future Provides a comprehensive review and comparison of the research directions for edge computing. prevalent MEC frameworks. The comparison is based on different parameters, such as system performance, network performance, overhead of deployment, and system migration overhead. [82] Edge general No Presents an overview of potential, trends, and challenges Discusses performance challenges related to IoT applica- of edge computing referring to the IoT Applications. tions. [63] MEC for IoT Yes Holistic overview on the exploitation of the MEC tech- Presents research topics related to MEC service level nology for the realisation of IoT applications. congestion control, latency aware routing, and dynamic application routing. [66] Edge general No Surveys edge computing, identifies requirements and Presents a comparison of MEC solutions aimed to opti- discusses open challenges. mize the execution cost and deployment, reduce network latency, minimize energy consumption, and maximize throughput. [70] MEC general Yes Surveys MEC fundamentals, discussing its integration Discusses the lessons learned, open challenges and future within the 5G network and relation to similar UAV directions on the performance of MEC integrated with the communication, IoT, machine learning and others. forthcoming 5G technologies. [72] MEC general Yes Surveys the MEC and focuses on the optimization of the Provides a state-of-the-art study on the different ap- MEC resources. proaches that optimize the MEC resources and its QoS parameters (i.e., processing, storage, memory, bandwidth, energy and latency). [86] Wireless edge No Discusses the feasibility and potential of providing edge Overviews the challenges and the enablers for achieving computing services with latency and reliability guaran- low-latency and high-reliability networking; discusses tees. network resources optimization techniques for a selection of use cases. [88] Vehicular Edge No Surveys the state of the art of vehicular edge computing. Presents a summary of techniques for caching, task Computing offloading, and data management highlighting the considered performance parameter. MEC and Game Yes Discusses how Game Theory can address the emerging Overviews of Game Theory models for achieving a [159] Theory requirements of MEC use cases. performance-cost balance in realistic edge network scenarios, and prospected future trends and research directions in the application of game theory in future MEC services. Edge and IIoT No Discusses the opportunities and challenges of edge com- Overviews of schemes for routing and task scheduling [160] puting in IIoT. for performance improvements. Describes challenges and potential solutions of applying some new technologies to edge-based IIoT. MEC for Indus- Yes Explores how the MEC is used and how it will enable Discusses MEC deployment bottlenecks and related [161] trial Verticals industrial verticals. solutions for performance improvements in a smart metropolitan area, where disparate verticals can coexist.

nical and business parameters, such as available site facilities, user device and the MEH hosting the application, which supported applications and their requirements, measured or enables low-latency services; estimated user load etc. As shown in Figure 5, the network - the access to user context, such as the user channel quality operator has four different deployment options. conditions or user location, which enables context-aware MEHs can be deployed in different network locations: from services, e.g. services adaptive to network conditions. near the gNB to a remote data network. Although running The design of MEHs location and the instantiation of MEC MEHs far from the edge can be useful in scenarios in which services require the analysis of multiple trade-offs for efficient compute power requirements are stricter than latency ones, usage of physical and virtualized resources. Depending on the most interesting scenario is represented by locating MEHs the use case, the processing power demands and the latency close to the user (e.g., at the gNBs of a 5G system). This requirements can be very heterogeneous. Other than the QoE scenario adds two very important features for enabling new of the users, the location of MEHs must consider the Total services: Cost of Ownership (TCO). On one hand, the centralized cloud - the reduction to low values of the delay between the end- decreases the TCO. On the other hand, the centralized cloud 23 fails to address the low latency requirements. A good trade- mize the data caching revenue of the operator [181], or that off is achieved by utilizing the existing infrastructures (e.g. improve content delivery speeds, network traffic congestion, Telco’s tower, central offices, and other Telco real estates). cache resource utilization efficiency, and users’ quality of ex- In the case of a mobile network, the MEHs can be located perience in highly populated cities [182]. Referring to the VNF in a different part of the network architecture, ranging from architecture, the SFC and VNF placement can be performed uCPEs (mainly at the customer premises) to RAN-edge (i.e., reducing the execution time and the resource utilization [183], with MEHs co-located with base stations), or to Smart Central taking into account both service requirements and the resource Offices, where the MEH could be, e.g., hosted as co-located capacity in the edge [184], maximizing revenue at the network with CRAN aggregation points, or again, to edge data centers level while matching demand [185], minimizing both energy at local/regional level. consumption and resource utilization [186], or maximizing The deployment problem has been analyzed by considering the number of user request admissions while minimizing different optimization approaches and performance parame- their admission cost (i.e., computing cost on instantiations of ters. For example, using the Shanghai Telecom’s base station requested VNF instances and the data packet traffic processing dataset, some works consider the MEH placement problem of requests in their VNF instances, and the communication with the goal of minimizing the energy consumption [172] or cost of routing data packet traffic of requests between users balancing the workloads of MEHs while minimizing the access and the MEH hosting their requested VNF instances) [187]. delay between the mobile user and MEH [173] [174]. Other c) User association: The most simple strategy for the recent works aim to minimize the cost of service providers user-MEH association is to allocate the nearest MEH that while guaranteeing the completion time of services [175], offers the requested service. Indeed, this approach agrees on or to minimize the number of MEHs while ensuring some the proximity strategy that is desired from the performance QoS requirements [176]. In general, the deployment of MEHs perspective, e.g. latency, energy consumption. However, this requires to define multi-objective problems, such as i) find simple strategy can lead to performance degradation when a Pareto front optimizing the time cost of IoT applications, other aspects, such as the load of the MEHs, the available load balance and energy consumption of MEHs [173], ii) transmission capacity, and the users’ mobility are not consid- optimizing the response time taking into account heterogeneity ered. For example, an increase in requests for MEC services of MEC/cloud systems and the response time fairness of base in a given area can lead to an overload of the MEC resources, stations, which may significantly degrade the system quality of generating performance bottlenecks. To avoid this problem, services to mobile users [177], iii) finding an optimized trade- the user-MEH association strategy needs to take into account off between response delay and energy consumption [178]. the load status of the MEC infrastructure. Different works An interesting issue is the update of the MEC infrastructure. propose optimization solutions for some metrics, such as As an example, in [179] the authors consider the problem latency, QoE, that simultaneously distribute the load between of scaling up an edge computing deployment by selecting the different servers, e.g. [188]. Furthermore, in the case of mobile optimal number of new MEHs and their placement and re- users, the selection of the edge cloud becomes crucial due to allocating access points optimally to the old and new MEHs. the uncertainty of movement and wireless conditions. In this In this case, the considered performance is the Quality of scenario, other than the MEC resources, the user mobility im- Experience of users and the QoS of the network operator. As pacts also the available network capacity, given the uncertainty concerning the integration MEC-5G, an interesting problem is of the number of users sharing the resources and, in the case of the determination of the MEH and UPFs optimal number and wireless access network, of the wireless channel that impacts locations to minimize overall costs while satisfying the service the radio resource efficiency. requirements [180]. The user-MEH association problem can consider different b) Resource allocation: Taking into account the avail- aspects, such as i) the enhancement of the user allocation able resources, the network operator can solve the service rate, server hiring cost, and energy consumption by means placement problem. Placing MEC services over a number of of an association scheme able to consider that edge users’ MEHs can prove to be critical for the user QoE and should resource demands arrive and depart dynamically [189], ii) the take into account gravity points, e.g., shopping malls, which dynamic balancing of the computation load for neighboring attract a plethora of users. Indeed, a bad design of the service MEHs [190]. Other approaches jointly consider the MEH placement can lead to the saturation of the available resources placement and association problem with the goal of minimiz- with consequent rejection of service requests or a worsening ing the deployment cost [191] or number of MEH [192] while of the user experience. The flexible availability of resources guaranteeing certain end-to-end service latency. plays a crucial role in the performance of a service. For 3) General challenges: example, wireless bandwidth and computing resources can a) Service continuity and user mobility: Service mobility be dynamically assigned to a service for achieving optimal is a key aspect that can impact the MAC performance in a benefits from MEC system. Orchestrating a MEC system in mobile network scenario. Indeed, to maintain a high QoE it terms of resource allocation and service placement is critical is of primary importance not only to establish a connection for assuring efficient network resource utilization, QoE, and between the end user and MEC resources but also to maintain reliability. it throughout all the necessary stages. Optimal end-to-end A set of works considers the data caching problem in the session connectivity needs to be maintained for the entire edge computing environment, proposing schemes that maxi- course of service usage. To achieve this goal, the MEC services 24 should be able to migrate quickly depending on user movements. The user movements can frequently change the anchor points of MEC services (e.g., from one MEH to another). In this scenario, ensuring optimum QoE for a delivered MEC service becomes challenging, especially for delay sensitive applications. At the IP level, the Distributed Mobility Man- agement (DMM) [193] represents a notable solution towards managing user mobility, overcoming also the scalability and reliability drawbacks of centralized mobility schemes. At the service level, the management of IP mobility is not sufficient to avoid the quality degradation due to the redirection of MEC service requests of the mobile user to a distant edge hosting the service. The MEC architecture aims for a single Fig. 11. Examples of Conflicts hop connectivity to the service. When the user is moving away from the MEH serving its request, the MEC service should migrate from the old to a new MEH closer to the new user Three examples of conflicts are shown in Figure 11. Reading location for maintaining the same performance. Procedures the figure clockwise, the examples are the following: the usage for service migration take time, especially when these require of encryption to gain security (more precise, confidentiality) moving large amounts of data. The user can experience a causes a delay and therefore a reduction of the performance; degradation of application performance, and in some cases, the the reduction of the energy consumption can be achieved service continuity cannot be guarantee. Numerous studies have by consolidation (i.e., reduction of the active elements), but addressed the problem of VM migration, but new technical the consolidation can create a single point of failure and challenges have emerged from the analysis of the problem therefore a reduction of the dependability; the dependability from a service point perspective [194]. In particular, the time can be achieved by redundancy (i.e., deployment of multiple it takes to prepare a VM for the target MEH, transfer it over the elements that perform the same functionality), the redundancy network, and finally deal with the problem of changing the IP can increase the exposure risk since more elements can be address after relocating the VM, makes it difficult to achieve attacked. service continuity. The analysis of this problem starts from the All potential conflicts will need to be studied in the design study of the features of different virtualization technologies. and operations of the future 5G-MEC systems. In this section, Other than presenting remarkable overhead in terms of both we present the challenges when these three aspects are con- processing and storage capabilities, hypervisor-based virtual- sidered together. The presented challenges are summarized in ization techniques show high latency for start-up activation and Table XV. migration procedures. Container-based virtualization enables 1) MEC host level: high density deployment of services, has limited features a) Physical device: Section IV-C has introduced that to support stateful service migration between different host physical dependability can be achieved by using redundant MEHs. In both classes, the main drawbacks derive from the hardware and software within a MEH. Similarly, Section III-C stateful nature of the application, which implies that the state has introduced the challenges related to physical security of the application and the network stack must be preserved in at the MEH level. The redundancy can have an impact on the case of migration or failure. To alleviate these drawbacks, performance and security. For example, the adding of redun- the stateful "Follow me Cloud" paradigm has been proposed dant physical devices increases the vulnerability related to [195]. Based on this new paradigm, some recent works propose direct physical access (e.g., the attacker can perform physical mechanisms for fast container-based live migration, such as modifications on the device or on the communication link). [196], [197]. In [198], the authors present a survey on the The countermeasures against these threats increase the costs techniques for service migration at the edge of the network. and can impact the performance. On the other hand, it was The development of stateless applications relies on user inputs already mentioned that possible alteration or deletion of data or distributed shared storage, avoiding the store internal states. (as a consequence of an attack) requires adequate backup This feature allows a stateless application to be replicated and recovery techniques, which are feasible by means of on different MEHs. Based on the specific offloading request dependability. Cryptographic primitives such as secret sharing and current connectivity quality, the most appropriate instance schemes can be useful to allow recovery capabilities with could be selected. Examples of studies on the performance of lower duplication rates. Moreover, physical security mecha- stateless migration are [199] [200]. nisms are normally beneficial for performance, as they usually introduce less latency [62], [70], [115]. VI.MULTI-ASPECT CHALLENGES b) Virtualization technology: In the previous sections, Dependability, security, and performance are all three im- the importance of virtualization for each aspect has been portant aspects in 5G MEC. These aspects are usually ad- introduced. dressed individually but they are not independent and they From dependability perspective, the live VM migration in can be actually conflicting (i.e., a solution for improving one Openstack [132] and VNF placement in a MEC-NFV envi- aspect may impact the others). ronment [133], [134] have been introduced in Section IV-C. 25

TABLE XV SUMMARY OF THE MULTI-ASPECT CHALLENGES

Topic Challenges MEC Host Level Physical device - Evaluate the impact on performance and security when redundant hardware and software is used to increase the dependability of a MEH. - Investigate the use of cryptographic primitives (e.g., secret sharing schemes) to achieve dependability with a decreated rate of duplicated data and lightweight cryptography to achieve better performance.

Virtualization technology - Develop VM migration and VNF allocation algorithms able to jointly take into the requirements in all the three aspects. - Explore new methods aimed to secure the container-based virtualization without degrading its performance. - Develop new lightweight and easy-to-instantiate virtualization. technologies or new mixing and nesting models of virtualization technologies able to support MEC URLL services with security constraints.

MEC System Level Deployment and design - Evaluate the deployment of MEHs including dependability (failover), performance and security requirements. - Develop new failover mechanisms able to jointly tale into account security and performance constraints. - Deﬁne robust design models aimed to optimize performance while satisfying dependability constraints.

Resource allocation - The allocation of computing and storage resources, also called task ofﬂoading, should not only aim to performance targets (such as energy efﬁciency or latency reduction), but also consider dependability and security requirements.

User association - New multi-constrained path computation algorithms need to be developed for establishing user association methods able to jointly satisfy the constraints imposed by the three aspects. - Other than performance-related metrics, new metrics need to be defined for security and dependability to find finding Pareto frontier when the three aspects are jointly considered.

MEO - The reliability and the performance of the MEO system can be increased by the design of distributed MEO architecture, which must be deeply analyzed also from the security perspective. - The MEO functionalities should aim to orchestrate and manage the MEC system by jointly considering the three aspects.

Consistency - The consistency is an important requirement given the distributed and redundant nature of the system. These two features aim to improve the scalability and dependability of the system, but their impact on performance and dependability should be investigated. - Deeply analyse new methods proposed for guaranteeing consistency (even in the case of active adversaries).

General Challenges Modelling - Develop new models for a joint evaluation of the 5G-MEC system from the three different perspectives.

Network connectivity - The network connectivity has an important impact on dependability and performance requirements and is source of potential threats. - New technologies are under development for increasing connectivity performance, such as Teraherz communications. This activity must be integrated by the development of new models for analysing the impact of these technologies on the MEC system reliability and performance jointly. - Develop new solutions for mMTC and URLLC use cases, such as those based on NOMA, able to improve system capacity, while satisfying the constraints on dependability and security, in scenarios characterized by a large number of users exchanging small blocks of data.

Service continuity and user - The solutions for achieving service continuity (even in case of user mobility) may lead to conﬂicts mobility between the three aspects. New solutions must be developed considering the effects on these three aspects jointly, trying to achieve Pareto optimal.

Monitoring and detection - The monitoring and detection for security, dependability, and performance should be jointly executed to exploit the cross information. For the joint monitoring and detection, AI-based techniques can provide extra capabilities.

In both cases, the dependability is investigated together with the performance. In [132], the authors analyze the impact of 26 the system pressures and network failures on the performance to runtime errors caused by various events, such as software of VM live migration by considering the migration time. In exceptions, hardware errors, cyber attacks, etc., similar to [133], the authors address the problem of VNF placement by what occurs in cloud servers [201]. When this failure events considering two conflicting objectives, namely minimizing the happen, the design objectives and corresponding constraints access latency and maximizing the service availability. can easily be violated. The quality of experience will decrease The MEC scenario can be characterized by lower computa- immediately and significantly, especially those of latency- tions and storage resources with respect to the cloud scenario. sensitive applications. During a failure event, mobile users The hypervisor-based virtualization becomes unsuitable given can be disconnected from the MEC infrastructure and the that the image files of VMs are large and its overhead is non- related services are unavailable. This scenario can have a negligible. Furthermore, when MEC is integrated with 5G, disruptive impact on the perceived quality of the offered the user mobility adds new requirements to the virtualization services particularly in areas with a high density of users. techniques. In particular, the service continuity and the fast Given the negative effects of the MEH failures, the design of migration of service between MEHs extend the popularity the MEC infrastructure must consider the reliability aspects of container-based technology, which allows to easily build, of the system. The robustness of the MEC infrastructure run, manage, migrate and remove containerized applications. requires coverage of a specific area with a number of MEHs The different container solutions (e.g. LXC, LXD, Singularity, greater than strictly necessary to guarantee performance in Docker, Kata Containers, etc.) offer diverse performance in ideal situations. Making a MEC design considering only the terms of CPU and memory load, networking bandwidth, and performance, without taking into account the robustness of disk I/O, and configuration options that can further improve the the infrastructure, leads to localize the MEHs, minimizing the performance in some specific scenarios. However, the isolation overlap of the coverage of each of them. Simply maximizing mechanism of existing container-based technologies is weak, the collective coverage of the MEC infrastructure could lead due to the sharing of one kernel among multiple isolated envi- to situations where no MEH can take control of mobile users ronments. This feature adds four types of problems to consider disconnected from a failed MEHs. Such a MEC infrastructure for container security: (i) protecting a container from applica- is highly vulnerable to runtime errors. tions inside it, (ii) inter-container protection, (iii) protecting As already presented in Section IV-C, to alleviate this prob- the host from containers, and (iv) protecting containers from lem, recent studies [135], [136] investigate the dependability a malicious or semi-honest host. A big challenge is to explore in the deployment of the MEHs because of its impact on the new methods (e.g., using trusted images, managing container effectiveness of the failover mechanisms. Of course, consider- secret, securing the runtime environment, and vulnerability ing only dependability target is also not correct. The depend- scanning), which can secure the container-based virtualiza- ability should be jointly considered with the performance. For tion without degrading the performance in terms of agility, example, if both dependability and latency prefer a denser resource consumption, and computation delay. Alternatively, deployment of MEHs, energy consumption and economical the unikernel technology guarantees security by the isolation costs push for a less dense deployment. For this reason, the provided by the underlying operating system or hypervisor. deployment strategies will aim to find the best trade off in the Unikernel is more secure than the container. However, from given scenario. the performance perspective, unikernel presents low usability Moreover, the failover mechanisms themselves should con- given that it does not have a shell and does not support online sider performance but also security aspects. For example, the debugging, online upgrades and updates either. In the case failover mechanisms should be fast (for example, it can be of failure of a unikernel application, only the reboot solves proactive [137]) in order to reduce the delay. Moreover, using the problem. Application and/or configuration updates require other users as rely on to reach distant MEHs [138] might to recompile the source code to produce a new unikernel and introduce severe security threats. deploy a new version. This action can be very costly and some- b) Resource allocation: As we have already mentioned times prohibitive. This weak point of unikernel technology in previous sections, another challenge addressed by current represents a big challenge that needs to be considered in order studies is the allocation of resources, usually computing and to evaluate the most suitable virtualization technologies for storage, in the different MEHs. As mentioned in Section IV-C, MEC, able to achieve a good tradeoff between performance, the resource allocation in MEC is often called task allocation, dependability, and security. This evaluation can converge to since an application or procedure (task) is moved (offloaded) a single new virtualization technology or a set of them, each from the local execution in the mobile device to a remote one achieving the best tradeoff in some specific use cases. execution on a MEH. 2) MEC system level: Most of the current works have as target the energy ef- a) Deployment and design: The problem of the deploy- ficiency. As mentioned in Section IV-C, several works also ment of MEHs in a particular area is commonly addressed consider the dependability metrics as requirements [139], with the aim of improving performance. Many existing studies [140]. Some works consider both reliability and latency as focus on different goals, for example, to minimize overall constraints [141], [142] or as targets [146]. Other works latency of mobile users, to maximize overall throughput of implicitly consider reliability and latency by focusing on the the MEH network, etc. [172], [174], [177]. Many of these queue length [143]–[145]. studies assume that the MEHs are free of failures. However, To the best of the authors’ knowledge, no work considers in a dynamic and volatile network scenario, MEHs are subject security aspects in depth. The importance must be better 27 investigated. For example, the task can have different security of risks that a potential attacker may exploit (i.e., an active requirements, and the MEHs have different security guaran- attacker might aim to break the consistency of the system). tees. A general good practice is the adding of a threshold Moreover, the information exchange can use resources and for resource consumption and guarantee of some resource impact the performance of the system. availability for security functionalities [25]. Otherwise, for 3) General challenges: example, by compromising the user apps an adversary can a) Modelling: A joint evaluation and analysis of security, use too many resources and thus affect performance. More performance, and dependability of 5G-MEC is needed. In specific approaches consider for example the usage of a low the literature, there are several works that focus on joint resource Intrusion Detection System (IDS) [68] or lightweight modeling: performance and dependability [203]; security and cryptography, as already discussed in Section III-C. dependability [128], [204]. There are also tools that aim to c) User association: As mentioned in Section IV-C, jointly evaluate these aspects, such as Möbius [205]. To the some works address the task allocation together with the user- best of the authors’ knowledge, there is no current work that host association [145]. For some critical applications requiring jointly evaluates the three aspects in MEC. low latency and high reliability, such as in some V2X and b) Network connectivity: The network is a key part of a Industrial IoT scenarios, the design of user association algo- MEC system. It includes the network access to reach the user, rithms jointly considering the three aspects is of paramount and the edge network to interconnect the MEHs and other importance. Multi-constrained optimal path computation algo- MEC elements. The network has an impact on the performance rithms have been proposed for solving routing problems [202]. (lack of the requirements), dependability (lack or degraded Some of these can be extended to solve the user association connectivity), and security (the MEC system is exposed via problem. Other than performance-related metrics, new metrics the network). summarizing the security and the reliability level of a link and Recently, new studies are focused on the definition of a node need to be studied and jointly considered in the path new wireless communications systems, such as mmWave and computation towards alternative MEHs. Through these tools, TeraHertz. This trend is spurred by the need of having a very path computation solutions towards different MEHs can be high data rate, necessary for reducing the delay related to task then compared to select the MEH satisfying the user require- offloading of applications. However, these new communication ments on the three different aspects. To reduce the complexity systems are vulnerable to blocking events due to obstacles or added by the path computation problem, similar metrics can beam collisions, which can interrupt the data exchange. As be defined only for the MEH to develop a user association a concern, this trend, one set of challenges is related to the algorithm aimed at finding the best trade-off among the three study of new beamforming techniques based on antenna arrays aspects. both at the transmitter and at the receiver side, and to the Concerning security and performance, whenever possible, usage of multi-link communications. Another set of challenges perform user security services locally, with no need for cen- is related to the simultaneous study of the performance and tralization (e.g., identification and authentication [62]). the reliability of the network connectivity with the goal of d) MEO: The MEO is an important element of the improving some performance parameters, such as latency, MEC architecture. It is a logically centralized element that energy consumption, and deployment costs while achieving is the main responsible for the system-level management of the reliability requirements. As an example, the reliability can MEC. The ETSI MEC architecture [8] specifies that the MEC be improved allowing the UE to send information to different applications have a certain number of resource requirements, MEHs, via different mmWave beams and over all the available and these requirements are validated by the MEO. UE-MEHs links simultaneously. More effort is necessary to As introduced in Section IV-C, the MEO is crucial for define strategies, such as the Parallel Redundancy Protocol the availability and reliability of the MEC system because it (PRP), in order to find an acceptable trade-off between net- orchestrates the creation of a MEC and the fault management work resource consumption and achieved reliability. of the other MEC elements. The solutions aimed to have For mMTC and some URLLC use cases, the Non- a dependable MEO, such as physical distributed MEO, can Orthogonal Multiple Access (NOMA) represents a key tech- generate challenges from a security perspective given the nology of 5G for improving the system capacity and the increment of the exposure risk. As discussed in Section III-C, connection density [206]. The key idea of NOMA is to share special attention must be given to the virtualisation security. a given resource slot (e.g., time/frequency) among multiple e) Consistency: In Section IV, consistency has been pre- users, and applying Successive Interference Cancellation (SIC) sented as a property needed when we use redundant elements to decode the transmitted information. NOMA allows the for improving the dependability of a system. This is valid grant-free transmission. This feature is particularly important for the MEO but also for the MEHs (during the handover). for the uplink transmission of small data blocks, such as in Consistency is also important in the case of distributed sys- the IoT scenarios. In this case, the control signaling overhead tems, which aim, not only to increase dependability, but also to is reduced, as well as the transmission latency and the power increase scalability. For achieving consistency, the redundant consumption of the terminal device. However, NOMA adds and/or distributed elements need to exchange information some security issues that need to be carefully studied. In the about their states in order to have a consistent global view of case the NOMA connection is used by two users for offloading the system. This information exchange can have an impact on tasks to a MEH at the same time when SIC is performed, the security and performance of the system and can be a source one user can decode the other user’s message. During this 28 period, an eavesdropper or an attacker may attempt to decode First, the ETSI MEC architecture has been introduced the mobile user’s message. To cope with this problem, key including the NFV-based version and the integration with 5G. challenges are to combine physical-layer security and NOMA- Second, for each aspect, the taxonomy, the state of the art, MEC in order to find solutions that can achieve a good and the challenges have been presented. The taxonomy has trade-off between security and performance. Given the NOMA given a general introduction to the aspects to the readers that features, the performance is related to the latency, system are only experts in the other aspects. The state of the art has capacity, and energy consumption at the user side. introduced the standards and the current surveys that address Concerning both security and performance, Soft-VPLS (dis- the investigated aspect on 5G MEC. The requirements for each cussed in Section III-C) allows different traffic categories element of the ETSI MEC architecture have been highlighted. (e.g., MEC service requests, user data, control statistics) to The challenges of the investigated aspect on 5G MEC have be routed via distinct tunnels, aiming to enhance both end-to- been presented by categorizing them in MEC host level, MEC end security and overall communication performance [68]. system level, and general challenges. This study has shown c) Service continuity and user mobility: The service that, although the main target of 5G MEC is to improve continuity has been defined in Section IV and is critical even the performance of network services, many works have ad- because the user(s) might be mobile. A lack of service continu- dressed the security and strict security requirements have been ity has a huge impact of both performance and dependability. specified. Fewer works have addressed the dependability on However, it is often not easy to implement securely while 5G MEC, even though URRLC services and mission-critical satisfying the performance needs. applications have strict requirements on dependability. Solutions to improve the service continuity can lead to Finally, the challenges of jointly addressing security, de- a conflict between performance and dependability. In MEC- pendability, and performance have been investigated by using host-assisted proactive state relocation for UE in connected the same categorization. The investigation has shown the im- mode [42], the improved latency is accomplished at the portance of jointly address the three aspects and how focusing price of relocation success, because failed handover operation on only one aspect can cause the failure of meeting the may nullify the state transition preparation made by the requirements on the other aspects. In the same time, our study MEHs. Therefore, a trade-off between latency and reliability highlights different aspects that need to be further investigated, is needed. being a starting point for further research. Similarly, solutions to achieve secure user mobility can lead to a conflict between security and performance. Special security protocols need to be set in place to allow secure mobility REFERENCES of users (e.g., from moving from one MEH to another) so that [1] ITU-R, “IMT Vision – Framework and overall objectives of the future this process does not expose sensible data (e.g., identifiers, development of IMT for 2020 and beyond (Recommendation ITU-R M.2083-0),” Sept 2015. keys, sensible data). Forward and backward security play [2] “2020 Research Report: 5G Enterpise Commercial Use Cases an important role in these scenarios (e.g., compromising a Update and Telco Go-To-Market Strategies with XL Axiata, key shared between the user and a MEH should not expose Indonesia, mMTC, AT&T, US, URLLC Use Case Snapshots,” https://www.globenewswire.com/news-release/2021/04/12/2208150/ previous or further keys shared by the user with past or future 28124/en/2020-Research-Report-5G-Enterpise-Commercial-Use- MEHs). Protocols such as key update or setting up a new Cases-Update-and-Telco-Go-To-Market-Strategies-with-XL-Axiata- security context (if needed) introduce some latency that, if Indonesia-mMTC-AT-T-US-URLLC-Use-Case-Snapshots.html, Accessed: 24-07-2021. not properly adopted, might affect performance. [3] 6G Flagship, “6G White Paper on Edge Intelligence – 6G Research Vi- d) Monitoring and detection: As presented in Sec- sions, No. 8,” https://www.6gchannel.com/items/6g-white-paper-edge- tion III-C, monitoring and detection are key elements for the intelligence/, June 2020. security in 5G MEC, but they are also important for fault and [4] “ETSI Industry Specification Group (ISG) on Multi-Access Edge Com- puting (MEC),” https://www.etsi.org/committee/mec, accessed: 13-04- performance management. The monitoring and detection for 2021. security, dependability, and performance should be addressed [5] M. Satyanarayanan, P. Bahl, R. Caceres, and N. Davies, “The case for jointly in order to exploit the cross information and provide ad- vm-based cloudlets in mobile computing,” IEEE Pervasive Computing, vol. 8, no. 4, pp. 14–23, 2009. vanced features. As already mentioned for security, AI-based [6] “OpenFog Consortium,” http://www.openfogconsortium.org/, techniques can be developed to provide advanced monitoring accessed: 13-04-2021. and detection functionalities [48]. Moreover, Virtual Machine [7] ETSI, “GS MEC 001 V2.1.1: Multi-access Edge Computing (MEC); Terminology,” Jan 2019. Introspection (VMI), and hypervisors machine introspection [8] ——, “GS MEC 003 V2.2.1: Multi-access Edge Computing (MEC); should monitor the activities in terms of resource utilization to Framework and Reference Architecture,” Dec 2020. prevent performance issues and DoS attacks [68]. As already [9] ——, “Developing Software for Multi-Access Edge Computing - White Paper No. 20,” 2019. discussed in Section III-C a good tradeoff between local and [10] ——, “GS MEC 011 V2.2.1: Multi-access Edge Computing (MEC); global monitoring techniques would be beneficial for security Edge Platform Application Enablement,” Dec 2020. in relation to performance. [11] OpenStack, “Cloud Edge Computing: Beyond the Data Center,” 2018. [12] ETSI, “GS MEC 002 V2.1.1: Multi-access Edge Computing (MEC); Phase 2: Use Cases and Requirements,” Nov 2018. VII.CONCLUSIONS [13] ——, “GS NFV 002 V1.2.1: Network Functions Virtualisation (NFV); In this survey, the state of the art and the challenges of Architectural Framework,” Dec 2014. [14] J. M. Halpern and C. Pignataro, “Service function chaining (SFC) the 5G MEC have been studied with respect to three aspects: architecture,” Internet Engineering Task Force, Fremont, CA, USA, RFC security, performance, and dependability. 7665, Oct 2015. 29

[15] ETSI, “GR MEC 017 V1.1.1: Mobile Edge Computing (MEC); De- [47] ——, “Network Transformation; (Orchestration, Network and Service ployment of Mobile Edge Computing in an NFV environment,” Feb Management Framework) - White Paper No. 32,” Oct 2019. 2018. [48] ——, “Artificial Intelligence and future directions for ETSI - White [16] ——, “GR MEC 027 V2.1.1: Multi-access Edge Computing (MEC); Paper No. 34,” Jun 2020. Study on MEC support for alternative virtualization technologies,” Nov [49] ——, “Harmonizing standards for edge computing - A synergized 2019. architecture leveraging ETSI ISG MEC and 3GPP specifications - [17] G. Nencioni, R. G. Garroppo, A. J. Gonzalez, B. E. Helvik, and White Paper No. 36,” Jul 2020. G. Procissi, “Orchestration and Control in Software-Defined 5G Net- [50] ——, “Enhanced DNS Support towards Distributed MEC Environment works: Research Challenges,” Wireless Communications and Mobile - White Paper No. 39,” Sep 2020. Computing, 2018. [51] ——, “MEC security: Status of standards support and future evolutions [18] ETSI, “MEC in 5G networks - White Paper No. 28,” Jun 2018. - White Paper No. 46,” May 2021. [19] ——, “GR MEC 031 V2.1.1: Multi-access Edge Computing (MEC); [52] Y. Yu, “Mobile edge computing towards 5G: Vision, recent progress, MEC 5G Integration,” Oct 2020. and open challenges,” China Communications, vol. 13, no. 2, pp. 89– [20] 3GPP, “TS 23.501 V17.0.0: 3rd Generation Partnership Project; Tech- 99, 2016. [Online]. Available: https://ieeexplore.ieee.org/document/ nical Specification Group Services and System Aspects; System archi- 7405725/ tecture for the 5G System (5GS); Stage 2 (Release 17),” Mar 2021. [53] B. Hibat Allah and I. Abdellah, “MEC towards 5G: A Survey [21] ETSI, “MEC Deployments in 4G and Evolution Towards 5G - White of Concepts, Use Cases, Location Tradeoffs,” Transactions on Paper No. 24,” Feb 2018. Machine Learning and Artificial Intelligence, vol. 5, no. 4, [22] ——, “GS MEC 028 V2.1.1: Multi-access Edge Computing (MEC); Aug. 2017. [Online]. Available: http://www.scholarpublishing.org/ WLAN Information API ,” Jun 2020. index.php/TMLAI/article/view/3215 [23] ——, “Cloud RAN and MEC: A Perfect Pairing - White Paper No. [54] T. Taleb, K. Samdanis, B. Mada, H. Flinck, S. Dutta, and D. Sabella, 23,” Feb 2018. “On Multi-Access Edge Computing: A Survey of the Emerging [24] A. J. Gonzalez, J. Ordonez-Lucena, B. E. Helvik, G. Nencioni, M. Xie, 5G Network Edge Cloud Architecture and Orchestration,” IEEE D. R. Lopez, and P. Grønsund, “The Isolation Concept in the 5G Communications Surveys & Tutorials, vol. 19, no. 3, pp. 1657– Network Slicing,” in European Conference on Networks and Commu- 1681, 2017. [Online]. Available: http://ieeexplore.ieee.org/document/ nications (EuCNC), Dubrovnik, Croatia, June 2020. 7931566/ [25] R. F. Olimid and G. Nencioni, “5g network slicing: A security [55] S. Wang, X. Zhang, Y. Zhang, L. Wang, J. Yang, and W. Wang, overview,” IEEE Access, vol. 8, pp. 99 999–100 009, 2020. “A Survey on Mobile Edge Networks: Convergence of Computing, [26] ETSI, “GR MEC 024 V2.1.1: Multi-access Edge Computing (MEC); Caching and Communications,” IEEE Access, vol. 5, pp. 6757– Support for network slicing,” Nov 2019. 6779, 2017. [Online]. Available: http://ieeexplore.ieee.org/document/ [27] ——, “GS MEC-IEG 006 V1.1.1: Mobile Edge Computing;Market 7883826/ Acceleration; MEC Metrics Best Practice and Guidelines,” Jan 2017. [56] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A Survey [28] ——, “GS MEC 009 V3.1.1: Multi-access Edge Computing (MEC); on Mobile Edge Computing: The Communication Perspective,” IEEE General principles, patterns and common aspects of MEC Service Communications Surveys & Tutorials, vol. 19, no. 4, pp. 2322– APIs,” Jun 2021. 2358, 2017. [Online]. Available: http://ieeexplore.ieee.org/document/ [29] ——, “GS MEC 010-1 V1.1.1: Mobile Edge Computing (MEC); Mo- 8016573/ bile Edge Management; Part 1: System, host and platform management [57] P. Mach and Z. Becvar, “Mobile Edge Computing: A Survey on management,” Oct 2017. Architecture and Computation Offloading,” IEEE Communications [30] ——, “GS MEC 010-2 V2.1.1: Multi-access Edge Computing Surveys & Tutorials, vol. 19, no. 3, pp. 1628–1656, 2017. [Online]. (MEC);MEC Management; Part 2: Application lifecycle, rules and Available: http://ieeexplore.ieee.org/document/7879258/ requirements management,” Nov 2019. [58] S. Shahzadi, M. Iqbal, T. Dagiuklas, and Z. U. Qayyum, “Multi-access [31] ——, “GS MEC 012 V2.1.1: Multi-access Edge Computing (MEC); edge computing: open issues, challenges and future perspectives,” Radio Network Information API,” Dec 2019. Journal of Cloud Computing, vol. 6, no. 1, p. 30, 2017. [32] ——, “GS MEC 013 V2.1.1: Multi-access Edge Computing (MEC); [59] S. N. Shirazi, A. Gouglidis, A. Farshad, and D. Hutchison, “The Location API,” Sep 2019. extended cloud: Review and analysis of mobile edge computing and fog [33] ——, “GS MEC 014 V2.1.1: Mobile Edge Computing (MEC);UE from a security and resilience perspective,” IEEE Journal on Selected Identity API,” Mar 2021. Areas in Communications, vol. 35, no. 11, pp. 2586–2595, 2017. [34] ——, “GS MEC 015 V2.1.1: Multi-Access Edge Computing (MEC); [60] K. Bilal, O. Khalid, A. Erbad, and S. U. Khan, “Potentials, trends, and Traffic Management APIs,” Jun 2020. prospects in edge technologies: Fog, cloudlet, mobile edge, and micro [35] ——, “GS MEC 016 V2.2.1: Multi-access Edge Computing (MEC); data centers,” Computer Networks, vol. 130, pp. 94–120, 2018. Device application interface,” Apr 2020. [61] K. Peng, V. C. M. Leung, X. Xu, L. Zheng, J. Wang, and Q. Huang, [36] ——, “GS MEC 021 V2.1.1: Multi-access Edge Computing (MEC); “A Survey on Mobile Edge Computing: Focusing on Service Application Mobility Service API,” Jan 2020. Adoption and Provision,” Wireless Communications and Mobile [37] ——, “GS MEC 026 V2.1.1: Multi-access Edge Computing (MEC); Computing, vol. 2018, pp. 1–16, Oct. 2018. [Online]. Available: Support for regulatory requirements,” Ian 2019. https://www.hindawi.com/journals/wcmc/2018/8267838/ [38] ——, “GS MEC 029 V2.1.1: Multi-access Edge Computing (MEC); [62] R. Roman, J. Lopez, and M. Mambo, “Mobile edge computing, Fog et Fixed Access Information API ,” Jul 2019. al.: A survey and analysis of security threats and challenges,” Future [39] ——, “GS MEC 030 V2.1.1: Multi-access Edge Computing (MEC); Generation Computer Systems, p. 19, 2018. V2X Information Service API,” Apr 2020. [63] P. Porambage, J. Okwuibe, M. Liyanage, M. Ylianttila, and [40] ——, “GS MEC-DEC 032-1 V2.1.1: Multi-access Edge Computing T. Taleb, “Survey on Multi-Access Edge Computing for Internet (MEC); API Conformance Test Specification; Part 1: Test Require- of Things Realization,” IEEE Communications Surveys & Tutorials, ments and Implementation Conformance Statement (ICS),” Dec 2020. vol. 20, no. 4, pp. 2961–2991, 2018. [Online]. Available: https: [41] ——, “GS MEC-DEC 032-2 V2.1.1: Multi-access Edge Computing //ieeexplore.ieee.org/document/8391395/ (MEC); API Conformance Test Specification; Part 2: Test Purposes [64] H. Tanaka, M. Yoshida, K. Mori, and N. Takahashi, “Multi- (TP),” Dec 2020. access Edge Computing: A Survey,” Journal of Information [42] ——, “GR MEC 018 V1.1.1: Mobile Edge Computing (MEC); End to Processing, vol. 26, no. 0, pp. 87–97, 2018. [Online]. Available: End Mobility Aspects,” Oct 2017. https://www.jstage.jst.go.jp/article/ipsjjip/26/0/26_87/_article [43] ——, “GR MEC 022 V2.1.1: Multi-access Edge Computing (MEC); [65] N. Abbas, Y. Zhang, A. Taherkordi, and T. Skeie, “Mobile Study on MEC Support for V2X Use Cases,” Sep 2018. Edge Computing: A Survey,” IEEE Internet of Things Journal, [44] ——, “GR MEC-DEC 025 V2.1.1: Multi-access Edge Computing vol. 5, no. 1, pp. 450–465, Feb. 2018. [Online]. Available: (MEC); MEC Testing Framework,” Jun 2019. http://ieeexplore.ieee.org/document/8030322/ [45] ——, “GR MEC 035 V3.1.1: Multi-access Edge Computing (MEC); [66] W. Z. Khan, E. Ahmed, S. Hakak, I. Yaqoob, and A. Ahmed, “Edge Study on Inter-MEC systems and MEC-Cloud systems coordination,” computing: A survey,” Future Generation Computer Systems, vol. 97, Jun 2021. pp. 219–235, 2019. [46] ——, “MEC in an Enterprise Setting: A Solution Outline - White Paper [67] A. Yousefpour, C. Fung, T. Nguyen, K. Kadiyala, F. Jalali, A. Ni- No. 30,” Sept 2018. akanlahiji, J. Kong, and J. P. Jue, “All one needs to know about fog 30

computing and related edge computing paradigms: A complete survey,” [87] M. Yahuza, M. Y. I. B. Idris, A. W. B. A. Wahab, A. T. Ho, S. Khan, Journal of Systems Architecture, vol. 98, pp. 289–330, 2019. S. N. B. Musa, and A. Z. B. Taha, “Systematic review on security and [68] P. Ranaweera, A. D. Jurcut, and M. Liyanage, “Realizing multi-access privacy requirements in edge computing: State of the art and future edge computing feasibility: Security perspective,” in 2019 IEEE Con- research opportunities,” IEEE Access, vol. 8, pp. 76 541–76 567, 2020. ference on Standards for Communications and Networking (CSCN). [88] L. Liu, C. Chen, Q. Pei, S. Maharjan, and Y. Zhang, “Vehicular IEEE, 2019, pp. 1–7. Edge Computing and Networking: A Survey,” Mobile Networks and [69] Z. Li, X. Zhou, and Y. Qin, “A Survey of Mobile Edge Applications, Jul. 2020. [Online]. Available: https://doi.org/10.1007/ Computing in the Industrial Internet,” in 2019 7th International s11036-020-01624-1 Conference on Information, Communication and Networks (ICICN). [89] R. Khan, P. Kumar, D. N. K. Jayakody, and M. Liyanage, “A survey Macao, Macao: IEEE, Apr. 2019, pp. 94–98. [Online]. Available: on security and privacy of 5g technologies: Potential solutions, recent https://ieeexplore.ieee.org/document/8834959/ advancements, and future directions,” IEEE Communications Surveys [70] Q. Pham, F. Fang, V. N. Ha, M. J. Piran, M. Le, L. B. Le, W. Hwang, & Tutorials, vol. 22, no. 1, pp. 196–248, 2019. and Z. Ding, “A survey of multi-access edge computing in 5g and [90] National Institute of Standards and Technologies (NIST), Computer beyond: Fundamentals, technology integration, and state-of-the-art,” Security Reserch Center (CSRC), “Glossary,” 2021. IEEE Access, vol. 8, pp. 116 974–117 017, 2020. [91] H. C. Van Tilborg and S. Jajodia, Encyclopedia of cryptography and [71] P. Habibi, M. Farhoudi, S. Kazemian, S. Khorsandi, and A. Leon- security. Springer Science & Business Media, 2014. Garcia, “Fog Computing: A Comprehensive Architectural Survey,” [92] A. J. Menezes, P. C. Van Oorschot, and S. A. Vanstone, Handbook of IEEE Access, vol. 8, pp. 69 105–69 133, 2020. [Online]. Available: applied cryptography. CRC press, 2018. https://ieeexplore.ieee.org/document/9046806/ [93] M. Liyanage, A. Gurtov, and M. Ylianttila, Software defined mobile [72] A. Filali, A. Abouaomar, S. Cherkaoui, A. Kobbane, and M. Guizani, networks (SDMN): beyond LTE network architecture. John Wiley & “Multi-access edge computing: A survey,” IEEE Access, vol. 8, pp. Sons, 2015. 197 017–197 046, 2020. [94] M. Liyanage, A. Braeken, A. D. Jurcut, M. Ylianttila, and A. Gur- [73] F. Vhora and J. Gandhi, “A Comprehensive Survey on Mobile tov, “Secure communication channel architecture for software defined Edge Computing: Challenges, Tools, Applications,” in 2020 mobile networks,” Computer Networks, vol. 114, pp. 32–50, 2017. Fourth International Conference on Computing Methodologies and [95] J. McCumber, “Information systems security: A comprehensive model,” Communication (ICCMC). Erode, India: IEEE, Mar. 2020, pp. 49–55. in Proceedings 14th National Computer Security Conference, 1991, pp. [Online]. Available: https://ieeexplore.ieee.org/document/9076528/ 328–337. [74] P. Ranaweera, A. D. Jurcut, and M. Liyanage, “Survey on multi-access [96] ZTE, “5G Security White Paper - Security makes 5G Go further,” May edge computing security and privacy,” IEEE Communications Surveys 2019. & Tutorials, pp. 1–1, 2021. [97] NGMN, “5G Security – Package 3: Mobile Edge Computing / Low [75] S. D. A. Shah, M. A. Gregory, and S. Li, “Cloud-native network slicing Latency / Consistent User Experience,” Feb 2018. using software defined networking based multi-access edge computing: [98] X. Tan, H. Li, L. Wang, and Z. Xu, “Global orchestration of coop- A survey,” IEEE Access, vol. 9, pp. 10 903–10 924, 2021. erative defense against ddos attacks for mec,” in 2019 IEEE Wireless [76] H. Madsen, B. Burtschy, G. Albeanu, and F. Popentiu-Vladicescu, “Re- Communications and Networking Conference (WCNC). IEEE, 2019, liability in the utility computing era: Towards reliable fog computing,” pp. 1–6. in 2013 20th International Conference on Systems, Signals and Image [99] H. Li and L. Wang, “Online orchestration of cooperative defense Processing (IWSSIP), 2013, pp. 43–46. against ddos attacks for 5g mec,” in 2018 IEEE Wireless Communica- [77] P. Hu, S. Dhelim, H. Ning, and T. Qiu, “Survey on fog computing: tions and Networking Conference (WCNC). IEEE, 2018, pp. 1–6. architecture, key technologies, applications and open issues,” Journal [100] A. S. Mamolar, Z. Pervez, J. M. A. Calero, and A. M. Khattak, of Network and Computer Applications, vol. 98, pp. 27–42, Nov. “Towards the transversal detection of ddos network attacks in 5g multi- 2017. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/ tenant overlay networks,” Computers & Security, vol. 79, pp. 132–147, S1084804517302953 2018. [78] R. Mahmud, R. Kotagiri, and R. Buyya, “Fog Computing: [101] P. Krishnan, S. Duttagupta, and K. Achuthan, “SDNFV based threat A Taxonomy, Survey and Future Directions,” arXiv:1611.05539 monitoring and security framework for multi-access edge computing [cs], pp. 103–130, 2018, arXiv: 1611.05539. [Online]. Available: infrastructure,” Mobile Networks and Applications, vol. 24, no. 6, pp. http://arxiv.org/abs/1611.05539 1896–1923, 2019. [79] Z. Bakhshi and G. Rodriguez-Navas, “A preliminary roadmap [102] L. Xiao, X. Wan, C. Dai, X. Du, X. Chen, and M. Guizani, “Security for dependability research in fog computing,” SIGBED Rev., in mobile edge caching with reinforcement learning,” IEEE Wireless vol. 16, no. 4, p. 14–19, Jan. 2020. [Online]. Available: https: Communications, vol. 25, no. 3, pp. 116–122, 2018. //doi.org/10.1145/3378408.3378410 [103] A. Abeshu and N. Chilamkurti, “Deep learning: The frontier for [80] P. Garcia Lopez, A. Montresor, D. Epema, A. Datta, T. Higashino, distributed attack detection in fog-to-things computing,” IEEE Com- A. Iamnitchi, M. Barcellos, P. Felber, and E. Riviere, “Edge-centric munications Magazine, vol. 56, no. 2, pp. 169–175, 2018. computing: Vision and challenges,” 2015. [104] I. Ahmad, T. Kumar, M. Liyanage, J. Okwuibe, M. Ylianttila, and [81] X. Huang, R. Yu, J. Kang, Y. He, and Y. Zhang, “Exploring A. Gurtov, “Overview of 5g security challenges and solutions,” IEEE Mobile Edge Computing for 5G-Enabled Software Defined Vehicular Communications Standards Magazine, vol. 2, no. 1, pp. 36–43, 2018. Networks,” IEEE Wireless Communications, vol. 24, no. 6, pp. 55–63, [105] J. Zhang, B. Chen, Y. Zhao, X. Cheng, and F. Hu, “Data security Dec. 2017. [Online]. Available: http://ieeexplore.ieee.org/document/ and privacy-preserving in edge computing paradigm: Survey and open 8246847/ issues,” IEEE Access, vol. 6, pp. 18 209–18 237, 2018. [82] W. Yu, F. Liang, X. He, W. G. Hatcher, C. Lu, J. Lin, and X. Yang, [106] R. Rapuzzi and M. Repetto, “Building situational awareness for “A Survey on the Edge Computing for the Internet of Things,” network threats in fog/edge computing: Emerging paradigms beyond IEEE Access, vol. 6, pp. 6900–6919, 2018. [Online]. Available: the security perimeter model,” Future Generation Computer Systems, http://ieeexplore.ieee.org/document/8123913/ vol. 85, pp. 235–249, 2018. [83] M. Du, K. Wang, Y. Chen, X. Wang, and Y. Sun, “Big data privacy [107] D. Thembelihle, M. Rossi, and D. Munaretto, “Softwarization of preserving in multi-access edge computing for heterogeneous internet mobile network functions towards agile and energy efficient 5g archi- of things,” IEEE Communications Magazine, vol. 56, no. 8, pp. 62–67, tectures: A survey,” Wireless Communications and Mobile Computing, 2018. vol. 2017, 2017. [84] J. Ni, X. Lin, and X. S. Shen, “Toward edge-assisted internet of things: [108] D. He, S. Chan, and M. Guizani, “Security in the internet of things sup- From security and efficiency perspectives,” IEEE Network, vol. 33, ported by mobile edge computing,” IEEE Communications Magazine, no. 2, pp. 50–57, 2019. vol. 56, no. 8, pp. 56–61, 2018. [85] S. Bagchi, M.-B. Siddiqui, P. Wood, and H. Zhang, “Dependability [109] J. Okwuibe, M. Liyanage, I. Ahmad, and M. Ylianttila, “Cloud and in edge computing,” Communications of the ACM, vol. 63, no. 1, pp. mec security,” in A Comprehensive Guide to 5G Security. Wiley, 58–66, 2019. 2018, p. 373. [86] M. S. Elbamby, C. Perfecto, C.-F. Liu, J. Park, S. Samarakoon, X. Chen, [110] ENISA, “Research topics - ENISA Threat Landscape,” Oct 2020. and M. Bennis, “Wireless edge computing with latency and reliability [111] I. Farris, J. B. Bernabé, N. Toumi, D. Garcia-Carrillo, T. Taleb, guarantees,” Proceedings of the IEEE, vol. 107, no. 8, pp. 1717–1737, A. Skarmeta, and B. Sahlin, “Towards provisioning of sdn/nfv-based 2019. security enablers for integrated protection of iot systems,” in 2017 31

IEEE Conference on Standards for Communications and Networking IEEE Journal on Selected Areas in Communications, vol. 38, no. 7, pp. (CSCN). IEEE, 2017, pp. 169–174. 1505–1514, 2020. [112] A. M. Zarca, J. B. Bernabe, R. Trapero, D. Rivera, J. Villalobos, [135] B. Li, Q. He, G. Cui, X. Xia, F. Chen, H. Jin, and Y. Yang, “Read: A. Skarmeta, S. Bianchi, A. Zafeiropoulos, and P. Gouvas, “Security Robustness-oriented edge application deployment in edge computing management architecture for nfv/sdn-aware iot systems,” IEEE Internet environment,” IEEE Transactions on Services Computing, pp. 1–1, of Things Journal, vol. 6, no. 5, pp. 8005–8020, 2019. 2020. [113] I. Petri, O. F. Rana, Y. Rezgui, and G. C. Silaghi, “Trust modelling [136] F. Tonini, B. M. Khorsandi, E. Amato, and C. Raffaelli, “Scalable edge and analysis in peer-to-peer clouds,” International Journal of Cloud computing deployment for reliable service provisioning in vehicular Computing, vol. 1, no. 2-3, pp. 221–239, 2012. networks,” Journal of Sensor and Actuator Networks, vol. 8, no. 4, [114] Y. He, F. R. Yu, N. Zhao, and H. Yin, “Secure social networks in 5g 2019. [Online]. Available: https://www.mdpi.com/2224-2708/8/4/51 systems with mobile edge computing, caching, and device-to-device [137] H. Huang and S. Guo, “Proactive failure recovery for nfv in distributed communications,” IEEE Wireless Communications, vol. 25, no. 3, pp. edge computing,” IEEE Communications Magazine, vol. 57, no. 5, pp. 103–109, 2018. 131–137, 2019. [115] T. Bai, J. Wang, Y. Ren, and L. Hanzo, “Energy-efficient computation [138] D. Satria, D. Park, and M. Jo, “Recovery for overloaded mobile offloading for secure UAV-edge-computing systems,” IEEE Transac- edge computing,” Future Generation Computer Systems, vol. 70, pp. tions on Vehicular Technology, vol. 68, no. 6, pp. 6074–6087, 2019. 138–147, 2017. [Online]. Available: https://www.sciencedirect.com/ [116] J. Xu and J. Yao, “Exploiting physical-layer security for multiuser science/article/pii/S0167739X16302096 multicarrier computation offloading,” IEEE Wireless Communications [139] C.-A. Chen, M. Won, R. Stoleru, and G. G. Xie, “Energy-efficient Letters, vol. 8, no. 1, pp. 9–12, 2018. fault-tolerant data storage and processing in mobile cloud,” IEEE [117] A. Mtibaa, K. Harras, and H. Alnuweiri, “Friend or foe? detecting and Transactions on Cloud Computing, vol. 3, no. 1, pp. 28–41, 2015. isolating malicious nodes in mobile edge computing platforms,” in 2015 [140] C.-A. Chen, R. Stoleru, and G. G. Xie, “Energy-efficient and fault- IEEE 7th International Conference on Cloud Computing Technology tolerant mobile cloud storage,” in 2016 5th IEEE International Con- and Science (CloudCom). IEEE, 2015, pp. 42–49. ference on Cloud Networking (Cloudnet), 2016, pp. 51–57. [118] W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. Jiao, Y.-C. Liang, Q. Yang, [141] C.-F. Liu, M. Bennis, and H. V. Poor, “Latency and reliability-aware D. Niyato, and C. Miao, “Federated learning in mobile edge networks: task offloading and resource allocation for mobile edge computing,” in A comprehensive survey,” IEEE Communications Surveys & Tutorials, 2017 IEEE Globecom Workshops (GC Wkshps), 2017, pp. 1–7. vol. 22, no. 3, pp. 2031–2063, 2020. [142] H. Liu, L. Cao, T. Pei, Q. Deng, and J. Zhu, “A fast algorithm for [119] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: energy-saving offloading with reliability and latency requirements in Challenges, methods, and future directions,” IEEE Signal Processing multi-access edge computing,” IEEE Access, vol. 8, pp. 151–161, 2020. Magazine, vol. 37, no. 3, pp. 50–60, 2020. [143] M. Merluzzi, N. di Pietro, P. Di Lorenzo, E. Calvanese Strinati, [120] LibVMI, “Virtual Machine Introspection,” 2021. and S. Barbarossa, “Network energy efficient mobile edge computing 2019 IEEE Global Communications [121] M. A. Kumara and C. Jaidhar, “Leveraging virtual machine intro- with reliability guarantees,” in Conference (GLOBECOM) spection with memory forensics to detect and characterize unknown , 2019, pp. 1–6. malware using machine learning techniques at hypervisor,” Digital [144] M. Merluzzi, P. D. Lorenzo, S. Barbarossa, and V. Frascolla, “Dy- Investigation, vol. 23, pp. 99–123, 2017. namic computation offloading in multi-access edge computing via ultra-reliable and low-latency communications,” IEEE Transactions on [122] S. F. Mjolsnes and R. F. Olimid, “Private identification of subscribers Signal and Information Processing over Networks, vol. 6, pp. 342–356, in mobile networks: Status and challenges,” IEEE Communications 2020. Magazine, vol. 57, no. 9, pp. 138–144, 2019. [145] C.-F. Liu, M. Bennis, M. Debbah, and H. V. Poor, “Dynamic task [123] ETSI, “TS 103 458 V1.1.1: CYBER; Application of Attribute Based offloading and resource allocation for ultra-reliable low-latency edge Encryption (ABE) for PII and personal data protection on IoT devices, computing,” IEEE Transactions on Communications, vol. 67, no. 6, pp. WLAN, cloud and mobile services - High level requirements,” Jun 4132–4150, 2019. 2018. [146] J. Liu and Q. Zhang, “Offloading schemes in mobile edge computing 2018 Crypto [124] K. Wüst and A. Gervais, “Do you need a blockchain?” in for ultra-reliable low latency communications,” IEEE Access, vol. 6, Valley Conference on Blockchain Technology (CVCBT) . IEEE, 2018, pp. 12 825–12 837, 2018. pp. 45–54. [147] P. A. Apostolopoulos, E. E. Tsiropoulou, and S. Papavassiliou, “Risk- [125] ETSI, “Industry Specification Group (ISG)Permissioned Distributed aware data offloading in multi-server multi-access edge computing Ledger (PDL).” environment,” IEEE/ACM Transactions on Networking, vol. 28, no. 3, [126] A. Avizienis, J. . Laprie, B. Randell, and C. Landwehr, “Basic pp. 1405–1418, 2020. concepts and taxonomy of dependable and secure computing,” IEEE [148] A. J. Gonzalez, G. Nencioni, A. Kamisinski,´ B. E. Helvik, and P. E. Transactions on Dependable and Secure Computing, vol. 1, no. 1, pp. Heegaard, “Dependability of the nfv orchestrator: State of the art and 11–33, 2004. research challenges,” IEEE Communications Surveys Tutorials, vol. 20, [127] “ResliNets Wiki,” https://resilinets.org/, accessed: 22-01-2021. no. 4, pp. 3307–3329, Fourthquarter 2018. [128] K. S. Trivedi, D. S. Kim, A. Roy, and D. Medhi, “Dependability and [149] A. J. Gonzalez, G. Nencioni, B. E. Helvik, and A. Kamisinski, “A security models,” in 2009 7th International Workshop on Design of fault-tolerant and consistent sdn controller,” in 2016 IEEE Global Reliable Communication Networks, 2009, pp. 11–20. Communications Conference (GLOBECOM), 2016, pp. 1–6. [129] 5GAA Automotive Association, “Toward fully connected vehicles: [150] S.-C. Wang, W.-S. Hsiung, C.-F. Hsieh, and Y.-T. Tsai, “Reliability Edge computing for advanced automotive communications,” Dec 2017. enhancement of edge computing paradigm using agreement,” [130] ETSI, “ETSI TS 132 111-2 V16.0.0: Digital cellular telecommunica- Symmetry, vol. 11, no. 2, 2019. [Online]. Available: https: tions system (Phase 2+) (GSM); Universal Mobile Telecommunications //www.mdpi.com/2073-8994/11/2/167 System (UMTS); LTE; Telecommunication management; Fault Man- [151] G. Nencioni, B. E. Helvik, and P. E. Heegaard, “Including failure agement; Part 2: Alarm Integration Reference Point (IRP): Information correlation in availability modeling of a software-defined backbone Service (IS) (3GPP TS 32.111-2 version 16.0.0 Release 16),” Aug network,” IEEE Transactions on Network and Service Management, 2020. vol. 14, no. 4, pp. 1032–1045, 2017. [131] ——, “TS 132 332 V16.0.0: Digital cellular telecommunications [152] B. Tola, G. Nencioni, and B. E. Helvik, “Network-aware availability system (Phase 2+) (GSM); Universal Mobile Telecommunications modeling of an end-to-end nfv-enabled service,” IEEE Transactions System (UMTS); LTE; Telecommunication management; Notification on Network and Service Management, vol. 16, no. 4, pp. 1389–1403, Log (NL) Integration Reference Point (IRP); Information Service (IS) 2019. (3GPP TS 32.332 version 16.0.0 Release 16),” Aug 2020. [153] H. Raei and N. Yazdani, “Performability analysis of cloudlet in [132] J. Hao, K. Ye, and C.-Z. Xu, “Live migration of virtual machines in mobile cloud computing,” Information Sciences, vol. 388-389, pp. openstack: A perspective from reliability evaluation,” in International 99–117, 2017. [Online]. Available: https://www.sciencedirect.com/ Conference on Cloud Computing. Springer, 2019, pp. 99–113. science/article/pii/S0020025517301330 [133] L. Yala, P. A. Frangoudis, and A. Ksentini, “Latency and availability [154] P. E. Heegaard, B. E. Helvik, G. Nencioni, and J. Wäfler, “Managed driven vnf placement in a mec-nfv environment,” in 2018 IEEE Global dependability in interacting systems,” in Principles of Performance and Communications Conference (GLOBECOM), 2018, pp. 1–7. Reliability Modeling and Evaluation. Springer, 2016, pp. 197–226. [134] H. D. Chantre and N. L. Saldanha da Fonseca, “The location problem [155] M. Le, Z. Song, Y.-W. Kwon, and E. Tilevich, “Reliable and efficient for the provisioning of protected slices in nfv-based mec infrastructure,” mobile edge computing in highly dynamic and volatile environments,” 32

in 2017 Second International Conference on Fog and Mobile Edge [Online]. Available: http://www.sciencedirect.com/science/article/pii/ Computing (FMEC), 2017, pp. 113–120. S1574119220301401 [156] P. Kochovski and V. Stankovski, “Supporting smart construction with [176] F. Zeng, Y. Ren, X. Deng, and W. Li, “Cost-effective edge server dependable edge computing infrastructures and applications,” Automa- placement in wireless metropolitan area networks,” Sensors, vol. 19, tion in Construction, vol. 85, pp. 182–192, 2018. [Online]. Available: no. 1, 2019. [Online]. Available: https://www.mdpi.com/1424-8220/ https://www.sciencedirect.com/science/article/pii/S0926580517304776 19/1/32 [157] D. Malas and A. Morton, “Basic telephony sip end-to-end performance [177] K. Cao, L. Li, Y. Cui, T. Wei, and S. Hu, “Exploring placement of metrics,” IETF RFC 6076IEEE, 2011. heterogeneous edge servers for response time minimization in mobile [158] T. Hoßfeld, P. E. Heegaard, M. Varela, and S. Möller, “Qoe beyond edge-cloud computing,” IEEE Transactions on Industrial Informatics, the mos: an in-depth look at qoe via better metrics and their relation vol. 17, no. 1, pp. 494–503, 2021. to mos,” Quality and User Experience, vol. 1, no. 1, 2016. [178] B. Li, P. Hou, H. Wu, R. Qian, and H. Ding, “Placement of edge server [159] J. Moura and D. Hutchison, “Game theory for multi-access edge com- based on task overhead in mobile edge computing environment,” puting: Survey, use cases, and future trends,” IEEE Communications Transactions on Emerging Telecommunications Technologies, vol. n/a, Surveys & Tutorials, vol. 21, no. 1, pp. 260–288, 2019. no. n/a, p. e4196. [Online]. Available: https://onlinelibrary.wiley.com/ [160] T. Qiu, J. Chi, X. Zhou, Z. Ning, M. Atiquzzaman, and D. O. Wu, doi/abs/10.1002/ett.4196 “Edge computing in industrial internet of things: Architecture, advances [179] L. Lovén, T. Lähderanta, L. Ruha, T. Leppänen, E. Peltonen, J. Riekki, and challenges,” IEEE Communications Surveys & Tutorials, vol. 22, and M. J. Sillanpää, “Scaling up an edge server deployment,” in no. 4, pp. 2462–2488, 2020. 2020 IEEE International Conference on Pervasive Computing and [161] F. Spinelli and V. Mancuso, “Toward enabled industrial verticals in 5g: Communications Workshops (PerCom Workshops), 2020, pp. 1–7. A survey on mec-based approaches to provisioning and flexibility,” [180] I. Leyva-Pupo, A. Santoyo-González, and C. Cervelló-Pastor, “A IEEE Communications Surveys & Tutorials, vol. 23, no. 1, pp. 596– framework for the joint placement of edge service infrastructure and 630, 2021. user plane functions for 5g,” Sensors, vol. 19, no. 18, 2019. [Online]. [162] R. Mijumbi, J. Serrat, J. Gorricho, N. Bouten, F. De Turck, and Available: https://www.mdpi.com/1424-8220/19/18/3975 R. Boutaba, “Network function virtualization: State-of-the-art and [181] Y. Liu, Q. He, D. Zheng, X. Xia, F. Chen, and B. Zhang, “Data caching research challenges,” IEEE Communications Surveys Tutorials, vol. 18, optimization in the edge computing environment,” IEEE Transactions no. 1, pp. 236–262, 2016. on Services Computing, pp. 1–1, 2020. [163] V. Aggarwal and B. Thangaraju, “Performance analysis of virtualisation [182] H. Sinky, B. Khalfi, B. Hamdaoui, and A. Rayes, “Adaptive edge- technologies in nfv and edge deployments,” in 2020 IEEE International centric cloud content placement for responsive smart cities,” IEEE Conference on Electronics, Computing and Communication Technolo- Network, vol. 33, no. 3, pp. 177–183, 2019. gies (CONECCT) , 2020, pp. 1–5. [183] M. Wang, B. Cheng, and J. Chen, “An efficient service function [164] R. Perez and et al., “A performance comparison of virtualization chaining placement algorithm in mobile edge computing,” ACM Trans. techniques to deploy a 5g monitoring platform,” in EUCNC, 6G Internet Technol., vol. 20, no. 4, Oct. 2020. [Online]. Available: Summit, 2021. https://doi.org/10.1145/3388241 [165] F. Manco, C. Lupu, F. Schmidt, J. Mendes, S. Kuenzer, S. Sati, [184] L. Gu, J. Hu, D. Zeng, S. Guo, and H. Jin, “Service function K. Yasukata, C. Raiciu, and F. Huici, “My vm is lighter (and safer) chain deployment and network flow scheduling in geo-distributed data than your container,” in Proceedings of the 26th Symposium on centers,” IEEE Transactions on Network Science and Engineering, Operating Systems Principles, ser. SOSP ’17. New York, NY, USA: vol. 7, no. 4, pp. 2587–2597, 2020. Association for Computing Machinery, 2017, p. 218–233. [Online]. [185] M. Siew, K. Guo, D. Cai, L. Li, and T. Q. S. Quek, “Let’s share vms: Available: https://doi.org/10.1145/3132747.3132763 Optimal placement and pricing across base stations in mec systems,” [166] Z. Yu, “The Application of Kata Containers in Baidu AI Cloud,” 2019. 2021. [167] R. Behravesh, E. Coronado, and R. Riggio, “Performance evaluation on virtualization technologies for nfv deployment in 5g networks,” in [186] M. Chen, Y. Sun, H. Hu, L. Tang, and B. Fan, “Energy-saving and 2019 IEEE Conference on Network Softwarization (NetSoft), 2019, pp. resource-efficient algorithm for virtual network function placement IEEE Transactions on Green Communications 24–29. with network scaling,” and Networking, vol. 5, no. 1, pp. 29–40, 2021. [168] D. Gupta, L. Cherkasova, R. Gardner, and A. Vahdat, “Enforcing performance isolation across virtual machines in xen,” in Proceedings of [187] Y. Ma, W. Liang, M. Huang, W. Xu, and S. Guo, “Virtual network the ACM/IFIP/USENIX 2006 International Conference on Middleware, function service provisioning in mec via trading off the usages be- ser. Middleware ’06. Berlin, Heidelberg: Springer-Verlag, 2006, p. tween computing and communication resources,” IEEE Transactions 342–362. on Cloud Computing, pp. 1–1, 2020. [169] L. Cherkasova, D. Gupta, and A. Vahdat, “Comparison of the [188] Q. Liu, R. Mo, X. Xu, and X. Ma, “Multi-objective resource allocation three cpu schedulers in xen,” SIGMETRICS Perform. Eval. Rev., in mobile edge computing using paes for internet of things,” Wireless vol. 35, no. 2, p. 42–51, Sep. 2007. [Online]. Available: https: Networks, 2020. //doi.org/10.1145/1330555.1330556 [189] C. Wu, Q. Peng, Y. Xia, Y. Ma, W. Zheng, H. Xie, S. Pang, [170] J. Li, S. Xue, W. Zhang, R. Ma, Z. Qi, and H. Guan, “When i/o interrupt F. Li, X. Fu, X. Li, and W. Liu, “Online user allocation in mobile becomes system bottleneck: Efficiency and scalability enhancement for edge computing environments:a decentralized reactive approach,” sr-iov network virtualization,” IEEE Transactions on Cloud Computing, Journal of Systems Architecture, p. 101904, 2020. [Online]. Available: vol. 7, no. 4, pp. 1183–1196, 2019. http://www.sciencedirect.com/science/article/pii/S1383762120301739 [171] Z. Kotulski, T. W. Nowak, M. Sepczuk, and M. A. Tunia, “5g [190] L. Wang, Y. Zhang, and S. Chen, “Computation offloading via networks: Types of isolation and their parameters in ran and cn slices,” sinkhorn’s matrix scaling for edge services,” IEEE Internet of Things Computer Networks, vol. 171, p. 107135, 2020. [Online]. Available: Journal, pp. 1–1, 2020. http://www.sciencedirect.com/science/article/pii/S1389128619304797 [191] Z. Liu, J. Zhang, Y. Li, and Y. Ji, “Hierarchical mec servers [172] Y. Li and S. Wang, “An energy-aware edge server placement algorithm deployment and user-mec server association in c-rans over wdm in mobile edge computing,” in 2018 IEEE International Conference on ring networks,” Sensors, vol. 20, no. 5, 2020. [Online]. Available: Edge Computing (EDGE), 2018, pp. 66–73. https://www.mdpi.com/1424-8220/20/5/1282 [173] S. Wang, Y. Zhao, J. Xu, J. Yuan, and C.-H. Hsu, “Edge [192] S. Lee, S. Lee, and M. K. Shin, “Low cost mec server placement server placement in mobile edge computing,” Journal of Parallel and association in 5g networks,” in 2019 International Conference and Distributed Computing, vol. 127, pp. 160 – 168, 2019. on Information and Communication Technology Convergence (ICTC), [Online]. Available: http://www.sciencedirect.com/science/article/pii/ 2019, pp. 879–882. S0743731518304398 [193] F. Giust, L. Cominardi, and C. J. Bernardos, “Distributed mobility [174] S. K. Kasi, M. K. Kasi, K. Ali, M. Raza, H. Afzal, A. Lasebae, management for future 5g networks: overview and analysis of existing B. Naeem, S. u. Islam, and J. J. P. C. Rodrigues, “Heuristic edge server approaches,” IEEE Communications Magazine, vol. 53, no. 1, pp. 142– placement in industrial internet of things and cellular networks,” IEEE 149, 2015. Internet of Things Journal, pp. 1–1, 2020. [194] L. F. Bittencourt, M. M. Lopes, I. Petri, and O. F. Rana, “Towards vir- [175] B. Li, P. Hou, H. Wu, and F. Hou, “Optimal edge server deployment tual machine migration in fog computing,” in 2015 10th International and allocation strategy in 5g ultra-dense networking environments,” Conference on P2P, Parallel, Grid, Cloud and Internet Computing Pervasive and Mobile Computing, vol. 72, p. 101312, 2021. (3PGCIC), 2015, pp. 1–8. 33

[195] T. Taleb and A. Ksentini, “Follow me cloud: interworking federated clouds and distributed mobile networks,” IEEE Network, vol. 27, no. 5, pp. 12–19, 2013. [196] R. A. Addad, D. L. C. Dutra, M. Bagaa, T. Taleb, and H. Flinck, “Fast service migration in 5g trends and scenarios,” IEEE Network, vol. 34, no. 2, pp. 92–98, 2020. Gianfranco Nencioni is Associate Professor with [197] L. Ma, S. Yi, N. Carter, and Q. Li, “Efficient live migration of edge the University of Stavanger, Norway, from 2018. services leveraging container layered storage,” IEEE Transactions on He received the M.Sc. degree in telecommunication Mobile Computing, vol. 18, no. 9, pp. 2020–2033, 2019. engineering and the Ph.D. degree in information [198] H. Abdah, J. P. Barraca, and R. L. Aguiar, “Qos-aware service engineering from the University of Pisa, Italy, in continuity in the virtualized edge,” IEEE Access, vol. 7, pp. 51 570– 2008 and 2012, respectively. In 2011, he was a 51 588, 2019. visiting Ph.D. student with the Computer Laboratory, [199] G. Avino, M. Malinverno, F. Malandrino, C. Casetti, and C. F. University of Cambridge, U.K. He was a Post- Chiasserini, “Characterizing docker overhead in mobile edge Doctoral Fellow with the University of Pisa from computing scenarios,” in Proceedings of the Workshop on Hot Topics 2012 to 2015 and the Norwegian University of Sci- in Container Networking and Networked Systems, ser. HotConNet ’17. ence and Technology, Norway, from 2015 to 2018. New York, NY, USA: Association for Computing Machinery, 2017, p. He is currently the head of the Computer Networks (ComNet) research group 30–35. [Online]. Available: https://doi.org/10.1145/3094405.3094411 and leader of the 5G-MODaNeI project funded by the Norwegian Research [200] T. V. Doan, G. T. Nguyen, H. Salah, S. Pandi, M. Jarschel, R. Pries, Council. His research activity regards modelling and optimization in emerging and F. H. P. Fitzek, “Containers vs virtual machines: Choosing the right networking technologies (e.g., SDN, NFV, 5G, Network Slicing, Multi-access virtualization technology for mobile edge cloud,” in 2019 IEEE 2nd Edge Computing). His past research activity has been focused on energy-aware 5G World Forum (5GWF), 2019, pp. 46–52. routing and design in both wired and wireless networks and on dependability [201] I. Benkacem, T. Taleb, M. Bagaa, and H. Flinck, “Optimal vnfs of SDN and NFV. placement in cdn slicing over multi-cloud environment,” IEEE Journal on Selected Areas in Communications, vol. 36, no. 3, pp. 616–627, 2018. [202] R. G. Garroppo, S. Giordano, and L. Tavanti, “A survey on multi-constrained optimal path computation: Exact and approximate algorithms,” Comput. Netw., vol. 54, no. 17, p. 3081–3107, Dec. 2010. [Online]. Available: https://doi.org/10.1016/j.comnet.2010.05.017 [203] K. S. Trivedi, J. K. Muppala, S. P. Woolet, and B. R. Haverkort, “Composite performance and dependability analysis,” Performance Rosario G. Garroppo is Associate Professor at the Evaluation, vol. 14, no. 3, pp. 197–215, 1992, performability Modelling Dipartimento di Ingegneria dell’Informazione of the of Computer and Communication Systems. [Online]. Available: University of Pisa. His expertise is on networking https://www.sciencedirect.com/science/article/pii/016653169290004Z and his main research activities are focused on [204] D. Nicol, W. Sanders, and K. Trivedi, “Model-based evaluation: from experimental measurements and traffic modelling in dependability to security,” IEEE Transactions on Dependable and broadband and wireless networks, MoIP systems, Secure Computing, vol. 1, no. 1, pp. 48–65, 2004. traffic control techniques for multimedia services in [205] T. Courtney, S. Gaonkar, K. Keefe, E. W. D. Rozier, and W. H. wireless networks, network optimization, and green Sanders, “Möbius 2.3: An extensible tool for dependability, security, networking. On these topics, he has published more and performance evaluation of large and complex system models,” than 100 peer-reviewed papers in international jour- in 2009 IEEE/IFIP International Conference on Dependable Systems nal and conference proceedings, and won a Best Networks, 2009, pp. 353–358. Paper Award at the 4-th International Workshop on Green Communications [206] Y. Yuan, Z. Yuan, and L. Tian, “5g non-orthogonal multiple access (2011). He served as Technical Program Committee member of several study in 3gpp,” IEEE Communications Magazine, vol. 58, no. 7, pp. international conferences on wireless networks, and as referee for several 90–96, 2020. international journals and conferences. He was co-creator and co-organizer of the international IEEE Workshop on advanced EXPerimental activities ON WIRELESS networks and systems (EXPONWIRELESS), held in conjunction with IEEE WoWMoM since 2006 until 2009.

Ruxandra F. Olimid is Associate Professor at the Department of Computer Science, University of Bucharest and Adjunct Associate Professor at the Department of Information Security and Com- munication Technology, Norwegian University of Science and Technology, Trondheim. She received her Ph.D. in Computer Science from the University of Bucharest in 2013. She has a background in both computer science (BSc and MSc from the Uni- versity at Bucharest, 2008 and 2010) and telecommunications (BSc from the University Politehnica of Bucharest, 2009). She was a post-doctoral researcher with Norwegian University of Science and Technology, Norway, from 2016 to 2018. Her past experience includes Cisco certiﬁcations (CCNA, WLAN/FE) and almost 10 years in Orange Romania. Her research interests include cryptography and privacy, with a current focus on privacy and security in communication networks.