Resource Management in Fog/Edge Computing: A Survey Cheol-Ho Hong and Blesson Varghese Abstract—Contrary to using distant and centralized cloud data center resources, employing decentralized resources at the edge of a network for processing data closer to user devices, such as smartphones and tablets, is an upcoming computing paradigm, referred to as fog/edge computing. Fog/edge resources Cloud Data Center are typically resource-constrained, heterogeneous, and dynamic compared to the cloud, thereby making resource management an important challenge that needs to be addressed. This article re- views publications as early as 1991, with 85% of the publications between 2013–2018, to identify and classify the architectures, in- frastructure, and underlying algorithms for managing resources in fog/edge computing. Fog/Edge Nodes I. INTRODUCTION Accessing remote computing resources offered by cloud data centers has become the de facto model for most Internet- based applications. Typically, data generated by user devices User Devices such as smartphones and wearables, or sensors in a smart city /Sensors or factory are all transferred to geographically distant clouds to be processed and stored. This computing model is not practical Fig. 1. A fog/edge computing model comprising the cloud, resources at the for the future because it is likely to increase communication edge of the network, and end-user devices or sensors latencies when billions of devices are connected to the In- ternet [1]. Applications will be adversely impacted because of the increase in communication latencies, thereby degrading the Contrary to cloud resources, the resources at the edge are: overall Quality-of-Service (QoS) and Quality-of-Experience (i) resource constrained - limited computational resources (QoE). because edge devices have smaller processors and a limited An alternative computing model that can alleviate the above power budget, (ii) heterogeneous - processors with different problem is bringing computing resources closer to user devices architectures, and (iii) dynamic âA¸Stheir˘ workloads change, and sensors, and using them for data processing (even if only and applications compete for the limited resources. Therefore, partial) [2], [3]. This would reduce the amount of data sent managing resources is one of the key challenges in fog and to the cloud, consequently reducing communication latencies. edge computing. The focus of this article is to review the To realize this computing model, the current research trend architectures, infrastructure, and algorithms that underpin re- arXiv:1810.00305v1 [cs.DC] 30 Sep 2018 is to decentralize some of the computing resources available source management in fog/edge computing. Figure 2 presents in large data centers by distributing them towards the edge of the areas covered by this article. the network closer to the end-users and sensors, as depicted Figure 3 shows a histogram of the total number of re- in Figure 1. These resources may take the form of either search publications reviewed by this article between 1991 and (i) dedicated ‘micro’ data centers that are conveniently and 2018 under the categories: (i) books and book chapters, (ii) safely located within public/private infrastructure or (i) Inter- reports, including articles available on pre-print servers or net nodes, such as routers, gateways, and switches that are white papers, (iii) conference or workshop papers, and (iv) augmented with computing capabilities. A computing model journal or magazine articles. Similar histograms are provided that makes use of resources located at the edge of the network for each section. More than 85% of the articles reviewed were is referred to as ‘edge computing’ [4], [5]. A model that makes published from 2013. use of both edge resources and the cloud is referred to as ‘fog computing’ [6], [7]. The remainder of this article is structured as follows. Sec- tion II discusses resource management architectures, namely Cheol-Ho Hong is with the School of Electrical and Electronics Engineer- the dataflow, control, and tenancy architectures. Section III ing, Chung-Ang University, South Korea. E-mail: [email protected] presents the infrastructure used for managing resources, such Blesson Varghese is with the School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, United Kingdom. E-mail: as the hardware, system software, and middleware employed. [email protected] Section IV highlights the underlying algorithms, such as 1 System Virtualization Network Virtualization Volunteer Edge Computing Volunteer Computing Hierarchical Fog/Edge Computing Mobile Fog/Edge OrchestrationManagement Cloud Computation Devices Computation Network Devices Aggregation Sharing Offloading Single Multi Centralized Distributed Hierarchical Hardware SystemSoftware Middleware Dataflow Control Tenancy Architectures Infrastructure Resource Management in Fog/Edge Computing Fog/Edge in 2 Algorithms Discovery Placement Benchmarking Load-balancing Fig. 2. A classification of the architectures, infrastructure, and algorithms for resource management in fog/edge computing Message Passing Graph-based Iterativetechniques Handshaking Protocol Breadth First Search Dynamic Condition-aware Programming Infrastructure Integrated Benchmarking Application Benchmarking Cooperative Load-balancing Particle Swarm Optimization Evaluating Functional Properties Evaluating 60 B R C J Tree-based 50 Graph-based Directed Graph-based 40 Cluster-based 30 For Modeling and Implementing Aggregation Petri net-based Frequency 20 Decoupled 10 Batch 0 Aggregation Hybrid Techniques 1991 2002 2003 2004 2005 2006 2007 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 Bandwidth Ye a r Efficiency-aware Latency Energy Fig. 3. A histogram of the total number of research publications on resource For Improving management in fog/edge computing reviewed by this article. Legend: B - Aggregation Quality-aware books or book chapters; R - reports, including articles available on pre-print Security and servers or white papers; C - conference or workshop papers; J - journal or Privacy-aware magazine articles. Heterogeneity-aware 25 B R C J Fig. 5. A classification of aggregation techniques 20 15 A. Data Flow 10 Frequency This survey identifies key data flow architectures based 5 on how data or workloads are transferred within a fog/edge 0 computing environment. This section considers three data flow architectures, namely aggregation, sharing, and offloading. 2002 2003 2004 2005 2006 2007 2011 2012 2013 2014 2015 2016 2017 2018 1) Aggregation: In the aggregation model, an edge node Ye a r obtains data generated from multiple end devices that is then partially computed for pruning or filtering. The aim in the Fig. 4. A histogram of publications reviewed for the classification of architectures for resource management in fog/edge computing. Legend: B - aggregation model is to reduce communication overheads, in- books or book chapters; R - reports, including articles available on pre-print cluding preventing unnecessary traffic from being transmitted servers or white papers; C - conference or workshop papers; J - journal or beyond the edge of the network. Research on aggregation magazine articles. can broadly be classified on the basis of (i) techniques for modeling and implementing aggregation, and (ii) techniques discovery, benchmarking, load balancing, and placement. Sec- for improving aggregation, as shown in Figure 5. tion V suggests future directions and concludes the paper. i. Techniques for Modeling and Implementing Aggregation: The underlying techniques implemented for supporting ag- II. ARCHITECTURES gregation have formed an important part of Wireless Sensor In this survey, the architectures used for resource manage- Networks (WSNs) [8] and distributed data stream process- ment in fog/edge computing are classified on the basis of data ing [9]. Dense and large-scale sensor networks cannot route flow, control, and tenancy. all data generated from sensors to a centralized server, but • Data flow architectures: These architectures are based on instead need to make use of intermediate nodes along the the direction of movement of workloads and data in the data path that aggregate data. This is referred to as in-network computing ecosystem. For example, workloads could be data aggregation [10]. We consider WSNs to be predecessors transferred from the user devices to the edge nodes or of modern edge computing systems. Existing research in the alternatively from cloud servers to the edge nodes. area of in-network data aggregation can be classified into the • Control architectures: These architectures are based on how following six ways on the basis of the underlying techniques the resources are controlled in the computing ecosystem. used for modeling and implementing aggregation: For example, a single controller or central algorithm may be a. Graph-based Techniques: In this survey, we report two used for managing a number of edge nodes. Alternatively, graph-based techniques that are used for data aggregation, a distributed approach may be employed. namely tree-based and directed graph-based techniques. • Tenancy architecture: These architectures are based on the Tree-based Techniques: Two examples of tree-based tech- support provided for hosting multiple entities in the ecosys- niques are Data Aggregation Trees (DATs) and spatial index tem. For example, either a single application or multiple trees. DATs are commonly used for aggregation in WSNs applications could be hosted on an edge