Emerging Edge Computing Technologies for Distributed Iot Systems

ACCEPTED FROM OPEN CALL

Emerging Edge Computing Technologies for Distributed IoT Systems

Ali Alnoman, Shree Krishna Sharma, Waleed Ejaz and Alagan Anpalagan

Abstract This remarkable increase in the number of connected devices needs to be accompanied by an The ever-increasing growth of connected equivalent increase in resource provisioning to avoid smart devices and IoT verticals is leading to the any sort of service disruption. Although the existing crucial challenges of handling the massive amount cloud computing paradigm is highly capable of han- of raw data generated by distributed IoT systems dling the massive amount of data, it is not suitable and providing timely feedback to the end-users. for distributed IoT systems due to the potentially Although the existing cloud computing paradigm incurred delays [2]. For this reason, providing com- has an enormous amount of virtual computing puting, storage, and communication functionalities power and storage capacity, it might not be able at the network edge helps not only in reducing the to satisfy delay-sensitive applications since com- end-to-end delay, but also can alleviate the burdens puting tasks are usually processed at the distant on cloud-servers and backhaul links. Furthermore, cloud-servers. To this end, edge/fog computing due to the physical proximity of edge devices with has recently emerged as a new computing par- end-users, edge computing can support distributed adigm that helps to extend cloud functionalities IoT applications that require location awareness and to the network edge. Despite several benefits of higher Quality of Service (QoS) [2, 3]. edge computing including geo-distribution, mobil- In contrast to the conventional IoT architec- ity support and location awareness, various com- ture, where storage and computing operations munication and computing related challenges are mostly performed in the cloud-center, distrib- need to be addressed for future IoT systems. In uted IoT systems incorporate nodes/gateways/ this regard, this article provides a comprehensive servers at the network edge to fulfill heterogenous view of the current issues encountered in distrib- IoT requirements with less delay and energy con- uted IoT systems and effective solutions by clas- sumption. However, edge nodes might not always sifying them into three main categories, namely, have sufficient computing and storage resources radio and computing resource management, intel- to process the massive amount of IoT data; there- ligent edge-IoT systems, and flexible infrastruc- fore, the cooperation between edge and cloud ture management. Furthermore, an optimization entities is indispensable to take the best of both framework for edge-IoT systems is proposed by computing paradigms [2]. In addition to providing considering the key performance metrics includ- computing capabilities at the vicinity of IoT users, ing throughput, delay, resource utilization and edge devices can perform various pre-process- energy consumption. Finally, an ML based case ing tasks such as data classification and filtration, study is presented along with some numerical service-level agreement ranking, and parameter results to illustrate the significance of ML in edge- measurements before involving the central-cloud. IoT computing. One of the crucial challenges in IoT systems is the limited radio resources required to provide reli- Introduction able connectivity to the massive number of devic- The next generation of Information and Com- es. Herein, one of the envisioned solutions to cope munication Technology is characterized by the with the scarcity of radio resources is to exploit and ubiquity of smart devices and machines that integrate all available communications, caching perform intelligent functions by autonomously and computing resources and radio access tech- sensing, analyzing, and exchanging information nologies (RATs) such as 5G, LTE and WiFi by using via the Internet. From E-health, smart homes and efficient resource allocation schemes. In addition, intelligent transportation to industrial manufac- harnessing large numbers of low-power small-cell turing and supply chain, Internet of Things (IoT) base stations (SBSs) can improve the cellular net- is intended to provide humanity with an easier, work capacity by allowing spatial frequency reuse safer and more intelligent lifestyle. However, the over small geographical areas. However, dealing rapid growth of IoT applications has increased the with systems characterized by such resource het- number of connected “Things” to unprecedented erogeneity requires sophisticated management levels. The number of connected devices is fore- and control schemes. To this end, implementing casted to reach about 125 billion (IHS Markit) by softwarized and virtualized platforms such as Soft- 2030, and Machine-to-Machine (M2M) commu- ware-Defined Networking (SDN) and Network nications, which constitutes a large proportion of Function Virtualization (NFV) technologies can sig- IoT applications, is expected to occupy almost 45 nificantly ease and automatize the entire network percent of the entire network traffic by 2022 [1]. control [4].

Digital Object Identifier: Ali Alnoman and Alaga Anpalgan are with Ryerson University; Shree Krishna. Sharma (corresponding author) is with the University of Luxembourg; 10.1109/MNET.2019.1800543 Waleed Ejaz is with Thomson River University.

140 0890-8044/19/$25.00 © 2019 IEEE IEEE Network • November/December 2019 Authorized licensed use limited to: Ryerson University Library. Downloaded on March 03,2020 at 17:48:05 UTC from IEEE Xplore. Restrictions apply. Most existing research works have focused on Domain Challenges Potential solutions centralized IoT systems without providing a high-level coordination among the distributed communi- Delay Intelligent task offloading mechanisms cation and computing entities. Furthermore, SDN, BBU pool, distributed fog nodes, and NFV, big data analytics and artificial intelligence are Mobility usually introduced as application-specific technolo- Heterogeneity of IoT ad-hoc fogs gies/platforms rather than being adopted within a systems fundamental optimization framework. In this work, IoT authentication, access control, ML-based we aim to present comprehensive insights on dis- Security and privacy malware detection, pseudonymization tributed IoT systems by taking into account the chal- techniques, and secured task offloading lenges that encounter both radio and computing Insufficient computing Bi-directional resource sharing between elements. Moreover, effective potential solutions resources edge and cloud servers that foster adaptivity, elasticity, and self-learning capabilities are also introduced. The main contribu- High demand for In-network caching tions of this article are highlighted below: particular content • Introduce the main practical challenges fac- Resource management ing distributed IoT computing systems and Redundant data Data aggregation and analysis highlight the potential solutions. transmission • Provide a classification of the emerging tech- More demand on the nologies in distributed IoT systems into differ- Scheduling techniques ent sub-categories along with their relevant uplink discussions. • Propose an optimization framework to tackle Various computing Cooperative hierarchical architectures various system-level aspects such as comput- entities (e.g., fog-to-cloud and cloud-to-fog) ing, delay, scheduling, and energy consump- Performance Standardization and utilization of tion. Multiple service providers coordination compatible infrastructures and platforms • Present a Machine Learning (ML)-based case study for efficient IoT device clustering in the Network slicing, cross-layer optimization, Interoperability context of edge-IoT system optimization. and load sharing Computing in Distributed IoT Systems: TABLE 1. Challenges and potential solutions for Edge-IoT systems. Challenges and Potential Solutions The ever-increasing number of IoT devices and the context of cellular IoT networks, the baseband the heterogeneous nature of their demands pose unit (BBU) pool that supports the architecture of many practical challenges, especially in regard to Heterogeneous Cloud Radio Access Networks system management and resource provision. Edge (H-CRANs) can provide significant assistance since computing can help resolve these challenges by all network resources are virtualized and managed exploiting the physical proximity with IoT devices by a unified controller. toward supporting context-awareness, data filtra- From the information security perspective, tion, and on-demand resource provision at the the limited computing resources of IoT devices network edge. In this section, we categorize the may lead to serious security challenges, especially main challenges in edge-IoT systems into three when tasks are offloaded to remote servers. Thus, main domains, and present potential enabling it is essential to harness the powerful edge com- solutions as listed in Table 1. puting resources and support IoT devices with ML and self-organizing capabilities to reduce mali- Heterogeneity of IoT Systems cious attacks. Also, it is important for IoT devices The ubiquity of IoT devices in a variety of applica- to consider the potential delay and energy con- tions diversifies the Quality of Experience (QoE) sumption when making an offloading decision [6]. requirements. One of the important QoE parameters is the end-to-end delay experienced by IoT Resource Management devices. While some IoT applications such as cli- The spatial and temporal variations of IoT devices’ mate monitoring can tolerate up to several minutes demands necessitate the bi-directional load shar- of delay, other IoT applications including autono- ing mechanisms between cloud and edge serv- mous driving and biomedical sensors can tolerate ers to maximize computing resource utilization only a few milliseconds. In addition, machine-type and reduce resource wastage. In a similar context, communications is generally characterized by a caching popular content at the network edge is bursty and low data-rate transmission, and with the considered a promising solution to reduce ser- massive number of connected machines, traffic vice delay and load at the central-cloud. Content burstiness can overload or even crash the cellular caching along with data aggregation and analysis network, in particular, the mobility management can also help reduce redundant data storage and units. For instance, according to real-time mea- transmission. Since IoT devices in general have surements in a wideband network, the duration more demand on the uplink (e.g., IoT sensors), between 90 percent of subsequent cell conges- efficient frequency allocation schemes need to tion occurrences lasts for less than 13 minutes, be developed to accommodate the massive num- and about 90 percent of these occurrences last for ber of connected IoT devices. To this end, hybrid only 1.2 seconds [5]. Hence, there is an indispens- multiple access schemes help combine the mer- able need to devise flexible radio resource allo- its of both the scheduled and random multiple cation schemes that can adapt to such burstiness access schemes depending on the real-time sys- and temporary variations in edge-IoT networks. In tem parameters [7].

IEEE Network • November/December 2019 141 Authorized licensed use limited to: Ryerson University Library. Downloaded on March 03,2020 at 17:48:05 UTC from IEEE Xplore. Restrictions apply. Emerging Technologies for Edge-IoT Systems

Resource Management

Joint Communication and Communication Resources Computing Resources Computing Resources •Dynamic resource allocation • Data filtration and analysis • Task offloading • All-to-one transmission • Cloud-edge paradigms • Dynamic code partitioning • Cellular-assisted D2D • IoT device clustering • Vehicular fog computing

Intelligent Edge-IoT Systems

Context-aware Self-organizing Big data analytics Deep learning In-network caching computing capabilities

Flexible Infrastructure Management

Reconfigurable Interoperability Self-organizing SDN NFV FPGA systems and compatibility features

FIGURE 1. Classification of emerging edge computing technologies for distributed IoT systems.

Performance Coordination Coordinated Resource Management Integrating a wide variety of services, devices, and Communication Resources: Toward improving RANs in the IoT paradigm imposes many chal- both spectral and energy efficiencies, and reduc- lenges in regard to QoE provision, compatibility, ing the overhead of information exchange among load sharing, and network-wide synchronization. the cellular nodes, the emerging C-RAN architec- In addition, the high complexity of IoT systems ture can maintain efficient coordination among necessitates the need for distributed network the coexisting RATs in the unified SDN-based BBU intelligence to provide edge nodes with deci- pool. For instance, 5G, LTE, WiFi, Coordinated Multi- sion-making capabilities and to establish a stand- Point (CoMP), millimeter wave (mmWave), mas- alone IoT environment [8]. For instance, although sive-MIMO, and Non-Orthogonal Multiple Access cloud-servers have more powerful capabilities, (NOMA) technologies can operate concurrently in the round-trip time between IoT devices and different network layers and nodes with less com- the cloud might not satisfy the desired QoE for munication overhead [7]. Moreover, Device-to-De- delay-sensitive applications. Therefore, intelligent vice (D2D) communications can reduce burdens coordination among different fog layers on one on both cellular and computing infrastructures by hand, and between fog layers and the cloud-cen- allowing mobile IoT devices to use out-of-band fre- ter on the other hand, must be maintained to min- quencies and benefit from the available computing imize the latency experienced by the end-users. resources of nearby devices [8]. Another major issue in the implementation of Computing Resources: The hierarchical com- distributed IoT computing is standardization. Unlike puting operations including the cloud-to-fog and Mobile Edge Computing (MEC), which was stan- fog-to-cloud paradigms helps to improve the utili- dardized by the European Telecommunications zation of computing resources by leveraging intel- Standards Institute (ETSI), the IoT community is still ligent task offloading decisions. It is also expected facing challenges in making global IoT standards in future computing networks that every mobile toward better flexibility and interoperability [9]. user with computing capability can take part in the IoT standardization involves the development of global computing process. The concept of consum- technical standards in regard to its architecture, er-as-a-provider is one of the implementations that protocols, identification, and security. On the allow user devices to share their available comput- communication side, standardization seems more ing capabilities with ambient devices, or further, encouraging as the 3GPP Release 13 has already with the fog and cloud servers. Furthermore, the revealed the Narrowband IoT technology to enable volatile vehicular and drone-based fogs can play a low-power wide area networking for IoT systems. role in providing on-demand computing services for adjacent IoT users with reduced latency. Classification of Joint Communication and Computing Resources: Providing SBSs with cloud-like com- Emerging Edge-IoT Technologies puting capabilities can transform those SBSs from Since IoT systems inherently integrate both cellu- just radio access nodes into the so-called “smart lar and computing functions, several technologies SBSs.” In LTE-based systems, the small-cell cloud should be considered to enable the joint operation enhanced eNodeB paradigm is a practical imple- of radio and computing infrastructures. Herein, we mentation of smart SBSs wherein both cellular classify the enabling technologies for edge-IoT sys- and edge functionalities can be attained [10]. On tems into three categories, as highlighted in Fig. 1, one side, SBSs in this paradigm are connected and provide their brief discussions below. with the cellular core via backhaul links; on the

142 IEEE Network • November/December 2019 Authorized licensed use limited to: Ryerson University Library. Downloaded on March 03,2020 at 17:48:05 UTC from IEEE Xplore. Restrictions apply. BBU pool

Heating and cooling control Smart meter and display

Smart appliances Fronthaul Backhaul

MBS Smart home

Cloud SBS

Edge Smart device city

FIGURE 2. Integrated cellular-computing architecture for edge-IoT systems. The cellular part of the system is represented by the MBS, SBSs, and the BBU pool, whereas the computing part includes edge devices and the central-cloud. other side, SBSs are associated with mobile devic- cations, especially those equipped with signal pro- es to provide computing services. By exploiting cessing functions [3]. Aided by the emerging ML the accessibility of network-wide content statistics, techniques, big data analytics can be harnessed to SBSs can achieve accurate content caching which make accurate predictions of content popularity, helps reduce both latency and communication cache this content in the network edge, and pro- overhead. Therefore, both network providers and vide timely feedback to end-users without adding cloud operators will have to collaborate in order burdens on backhaul resources [2, 11]. However, to optimize the allocation of backhaul, frequen- the complicated data analytic algorithms that are cy, and computing resources. Furthermore, the suitable for the powerful cloud need to be simplified interplay between cloud and edge providers can in order to match with the resource-constrained and support a broad diversity of IoT applications that computing-constrained edge-devices [8]. have various QoE requirements [2]. Machine/Deep Learning: The decentralized operation of distributed IoT systems can be real- Intelligent Edge-IoT Systems ized via the application of efficient ML techniques The enormous amount of data collected from that provide IoT devices and service provid- IoT devices form an important source of data- ers with self-organizing and self-healing features. sets that can train and improve the accuracy of To this end, several ML-based approaches (e.g., ML-based schemes. In the context of IoT, efficient neural networks (NNs), support vector machines ML schemes can assist in a variety of computing (SVMs), K-Means and linear regression) [12] are aspects such as content caching, task offloading, available to make intelligent decisions in several and device clustering that empower the decen- network aspects including load balancing, fault tralized operation of IoT systems. In the following, management, and adaptive resource allocation. we discuss the important components of intelli- Also, it should be noted that training data have to gent edge-IoT systems highlighted in Fig. 2. be carefully determined to achieve the best ML Content Caching: The recent advances in the performance. For instance, applications that have caching-enabled cellular architectures will help stringent delay requirements such as autonomous network elements to carry out a comprehensive driving can be guided using real-time information. reasoning and prediction about network condi- On the other hand, feedback made for applications tions and resources. In addition, the existence of that are not delay-sensitive can be made based on SBSs in the vicinity of mobile users assists in prolong-term statistics [13]. To achieve a higher level viding accurate spatio-temporal statistics about of accuracy, ML can be upgraded to the so-called popular contents; as a result, more accurate con- “deep learning” such as deep NNs that contain tent caching and better usage of the limited stor- more hidden layers and mathematical operations age capacity can be achieved. to enhance the feature extraction performance. Big Data Analytics: Performing computing tasks From the data security perspective, several ML at the network edge simplifies the evaluation and techniques including supervised, unsupervised, analysis of IoT data through aggregation, filtering, and reinforcement learning (RL) can be applied. and pre-processing prior to sending those data to For instance, SVMs and NNs are supervised learn- the distant cloud. As a result, less communication ing techniques that can be used in network intru- overhead and processing delay can be achieved, sion detection. Multivariate correlation analysis is and that is important for computing-hungry appli- an unsupervised learning technique that can be

IEEE Network • November/December 2019 143 Authorized licensed use limited to: Ryerson University Library. Downloaded on March 03,2020 at 17:48:05 UTC from IEEE Xplore. Restrictions apply. Ultra low latency for delay-sensitive applications

CPU speed Task Context offloading awareness s e c r u o s e r Spectrum Coordinated g g

n efficiency computing i ti n n u o p i s m i o v c o

r d p

Machine/deep n e NOMA a

t learning a o r i

d a t a r a

f d

h e g Energy Device i Multiple g a

H clustering and RATs s u

IoT gateways l a ti m p O

Dynamic power Task Discontinuous allocation offloading transmission

Less energy wastage and CO2 emissions

FIGURE 3. Proposed optimization framework for distributed IoT systems considering delay, resource utilization, energy and throughput as the optimization objectives. Potential solutions are identified for each objective to effectively reach the desired optimization goals.

used to detect the denial of service (DoS) attacks. necessitates the implementation of flexible man- Also, RL-based techniques such as Q-learning can agement technologies to maintain network adapt- be used to improve authentication and malware ability, re-configurability, and scalability. In the detection in IoT systems [14]. following, we discuss some of the key enablers for IoT Device Clustering: To avoid redundant data flexible infrastructure management. transmission, IoT devices can be classified and clus- SDN: As mentioned earlier, the layered fog- tered according to predetermined criteria such as cloud architecture is a promising solution to functionality or geographic location. For instance, achieve efficient resource management. How- home sensors can be processed by a central home ever, this paradigm requires flexible packet for- controller to detect unusual data (e.g., air pollu- warding over the participating fog layers on one tion) and forward these data to the homeowner or side, and between fog layers and the cloud layer specialized authorities. Moreover, within a neigh- on the other side. SDN technology can effective- borhood, data collected from the ambient sensors ly facilitate this process and reduce the inherent can be processed and verified by a neighborhood complexity of the bi-directional fog-cloud oper- controller (e.g., nearby edge device). That way, ation. In addition, the SDN-based cellular core redundant data transmission from thousands of which adopts OpenFlow controllers and protocols devices can be avoided allowing only abstracted can further ease the data forwarding process by meaningful information through the cloud-center. facilitating control, mobility management, authen- The clustering process can be managed by tication and network virtualization. For instance, edge devices that virtually cluster ambient IoT when an IoT device roams between different fog devices, extract correlated features of these clus- nodes, SDN technology helps calculate the delay ters, and make necessary decisions such as pre- cost of virtual machine (VM) migration, and then dicting future behavior of devices or cache the decides on whether to trigger the VM migration popular content for each particular cluster. A case or not based on the cost evaluation [15]. study will be presented later to illustrate the effec- NFV: By virtualizing multiple network functions tiveness of IoT device clustering on resource utili- on shared hardware, NFV technology helps not zation and QoE performance. only to reduce the complexity of both network administration and IoT system management [4], Flexible Infrastructure Management but also to improve system scalability by allowing Due to the unprecedented density and hetero- resource sharing based on runtime needs [10]. geneity of the connected IoT devices, network Examples of NFV technology include the Content management can be quite challenging. This Delivery Network (CDN) and Platform as a Ser-

144 IEEE Network • November/December 2019 Authorized licensed use limited to: Ryerson University Library. Downloaded on March 03,2020 at 17:48:05 UTC from IEEE Xplore. Restrictions apply. vice (PaaS) paradigms that allow users to access sitive applications such as autonomous driving and particular content on shared machines. health-monitoring, in which delay consequences Flexible Radio: To maintain flexible radio can be critical. Therefore, decisions on whether to resource provision, it is essential for cellular net- process tasks using the on-device processor or to works to conduct elastic frequency allocation offload tasks to the edge-device have to be care- strategies that can adapt to the real-time demands fully made based on accurate estimation of the of IoT devices. Here, implementing the cog- round-trip time and the amount of required CPU nitive radio technology that supports adaptivity cycles that satisfy the task completion deadline in resource allocation and network tier associa- set by IoT applications. Moreover, the arrival rate tion (e.g., macro, pico, femto cells), can play an of incoming tasks to edge-devices must be incor- important role in realizing the goals of scalabili- porated in such decisions to avoid overloading ty and flexibility in IoT systems, especially when server buffers. equipped with efficient ML capabilities. Resource Utilization: Establishing a high-level coordination among IoT devices, edge-devices, Proposed Optimization Framework for and the central-cloud is crucial to take full advan- tage of all available computing resources. From Edge-IoT Systems the communication point of view, the SDN-based Herein, we present an integrated cellular-com- BBU pool helps to optimally allocate frequency puting architecture which is comprised of multi- resources to IoT devices by taking the bandwidth ple cellular elements, namely, SBSs, MBS, BBU and interference constraints into account. In the pool, and backhaul links, whereas the computing context of smart IoT systems, it is important to elements are represented by the cloud and edge support IoT gateway systems with ML-based devices as depicted in Fig. 2. algorithms and protocols to dynamically adapt As depicted in Fig. 3, the proposed optimiza- to the large-scale heterogeneity of IoT devices, tion framework aims to achieve the following four and to automatize the process of registering objectives: newly added IoT devices in the network database. • Ultra-low latency for delay-sensitive IoT appli- Moreover, the formation of volatile on-demand cations. fog nodes and device clustering can optimize • Optimal usage of available radio and com- resource usage in edge-IoT systems. puting resources. Energy: As a major challenge in future commu- • Less energy consumption and CO2 emis- nication and computing systems, energy has to be sions. addressed not only to prolong the lifetime of the • High data rate provisioning. on-device batteries, but also to reduce the exces- The first objective can be achieved via opti- sive amounts of CO2 emission. In this direction, mal CPU speed determination, smart offloading some promising solutions include efficient power decisions, and context awareness. In the second allocation, short-range transmission via SBSs, CPU objective, resource utilization can be enhanced cycle optimization, task offloading, discontinuous by using SDN technology, coordinated comput- transmission, and hardware sleeping mechanisms. ing, and efficient device clustering mechanisms. Throughput: Providing IoT devices with the The third objective deals with energy efficiency powerful edge computing capabilities must be which is a major concern in IoT systems. Energy accompanied by a high data-rate provisioning to consumption can be optimized using different avoid undesired latencies. To this goal, exploit- techniques such as dynamic power allocation, ing the spatial frequency reuse of SBSs over energy-aware task offloading, and discontinuous small areas can increase the per-user data rate. In transmission. Maximizing data rate is the fourth addition, the recent advances in communication objective and can be fulfilled using several technologies including NOMA, massive MIMO, enabling technologies such as NOMA, distributed 5G, LTE-A and WiFi can diversify the supply of fre- small cells, and multi-RAT network planning. quency resources, and thus improve the spectral Since IoT systems constitute a mixture of both efficiency. computing and communication networks, data security and user privacy are inherently involved Use Case Study in the aforementioned enabling technologies. Toward avoiding the congestion of backhaul links For instance, IoT devices have to recognize the and saving both computing and radio resourc- secure fog-servers to avoid identity-based attacks es, IoT devices can be virtually clustered at the such as spoofing. Moreover, IoT resources in the network edge forming virtual IoT groups. Herein, edge nodes must be secured by undertaking effi- we present a case study to demonstrate the sig- cient authentication and access control strategies. nificance of RL-assisted solutions in forming IoT Coordinated computing schemes can provide device clusters. In such a scenario, RL-based data IoT devices with enhanced defensive strategies classification and aggregation can help not only benefiting from the powerful computing capa- to save computing resources but also to support bilities of the cloud and edge processors. To this a scalable computing paradigm that can adapt end, ML-based data security techniques such to sudden changes in data volumes and traffic as authentication, access control, and malware variations. Figure 4 demonstrates the proposed detection need to be implemented to strengthen case study. IoT defense strategies against potential threats We consider a distributed IoT system where [14]. In the following, we briefly describe some smart home devices and sensors offload their promising solutions to improve the four perfor- tasks to the edge-devices for processing as mance metrics that are highlighted in Fig. 3. shown in Fig. 4a. The clustering is managed by Delay: Minimizing delay is one of the key the edge-device which is located at the vicinity of objectives in IoT systems, especially for delay-sen- IoT devices and possesses K servers (VMs). The

IEEE Network • November/December 2019 145 Authorized licensed use limited to: Ryerson University Library. Downloaded on March 03,2020 at 17:48:05 UTC from IEEE Xplore. Restrictions apply. ered “delayed,” the RL-based controller changes the reward of incrementing and decrementing the Edge device number of members into –10 and 5, respectively. The used values for the discount factor and learn-

VM 1 ing rate in the RL-based scheme are 0.9 and 0.1,

respectively. g n i VM 2

e r Figure 5 depicts the system performance using s t . the RL-based and equal-size clustering schemes. Cl u . In the “equal-size” clustering scheme, IoT devices VM K are evenly distributed among all available VMs in the system regardless of the data size and task completion deadline. Unlike the “equal-size” (a) clustering scheme which lacks QoE awareness, the RL-based scheme monitors the QoE experi- Agent (Edge device) enced by IoT devices. It can be observed in Fig. 5a that the RL-based scheme provides better performance in regard to reducing the amount State Reward Action of required computing resources represented by the number of VMs. The “equal-size” cluster- Environment ing scheme utilizes all available VMs all the time (QoE) and that keeps the number of used resources (b) fixed at 5, while the RL-based scheme is capable of increasing the percentage of VM utiliza- FIGURE 4. Illustration of the proposed virtualized IoT tion by filling up VMs more quickly compared to device clustering: a) system layout; b) RL-based the other scheme, which explains the non-linear learning mechanism. behavior of the RL-based scheme as shown in Fig. 5b. An interesting observation in this study is that despite all VMs being utilized in the “equal-size” edge-device aims to maximize each VM’s utiliza- scheme, the RL-based scheme can provide better tion by aggregating IoT tasks under the constraint QoE performance (fewer delayed IoT devices) of VM capacity and task completion deadline of as observed in Fig. 5c. This is due to the diversi- IoT devices. Figure 4b illustrates the RL-based ty of IoT devices’ demands in regard to the data scheme where the edge-device is responsible for size and task completion deadline which requires taking the increment and decrement action on a QoE-aware and adaptive resource allocation the number of IoT devices associated with each mechanism. In other words, due to the variety of VM. The edge-device then monitors the QoE task sizes and task completion deadlines among that results from the taken action; in particular, IoT devices, some IoT clusters might have higher it monitors the delay experienced by IoT devices. processing demands compared to others, and The number of states in the system corresponds that results in some clusters being satisfied while to the number of IoT devices whereby the agent others not. Also, it can be noted that having more (edge-device) moves through these states and stringent delay requirements (i.e., m = 0.5 s) neces- takes the required actions. sitates the utilization of more computing resourc- Each IoT device is assumed to have a particu- es (VMs), and increases the number of delayed lar packet size and a task completion deadline to devices since the task completion deadline can be accomplish the task. The packet size of IoT devic- exceeded more easily. es is uniformly distributed between 500 kB and 4 MB. The task completion deadline is assumed Conclusions to differ among IoT devices as follows; the first Edge computing along with the central-cloud con- group has a deadline range between 100–900 ms stitute a powerful computing paradigm for the (i.e., mean delay m = 0.5 s), whereas the second practical implementation of distributed IoT sys- group has a deadline range of 500–1500 ms (i.e., tems. However, there still exist several challenges mean delay m = 1 s). The edge device has K = 5 from both the communication and computing VMs each of which has a processing capacity of perspectives. Various technologies including 500 MHz. The aim of this study is to investigate cooperative resource management, ML, con- the optimal number of IoT devices associated text-aware computing, and flexible infrastructure with each VM such that fewer VMs are to be used management have emerged in this direction as while satisfying the deadline requirement of IoT promising solutions. This article presented a com- devices. To achieve this goal, Q-learning, which is prehensive view of the existing research issues, an RL-based technique, is used to adaptively clus- and introduced potential solutions along with an ter IoT devices into available VMs. Two actions optimization framework taking into account the are considered during the Q-learning process, key performance metrics in edge-IoT systems. namely, “increment” and “decrement,” where the The upcoming IoT era calls for serious efforts reward of incrementing and decrementing the in the direction of ML, SDN, and NFV to establish number of cluster members is +5 and –1, respec- self-organized and self-resilient computing systems tively. Hence, increasing the number of IoT devic- that can cope with the heterogeneity of IoT ser- es per cluster is always preferred; however, since vices. Furthermore, since both communication and the VM capacity is limited, cluster members can computing parties are inherently integrated in IoT experience more delay, and the task completion systems, system optimization must consider the deadline will be violated. To tackle the aforemen- constraints imposed by the limitations of cellular tioned problem, when any IoT device is consid- networks such as insufficient resources, spatio-tem-

146 IEEE Network • November/December 2019 Authorized licensed use limited to: Ryerson University Library. Downloaded on March 03,2020 at 17:48:05 UTC from IEEE Xplore. Restrictions apply. poral variations of wireless links, and other issues including energy consumption and cost. In other 8 Equal-size words, a high-level coordination among the cellular RL-based, =0.5s and edge nodes is essential to fulfill the deploy- RL-based, =1s ment of efficient edge-IoT computing systems. 6 eferences R 4 [1] A. Al-Fuqaha et al., “Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications,” IEEE Commun. Surveys Tuts., vol. 17, no. 4, Fourth Quarter 2015, pp. 2

2347–76. Number of clusters [2] S. K. Sharma and X. Wang, “Live Data Analytics with Col- laborative Edge and Cloud Processing in Wireless IoT Net- 0 works,” IEEE Access, vol. 5, Mar. 2017, pp. 4621–35. 5 10 15 20 25 30 [3] N. Abbas et al., “Mobile Edge Computing: A Survey,” IEEE Internet of Things J., vol. 5, no. 1, Feb. 2018, pp. 450–65. Number of IoT devices [4] F. van Lingen et al., “The Unavoidable Convergence of NFV, (a) 5G, and Fog: A Model-Driven Approach to Bridge Cloud 100 and Edge,” IEEE Commun. Mag., vol. 55, no. 8, Aug. 2017, Equal-size, =0.5s pp. 28–35. [5] X. Zhou et al., “Toward 5G: When Explosive Bursts Meet 80 Equal-size, =1s Soft Cloud,” IEEE Network, vol. 28, no. 6, Dec. 2014, pp. RL-based, =0.5s 12–17. RL-based, =1s [6] X. Meng, W. Wang, and Z. Zhang, “Delay-Constrained 60 Hybrid Computation Offloading with Cloud and Fog Com- puting,” IEEE Access, vol. 5, Sept. 2017, pp. 21355–67. 40 [7] A. Alexiou, “Wireless World 2020: Radio Interface Challeng-

es and Technology Enablers,” IEEE Veh. Technol. Mag., vol. VM usage (%) 9, no. 1, Mar. 2014, pp. 46–53. 20 [8] Y. Sahni et al., “Edge Mesh: A New Paradigm to Enable Dis- tributed Intelligence in Internet of Things,” IEEE Access, vol. 0 5, Aug. 2017, pp. 16441–58. 5 10 15 20 25 30 [9] S. Chen et al., “A Vision of IoT: Applications, Challenges, and Opportunities with China Perspective,” IEEE Internet of Number of IoT devices Things J., vol. 1, no. 4, Aug. 2014, pp. 349–59. (b) [10] S. Barbarossa, S. Sardellitti, and P. Di Lorenzo, “Communi- 3 cating while Computing: Distributed Mobile Cloud Comput- Equal-size, =0.5s ing over 5G Heterogeneous Networks,” IEEE Signal Process. Mag., vol. 31, no. 6, Nov. 2014, pp. 45–55. Equal-size, =1s [11] E. Zeydan, “Big Data Caching for Networking: Moving 2 RL-based, =0.5s from Cloud to Edge,” IEEE Commun. Mag., vol. 54, no. 9, RL-based, =1s Sept. 2016, pp. 36–42. [12] C. Jiang et al., “Machine Learning Paradigms for Next-Gen- eration Wireless Networks,” IEEE Wireless Commun., vol. 24, no. 2, Apr. 2017, pp. 98–105. 1 [13] Y. Mao et al., “Stochastic Joint Radio and Computational Resource Management for Multi-User Mobile-Edge Comput- ing Systems,” IEEE Trans. Wireless Commun., vol. 16, no. 9, Sept. 2017, pp. 5994–6009. 0 [14] L. Xiao et al., “IoT Security Techniques Based on Machine Number of delayed devices 5 10 15 20 25 30 Learning: How Do IoT Devices Use AI to Enhance Secu- rity?” IEEE Signal Process. Mag., vol. 35, no. 5, Sept. 2018, Number of IoT devices pp. 41–49. (c) [15] X. Sun and N. Ansari, “EdgeIoT: Mobile Edge Computing for the Internet of Things,” IEEE Commun. Mag., vol. 54, no. 12, pp. 22–29, Dec. 2016. FIGURE 5. Performance of the proposed IoT device clustering method: a) number of required clus- Biographies ters under different schemes; b) percentage of Ali Alnoman received his B.Sc. and M.Sc. degrees in electrical computing resource utilization; c) average num- engineering from the University of Baghdad, Iraq, in 2009 and ber of delayed IoT devices. 2012, respectively. He is currently pursuing the Ph.D. degree in the Department of Electrical and Computer Engineering at Ryerson University, Toronto, Canada. His research interests ty, and Queen’s University in Canada. He received the Ph.D. include energy efficiency and resource allocation in HetNets, degree in information and communication engineering from IoT, and cloud computing. During 2012-2014, he worked as a Sejong University, Republic of Korea. He has co-authored over faculty member at Ishik University, Erbil, Iraq. He also served as 90 papers in prestigious journals and conferences, and three a technical program committee member for the IEEE Vehicular books. His current research interests include Internet of Things Technology Conference VTC2017-Fall in Toronto. (IoT), energy harvesting, 5G and beyond networks, and mobile edge computing. He is a registered Professional Engineer in the Shree Krishna Sharma is currently a research scientist at the province of British Columbia, Canada. Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg. Prior to this, he held postdoctoral Alagan Anpalagan received the BASc, MASc, and Ph.D. research positions at the University of Western Ontario, Canada degrees in electrical engineering from the University of Toron- and at the SnT after obtaining his Ph.D. degree from the Uni- to, Canada. He is a full professor in the ELCE Department at versity of Luxembourg in 2014. He has published more than 80 Ryerson University, Canada where he served as Associate Chair technical papers in scholarly journals and international confer- for Graduate Studies, EE Program Director, and Graduate Pro- ences, and has over 1500 Google Scholar citations. His current gram Director. He has co-authored four edited books, including research interests include 5G and beyond wireless, IoT, machine Design and Deployment of Small Cell Networks, Powerline Com- learning and edge computing. He is Senior Member of IEEE and munication Systems for Smart Grids, and Handbook on Green an associate editor of IEEE Access. Information and Communication Systems. He was a recipient of the IEEE Canada J.J. Ham Outstanding Engineering Educa- Waleed Ejaz is an assistant professor in the Department of tor Award (2018), Ryerson YSGS Outstanding Contribution to Applied Science & Engineering at Thompson Rivers Universi- Graduate Education Award (2017), and IEEE M.B. Broughton ty, Kamloops, BC, Canada. Previously, he held academic and Central Canada Service Award (2016). He is a Fellow of IET and research positions at Ryerson University, Carleton Universi- a Fellow of the Engineering Institute of Canada.

IEEE Network • November/December 2019 147 Authorized licensed use limited to: Ryerson University Library. Downloaded on March 03,2020 at 17:48:05 UTC from IEEE Xplore. Restrictions apply.