arXiv:1610.00620v1 [cs.NI] 3 Oct 2016 aec fmsae xhne sn a using end-to-en exchanged average messages the example of For latency latency. of source beco major paths a multi-hop in queuing packets [4]. successive [3], broker, Interne edge network large-scale the of near realization applications the (IoT) laten for end-to-end Things tail cornerstone bounded a A [2]. is [1], feasible applications be situation-awa not ms, real-time may 150-300 and than surgery remote-assisted longer like if for delayed Even were services. geographically sages cloud between and sub-systems devices messaging distributed on rely they differen to manner. migrate distributed a autonomously clon in platforms allow and cloud to select heterogeneous behavior dynamically flocking-like to mimicking algorithm an n eorealcto,dsrbtdITapplications. IoT distributed allocation, resource ing eiescoecnpoiemsaeboeigsries tha so service brokering message provide can II). clone Section device’s the (see brokering broker to away one-hop compared a when through latency messages dou end-to-end experience the can ble messages latency (D2D) device-to-device of sh direct Experiments source applications. major distributed large-scale another for of become an number processing capacities limited large memory Devices’ a devices. many communicates with device messages problem. a queuing Broker-less applications, not packets IoT devices. successive does the from communication solve away necessarily device-to-device one-hop using deployi broker messaging than longer same times three the is data-center amazon close analyt de in data Clones to near-the-edge nodes. subscribes compute facilitate cloud that resourceful clones service across cloning The device brokers messages. capacity message high device, of a se each For migration discovery, platforms. cloud online heterogeneous autonomous limitations and enabling the deployment, by overcomes that systems system conventional broker message based antsaewl slrenmeso eorelmtddevi propose resource-limited we of paper, numbers this In large messages. as exchange well scale technolog limit cannot communication the (D2D) syste on device-to-device dependent messaging direct highly of broker-less are latency, hand, low other ensure though the a numbers On through broker. massive having communicate sage of devices result distributed a geographically as brokers message tional hndvcscmuiaetruhamdl message middle a through communicate devices When as latency to sensitive are applications large-scale Many Terms Index hndvcsaecoe naoehpaa lult[] a [6], cloudlet away one-hop a in cloned are devices When Abstract FogMQ itiue,Itre-cl o plctosover Applications IoT Internet-Scale Distributed, Ecsieti n-oedltnyocr ihconven- with occurs latency end-to-end tail —Excessive Msaeboeigacietr,codcomput- cloud architecture, brokering —Message .I I. esg rkrSse o Enabling for System Broker Message A : eeoeeu lu Platforms Cloud Heterogeneous NTRODUCTION rgnSaeUiest,abdelwas,[email protected] University, State Oregon hrfAdlaa n ehrHamdaoui Bechir and Abdelwahab Sherif Redis FogMQ FogMQ FogMQ rkr[]i a in [5] broker 10% apply cloud- a , provides fmes- of wthat ow e,and ies, reness Flock c in ics ation mes- mes of t vice ms, ces ng cy lf- In es of of d d - t t , oe.O ore omnctn hog n-o away one-hop long resourceful a cause to still through may computation communicating clone their course, Of offload while nodes. latency to low them with allowing communicate can devices interacting irto a ii id okn n rv hti sstable is is clone that it latency autonomous that tight prove a how and achieves flocking and show birds mimic We can latency. migration self-migra weighted bounded end-to-end can a ensure tail always clones to platforms platforms, cloud between cloudlets/cloud other and of messages, by experienced latency easmsae odsatdvcs facoecnmeasure: can clone a If service devices. broker distant the 1) to when messages - relays clones/devices communicati plus between computation of latencies percentile 99-th the ering features: egtdti n-oedltny hrb tblzn clo stabilizing latency. thereby low achieving latency, and deployments end-to-end self-measu to to tail according migrate weighted platforms autonomously hosting and cloud self-discover potential to clones allows ta fteaeaeRTo single-hop. a of RTT average the of the stead where clones latency end-to-end other and ge several devices with communicate distributed clones environment, fog this heterogeneou In adminis in centralized without clones (cloudlet/clouds) necessita inter-networking platforms [4] of computing deployment fog the in ch applications locations devices However, single-h if [10]. only (average migration metric trigger and migration latency) constraining as Time units too Trip control adopt optimal central make Round using to despite struggle single-hop decisions schemes migration Such average ms. migrate 10 an nearly [9] at and (RTT) sustain [8], milliseconds, FollowMe to of and tens clones MobiScud to like seconds singl 0.5-1 technologies the from reduce [7] latency Cloudlets hop new: not are problems latency .Esrsbuddmsaeltnyo o applica- IoT of latency message bounded Ensures 2. multi-hop from arise that delays end-to-end Reduces 1. .Atnmul icvr n irtst heterogeneous, to migrates and discovers Autonomously 3. epropose We solve to migration service dynamic and cloudlets of use The esgn eadwt te devices/clones, other with demand messaging in,otefrigcnetoa esg rkr,like brokers, Redis message conventional devices’ outperforming tions, the for patterns; accounting traffic the relationship while at and communication and brokers message edge, deploying network by queuing message and FogMQ Nats eoe h rmr aec esr in- measure latency primary the becomes ; du ln rkrn ytmdsg that design system brokering clone a , aledt-n latency end-to-end tail 1+ (1 3) FogMQ ǫ h oeta latency potential the ) − a rmoptimal. from far alweighted tail a orkey four has 2) consid- - h tail the tration. they: ange red op on ne o- e- te te s 1
-14% 0.8 high edge load
Redis-Pub 0.6 Device Redis-Pub A 3x high device load x CPU Nearest EC2 DC 0.4 36.8% Redis Server 10x CPU Redis Server ZMQ-PUB/SUB 10x CPU 0.2 Edge-Redis 55% Wifi Access Point Nearest EC2-Redis Device D2D-ZMQ Device DeviceB 0 BB Redis-Sub 0 1 2 3 x CPU 10 10 10 10 end-to-end latency [ms] (a) Experiments setup. (b) CDF of latencies. (c) Tail latency in D2D vs edge brokers.
Fig. 1: Motivating experiments: the sources of latency in devices publish/subscribe communication.
Internet-scale cloud/edge platforms without needing cen- capacity VM in a host co-located with our WiFi access point tralized monitoring and control; (Edge-Redis). Our host runs other workloads. We emulate 4. Simple and requires no change to existing cloud platforms devices as simple processes running on another host that uses controllers. the same access point. We ensure that all VMs and hosts are time-synchronized with zero delay and jitter during the II. MOTIVATION AND CHALLENGES experiment execution time. A device emulator A publishes We first show that multi-hop queuing along Internet paths 10K messages to either Redis servers, and another device is a major source of end-to-end latency for IoT applications. emulator B subscribes to A’s messages. Fig. 1(b) shows the We then show that devices’ limited compute and memory Cumulative Distribution Function (CDF) of the end-to-end resources standstill against latency reduction by direct device- latency, measured as the time between receiving a message to-device communication. at B and publishing it from A. The tail end-to-end latency for Edge-Redis is 15.6ms, while it measured at 24.2ms for EC2- A. Sources of latency Redis accounting for 1.5x tail end-to-end latency improvement When a message broker is hosted in a multi-tenant cloud by avoiding multi-hop path to the closest EC2 instance and 3x platform, the end-to-end latency degrades due to network improvement on average. interference. Network interference occurs when the broker B. Why broker-less is not always the answer? messages share: ingress/egress network I/O of its host, and one or more queues in the data-center network switches Benchmark 50% 99th% [11]. Hosts and network resources become spontaneously Redis, 1000 messages 523.2 1, 276.0µs ZeroMQ, 1000 messages 314.8 647.6µs congested by traffic-demanding applications. For a single- Redis, 10, 000 messages 620.1 2, 010.5µs authority cloud, an operator can control network interference ZeroMQ, 10, 000 messages 320.3 652.9µs of latency-sensitive applications with: switching, routing, and th TABLE I: Median and 99 end-to-end latency of Redis and queuing management policies, besides controlling contention ZeroMQ measured under different loads (number of messages). for hosts’ compute, memory, and I/O resources [12], [13]. Network interference is harder to control for IoT applica- Direct device-to-device communication using broker-less tions. As devices communicate using our cloud-hosted bro- message queues can thought of to be better than using message ker, messages share network resources of multi-hop paths brokers. The obvious reasons for broker-less queues, such as with diverse and unmonitored traffic. Multiple, unfederated ZeroMQ [14], superiority are: their lightweight implementa- authorities manage network resources along these paths, which tion, and their usage of minimal number of shared queues, makes it hard to enforce unified traffic shaping or queuing switches, routers, and access points, between communicating management polices. Adding also variations in devices’ traffic devices. TABLE I shows the median and tail end-to-end demand, communication pattern with other devices, and mo- latency of ZeroMQ and Redis under different loads, where bility, it becomes particularity hard to trace devices’ traffic, ZeroMQ can deliver 10, 000 messages three times faster than delays, and infrastructure conditions to find optimal policies Redis. with centralized solutions. Multi-hop queuing along Internet Unfortunately, if the devices are resource limited, the la- paths can account for a 3x degradation in end-to-end latency tency superiority of direct device-to-device communication on average. is not always maintained. Let us return to our motivating Fig. 1(a) illustrates an experiment setup to quantify this experiment in Fig. 1(a). We limit the resources used by the latency degradation. We install a Redis server in a Virtual device emulators using Linux cgroups such that a device Machine (VM)-instance in the nearest Amazon EC2 data- emulator can use no more than 10% of the CPU time compared center (EC2-Redis), and install another Redis server in a same to EC2-Redis or Edge-Redis. Fig. 1(b) shows that the average end-to-end latency of D2D-ZeroMQ is 7 times longer than Push x + y Edge-Redis, and the tail end-to-end latency is 4 times longer. Several factors can contribute to this deteriorated performance Cloud Device including the wireless environment loading and implementa- B tion details of either Redis or ZeroMQ. However, the main Clone B factor that limits direct device-to-device latency is the limited Subscribe Subscribe compute resources of the devices emulators. Cloud/ Cloud To emphasis this observation, we increase the number Publish x Publish y of the publishing device peers (i.e. number of subscribing Clone A Infrastructure Clone C Device Network Device device emulators) until the Edge-Redis server becomes loaded. A C Fig. 1(c) shows the tail end-to-end latency for different num- bers of peers. As we increase the number of peers, the latency Fig. 2: Example overlay aggregation tree formed by device clones. superiority of the Edge-Redis starts to diminish, until we reach the 200 peers points at which our host becomes loaded at 90% utilization and the tail end-to-end latency of broker-less D2D- participating in the execution of the IoT application publishes ZeroMQ becomes better by 14%. Broker-less device-to-device its data to its brokering clone. On the other hand, clones messaging is only better if a device computational resources subscribe to each other according to the application-modeled are sufficiently large, which is an unrealistic assumption for graph, which forms an overlay network of clones to enable the most IoT devices. IoT application. Upon completion of their executions, clones may push the results back to devices. III. FogMQ SYSTEM Fig. 2 illustrates a simple tree aggregation application for Our motivating experiments show that multi-hop queuing data retrieved from three devices. Device A and C publish along Internet paths is a major source of tail end-to-end la- their data x and y to their clones. As device B is interested tency for cloud-based messaging systems, and that the latency in the result x + y, clone B subscribes to data from clones improvement promise from device-to-device communication A and C to retrieve x and y, evaluates x + y, and pushes cannot always be attained due to limited devices resources. the result to its device. The advantages of using a pub/sub FogMQ tackles multi-hop queuing by reducing the queuing of system for interacting between clones and the devices are: 1) messages: primarily, if message brokers can self-deploy and providing an efficient messaging middleware to manage large- migrate in cloudlets according to the communication pattern of scale graph structures and multiple applications, 2) relying the devices, then the impact of multi-hop queuing delay can on the already in-place subscription and matching languages be diminished. In the extreme case, if two resource-limited to effectively route information between devices and clones devices communicate through brokers in the same cloudlet and inter-clones, and 3) simplifying the design of large-scale using the same access point, we can achieve a finite minimal applications as overlay networks of and among the clones. bound on the latency. Autonomous brokers migration is the Generally speaking, the overlay network design of the foundation idea of FogMQ. clones is either structured or unstructured, and focuses mainly In this section, we derive an intuitive design of FogMQ by on minimizing a brokers fanout to minimize the communica- which we bound weighted latency given an arbitrary network tion between the clones. For example, topic-connected overlay of heterogeneous cloud platforms. Although the stability and networks are designed such that devices interested in the bounded performance of our design are intuitive, we solidify same topic are organized in a direct connected dissemination this intuition by relating the design to the theory of singleton overlay [21]. The overlay network forms the foundation for weighted congestion games [15], [16], where we show that distributed pub/sub, and directly impacts the system scalability self-deploying clones reach a Nash Equilibrium (NE) and and application performance [22], [23], [24]. We assume that tightens the Price of Anarchy (PoA) of the weighted end-to- an overlay topology of clones is given and we model it as a end delay. social network. We model the social network of clones as a graph G = (V, P ), where V denotes the set of n clones and A. Network of clones P denotes the set of all clone pairs such that p = (i, j) ∈ P To begin, we assume that devices communicate with each if the i-th and j-th clones communicate with each other. others according to IoT applications’ requirements and form a social network of devices. Typically, the convergence of B. FogMQ architecture man-machine interactions in IoT will derive devices to form We now describe how FogMQ initially creates device a social network [17]. This network can form according to clones, as well as the FogMQ’s architectural design tradeoffs. existing social network structure of users or according to the Fig. 3 illustrates the architectural elements of FogMQ. required communication among devices that is inherited from FogMQ initially clones a device at the closest cloud/cloudlet application design. from a set A of m cloud/cloudlets that are available to all The idea of application design for IoT is simple. An applica- devices and that can communicate over the Internet. An RPC tion is modeled as a graph (e.g. [18], [19], [20] ). Each device client in the device is responsible for the clone initiation and Cloud/Cloudlet resubscribes to the devices’ messages, allowing the device to Controller Server continue publishing its messages without needing the clone to FogMQ RPC Server RPC notify the device of its migration.
Device A Clone A clone can process its device’s messages in its computation Device A Overlay Cloud Controller offloading module (see Fig. 3) with high processing, memory, Sensors FogMQ Optimization RPC Client and storage capacities. The computation offloading module Readings RPC Client Computation offloading of the clone also executes distributed applications defined as overlay networks that interconnect several clones. To exchange Processing Pub Sub Pub messages between peer-clones of an overlay network, a clone
Peers SubSub Routing creates a separate process for each of its peers. Each process Pull Push SubSub Messages Peers sub flockingdecider subscribes to published messages from its corresponding peer- ClonesPeer-to-Peer processes Clonemonitoring and clone to make messages from peers available for the compu- Peers tation offloading module. If needed a clone pushes messages
Device B Device B Clone and/or computation results back to its device using a push/pull Sensors FogMQ FogMQ RPC Server messaging pattern. Fig. 3 illustrate messages flow between Readings RPC Client different modules for a simple example in which Device A Processing Pub Sub Pub is a peer to Device B. The overlay optimization module and the peer-to-peer rout- ing module are responsible for optimizing the fan-out of Fig. 3: FogMQ Architecture overlay networks and the routing decisions as we described earlier. Although the design of these mechanisms are integral to the performance of the overall system, the details overlay peer relationship definition with other devices by which a design and routing optimization algorithms are orthogonal to device participates in the execution of a distributed application the scope of this paper in which we focus on autonomous described as an overlay network of clones. Each cloud/cloudlet migration decision that minimize the tail end-to-end latency. runs FogMQ RPC server as a middleware around the cloud/- cloudlet controller which implements different solutions that C. Latency and peer-demand monitoring enable FogMQ to interact with heterogeneous cloud platforms (e.g. EC2, AWS, OpenStack cloudlet, or even a standalone Each device clone self-monitors and characterizes the de- host). A typical approach that clients can use to initiate clones mands with its peers, and evaluates latencies with the assis- + is to query a global geo-aware domain name service load tance of the hosting cloud. Let dij ∈ R denote the traffic balancer to retrieve the IP address of the nearest FogMQ RPC demand between i and j and assume that dij = dji. Let server. With the integration of cloud computing in cellular xi ∈ A denote the cloud that hosts i and l(xi, xj ) > 0 systems [25], devices can also use native cellular procedures be the average latency between i and j when hosted at xi to initiate clones in the device’s nearest cellular site. and xj , respectively (Note: if i and j are hosted at the The FogMQ RPC server realizes the device clone in a cloud same cloud, xi = xj ). We assume that l is reciprocal and platform as a virtual machine, container, or native process, monotonic. Therefore, l(xi, xj ) = l(xj , xi) and there is an where processes are always a favorable design choice to avoid entirely nondecreasing order of A → A′ such that for any ′ ′ ′ latency overhead. Recent Linpackbenchmark [26] shows that consecutive xi, xi ∈ A , l(xi, xj ) ≤ l(xi, xj ). The reciprocity containers and native processes can achieve a comparable condition ensures that measured latencies are aligned with number of floating point arithmetic per second, at least 2.5x peer-VMs and imitates the alignment rule in bird flocking. We greater than virtual machines. Despite that containers have model l(xi, xj )= τ(xi, xj )+ρ(xi)+ρ(xj ), where τ(xi, xj ) is better privacy and security advantage, as they provide a the average packet latency between xi and xj , and τ(xi, xj )= better administration, network, storage, and compute isolation, τ(xj , xi). The quantity ρ(x) is the average processing delay containers networking configuration can account for 30µs − of x modeled as: ρ(x) = δ Pi∈V :xi=x Pj∈V dij /(γ(x) latency overhead when compared to that incurred by native Pi∈V :xi=x Pj∈V dij ), where δ is an arbitrary delay constant processes [27]. As we will discuss later, implementing clones and γ(x) denotes the capacity of x to handle all demanded as processes has an advantage over both virtual machines and traffic of its hosted VMs. containers as they incur lesser migration overhead. Once creates a clone for a device, the clone FogMQ D. Learning new targets subscribes to the devices’ published messages that contain preprocessed sensors reading. Subscribing to the devices’ Each cloud runs FogMQ RPC server as a middleware. The messages eases clone migration processes as we will detail FogMQ servers in different clouds form a peer-to-peer net- later. If a clone migrates from one cloud to the other, any work that evolves autonomously. Bootstrap nodes assist newly changes to the clone IP address or network configuration joined FogMQ servers to discover other servers. Gossip proto- become transparent to the device. Upon migration, the clone col is used to spread information about new FogMQ servers. IV. Flock—AN ADAPTIVE CLONE MIGRATION the worst case of k is O(n log(nfmax)) where fmax is the ALGORITHM maximum value of the regularization function f [29], the Clones should adapt themselves to changes in the infras- figure shows that Flock scales better than O(n) on average. tructure network interconnecting the heterogenous cloud plat- forms, and should change according to the network state, struc- 120 ture, and applications’ requirements. We propose an adaptive, 110 fully distributed algorithm for dynamic cloud selection. The al- 100 ⊆ gorithm allows each VM i to learn a set Ai A (referred to as 90 i’s strategy set) from its hosting cloud xi, and to autonomously select its hosting cloud based on local measurements only. 80
k 70 Every cloud x updates its weight wx = Pi:xi=x ui(x) and broadcasts a monotonic, non-negative regularization function + + 60 f(wx): R → R with α