Understanding Operational 5G: A First Measurement Study on Its Coverage, Performance and Energy Consumption Dongzhu Xu+, Anfu Zhou+, Xinyu Zhang◇, Guixian Wang+, Xi Liu+ Congkai An+, Yiming Shi+, Liang Liu+, Huadong Ma+ +Beijing University of Posts and Telecommunications {xdz9601, zhouanfu, wangguixian, 2016213522, ACK, syming111, liangliu, mhd}@bupt.edu.cn ◇ University of California San Diego [email protected] ABSTRACT 1 INTRODUCTION 5G, as a monumental shift in cellular communication technology, We are standing on the eve of the 5G era. Major US cellular oper- holds tremendous potential for spurring innovations across many ators such as Verizon and AT&T already rolled out their first 5G vertical industries, with its promised multi-Gbps speed, sub-10 ms deployment in 2019. Meanwhile, China’s three major mobile service low latency, and massive connectivity. On the other hand, as 5G providers officially launched commercial 5G services, and have de- has been deployed for only a few months, it is unclear how well ployed more than 150 thousand 5G base stations by the end of 2019 and whether 5G can eventually meet its prospects. In this paper, we [77]. 13.7 million of 5G-enabled smartphones have been sold within demystify operational 5G networks through a first-of-its-kind cross- less than a year [94]. It is widely reported that 5G represents a giant layer measurement study. Our measurement focuses on four major leap beyond 4G. It is expected to attain multi-Gbps wireless bit-rate perspectives: (i) Physical layer signal quality, coverage and hand-off for bandwidth-hungry applications like 4K/8K UHD video/VR trans- performance; (ii) End-to-end throughput and latency; (iii) Quality mission, ultra reliable and low latency communication (uRLLC) for of experience of 5G’s niche applications (e.g., 4K/5.7K panoramic auto-driving or telesurgery [92] and also the massive machine to video telephony); (iv) Energy consumption on smartphones. The machine communication for IoT [66, 87]. Overall, 5G is predicted to results reveal that the 5G link itself can approach Gbps through- generate new economic revenue up to $12.3 trillion across a broad put, but legacy TCP leads to surprisingly low capacity utilization range of industries [73]. (<32%), latency remains too high to support tactile applications Despite the huge potential, one should be cautious that it takes and power consumption escalates to 2 − 3× over 4G. Our analysis time for 5G to evolve and mature. The most recent 5G standard suggests that the wireline paths, upper-layer protocols, computing (3GPP Release-15 [66], standardized in March 2019) focuses on and radio hardware architecture need to co-evolve with 5G to form enhancing network capacity, while low-latency and machine-type an ecosystem, in order to fully unleash its potential. communication tasks are still in progress. Moreover, the current 5G deployment commonly follows the pragmatic Non-standalone CCS CONCEPTS (NSA) mode, reusing the legacy 4G infrastructure to reduce cost. • Networks → Network measurement; Network perfor- On the other hand, whereas 5G optimization mainly resides on mance analysis; the edge (i.e., the radio access network, fronthaul/backhaul and the cellular core network), the end-to-end performance of mobile KEYWORDS applications also depends on the wireline paths, cloud servers and 5G, Network Measurement, Network Coverage, End-to-end Perfor- even the processing capacity of the mobile devices. All in all, at this mance, TCP, Energy Efficiency early stage, one natural question is: How far away is 5G from its prospects and what does it take to reach the tipping point of the ACM Reference Format: 5G ecosystem? Dongzhu Xu, Anfu Zhou, Xinyu Zhang, Guixian Wang, Xi Liu, Congkai In this paper, we perform a measurement study on one of the An, Yiming Shi, Liang Liu, Huadong Ma. 2020. Understanding Operational 5G: A First Measurement Study on Its Coverage, Performance and Energy world’s earliest commercial 5G networks, deployed in an urban Consumption. In Annual conference of the ACM Special Interest Group on environment and running on the sub-6 GHz spectrum. Using 5G- Data Communication on the applications, technologies, architectures, and enabled smartphones and custom-built tools, we conduct in-depth protocols for computer communication (SIGCOMM ’20), August 10–14, 2020, active-passive measurements to characterize 5G from the physical Virtual Event, NY, USA. ACM, New York, NY, USA, 16 pages. https://doi.org/ layer to application layer, with particular emphasis on its compar- 10.1145/3387514.3405882 ison against 4G LTE. Specifically, we build a software toolset to Permission to make digital or hard copies of all or part of this work for personal or log 5G’s physical layer information (e.g., channel quality and bit- classroom use is granted without fee provided that copies are not made or distributed rate) and fine-grained energy consumption traces to enable passive for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM diagnosis. In addition, we leverage high-bandwidth cloud servers must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to set up an application service pipeline that can take advantage to post on servers or to redistribute to lists, requires prior specific permission and/or a of the massive capacity of 5G, so as to enable active probing on fee. Request permissions from [email protected]. SIGCOMM ’20, August 10–14, 2020, Virtual Event, NY, USA the interactions among applications, networking protocols and the © 2020 Association for Computing Machinery. radio layer. ACM ISBN 978-1-4503-7955-7/20/08...$15.00 https://doi.org/10.1145/3387514.3405882 479 Measurement perspectives. Our measurement aims to demys- it is dominated by the wire-line paths. The results hint that the tify 5G from four major perspectives: legacy Internet infrastructure also needs to be retrofitted to meet (i) 5G coverage (Sec. 3). In theory, due to its usage of higher fre- the prospects of the low-latency 5G. On the other hand, mobility quencies than 4G, the 5G links suffer from more severe attenuation worsens 5G latency. We find the cross-cell hand-off takes around and penetration loss, leading to poor coverage. To understand the 108.4ms, 3:6× longer compared to 4G, mainly due to the use of the coverage issues in practice, we develop a 5G channel analytics tool NSA architecture. based on XCAL-Mobile [4] – a commercial 5G air-interface moni- (iii) As for application performance, we find that 5G offers negli- toring framework. As a result, we can profile a comprehensive set of gible benefits to mobile Web loading, whose latency is dominated physical layer metrics, including signal strength, bit-rate, hand-off by either page rendering time or TCP’s transient behavior which timing, etc, separated on a per-cell basis. severely under-utilizes the network bandwidth. For 4K panoramic (ii) End-to-end throughput and delay (Sec. 4). 5G claims to support video telephony, 5G can improve video quality and smoothness ow- Gbps bit-rate and sub-10 ms latency, through its New Radio (NR) ing to its high throughput. However, the codec/processing latency technology and a more flat core network architecture. However, tends to outweigh transmission time by 10×, i.e., the computing its practical performance faces many attrition factors, e.g., limited modules become the bottleneck in such demanding 5G use cases. capacity of wire-line paths, poor interaction across layers within (iv) We find the 5G module results in alarmingly high power con- the network stack and link quality disruptions due to frequent sumption, 2−3× over 4G and 1:8× over screen display which used to hand-off across cells (each with limited coverage). To understand dominate the 4G phone power budget [42]. More interestingly, such how these factors manifest in practice, we measure the end-to- high power consumption is intrinsic to the 5G radio hardware and end performance of mainstream transport-layer protocols, along DRX state machine, which makes standard power-saving schemes with a breakdown of network latency. Our measurement identifies ineffective. Our trace-driven simulation shows that an oracle sleep the bottlenecks and sources of anomalies that prevent 5G from scheduling mechanism can only reduce 5G power consumption delivering its expected performance. by 16.02%, 12.24% and 11.17% for web browsing, video telephony (iii) Application performance (Sec. 5). Besides network perfor- and bulk file transfer, respectively; Whereas our heuristic-based mance, the application quality of experience (QoE) also depends scheme, which opportunistically offloads certain traffic to 4G,can on other factors, especially the processing capabilities of end-user achieve 25.04% power saving compared to the 4G module. devices. We thus investigate the inter-play between the communica- Our contributions. To our knowledge, this work represents tion and computing factors, by implementing a 4K/5.7K panoramic the first cross-layer study of operational 5G New Radio (NR, or sub-6 real-time video delivery system and characterizing the feasibility GHz) networks, through a comprehensive measurement toolset. Our and challenges of the much anticipated immersive technologies main contributions can be summarized as follows: (i) Quantita- over 5G. tive characterization of 5G’s coverage in comparison to 4G, which (iv) 5G smartphone energy consumption (Sec. 6). 5G’s high bit-rate offers hints for optimizing deployment and hand-off/mobility man- comes at the cost of power-hungry signal/packet processors and agement. (ii) Identifying an alarming TCP anomaly that severely RF hardware. In this paper, we develop an energy profiling tool - underutilizes 5G capacity, diagnosing the root causes and proposing pwrStrip, to quantitatively analyze the power consumption on a practical solutions.