Received June 3, 2020, accepted July 7, 2020, date of publication July 9, 2020, date of current version July 29, 2020. Digital Object Identifier 10.1109/ACCESS.2020.3008250 Hardware-Accelerated Platforms and Infrastructures for Network Functions: A Survey of Enabling Technologies and Research Studies PRATEEK SHANTHARAMA 1, (Graduate Student Member, IEEE), AKHILESH S. THYAGATURU 2, (Member, IEEE), AND MARTIN REISSLEIN 1, (Fellow, IEEE) 1School of Electrical, Computer, and Energy Engineering, Arizona State University (ASU), Tempe, AZ 85287, USA 2Programmable Solutions Group (PSG), Intel Corporation, Chandler, AZ 85226, USA Corresponding author: Martin Reisslein ([email protected]) This work was supported by the National Science Foundation under Grant 1716121. ABSTRACT In order to facilitate flexible network service virtualization and migration, network func- tions (NFs) are increasingly executed by software modules as so-called ``softwarized NFs'' on General- Purpose Computing (GPC) platforms and infrastructures. GPC platforms are not specifically designed to efficiently execute NFs with their typically intense Input/Output (I/O) demands. Recently, numerous hardware-based accelerations have been developed to augment GPC platforms and infrastructures, e.g., the central processing unit (CPU) and memory, to efficiently execute NFs. This article comprehensively surveys hardware-accelerated platforms and infrastructures for executing softwarized NFs. This survey covers both commercial products, which we consider to be enabling technologies, as well as relevant research studies. We have organized the survey into the main categories of enabling technologies and research studies on hardware accelerations for the CPU, the memory, and the interconnects (e.g., between CPU and memory), as well as custom and dedicated hardware accelerators (that are embedded on the platforms); furthermore, we survey hardware-accelerated infrastructures that connect GPC platforms to networks (e.g., smart network interface cards). We find that the CPU hardware accelerations have mainly focused on extended instruction sets and CPU clock adjustments, as well as cache coherency. Hardware accelerated interconnects have been developed for on-chip and chip-to-chip connections. Our comprehensive up-to-date survey identifies the main trade-offs and limitations of the existing hardware-accelerated platforms and infrastructures for NFs and outlines directions for future research. INDEX TERMS Central processing unit (CPU), hardware accelerator, interconnect, memory, software defined networking (SDN), virtualized network function (VNF). I. INTRODUCTION layer (e.g., for the datagram routing NF and Internet Protocol A. TREND TO RUN SOFTWARIZED NETWORK FUNCTIONS firewall NF). These low-level NFs were usually executed ON GENERAL-PURPOSE COMPUTING (GPC) PLATFORMS in specially designed dedicated (and typically proprietary) Traditionally, the term ``network function (NF)'' applied networking equipment, such as switches, routers, and gate- primarily to functions of the lower network protocol lay- ways. Recently, the definition of an NF has been broadened ers, i.e., mainly the data link layer (e.g., for the data link to describe networking related tasks spanning from low- layer frame switching NF, virtual local area network NF, level frame switching and Internet Protocol (IP) routing to and medium access control security NF) and the network high-level cloud applications [1]–[3]. The area of network- ing currently undergoes an unprecedented transformation in The associate editor coordinating the review of this manuscript and moving towards implementing NFs as software entities— approving it for publication was Junaid Shuja . so-called ``softwarized NFs'’—that run on General-Purpose This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ VOLUME 8, 2020 132021 P. Shantharama et al.: Hardware-Accelerated Platforms and Infrastructures for NFs FIGURE 1. Illustration of GPC platform hardware to process Network Functions (NFs). An NF can be implemented as a Bare Metal NF, Application NF (not shown), Virtual NF (VNF), or Container NF (CNF). Computing (GPC) platforms and infrastructures as opposed The push towards ``softwarized NFs'' is to reduce the hard- to dedicated networking equipment hardware. ware dependencies of NFs for function implementation so as In order to motivate this survey on hardware-accelerated to maximize the flexibility for operations, e.g., to allow for platforms and infrastructures for softwarized NFs, we briefly the flexible scaling and migration of NF services. Softwarized introduce the basic concepts of softwarized NFs, includ- NFs enable compute operations to be implemented as generic ing their computation and management on GPC platforms executing programs in the form of applications that can be run and infrastructures, in the following paragraphs. We then on a traditional OS or isolated environments, such as Virtual explain the need for hardware-acceleration of softwarized Machines (VMs) [4] or containers [5], on GPC platforms. NFs on GPC platforms and infrastructures in Section I-B, Analogous to the broad term ``Network Function (NF)'', followed by an overview of the contributions of this survey the term ``Network Function Virtualization (NFV)'' broadly in Section I-C. refers to NF implementation as a virtualized entity, typically as an application (which itself could run inside a container), and running inside a VM (see Fig. 1). Thus, NFV is an 1) NETWORK FUNCTIONS (NFs) AND NETWORK implementation methodology of an NF; while the term NF FUNCTION VIRTUALIZATION (NFV) broadly refers to compute operations related to general packet The term ``Network Function (NF)'' broadly encompasses processing. Moreover, the term ``Virtual Network Function the compute operations (both logical [e.g., bitwise AND (VNF)'' refers to an NF that is implemented with the NFV or OR] and mathematical scalar and vector [e.g., integer and methodology. floating point arithmetic]) related either directly or indirectly to data link layer (Layer 2) frames, network layer (Layer 3) datagrams or packets, and network application data (higher 2) ROLE OF SOFTWARE DEFINED NETWORKING (SDN) protocol layers above Layer 3). For instance, a packet filter IN THE MANAGEMENT OF NFs is a direct logical NF that compares header data to allow or Software Defined Networking (SDN) [6]–[9] is a paradigm block packets for further processing, while a jitter and latency in which a logically centralized software entity (i.e., the SDN estimator function is an example of an indirect arithmetic NF. controller) defines the packet processing functions on a An NF that requires dedicated processing with a strict dead- packet forwarding node. The notion of centralized decision line, e.g., an NF to verify a medium access control (MAC) making for the function implementation and configuration frame error through a Cyclic Redundancy Coding (CRC) of forwarding nodes implies that the network control plane check, is preferably implemented as a hardware component. (which makes the decisions on the packet processing) is On the other hand, an NF with relaxed timing requirements, decoupled from the network data plane (which forwards the e.g., TCP congestion control, can be implemented as a soft- packets). Extending the principles of SDN from forwarding ware entity. nodes to the broad notion of compute nodes can achieve 132022 VOLUME 8, 2020 P. Shantharama et al.: Hardware-Accelerated Platforms and Infrastructures for NFs more flexibilities in the deployment of NFs on GPC platforms and virtualization of NFs can reduce CAPEX and OPEX of in terms of scalability and management [10], [11]. More the network operator. In particular, the network operator can precisely, SDN can be applied for two primary purposes: upgrade, install, and configure the network with a centralized i) macro-scale NF deployments, where the decisions involve control entity. Thus, MEC and virtualization are seen as key selecting a specific platform for NF deployments based on building blocks of future network infrastructures, while SDN decision factors, such as physical location, capabilities, and enables efficient network service management. availability, and ii) micro-scale NF deployments, where the decisions involve reconfiguring the NF parameters during B. NEED FOR NF HARDWARE ACCELERATION on-going operations based on run-time requirements, such as ON GPC PLATFORM traffic loads, failures and their restoration, as well as resource The NF softwarization makes the overall NF development, utilization. deployment, and performance characterization at run time more challenging [11]. Softwarized NFs rely on GPC central 3) COMPUTE NODES FOR RUNNING NFs processing units (CPUs) to accomplish computations and In general, the compute nodes running the NFs as applications data movements. For instance, data may need to be moved (VMs and containers) can be deployed on platform installa- between input/output (I/O) devices, e.g., Network Interface tions ranging from large installations with high platform den- Cards (NICs), and system memory. However, the GPC plat- sities (e.g., cloud and data-centers) to distributed and singular forms, such as the Intel R x86–64 [32] and AMD R [33] platform installations, such as remote-gateways, clients, and CPU platforms, are not natively optimized to run NFs that mobile nodes. The cloud-native approach [12] is the most include routine packet processing procedures across the I/O