The Network as a Programmable Platform Realizing Verifiable Closed-Loop Control in Networks

Larry Peterson Open Networking Foundation Princeton University

Co-Conspirators: Nate Foster, Nick McKeown, Jen Rexford, Guru Parulkar and Oguz Sunay. THIS TALK

Network as a Platform • Where Software-Defined Networking (SDN) is taking us What that means for networking in general • And the in particular An opportunity to get involved • Aether: An Open 5G-Enabled Edge Cloud for Enterprises SDN JOURNEY (SO FAR)

Server running control plane

Compute Network mainframes mainframes Whitebox switch

Closed & proprietary Hard to manage Merchant silicon Easier to fix Bloated & complex Hard to secure Open source Easier to innovate Little incentive to improve Hard to change Cloud practices Simpler to operate

2005 2020 SDN JOURNEY (IN PROGRESS)

P4 Program

compiler

Match+Action

Mem ALU Actions Actions Actions Hdr Hdr Hdr ACL Actions L2Table ACLTable v4 v4 IPv4 Table IPv4 Table IPv6 v6 v6 L2 L2

Fixed Programmable Programmable Match-Action Parser Fixed Header Processing Pipeline Parser Pipeline

Fixed function switches and NICs Programmable switches and smartNICs What does this all mean for networks? PROGRAMMABLE END-TO-END & TOP-TO-BOTTOM BGP Load Traffic Traffic Overlay

Balancing … Engineering Virtualization

SDN Controller

DPDK DPDK User space User space

Kernel stack NIC NIC Kernel stack XDP/eBPF XDP/eBPF PROGRAMMABLE END-TO-END & TOP-TO-BOTTOM Specified Behavior BGP

CompileLoad Traffic Traffic Overlay

Balancing … Engineering Virtualization

SDN Controller Partition

Compile Compile Compile

DPDK DPDK User space User space

Kernel stack NIC NIC Kernel stack XDP/eBPF XDP/eBPF VERIFIABLE CLOSED-LOOP CONTROL

High-Level Specificatio n BGP Load Traffic Overlay Balancing Engineering Virtualization …

SDN Controller Verifier

NIC NIC

Fine-Grained Network Telemetry PROGRAMMABLE END-TO-END & TOP-TO-BOTTOM

Networks, for the first time, will be: This creates new possibilities: • Programmable end-to-end • Network is correct-by-construction BGP Load Traffic Traffic • Specified top-to-bottom Overlay • Verify against network specification Balancing … Engineering • Defined entirely by software Virtualization • Correct through closed-loop control

SDN Controller

DPDK DPDK User space User space

Kernel stack NIC NIC Kernel stack XDP/eBPF XDP/eBPF PROGRAMMABLE END-TO-END & TOP-TO-BOTTOM Link BGP Slice Load Load Traffic Traffic Control Overlay Balancing Handover Balancing … Engineering Aggregation Management Virtualization

SDN Controller

DPDK DPDK User space User space

Kernel stack NIC NIC Kernel stack XDP/eBPF 5G Mobile XDP/eBPF Network ? 5G TRANSFORMATION ßAccess Cloudà 4G eNodeB EPC: Evolved Packet Core

(Closed, Telephony-Based) (Proprietary, Bundled) Open Source Microservice-based

NG-Core 5G Radio Unit Distributed Unit Central Unit PROGRAMMABLE END-TO-END & TOP-TO-BOTTOM

Management Plane Policy

>> 1s Control Loop (Non-RT) Link Slice Load Control Balancing > 10ms Control Loop (Near-RT) Handover … Aggregation Management < 1ms Control Loop (RT) SDN Controller SDN Controller (RIC)

NG-Core

Radio Unit Distributed Unit Central Unit UPF DEEP PROGRAMMABILITY

…Control Plane… Software

Hardware UPF

PFCP Agent Server HQoS & Buffering vSwitch

NIC smartNIC

GTP-U Termination

Programmable Switch SW Path HW Path Radio Unit ONOS-BASED RIC Link Link Load Balancing Load Interference Handover Aggregation Control Service Mgmt Control Management Applications … and Orchestration Handover Performance (200-300 per-second)

C1 Server A1 Server Latency Number of UEs Northbound I/O Controller 10,000 25,000 100,000

ONOS RIC: Average 0.8 ms 1.3 ms 1.5 ms RAN Control Telemetry Topology K/V TSDB 90% 1.3 ms 1.5 ms 1.5 ms Intelligent Service Service Service Controller 95% 1.9 ms 2.5 ms 1.9 ms

Southbound I/O Controller 99% 4.6 ms 8.3 ms 4.4 ms

E2 Client E2 Client E2 Client E2 Client

Compute Resources: 1x 3x RAN Elements Managed 4G/5G Edge Cloud for Enterprises (an ONF Project) AETHER OVERVIEW

IoT

Sensors Small Cell RAN Mobile Core Central Unit User Plane Enterprise Surveillance Control Portal

Control and Edge Apps Management Platform Mobile Core Central Apps Small Cell Multimedia Control Plane Enterprise Edge Central Cloud

Employees Distributed Mobile Core User Plane provides local breakout at all remote Edge sites Visitors Small Cell AETHER COMPONENTS

xApps Trellis UPF Verify

RIC ONOS TSDB

Aether Connectivity fabric.p4 fabric.p4 Control (ACC) Stratum Stratum INT INT Aether Management Tofino Tofino Platform (AMP)

fabric.p4 fabric.p4 Stratum Stratum Central Cloud INT INT Tofino Tofino

RAN Aether Connected Edge (ACE) AETHER HAS BEEN OPERATIONAL SINCE DECEMBER ’19

AMP, ACC

ACE MORE INFORMATION

Deep Programmability • ACM SIGCOMM Computer Communication Review, Oct 2020 (to appear) Aether • https://www.opennetworking.org/aether/ SD-RAN • https://www.opennetworking.org/sd-ran/ Background Tutorials • https://5G.systemsapproach.org • https://SDN.systemsapproach.org