New IP and Market Opportunities

Richard Li Chief Scientist, Network Technologies Futurewei, USA

IEEE International Conference on High Performance Switching and Routing (HPSR) 2020 May 11-14, 2020

Futurewei Technologies, Inc. Acknowledgement and Disclaimer

• New IP is under research and development by research scientists and professional engineers across different countries and different organizations

• Some results of this talk may have appeared in ITU, IEEE and ACM publications

2 The has been very successful! But can it sustain all new applications in decades to come? Can we rest here and take the Internet as it is forever?

Paul Baran Frank Heart &Team, Queen Elizabeth II TCP/IP ARPANET Sends her first email. Inventor Packet Switching BBN IMP Spec Standardized ceased 1961 1968 1976 1980 2000 Conceptual Experimental Standardization Explooooosion

1965 1974 1979 1989 Cyclades at INRIA, IEEE, "A Protocol for Packet Network WWW, CERN Inventor Packet Switching France, 1971-1979, Intercommunication” , 1974 Tim Berners-Lee Don Davis, NPL, UK Vinton Cerf and

Packet Switching Landmark Projects Commercialization Page 3 Agenda

• Market Opportunities and New Requirements ▪ Need for high-precision communications for industrial machine-type communications ▪ Need for free choice addressing in the age of ManyNets for economy and democracy ▪ Need for qualitative communications for very large volumetric communications • Current IP ▪ Design decisions: e.g. Fixed Addressing, Inherently Best Effort ▪ Packet Loss, Retransmission, and Host-based Congestion Control ▪ Cerf-Kahn-Mathis Equation • New IP ▪ Free-Choice Addressing ▪ High-Precision Communications ▪ Qualitative Communications

4 ITU-T Focus Group on Network 2030

ITU General Secretariat Radiocommunication Standardization Development ITU Telecom Members’ Zone Join ITU

About ITU-T Study Groups Events All Groups Join ITU-T Standards Resources Regional Presence BSG

Identify future use Study new capabilities Explore new concepts, Review Protocol Stack, cases and new of networks for the year principles, mechanisms, and outline requirements 2030 and beyond and architectures future directions

https://www.itu.int/en/ITU-T/focusgroups/net2030/Pages/default.aspx

nd 2 Meeting rd th Establish 1st Meeting 3 Meeting 4 Meeting 5th Meeting 6th Meeting December 18 – 21, 2018 July 16 – 27, 2018 October 2 – 4, 2018 February 18-20, 2019 May 21-23, 2019 October 14-18, 2019 January 13-15, 2020 Hong Kong Geneva New York London St. Petersburg Geneva Lisbon

Page 5 Driverless Vehicles and Remote Operations

Hazardous environment

6 Latency and Packet Loss vs Safety of Life

30 km/hour = 8.3m/sec. distance = 8.4m/sec x 60ms = 0.5m 60 km/hour = 16.7m/sec. distance = 16.7m/sec x 60ms = 1m Packet Loss is a Serious Issue

Collision-Avoidance Distance

Autonomous Driving Cloud Driving Local Sensory Input

Local Latency Local Driving Sensory Input Remote Remote Driving Latency Decision Point Contextual Advisory Packet Loss Decision Point Packet Loss

Page 7 Industrial Machine-Type Communications and Control

Cloudified PLC

Major Challenges:

▪ Latency ▪ Availability ▪ Elasticity ▪ Security ▪ Usability

Source: Texas Instruments

Page 8 ManyNets: Embracing Diversity, Variety, Economy, and Autonomy

OneWeb

Spread Networks

Starlink

Non-IP Networks Private Global Backbones Emerging Satellite Constellations (Global Broadband connectivity for 4 billion people (Growing market segment) (No Need for Internet Transit) who are not connected to any network today)

1) Geoff Huston, The Death of Transit and the Future Internet, Keynote Speech at 2nd ITU-T Workshop on Network 2030, Hong Kong, Dec. 2018 2) Mostafa Ammar, Service-Infrastructure Cycle, Ossification, and the Fragmentation of the Internet, Keynote Speech at 3rd ITU-T Workshop on Network 2030, London, UK, Feb. 2019 Page 9 Space Internet How do we integrate terrestrial and space internet?

Routing in space Co. Support Scale Starlink SpaceX, Google 4K by 2019, then 12K LEOs or MEOs Oneweb Blue Origin (Bezos), Virgin 650 by 2019 Orbit LT Boeing Apple (spec) 2956, 1350 in 6 yrs E Laser Groun d O3Nb Virgin group, SES 400 Statio n CASIC China 300 (54 trial)

Distances Bandwidth delay (LEO) 1—200 Gbps 35ms 900-1200 KM Public Transit Backbone (MEO) 1-200 Gbps ~60ms ~2000 KM Private Transit Backbone Space to space ~100 KM – ~Tbps ~1000 KM ~10 Gbps Private Transit Backbone

Source: Internet

Page 10 We are in the dawn of a big change from digital society to holographic society

Healthcare Non-contact Agriculture Haptic Virtual / Surgery Force Remote Touch Movement Technology feedback

Shopping Education Holographic Tactile Remote Fast Response High Precision Internet Control Society

Entertain Industry ment Digital Senses Sight Hearing Touch Smell Taste Tourism

11 Holograms and Holographic Type Communications How do we transport very large volumetric data? Motion-to-Photon Time: Total 20 ms 20” wide

Image Framing Encoding Capture Streaming 4” 5-7 ms 4” Network Display Decoding Transport /VR

Throughput 6’0” tall 6’0”

4K/8K HD band AR/VR band Hologram Dimensions Raw Data width width 35Mbps~140Mbps 25Mbps~20Gbps 10 Gbps~10 Tbps

Tile 4 x 4 inches 30 Gbps Synchronization of parallel streams

(reference: 3D Holographic Display and Its Data Transmission Requirement, 4K/8K HD VR/AR Hologram 10.1109/IPOC.2011.6122872), derived from for streams streams ‘Holographic three-dimensional telepresence’; N. Peyghambarian, University of Arizona) Human 72 x 20 inch 4.32 Tbps ~thousands Raw data; no optimization or compression. color, FP Audio/Video(2) Multiple tiles (12) (full parallax), 30 fps (view-angles) 360 degrees of view 6 degrees of freedom

Page 12 Attaching Digital Senses to Holograms

IEEE Digital Senses Initiative Media Evolution Coverage Model

D Transforming Industries Hologram 1T/s 1ms and more… Consumer Healthcare AR/VR Virtual Privacy Reality Augmented Reality D Video Human 1G/s 17ms Augmentation Identity Smart Robots Wearables Public Awareness Audio User Acceptance Security Sight Content Richness Image 100M/s 33ms Hearing Touch Ecosystem Readiness

Trust Human or Text Machine Respond Undersized Smell Taste Ethics Synthesize 64k/s 50ms Other Senses Reproduce Capture

Page 13 IP/MPLS in Mobile Backhaul Networks How do we provide end-to-end service guarantee for 5G/B5G/6G-enabled applications?

Inefficient use of protocols No guarantee on E2E throughput and latency by current TCP/IP Tunnels over tunnels Repeating header fields

App (user) App (user) App (user) App (user) App (user)

TCP (user) TCP (user) TCP (user) TCP (user) TCP (user)

IP (user) IP (user) IP (user) IP (user) IP (user)

Inefficient PDCP PDCP GTP-U (S1) GTP-U (S1) retransmission RLC RLC UDP (Nwk) UDP (Nwk) Not suitable for mMTC and uRLLC Radio retransmissions are not synchronized MAC MAC IP (Nwk) IP (Nwk) Low efficient user payload, unsuitable for mMTC and short messages with TCP flow control IP/MPLS IP/MPLS PHY PHY Retransmit wasteful Backhaul Backhaul No E2E QoS, unsuitable for uRLLC packets Eth/Nwk Eth/Nwk

Cellular network Fixed, IP based wireline network

Page 14 Market Opportunity and New Requirements

▪ Industrial Machine-type Communications ▪ In-Time Guaranteed Transport High-Precision ▪ ▪ Industrial Control and Manufacturing Communications On-Time Guaranteed Transport ▪ Industrial Internet ▪ Lossless Transport ▪ Tactile Internet ▪ Autonomous Driving ▪ Mix and Match ▪ ▪ Cloud Driving Free-Choice Industrial Machine-Type Addressing ▪ Digital Twin Addressing ▪ Machine ID located in a Workshop ▪ Power-Efficient Addressing ▪ Holographic Twin ▪ ManyNets ▪ Holographic Society ▪ Network and Computing Convergence ▪ Entropy-Based Communications ▪ Smart City Qualitative Communications ▪ Semantics-Based Communications ▪ Smart Agriculture ▪ Knowledge-Based Communications ▪ Smart healthcare ▪ ManyNets ▪ Premium services ▪ Space-Terrestrial Integrated Network ▪ Adaptable Responsive Moving Beyond ▪ User Programmable ▪ Private Internet Native Multiplexing ▪ Non-IP Networks ▪ Business Easing ▪ Compute Power Network

15 Agenda

• Market Drivers and New Requirements ▪ Need for high-precision communications for industrial machine-type communications ▪ Need for free choice addressing in the age of ManyNets for economy and democracy ▪ Need for qualitative communications for very large volumetric communications • Current IP ▪ Design decisions: e.g. Fixed Addressing, Inherently Best Effort ▪ Packet Loss, Retransmission, and Host-based Congestion Control ▪ Cerf-Kahn-Mathis Equation • New IP ▪ Free-Choice Addressing ▪ High-Precision Communications ▪ Qualitative Communications ▪ Better Traffic Engineering

16 and Packet Statistical Multiplexing Maximal Resource Utilization Inherently Best Effort Forwarding

Ingress Card Switch Fabric Egress Card Burst

MAC NP TM/FIC Serdes IO Serdes Serdes IO Serdes Core

FIB

DDR接口

DDR FIB

Transmission Header Processing Delay Queueing Delay Delay

Packets are dropped when Packets are dropped when buffer is full buffer is full

17 IP is an artifact (made up of a few design decisions)

Statistical Multiplexing One Size Fits All Capabilities and Services (Fixed Addressing)

Best Effort. Default and Most Popular

DiffServ (on a per-hop basis)

Traffic Engineering ▪ Traffic Steering (Explicit Path) ▪ Minimal Bandwidth Guarantee ▪ Fast Re-Route

Maximize network utilization: Matching One common network layer to traffic demand to available capacity connect everything globally

Packet switching: 59 years (2020-1961) TCP/IP (Cerf’s paper): 46 years (2020-1974) IPv4 (RFC 791): 39 years (2020-1981) IPv6 (RFC 1883): 25 years (2020-1995) MPLS (RFC 3031): 19 years (2020-2001) As of now, total RFCs: 8778 (If an engineer studies 5RFCs a week, it takes 33.76 years to finish all them) Futurewei Technologies, Inc. Page 18 Transport of Traffic: Packet Loss and Retransmission

Packet Loss

Transmission Retransmission Congestion Control

퐖퐢퐧퐝퐨퐰퐒퐢퐳퐞 퐌퐒퐒 퐂 퐓퐡퐫퐨퐮퐠퐡퐩퐮퐭 ≤ 퐦퐢퐧(퐁퐖, , × ) 퐑퐓퐓 퐑퐓퐓 훒

Page 19 Transport of Traffic: Improvements and Enhancements

Flow LB

Load Balancing ECMP Packet LB

Rerouting Flowlet LB In Network Priority Queuing Rate Limiting

Back Pressure Multi-Path Congestion In Host Control Pacing

See ACM SIGCOMM 2019 Keynote (Mark Handley) for his personal achievements Scheduling

It is an eternal and never-stopping topic in ACM Sigcomm

Google Scholar returns 13,200 publications on “TCP Congestion Control”

No one size fits all Time to read: 36 years Read one paper a day: 13200/365

Page 20 Transport of Traffic: Extrapolation from Cerf-Kahn-Mathis Equation

TCP Throughput (Mbps) drops as Packet Loss Rate Ultra-low Latency (us) demands as Packet Loss Ratio increases, Guaranteed RTT= 5ms increases, Guaranteed Throughput = 12Gbps

800.0 738.7 (Mbps) 350.00 307.80 (us) 700.0 300.00 600.0 522.3 250.00 217.64 500.0 400.0 200.00 300.0 233.6 150.00

RTT (us) RTT 97.33 165.2 200.0 100.00 68.83 73.9 52.2 30.78

100.0 23.4 (Mbps) 50.00 21.76 9.73 (us) TCP Throughput (Mbps) Throughput TCP 0.0 0.00 0.001% 0.002% 0.010% 0.020% 0.100% 0.200% 1.000% 0.001% 0.002% 0.010% 0.020% 0.100% 0.200% 1.000% Packet Loss Ratio Packet Loss Ratio

If you lose 1 packet per 10,000 packets, your latency is 0.1 ms in order to yield a throughput of 12 Gbps

The result may vary with CPUs, Links, Buffers, etc, but throughput, latency and packet loss are coupled closely together

You can’t make omelet without breaking eggs!

• improvements exist, but the nature of the correlation between throughput, packet loss and latency keeps similar.

Page 21 Agenda

• Market Drivers and New Requirements ▪ Need for high-precision communications for industrial machine-type communications ▪ Need for free choice addressing in the age of ManyNets for economy and democracy ▪ Need for qualitative communications for very large volumetric communications • Current IP ▪ Design decisions: e.g. Fixed Addressing, Inherently Best Effort ▪ Packet Loss, Retransmission, and Host-based Congestion Control ▪ Cerf-Kahn-Mathis Equation • New IP ▪ Free-Choice Addressing ▪ High-Precision Communications ▪ Qualitative Communications ▪ Better Traffic Engineering

22 What are new in New IP?

New IP

▪ High-Precision Communications ▪ In-Time Guaranteed Transport ▪ On-Time Guaranteed Transport ▪ Lossless Networking IPv4 and IPv6 ▪ Free-Choice Addressing ▪ Fixed Addressing ▪ Mix-and-Match ▪ Industrial Machine-Type Addressing ▪ Best-Effort ▪ Power-Efficient Addressing ▪ Domain-Specific Addressing in ManyNets ▪ DiffServ ▪ Qualitative Communications ▪ Entropy-Based Communications ▪ Traffic Engineering ▪ Semantics-Based Communications ▪ Knowledge-Based Communications ▪ Quantitative Communications ▪ Inherited and backward compatible ▪ Best-Effort ▪ DiffServ ▪ Traffic Engineering ▪ Quantitative Communications

Page 23 What Can We Learn from Postal Services?

IP datagram used to be called “letter-gram”, and it enjoys many analogies to postal letters. Today’s postal services are no longer the postal services of 50 years ago. Postal services have greatly evolved, but IP hasn’t!

Customize Delivery Time

Deliver to Another Address

Hold at FedEx Location

Sign for a Package

Provide Delivery Instructions

Sign for a Package

Customizable Trackable Assurable Billable Programmability Measurement and Telemetry Guaranteed New Source of Revenue

Page 24 Imagine a New IP Packet as a FedEx-like Datagram

IP Header Contract User Payload

A packet carries a contract between an application and the network. The network and routers fulfill the contract.

FedEx-like IP Datagram FedEx Package 1 The packet arrives in 35ms 1 The package arrives in 1 day

2 The packet arrives at 35ms sharp, no sooner no later 2 The package arrives at 9:00am next day

3 It requires a throughput of 12 Gbps 3 The weight is 12kg

4 No packet loss. If lost, you get a compensation. 4 No package loss. If lost, you get a refund of $$$.

5 Track it 5 Status track

Ref: Richard Li, et al, A New Framework and Protocol for Future Networking Applications, ACM Sigcomm 2018 NEAT Workshop, Budapest, Hungary, August 2018

Page 25 What Can a Contract Do in New IP?

High-Precision User Network In-Band High-Precision User-Defined Communications Interface (UNI) Signaling Telemetry Networking

❖ Lossless networking Application-Specific ❖ Throughout guarantee Programmability ❖ Latency Guarantee Preferred Path Routing (in-time, on-time, . . . (PPR) coordinated)

Page 26 E Pluribus Unum: Mix-and-Match, Flexible Addressing System

Raw New IP 3 New IP 5 1 Data LAN Backbone Network

4 Raw Data 2 DIP:D (128 bit) GW3

IoT SIP: x (24 bit) Prefix P, 104 bits Server Device DIP: D (128 bit) #1 SIP: P + x (128 bit)

DIP: y (24 bit) SIP: x (24 bit)

2 IoT Device #2

IPv4 host directly talks with OAM from IPv4 hosts Satellites of different IPv6 server, and vice versa to non-IPv4 nodes companies talk with each other

Comparison In current IP, the source and destination addresses are of the same type, but cannot be mixed

Page 27 Moving Satellites Topology-based & Geography-based Addressing

(12, 10) (14, 10) (18, 10)

New IP Packet Header (12, 8) (16, 8) IP ADDR 2:2::2 Geo ADDR (20, 10) Geo IP Prefix payload (10, 10) Address (20, 10) 3:3::3/32 (30, 15) 2:2::2/32 (20, 10)

2:2::2/32

New IP Packet Header AS 1 AS 2 AS 3 New IP Packet Header IP ADDR 2:2::2 IP ADDR 2:2::2 payload payload Sender Receiver

Page 28 Current IP: Quantitative Communications

Syntax Sender Receiver What is received = What is sent

Every bit and byte has the same significance to routers/switches

Packet Packet Good for • File/Document Transfer • Banking, Shopping

Overkill for some applications • Holograms • Disaster Environment Packet Corrupted Packet

Page 29 New IP: Qualitative Communications

Semantics Sender Receiver What is received = What is maximally meant

In payload, bits and bytes are not equally Noisy Link significant. Instead, they are differential in their entropies

Congested Node Less significant bits and bytes may be dropped

Partial or degraded, yet useful, packets may be Qualitative Packet repaired and recovered before being rendered Congested Node Good for • Large volume of image-like data

Congested Node • Holographic type communications • Media with digital senses Ref: A Framework for Qualitative Communications using Big Packet Protocol, ACM Sigcomm 2019 NEAT Workshop, Beijing, August 19, 2019. Available at: https://dl.acm.org/citation.cfm?id=3342201 • Disaster Environment

Page 30 Progression and Evolution

IPv4/IPv6 Header User Payload

Enhancement with “Contract”

Header Contract User Payload

BBE-Beyond Best Effort Header Evolution HPC- High Precision Communications VLV&TIC Payload Evolution User-Defined Networking Better Traffic Engineering

New IP Shipping Directive Contract User Payload

ManyNets VLV&TIC Free-Choice Addressing Qualitative Communications

31 Deployment under Consideration

❖ New IP can be deployed within an autonomous system (AS) in the existing framework.

interconnected ❖ New IP is especially good for industrial machine- type communications that often require high- precision KPI, ultra-low latency, and whose devices may have addresses other than IPv4/IPv6.

❖ Examples of New IP Trial Deployment: Autonomous System Autonomous System Autonomous System ❖ Metropolitan Networks IPv4/IPv6 IPv4/IPv6 New IP ❖ Mobile Backhaul for 5G/B5G/6G ❖ Industrial Manufacturing Facility Existing Some Industrial Use ❖ Industrial Park and Campus (Stringent KPI) and Industrial Use ❖ Autonomous Driving and Remote Cloud Driving Most Networks (Non-Stringent KPI) + + Consumer Use Consumer Use

Page 32 Concluding Remarks ▪ While the existing IP is pretty much like classical postal letters, New IP is more like modern FedEx-style packages.

▪ New IP is motivated by • Industrial machine-type communications (Industry 4.0, Industrial Internet, Smart World) • Mobile Backhaul Transport for 5G/B5G/6G • Emerging Industry Verticals (driverless vehicles) • ITU-T Network 2030

▪ Research and empirical results have been published in ITU, IEEE, and ACM. Some industrial manufacturing-related companies, service providers and network operators have shown their interest in New IP

▪ Some prototypes have been implemented by different organizations in different countries. However, Standards Developing Organizations (SDO) have yet to standardize it.

▪ New IP has been, unfortunately, politicized in media, and misinformation about it has been spread 33 Thank You!

34