SONiC: Software for Open Networking in the Cloud

Lihua Yuan Azure Network Team for the SONiC Community

SONiC Community @ OCP Mar 2018 Application & Management Application Management &

SONiC [Software For Open Networking in the Cloud] Switch Platform Platform Switch

SAI [Switch Abstraction Interface] Silicon/ASIC Microsoft Cloud Network - 15 Billion Dollar Bet SONiC Is Powering Microsoft Cloud At Scale

Tier 3 - Regional Spine T3-1 T3-2 T3-3 T3-4

… … … Tier 2 - Spine T2-1-1 T2-1-2 T2-1-8 T2-4-1 T2-4-2 T2-4-4

… … … T1-1 T1-2 SONICT1-7 TSONIC1-8 T1-1 T1-2 T1-7 T1-8 Tier 1 – Row Leaf SONICT1-1 SONICT1-2 SONICT1-7 T1SONIC-8 SONIC SONIC SONIC SONIC SONIC SONIC

… … … Tier 0 - Rack SONICT0-1 SONICT0-2 SONICT0-20 SONICT0-1 SONICT0-2 T0SONIC-20 SONICT0-1 SONICT0-2 SONICT0-20 Servers Servers Servers Goals of SONiC

Faster Reduce Technology Operational Evolution Burden

Disaggregation with SONiC

Open & Choices of Vendors & Modular Platforms Software SONiC: Software for Open Networking in the Cloud

• Switch Abstraction Interface (SAI) • Cross-ASIC portability

• Modular Design with Switch State Service (SwSS) • Decoupling software components • Consistent application development model

• Containerization of SONiC • Serviceability • Cross-platform portability

• SONiC Operational Scenarios • Hitless upgrade • Network emulation with CrystalNet Switch Abstraction Interface (SAI)

NetworkNetwork ApplicationsApplications Simple, consistent, and Network Applications stable network application stack Hello

Switch Abstraction Interface Helps consume the underlying complex, heterogeneous hardware Hola Aloha easily and faster 你好 नम Bonjour مرحبا ते Switch Abstraction Interface

• CRUD operation over extensible Entity/Attribute/Value data model

• Reference data-plane behavior model supports various devices

• Significant feature/partner growth since announcement in 2015 • https://github.com/opencomputeproject/SAI SAI Function Calls & Pipelines in SONiC

• Register SAI APIs to achieve specific functions and control logging separately • Construct the right order to execute SAI function calls • Refer to SAI Pipelines to create the right packet action flow

create_router_interface create_next_hop create_router_interface get_switch_attribute get_switch_attribute create_route create_neighbor create_host_interface create_host_interface

Next Port RIF Route Neigh RIF Port Hop SONiC: Software for Open Networking in the Cloud

• Switch Abstraction Interface (SAI) • Cross-ASIC portability

• Modular Design with Switch State Service (SwSS) • Decoupling software components • Consistent application development model

• Containerization of SONiC • Serviceability • Cross-platform portability

• SONiC Operational Scenarios • Hitless upgrade • Network emulation with CrystalNet SONiC High Level Architecture

Linux Network Applications Object Library

Switch State Service

SAI

ASIC SDK User User Space

PAL - sysfs netdev ASIC PCI driver

Kernel HW Peripheral Network drivers device drivers

Various OCP Chassis Switch Hardware Supplier

ASIC Supplier Fan/Power/LED Transceiver ASIC

Linux Switch State Service (SwSS)

Network SAI DB: persist SAI objects Applications App DB: persist App objects

DB backend: redis with object library APP SyncD: sync SAI objects between software and DB hardware Orchestration Agent

Orchestration Agent: translation between apps and Backend Redis SAI objects, resolution of dependency and conflict SAI DB w/ Library Object

Key Goal: Evolve components independently SyncD

SAI

ASIC How SAI Objects are Stored in SAI Database

• APP DB: • Application oriented SAI DB • Human readable • SAI DB: • SAI oriented • Query-able and machine parse-able KEY SAI_OBJECT_TYPE:SAI_OBJECT_ID • Snapshot of the current SAI state SAI_OBJECT_ATTR VALUE SAI_OBJECT_ATTR VALUE • APIs: SAI_OBJECT_ATTR SAIVALUE DB • sai_create_api SAI_OBJECT_TYPE_NEXT_HOP:00000001 • sai_set_api SAI_NEXT_HOP_ATTR_IP 10.0.0.1 • sai_get_api* SAI_NEXT_HOP_ATTR_TYPE SAI_NEXT_HOP_IP • sai_remove_api *Used only when querying ASIC settings and configurations

13 How Routing Works in SONiC

BGP BGPd Zebra fpmsyncd Neighbor

APP DB

Orchestration Agent

SAI Redis SAI DB

socket

Host Intf SyncD netdev

ASIC SAI Route

Transceivers How LAG Works in SONiC

LACP teamD teamsyncd Neighbor

APP DB

Orchestration Agent socket

teamD SAI Redis SAI DB netdev Host Intf netdev SyncD

ASIC SAI lag

Transceivers SONiC Containerization

• TEAMD LLDP SNMP BGP • LLDP • BGP: Quagga or FRR DHCP TEAMD • SNMP: Net-SNMP + SNMP subagent

SYNCD SWSS Database • DHCP Relay: isc dhcp • Platform: sensors Utility Platform • SWSS: switch state service • DB: Redis • SONiC Base Image Syncd: sairedis + syncd agent SONiC Containerization

Container Strength SONiC Benefits • Clean isolation • Serviceability • Easy deployment • Extensibility • Transactional • Development agility • Run universal • Cross-platform SONiC Containerization • Quagga Components developed in Routing different environments

LLDP SNMP FRR • may not be DHCP TEAMD available GoBGP Databa SYNCD SWSS se Arista • Enables choices on a per- Utility Platform BGP component basis

SONiC Base Image Hitless Upgrade Through Docker

• Cloud users demand zero-down time, but SPINE SPINE …… SPINE 1 2 1 • Security patches, bug LEAF 6 Server: fixes need to be rolled SONiC SWARM out in hours 1 2 1 6 • New features need to be TOR TOR …… TOR rolled out in days

Before they fly the space shuttle …. They practice all activities and test all equipment extensively in a high-fidelity simulators CrystalNet: A high-fidelity, cloud-scale network emulator

SONiC Container Soft Switch

25 Open Invitation

• Inviting contributions in all areas

• New ideas on white/open network devices • New features, applications and tools • Download it, Test, Deploy!

Website: https://azure.github.io/SONiC/ Mailing list: [email protected] GitHub: https://github.com/Azure/SONiC Wiki: https://github.com/Azure/SONiC/wiki/