#CLUS Introduction to the Next Generation IOS-XR architecture Bhavna Prasad, Product Manager ([email protected]) Patrick Warichet, Technical Marketing Engineer ([email protected]) BRKSPG-2069

#CLUS Agenda • Introduction

• Software Architecture Overview

• Flexible Packaging

• Application Hosting

• Configuration, Monitoring and Troubleshooting

• Examples

• Conclusions

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 3 Agenda 2

• Introduction

• Software Architecture Overview

• Flexible Packaging

• Application Hosting

• Configuration, Monitoring and Troubleshooting

• Examples

• Conclusions

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 Cisco Webex Teams

Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How 1 Find this session in the Cisco Events App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKSPG-2069 by the speaker until June 18, 2018.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5 Introduction An Evolved Network Operating System for a Rapidly Changing Web and SP Landscape

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 Key Learning Objectives

• At the end of the session, you should be able to:

• Appreciate rapidly evolving network infrastructure landscape

• Understand how Cisco is evolving IOS-XR network operating systems to meet customer needs

• Identify technical principles of Evolved IOS-XR

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 Digitization Drives Data and Infrastructure to the Edge

INCREASING DIGITIZATION A shift in the network model must 2014—3.4 ZB 2019—10.4 ZB occur

60% 31% 37% Inter-DC Most Data Will Cloud Workload Traffic Be Processed Increase Growth at the Edge

Source: Cisco Consulting Services Global IoT Study, 2014; 1230 respondents Cisco Global Cloud Index Forecast, 2014–2019 #CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 Automating Provisioning Down to Minutes

Traditional Cloud Scale Day 0 Day 1 Day 2 Network Operations Network Operations

INSTALL CONFIGURE Plug in new device Plug in new device

Operating System Automated Boot Network Operations Center Upgrade (Manual)

Manual staging CLI Script Operating Configuration System Configure new device remotely Auto-provisioning (Manual)

Traditional Cloud Scale Device up and running Device up and running Networking Operations Networking Operations • Manual device onboarding • Reduced time to install and provision new devices • Lengthy installation and 4 Hours to 3 Days 10 to 15 Minutes provisioning procedures Operating • Faster disaster recovery System • Automated device onboarding

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 Traditional NetOps Upended by Evolved Needs of SP Customers

Traditional Network Evolved SP/Cloud Scale DevOps Operations Network Operations

Day 0

Manual INSTALL Automation Provisioning • Automated services • Simple to scale

Day 1 • Agile, open software SW Inflexible Modularity SW & CONFIGURE Extensibility Fragmented Topology Visibility & Day 2 View and Control Complex Routing MANAGE & OPTIMIZE * Source: Google

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 Cisco Evolved IOS XR Innovations Designed for Operational Fit

Software Modularity Automation Visibility and Control and Extensibility Data Third- Service Agility Model party Simplification Driven Agents Telemetry

Asynchronous Upgrade APIs Support Visibility

Granular Packaging Modularity Better Customer Open Innovation Experience Operational Efficiency Docker Application Engineered Application Automated Boot and Routing Hosting Auto-provisioning Control Containers

Extensibility Visibility and Control

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 Evolved IOS-XR Enablement across Platforms

NCS5501/SE

NCS5504 NCS5502/SE

NCS5001 NCS6000

NCS5002 NCS1002 with MACSec NCS-55A1-24H

NCS5011 NCS5508 NCS5516 NCS-55A1-36H-S ASR9000 64-bit NCS-55A2-MOD NCS1002 Now Shipping 6.5.1

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 Future: Flexible Architecture for all Deployments

Elastic Cost Optimized Ultra-high Density Critical Infrastructure Programmable and Automatable

Virtual Network Function Cisco HW with Merchant Si Cisco HW with Custom Si

Fixed and modular Fixed, modular, and multi- x86 data plane hardware, merchant data chassis hardware; Cisco plane data plane

Common Operating System Across Physical and Virtual Data Planes

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 Pillars of Evolved IOS-XR

Visibility & • Operational Data, Deep analytical hooks Telemetry • Model-based, flexible, Push Model

Evolved • Data accessible via published model driven interfaces (Openconfig & Netconf) • Machine friendly Programmability • Enables automation @ scale

Application • Ability to run 3rd party off the shelf applications built with Linux tool chains Hosting • Run custom applications inside a separate LXC container

• Packages can be inspected on box using standard tool chain (RPM tools) Flexible Platform • Automated installation with dependency checking and Packaging • Open Bootloaders (iPXE) and end-to-end auto-provision • 64-bit Linux

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 Software Architecture IOS-XR 6.X: A New Software Infrastructure

• 64-bit OpenEmbedded Linux support. Classic XR XR 6.X • Processes containerization.

• Standard Linux toolchain. System System Control Admin • Third-Party applications. Control Admin

• NCS5500, NCS5000 and 32 bit QNX NCS1002 support 64-bit Linux. 64 bit Linux NPU X86 Hardware • ASR 9000 supports 64-bit Linux starting with 6.1.1.

• Will still have 32-bit QNX support.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 Introduction to Virtualization

• With eXR, the Admin plane and the Control Plane run in isolated environment. NCS5000-5500 ASR9K • ASR9K uses Virtual Machines (VMs) App App • NCS5000 and NCS5500 use containers (LXCs) App App … Lib Lib … • Provide the same functionalities. Lib Lib OS OS • Use same tools (libvirsh) to Provide the virtualization services Operating System Hypervisor

• LXCs are lightweight no ISSU support. X86 Hardware Operating System X86 Hardware • VMs are heavier but provide ISSU support.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 Why Virtualization ?

• LXCs and VMs Provide HW abstraction Each instance has its • own network interfaces. • - Can be bridged, routed... • own filesystem • - e.g.: RedHat host can run container or VM.

• LXCs and VMs provide Isolation between instances: • Security. • - Two containers or VMs can't harm (or even see) each other. • Resource usage. • - Soft & Hard quotas for RAM, CPU, I/O can be assigned on Containers or Vms.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 IOS-XR Entities Roles: The Host

• Runs Yocto based 64-bit Open Embedded . • Built using Windriver 7. • The main functions of the host are: • Interact directly with the underlying hardware. • Provide kernel services for the LXCs/VMs • Provide libraries, tools, and utilities to help Control Admin Third launch, monitor, and maintain containers. Plane Plane Party • Provide the network infrastructure to allow containers to communicate. 64-bit Host OS

Routing Processor

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 Container Architecture NCS 5500 and NCS 5000 Runs processes responsible to perform Cisco developed Runs any 64-bit Linux system diags, monitor packages for core distribution. env. variables, and network functions Launched from the XR manage hardware (BGP, MPLS, etc.) container using virsh components and libvirtd. Yocto packages for First container to be standard Linux tools Access Network Control Admin booted by the host, Third and libraries Interface through the Plane Plane responsible for the Party (bash, python, Global VRF Network start and maintenance tcpdump, etc.). Name Space. of the Control Plane container

64-bit Host OS

Routing Processor

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Container Architecture (ASR9K)

VM LXC Runs any 64-bit VM Runs processes Cisco developed . responsible to perform packages for core Launched from the system diags, monitor env. network functions XR virtual machine variables, and manage (BGP, MPLS, etc.) using virsh and hardware components Admin Control Third libvirtd. Yocto packages Plane Party Plane First VM to be booted by for standard Linux Access Network the Hypervisor, responsible tools and libraries Interface through the for the start and (bash, python, Global VRF Network maintenance of the Control tcpdump, etc.). Name Space. Plane VM

Hypervisor 64-bit Host OS Routing Processor

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 IOS-XR 6.X Boot Process - iPXE

• All NCS routers are equipped with a UEFI 64-bit Firmware (aka BIOS).

• Supports initial booting from USB drive or iPXE.

• iPXE is an open source boot firmware.

• Fully backward compatible with PXE with several enhancements. • Boot from a web server via HTTP. • Control the boot process with scripts and menus. • DNS support.

• iPXE is supported on the management interfaces.

• Supports both IPv4 and IPv6.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 26 IOS-XR 6.X Image Format and Boot Process • IOS-XR 6.X images are released in the form of bootable self-extracting ISOs. • Similar to any Linux distribution. • Include a signature for integrity verification.

• IOS-XR 6.X comes with a Zero Touch Provisioning (ZTP) process. • Executed at the end of the control-plane boot sequence. • Executed inside a Shell environment • Can be Shell (e.g.: bash) or Python scripts (starting with IOS-XR 6.2.5) • Can be initiated from the management or any IOS-XR interface (starting with 6.5.1) • Can execute scripts or apply static configuration.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 DHCP SERVER

IOS-XR Boot Process with iPXE IP address Next-server iPXE Filename=http:///new-img.ISO N boot 1 Y IP address Next-server 2 iPXE Filename=http:///script-SN.sh or Filename=http:///config-SN.txt XR Install HTTP SERVER XR Boot 3 Image-new.ISO 4 GET script-SN.sh or config-SN.txt AutoProvision script-SN.sh execution config-SN.txt

5 Additional Apply GET addon scripts/packages/configuration Scripts Execute script configuration Packages, etc… #CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 ZTP Flow of Operation (6.5.1)

ZTP Start

Mgmt Y Start ZTP on link up ? mgmt. port wait for N 120 sec timeout Start ZTP on N DHCP N data ports Offer ?

DHCP Y Reverse N Offer N wait for Reverse HTTP configuration with N configuration 120 sec get 43/17 timeout success N Y Y N N HTTP Exit 0 ? get Proceed with Exit 0 ? success mgmt. ZTP Y Y Y Infinite Loop toggling ZTP ZTP Proceed with between mgmt. and data port end end Data Port ZTP #CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 Single DHCP Server Configuration host NCS5500-rp0 {

hardware ethernet e4:c7:22:be:10:ba; DHCP fixed-address 192.168.0.10; option 77 If exists user-class and option user-class = "iPXE" { DHCP # Image request, provide ISO option 67 filename "http://192.168.0.10/images/ncs5500-mini-x-6.3.2.iso"; } elsif exists user-class and option user-class = "exr-config" { # Auto-provision request, provide script or configuration DHCP option 77 filename "http://192.168.0.10/scripts/ncs5500-rp0.sh"; } DHCP } option 67

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 Flexible Packaging Networking Software Delivery Direction

• Server-like Workflows

• Modular Disaggregated

• Update, Upgrade, Remove • RPM – RPM

• Dependency management • – Yellowdog Updater Modified • Secure • New Install Commands • Repository Support

• Third Party Applications

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 RPM: IOS-XR New Package Format

• RPM Package Manager is the new Package format starting with IOS-XR 6.0.

• Packages are placed in a reachable repository and accessed via FTP/SFTP/SCP/TFTP or HTTP or pre-staged on the box

• Third Party packages are installed with RPM or YUM inside the Shell.

• IOS-XR packages are installed with “install update/upgrade”.

• Install commands are a wrapper around YUM to provide multi-arch support.

• Both YUM and install commands provide dependency verification/resolution.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33 Anatomy of RPM Packages

RPM

Archive binary archive

Describe package contents MetaData Install structure /var/lib/rpm Dependencies

Scriptlet Pre and Post install instructions Database of installed packages

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34 XR Packages Naming Convention

Package --..rpm

ncs5500-mgbl-4.0.0.0-r632.x86_64.rpm

Major.Minor.Build.SMU SMU --...rpm

ncs5500-mgbl-4.0.0.1-r632.CSCab12345.x86_64.rpm

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 Package Repository www.cisco.com Local Repository

Mini ISO Platform + Pkgs 6.3.2 Release Mini ISO ncs5500-mini-x-6.3.2.iso + k9 pkg ncs5500-mpls-te-rsvp-2.2.0.0-r632.x86_64.rpm + XR Software Pkgs ncs5500-isis-1.3.0.0-r632.x86_64.rpm

Full K9 ISO ncs5500-mcast-2.1.0.0-r632.x86_64.rpm ncs5500-k9sec-4.1.0.0-r632.x86_64.rpm Full ISO ncs5500-mgbl-4.0.0.0-r632.x86_64.rpm

XR SMUs DDTS SMU ncs5500-mpls-2.1.0.0-r632.x86_64.rpm ncs5500-ospf-2.0.0.0-r632.x86_64.rpm ncs5500-li-1.0.0.0-r632.x86_64.rpm Svc Pak DDTS SMUs ncs-5500-mgbl-4.0.0.1-r632.CSCab12345.x86_64.rpm

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36 ISO Customization : Golden ISO iPXE / ZTP Golden ISO

gisobuild.py Open ISO Boot ISO Install Install PKG/SMUs PKG/SMUs Apply Configuration Reboot iPXE Apply USB Configuration System Upgrade

Golden ISO #CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37 Updating XR Packages

Command Line Behavior install update source No package specified, update latest SMUs of all installed packages install update source ncs5500-mpls Package name specified, will install that package, update all latest SMUs of that package(s) (along with its dependencies). install update source ncs5500- SMU installation: the SMU will be downloaded mpls-2.0.0.0-r611.CSCab12345.x86_64.rpm and installed (along with its dependent SMUs). install update source ncs5500- Asynchronous package upgrade, that package mpls-2.1.0.0-r632.x86_64.rpm will be installed (along with its dependent SMUs). Available since 6.1.1

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38 Upgrading XR Packages

Command Line Behavior install upgrade source version 6.5.1 Upgrade the base image to the specified version. All installed packages will be upgraded to same release as the base package. install upgrade source version 6.5.1 Perform install upgrade and install update for a ncs5500-mpls-2.2.0.0-r651.x86_64.rpm specific package(s) in one operation.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 Asynchronous Package Update

• Deliver a functional feature (such as BGP) independent of XR release.

• Installer checks dependency.

BGP 6.2.3 RPM BGP 6.2.2 BGP 6.2.2 BGP 6.2.2 BGP 1.1.0.0 ISIS ISIS BGP 1.1.0.0 6.2.2 RPM 6.2.3 RPM RPM ISIS 2.2.0.0 ISIS ISIS ISIS 2.2.0.0 ISIS 2.3.0.0 2.3.0.0 2.3.0.0 Release RPM Install Release RPM Update Release Release 6.2.2 Base 6.2.2 Base 6.2.2 Base 6.2.2 Base

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 PIE Install RPM Install

• No dependency management • Dependency management

• Offline process required to copy • Online process over secure packages transport • Require multiple operations • Single operation • install add • install update or install upgrade • install activate • On-box / Off-box package • install commit inspection using rpm tool • Cisco Software Manager • Description (CSM) for package content • Dependencies

• Content

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41 Installing and Updating Third Party Packages

• Third Party Packages are traditional Linux tools available from the Shell • Communication: lighttpd ,openssh, wget, curl, etc. • Programming: python, ruby, perl, etc. • Utilities: sed, gawk, tar, , vi, etc.

• Additional packages provided by vendors (No Cisco Support) • Chef • Puppet

• Installed using yum or rpm yum-config-manager --add-repo=http://192.168.0.254/XR/6.X.0 yum install chef –y

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42 Ansible, Puppet and Chef

• Open Source Software with Commercial Support or Add-Ons.

• Configuration management tools.

• Idempotent, convergent.

• Describe infrastructure as code. • Can then be managed from within a VCS such as git. • Changes are attempted, shared, rolled forward and rolled back in a much more frictionless way than traditionally.

• Agentless • Agent based • Agent based • Uses Python 2.7 • Uses Ruby based DSL for • Uses a subset of Ruby for configuration configuration • Communicates over SSH

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43 Application Hosting Application Hosting

Inside the Control Plane Inside the Third Party LXC

• Visibility of all XR processes • No direct visibility of XR processes

• Visibility of all XR resources • No direct visibility of XR resources

• Run alongside XR Processes • Run in a separate namespace • No isolation • Isolation

• Must be built for WRL7 • Must be built for the third

distribution VM/LXC party LXC distribution LXC LXC

Control Third Third Plane Party Party

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 Global VRF Network Name Space

• Provides visibility of fabric attached Control Plane Admin Plane Third Party interfaces outside of Global-VRF Global-VRF XR CLI. Mgmt Mgmt Internal Gig Gig IPC • Available to processes TenGig Interfaces TenGig inside the Control HunGig HunGig Plane or Third Party containers.

• Requires that the Global-VRF interface is up. 64-bit Host OS Routing Processor • Routing handled by XR.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46 Third Party Container Application Workflow

2 Deploy 3 Launch • Create the Container archive on a Linux Server. Server • Copy the to /misc/app_host.

• Unarchive in a rootfs directory. Control Admin Third TAR Plane Plane Party • Create XML file specifying LXC parameters. 1 Create 64-bit Host OS • Run virsh command. Routing Processor

virsh create

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 XML File example xr-lxc-app 327680 exe /sbin/init 1 /usr/lib64/libvirt/libvirt_lxc destroy restart destroy

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48 Configuration, Monitoring and Troubleshooting Model-Driven Programmability Stack

Model-Driven Apps App1 App2 App3 Configuration

Model-Driven APIs APIs YANG Development Kit (YDK)

Protocol NETCONF RESTCONF gRPC

Encoding XML JSON GPB

Transport SSH HTTP HTTP/2

YANG Models Model- Models (native, open) Driven Telemetry

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50 Programmatic Configuration using Data Models

• YANG: A Data Modeling Language • Explicitly and precisely determines the structure, syntax and semantics of the Protocols data… • …that is externally visible • Consistent and complete Instance

• NETCONF, RESTCONF, GRPC: Data Protocols • Provides primitives to view and Data manipulate data Model • Encoding of the data as defined by the data model

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 51 Benefits of Model-Driven Programmability

• Model based, structured, computer friendly.

• Multiple model types (native, OpenConfig, IETF, etc.).

• Models decoupled from transport, protocol end encoding.

• Choice of transport, protocol and encoding.

• Model-driven APIs for abstraction and simplification.

• Wide standard support while leveraging open source.

Reduce Operational Costs

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 52 Introduction to YANG

Leaf Leaf list • Modeling language (initially for NETCONF)

• Main node types Container (grouping) • Leaf – An end node of data List • Leaf list – A list of single items key • Container – Groups things together Container • List – A collection of containers (presence)

Node without a value Node with a value

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53 YANG Model Example

YANG CLI container community-sets { community-set CSET1 description “Container for community sets"; 65172:1, list community-set { 65172:2, key community-set-name; 65172:3 description "Definitions for community sets"; end-set leaf community-set-name { ! type string; community-set CSET10 description "name of the community set"; 65172:10, } 65172:20, leaf-list community-member { 65172:30 type string { end-set pattern '([0-9]+:[0-9]+)'; ! } description "members of the community set"; } } }

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54 Model Data Example

JSON CLI { "community-sets": { community-set CSET1 "community-set": [ 65172:1, { "community-set-name": "CSET1", 65172:2, "community-member": [ 65172:3 "65172:1", end-set "65172:2", ! "65172:3" ] community-set CSET10 }, 65172:10, { "community-set-name": "CSET10", 65172:20, "community-member": [ 65172:30 "65172:10", end-set "65172:20", ! "65172:30" ] } ] } }

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55 YANG Models (Github)

IOS XR Models XR models per release

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56 NETCONF Protocol Overview

• Rich functionality to manage configuration and operational (state) data XML

• Operations defined as RPCs (request / reply) in XML NETCONF • Client/app initiate request towards server/device

• Supports running, candidate and startup SSH configurations

• Capability exchange during session YANG initiation

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 57 Network Configuration Protocol (NETCONF)

• NETCONF provides mechanisms to install, manipulate, and delete device configuration.

• It uses an XML based data encoding for the configuration data as well as the protocol messages.

• NETCONF protocol operations are realized as remote procedure calls (RPCs) over SSH. SSH Clients Networking Device

SSH Retrieve, Configure, Copy, and Delete

RPC calls are divided in 4 groups of methods: Retrieve, Configure, Copy, and Delete. Additional methods can be supported

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58 Model-Driven APIs

Class Hierarchy • Simplify app development YANG Model (Python, C++, Ruby, Go) • Abstract transport and encoding

• Generated from YANG model

• One-to-one correspondence between model and class hierarchy

• Multi-language (Python, C++, Ruby, Go, etc.)

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60 Generation of Model-Driven APIs Using YANG Development Kit (YDK)

YDK Python Docs YDK-Py

YDK-gen Data C++ Docs YDK-CPP Models API YANG YANG(YANG) Ruby Docs YANG Generator

go Docs YDK-Go : :

C♯ Docs

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 61 Programmatic Interface

Secure Connection Negotiate RPC-based Configuration Manager Transport Oriented Capabilities communication model

Backend Apps XR SSH NETCONF YANG SSHd Programmatic Proxy Agent Framework SysDB Interface Server Client SysDB Library

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 62 Overview of gRPC on Cisco IOS XR

• Google RPC provides a general (open source) RPC framework JSON CLI

• Interface definition in Cisco IOS XR specifies device operations Cisco gRPC

• Functional subset of NETCONF HTTP/2 • Simple client development • High performance YANG

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 65 Streaming Telemetry Traditional Monitoring Concepts No Longer suited for Cloud-Scale Network Operations Where Data Is Created Where Data Is Useful

SNMP

syslog Sensing & Measurement Storage & Analysis CLI Strong burden on back-end Normalize different encodings, transports, data models, timestamps

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 68 Streaming Telemetry Concepts Better suited for Cloud-Scale Network Operations

Where Data Is Created Streaming Telemetry Where Data Is Useful

Push paradigm

One consistent way to access Statistics, Oper Sensing & state & Events @ all layers Measurement High Performance: 10 sec Storage & Analysis Multiple encodings & Transport Analytics-Ready Data

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 69 How Do You See Telemetry?

Transport 1 2 3

Encoding 1 2 3

Models 1 2 3

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 70 “Pushing” More Data Really Does Work Better Counters CPU load 400 30 300 20% 20 14% 200 10 7% 7% 7% 8%

100 Thousands 0 0 1 2 3 5s 10s 15s 20s Destinations

Time to collect all data (NCS5516, 576х100GE)

Interface counters (In/Out pkts, In/Out Discards, In/Out Errors)

MemAllocated Telemetry SNMP 0 5 10 15 20 25 Seconds #CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 71 Cisco IOS XR Telemetry Support

Classic XR Evolved XR NCS5500 NCS6k/CRS ASR9k ASR9k MDT support 6.1.1 6.1.1 6.1.1 6.1.3 YANG YANG YANG YANG Data models (native, OC) (native, OC) (native, OC) (native, OC) Transport gRPC gRPC TCP, TCP, (Control (dial-in, dial-out), (dial-in, dial-out), UDP (6.2.1) UDP (6.2.1) protocols) TCP, UDP (6.2.1) TCP, UDP (6.2.1) GPB / GPB / GPB / GPB / Encoding GPB-KV / GPB-KV / GPB-KV / GPB-KV / JSON (6.3.1) JSON (6.3.1) JSON (6.3.1) JSON (6.3.1)

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 72 Model-Driven Telemetry Example telemetry model-driven destination-group DGroup1 address-family ipv4 10.30.110.38 port 5432 encoding self-describing-gpb protocol tcp ! GRPC with GPB Where to Send and How ! encoding address-family ipv6 2001::1 port 2345 encoding json protocol grpc no-tls GRPC with JSON encoding ! What Data to Collect sensor-group SGroup1 sensor-path Cisco-IOS-XR-infra-statsd-oper:infra- statistics/interfaces/interface/latest/generic-counters How Often ! subscription Sub1 If bored with manual, automate MDT configuration with YDK sensor-group-id SGroup1 sample-interval 1000 (http://ydk.io)! destination-id DGroup1 Examples of Telemetry config with Native Models: https://goo.gl/Adjcgu, and with Openconfig Models: #CLUS https://goo.gl/Ubb2ZNBRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 74 Event Driven Telemetry To make telemetry even better

Model-Driven Telemetry Event-Driven Telemetry

Router X Router X

t0 t0

t1 t1

100 interfaces UP / 0 interfaces DOWN 100 interfaces UP / 0 interfaces DOWN t0 t0 100 interfaces UP / 0 interfaces DOWN

100 interfaces UP / 0 interfaces DOWN

99 interfaces UP / 1 interfaces DOWN Time interface X state DOWN Time t1 t1 99 interfaces UP / 1 interfaces DOWN

99 interfaces UP / 1 interfaces DOWN

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 75 Event Driven Telemetry Easy to configure

6.3.1 6.3.2 6.5.1* Interface OpenConfig RIB LLDP BGP, ISIS, Syslog MPLS, etc.

telemetry model-driven subscription interface sensor-group-id interface sample-interval 0

* The full list will be available after XR 6.5.1 is published on CCO

#CLUS BRKSPG-2004 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 76 Service Level API: De-Layering The Network Stack

3rd Party Agent + Telemetry OSS Management/Presentation Layer NBI Provides access to configure and manage the stack through Network config/oper DB: Yang Models, CLI. Management CLI, Netconf, SNMP, Syslog, SSH APL Application/Protocol Layer Provides APIs for the Routing Protocols (BGP, IGP, etc.) Applications / Protocol Stack BGP, ISIS, OSPF, LDP, SR, L2 Protocols Network Infrastructure Layer - Service Adaptation Layer SAL SL API Bridge between the Application Layer and the HW Network Infrastructure / Service Adaptation Presents abstractions to the Application/Protocol Layer RIB, Label Manager, BFD, Interface and more System OS - Linux Kernel BSP (Board Support Package) - Boot Loader, Device ASIC SDK System OS + BSP Drivers, etc. ASIC SDK and drivers for the SDK

HW/Data Plane Hardware Fans, Sensors, NPU ASIC CPU Optics, etc. ASIC/Switching Silicon + CPU, Fans, Sensors

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 77 Off-Box Service Layer API Architecture Request gRPC Client stub

On-Box Python, C++, go gRPC Client stub

Python, C++, go Response gRPC Server (Router)

Protobuf Schema/Model

Initialization RIB MPLS Interface BFD …… RPCs RPCs RPCs RPCs RPCs

SL-API Functionality Domains

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 78 SL-API Example Use Cases

Traffic Engineering and Programmable Route Bring your own Path Selection Downloads Protocol/Agent

Controller Controller

SL SL API API

Engineering paths for Programmable route On-box agents and custom applications through downloads to CDN PoP routers protocols that co-exist with Route/label manipulation, all to optimize TCAM space standard protocols to influence based on user specific logic routing

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Cisco Service Layer APIs

• Github: https://github.com/Cisco-Service-Layer/service-layer-objmodel

• Proto definitions of the latest RPC versions

• Getting started tutorials

• Exhaustive python Unit Tests @xrdocs: https://xrdocs.github.io/cisco-service-layer/

• Blogs, Tutorials on Using Service Layer APIs and associated Libraries APIdocs: https://xrdocs.github.io/cisco-service-layer/apidocs/

• Doxygen based documentation, auto-generated from the proto files

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 80 Examples iPerf Based Path Selection in OSPF

3rd Party LXC 3rd Party LXC

Control Admin Admin Control Plane Plane Plane Plane iPerf Session

rtr1 rtr2 OSPF • Each router is running a Linux distribution (Debian, Red Hat, etc…) inside the 3rd Party LXC • iPerf is running inside the container and a session is created between the two routers • Link degradation is detected by iPerf inside the container • Netconf client inside the container will update the OSPF cost of links based on iPerf result

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 82 Open/R on IOS-XR

Open/R running inside Docker on XR Open/R

IPv6 Hellos, 0MQ neighbors messages

Service Layer Telemetry APIs Packet Routes Interface IO Open/R hellos and adjacencies Notifications

IOS-XR IOS-XR IOS-XR RIB SYSDB Interfaces

https://github.com/akshshar/openr-xr

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 84 Conclusions Key Takeaways

• No Change in traditional CLI Operation with IOS-XR 6.X

• 6.X is available on NCS5500, NCS5000 and NCS1000 Platforms

• 6.1.1 is available on ASR9k (traditional 32-bit QNX offering will still be supported)

• Operational enhancement for Automated Operations

• Operating System change for Open Source tooling

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 86 IOS-XR 6.X Operational Enhancements

IOS-XR 6.X

& &

Packaging

Flexible Platform

ApplicationHosting

Visibility &Telemetry Visibility Evolved Evolved programmability

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 87 Related Sessions • BRKSPG-2004: Advanced Topics in Telemetry

• LTRSPG-2601: Cisco IOS XR Programmability

• DEVNET-1775: Introduction to OpenConfig

• BRKSPG-2303: Model-driven programmability for Cisco IOS XR

• BRKSPG-3000: Advanced IOS-XR Programmability Using Service Layer APIs

• DEVNET-2936: Bringing Programmability to Service Provider deployments with IOS-XR

• LTRSDN-2260: Network Automation using YANG Models and DevOps Tools across IOS-XE, IOS- XR, and NX-OS

• BRKARC-2022: Introduction to High Performance Network Convergence System Routers

• BRKARC-3000: NCS5500: Deepdive in the Merchant Silicon High-end SP Routers

• DEVNET-1002: DevNet Workshop - NetDevOps for the Network Dude - How to get started with API's, Ansible and Python

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 88 References

• IOS-XR Documentation, Blogs Tutorials and more : https://xrdocs.github.io/

• IOS-XR YANG Models: https://git.io/vg7fk YANG Developer Kit: http://ydk.cisco.com/py/docs/

• Getting Started With OpenConfig in Cisco IOS XR: https://git.io/vg7vF

• Getting Started With gRPC in Cisco IOS XR: https://git.io/vg7vP

• IOS-XR Ansible: https://github.com/ios-xr/iosxr-ansible

• IOS-XR Puppet: https://docs.puppet.com/pe/latest/install_iosxr.html

• IOS-XR Chef: https://downloads.chef.io/chef-client/iosxr/

• Cisco IOS XR 6.0 Application Hosting Lab v1: https://dcloud-cms.cisco.com/demo/cisco-ios-xr-6-0-application-hosting-lab-v1 • Cisco IOS XR 6.0 Software Management Lab v1: https://dcloud-cms.cisco.com/demo/cisco-ios-xr-software-management-lab-v1 • Cisco IOS XR 6.0 End-to-End Automation Lab v1: https://dcloud-cms.cisco.com/demo/cisco-ios-xr-6-0-end-to-end-automation-lab-v1

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 89 Complete your online session evaluation

Give us your feedback to be entered into a Daily Survey Drawing. Complete your session surveys through the Cisco Live mobile app or on www.CiscoLive.com/us.

Don’t forget: Cisco Live sessions will be available for viewing on demand after the event at www.CiscoLive.com/Online.

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 90 Continue Demos in Walk-in Meet the Related your the Cisco self-paced engineer sessions education campus labs 1:1 meetings

#CLUS BRKSPG-2069 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 91 Thank you

#CLUS #CLUS