BRKSPG-2069

Introduction to the Next-Generation IOS-XR architecture

@irakshat

@xrdocs

Akshat Sharma, Technical Marketing Engineer, Cisco Cisco Webex Teams

Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How 1 Find this session in the Cisco Events Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space cs.co/ciscolivebot#BRKSPG-2069

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 3 Agenda • Introduction

• Software Architecture Overview

• Flexible Packaging

• Application Hosting

• Configuration, Monitoring and Troubleshooting

• Examples

• Conclusions

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 Introduction An Evolved Network for a Rapidly Changing Web and SP Landscape

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 Key Learning Objectives

• At the end of the session, you should be able to:

• Appreciate rapidly evolving network infrastructure landscape

• Understand how Cisco is evolving IOS-XR network operating systems to meet customer needs

• Identify technical principles of Evolved IOS-XR

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 Automating Provisioning Down to Minutes

Traditional Cloud Scale Day 0 Day 1 Day 2 Network Operations Network Operations

INSTALL CONFIGURE Plug in new device Plug in new device

Operating System Automated Boot Network Operations Center Upgrade (Manual)

Manual staging CLI Script Operating Configuration System Configure new device remotely Auto-provisioning (Manual)

Traditional Cloud Scale Device up and running Device up and running Networking Operations Networking Operations • Manual device onboarding • Reduced time to install and provision new devices • Lengthy installation and 4 Hours to 3 Days 10 to 15 Minutes provisioning procedures Operating • Faster disaster recovery System • Automated device onboarding

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 Traditional NetOps Upended by Evolved Needs of SP Customers

Traditional Network Evolved SP/Cloud Scale DevOps Operations Network Operations

Day 0

Manual INSTALL Automation Provisioning • Automated services • Simple to scale

Day 1 • Agile, open software SW Inflexible Modularity SW & CONFIGURE Extensibility Fragmented Topology Visibility & Day 2 View and Control Complex Routing MANAGE & OPTIMIZE * Source: Google

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 Cisco Evolved IOS XR Innovations Designed for Operational Fit

Software Modularity Automation Visibility and Control and Extensibility Data Third- Service Agility Model party Simplification Driven Agents Asynchronous Upgrade Support Telemetry

Granular Packaging Visibility Modularity Operational Efficiency Better Customer Open Innovation Experience

Docker Application Automated Boot and Application Hosting Auto-provisioning Engineered Routing

Containers Control Extensibility

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 Evolved IOS-XR Enablement across Platforms

NCS5501/SE

NCS5504 NCS5502/SE

NCS5001 NCS6000

NCS5002 NCS1002 with MACSec NCS-55A1-24H

NCS5011 NCS5508 NCS5516 NCS-55A1-36H-S ASR9000 64-bit NCS-55A2-MOD NCS1002 Now Shipping 6.5.1

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 Pillars of Evolved IOS-XR

Visibility & • Operational Data, Deep analytical hooks Telemetry • Model-based, flexible, Push Model

Evolved • Data accessible via published model driven interfaces (Openconfig & Netconf) • Machine friendly Programmability • Enables automation @ scale

Application • Ability to run 3rd party off the shelf applications built with Linux tool chains Hosting • Run custom applications inside a separate LXC container

• Packages can be inspected on box using standard tool chain (RPM tools) Flexible Platform • Automated package installation with dependency checking and Packaging • Open (iPXE) and end-to-end auto-provision • 64-bit Linux

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 Software Architecture IOS-XR 6.X: A New Software Infrastructure

• 64-bit OpenEmbedded Linux support. Classic XR XR 6.X • Processes containerization. • Standard Linux toolchain. System • Third-Party applications. System Control Admin Control Admin • NCS5500, NCS5000 and NCS1002 support 64-bit Linux. 32 bit QNX 64 bit Linux • ASR 9000 supports 64-bit Linux NPU Hardware starting with 6.1.1. • Will still have 32-bit QNX support.

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 Introduction to

• With eXR, the Admin plane and the Control Plane run in isolated NCS5000-5500 ASR9K environment. • ASR9K uses Virtual Machines (VMs) App App App App • NCS5000 and NCS5500 use containers … Lib Lib … (LXCs) Lib Lib OS OS • Provide the same functionalities. Operating System • Use same tools (libvirsh) to Provide the virtualization services X86 Hardware Operating System X86 Hardware • LXCs are lightweight no ISSU support. • VMs are heavier but provide ISSU support.

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 IOS-XR Entities Roles: The Host

• Runs Yocto based 64-bit Open Embedded . • Built using Windriver 7. • The main functions of the host are: • Interact directly with the underlying hardware. • Provide kernel services for the LXCs/VMs • Provide libraries, tools, and utilities to help launch, monitor, and maintain containers. Control Admin Third • Provide the network infrastructure to allow containers to Plane Plane Party communicate.

64-bit Host OS

Routing Processor

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 Container Architecture NCS 5500 and NCS 5000 Runs processes responsible to perform Cisco developed Runs any 64-bit Linux system diags, monitor packages for core distribution. env. variables, and network functions Launched from the XR manage hardware (BGP, MPLS, etc.) container using virsh components and libvirtd. Yocto packages for First container to be standard Linux tools Access Network Control Admin booted by the host, Third and libraries Interface through the Plane Plane responsible for the Party (bash, python, Global VRF Network start and maintenance tcpdump, etc.). Name Space. of the Control Plane container

64-bit Host OS

Routing Processor

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 Container Architecture (ASR9K)

VM LXC Runs any 64-bit VM Runs processes Cisco developed Linux distribution. responsible to perform packages for core Launched from the system diags, monitor env. network functions XR variables, and manage (BGP, MPLS, etc.) using virsh and hardware components Admin Control Third libvirtd. Yocto packages Plane Party Plane First VM to be booted by for standard Linux Access Network the Hypervisor, responsible tools and libraries Interface through the for the start and (bash, python, Global VRF Network maintenance of the Control tcpdump, etc.). Name Space. Plane VM

Hypervisor 64-bit Host OS Routing Processor

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 Zero Touch Provisioning and iPXE Automated Installation of IOS-XR images IOS-XR images

• Cisco Platforms: ISO format for iPXE • Supported Whitebox Platforms: ONIE compatible installer

Web Server DHCP Server

Route r

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 Automated Provisioning using ZTP ZTP Artifacts

• CLI Configuration • ZTP script: Support for Data ports  Native Python scripts  Native bash scripts, Starting with Release 6.5.1!  Golang/c++/pyinstaller binaries

Web Server DHCP Server

IOS-XR

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Sample DHCP Server Configuration DHCP option space cisco-vendor-id-vendor-class code width 1 length width 1; option 124 option vendor-class.cisco-vendor-id-vendor-class code 9 = {string}; class "ncs5508" { match if (substring(option dhcp-client-identifier,0,11) = "FGE194714QS");} pool { DHCP https://xrdocs.io/device- allow members of "ncs5508"; lifecycle/blogs/2017-09-21- option 77 DHCP range 11.11.11.47 11.11.11.50; -xr-ztp-learning-through- option 67 packet-captures/ if exists user-class and option user-class = "iPXE" { filename="http://11.11.11.2:9090/ncs5500-mini-x.iso"; DHCP } option 77 DHCP if exists user-class and option user-class = "exr-config" { option 124 if (substring(option vendor-class.cisco-vendor-id-vendor-class,3,11)="FGE194714QS") { DHCP if (substring(option vendor-class.cisco-vendor-id-vendor-class,19,99)="NCS-5508") option 124 { filename="http://11.11.11.2:9090/scripts/ztp_script.py"; DHCP option 67 } }}}

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 IOS-XR Boot Process with iPXE and ZTP

DHCP SERVER

N iPXE IP address Next-server boot 1 Filename=http:///new-img.ISO Y 2 IP address iPXE Next-server Filename=http:///script-SN.sh or XR Install Filename=http:///config-SN.txt

XR Boot 3 HTTP(S) SERVER

ZTP GET script-SN.sh or config-SN.txt execution 4 Image-new.ISO

config executable 5 script-SN.sh GET addon scripts/packages/configuration Apply config-SN.txt configuration Execute script Additional Scripts Retry Success Retry Success Packages, etc…

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 26 ZTP Functions

Apply Configuration Deploy Applications (Containers, Binaries, cron-jobs, etc.)

ZTP

Software Upgrade/Downgrade Provide on-box automation libraries: Cli, Yang-APIs etc.

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 Model-Driven ZTP Automation

Introducing on-box Yang API hooks Yang based provisioning from Day 0!

• Introducing Yang-APIs using ZTP Python on-box netconf client in Release APIs

7.0.1 IOS-XR CLI

Yang over • On-box netconf client is netconf transport free - No SSH port to Yang over GNMI connect to for ztp script/client.

• Localhost based GNMI capability coming soon

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 ZTP Script: Available and Upcoming Options

ZTP Scripts

Python Library Bash Library Binaries

IOS-XR CLI Pyinstaller (ncclient, gNMI/gRPC) • Show commands • commands IOS-XR CLI YANG-XML (Actions) golang • Merge Config (gNMI/gRPC) • Show • Ncclient • Replace config commands • Native • Exec commands netconf C++ (Actions) client (gNMI/gRPC) • Merge Config • gNMI/gRPC • Replace config * Local gNMI/gRPC support coming soon.

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 Want to know more?

• IOS-XR Documentation, blogs, tutorial, etc .. • https://xrdocs.io/ • IPXE Deep Dive • https://xrdocs.io/device-lifecycle/tutorials/2016-07-27-ipxe-deep-dive/ • Working with ZTP • https://xrdocs.io/device-lifecycle/tutorials/2016-08-26-working-with-ztp/ • https://xrdocs.io/device-lifecycle/blogs/2017-09-21-ios-xr-ztp-learning- through-packet-captures/ • https://github.com/ios-xr/iosxr-ztp-python • Software Management blogs, tutorial, etc … • https://xrdocs.io/device-lifecycle/

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 31 Flexible Packaging Networking Software Delivery Direction

• Server-like Workflows • Modular Disaggregated • Update, Upgrade, Remove • RPM – RPM • Dependency management • YUM – Yellowdog Updater Modified • Secure • New Install Commands • Repository Support • Third Party Applications

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 33 RPM: IOS-XR New Package Format

• RPM Package Manager is the new Package format starting with IOS-XR 6.0. • Packages are placed in a reachable repository and accessed via FTP/SFTP/SCP/TFTP or HTTP or pre-staged on the box • Third Party packages are installed with RPM or YUM inside the Shell. • IOS-XR packages are installed with “install update/upgrade”. • Both YUM and install commands provide dependency verification/resolution.

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 34 Anatomy of RPM Packages

RPM

Archive CPIO binary archive

Describe package contents MetaData Install structure /var/lib/rpm Dependencies

Scriptlet Pre and Post install instructions Database of installed packages

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 XR Packages Naming Convention

Package --..rpm

ncs5500-mgbl-4.0.0.0-r632.x86_64.rpm

Major.Minor.Build.SMU SMU --...rpm

ncs5500-mgbl-4.0.0.1-r632.CSCab12345.x86_64.rpm

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 36 Package Repository www.cisco.com Local Repository

Mini ISO Platform + Pkgs 6.3.2 Release Mini ISO ncs5500-mini-x-6.3.2.iso + k9 pkg ncs5500-mpls-te-rsvp-2.2.0.0-r632.x86_64.rpm + XR Software Pkgs ncs5500-isis-1.3.0.0-r632.x86_64.rpm

Full K9 ISO ncs5500-mcast-2.1.0.0-r632.x86_64.rpm TAR ncs5500-k9sec-4.1.0.0-r632.x86_64.rpm Full ISO ncs5500-mgbl-4.0.0.0-r632.x86_64.rpm

XR SMUs DDTS SMU ncs5500-mpls-2.1.0.0-r632.x86_64.rpm ncs5500-ospf-2.0.0.0-r632.x86_64.rpm ncs5500-li-1.0.0.0-r632.x86_64.rpm Svc Pak DDTS SMUs ncs-5500-mgbl-4.0.0.1-r632.CSCab12345.x86_64.rpm

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 37 Updating XR Packages

Command Line Behavior install update source No package specified, update latest SMUs of all installed packages install update source ncs5500-mpls Package name specified, will install that package, update all latest SMUs of that package(s) (along with its dependencies). install update source ncs5500- SMU installation: the SMU will be downloaded mpls-2.0.0.0-r611.CSCab12345.x86_64.rpm and installed (along with its dependent SMUs). install update source ncs5500- Asynchronous package upgrade, that package mpls-2.1.0.0-r632.x86_64.rpm will be installed (along with its dependent SMUs). Available since 6.1.1

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 38 Upgrading XR Packages

Command Line Behavior install upgrade source version 6.5.1 Upgrade the base image to the specified version. All installed packages will be upgraded to same release as the base package. install upgrade source version 6.5.1 Perform install upgrade and install update for a ncs5500-mpls-2.2.0.0-r651.x86_64.rpm specific package(s) in one operation.

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 PIE Install RPM Install

• No dependency management • Dependency management • Offline process required to copy • Online process over secure packages transport • Require multiple operations • Single operation • install add • install update or install upgrade • install activate • install commit • On-box / Off-box package inspection using rpm tool • Cisco Software Manager (CSM) • Description for package content • Dependencies • Content

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 Introducing Golden ISO (GISO) workflows Creating Deployment-Ready Golden ISOs

Base ISO: Golden ISO Golden ISO++

• Minimal Software image Minimal Software image + Minimal Software image • Use ZTP or day 1 operations + to install/apply: Functionality packages/RPMs + Functionality packages/RPMs Base Configuration +  Functionality Base Configuration packages/RPMs  IOS-XR Configuration  Third-party applications Third-party applications installed Third-party applications during ZTP or Day-1 Ops

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 42 Golden ISO Workflow

IOS- XR RPMs Deploy using iPXE or XR install RPM Repo

GISO tool

Base ISO

Build ISO DHCP Server Web Server

Router

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 43 Installing and Updating Third Party Packages

• Third Party Packages are traditional Linux tools available from the Shell • Communication: lighttpd ,openssh, wget, , etc. • Programming: python, ruby, , etc. • Utilities: sed, gawk, tar, gzip, vi, etc.

• Additional packages provided by vendors (No Cisco Support) • Chef • Puppet

• Installed using yum or rpm

yum-config-manager --add-repo=http://192.168.0.254/XR/6.X.0 yum install chef –y

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 44 Ansible, Puppet and Chef

• Open Source Software with Commercial Support or Add-Ons. • Configuration management tools. • Idempotent • Describe infrastructure as code. • Can then be managed from within a VCS such as git. • Changes are attempted, shared, rolled forward and rolled back in a much more frictionless way than traditionally.

• Agentless • Agent based • Agent based • Uses Python 2.7 • Uses Ruby based DSL for • Uses a subset of Ruby for • Communicates over SSH configuration configuration

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 Application Hosting Application Hosting Inside the Inside the Control Plane Third Party LXC/

• Visibility of all XR processes • No direct visibility of XR processes

• Visibility of all XR resources • No direct visibility of XR resources

• Run alongside XR Processes • Run in a separate namespace • No isolation • Isolation

• Must be built for WRL7 • Must be built for the third

distribution VM/LXC party LXC distribution Binary LXC/ Docker Control Third Third Plane Party Party

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 Third Party Container Application Workflow

2 Deploy 3 Launch • Create the Container archive on a Linux Server.

• Copy the archive file to /misc/app_host. Server • Unarchive in a rootfs directory.

• Create XML file specifying LXC parameters.

• Run virsh command. Control Third TAR Plane Party

1 Create 64-bit Host OS virsh create Routing Processor virsh list

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 48 Docker Container Workflow

2 Pull 3 Launch • Create a docker image

• Pull down using docker client in XR control plane Server • Spin up the docker container on host OS by executing “docker run” in XR linux shell.

Control Plane

docker pull <> 1 Create 64-bit Host OS docker run <> docker exec <> Routing Processor

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 49 Configuration, Monitoring and Troubleshooting Modeled APIs at every layer of the Stack De-Layering The Network Stack

3rd Party Agent + Telemetry OSS

NBI Management/Presentation Layer

Management Provides access to configure and manage the stack through CLI, Netconf, SNMP, Syslog, SSH Network config/oper DB: Yang Models, CLI. APL Application/Protocol Layer Applications / Protocol Stack Provides APIs into the Routing Protocols (BGP, IGP, SR, etc.) BGP, ISIS, OSPF, LDP, SR, L2 Protocols SL API SAL Network Infrastructure Layer / Service Adaptation Layer

Network Infrastructure / Service Adaptation • Acts as the bridge between the Application Layer and the HW RIB, Label Manager, BFD, Interface and more • Presents abstractions to the Application/Protocol Layer

System OS - Linux Kernel BSP(Board Support Package) - Boot Loader, Device Drivers, ASIC SDK etc. System OS + BSP ASIC SDK and drivers for the SDK

HW/Data Plane Hardware Fans, Sensors, Optics, NPU ASIC CPU Consists of ASIC/Chipset from HW vendors + CPU, Fans, etc. Sensors

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 52 Model Driven Manageability Cisco IOS XR Device Programmability

Model-Driven Manageability Service Layer API

App App

Controller Controller Orchestrator Orchestrator

Management Plane

Control Plane Control Plane

Data Plane Data Plane

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 54 Model-Driven Manageability

Apps App App App Controller Orchestrator Model-Driven SDKs SDK YANG Development Kit (YDK)

Model-Driven Telemetry Protocol NETCONF gRPC

Closed-loop automation Encoding XML JSON GPB

Model-Driven Configuration Transport SSH TCP HTTP

YANG Models Network Models Device (native, open)

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 55 Introduction to YANG

• Modeling language Leaf Leaf list (initially for NETCONF)

• Main node types • Leaf – An end node of data Container (grouping) • Leaf list – A list of single items List • Container – Groups things together • List – A collection of containers key Container (presence)

Node without a value Node with a value

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 56 Benefits of Model-Driven APIs

• Model based, structured, friendly.

• Multiple model types (native, OpenConfig, IETF, etc.).

• Models decoupled from transport, protocol end encoding.

• Choice of transport, protocol and encoding.

• Model-driven APIs for abstraction and simplification.

• Wide standard support while leveraging open source.

Reduce Operational Costs

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 57 YANG Model Example

YANG CLI container community-sets { community-set CSET1 description “Container for community sets"; 65172:1, list community-set { 65172:2, key community-set-name; 65172:3 description "Definitions for community sets"; end-set leaf community-set-name { ! type string; community-set CSET10 description "name of the community set"; 65172:10, } 65172:20, leaf-list community-member { 65172:30 type string { end-set pattern '([0-9]+:[0-9]+)'; ! } description "members of the community set"; } } }

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 58 Model Data Example

JSON CLI { "community-sets": { community-set CSET1 "community-set": [ 65172:1, { "community-set-name": "CSET1", 65172:2, "community-member": [ 65172:3 "65172:1", end-set "65172:2", ! "65172:3" ] community-set CSET10 }, 65172:10, { "community-set-name": "CSET10", 65172:20, "community-member": [ 65172:30 "65172:10", end-set "65172:20", ! "65172:30" ] } ] } }

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 59 YANG Models (Github)

IOS XR Models XR models per release

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 60 YANG Development Kit

• SDK simplifying client development for model-driven programmability • Rich protocol support (NETCONF, RESTCONF, gNMI) YDK-Py YDK-Cpp YDK-Go • Rich data model support (XR, XE, NX-OS, OC, IETF) • Rich language support (Python, NETCONF Go, C++) Model-driven RESTCONF • Built-in model data validation classes gNMI • Open source ydk.io

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 61 Model-Driven APIs

Class Hierarchy • Simplify app development YANG Model (Python, C++, Ruby, Go) • Abstract transport and encoding

• Generated from YANG model

• One-to-one correspondence between model and class hierarchy

• Multi-language (Python, C++, Ruby, Go, etc.)

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 62 gNMI at a Glance (Introduced in IOS-XR 6.5.1)

• OpenConfig network management protocol • Supports configuration management and telemetry • Efficient data streaming over gRPC • IOS XR 6.5.1 introduced support for Configuration Telemetry gNMI • YDK 0.8.0 enables simple client development

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 63 Understanding the Service-Layer API Locating the 3rd Party Agent + Telemetry OSS Service-Layer API

SL API SAL

Service Adaptation Network Infrastructure Layer / RIB, Label Manager, BFD, Interface and more Service Adaptation Layer

• Acts as the bridge between the Application Layer and the HW

• Presents abstractions to the Application/Protocol Layer

• Highly Performant API outside the Central Config/Oper Database context

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 65 Service Layer API Architecture

Off-Box gRPC Client CLI, Yang Models, Streaming Telemetry stub

Python, C++, go Manageability Layer

Request Response On-Box Client gRPC SYSDB stub Python, C++, go

gRPC Server () Service Layer API

Network Infrastructure Layer (Service Layer) Protobuf Schema/Model

BFD Service-Layer APIs bypass Initialization RIB MPLS Interface …… RPCs RPCs RPCs RPCs RPCs SysDB giving higher

performance SL-API Functionality Verticals

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 66 SL-API Example Use Cases

Traffic Engineering and Programmable Route Bring your own Path Selection Downloads Protocol/Agent

Controller Controller

SL API SL API

Engineering paths for Programmable route On-box agents and custom applications through downloads to CDN PoP routers protocols that co-exist with Route/label manipulation, all to optimize TCAM space standard protocols to influence based on user specific logic routing

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 67 Cisco Service Layer APIs Github

• Github: Check out the Obj-model repository on Github at

https://github.com/Cisco-Service-Layer/service-layer- objmodel

 Proto definitions  Exhaustive Unit Tests and tutorial apps

• @xrdocs: Blogs, Tutorials on Using Service Layer APIs APIdocs @xrdocs and associated Libraries: https://xrdocs.io/cisco-service-layer/

• APIdocs: Doxygen based documentation, auto-generated from the proto files: https://xrdocs.io/cisco-service-layer/apidocs/

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 68 Streaming Telemetry Traditional Monitoring Concepts No Longer suited for Cloud-Scale Network Operations Where Data Is Created Where Data Is Useful

SNMP

syslog Sensing & Measurement Storage & Analysis CLI Strong burden on back-end Normalize different encodings, transports, data models, timestamps

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 70 Streaming Telemetry Concepts Better suited for Cloud-Scale Network Operations

Where Data Is Created Streaming Telemetry Where Data Is Useful

Push paradigm

One consistent way to access Statistics, Oper Sensing & state & Events @ all layers Measurement High Performance: 10 sec Storage & Analysis Multiple encodings & Transport Analytics-Ready Data

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 71 How Do You See Telemetry?

Transport 1 2 3

Encoding 1 2 3

Models 1 2 3

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 72 “Pushing” More Data Really Does Work Better Counters CPU load 400 30 300 20% 20 14% 200 10 7% 7% 7% 8%

100 Thousands 0 0 1 2 3 5s 10s 15s 20s Destinations

Time to collect all data (NCS5516, 576х100GE)

Interface counters (In/Out pkts, In/Out Discards, In/Out Errors)

MemAllocated Telemetry SNMP 0 5 10 15 20 25 Seconds

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 73 Cisco IOS XR Telemetry Support

Classic XR Evolved XR NCS5500 NCS6k/CRS ASR9k ASR9k MDT support 6.1.1 6.1.1 6.1.1 6.1.3 YANG YANG YANG YANG Data models (native, OC) (native, OC) (native, OC) (native, OC) Transport gRPC gRPC TCP, TCP, (Control (dial-in, dial-out), (dial-in, dial-out), UDP (6.2.1) UDP (6.2.1) protocols) TCP, UDP (6.2.1) TCP, UDP (6.2.1) GPB / GPB / GPB / GPB / Encoding GPB-KV / GPB-KV / GPB-KV / GPB-KV / JSON (6.3.1) JSON (6.3.1) JSON (6.3.1) JSON (6.3.1)

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 74 Model-Driven Telemetry Example telemetry model-driven destination-group DGroup1 address-family ipv4 10.30.110.38 port 5432 encoding self-describing-gpb protocol tcp GRPC with GPB Where to Send and How ! encoding ! address-family ipv6 2001::1 port 2345 encoding json protocol grpc no-tls GRPC with JSON ! encoding sensor-group SGroup1 What Data to Collect sensor-path Cisco-IOS-XR-infra-statsd-oper:infra- statistics/interfaces/interface/latest/generic-counters ! subscription Sub1 How Often sensor-group-id SGroup1 sample-interval 1000 destination-id DGroup1 If bored with manual, automate MDT configuration with YDK (http://ydk.io)! Examples of Telemetry config with Native Models: https://goo.gl/Adjcgu, and with Openconfig Models: https://goo.gl/Ubb2ZN

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 76 Event Driven Telemetry To make telemetry even better

Model-Driven Telemetry Event-Driven Telemetry

Router X Router X

t0 t0

t1 t1

100 interfaces UP / 0 interfaces DOWN 100 interfaces UP / 0 interfaces DOWN t0 t0 100 interfaces UP / 0 interfaces DOWN

100 interfaces UP / 0 interfaces DOWN

99 interfaces UP / 1 interfaces DOWN Time interface X state DOWN Time t1 t1 99 interfaces UP / 1 interfaces DOWN

99 interfaces UP / 1 interfaces DOWN

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 77 Event Driven Telemetry Easy to configure

6.3.1 6.3.2 6.5.1* Interface OpenConfig RIB LLDP BGP, ISIS, Syslog MPLS, etc.

telemetry model-driven subscription interface sensor-group-id interface sample-interval 0

* The full list will be available after XR 6.5.1 is published on CCO

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 78 Examples iPerf Based Path Selection in OSPF

3rd Party LXC 3rd Party LXC

Control Admin Admin Control Plane Plane Plane Plane iPerf Session

rtr1 rtr2 OSPF • Each router is running a Linux distribution (Debian, Red Hat, etc…) inside the 3rd Party LXC • iPerf is running inside the container and a session is created between the two routers • Link degradation is detected by iPerf inside the container • Netconf client inside the container will update the OSPF cost of links based on iPerf result

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 80 Open/R integration with IOS-XR (using Service-Layer APIs) https://github.com/akshshar/openr-xr

IPv6 Blog on xrdocs.io neighbors

Telemetry

XR LXC Open/R running inside Docker on XR

IOS-XR RIB Routes Open/R Service Layer APIs IOS-XR Hellos, Interfaces 0MQ messages Interface Notifications

Packet Host IO

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 82 Conclusions Key Takeaways

• No Change in traditional CLI Operation with IOS-XR 6.X

• 6.X is available on NCS5500, NCS5000 and NCS1000 Platforms

• 6.1.1 is available on ASR9k (traditional 32-bit QNX offering will still be supported)

• Operational enhancement for Automated Operations

• Operating System change for Open Source tooling

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 84 IOS-XR 6.X Operational Enhancements

IOS-XR 6.X

& &

Packaging

Flexible Platform

ApplicationHosting

Visibility &Telemetry Visibility Evolved Evolved programmability

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 85 References • IOS-XR Documentation, Blogs Tutorials and more : https://xrdocs.github.io/

• IOS-XR Programmability Learning Labs on DevNet: https://learninglabs.cisco.com/tracks/iosxr- programmability

• IOS-XR YANG Models: https://git.io/vg7fk

• YANG Developer Kit: http://ydk.cisco.com/py/docs/

• IOS-XR Ansible: https://github.com/ios-xr/iosxr-ansible

• IOS-XR Puppet: https://docs.puppet.com/pe/latest/install_iosxr.html

• IOS-XR Chef: https://downloads.chef.io/chef-client/iosxr/

• Cisco IOS XR 6.0 Application Hosting Lab v1: https://dcloud-cms.cisco.com/demo/cisco-ios-xr-6-0-application-hosting-lab-v1

• Cisco IOS XR 6.0 Software Management Lab v1: https://dcloud-cms.cisco.com/demo/cisco-ios-xr-software-management-lab-v1

• Cisco IOS XR 6.0 End-to-End Automation Lab v1: https://dcloud-cms.cisco.com/demo/cisco-ios-xr-6-0-end-to-end-automation-lab-v1

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 86 Cisco Webex Teams

Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How 1 Find this session in the Cisco Events Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space cs.co/ciscolivebot#BRKSPG-2069

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 87 Complete your online session survey

• Please complete your Online Session Survey after each session

• Complete 4 Session Surveys & the Overall Conference Survey (available from Thursday) to receive your Cisco Live T- shirt

• All surveys can be completed via the Cisco Events Mobile App or the Communication Stations

Don’t forget: Cisco Live sessions will be available for viewing on demand after the event at ciscolive.cisco.com

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 88 Continue Your Education

Related Demos in Walk-in Meet the sessions the Cisco self-paced engineer Showcase labs 1:1 meetings

BRKSPG-2069 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 89 Thank you