BRKSDN-2119.Pdf

Home , Protocol Buffers, YAML, YAML (framework)

Programming Device APIs

Fabrizio Maccioni – Technical Marketing Engineer - Cisco BRKSDN-2119 Agenda

• Introduction

• Device APIs History • WSMA • NETCONF 1.0 • Model Driven Programmability • NETCONF 1.1 • RESTCONF • gRPC • REST APIs: APIC-EM

• Key Takeaways Introduction Programmability, Automation, Orchestration, APIs terminology

NETCONF Open & Programmable | Standards-Based | Secure Southbound APIs RESTCONF gRPC Flexible Infrastructure Programmability Physical & Virtual Infrastructure | App Hosting

Software Driven | Extensible | Cloud Enabled | Secure

CLI CLI

Physical and Virtual Network Infrastructure

No Human Task Easy To Special Friendly Oriented Replay Tools

No Syntax No No Error Transaction format Structured Reporting manageme changes output nt

Lack of Hard to Requires Security Writeable Replay / special Concerns MIBs Rollback application

Lousy Hard to Scalability Models Parse Issues ...

• SNMP works “reasonably well for Device Monitoring”

• Does not work well for Configuration

CLIs are for humans

MachinesCLI needCLI APIs (Open Programmable Interfaces)

Physical and Virtual Network Infrastructure

• Programming Building block • APIs can have various Properties • Transport (i.e. SSH, HTTP) • Encoding (i.e. XML, JSON, ProtoBuffer) • Data structure (Data Models) • Some Examples of APIs • Twitter API • Java API • APIC-EM API

CLI NX-API NETCONF/ NX-API REST YANG CLI

NETCONF NGNIX CLI Parser NGNIX Data Model (YANG) Data Model (DME)

Device Features Device Features Device Features Device Features

BGP QoS ACL … BGP QoS ACL … BGP QoS ACL … BGP QoS ACL …

Catalyst 4K Catalyst 3K Nexus 7K Nexus 9K

“XML, JSON, YAML are Text-file formats used to store structured data for embedded and Web applications”

• Compare to [traditional] HTML • HTML to display data, case-insensitive • XML to describe data, case-sensitive

• Readable format for structuring data

value • Transmits data between servers and web apps

• Comprised of “Tags” and “Value”

“XML is a markup language that defines a set of rules for encoding documents in a format that is both human-readable and machine-readable”

• Readable format for structuring data “key”: ”value” • Comprised of “Key/Value” pairs

• Value: Array, Boolean, Number, String

• Largely replacing XML

“JSON is an open-standard format that uses human-readable text to transmit data objects consisting of attribute–value pairs. It is the most common data format used for asynchronous browser/server communication,”

• No marked blocks (like Python)

key: value • Simpler than XML/JSON • Comprised of Key/Value pairs

• It is standard (RFC 2822)

“YAML is a human-readable data serialization language that takes concepts from programming languages such as C, Perl, and Python, and ideas from XML (RFC 2822)”

{ "ietf-interfaces:interfaces": { --- "interface": [ ietf-interfaces:interfaces: { interface: eth0 "name": "eth0”, name: eth0 ethernetCsmacd "type": "ethernetCsmacd”, type: ethernetCsmacd 0 "location": "0”, location: 0 true "enabled": true, enabled: true 2 "if-index": 2 if-index: 2 } ] } }

Elements <> | Parsers and Validation Key-Value Pairs | Readable | Simpler

• A framework– not a standard GET

POST • Architectural constraints API • Client–server PUT • Stateless

DELETE • Cacheable • Layered system • Uniform interface

NETCONF 1.0 WSMA onePK NETCONF 1.1

2005 2007 2012 2014

API: Application Programming Interface

show ip int br show ip interface brief

WSMA

HTTP / HTTPS

GigabitEthernet1 10.203.30.98 YES manual up up

ip http authentication local ip http secure-server

wsma agent exec profile httpslistener wsma agent config profile httpslistener wsma agent filesys profile httpslistener wsma agent notify profile httpslistener ! wsma profile listener httpslistener transport https

csr# show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet1 10.203.30.98 YES manual up up csr# show ip interface brief | format built-in GigabitEthernet1 10.203.30.98 YES manual up up

cisco cisco show ip int br > https://10.203.30.98/wsma POST

Demo GitHub Repo https://github.com/CiscoDevNet/dp-workbench

© 2017 u and/or its affiliates. All rights reserved. Cisco Public NETCONF NETCONF roots • June of 2002, the Internet Architecture Board (IAB) held invitational workshop on Network Management to • Identify a list of technologies relevant for network management with their strengths and weaknesses • Identify the most important operator needs

© 2017 u and/or its affiliates. All rights reserved. Cisco Public NETCONF definition “NETCONF is a protocol defined by the IETF to install, manipulate, and delete the configuration of network devices” Protocol Stack V 1.0 Content • XML • RFC 4741 1.0 Base NETCONF Protocol • get, get-config, Operations edit-config, etc. • RFC 4742 NETCONF over SSH Messages • rpc, rpc-reply

Transport • SSH

2006

Designed to address the shortcomings documented in RFC 3535

Main Operations Description (close to ‘show ?’) Retrieve running configuration and device state information (close to ‘show run’) Retrieve all or part of specified configuration datastore (close to ‘conf t’) Loads all or part of a configuration to the specified configuration datastore

Other Operations Description Replace an entire configuration datastore with another Delete a configuration datastore Copy candidate datastore to running datastore (ex: XR) / Lock or unlock the entire configuration datastore system Graceful termination of NETCONF session Forced termination of NETCONF session

© 2017 u and/or its affiliates. All rights reserved. Cisco Public NETCONF Datastores Target of Operations “A Datastore holds a copy of the configuration data that is required to get a device from its initial default state into a desired operational state”

Running running-config

Start-up startup-config

Available on IOS-XR Candidate work place for creating and manipulating configuration data

Running is the only mandatory Datastore

Transaction Management • Either all configuration is applied or nothing • Avoids inconsistent state • Both at Single Device and Network-wide level Error Management • All operations return OK or error code

Capability Exchange defined by the RFC

ssh -p 830 [email protected] -s netconf

Models Download from a Device

This is great for integration

Simplified View

NETCONF RESTCONF gRPC

NETCONF RESTconf gRPC Protocols CLI Parser Data Model Models

Device Features

Interface BGP QoS ACL …

Interface Model definition

“A Data-Model Explicitly and precisely defines Data Structure, Syntax and Semantics”

“YANG - A Data Modeling Language for NETCONF”

Protocol

The Data is NOT Data-Model Data defined by NETCONF RFC!

• YANG used to describe how to structure the Data to send/receive

• Standard defined in RFC 6020 YANG IETF https://tools.ietf.org/html/rfc6020

Interface Model definition Interface Model Instances in XML

interface: list, key = name

name: string

speed: string

duplex: string

YANG Models à Data Models defined using the YANG language

NETCONF RESTCONF gRPC

NETCONF RESTconf gRPC (YANG) Data Model Models Data Model Open Native Open Native Models Models Models Models

Device Features Configuration Operation Data

Interface BGP QoS ACL …

Config-data Operational-data

What the device is told to do What the box is actually doing It is (currently) the way It is (currently) what you get out of you express intent SNMP and most show commands

Examples: Examples: switch> show run interface Loopback0 switch> show interface Loopback0 switch(config)# interface Loopback0 ‘snmpget’ results

Open Native Models Models

Industry definition Cisco definition Compliant with standard Unique to a Cisco operating (IETF, ITU, OpenConfig, etc) system

Example: ietf-diffserv-policy.yang Example: Cisco-IOS-XR-ipv4-bgp-cfg.yang (IETF Diffserv data model) (IOS-XR BGP data model)

Open Models are a subset of the Native Models

Benefits:

• Use open models where

available Open Models (i.e. OpenConfig)

• Use native models for Map functionality not yet in open models Platform Native Models

Device Features

Interface BGP QoS ACL …

Model URL IEEE https://github.com/YangModels/yang/tree/master/standard/ieee

IETF https://github.com/YangModels/yang/tree/master/standard/ietf

Vendors https://github.com/YangModels/yang/tree/master/vendor/

OpenConfig https://github.com/openconfig/public

https://github.com/YangModels/yang/tree/master/vendor/cisco

http://www.openconfig.net/

NETCONF RESTCONF NETCONF RESTconf gRPC gRPC Protocols

NETCONF RESTconf gRPC XML JSON GPB (XML) (XML/JSON) Encoding

Data Model SSH HTTP HTTP/2 Transport Device Features

Interface BGP QoS ACL …

V 1.0 V 1.1 Extensions Protocol Stack

Content • XML • RFC 4741 1.0 • RFC 6241 – 1.1 • RFC 5277 Base NETCONF Base Notifications Protocol NETCONF • RFC 5717 Partial • get, get-config, Protocol Locking Operations edit-config, etc. • RFC 4742 NETCONF over • RFC 6242 – • RFC 6243 With SSH NETCONF over defaults Messages • rpc, rpc-reply SSH • RFC 6020 YANG

Transport • SSH

2006 2011

“REST-like protocol running over HTTP for accessing data defined in YANG using Datastores defined in NETCONF”

• It defines how a YANG model is mapped to a RESTful interface

• Supports OPTIONS, GET, PUT, POST, DELETE operations

• Request and response in XML or JSON format

• HTTP transport

• Now an IETF Standard: RFC 8040

IETF https://tools.ietf.org/html/rfc8040

RESTCONF As compared to NETCONF GET ,

POST (operation=“create”)

PUT (operation=“create/replace”)

DELETE (operation=“delete”)

“gRPC is an open source RPC (Remote Procedure Call) system developed at Google”

• Google ecosystem with automatic GPB integration

• cross-platform client and server bindings for many languages: C, C++, C#, Go, Java, Node.js, Objective-C, PHP, Python, Ruby

• Feature rich: authentication, bidirectional streaming and flow control, blocking/nonblocking bindings, cancellation and timeouts

• HTTP/2 transport

• Not a standard!

• Method of serializing structured data

• Data structures (messages) defined in a definition file (.proto)

• Smaller, Faster than XML/JSON/YAML

• Origin: Google's internal data interchange

The sender uses the .proto to serialize the data, the receiver uses the .proto to de-serialize the data. https://developers.google.com/protocol-buffers/ © 2017 u and/or its affiliates. All rights reserved. Cisco Public Streaming Telemetry Telemetry Collector

Subscription Periodic or on-change

Transport NETCONF RESTCONF gRPC

Encoding XML JSON GPB (YANG) Data Model Models Data Model Open Native Open Native Models Models Models Models

Device Features Configuration Operation Data

Interface BGP QoS ACL … Operational Data only

• Same concepts:

• Pub Sub is standardized at the IETF (by Cisco)

• Telemetry • Some differences:

• OpenConfig-telemetry is a subset of IETF effort

IETF PUB/SUB https://tools.ietf.org/html/draft-ietf-core-coap-pubsub-00 OC Telemetry http://www.openconfig.net/projects/streaming-telemetry/

rd 3 Party Custom

NETCONF RESTconf gRPC Protocols

Data Model

Device Features

Interface BGP QoS ACL …

“A GUI driven tool to test NETCONF and RESTCONF interfaces defined by YANG models”

• Load YANG models from device

• Browse YANG models

• Execute NETCONF or RESTCONF Operations

• Generate self-contained Python scripts

• Open Source!!

YangExplorer https://github.com/CiscoDevNet/yang-explorer

Python Scripts

Check Capabilities (2) Browse Models

(4) Create RPC

(5) Run RPC

RESTful APIs Programmability, Automation, Orchestration, APIs terminology

NETCONF Open & Programmable | Standards-Based | Secure Southbound APIs RESTCONF gRPC Flexible Infrastructure Programmability Physical & Virtual Infrastructure | App Hosting

Software Driven | Extensible | Cloud Enabled | Secure

APIC-EM APIC-EM Turn-key Solution Build Your Own APP GUI APIs

“Controller”

https:///api/v1/flow-analysis POST { "sourceIP" : "65.1.1.86", "destIP » : "207.1.10.20", "sourcePort" : "1000", "destPort » : "2000", "protocol » : "tcp", "inclusions » : ["QOS-STATS","INTERFACE-STATS","DEVICE-STATS","PERFORMANCE-STATS","ACL-TRACE"], "periodicRefresh » : true } API Response: { "response": { "flowAnalysisId": "5755deb8-e201-4770-8a18-65572b1ea4a2", "taskId": "57d77c04-9aa8-49f6-aed9-a6f590658241", "url": "/api/v1/flow-analysis/5755deb8-e201-4770-8a18-65572b1ea4a2" }, "version": "1.0" }

https:///api/v1/flow-analysis/5755deb8-e201-4770-8a18-65572b1ea4a2 GET

API Response: "lastUpdate": "Mon Jun 06 06:33:36 UTC 2016", (Continued) "networkElementsInfo": [ { { "response": { "id": "1aaa159e-5814-454a-8604-2645931eb4b6", "request": { "type": "wireless", "sourceIP": "65.1.1.86", "ip": "65.1.1.86", hop 1 "sourcePort": "1000", "linkInformationSource": "Switched" "destIP": "207.1.10.20", }, "destPort": "2000", { "protocol": "tcp", "id": "17184480-2617-42c3-b267-4fade5f794a9", "periodicRefresh": true, "name": "AP7081.059f.19ca", "inclusions": [ "type": "Unified AP", "ACL-TRACE", "ip": "55.1.1.3", "PERFORMANCE-STATS", "role": "ACCESS", hop 2 "DEVICE-STATS", "linkInformationSource": "Switched", "INTERFACE-STATS", "tunnels": [ "QOS-STATS" "CAPWAP Tunnel" ], ] "id": "5755deb8-e201-4770-8a18-65572b1ea4a2", }, "status": "COMPLETED", { "createTime": 1465194237489, "id": "24ac6aa8-7759-44d5-90a3-00c83e96583d", "lastUpdateTime": 1465194787854 "name": "CAMPUS-Access1", }, "type": "Switches and Hubs", hop 3 "ip": "212.1.10.1", ...

APIC-EM GUI DevNet

http://devnetapic.cisco.com

CLIs for Humans

APIs for Machines NETCONF/RESTCONF, gRPC Open/Standard RFC6241, OpenConfig

WSMA For legacy platforms

Beginners: • Code Academy (interactive) https://www.codecademy.com/learn/python • Coursera (online) https://www.coursera.org/learn/interactive-python-2 • TutorialsPoint (online) http://www.tutorialspoint.com/python/ • Python.org (online) https://docs.python.org/3/tutorial/index.html • The Hard Way (eBook) https://learnpythonthehardway.org/ • Dive into Python 3 (online) http://www.diveintopython3.net/ Intermediate: • Google (eBook) https://developers.google.com/edu/python/?hl=en • Crash Course (online) https://stephensugden.com/crash_into_python/

https://learninglabs. cisco.com/labs with NETCONF and YANG tags

• The tail-f education page is an excellent source http://www.tail-f.com/education/ View the 3 recordings under “NETCONF and YANG Tutorial Series”

• Here is a short reference

http://www.tail-f.com/wordpress/wp-content/uploads/2014/02/Tail-f-instant-YANG.pdf

• Tail-f training video collection

http://www.tail-f.com/confd-training-videos/

• In-depth NETCONF/YANG Tutorial deck

http://www.slideshare.net/tailfsystems/netconf-yang-tutorial

• NETCONF Central: http://www.netconfcentral.org/

• Model-Based Management Jive Page: https://cisco.jiveon.com/groups/model-based-management

© 2017 u and/or its affiliates. All rights reserved. Cisco Public http://www.yang-central.org Tooling – Exploring and using NETCONF/YANG • Editor plug-ins • emacs (yang-mode.el) • vim (yang.vim) • sublime text (sublime-yang-syntax)

• pyang • an extensible YANG validator written in Python. (Video trainining: pyang) • can be used standalone as a validator of YANG modules, or to generate YIN, YANG, DSDL and XSD from YANG and YIN. • https://github.com/mbj4668/pyang • http://www.yangvalidator.com/

• libsmi • A library allowing the generation of YANG models from SMI/SMIv2 compliant MIBs

• ncclient • a Python library that facilitates client-side scripting and application development around the NETCONF protocol (only supports NETCONF 1.0)

• Postman • a Chrome plugin for RESTCONF, allowing for customized sets of REST snippets to be easily built, maintain and shared. Useful for NETCONF via RESTCONF, for example Open Daylight

• OpenDaylight • enables auto-generation of RESTconf APIs from YANG models, with NETCONF client support • APIdocs feature provides a way to explore both local and mounted YANG models

Starting point: https://developer.cisco.com/site/ydk/ GitHub

• YDK Python API – YDK-Py (https://git.io/vaWsg)

• YDK-Py sample apps (https://git.io/vaw1U)

• YDK Generator – YDK-gen (https://git.io/vaw1M)

• YANG Explorer (https://git.io/vg7Jm)

DevNet

• YDK at DevNet (https://goo.gl/Wqwp3C) - Live by Apr 25

• Cisco IOS XR 6.0 at DevNet (https://goo.gl/uaxrpN)

• Ansible Network Automation: ansible.com/network-automation

• IOS Modules: docs.ansible.com/ansible/list_of_network_modules.html#ios

• Ansible and Cisco: ansible.com/ansible-cisco

• Tools on GitHub https://github.com/cisco/bigmuddy-network-telemetry-stacks https://github.com/cisco/bigmuddy-network-telemetry-collector https://github.com/cisco/xr-telemetry-m2m-web • Demos and Labs https://dcloud-cms.cisco.com/?p=22317 (dCloud telemetry lab) https://www.sdxcentral.com/resources/sdn-demofriday/cisco-ios-xr-signalfx-demo-monitoring-your-modern-network/ (demo with signalFX) https://youtu.be/F_S9-ctNFe0

Use Cisco Spark to communicate with the Speaker and fellow participants after the session

Download the Cisco Spark app from iTunes or Google Play

1. Go to the Cisco Live Melbourne 2017 Mobile app 2. Find this session 3. Click the Spark button under Speakers in the session description 4. Enter the room, room name = BRKSDN-2119 5. Join the conversation!

The Spark Room will be open for 2 weeks after Cisco Live

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 86 Complete Your Online Session Evaluation Give us your feedback and receive a Cisco Live 2017 Cap by completing the overall event evaluation and 5 session evaluations.

All evaluations can be completed via the Cisco Live Mobile App.

Caps can be collected Friday 10 March Learn online with Cisco Live! at Registration. Visit us online after the conference for full access to session videos and presentations. www.CiscoLiveAPAC.com