Open-Source NFV: OPNFV and ONAP

#CLUS Open-source NFV: OPNFV and ONAP

Frank Brockners BRKSDN-2333

#CLUS Cisco Webex Teams

Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How 1 Find this session in the Cisco Events App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKSDN-2333 by the speaker until June 18, 2018.

Toby Ford, FD.io Mini-Summit Sept, 2016

Service Model WorkFlow Topology App Intent

VM Policy, Network Policy Service Provisioning Service Provisioning, Service Workload Placement Service/WF Life Cycle Configuration Service Chaining, Service Monitoring Service Configuration Manager Auto Recovery, Elastic Scaling, Workload Placement, Service Assurance VM/Container Policy Service Monitoring Virtual Machine/Container Auto Healing Life Cycle Manager Elastic Scaling

Phys./virtual Network Control Network Controller; Group Policy, Chaining Hypervisor/Host//Container High-Performance IO Abstraction & Feature Compute Network Storage Flexible Feature Paths Path

Additional

Application Layer / App Server PaaS platforms PaaS

Network Data Analytics

Orchestration

Virtualization Control

Cloud Cloud Infra Tooling& Network & Connectivity

Operating Systems

IO Abstraction & Feature Path

Infrastructure Hardware

Integrate/Deploy/Test & Tooling CI/CD

Application Layer / App Server PaaS

Network Data Analytics ONAP

Orchestration

Virtualization Control OPNFV • Compose/Integrate Cloud Cloud Infra Tooling& Network & Connectivity • Deploy

Operating Systems • Test

IO Abstraction & Feature Path • Evolve

Infrastructure Hardware • Iterate

Integrate/Deploy/Test & Tooling CI/CD

• OPNFV – NFVI, CI/CD, Tooling

• ONAP – Design, Orchestration

• A glimpse at ONAP as a User

#CLUS BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 Open Platform for NFV (OPNFV) facilitates the development and evolution of NFV components across various open source ecosystems.

Through system level integration, deployment and testing, OPNFV creates a reference NFV platform to accelerate the transformation of enterprise and service provider networks.

• OPNFV performs System Integration to create an NFV- Infrastructure (NFVI) layer as an open community effort:

• Create/Evolve Components (together with Upstream Communities) Integration Testing New Features • Compose / Deploy / Test

• Iterate (in a distributed, multi-vendor CI/CD system) Alignment Functional • Compose: NFVI solution stacks for VMs (OpenStack based) and Installation System NFV Features Containers (Kubernetes based)

• Approx. 50 different stacks (“scenarios”) deployed/tested in CI/CD Composition Performance pipeline

• Includes stacks with FD.io/VPP (see e.g. FastDataStacks project)

• Deploy and Test: Maintain CI/CD pipeline for systems level deployment and test Continuous Integration / Continuous Deployment

• Several install tools and a wide variety of testing tools used Documentation • Operate: NFVI operations support tools development

• Calipso (monitor a deployed NFVI stack), NFVbench (performance Security benchmarking of an NFVI stack)

• Project launched: September 2014 • Since January 2018 OPNFV is part of the Linux Foundation Networking Fund (LFN) • Cisco Technical Steering Committee (TSC) member: Frank Brockners

• Number of code contributing organizations: 39 • OPNFV contribution stats details: OPNFV Bitergia

• Number of releases so far: 5 • Approx. 50 different solution stacks continuously (Ci/CD) composed, deployed, tested

• Releases in 2018: “Fraser” (May) + “Gambia” (October)

• Number of Projects: 30 (for Beijing)

Iterate.

NFV Telemetry Data Acquisition Network Data Analytics (VNF Event Stream, Barometer)

NFV deployment architectures MANO solutions Orchestration (Multi-Site, EdgeNFV) Opera (Open-O), Orchestra (OpenBaton)

Fault Management/Localization, Audit Resource Control/Scheduling Model and Policy Driven Control (Doctor, Pinpoint, Prediction, (Copper, Movie, Models, Domino, Virtualization Control Bottlenecks, Inspector) (Promise, Resource Scheduler (RS)) Parser)

Cloud Cloud Infra Tooling& Network Policy NFV services (service chaining, VPN, ..) SDN Controller Performance Test Network & Connectivity (FastDataStacks, Copper, PolicyTest) (VNFFG, SFC, SDNVPN) (Cperf)

Hypervisor for NFV Operating Systems (KVM4NFV)

Dataplane Performance Test Dataplane Evolution & Test IO Abstraction & Feature Path (Vsperf, Fastpath) (DPACC, OVSDPDK, FastDataStacks)

Infrastructure Infra control & CI/CD Hardware (Pharos, Releng, Octopus)

Create Scenarios/Stacks Installation, Upgrade Documentation Integrate/Install (IPv6, FastDataStacks, Armband, Apex, Daisy, (Apex, Joid, Fuel, Compass, Daisy, (OPNFVdocs) ONOSFW, OpenContrail,Joid, Fuel, Compass) Escalator) System Testing Security

CI/CD Storage Performance Test System Test & Tooling (Yardstick, FuncTest, Qtip, NFVbench, Calipso) (Storperf) (Moon)

Present Solution & Reach & Integrate, Architecture demos, Requirement review develop test and & Gaps collect internally upstream document feedback

• Goal: • Develop and build fault management and maintenance framework for high availability of Network Services running on top of virtualized infrastructure.  Proposed with a very clear target / key feature: • Immediate notification of unavailability of virtualized resources from VIM to Consumer • Members: • NEC (PTL: Ryota Mibu), AT&T, Cisco, Cloudbase Solutions, Corenova, Ericsson, Hephaex, Huawei, Intel, KDDI, KT, Nokia, NTT DOCOMO, Spirent, Sprint, Telecom Italia, Vmsec, ZTE

• https://wiki.opnfv.org/display/doctor/

ARNO (May/2015) BRAHMAPUTRA (March/2016) COLORADO (Sept/2016) DANUBE (March/2017) - Requirements document - Ceilometer “Immediate - Nova: - Neutron “Port Status update” Notification” “Get valid server state” - Inspector design guidelines - Nova “Mark Host Down” - Integration of Congress - Performance profiler - Functional test cases as Doctor Inspector - Documentation updates - PoC demo at OPNFV - Extended functional tests Summit - PoC demo at OPNFV Summit and - Documentation updates OpenStack Summit Barcelona - Documentation updates

• Create a new stack which significantly evolves networking for NFV: Introduce Solution Stacks with FD.io/VPP

• OpenStack – ODL (Layer2) – VPP

• OpenStack – ODL (Layer3) – VPP

• OpenStack – VPP

• Work areas:

• OpenStack (ML2 driver: networking-vpp)

• ODL enhancements (GBP Neutron Mapper, VPP Renderer)

• FD.io - VPP/Honeycomb enhancements

• OPNFV Installer integration (APEX)

• OPNFV System-level testing

• Contributors:

https://wiki.opnfv.org/display/fds

• VPP: Universal data-plane for all use-cases – NC/Y REST gRPC ... cloud, NFV, etc. Management Agent • Consistent, predictable behavior and latency

• Highest performance in the industry: 18+ Mpps/Core

• Wide range of features supported

• Very easy to extend (can even support proprietary plugins)

• Runs 100% in userspace (DPDK supported): Enables rapid upgradability, high availability, no system call overhead, no dependency on Linux kernel networking community for features

• Industry proven: Shipping since 2002 as part of both embedded & server products, in volume Packet Processing: VPP

Network IO See also: https://fd.io/ (The Fast Data Project)

Hardware Platforms Routing Switching Network Services IPv4/IPv6 Pure Userspace - X86,ARM 32/64,Power VLAN Support DHCPv4 client/proxy 18+ MPPS, single core Raspberry Pi Single/ Double tag DHCPv6 Proxy Hierarchical FIBs L2 forwd w/EFP/BridgeDomain concepts MAP/LW46 – IPv4aas Multimillion FIB entries VTR – push/pop/Translate (1:1,1:2, 2:1,2:2) MagLev-like Load Source RPF Interfaces Mac Learning – default limit of 50k addr Identifier Locator Addressing Thousands of VRFs DPDK/Netmap/AF_Packet/TunTap Bridging NSH SFC SFF’s & NSH Proxy Controlled cross-VRF lookups Vhost-user - multi-queue, reconnect, Split-horizon group support/EFP Filtering LLDP, BFD Multipath – ECMP and Unequal Cost Jumbo Frame Support Proxy Arp, Arp termination Policer Multiple million Classifiers – IRB - BVI Support with RouterMac assigmt Arbitrary N-tuple Language Bindings Segment Routing Flooding, Input ACLs Interface cross-connect SR MPLS/IPv6 C/Java/Python/Lua L2 GRE over IPSec tunnels Inband OAM (IOAM) Including Multicast Telemetry export infra (raw IPFIX) Tunnels/Encaps Security iOAM for VXLAN-GPE (NGENA) LISP SRv6 and iOAM co-existence GRE/VXLAN/VXLAN-GPE/LISP-GPE/NSH LISP xTR/RTR Mandatory Input Checks: iOAM proxy mode / caching IPSEC L2 Overlays over LISP and GRE encaps TTL expiration iOAM probe and responder Including HW offload when available Multitenancy header checksum Multihome L2 length < IP length Map/Resolver Failover ARP resolution/snooping Monitoring MPLS Source/Dest control plane support ARP proxy MPLS over Ethernet/GRE Map-Register/Map-Notify/RLOC-probing SNAT Simple Port Analyzer (SPAN) Deep label stacks supported Ingress Port Range Filtering IP Flow Export (IPFIX) Per interface whitelists Counters for everything Policy/Security Groups/GBP (Classifier) Lawful Intercept

IPv6, 24 of 72 cores IPv4+ 2k Whitelist, 36 of 72 cores Zero-packet-loss Throughput [Gbps]] [Gbps]] for 12 port 40GE 600.0 600 Hardware: 400.0 400 Cisco UCS C460 M4 200.0 200 Intel® C610 series chipset 0 0.0 1518B 4 x Intel® Xeon® Processor E7-8890 64B 64B v3 (18 cores, 2.5GHz, 45MB Cache) 2133 MHz, 512 GB Total 9 x 2p40GE Intel XL710 18 x 40GE = 720GE !! 480Gbps zero frame loss IMIX => 342 Gbps,1518B => 462 Gbps Latency 18 x 7.7trillion packets soak test [Mpps] [Mpps] Average latency: <23 usec 300 300.0 Min Latency: 7…10 usec 200.0 200 Max Latency: 3.5 ms 100.0 100 Headroom 0.0 1518B 0 64B Average vector size ~24-27 64B Max vector size 255 Headroom for much more throughput/features NIC/PCI bus is the limit not vpp 200Mpps zero frame loss 64B => 238 Mpps

#CLUS BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 NFV is about forwarding delay: VPP means low delay Min Delay • Low long-term max packet delay Average Delay Max Delay with FD.io VPP < 10 usec < 23 usec < 3.5 msec • 0.007ms < 0.023ms < 3.5ms

• Other vSwitches • >120ms long term max delay

• Tests environment • stock Ubuntu 14.04.03 LTS Kernel: 3.13.0-63-generic (no Linux tuning) • Cisco UCS C460 M4 (4 x Intel® Xeon® Processor E7-8890 v3 (18 cores, 2.5GHz, 45MB Cache), 9 x 2p40GE Intel XL710)

• Integrate VPP into existing OPNFV scenarios Install Tools Apex Apex Apex • Initial scenarios • OpenStack – ODL (Layer2) – VPP VM Control OpenStack OpenStack OpenStack • OpenStack – ODL (Layer3) – VPP OpenDaylight OpenDaylight • OpenStack – VPP Network Control L2 L3 • ...

• Diverse set of contributors: Hypervisor KVM KVM KVM

Forwarder VPP VPP VPP

• https://wiki.opnfv.org/display/fds

• OpenStack ...

• Networking-vpp ML2 driver: Neutron https://github.com/openstack/networking-vpp ML2-driver: networking-vpp • FD.io • VPP – Enhancements etcd • CSIT – VPP component tests

• OPNFV • Overall System Composition – Integration into CI/CD

• Installer: Integration of VPP into APEX ML2-agent • System Test: FuncTest and Yardstick system test VPP application to FDS DPDK

System Install System Test (APEX) (FuncTest, Yardstick)

See also: FDS Architecture: https://wiki.opnfv.org/display/fds/OpenStack-ODL-VPP+integration+design+and+architecture

Neutron Server

ML2 VPP journaling Mechanism Driver

VM VM VM VM VM VM HTTP/json

vhostuser vhostuser

VPP VPP VPPAgent dpdk VPPAgent dpdk Compute Node Compute Node

vlan / flat network

Internet

External network i/f Controlnode-0

OVS (br-ex)

OpenStack Services qrouter (NAT) Computenode-0 Neutron Server Computenode-1 ML2 VPP tap Mechanism Driver

tap Bridge ML2 Agent DHCP ML2 Agent ML2 Agent Domain VPP

Tenant network i/f

vhost- Bridge Bridge vhost- VM 1 user Domain Domain user VM 2

VPP VPP

Tenant network i/f networkTenant i/f networkTenant

VLAN / Flat network

• Network types • Layer 3 Networking

• VLAN: supported since version 16.09 • North-South Floating IP

• VXLAN-GPE: supported since version 17.04 • North-South SNAT

• East-West Internal Gateway • Port types

• VM connectivity done using fast vhostuser interfaces • Robustness

• TAP interfaces for services such as DHCP • If Neutron commits to it, it will happen

• Component state resync in case of failure: recovers • Security from restart of Neutron, the agent and VPP • Security-groups based on VPP stateful ACLs

• Port Security can be disabled for true fastpath

• Role Based Access Control and secure TLS connections for etcd

• JSON Web Token (RFC 7519) with X.509 Certificate

• Tap as a Service See also: https://github.com/openstack/networking-vpp

1 5 networking-vpp/nodes/vpp- rocks/ports/c367e21f-ae39-4549-b87d- Request Notification 2e69636155c6

{"allowed_address_pairs": [], "segmentation_id": 194, "mtu": 1500, "binding_type": "plugtap", "physnet": "physnet", "mac_address": "fa:16:3e:03:ce:ff", "port_security_enabled": Neutron Server 1 false, "fixed_ips": [{"subnet_id": "006fce47- 6072-4099-a695-c3caa140fff7", "ip_address": "10.0.0.2"}, {"subnet_id": "81b2fbdc-c350- ML2 VPP 4f35-9b9b-909cf33a4426", "ip_address": Mechanism Driver "fd59:3bf6:c35d:0:f816:3eff:fe03:ceff"}], "network_type": "vlan", "security_groups": []} VM 2 3 2 vhostuser 5 3 VPP

dpdk VPPAgent 4 /networking-vpp/state/vpp-rocks/ports/d2069a46-3a47-4ec7- 94fb-3b1bcd4c6dc0 Compute Node {"net_data": {"segmentation_id": null, "if_physnet": "tap-2", "bridge_domain_id": 3, "if_uplink_idx": [3], "network_type": "flat", "physnet": "physnet"}, "bind_type": "plugtap", "ext_tap_name": "tapd2069a46-3a", "mac": "fa:16:3e:5d:fe:c4", "bridge_name": "br-d2069a46-3a", "int_tap_name": "vppd2069a46-3a", "iface_idx": 6} #CLUS BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 OPNFV FastDataStacks project – Timeline...

Colorado 1.0 Colorado 3.0 Danube 1.0 Danube 2.0 Euphrates 1.0 (Sep/2016) (Dec/2016) (March/2017) (May/2017) (November/2017) - Base O/S- - Enhanced O/S- - Enhanced O/S- - Enhanced O/S- - Enhanced O/S- ODL(L2)-VPP ODL(L2)-VPP stack ODL(L3)-VPP ODL(L3/L2)-VPP ODL-VPP stack: stack (Infra: (Infra complete: stack (Infra stack: HA for DVR with VPP Neutron / GBP Neutron / GBP Mapper complete: OpenStack and (pure L3 for east- Mapper / GBP / GBP Renderer / VBD Neutron / GBP ODL (clustering) west and north- Renderer / VBD / / Honeycomb / VPP) Mapper / GBP south Honeycomb / - Enhanced system-level Renderer / VBD / communication); VPP) testing Honeycomb / Dynamic VXLAN- - Automatic Install - L2 networking using VPP) tunnel control via - Basic system- ODL (incl. east-west - L2 and L3 LISP level testing security groups), L3 networking - L2 networking networking uses using ODL (incl. Faser 2.0 (May/2018) using ODL (no qrouter/OVS east-west - Enhanced O/S- east-west security security groups) VPP stack: Native groups), L3 - O/S-VPP (Infra: L3 (via VPP), networking uses Neutron ML2-VPP / VXLAN, L3 router qrouter/OVS Networking-vpp-agent / HA - Overlays: VXLAN, VPP) VLAN - Automatic Install, Overlays: VLAN #CLUS BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 Building/evolving features means contributing upstream Team Up To Drive Required Change Upstream

Swimming upstream Swimming upstream Team-up to increase is hard is dangerous the likelihood to succeed

Iterate.

Integrate & build* Test on as a system reference system(s)

Lab3 Install on Lab2 reference Lab1 system(s)

*OPNFV currently composes builts from existing artifacts (e.g. RPMs) rather Lab1 Lab3 Lab2 OPNFV CI/CD Projects: than builds from source Octopus, Releng

Google Cloud Storage

Patch Verified Poll changes Upload ISO +1/-1 Clone repo

Build

yes new Build changes? Build

Build Server

Patchset Push Image Verification no

Docker Hub

Google Cloud Docker Docker Storage Hub Hub

Pull Down Download Pull Down Functest Image ISO Yardstick Image

Labs connected 24x7 using JNLP

Initiate Deploy Functest Yardstick Run for certain Scenario Jumphost

Pharos POD (SUT)

Project Team OPNFV Gerrit OPNFV CI/Test OPNFV Release Upstream Gerrit Upstream CI/Test Upstream Release

Requirement

Patch Test Test Release Fast feedback Document

Downstream

Test Release • FastLong development development cycle cycle • FastDownstream feedback, will < delay1 day to next release • OPNFVSlow feedback, specific > issues 5 months can be • tested/detectedOPNFV specific inissues time; cannot be • Correctiontested/detected in same in time release

contribute contribute contribute contribute

pull from master, deploy, test and/or verify patchset, post feedback

• Goals:

• Facilitate collaborative testing

• Provide developers with substantial resources

• Ensure OPNFV applicability across architectures, environments and vendors https://www.opnfv.org/developers/pharos • Create more robust, https://wiki.opnfv.org/display/pharos/Pharos+Home interoperable releases

• Facilitate collaborative testing

• Ensure OPNFV applicability across architectures, environments and vendors

• Create more robust, interoperable releases https://www.opnfv.org/developers/pharos https://wiki.opnfv.org/display/pharos/Pharos+Home

Lab as a Service Resources booked via dashboard

Dedicated Resources

Installer X Scenario A

Dedicated Resources

Resources Dynamically Allocated

Installer Y Scenario B

NFV orchestration

Virtualized Network Functions VNF VNF VNF VNF

Cloud Management Infrastructure Virtualization Compute Storage Network

Functional Performance Stress Security

Infrastructure Network Network Upgradability Features Storage Storage Components Compute Compute Backup&Restore VNF Virtual layer … . MANO Traffic gen. . … … .

#CLUS BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 Test: Functional Testing: Example Project FuncTest vPing SSH ODL suite Promise Verify private & public connectivity Robot framework, ODL Resource reservation functional testing and management project vPing userdata Verify nova-metadata ONOS suite Doctor service and private TestON framework Fault management and connectivity maintenance project

Tempest test BGPVPN OpenStack native tests vIMS (200+ tests) OpenSource solution by OpenStack Clearwater Neutron BGPVPN project integration Rally bench tests Benchmark the OpenStack deployment

#CLUS BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 Operations and Troubleshooting support: Operating OpenStack Networking is not that simple https://www.openstack.org/videos/video/troublesho oting-neutron-physical-and-virtual-networks (“Operations War Stories” from OS Summit)

• Solves virtual networking discovery issues

• Monitoring support for ‘day2 net operations’

• Discover, Analyze, Monitor, Visualize

• Substantially lowering administration overhead for maintenance and troubleshooting cycles in cloud environments ( < 90% for discovery, analysis and monitoring of v-nets)

• Highly Customized and Modeled

• calipso.io and OPNFV Calipso documentation https://git.opnfv.org/calipso/

• Auto-discovery of virtual networking components, their detailed data for K8S and OpenStack

• Auto-discovery and end-to-end monitoring for virtual-physical-virtual for ACI/APIC use-cases

• Analysis of inter-connections and dependency (links) with embedded impact analysis

• Monitoring of virtual networking objects and links for health and status, some statistics

• Visualization of topologies, statuses, settings, browsing tree and an embedded search engine.

• API for Eco-system integrations.

• Support multi-region and multi-tenancy in multiple OpenStack, K8s environments

• Integrated monitoring and a plug-in framework, ready for many type of devices

• A unified ‘all-in-one’ Installer for simple deployment.

• Modeled, Federated and customized, ‘model driven design’ and ‘micro-services’ architectures.

• Full-Stack NFV blackbox NIC NFVbench (with Trex) NFV-benchmark: NFVbench tool • Different connection scenarios NIC VPP VNF1 Compute 1 HoneyComb Nova (PVP, PVVP, ..) • Different traffic patterns UCS-B • NDR/PDR detection Fabric NIC VPP VNF2 Compute 2 Switch HoneyComb Nova • Different VNFs

• Compare a FastDataStack to NIC VPP Controller legacy stacks with other virtual HoneyComb Controller UnderCloud switches, e.g. OVS

• NFVbench documentation NIC Jumphost OverCloud

• https://git.opnfv.org/nfvbench/ FastDataStack full NFV stack deployment

#CLUS BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56 Resource Allocation Network Intelligence, Guidance What the Resource Usage POLICY Data Access ENGINE industry had Control to invent APPLICATIONS ANALYTICS & PaaS

REACTIVE, COMPUTE

NETWORK, STORAGE AND Statistics, States, SECURITY – Workflow Objects and Events MODULAR, COMPOSABLE SERVICE ORCHESTRATION AND CONTROL ARCHITECTURES NETWORK

Programmability

Design-Time Run-Time

Dashboard/(G)UI External APIs / API Gateway Single Pane of Glass Orchestrator Inventory Analytics Infra- Installation, Design & Event/Data Movement & Storage Monitoring; Creation VNF Life Assurance Network Controller VM/Container Manager Cycle/Placement/Capacity Security

Virtualized Functions Policy Physical Network Storage Compute

System Integration

Single Pane Dashboard/(G)UI External APIs / API Gateway Infra- of Glass Monitoring/ Assurance Security Orchestrator Inventory Analytics

Design & Creation Event/Data Movement & Storage

Network Controller VNF Life Cycle/Placement/Capacity VM/Container Manager

Policy Virtualized Functions

Physical Network Storage Compute

System Integration

Application Layer / App Server PaaS

Network Data Analytics ONAP

Orchestration

Virtualization Control OPNFV • Compose/Integrate Cloud Cloud Infra Tooling& Network & Connectivity • Deploy

Operating Systems • Test

IO Abstraction & Feature Path • Evolve

Infrastructure Hardware • Iterate

Integrate/Deploy/Test & Tooling CI/CD

ONAP community now represents 60% of global subscribers

Note: With the move to Linux Source: http://www.lightreading.com/automation/onap-adds-verizon-claims-de-facto-title/d/d-id/739689?itc=lrnewsletter_mobiledaily Foundation Networking Fund*, there is nolonger a dedicated ONAP board

• Project launched: March, 2017 Top contributors

• Governance: Technical Steering Committee (TSC)

• Cisco TSC member: Frank Brockners

• Number of contributing organizations in 2017: 22 –Total Members 60

• Number of releases so far: 1 (“Amsterdam” - November 20, 2017)

• Releases in 2018: “Beijing” (June) & “Casablanca” (December)

• Number of Projects: 30 (for Beijing)

• ONAP is now a project within the Linux Foundation Networking Fund – LFN. Source: https://onap.biterg.io/app/kibana#/dashboard/Overview?_g=() • LFN provides a single board for the LF Networking projects. Currently those are: ONAP, OPNFV, OpenDaylight, FD.io, PNDA.io, SNAS.io

Single Pane Dashboard/(G)UI External APIs / API Gateway Infra- Usecase UI Monitoring/ of Glass External API Framework ONAP CLI Assurance

Orchestrator Inventory Analytics OOM: Service Orchestrator (SO) A & AI (Active and Available Inventory) DCAE (Data Collection, Analytics, and Operations (BPMN & AriaTOSCA) Events) Manager Holmes (alarm correlation & analytics) CLAMP Design & (design & Creation DMAAP* AAF Auth. Framework manage Microservices Bus (MSB) Event/Data Movement & Storage Logging control loops) Modeling Network Controller VNF Life Cycle/Placement/Capacity VM/Container Manager Service Design & SDN-C* (SDN controller) APPC (Application/VNF controller) Creation (SDC) Multi VIM/Cloud Common Controller SDK VF-C (Virtual Function controller) Common Controller SDK Policy VNF Requirements Virtualized Functions VNF SDK Policy Framework Physical Network Storage Compute

Documentation System Integration Integration

• Merges OpenO and OpenECOMP code bases

• Key use-cases • Residential vCPE – ATT driven • VoLTE – ChinaMobile driven

• VM-based installation option only: onap.readthedocs.io

• 2nd release (Beijing) focuses on platform maturity: https://wiki.onap.org/display/DW/Beijing+Release+Platform+Maturity

ONAP “Beijing” Architecture: High-Level View

(Utilities)

Integration

VNF Requirements

Modeling VNF Validation Program VNF Validation

• Deploy ONAP Open-Source Components • Typical solutions will leverage a subset of all the available ONAP components • Typical solutions will combine ONAP open-source components with commercial components • Complement ONAP components • Replace ONAP components with commercial components

• ONAP as an industry-standard architecture • Use ONAP as a reference architecture to create a multi-vendor deployment from commercial components

Single Pane Dashboard/(G)UI External APIs / API Gateway Infra- of Glass Monitoring/ API-Gateway Assurance Security Orchestrator Inventory Analytics

Crosswork Network Crosswork Change Automation Data Platform Design & Creation Event/Data Movement & Storage Crosswork NFVI Network Controller VNF Life Cycle/Placement/Capacity VM/Container Manager

NSO, VTS, ACI, XTC NSO, ESC CCP, Cisco VIM (Mercury) Policy

CSR1kv, vASA, ... Virtualized Functions

Physical Network Storage Compute ASR, NCS, Nexus, ... UCS,..

System Integration Advanced Services

• ONAP Operations Manager (primary choice for Beijing) • Life-cycle manager for ONAP components • Leverages Kubernetes (deployment & scheduling) and Consul (health monitoring) • Deployment, Configuration, Monitoring, Restart, Clustering and Scaling Upgrade, Deletion of ONAP components

• Heat template (legacy)

• Note: ONAP requires a pre-installed cloud to schedule VNFs on

See also: https://wiki.onap.org/display/DW/ONAP+Operations+Manager+Project

• Collaborative, catalog-driven “self-service” Catalog management design studio Onboard vendor provided Network • Define resources, services, and products Functions • Create and manage models, processes, policies, Design Services and and analytics for creation and lifecycle management Operations • Systematic evaluation, certification, and Test, certify, and distribute models onboarding of technology supply chain for Runtime Execution

• Institutionalize content and models for consistent implementation and technology Catalog insertion

• Single platform to define and deploy Policy instantiation, management, and control definitions and behaviors Policy Creation & Validation

• Import Management Functions • Design Service & Operations Methods

• Import ONAP base capability definitions from • Create Service models with resources Development Catalog to Design Catalog as • Design and associate operations processes & building blocks in a standard format policies (e.g., Instantiation, DCAE/Control Loop, Change Management, etc.) and configure them to • Create Flows & Policies be service specific • Create reusable management flows using building blocks & associated events & Policies • Certification

• Simulate & test the design in Sandbox environment • Onboard Network Functions • Certify Readiness & Adherence to Standards • Create VF model to include vendor’s description, scripts, & license model • Metadata Distribution • Customize models to include Service Provider • Publish certified models for distribution to runtime specific attributes catalog

• Notifications & Version Control

Asset name Description Resource Combination of one or more Virtual Function Components (VFCs), along with all the information necessary to instantiate, update, delete, and manage the Resource. A Resource also includes license-related information. There are three kinds of Resource: • Infrastructure (the Cloud resources, e.g., Compute, Storage) • Network (network connectivity functions & elements); example: a Virtual Network Function (VNF) • Application (features and capabilities of a software application); example: a load-balancing function

Service Well formed object comprising one or more Resources. Service Designers create Services from Resources, and include all of the information about the Service needed to instantiate, update, delete, and manage the Service Product Products includes one or more Services packaged with commercialization attributes for customer ordering, billing, and issue resolution. Products are created by Product Managers, and can have one or more "category" attributes assigned by Product Developers

Other bundling of Products with specific Marketing configurations for selling to customers

User: “Superuser” User: “Designer” User: “Tester” User: “Governor” User: “Operator”

Roles with SDC focus

Virtual Vendor Vendor Licence Software Software Model Product Product creation onboarding testing

User: “Designer” User: “Designer” User: “Tester”

Service Service Service Service Creation Testing Approval Distribution

User: “Designer” User: “Tester” User: “Governor” User: “Operator”

Component name Description

Repository for assets at the Resource, Service and Product levels. Catalog Assets are added to the Catalog using the Design Studio.

Design Studio Used to create, modify, and add Resource, Service, and Product definitions in the Catalog.

Used to test new assets at all levels. It will be used for sandbox experimentation, Certification Studio and will include support for automated testing.

Used to deploy certified assets. From the Distribution studio, new Product assets, including their underlying Resources and Services, are deployed into lab environments for testing Distribution Studio purposes, and into production after certification is complete. In a future release, there will be a way to export Product information to external Business Support Systems for customer ordering and billing.

• Policy Framework and SDC are the key ONAP design time components

• Policy Framework • Provides a logically centralized environment for the creation and management of modifiable configurations, rules, assertions and/or conditions to provide real-time decision making on conditions and events that underlie ONAP’s control, orchestration, and management functions • Supports specification, decomposition, distribution and enforcement for various types of policies such as microservice configuration policy, operational policy, decision policy, guard policy, etc. • Policy scopes include, but are not limited to, infrastructure/network management, products and services, operation automation, and security.

• Provides functionality for • Execution of specified processes • Automated sequencing of activities, task, rules and policies • Drives creation, modification, removal of network application, infrastructure services or resources via associated controllers

• Supports different specializations with specific orchestration scopes • Specialization scopes include, but are not limited to, PNF orchestration, Service and VF scaling, Homing and placement.

• Model Driven Orchestration and APIs • Runtime behavior driven by service and resource models and policies (including compound/nested services) designed in or onboarded into SDC • Orchestrates service delivery, change management as well as open and closed-loop control actions • Provides model driven APIs for requested actions • Tracks orchestrated activity for the life of the request, but doesn’t maintain state of orchestrated components

• Processing of Service Requests • Performs Decomposition, Recipe Selection, Recipe Execution (including Rainy Day) • Triggers and Records Results for: Homing, Validation, Monitoring, Assign/Create/Configure • Separate execution threads for service, decomposed resources, and any subtending service(s) provide nested service orchestration in a recursive manner

• Orchestration of Controllers • Coordinates activities across Multi-Cloud/SDN-C/Generic NF controllers, including data sourcing and mapping to Controller inputs

• Workflow and model-based orchestration integrated • BPMN – Camunda • TOSCA Orchestrator - ARIA

• SO Operations • Invoked from VID (Dashboard and UI) or via APIs, consumes models from SDC • Interfaces with SDNC, APPC, VFC, AAI and other ONAP components such as OOF

• SO initially focused on 3 use cases (Amsterdam Release): • Virtual Firewall and Virtual DNS (based on BPMN workflow) • VoLTE (based on BPMN workflow) • Residential Broadband vCPE (HEAT/TOSCA orchestrator calling out to APPC)

Service Request

Service Orchestrator (SO)

SDNC APPC VFC … Controller

SDN-C configures and maintains the health of L1-3 VNFs/PNFs and network services throughout their lifecycle

• Programmable network application management platform • Local source of truth

‒ Behavior patterns programmed via models and policies ‒ Manages inventory within its scope

‒ Standards based models & protocols for multi-vendor ‒ All stages/states of lifecycle implementation ‒ Configuration audits ‒ Extensible SB adapter set supporting various network config protocols, including 3rd party controllers • Key Attributes of Controllers

‒ Operational control, coordinated state changes across ‒ Intimate with network protocols devices, source of telemetry/events, etc. ‒ Manages the state of services • Manages the health of network services & VNFs/PNFs in ‒ Single service/network domain scope per instance its scope

‒ Policy-based optimization to meet SLAs

‒ Event-based control loop automation to solve local issues near real-time

‒ Action executor for outer control loop automation

• Model driven service abstraction layer: • API handlers, • Operational and configuration trees, • An adapter framework for integrating with controlled devices, virtual functions, and cloud infrastructure. • Within this framework the ONAP Service Logic Interpreter (SLI) addition provides an extensible scripting language for expressing service logic through a Directed Graph (DG) builder based on Node-Red. The SLI is extended by adding Java classes that can be called as a node in a DG to support frequent complex operations

• Based on OpenDaylight Controller framework

Artifact Distribution Orchestration Closed Loop Actions Inventory Updates SDC SO DCAE A&AI

MSB/Data Movement CE-1 CE-2 CE-3 CE-4 SDNC API Handler CI-1 CI-2 CI-3 Repository Service Control Operational Tree/ Config Tree Processing (Service Model) NB Service/Network Yang Models CI-4 CI-5 Service Logic IP/VRF Assign Service L2 Service Create Configuration Templates Logic L3 VPN Service Create Service/Network Design & Service SD-WAN Create Engineering Rules LogicService TE Tunneling Assigned Resources Inventory: Policies Logic BGP Config CI-7 Service Topology & SB Device Yang Models SW Upgrade VNF/PNF State

MSB/DataMovement … CI-6 Adapters Multi-Cloud NetConf/ BGP LS/ External System OpenFlow Others Network Adapter YANG PCEP … Adapter (s)

MSB/Data Movement CE-6 CE-5 External 3rd Party Controllers VNFs, PNFs Element Mgt. Systems Multi-VIM/Cloud CE-x Controller External API CI-x Controller Internal#CLUS API BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 93 SDNC: SDN Controller Overview & Architecture (L0-L3) Directed Graph Builder

Directed Network Data Service Data Security Control Loop Service Graph Model Model Applications Applications Orchestrators Files – XML Files – YANG Files - YANG (Eng Rules) (i.e. IPAG EMT) (i.e.UNI port)

API (REST) API (REST) API (REST) Service-related Artifacts for SLI, API Handlers, Network Adpaters

API Handlers External API calls

A&AI

OpenDaylight Service Logic REST with Interpreter SDN-C SDN-C Database Inventory customizations Service Logic/Eng Rules Assigned Resource Config Operational Inventory Tree Tree

Network Adapters ADMIN OpenStack NETCONF BGPCEP Etc. Adapter Adapter Portal

• Provides reference implementation of NFVO in ETSI MANO architecture

• Provides Network service life cycle management

• Provides NS/VNF layer’s FCAPS management

NFVO Functions GVNFM Functions

• NS Lifecycle Management including NS • VNF Lifecycle instantiate, scale, heal, operate (query Management, including SO Policy UUI /update/…)and terminate. Most of VNF deploy, scale, heal, NSLCM interfaces align with SOL005 operate (start/stop/ Os-Ma-nfvo reference point restart/…), update and terminate, etc NFVO • Integration with multi VNFMs via drivers which include vendors VNFM and generic • Support multiple VNFs VNFM. The interfaces between NFVO and multi-type VNFs and driver comply with Or-Vnfm from different vendors SVNFM GVNFM EMS reference point. • Support multiple VIM • Support integration with multi VIM via environments and Multi-Cloud multi-type VM Multi-Cloud VIM environments based on • NFVO also supports integration with VM or Docker vendor EMS via driver

SO Policy UUI Os-Ma-nfvo VNF Package ( VNFD) NFVO Instantiate NS Package ( NSD) Terminate Scale

Operate Heal

Or- Vnfm Or- Vnfm VNF Package Driver ( VNFD) Driver SVNFM GVNFM EMS Specific VNF-Mgr Or-Vi Generic VNF-Mgr Vi-Vnfm Multi-Cloud Vi-Vnfm Vi-Vnfm NFVI&VIM

• Application Controller (APPC) performs the functions to manage the lifecycle of VNFs and their components. • It provides a comprehensive set of controller actions such as Configure, Modify Configuration, Start, Stop, Migrate, Restart, Rebuild, and so on. • It supports a set of standard VNF interfaces (Netconf, Chef, Ansible.) • Is designed to be self-service using a model driven architecture that provides a layer of abstraction making APPC completely service, VNF, and site agnostic.

• L4-L7 focus (complements SDNC)

• Life Cycle Management of VNF’s (service Typical Application Controller Actions (APIs) workflow recipes) Configure Healthcheck LiveUpgrade Start Install Query Stop Reconfigure Test • The APPC infrastructure is implemented on Rebuild Rollback Scale virtual machines in an OpenStack cloud in Restart Audit SoftwareUpload Terminate BlockAudits ActionStatus the Amsterdam release. APPC is created Migrate Sync ConfigErase on top of ODL ModifyConfig VNFLock ConfigSave Evacuate ConfigSetBaseline • ODL with some modifications (Service Logic Interpreter)

• Interfaces: LCM/REST & LCM/DMaaP (bus)

• Origin: AT&T ECOMP

ONAP Applications – e.g., portals, orchestrators (SO), workflow engines Service Operations A&AI SO Applications Applications

DMaaP Bus

Service Logic Interpreter SDC Policy Directed Graphs APPC OpenDaylight Container Repository

Provider Plugins NetConf Ansible OpenStack Analytics Function Adapters Adapter Adapter Adapter

VNF

VNF VNF VNF

• AAI as central ONAP registry: • Maintain a live view of services and resources in the network, providing the state and relationships of the service components • Maintain the view of the managed systems services and resources, as well as information of the external systems that ONAP will connect to. • Provide real-time views of a managed systems resources, services and relationships with each other.

• Access via REST API and GUI

• • SDC AAI • • •

ESR

AAI-Model Loader AAI-Sparky-FE AAI-Sparky-BE AAI-Babel Input Abstraction

AAI- AAI-Resources AAI-Traversal AAI-Gizmo CRUD End common Points

AAI-logging AAI-Data AAI-Search Data Query Abstraction AAI-router- Router Service core

AAI-rest- client AAI-Champ Database Abstraction

Elastic Search + Graph Data Store

• Open, plug-able platform for “sensing and making sense” for ONAP

• Functional requirements • Of ONAP • Interfacing with other ONAP components • ”Model driven” • DCAE service components are modeled • Generated events are modeled • Operations are modeled • For ONAP • Able to incorporate the best collection and analytics technologies into a catalog • Able to collect, analyze, and generate actionable events following the requirements of control applications, e.g. delay, bandwidth, resource constrains, etc

DCAE Platform Core DCAE Control Platform (Docker Hosts) ONAP Lifecycle Manager Dispatcher Service Change (Cloudify Manager) (NB API) Handler DCAE PGaaS Service & Config Registry Policy Inventory (Consul) Handler

DCAE Collector Services (Docker Hosts) DCAE Service Infrastructure CDAP Broker (CDAP + Haddop Cluster)

Collector Collector Collector Analytics Analytics Analytics (VES) SNMP Telemetry (TCA) (Data Norm) (Correlation)

DCAE Platform Components DCAE Service Components

#CLUS BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 109 • Cloudify Manager: Cloudify, through its army of plugins, is capable of relationship topology base resource orchestration in many levels and cross different technologies. It is the lifecycle management engine of DCAE. Various resource deployment, change, allocation, configuration, etc, operations are all done through Cloudify. [Note: This component is part of the Common Controller SDK, or ccsdk project.]

• Consul: Consul is a service discovery technology for distributed fault detection and KV store. DCAE uses Consul for service and configuration registry. [Note: This component is part of the Common Controller SDK, or ccsdk project. ]

• Service infrastructure: DCAE platform supports two kinds of infrastructures, the Docker container hosts and CDAP/Hadoop clusters. The former is for running containerized applications and services. And the latter is for running CDAP-Hadoop based big data analytics.

• Dispatcher: Dispatcher is a NB API provider for the DCAE Services. Service related triggers, such as deploying/undeploying services, changing configurations, etc all arrive at the Dispatcher, which then enriches the request, and invokes the right Blueprints and calling Cloudify Manager plugins to complete the necessary changes in virtual resources.

• Inventory: Inventory tracks DCAE related resource information such as various Blueprints and templates that are used by Cloudify Manager to deploy and configure components, as well as inventory information extracted from A&AI that is related to but not really part of DCAE, such as the relationships between virtual network resources and their physical infrastructures.

• PGaaS: Inventory is backed by a PostgreSQL database for data storage.

• Policy Handler and Service Changing Handler: They are the interfacing modules for specific external components such as Policy, SDC,..

• CDAP Broker: CDAP Broker interfaces between CDAP and Cloudify Manager, supporting carrying out various Cloudify CDAP operations onto the CDAP. See also: https://wiki.onap.org/pages/viewpage.action?pageId=3247121

DCAE Data Platform (PNDA.io) DCAE Collector Services (Docker Hosts) PNDA Deployment Manager

Collector Collector Collector * Analytics ZTT * Analytics (TCA) Analytics (VES) SNMP Telemetry (Data Norm) (Grafana) (Correlation) * Existing PDNA Capabilities

DCAE Platform Components DCAE Service Components

User: “Superuser” User: “Designer” User: “Tester” User: “Governor” User: “Operator”

Virtual Vendor Vendor Licence Software Software Model Product Product creation onboarding testing

User: “Designer” User: “Designer” User: “Tester”

Service Service Service Service Creation Testing Approval Distribution

User: “Designer” User: “Tester” User: “Governor” User: “Operator”

Service Instance

Virtual Function Virtual Function

VF VF ... VF VF ...... Module Module Module Module

Packet Firewall Sink Generator VF VF VF

• VF run Ubuntu 14.04

• FD.io/VPP used as data-plane

• HoneyComb used as control plane

Service Instance

VNF: Packet-Gen VNF: Firewall

Traffic Traffic Firewall Generator Sink VM VM VM

SDC SO AAI APPC SDNC DCAE Policy

VNF resource onboarding VNF service onboarding VNF distribution

Artifact distribution Artifact distribution

Artifact distribution

VNF Preload Data (from Beijing onwards)

Telemetry to collect, closed loop (CLAMP) triggers

Control loop policies

Multi Cloud Hyper SDC VID SO AAI SDNC vFW VIM API Visor

Service and VF deployment SDNC preloading (Amsterdam only) Look up artifacts Look up cloud locations

VF-Module deployment Instantiate base VF modules Inventory update Generic VNF API (reserved)

VF Heat template + Env file VF Heat template + Env file Instantiate VF infra

complete Nova/Neutron complete complete Generic VNF API (activated) L3 network resource update complete

Now we have to fill these profiles (requires manual REST calls in Amsterdam)

Service Instance ID: 5b7594e7-11a4-4680-964f-d135e685cb10

VNF-Type: VfwclVfwsnkVf..base_vfw..module-0

Generic-vnf-name: vFW_SINC

Generic-vnf-type: vFWCL_vFWSNK-vf 0

Status Code: 200 OK Content-Type: application/json Server: Jetty(8.1.19.v20160209) Transfer-Encoding: chunked

VNF ID: ae875a89-b89e-4527-a38f-e0a597d09e9d

IP address: 10.4.2.166 (get e.g. via OpenStack)

192.168.10.240

XCI: www.onap.org Develop.Integrate: Deploy.Test Design, Management & Orchestration: (OPNFV, ODL, ONAP, wiki.onap.org FD.io, PNDA.io, ...)

Develop.Integrate: www.opnfv.org NFV-I & Tooling wiki.opnfv.org

Give us your feedback to be entered into a Daily Survey Drawing. Complete your session surveys through the Cisco Live mobile app or on www.CiscoLive.com/us.

Don’t forget: Cisco Live sessions will be available for viewing on demand after the event at www.CiscoLive.com/Online.

#CLUS BRKSDN-2333 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 194 Continue Demos in Walk-in Meet the Related your the Cisco self-paced engineer sessions education campus labs 1:1 meetings

#CLUS #CLUS