BRKDCN-2219.Pdf

Data Center Design for Media and Entertainment Networks

Ammar Latif, Principal Systems Engineer, Sales Rahul Parameswaran, Technical Marketing Engineer

BRKDCN-2219 Cisco Webex Teams

Questions? Use Cisco Webex Teams to chat with the speaker after the session How 1 Find this session in the Cisco Events Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space

Ammar Latif Rahul Parameswaran Principal Engineer – Media Technical Marketing Engineer – INSBU

• Digital Disruption In Media

• SMPTE 2110 and Move from SDI to IP

• Cisco’s IP Fabric for Media

• Time Synchronization with PTP

• Post Production workflows

• Architectures and Customer Deployments

• Ready for today’s new formats and data Transition to IP 10110101 1001010110 rates from SDI 101010101010 • Increased bandwidth with Ethernet • Deterministic networking

• Move away from specialized, dedicated, x86 expensive HW Workloads • Towards general purpose, low cost compute and storage

• Software Defined Networking (SDN), not Cloud manual Architectures • Application Policy Driven Infrastructure • Composable pools of resources

fx PRODUCTION POST-PROD GRAPHICS PLAYOUT DISTRIBUTION

Nevion Video Lawo VSM Evertz Magnum Imagine SDNO IPath

REST REST REST REST

REST EVS S-Core Master

Cisco IP Fabric for Media REST • Open Scalable SMPTE 2110 Network • SDI-to-IP Network Infrastructure • Fully Non-Blocking Multicast Architecture • Open APIs GV Convergent • Multicast Flow Visibility • Realtime Telemetry & Analytics

Cisco IP Fabric for Media

• ST-2110 and ST-2022-6/7 Network • AES 67 Audio Selenio • DANTE Audio versio SNP100 • Mixed workflows (Streaming + File Based) • Compressed (MPEG, ABR, J2K)

Cisco IP Fabric for Media

• Master Control Facility Operations • Compressed / Uncompressed Workflows Selenio • AES67 / DANTE Audio versio SNP100 • Disaster Recovery • Playout • Program Replacement / Insertion

Control Systems with Control Panel Playout Cameras and Microphones Monitoring REST API Systems

Video Server Network Video Switcher Relay and Clips Controller

Graphic Multiviewer Systems

Remote Source Audio Mixer IP Fabric

Unchanged Operator Workflow

Zero Packet Loss Reservation of network resources across redundant paths for zero congestion loss

Video/ Audio End Point Sync Precision Timing and Synchronization

Fast Switching Switching streams with minimal delay

System Availability Same or better than SDI-based system

Network Security Protect network operations from any malicious attacks

• SMPTE ST 2022-6 : SDI Over IP. Composite Media • SMPTE ST 2110 suite: SDI to IP Essence-based media • SMPTE ST 2059-2: Timing. Based on IEEE1588 • SMPTE ST 2022-7 : Redundancy • AMWA NMOS: • IS-04: Device Registration and Discovery • IS-05: Connection Management: Connection mgmt between sender and receiver • IS-06 Network Control API (Work-in-progress): SDN API for Network Services

• Composite to Essence + Timing SMPTE ST 2110 2110-40 VANC SMPTE ST 2022-6 (SDI over IP) Ancillary Data

VANC 2110-20 Ancillary Data

Video

HANC Video Audio

21102110-30-30 HANC Audio Straight encapsulation of SDI frames into 1428 byte IP packet PTP

and more …..

Video Router Broadcast Controller

SRC DST

-SDI router sets up path

• SDI endpoints • Native IP Support • Legacy • Cameras • • Playout Systems Requires IPGs • Connect directly to the fabric • IPGs can become SDI aggregation point

A SDI IPG IP B Connectivity

IPG Per flow 1.5, 3 or 12Gbps IPG Per Port 10, 25, 40, 50 or 100Gbps

IP Fabric IP Fabric IP Fabric Broadcast Broadcast Controller Controller Broadcast Controller

IGMP Dst Src Src Dst Src Dst

-BC instructs destination to switch using IS-05 -BC instructs destination to switch using IS-05 or -BC instructs destination to switch using IS-05 or or other control means other control means and instructs the destination other control means and instructs the source, spine -End Point requests path setup using IGMP network switch to route new flow using API (in and destination network switch to route new flow -Network Fabric protocols sets up path parallel) using API (in parallel) -Network Fabric protocols sets up path -BC sets up the end to end path (no network fabric protocols involved)

Single Modular Chassis Flexible Spine and Leaf Fabric

1.5G HD 3G HD 12G UHD

ip pim rp-address 10.237.1.1 group-list 224.0.0.0/4

interface Ethernet1/1 ip router ospf 1 area 0.0.0.0 IGP

PIM for Multicast ip pim rp-address 10.237.1.1 group-list Layer 3 Routing 224.0.0.0/4 Uplinks

QoS interface Ethernet1/1 ip pim sparse-mode policy-map type queuing PRIORITY class type queuing c-out-8q-q7 priority level 1 IGMP interface Ethernet1/1 ip igmp version 3 ip igmp immediate-leave class type queuing c-out-8q-q5 1.5G HD 3G HD 12G UHD ip igmp suppress v3-gsq bandwidth remaining percent 10

• Unlike how SDI routers were defined (example:1024x1024), IP Fabrics are defined based on type of end points and their port requirements

• Consider the following example • Gateways ; Ports Required 36x40G • IP Camera’s ; Ports Required 150x10G • Audio, Intercomm ; Ports Required 50x1G Copper ports

Bandwidth from Leaf to each Spine > Bandwidth of senders on that leaf Bandwidth from Leaf to all Spines (aggregate) > Bandwidth of receivers on that leaf

• 9336FX2 can be used as a Leaf for 40G Gateways • Each supports up to 25x40G end point and requires 10x100G uplink

• 93180YC-EX/FX can be used as a Leaf for 10G Camera’s • Each supports up to 48x10G end points and requires 6x100G uplinks

• 9348FXP can be used as a Leaf for 1G Audio/Intercomm • Each supports up to 48x1/10G T end points with 2x100G uplinks

Number of End Points Platform/Quantity Uplinks Gateways- 9336FX2/2 20x100G 36x40G Camers- 93180-EX/4 24x100G 160x10G Audio and InterComm- 9348FXP 2x100G 70x1G

9508 Spine

6x100G Uplinks from each 93180 Leaf

10x100G Uplinks 2x100G Uplinks from each 9336 from each 9348 Leaf Leaf

9336 9336 93180 93180 93180 93180 9348 9348 Leaf 1 Leaf 2 Leaf 3 Leaf 4 Leaf 5 Leaf 6 Leaf 7 Leaf 8

• Protocol Independent Multicast(PIM) enables multicast routing across subnets/networks.

• When multiple path exist between Source and Receivers , traffic is load balanced (ECMP)

• ECMP uses hashing to determine which path is used for a flow

ECMP hashing could result in traffic loss as it

40G Link is not bandwidth aware!

UHD Video 12G Audio 1Mbps CC 100kbps

SMPTE 2110

S1 S2 S3 S4 R1(S1) R2(S2) R3(S3) R4(S4) There is a need for Intelligent Multicast Transport

• NBM brings bandwidth awareness to PIM • Traffic load balanced using flow bandwidth as a parameter • Prevents link oversubscription by ensuring flows more than link capacity is not sent

40G Link

UHD Video 12G Audio 1Mbps CC 100kbps

SMPTE 2110

S1 S2 S3 S4 R1(S1) R2(S2) R3(S3) R4(S4) NBM Brings Bandwidth Awareness to PIM ECMP

• NBM active mode (default) • NBM passive mode (shipping starting 9.3(2))

• NBM orchestrates flows in the fabric • NBM exposes API using which external controller can program end to end flow path from source to • NBM finds and programs a optimized non-blocking destination path from source to destination

• Destination either use IGMP or Broadcast controller Broadcast uses API to request for traffic Controller

API

PIM+NBM signaling between switches

• NBM tracks flows a receiver is subscribed to

• If operator accidently routes more sources to receiver than interface capacity – NBM gracefully denies

the request 3x3G flow on 10G lane Join request for 4th source

NBM denies the request

feature pim feature msdp feature ospf nbm flow-policy policy Anc_1 ip pim rp-address 192.168.100.1 group-list 224.0.0.0/4 bandwidth 10000 kbps ip group 239.100.160.221 router ospf 1 policy Audio_1 bandwidth 40000 kbps interface Ethernet1/33 ip group-range 239.30.11.1 to 239.30.11.10 ip address 192.168.1.1/31 ip router ospf 1 area 0.0.0.0 policy Video_1 ip pim sparse-mode bandwidth 1500000 kbps no shutdown ip group-range 239.20.11.1 to 239.20.11.10

feature pim ip pim rp-address 192.168.100.1 group-list 224.0.0.0/4

interface Ethernet1/49 ip address 192.168.1.0/31 ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown

Cisco Nexus

9200 Series NX-OS : NBM Process 9300-EX/FX/FX2 Series + 9500 -R Series

Cisco Nexus 9000 Series Cisco DCNM software (Optional)

From Cisco’s Data Center Portfolio

• Fabric Bootstrap : Day 0 Provisioning • Topology and Endpoint Visibility • End to End Flow Visualization • Policy Manager • API Gateway for Broadcast Controller • Network Health Monitoring

DCNM uses NXAPI REST to push policies/configuration to switch

NBM Process streams state information to DCNM using Streaming Telemetry

Permit/Deny end points Sender to mcast group White list policy Admission Flow Receiver to mcast group Black list policy Control Authorization

Define flow size Analytics & Monitor flow stats Flow Policing Sender policing & shaping Notifications Notify packet drops Monitor packet drops Notify security events

policy Video • Flow Policy defines bandwidth and QoS bandwidth 3000000 kbps ip group-range 239.1.1.0 to 239.1.1.255 parameters for flow

• NBM programs a policer that ensures traffic from an endpoint does not exceed what is allowed in the policy

• NBM uses bandwidth parameter in flow policy to

determine bandwidth reservation in the fabric 3Gbps

12Gbps

BRKDCN-2219 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 Host Policy Authorizing Endpoint to Send or Receive Traffic nbm host-policy receiver default deny • With SDI security was not an issue 5 host 100.1.18.2 group 239.1.1.1/32 permit

• With IP anyone with an IP device can send and IGMP join and subscribe to a feed or spoof a source

• Host Policy provides ability to restrict a sender Join request to send to certain flow(s) or a receiver to 239.1.1.1 subscribe to a certain flow(s)

Join request 239.1.1.2

Non-Blocking Fabric Visibility and Analytics Low Latency Information Rich Topology 1/10/25/40/100 Gbps Flow Visibility Flow Statistics

Plug-n-Play HA & Scale Day 0 Deployment Spine-Leaf Topology Image and Configuration Updates Flow Migration Multicast and PTP Scaling

Security Open Standards Endpoint Admission Control SMPTE 2022-6 Flow Authorization SMPTE 2110 Flow Policing PTP – SMPTE 2059-2, AES67 AMWA Network Control REST API

Video IPATH

DCNM Control Plane Media Controller Media APIs/NXAPI/Netconf

IPFM Spine Spine

Leaf Leaf Leaf

VLI UC GV XP Node NK IP

Legacy SDI Layer

• Two Independent Fabric • Two Independent Network Controller • Single Broadcast Controller • End point connected to both fabrics

Fabric A Fabric B

End Points

• Studios spread across buildings/locations Broadcast Controller • Have Leaf Switches at Each Studio • Aggregate it to Centrally Located Spine

Master Control Room Network Controller

IP Fabric

Studio A Studio B Studio C

• Remote Leaf Suitable when number of end point in Remote Site Small • Most Traffic is sent to Main Site Challenges: • Latency in Control Network between Network Controller and Remote Leaf • How will Broadcast Controller control end points in Remote Site • Needs dark fiber (No Service provided or router in between remote leaf and Spine) • PTP distribution : GM in main site

Main Site Remote Leaf

End Points End Points

• Multi-Site and PIM Border is Full Mesh MSDP a feature with which the IPFM fabric can interconnect with another site / external PIM Router • Uses border-leaf to interconnect with an external site/router • The sites that are interconnected work independently.

External PIM Network

Full Mesh MSDP

Site A

ExS ExR

Layer 3 ports

• Two Way Time Transfer protocol (TWTT) • Accuracy in a well designed E2E model in the nanosecond range • Boundary Clocks (BC) and Transparent Clocks (TC) aim correcting delay variations, in both directions (asymmetry) Ordinary Boundary Clock Transparent Ordinary Clock grandmaster Clock Clock Leaf Slave

PTP PTP Reference Recovered Clock Clock PTP

Transparent Clock Boundary Clock GM GM

Spine Spine

Leaf Leaf Leaf Leaf

Better Scale Lower Load on the GM

• AES67 (Audio driven) • Based on IEEE 1588-2008 default profile /w specific message rates • Announce: 1 {0,4} • Sync: -3 {-4,1} • DelayReq: 0 {-3,Sync +5}* • Allow all devices to be either Master or Slave • Multicast messages only • Default PTP domain: 0 {0-255}

• SMPTE 2059 (Video driven) • Based on a 5 sec fast locking requirement for endpoints /w spec rates • Announce: -2 {-3,1} • Sync: -3 {-7,-1} • DelayReq: 0 {Sync +0, Sync +5} • Default Slave mode unless can operate as Master • Additional PTP TLV with media specific information • Daily Jam Time, Default Frame rate, … • Supports Multicast, Mixed (Mcast/Ucast) and Unicast modes • Default PTP domain: 127 {0-127}

Boundary Clock GM 1588v2

Spine

Leaf Leaf

ST2059 AES67 1588v2

GPS GPS GPS GPS

No PTP Extension

• Keep PTP separated between sites • Ensure the GMs at all sites are synced to GPS • Signal going from one site to another may need to be re-clocked

• show ptp brief • show ptp clock • show ptp parent

• sh ptp port interface ethernet 1/x counters • sh ptp corrections

Integration with Broadcast Unchanged Operator Workflow SDI-based Controllers Video Router

Deterministic Network Non Blocking Network Design (Zero Packet Loss, Ultra Low with Bandwidth Aware Flow Latency and Minimal Jitter) Setup

Precision Time for SMPTE 2059-2, AES67 profile Synchronization

System Availability Redundant Network (-7) IP Network

Network Security Flow and Host Policies

Packet Loss Media DC Remote Production

Media Controller

Border Leaf MPLS

WAN WAN Router link

• Detect gaps in RTP flows • uncompressed flows like SMPTE 2110-20 • compressed flows MPEG-4,H.264 etc. • Leveraging Cisco’s Nexus 9000 Cloud Scale ASIC capability • ASIC can inspect every packet of every flow (up to 24k per switch) • Parses up to 128B of packet header to detect gaps in RTP flow

IP UDP RTP SMPTE 2110 Payload

Media Controller Report No RTP RTP drops Drops Reports

Border Leaf MPLS

WAN WAN Router link

RTP Flow

!Configuration feature netflow ip flow rtp show flow rtp details -- Displays all active flows detected by the ASIC

RTP Flow timeout is 1440 minutes IPV4 Entries SIP DIP BD ID S-Port D-Port Intf/Vlan Name Packet Count BytesPerSec FlowStart 192.168.21.2 239.20.13.1 4139 5500 20000 Ethernet1/34 181699102492 157943157 22:25:08 UTC May 22 2019 192.168.100.164 239.101.160.22 4138 50000 20000 Ethernet1/33 41719636319 76983505 06:21:29 UTC May 30 2019 192.168.21.2 239.30.21.1 4141 5500 20000 Ethernet1/36 1388462946 299999 22:25:10 UTC May 22 2019 192.168.21.2 239.40.21.1 4141 5500 20000 Ethernet1/36 84118756 30686 22:25:11 UTC May 22 2019 show flow rtp errors active – Displays flows currently impacted

Leaf1# show flow rtp errors history –- Displays historic information of flows that where impacted and the time of impact (24hr on box history)

RTP Flow timeout is 1440 minutes IPV4 Entries SIP DIP BD ID S-Port D-Port Intf/Vlan Name Packet Count BytesPerSec FlowStart Packet Loss Loss Start Loss End 192.168.21.2 239.20.13.1 4150 5500 20000 Ethernet1/50 99713271686 157931882 06:21:22 UTC May 30 2019 307773 04:18:34 UTC Jun 08 2019 04:19:02 UTC Jun 08 2019 192.168.21.2 239.20.13.1 4150 5500 20000 Ethernet1/50 97761927229 157931915 06:21:22 UTC May 30 2019 27295 00:07:48 UTC Jun 08 2019 00:07:48 UTC Jun 08 2019

Leveraging Telemetry – Nexus9k notifies DCNM • DCNM overlays end when a loss is detected to end path of flow and the point in the network where loss is detected

• Time spent on troubleshooting flow loss goes from several hours to seconds!

• UK Premier League broadcast with BT Sport

• Live UHD broadcast from IP-enabled truck • Grass Valley IP Camera and GV Node • World’s first live 4K UHD broadcast • Platform: N9508

• Cisco on Cisco Story • Internal customer for IPFM • All IP Studio Production • Platform: N9300

• Another common use case • Connect multiple locations via dark fiber

• Platform: N9300 and N9508 - Spine and Leaf • Platform: N9300 and N9508

fx PRODUCTION POST-PROD GRAPHICS PLAYOUT DISTRIBUTION

Cisco IP Fabric for Media

Cisco Hyperflex

• Graphics Production • VFX • Compositing / Overlays • Rendering • Real-time GFX Processing

SMPTE 2110 LLQ Link Network control Control Files / Editing Best Effort

Graphics and Editing Graphics ST ST Storage 2110 2110

• The solution allows LIVE workflow to co-exist with file based workflows

• NBM places the flows in priority queue

• User defined QOS policies are used to classify and place unicast or file based traffic in low priority queues

• Allows a certain percentage of bandwidth to be reserved for unicast

fx PRODUCTION POST-PROD GRAPHICS PLAYOUT DISTRIBUTION

Service Providers • No RIRs / DNS • No address ownership

• Multi-Tenancy

• Administrative domains • Overlap of addresses

• Security

• IGMP support Production DC

Service Providers

Multicast NAT

S2,G2 S1,G1

Production DC

Translate Source, Group , and L4 port Programable through APIs

Service Providers

S2,G2

S1,G1 S3,G3

S4,G4 Production DC

Translate Source, Group , and L4 port Programable through APIs Scales to 1000s of entries

Switch changes Multicast mapping S1,G1 > S3,G3 S2,G2 > S3,G3

S1,G1 S2,G2 S3,G3

Destination static join to S3,G3

• IP Fabric is the Next Gen Digital Glue for Media DC

• Spine Leaf Distributed Architecture • IP Fabric for Media • Multicast rich

• SDN Controller for Deterministic and Secure Media Fabric

• Automation and Streaming Telemetry

• Cisco Media page @ http://www.cisco.com/go/media

• IP Fabric for Media on Cisco.com • Cisco IP Fabric for Media Design Guide • Precision Time Protocol for Timing in IP Fabric for Media Guide

• DCNM Page

BRKDCN-2219 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 107 Complete your online session • Please complete your session survey survey after each session. Your feedback is very important.

• Complete a minimum of 4 session surveys and the Overall Conference survey (starting on Thursday) to receive your Cisco Live t-shirt.

• All surveys can be taken in the Cisco Events Mobile App or by logging in to the Content Catalog on ciscolive.com/emea.

Cisco Live sessions will be available for viewing on demand after the event at ciscolive.com.

Demos in the Walk-in labs Cisco campus

Meet the engineer Related sessions 1:1 meetings