Ethernet OAM - Technical Overview and Deployment Scenarios Jaroslaw Grabowski Consultant System Engineer BRKNMS-2202 Agenda
• Introduction
• Fault Management
• Performance Management
• Use cases and configuration Snippets
• Platform Support
• Summary Ethernet OAM Overview & High Level Maps Short History of Ethernet
400G?/Flex? 10M IEEE 802. 100M 1G 10G 100G 1974 1983 1994 1995 1999 2002 2007 2010 Today 2016
IEEE 802.*** Not mandatory Only L3 tools additions
Robert Metcalfe First massive Ethernet Ethernet patent deployments in OAM Service Providers Era Networks OAM is needed!
5 Historical Map
Link Management IEEE 802.3ah (subset of Ethernet First Mile 2004, 2008)
CFM (Connectivity Fault IEEE 802.1ag (2007) Management) Architecture + Performance Management Full OAM architecture ITU-T Y.1731 / G.8013 (2008, 2011, 2013 published) Y.xxxx IP and next-generation networks G.xxxx Transmission systems ITU-T Y.1564 (2011) Activation
Metro Ethernet Forum E-LMI (2006) Adoption of 802.1ag & Y1731 in MetroEth Service Activation, Certification, etc…
Cisco IP SLA Framework Ethernet OAM Extensions, common CLI, NMS
6 Classical ISO/OSI Management Model FCAPS
Fault Configuration Accounting Performance Security Management Management Management Management Management
Fault Detection Service ... Frame Loss … Fault Verification Provisioning Measurement Fault Isolation Delay E-LMI Measurement Fault Recovery Delay Variation Fault Notification Measurement Availability Link OAM Measurement
CFM (Connectivity Fault Management) Performance Management (Y.1731) Service Activation (Y.1564)
7 Protocols Positioning Map
CFM (Connectivity Fault Management) End-to-End or Domain Management
Performance Management (Y.1731) & Y.1564 E-LMI
Backbone Backbone Bridges Bridges Provider Provider Bridges Bridges Customer Customer MPLS Link OAM MPLS OAM
8 Protocol Overview - Link OAM IEEE 802.3-2008 (Clause 57) Link OAM (IEEE 802.3ah, Clause 57) For your reference
• Provides mechanisms useful for “monitoring point-to-point link operation”, such as: Reserved Remote Stable Remote Local Stable Local Critical Event Dying Gasp Link Fault • Link Monitoring Evaluating Evaluating • Remote Failure Indication • Remote Loopback Control DMAC SMAC Type
Dest Address Src Address type subtype flags code datacode CRC 0180c20002 8809 03 • Defines an optional OAM sublayer
code local info remote info … 00 TLV TLV • Uses “Slow Protocol”1) frames called Information OAMPDU
code seq link event … OAMPDUs which are never forwarded by 01 # TLV Event Notification OAMPDU MAC clients (specific multicast Destination code var descriptor … (0180.c200.0002) and Ether-Type (8809) ) Var Request OAMPDU 02 (branch & leaf)
code var container … • Standardized: IEEE 802.3ah, clause 57 (now Var Response OAMPDU 03 (branch, leaf, width, value)
in IEEE 802.3-2008) code loopback Loopback Control OAMPDU 04 command
code 24-bit OUI … Vendor-Specific OAMPDU FE (1) No more than 10 Frames Transmitted in Any One-Second Period
10 Link OAM. Use Case For your Link monitoring and discovery reference
• Discovery process • OAMPDU are sent periodically and carrying platform capabilities and identity OUI (organization unique identifier) Provider • Link monitoring process Bridges Customer • Event notification: error rate exceeded a threshold. Link OAM switch#show ethernet oam summ Symbols: * - Master Loopback State, # - Slave Loopback State Capability codes: L - Link Monitor, R - Remote Loopback interface ….. U - Unidirection, V - Variable Retrieval no ip address Local Remote ethernet oam Interface MAC Address OUI Mode Capability
Gi3/2 0014.6a7c.c451 00000C active L R Gi3/48 0015.6282.25a7 00000C passive L
11 Link OAM. For your DTE Types reference
Link OAM Capability Active DTE Passive DTE Initiates OAM Discovery process Yes No
Reacts to OAM Discovery process initiation Yes Yes
Required to send Information OAMPDUs Yes Yes
Permitted to send Event Notification OAMPDUs Yes Yes
Permitted to send Variable Request OAMPDUs Yes No
Permitted to send Variable Response OAMPDUs Yes1 Yes
Permitted to send Loopback Control OAMPDUs Yes No
Reacts to Loopback Control OAMPDUs Yes1 Yes
Permitted to send Organization Specific OAMPDUs Yes Yes
12 Link OAM. Use Case Dying Gasp (Remote Failure Indication)
• OAM PDU is sent when station is about to go down. SNMP Dedicated codes are created for different events No SysLog • Administrative shutdown power! • Power Loss (extended HW capabilities are required) Shut down • Reboot • After receiving OAMPDU remote peers can react depending on configured actions Provider • SNMP trap Bridges Customer • sysLog message Link • Disable interface OAM • Activate Interworking mechanisms
Benefit: No need for sending service team in case of lost connectivity
13 Link OAM. Use Case For your Dying Gasp. reference interface ….. no ip address ethernet oam max-rate 5 SNMP ethernet oam min-rate 2 No SysLog ethernet oam timeout 30 ethernet oam remote-failure link-fault action error-disable-interface power! ethernet oam remote-failure dying-gasp action error-disable-interface Shut ethernet oam remote-failure critical-event action error-disable- down interface ethernet oam
snmp-server traps ethernet oam events Provider Bridges Customer Link OAM interface ….. template oam no ip address Could be ethernet oam max-rate 5 ethernet oam ethernet oam min-rate 2 defined as ethernet oam mode passive template ethernet oam timeout 30 …..
14 Link OAM. Use Case Remote Loopback • Ethernet data traffic can be looped back on a per port / per VLAN basis • Use cases: • Service turn-up • Post service turn-up troubleshooting Customer • Out-of-service throughput testing External M2 M1 • Enabled via CLI configuration • Configurable direction: M1 M2 • External Loopback (facing and testing wire) Internal • Internal Loopback (facing and testing wire & bridge) • Complements CFM Loopback • External central Test Head allows for flexible and Cisco extension: MAC address swap. sophisticated test traffic patterns Original 802.3ah assumes pure data loopback
15 Link OAM. Use Case For your Remote Loopback reference
Networ_Switch# ethernet oam remote-loopback start interface fas1/3
6d22h: %ETHERNET_OAM-SP-6-LOOPBACK: Interface Fa1/3 has entered the master loopback mode. Customer External
Syslog message Internal Customer_Switch#
Remote 1w0d: %ETHERNET_OAM-SP-6-LOOPBACK: Interface Fa4/3 has activation entered the slave loopback mode.
16 Protocol Overview Connectivity Fault Management CFM (Connectivity Fault Management) Overview
• Family of protocols that provides capabilities to detect, verify, isolate and report end-to-end Ethernet connectivity faults
• Employs regular Ethernet frames that travel in-band with the customer traffic • Devices that cannot interpret CFM Messages forward them as normal data frames
• CFM frames are distinguishable by Ether-Type (0x8902) and dMAC address (for multicast messages)
• Standardized • IEEE std. 802.1ag-2007 • Y.1731/ G.8013 (2008, 2011, 2013) Y.1731/G.8013 (Fault Management) was design to be compatible with 802.1ag. Frame formats (Multicast Address, Ethertype, and common OAM PDU fields) and base functionality are generally agreed upon across IEEE 802.1ag and Y.1731/ G.8013
18 CFM Concepts Maintenance Entity
Point A Point B
• Simply: “The entity between two of the flow/connection points“ • Extended definition in ITU documentation “G.8013/Y.1731” referring to “G.8001/Y.1354” finally referring to referring to “G.8010/Y.1306”
19 CFM Concepts Management hierarchy
CE CE
Service Provider Customer
• Minimum (but relevant) information principle. Customer is interested only with his/her service • Maintenance Domain from Customer perspective is defined end-to-end
20 CFM Concepts More levels
CE CE
Operator B
Service Provider Customer
• Service Provider should have full visibility inside owned network • If Service Provider is using 3rd Party network – acting as “customer”, visibility of Operator B network structure is not necessary. Very often this type of information is confidential. 21 CFM Concepts
CE CE
Operator A Operator B
Service Provider Customer
• Defined by Operational/Contractual Boundaries • e.g. Customer/Service Provider/Operator • ME may nest and touch, but never intersect • Up to 8 levels of “nesting”: ME Level (0..7) • The higher the level, the broader its reach • ME Name Format: null, MAC address, DNS or string-based 22 CFM Concepts Maintenance Entity Group
CE Operator A Operator B CE
MAID String, VLAN Maintenance Entity Groups
• Monitors connectivity of a particular service instance in a given MEG (e.g. 1 service traversing 4 MEGs = 4 MAs) • Identified by MEGID == “Short MA” Name + MEG Name • Short MA Name Format: Vlan-ID, VPN-ID, integer or string-based
23 CFM Concepts
CE Operator A Operator B CE
MEP MEP MEP MEP
MEP MEP MEP MEP
Maintenance Entity Group End Point (MEP) • Define the boundaries of a MEG • Support the detection of connectivity failures between any pair of MEPs in an MA • Associated per ME and identified by a MEPID (1-8191) • Can initiate and respond to CFM PDUs
24 MEG End Point (MEP) MEG (Maintenance Entity Group) How it works.
Intention to create: Maintenance Domain. Level 5.
L2 EVC service All frames (e.g. Specific VLAN) pass freely
25 MEG End Point (MEP)
CFM MEP level 5 is Process Intercept Frames with configured. Eth-Type = CFM (0x8902) & Level 5
Data pass freely (Frame with Eth-Type different then CFM!!)
OAM PDU EthType
26 MEG End Point (MEP) Lower level can not cross Higher level Other CFM frames
CFM MEP level 5 is Process DROP Frame MD Level < 5 Level (0,1,2,3,4) configured.
5
Frame MD Level =< 5 Level DROP Frames pass freely Drop (0,1,2,3,4,5) also (Frame with Eth-Type level 5!! different then CFM) Protection Mechanism!
OAM PDU EthType
27 MEG End Point (MEP) Other CFM frames
CFM MEP level 5 is Process configured.
Pass Higer Frame MD Level > 5 Level (6,7) Level 6-7
Frames pass freely (Frame with Eth-Type different then CFM) Pass Higer Frame MD Level > 5 Level (6,7) Level 6-7
OAM PDU EthType
28 MEG End Point (MEP) Full story
Assuming MEP CFM Drop level 5 is Process DROP Frame MD Level < MEP mdLevel Level 0-4 configured. Process Only Frame MD Level = MEP mdLevel Level 5 level 5 Pass Frame MD Level > MEP mdLevel Level 6-7
Drop DROP Data pass freely Level 0-5 Frame MD Level =< MEP mdLevel (Frame with Eth-Type different then CFM) Pass Frame MD Level > MEP mdLevel Level 6-7
OAM PDU EthType
29 Multiple MEG End Point (MEP)
CFM MEP level 5 is Process Intercept Frames with configured. Eth-Type = CFM (0x8902) MEP level 3 is & Level 5, 3 configured.
More MEPs on Data pass freely (Frame with Eth-Type the same point different then CFM!!) are allowed
OAM PDU EthType
30 CFM Concepts Down MEP
• CFM PDUs generated by the MEP are sent via the wire connected to the port where the MEP is configured Bridge / Router • CFM PDUs to be responded by the MEP are expected to arrive via the wire connected to the port where the MEP is configured Relay • Port MEP – special Down MEP at level zero Entity (0) used to detect faults at the link level (untagged - rather than service) Port A Port B • Applicable to routers and switches CFM PDUs
31 CFM Concepts Up MEP
• CFM PDUs generated by the MEP are sent towards the Bridge’s Relay Function and not via the wire connected to the port where the Bridge MEP is configured • CFM PDUs to be responded by the MEP are expected to arrive via the Bridge’s Relay Relay Function Entity • Applicable to switches Port A Port B • UP MEPs typically found on UNIs (e.g. Port A)
CFM PDUs
32 CFM Concepts
MEP MEP Level 7 Down Down
MEP Up Level 6
MEP Up Level 3
MEP Down Level 0
• Down and Up MEP in action
33 CFM Concepts
MEP MIP MIP MEP MEP MIP MIP MEP
MEP MIP MIP MEP MEP MIP MIPMIP MIP MEP
MIP MIP MIP MIP
• Maintenance Entity Group Intermediate Point (MIP) • Support the discovery of paths among MEPs and location of faults along those paths • Can be associated per MEG and VLAN / EVC (manually or automatically created) • Can add, check and respond to received CFM PDUs
34 CFM Protocol & Tools
• There are three (3) protocols defined by CFM
• Continuity Check Protocol • Fault Detection • Fault Notification • Fault Recovery
• Loopback Protocol • Fault Verification
• Linktrace Protocol • Path Discovery and Fault Isolation
35 CFM Protocols: Continuity Check Protocol
CE Operator A Operator B CE
Catalogue and Catalogue Catalogue Terminate MEP MIP MIP MEP
1 2 3 X1. Continuity Check Message (CCM)
• Used for Fault Detection, Notification and Recovery • Per-Maintenance Association multicast “heart-beat” messages • Configurable transmission interval by MEPs (3.3ms, 10ms, 100ms, 1s, 10s, 1min, 10min) • Uni-directional (no response required) • Carries status of port on which MEP is configured • Catalogued by MIPs at the same MEG-Level and terminated by remote MEPs in the same MA
36 CFM Protocols: Continuity Check Protocol CE Operator A Operator B CE
Catalogue and Catalogue Catalogue Terminate MEP MIP MIP MEP
1 2 3 UPE11#show ethernet cfm maintenance-points remote ------MPID Domain Name MacAddress IfSt PtSt Lvl Domain ID Ingress CCM RDI MA Name Type Id SrvcInst Database EVC Name Age ------3100 PROVIDER_DOMAIN aabb.cc00.0599 Up Up 4 PROVIDER_DOMAIN Et0/1.100 - customer_100_provider Vlan 100 N/A N/A 0s
Total Remote MEPs: 1
37 CFM Protocols: Loopback Protocol
CE Operator A Operator B CE
S D MEP MIP MIP MEP
1 11. Loopback Message(LBM) 2 22. Loopback Reply (LBR)
• Used for Fault Verification—Ethernet Ping • MEP can transmit a unicast LBM to a MEP or MIP in the same MA • Receiving MP responds by transforming the LBM into a unicast LBR sent back to the originating MEP
38 CFM Protocols: Link trace Protocol
CE Operator A Operator B CE
S 4 6 D 2 MEP MIP MIP MEP
1 3 5 LM LM 1.X 1, 3, 5 Linktrace Message (LM) Y2. 2, 4, 6 Linktrace Reply (LR) • Used for Path Discovery and Fault Isolation—Ethernet Traceroute • MEP can transmit a multicast message (LTM) in order to discover the MPs and path to a MIP or MEP in the same MA • MIPs along the path relay LTM message only when target MAC is known • Each MIP along the path and the terminating MP return a unicast LTR to originating MEP
39 Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31
MAC MAC MAC MAC aabb.cc00.0299 aabb.cc00.0399 aabb.cc00.0499 aabb.cc00.0599 MEP mpid UPE11#traceroute ethernet mpid 3100 domain PROVIDER_DOMAIN vlan 100 3100 <
40 LinkTrace flow LM: Linktrace Message STP • Sent by MEP to discover MEG topology blocked LM • Forwarded on MIP ports LM LM – Follows Spanning Tree active topology – Terminated by MEPs LM LM • Transparently forwarded by non-participating MAC DA = MIP nodes Same as for LTM LM MEP – e.g. nodes that do not support extension Maintenance Entity Group LR LR: Linktrace Reply • Response to an ELM message • All MIPs/MEPs on Node may reply, OR LR • Only MIPs/MEPs in Active STP Topology • Destined to ELM originating node MAC DA = LR ELM Src Node LR • Transparently forwarded by non-participating nodes
41 In-Depth Visibility into Carrier Ethernet Networks Troubleshooting via CFM (802.1ag)
Nested Hierarchies Easier Carrier Ethernet Maintenance troubleshooting with CFM Domains Maintenance Association leveraging Prime Network: CFM Levels • CFM hierarchy configuration discovery and reported in hierarchal views • CFM ping, CFM trace and other built-in scripts available
Local and Remote Maintenance end Points (MEPs)
42 Please note that Cisco CFM implementation was created CFM Terminology before Y.1731 name conventions were introduced. In order to keep compatibility some names in Cisco documentation using 802.1ag names
• Comparison ITU-T Y.1731 and IEEE 802.1ag IEEE 802.1ag ITU-T Y.1731 ME Maintenance Entity ME Maintenance Entity
MA Maintenance Association MEG ME Group
MAID MA Identifier MEGID MEG Identifier
MD Maintenance Domain --- No such construct available MD MEG MD Level MEG Level Level Level MEP MA End Point MEP MEG End Point
MIP MD Intermediate Point MIP MEG Intermediate Point
43 ITU-T Y.1731 vs IEEE CFM
OpCode OAM PDU ITU-T Y.1731 IEEE 802.1ag CFM OAMPDUs for IEEE 802.1ag. Value type Common with Y.1731 1 CCM √ √ 2 LBR √ √ CFM OAMPDUs for Y.1731 3 LBM √ √ 4 LTR √ √ 5 LTM √ √ 33 AIS √ -- 35 LCK √ -- AIS, RDI – suppress alarms (when MEP receives AIS frames it means that alarm is 37 TST √ -- already sent and no need to re-send it. LCK – information about administrative locking the service (no user traffic) • OAM PDU OpCode Assignments
44 ITU-T Y.1731 - ETH-AIS Northbound Alarms Without AIS LOC Traps SNMP Server
Customer Operator A Link Operator B Customer Equipment Bridges Failure Bridges Equipment
MEP(7) MIP(7) MEP(5) MIP(5) MEP(3) MEP(2) MIP(3) MIP(2)
SMEP 802.3ah/link status “Server” MEP
45 ITU-T Y.1731 - ETH-AIS Northbound LOC Link Traps suppressed Alarms Without AIS Failure SNMP Server
Customer Operator A Operator B Customer Equipment Bridges Bridges Equipment
MEP(7) MIP(7) MEP(5) MIP(5) MEP(3) MEP(2) MIP(3) MIP(2)
SMEP 802.3ah/link status “Server” MEP AIS 46 Comparison. IEEE 802.3ah (Clause 57) OAM and IEEE 802.1ag CFM
Link OAM Connectivity Fault Management IEEE 802.3, Clause 57 IEEE 802.1ag/Y.1731 Operates on Physical Link only, Cannot May Be Per-Service or Per-Wire; Pass Through A Bridge Passes End-to-End Through Bridges Discovery, Statistics Enquiry, Keepalive, Connectivity Verification, Traceroute, Loopback (Mirror) Mode Ping, Alarm Indication/Suppression Multiple Instances Operating Single Instantiation per Wire Multiple Levels Simultaneously Can Be Used over any Medium Defined Only for 802.3 Media that Can Carry 802 Frames Performs a Continuity Check Can Be Programmed to Perform the Once per Second Continuity Check over a Range of Intervals 802.1ag Connectivity Fault Management Provides Service Level OAM, and Therefore Is Typically Higher Priority than 802.3 OAM
47 Summary of CFM
• Demonstration using one of the Cisco products
• Ethernet could be manageable
• Not all Ethernet implementations are manageable
• More efforts = better results
48 Protocol Overview Ethernet Performance Management Key Performance Indicators (KPIs)
Service “Pipe” What is the delay traffic is experiencing? Frame Delay (FD) Delay Measurement (MD) How much traffic is lost? What is available bandwidth? Frame Loss What are the delay Ratio (FLR) changes (jitter)? How much traffic customer is sending Frame Delay Variation (FDV) (billing & reporting) Summarized availability of the service
50 Key Performance Indicators (KPIs)
• Frame Loss Ratio (FLR) – percentage (%) of service frames* (or synthetic frames) not delivered / Total number of service frames (or synthetic frames) transmitted in T time interval
• Frame Delay (FD) – round-trip/one-way delay for a service frame
• Frame Delay Variation (FDV) – Variation in frame delay between a pair of service frame
• Service Availability
• * Service frames (Green) are frames that conform to agreed upon level of bandwidth profile conformance
51 Ethernet Performance Management ITU-T Y.1731 – Technology Overview Ethernet Ethernet Delay Measurement Ethernet Loss Measurement Synthetic Loss ETH-DM ETH-LM Measurement ETH-SLM
One-Way ETH- Two-Way ETH- Single-Ended Dual-Ended ETH- Single-Ended DM DM ETH-LM LM ETH-SLM Scope • One-Way delay • Two-Way delay • Unidirectional frame • Unidirectional frame • Unidirectional frame measurements measurements loss measurements loss measurements loss measurements • Also One-Way delay • Applicable to P2P • Applicable to P2P • Applicable to P2P and Specifics when synchronized services only services only MP services • 1DM PDU • DMM / DMRPDUs • LMM / LMRPDUs • CCMPDUs • SLM / SLRPDUs • Synthetic traffic with • Synthetic traffic with • On-demand operation • Proactive operation • Based on statistical two (2) timestamps two (2) mandatory • Based on actual • Based on actual sampling • Need for Time-of-Day and two (2) optional Service Frame Loss Service Frame Loss • Exchange of synthetic synchronization timestamps • Exchange of service • Exchange of service frame counters frame counters frame counters 52 ITU-T Y.1731 Frame Delay Measurement
• Frame Delay calculated based on Ethernet Delay timestamps applied to synthetic Measurement ETH-DM traffic
• Applicable to point-to-point and multipoint services One-Way ETH-DM Two-Way ETH-DM • Two (2) mechanisms defined • One-Way ETH-DM • Two-Way ETH-DM Performance Metrics Performance Metrics • One-Way FD/FDV • Two-Way FD/FDV • Time-of-Day (ToD) • One-Way FD/FDV when synchronization required for synchronized OW FD
53 ITU-T Y.1731 Overview Two-way ETH-DM
Customer Service Provider Customer Equipment Equipment
Probe Initiator & Statistics Service Calculation Frames MEP MEP Probe Responder
TxTimeStampf 0 DMM0 TxTimeStampf 0 DMM RxTimeStampf 0 DMM 0 TxTimeStampf TxTimeStampf Optional RxTimeStampf RxTimeStampf DMR DMRTxTimeStampb Timestamps TxTimeStampb RxTimeStampb DMR 0 Two-Way Delay One-Way Delay (Forward) One-Way Delay (Backward) (RxTimeStampb – TxTimeStampf) – RxTimeStampf – TxTimeStampf RxTimeStampb – TxTimeStampb (TxTimeStampb – RxTimeStampf) DMM – Delay Measurement Message DMR – Delay Measurement Reply 54 ITU-T Y.1731 Overview One-way ETH-DM
Customer Service Provider Customer Equipment Equipment
Service Probe Frames Initiator MEP MEP Statistics Calculation
TxTimeStampf 1DM 0 1DM tp TxTimeStampf RxTimeStampf 1DM
TxTimeStampf 1DM 0 tc TxTimeStampf 1DM 1DM RxTimeStampf
One-Way Delay One-Way Delay Variation
Delay = RxTimeStampf – TxTimeStampf Delay Var.[tc] = Delay[tc] – Delay[tp]
tp – Time Previous 1DM – One-way Delay Measurement tc – Time Current 55 For your ASR 9000 Cisco IOS XR Y.1731 Implementation reference
Global Configuration Interface Configuration ethernet sla interface TenGigE0/0/2/3.100 l2transport profile SAMPLE type cfm-delay-measurement encapsulation dot1q 100 probe ethernet cfm send burst every 60 seconds packet count 60 mep domain DOMAIN-L5 service SAMPLE mep-id 103 interval 1 seconds sla operation profile SAMPLE target mep-id 101 priority 1 ! ! schedule every 1 minutes for 1 minutes ! statistics measure round-trip-delay buckets size 1 probes buckets archive 2 TenGiGE0/0/2/3.100 ! dot1q 100 measure one-way-jitter-sd buckets size 1 probes buckets archive 2 ! ! UP MEP 56 ITU-T Y.1731 Frame Loss Measurement Ethernet Loss • Frame Loss calculated based on Measurement • Service frame counters (ETH-LM • Synthetic frame counters (ETH-SLM)
Single-Ended ETH-SLM Single-Ended ETH-LM • Measures FLR in • Point-to-Point EVCs only (ETH-LM) • P2P and Multipoint EVCs (ETH-SLM) Performance • Provides Metrics • Near End Frame Loss • Far End Frame Loss • Unidirectional Frame Loss • Frame Loss Ratio • Availability
57 ITU-T Y.1731 Overview Single-Ended ETH-LM Customer Service Provider Customer Equipment Equipment
Probe Initiator & Statistics Service Calculation MEP Frames MEP Probe Responder Per-CoS Service Frame counters TxFCf TxFCf TxFCl LMM 0 LMM 0 LMM RxFCl 0 0 t Per-CoS Service TxFCf p LMR TxFCf LMR LMR RxFCf RxFCf Frame counters TxFCb TxFCb RxFCl LMM TxFCl t c LMR Frame Loss (Far-end) Frame Loss (Near-end)
(TxFCf[tc] – TxFCf[tp]) – (RxFCf[tc] – RxFCf[tp]) (TxFCb[tc] – TxFCb[tp]) – (RxFCl[tc] – RxFCl[tp])
tp – Time Previous LMM – Loss Measurement Message tc – Time Current LMR – Loss Measurement Reply
58 Frame Loss in Multi-point service . Due to Broadcast and Multicasts comparing counters in Multi-point L2VPN does not provide information about lost frames
59 Frame Loss in Multi-point service . Due to Broadcast and Multicasts comparing counters in Multi-point L2VPN does not provide information about lost frames . Synthetic and targeted traffic needs to be generated . Counters are compared for synthetic traffic only. . Statistically, frame loss for synthetic traffic should give an approximation to the loss of data traffic
SLM – Synthetic Loss Message SLR – Synthetic Loss Reply
60 ITU-T Y.1731 Overview Single-Ended ETH-LM
Customer Service Provider Customer Equipment Equipment
Probe Initiator & Statistics Synthetic Calculation MEP Frames MEP Probe Responder Per-Test ID Synthetic Frame counters
TxFCl TxFCf TxFCf RxFCl SLM SLM 0 SLM 0 Per-Test ID Synthetic t Frame counters TxFCf p SLR TxFCf SLR SLR TxFCb TxFCb RxFCl TxFCl SLM RxFCl = TxFCl at tc SLR Responder Frame Loss (Far-end) Frame Loss (Near-end)
(TxFCf[tc] – TxFCf[tp]) – (TxFCb[tc] – TxFCb[tp]) (TxFCb[tc] – TxFCb[tp]) – (RxFCl[tc] – RxFCl[tp])
tp – Time Previous – Start of Measurement Period SLM – Synthetic Loss Message tc – Time Current – End of Measurement Period SLR – Synthetic Loss Reply 61 Y.1731 ETH-SLM IOS Implementation ip sla 100 Continuous Transmission Mode ethernet y1731 loss SLM domain cisco evc evc2 mpid 22 cos 5 source mpid 11 frame size 1300 frame interval 100 owner CiscoLive Time history interval 4 Frame Interval – inter-packet gap aggregate interval 60 Number of FLRs in frame consecutive 9 Aggregation Interval (time) frame consecutive loss-ratio 5 Availability Window ! Agg. Interval 1 Agg. Interval 1 ip sla schedule 100 life forever start-time now
Burst Transmission Mode
Time Frame Interval – inter-packet gap Frame Burst – frames per burst Number of SLM frames Frequency (time) – time between bursts used to calculate one Aggregation Interval – number of bursts FLR
62 Recording statistics
• This is what the results will look like. One bucket per hour containing a packets sent count and packets lost count for each consecutive pair of LMRs received.
Sent(1) = … Sent(1) = … Sent(1) = … Lost(1) = … Lost(1) = … Lost(1) = …
1 hr 2 hr 3 hr
63 Bin TwoWay: Interval 2 Bin Range (microsec) Total observations Start time: 21:25:48.600 PST Thu Feb 17 2011 0 - < 15 0 End time: 21:25:53.600 PST Thu Feb 17 2011 Show history.. 15 - < 30 0 Number of measurements initiated: 50 30 - < 45 0 Number of measurements completed: 50 45 - < 60 0 Flag: OK 60 - < 4294967295 50 Router#show ip sla history 10 interval-statistics Delay Statistics for Y1731 Operation 10 Delay Variance: Delay: Type of operation: Y1731 Delay Measurement Number of TwoWay positive observations: 38 Number of TwoWay observations: 50 Latest operation start time: 21:25:53.600 PST Thu Feb 17 Min/Avg/Max TwoWay positive: 0/0/0 (microsec) Min/Avg/Max TwoWay: 60/60/61 (microsec) 2011 Time of occurrence TwoWay positive: Time of occurrence TwoWay: Latest operation return code: OK Min - 21:25:57.235 PST Thu Feb 17 2011 Min - 21:25:53.600 PST Thu Feb 17 2011 Distribution Statistics: Max - 21:25:58.604 PST Thu Feb 17 2011 Max - 21:25:53.600 PST Thu Feb 17 2011 Number of TwoWay negative observations: 11 Min/Avg/Max TwoWay negative: 0/0/1 (microsec) Bin TwoWay: Interval 1 Time of occurrence TwoWay negative: Bin Range (microsec) Total observations Start time: 21:25:53.600 PST Thu Feb 17 2011 Min - 21:25:57.235 PST Thu Feb 17 2011 0 - < 15 0 End time: 21:25:58.604 PST Thu Feb 17 2011 Max - 21:25:58.604 PST Thu Feb 17 2011 15 - < 30 0 Number of measurements initiated: 52 30 - < 45 0 Number of measurements completed: 50 Bin TwoWay positive: 45 - < 60 0 Flag: OK Bin Range (microsec) Total observations 60 - < 4294967295 50 0 - < 2 38 2 - < 4 0 Delay Variance: Delay: 4 - < 6 0 Number of TwoWay positive observations: 40 Number of TwoWay observations: 50 6 - < 8 0 Min/Avg/Max TwoWay positive: 0/0/0 (microsec) Min/Avg/Max TwoWay: 60/60/61 (microsec) 8 - < 4294967295 0 Time of occurrence TwoWay positive: Time of occurrence TwoWay: Bin TwoWay negative: Min - 21:25:52.133 PST Thu Feb 17 2011 Min - 21:25:58.604 PST Thu Feb 17 2011 Bin Range (microsec) Total observations Max - 21:25:53.600 PST Thu Feb 17 2011 Max - 21:25:58.604 PST Thu Feb 17 2011 0 - < 2 11 Number of TwoWay negative observations: 9 2 - < 4 0 Min/Avg/Max TwoWay negative: 0/0/1 (microsec) 4 - < 6 0 Time of occurrence TwoWay negative: 6 - < 8 0 Min - 21:25:52.133 PST Thu Feb 17 2011 8 - < 4294967295 0 Max - 21:25:53.600 PST Thu …………………….. 64 Prime Performance . Web-based interface Prime Prime providing more than 5000 Performance Manager Provisioning predefined reports Prime Central . Provides information on Prime Optical Prime Network application and IP traffic, availability, IP QoS, resources and transport statistics . Report extensions through format customization and MIBs support . Threshold crossing alerts . Support for Cisco and third- party devices
65 L2 Service Activation Y.1564 Y.1564 Service Activation Measurements • Y.1564 is an Ethernet service activation /performance test methodology for turning up, Traffic installing and troubleshooting Ethernet-based Generator services. • Extension to the RFC 2544 (For IP services) EVC • “Intrusive” mode with synthetic traffic generation. Traffic – To ensure that all services carried by the network meet Remote Analysis their SLA objectives at their maximum committed rate, Loopback proving that under maximum load network devices and paths can support all the traffic as designed • “Passive” mode with only statistics collection • Tests requires configuration of : – Headend with traffic generator (automatic analysis) – Remote data loopback (no automatic signaling between loopback and Headend)
67 Y.1564 Service Activation Measurements • Packets Generation Traffic – Color-aware Packet generation and measurement for Generator Ethernet (802.1p cos) – IP/Layer3 Packet generation and measurement EVC – IMIX Traffic generation (Combination of 64, 512, 1518 byte packets)
Traffic • Traffic Analysis Remote Analysis – Throughput, Latency, Loss, Jitter Loopback • Directions of Packet generators – Internal. Going thorough Internal Forwarding Entity. (Similarly to Internal Generation UP MEP) (e.g Bridge Domain) – External. Similar to DOWN MEP
Relay Port A Entity Port B External Generation (e.g. Interface) 68 ITU-T Y.1564 vs RFC2544
• RFC2544 methodology shortcomings (as stated by Y.1564) – Not Ethernet service aware – Tests run as a single flow at a time – Tests are performed sequentially – Does not measure Frame Delay Variation – Does not verify CIR, CBS, EIR, EBS and CM • Y.1564 advantages – Tests all KPIs at same time – Test to CIR to verify SLA performance (FD, FDV, FLR) – Tests to EIR limit and just beyond to verify policing behavior (no SLA performance expected for yellow frames (above CIR and below CIR+EIR)
69 “Service Instance” Service Type with Internal direction Y.1564 ip sla 400 service-performance type ethernet dest-mac-addr 0000.0400.0401 Configuration (IOS XE) interface GigabitEthernet0/12
ASR920, ASR900 service instance 400 measurement-type direction internal loss Traffic throughput profile packet Direction Generator outer-vlan 400 packet-size 1028 src-mac-addr 0003.0033.0031 EVC profile traffic direction internal rate-step kbps 5000 “Interface” Service Type with External direction Traffic Remote ip sla 500 Analysis service-performance type ethernet dest-mac-addr 0000.0500.0501 Loopback interface GigabitEthernet0/12 measurement-type direction external Internal Generation loss throughput (e.g Bridge Domain) profile packet outer-vlan 500 inner-vlan 200 packet-size 1518 Relay src-mac-addr 0001.0051.0055 Port A Entity Port B profile traffic direction external External Generation rate-step kbps 4000 (e.g. Interface) 7070 Y.1564 Results Configuration (IOS XE) > show ip sla statistics 12345 ASR920, ASR900 IPSLAs Latest Operation Statistics
IPSLA operation id: 12345 Traffic Type of operation: Ethernet Service Performance Test mode: Two-way Measurement Generator Steps Tested (kbps): 10000 20000 25000 Test duration: 20 seconds
EVC Latest measurement: *15:54:44.007 IST Mon May 18 2015 Latest return code: Oper End of Life
Traffic Overall Throughput: 24850 kbps Remote Analysis Loopback Step 1 (10000 kbps): Stats: IR(kbps) FL FLR Avail Internal Generation 9944 0 0.00% 100.00% (e.g Bridge Domain) Tx Packets: 16377 Tx Bytes: 24860286 Rx Packets: 16377 Rx Bytes: 24860286 Step Duration: 20 seconds
Relay Port A Entity Port B External Generation (e.g. Interface) 7171 Simple implementation examples Bringing CFM & PM together Real live “minimum” scenario Objectives
CPE-B CPE-A Access Access Usually MPLS cloud
EVC service
• Objective 1. Monitor faults and connectivity problem (CFM) • Objective 2. Collect basic performance statistics between CPE A&B – Delay (DMM Two-Way delay measurements) – Loss Ratio (SLM Synthetic Loss Measurement) • Objective 3. Minimize configuration overhead. CFM & PM configuration Only mandatory values!! is under common framework
73 Real live scenario Necessary objects
Access CPE-A Access CPE-B
EVC service
MEPs will Probe DMM respond to Probe SLM probes automatically MIP L7 MIP L7 MEP Down L7
MEP Up L5 MEP Up L5
74 Real live scenario Distinctive objects and their Identity
Access CE-A Access CE-B
Domain_Name Domain_Level EVC service MEPs will Y1731_DMM_ID respond to probes Y1731_SLM_ID automatically
Service_Name MIP L7 MIP L7 Service_Name Service_VLAN_A Service_VLAN_B InterfaceToNetwork_A InterfaceToNetwork_B
MPID-A MPID-B
75 Real live scenario Identities’ characteristics Name Type A & B Remarks Could be the same for all Domain_Name String Identical Access CPE CPE-A Access CE-B Integer Domain_Level Identical Same for all CPE (Level=7) Domain_Name [0-7] Domain_Level EVCY1731_DMM_ID service String A-only Probe on site A Y1731_SLM_ID String A-only Probe on site A Y1731_DMM_ID Global – OSS/BSS Service_Name String Identical Y1731_SLM_ID or Local “Service_VLAN-ID” Integer Locally significant VLAN ID Service_VLAN_A Local Service_Name Service_Name MIP L7 [1-4093]MIP L7 (OSS/BSS). LocallyService_VLAN_B significant port ID Service_VLAN_A InterfaceToNetwork_A String Local (OSS/BSS). InterfaceToNetwork_A Integer MPID_A Unique! Unique per service & VLAN [1-8191] MPID_A MPID-B
76 Real live scenario CPE-A configuration template (OAM relevant) IOS, IOS XE IDs characteristics …… ethernet cfm domain Domain_Name level Domain_Level service Service_Name vlan Service_VLAN_A Access CPE-A Access continuity-check CE-B continuity-check interval 1 …… Domain_Name ip sla Y1731_DMM_ID Domain_Level ethernet y1731 delay DMM domain Domain_Name vlan Service_VLAN_A EVC service mpid MPID_B cos 0 source mpid MPID_A ip sla Y1731_SLM_ID ethernet y1731 loss SLM domain Domain_Name vlan Service_VLAN_A Y1731_DMM_ID mpid MPID_B cos 1 source mpid MPID_A
Y1731_SLM_ID ip sla schedule IPSLA_DMM_1 life forever start-time now ip sla schedule IPSLA_SLM_1 life forever start-time now …… Service_Name Service_Name MIP L7 MIP L7 Service_VLAN_A interface InterfaceToNetwork_A Service_VLAN_B description (to ACCESS side with proper VLAN_A enabled) InterfaceToNetwork_A ….snip…. ethernet cfm mep domain Domain_Name mpid MPID_A service Service_Name MPID_A MPID-B
77 Real live scenario CPE-A configuration template (Minimum variables) IOS, IOS XE Exemplary values …… ethernet cfm domain MY_SP level 7 service Service_44 vlan 44 CPE-A Access continuity-check Access CE-B continuity-check interval 1 Domain_Name …… =MY_SP ip sla Y1731_DMM_44 Domain_Level=7 ethernet y1731 delay DMM domain Domain_Name vlan 44 EVC service mpid 222 cos 0 source mpid 111 ip sla Y1731_SLM_44 ethernet y1731 loss SLM domain Domain_Name vlan 44 Y1731_DMM_44 mpid 222 cos 1 source mpid 111
Y1731_SLM_44 ip sla schedule Y1731_DMM_44 life forever start-time now ip sla schedule Y1731_SLM_44 life forever start-time now Service_Name Service_Name …… MIP L7 =Service_44 MIP L7 interface InterfaceToNetwork_A Service_VLAN_B Service_VLAN_A=44 description (to ACCESS side with proper VLAN 44 enabled) ….snip…. InterfaceToNetwork_A ethernet cfm mep domain MY_SP mpid 111 service Service_44 MPID_A=111 (local with probes) MPID-B Remote MPID_B=222 78 IOS, IOS XE Real live scenario CPE-A configuration template (Minimum variables) Exemplary values ethernet cfm domain MY_SP level 7 mep crosscheck mpid 222 vlan 44 service Service_44 vlan 44 < monitor MPID_B CPE-A Access continuity-check Access CE-B continuity-check interval 1 …… ip sla Y1731_DMM_44 ethernet y1731 delay DMM domain Domain_Name vlan 44 EVC service mpid 222 cos 0 source mpid 111 ip sla Y1731_SLM_44 ethernet y1731 loss SLM domain Domain_Name vlan 44 mpid 222 cos 1 source mpid 111
ip sla schedule Y1731_DMM_44 life forever start-time now ip sla schedule Y1731_SLM_44 life forever start-time now …… Service_Name MIP L7 MIP L7 interface InterfaceToNetwork_A Service_VLAN_B description (to ACCESS side with proper VLAN 44 enabled) ….snip…. ethernet cfm mep domain MY_SP mpid 111 service Service_44 Device# ethernet cfm mep crosscheck enable level 7 vlan 44 MPID_A=111 (local with probes) MPID-B Remote MPID_B=222 79 Cisco EPN and OAM
Apr2014 Oct2014 EPN and Cisco Cisco assumptions for next are Feb2013 Sep2013 EPN v3.0 EPN v4.0 defined Cisco Cisco
FMC v1.0 FMC v2.0 Evolve Programmable Networks Dec2011 Apr2012 Sep2012 Adding (SDN type) programmability Cisco Cisco Cisco Fixed and Mobile Convergence and orchestration UMMT v1.0 UMMT v2.0 UMMT v3.0 Adding wireline Corporate/Residential Cisco UMMT OAM definitione for Transport and services Unified MPLS for Mobile Transport • Mobile Services Migration from SDH to packet networks • MEF (L2) • Enterprise (L3VPN)
80 Mobile Services OAM and PM
LTE, IPSLA IPSLA PM IPSLA 3G IP UMTS, Probe Probe
Transport VRF VRF
MPLS VRF OAM Service OAM
IPSLA IPSLA IPSLA PM (future PW PM) 3G ATM UMTS, Probe Probe 2G TDM, Transport MPLS VCCV PW OAM
Transport OAM
End-to-end LSP IP OAM over inter domain LSP (future LSP OAM) With unified MPLS
NodeB CSG MTG RNC/BSC/SAE GW
81 MEF L2VPN Services OAM and PM
IPSLA IPSLA Ethernet Ethernet IPSLA PM Probe Probe Down MIP L7 Ethernet CFM MIP L7 Down MEP L7 MEP L7 Y.1731 Y.1731 Up AIS/LCK Ethernet CFM AIS/LCK Up MEP L6 MEP L6 Managed CPE MPLS VCCV PW OAM Service OAM
Y.1731 Y.1731 Y.1731 PM Unmanaged CPE Up Probe Probe Up MEP L6 Ethernet CFM MEP L6
E-LMI E-LMI MPLS VCCV PW OAM
Link OAM MPLS LSP OAM Link OAM Transport OAM
NID CSG/FAN/PAN-SE/AGN-SE CSG/FAN/PAN-SE/AGN-SE NID 82 Enterprise L3VPN Services OAM and PM
IPSLA PM
IP SLA PM
VRF VRF IPSLA IPSLA Probe MPLS VRF OAM Probe ELMI ELMI MPLS VCCV PW OAM MPLS VCCV PW OAM
Service OAM
MPLS LSP OAM
Link OAM MPLS LSP OAM MPLS LSP OAM Link OAM
Transport OAM
CPE CSG/FAN/PAN PA-SE/AGN+SE PAN-SE/AGN+SE CSG/FAN/PAN CPE 83 EPN Manager MEF CE2.0 Provisioning
84 EPN Manager UNI, QoS, OAM
• Service Provisioning includes EOAM configuration: • When provisioning the service • Can be added/modified once the service is provisioned
• EOAM settings include: • ELMI • Link OAM • CCM • Y.1731
• Leveraged by Fault and Performance
85 EPN Manager Service Testing UI
86 CFM Other applications G.8032 Ring Protection CCM for link status detection • ITU G.8032 is the Industry standardized Ethernet ring protection protocol. It can provide sub-50 ms Dedicated VLAN protection and recovery switching for Ethernet CCM – Continuity Check Message traffic in a ring topology while ensuring a loop free every 3.3ms (optional). topology • Primary Fault Detection mechanism based on PHY carrier signal down (1-2ms). Applicable mainly for FO cables • CCM is optional feature used to assure fast Fault Detection when: – PHY has no carrier signal (some copper PHYs, other transport systems) – When it is desired to detect “brain dead” node failures which would result in traffic stoppage without PHY going down Blocking point for Data VLANs
88 Ethernet Fault Detection (EFD)
DOWN • Mechanism for E-OAM UP protocol to bring down L2VPN IPv4 IPv6 MPLS interface “line protocol” state
when a defect is detected. Packet Packet I/O Packet Packet I/O • No traffic flows; Interface is Link OAM “down” to routing/switching DOWN EFD UP MAC layer EFD CFM protocols (MSTP, ARP, IGPs, BGP) – will trigger convergence.
• E-OAM protocol continues to
UP Interface operate; will bring interface up automatically when defect is resolved.
89 EFD - Triggers
• CFM: • Peer MEP defects (wrong MAID/level/interval, bad source MAC/MEP-ID, etc) • Peer interface down • Missing peer MEPs (if cross-check is configured) • Received AIS or RDI
• Link OAM: • Session Up/Down • Capabilities conflict • Discovery Timeout • Link Fault event received • Miswiring detected
90 EFD: CFM MEPs and the ring links G.8032 & CFM ethernet cfm Domain, Direction & domain domain1 Configuration (IOS XR) service link1 down-meps Interval continuity-check interval 3.3ms efd mep crosscheck Remote MEPs mep-id 2 Dedicated VLAN efd protection-switching mep crosscheck CCM – Continuity Check Message mep id 2 Different every 3.3ms (optional). Interface Gig 0/0/0/0 Domains! ethernet cfm mep domain domain1 service link1 mep-id 1 Interface Gig 1/1/0/0 ethernet cfm mep domain domain2 service link2 mep-id 1 Enables G.8032 ERP instance under L2VPN L2vpn ethernet ring g8032 RingA port0 interface g0/0/0/0 port1 interface g0/1/0/0 instance 1 Services rpl port0 owner bridge group ABC vlan-ids 10-100 bridge-domain BD2 aps channel interface Gig 0/0/0/0.2 level 3 interface Gig 0/1/0/0.2 Blocking point for Data VLANs port0 interface g0/0/0/0.1 interface Gig 0/2/0/0.2 port1 interface g1/1/0/0.1
91 G.8032 & CFM EFD: CFM MEPs and the ring links Domain, Direction & ethernet cfm domain g8032_domain level 1 Interval Configuration (IOS XE) service g8032_service evc evc_name vlan 10 direction down continuity-check continuity-check interval 3.3ms Remote MEPs Single interface Dedicated VLAN interface GigabitEthernet0/0/0 service instance 1 ethernet evc_name CCM – Continuity Check Message encapsulation dot1q 10 bridge-domain 10 every 3.3ms (optional). cfm mep domain g8032_domain mpid 1 continuity-check static rmep rmep mpid 2 Enables G.8032 interface GigabitEthernet0/0/1 …… Similar configuration ERP instance under L2VPN ethernet ring g8032 g8032_ring port0 interface GigabitEthernet0/0/0 port1 interface GigabitEthernet0/0/1 instance 1 rpl port0 owner inclusion-list vlan-ids 10,1000-2999 aps-channel Blocking point for Data VLANs port0 service instance 1 port1 service instance 1
92 Microwave ACM ACM (Adaptive Code Modulation) Y.1731 Signaling + EEM Logic
• Traditional Ethernet link protection mechanisms such as STP, REP and G.8032 work based on total loss of link connectivity (ON/OFF)
• Inclement weather often affects signal propagation of microwave radio links, which causes the radio to use more robust modulation (lower bandwidth with ACM) Microwave • Following ACM changes, respective BW-VSM Links frame is send by Microwave system.
• Signal is intercepted and EEM script modify: • HQoS settings CFM 802.1ag • Change of IGP routing metrics to steer traffic into more /Y.1731 optimum paths with pre-emptive protection BW-VSM Vendor Specific
93 Protocol Overview Ethernet Local Management Interface Ethernet LMI
• Provides protocol and mechanisms used for: User Network Interface • Notification of EVC addition, deletion or status (Active, Not Active, Partially Active) (UNI) to CE • Communication of UNI and EVC attributes Customer to CE (e.g. CE-VLAN to EVC map) CE Cisco • CE auto-configuration Enhancement • Notification of Remote UNI name and status to CE
• Asymmetric protocol based on Frame Relay LMI, mainly applicable UNI-C UNI-N to the UNI (UNI-C and UNI-N)
E-LMI • Specification completed by MEF: http://www.metroethernetforum. org/PDFs/Standards/MEF16.doc
95 Ethernet LMI
CE Notification
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31
CE11#show ethernet lmi evc map interface gig0/0 CE11#show ethernet lmi evc detail EVC_P2P_100CE11#show ethernet lmi evc detail EVC_MP_250 UNI Id: CE11_UNI EVC Id: EVC_P2P_100 EVC Id: EVC_MP_250 St Evc Id CE-VLAN interface GigabitEthernet0/0 interface GigabitEthernet0/0 ------Time since Last Full Report: 00:21:23 Time since Last Full Report: 00:25:54 A EVC_MP_250 250 Ether LMI Link Status: Up Ether LMI Link Status: Up A EVC_P2P_100 100 UNI Status: Up UNI Status: Up Key: St=Status, A=Active, P=Partially Active, I=Inactive, UNI Id: CE11_UNI UNI Id: CE11_UNI *=Default EVC, ?=Link Down CE-VLAN/EVC Map Type: Bundling CE-VLAN/EVC Map Type: Bundling VLAN: 100 VLAN: 250
EVC Status: Active EVC Status: Active EVC Type: Point-to-Point EVC Type: Multipoint-to-Multipoint Remote UNI Count: Configured = 1, Active = 1 Remote UNI Count: Configured = 2, Active = 2
UNI Id UNI Status Port UNI Id UNI Status Port ------CE31_UNI Up Remote CE12_UNI Up Remote CE31_UNI Up Remote
96 Configuration Snippets Connectivity Fault Management ME-3400 Cisco IOS CFM Implementation For your 3750-ME reference 6500 Example 1: CFM on Switchport MWR 2941
CE Operator A Operator B CE
MEP MIP GE2/7 GE2/1 UNI NNI interface GigabitEthernet2/7 interface GigabitEthernet2/1 switchport trunk allowed vlan 500 switchport trunk allowed vlan 500 switchport mode trunk switchport mode trunk ethernet cfm mep domain Domain_L4 mpid 111 vlan 500 ethernet cfm mip level 4 vlan 500 ethernet cfm ieee ethernet cfm global Global Configuration UP MEP ! MD and MD Level Manual ethernet cfm domain Domain_L4 level 4 MIP service customer_500_provider vlan 500 MA continuity-check continuity-check interval 1s ! ethernet cfm logging alarm cisco ethernet cfm logging alarm ieee 98 Cisco IOS CFM Implementation For your ISR Routers reference ASR 1000 Example 2: CFM on Layer 3 interfaces
Interface Configuration interface GigabitEthernet0/1 DOWN MEPs ethernet cfm mep domain DOMAIN-L7 mpid 111 vlan 4000 Global Configuration ethernet cfm mep domain DOMAIN-L7 mpid 111 vlan 3000 ethernet cfm mep domain DOMAIN-L7 mpid 111 vlan 2000 ! ethernet cfm ieee interface GigabitEthernet0/1.2000 ethernet cfm global encapsulation dot1Q 2000 ! ip address 1.1.1.1 255.255.255.0 ethernet cfm domain DOMAIN-L7 level 7 ipv6 address 2001:1.1::1/64 service EVPL-2000 vlan 2000 direction down ! continuity-check interface GigabitEthernet0/1.3000 continuity-check interval 1s encapsulation dot1Q 3000 service EVPL-3000 vlan 3000 direction down ip address 2.2.2.1 255.255.255.0 continuity-check ipv6 address 2001:2.2::1/64 continuity-check interval 1s ! service EVPL-4000 vlan 4000 direction down interface GigabitEthernet0/1.4000 continuity-check encapsulation dot1Q 4000 continuity-check interval 1s ip address 3.3.3.1 255.255.255.0 ipv6 address 2001:3.3::1/64
99 For your 7600 Cisco IOS CFM Implementation reference Example 3: CFM on Service Instance with bridge-domain Global Configuration Interface Configuration ethernet cfm ieee interface GigabitEthernet2/1 ethernetcfm global description UNI ! service instance 665 ethernetsampleEVC ethernetcfm domain Domain_L5 level 5 encapsulation dot1q100 service E2E_MA_name evc sampleEVC vlan 777 rewrite ingress tag pop 1 symmetric continuity-check bridge-domain 777 continuity-check interval 1s cfm mep domain Domain_L5mpid 1111 ! cfm mip level 6 ethernet evc sampleEVC ! ! interface GigabitEthernet2/2 ethernet cfm logging alarm cisco description EVCNNI ethernet cfm logging alarm ieee service instance 666 ethernetsampleEVC 2/2 encapsulation dot1q200 dot1q 200 Example includes Service rewrite ingress tag pop 1 symmetric UP MEP BD 777 Instance (with BD) and bridge-domain 777 switchports in the same cfm mip level 5 broadcast domain ! interface GigabitEthernet2/3 2/3 description switchportNNI switchport trunk allowed vlan777 dot1q 777 switchport mode trunk switchport ethernet cfm mip level 5 vlan 777
100 For your 7600 Cisco IOS CFM Implementation reference Example 4: CFM on Service Instance with Xconnect
Global Configuration Interface Configuration ethernet cfm ieee interface GigabitEthernet2/1 ethernet cfm global description PW AC ! service instance 665 ethernet sampleEVC ethernet cfm domain Domain_L3 level 3 encapsulation dot1q 100 service sampleMA evc sampleEVC rewrite ingress tag pop 1 symmetric continuity-check xconnect 2.2.2.2 123 pw-class vlan-xconnect continuity-check interval 1s cfm mip level 4 ! cfm mep domain Domain_L3 mpid 1111 ethernet evc sampleEVC ! ! pseudowire-class vlan-xconnect encapsulation mpls PW ! ethernet cfm logging alarm cisco ethernet cfm logging alarm ieee 2/1 dot1q 100 xconnect
UP MEP
101 For your ASR 9000 Cisco IOS XR CFM Implementation reference Example 5: CFM on Layer 2 and Layer 3 Interfaces
Global Configuration Interface Configuration ethernet cfm interface GigabitEthernet0/1/0/1.156 l2transport domain Domain_L2 level 2 encapsulation dot1q 156 service sampleMA1 down-meps ethernet cfm continuity-check interval 1s mep domain Domain_L2 service sampleMA1 mep-id 15 log continuity-check errors ! log continuity-check mep changes interface GigabitEthernet0/1/0/2 ! ipv4 address 20.20.14.15 255.255.255.0 service sampleMA2 down-meps ethernet cfm continuity-check interval 1s mep domain Domain_L2 service sampleMA2 mep-id 15 log continuity-check errors log continuity-check mep changes 0/1/0/1.156 ! dot1q 156 PW ! EoMPLS AC !
0/1/0/2 DOWN MEP L3 interface
102 Interworking scenarios Ethernet OAM Interworking Interworking Scenarios Main Examples
Link OAM CFM
CFM E-LMI
MPLS PW OAM E-LMI
104 Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE Access Aggregation Aggregation Access CE
Point-to-Point Ethernet Service
105 Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE Access Aggregation Aggregation Access CE
E-LMI CFM E-LMI Link OAM Link OAM
CFM CFM to E-LMI IW to E-LMI IW
Link OAM Link OAM to CFM IW to CFM IW
OAM protocol positioning
106 Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services CE Notification
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31
ELMI Status Enquiry message (Full Status report) ELMI Status message (Full Status report)
Local UNI ID Example: CE11_UNI CE-VLAN/EVC Map type Service_Multiplexing EVC ID EVC_P2P_100 EVC Type Point_to_Point CE-VLAN/EVC Map vlan 100 EVC Status New, Active Remote UNI count – configured 1 Remote UNI count – active 1 Cisco enhancements to ELMI Remote UNI ID CE31_UNI Remote UNI status UP
107 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services CE Notification
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31
CE11#show ethernet lmi evc map interface gig0/0 CE11#show ethernet lmi evc detail EVC_P2P_100CE11#show ethernet lmi evc detail EVC_MP_250 UNI Id: CE11_UNI EVC Id: EVC_P2P_100 EVC Id: EVC_MP_250 St Evc Id CE-VLAN interface GigabitEthernet0/0 interface GigabitEthernet0/0 ------Time since Last Full Report: 00:21:23 Time since Last Full Report: 00:25:54 A EVC_MP_250 250 Ether LMI Link Status: Up Ether LMI Link Status: Up A EVC_P2P_100 100 UNI Status: Up UNI Status: Up Key: St=Status, A=Active, P=Partially Active, I=Inactive, UNI Id: CE11_UNI UNI Id: CE11_UNI *=Default EVC, ?=Link Down CE-VLAN/EVC Map Type: Bundling CE-VLAN/EVC Map Type: Bundling VLAN: 100 VLAN: 250
EVC Status: Active EVC Status: Active EVC Type: Point-to-Point EVC Type: Multipoint-to-Multipoint Remote UNI Count: Configured = 1, Active = 1Remote UNI Count: Configured = 2, Active = 2
UNI Id UNI Status Port UNI Id UNI Status Port ------CE31_UNI Up Remote CE12_UNI Up Remote
CE31_UNI Up Remote 108 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services
CE Notification—VLAN ID Mismatch
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31
CE Configured with the correct C-VLAN (e.g. vid 100)
CE11(config)#interface gig0/0.100 CE11(config-subif)#encapsulation dot1Q 100
CE11#show ip interface brief Interface IP-Address OK? Method Status Protocol
109 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services CE Notification—VLAN ID Missmatch
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31
CE Configured with the incorrect C-VLAN (e.g. vid 1300)
CE11(config)#interface gig0/0.100 CE11(config-subif)#encapsulation dot1Q 1300
Jan 26 00:15:39.546: %ETHER_LMI-6-MISMATCHED_VLAN_NOT_CONFIGURED: VLAN 100 not Configured but in VLAN mapping for UNI GigabitEthernet0/0
Jan 26 00:15:39.546: %ETHER_LMI-6-MISMATCHED_VLAN_CONFIGURED: VLAN 1300 configured but not in VLAN mapping for UNI GigabitEthernet0/0 Interface
CE11#show ip interface brief Interface IP-Address OK? Method Status Protocol Proactive ELMI Action
110 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services
Failure Scenario: Network Failure
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31 X
CFM remote MEP timeout CFM remote MEP timeout
MEP Down (timeout) alarm MEP Down (timeout) alarm DefRemoteCCM IEEE alarm DefRemoteCCM IEEE alarm EVC declared Inactive EVC declared Inactive CFM to ELMI Interworking CFM to ELMI Interworking
ELMI Status message ELMI Status message Async EVC report Async EVC report
ELMI action: ELMI action: CE brings down CE brings down (sub)interface (sub)interface
111 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31 X
UPE11# *Apr 8 04:33:44.911: %E_CFM-3-REMOTE_MEP_DOWN: Remote MEP mpid 3100 vlan 100 MA name Cisco-defined customer_100_provider in domain PROVIDER_DOMAIN changed state to down with event code TimeOut. alarm *Apr 8 04:33:44.911: %ETHER_SERVICE-6-EVC_STATUS_CHANGED: status of EVC_P2P_100 changed to InActive
*Apr 8 04:33:47.587: %E_CFM-3-FAULT_ALARM: A fault has occurred in the network for the local MEP having mpid 1100 vlan 100 for service MA name customer_100_provider with the event code DefRemoteCCM. IEEE-defined UPE11#show ethernt cfm errors alarm ------MPID Domain Id Mac Address Type Id Lvl MAName Reason Age ------Error DB 3100 PROVIDER_DOMAIN aabb.cc00.0599 Vlan 100 4 customer_100_provider Lifetime Timer Expired 119s
112 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31
UPE11#ping ethernet aabb.cc00.0599 domain PROVIDER_DOMAIN vlan 100 Type escape sequence to abort. Sending 5 Ethernet CFM loopback messages to aabb.cc00.0599, timeout is 5 seconds :..... Success rate is 0 percent (0/5)
UPE11#traceroute ethernet aabb.cc00.0599 domain PROVIDER_DOMAIN vlan 100 Type escape sequence to abort. TTL 64. Linktrace Timeout is 5 seconds Tracing the route to aabb.cc00.0599 on Domain PROVIDER_DOMAIN, Level 4, vlan 100 Traceroute sent via Ethernet0/1.100, path found via MPDB
B = Intermediary Bridge ! = Target Destination * = Per hop Timeout ------MAC Ingress Ingr Action Relay Action Hops Host Forwarded Egress Egr Action Previous Hop ------B 1 AGG11 aabb.cc00.0399 Et0/0.100 IngOk RlyMPDB Forwarded Et0/1.100 EgrOK aabb.cc00.0299
* 113 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services
CE 11 uPE 11 AGG 11 AGG 31 uPE 31 CE 31 X CE11# *Apr 8 04:33:44.991: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0/0.100, changed state to down CE11#show ethernet lmi evc detail EVC_P2P_100 EVC Id: EVC_P2P_100 interface Ethernet0/0 Time since Last Full Report: 00:01:13 Ether LMI Link Status: Up UNI Status: Up UNI Id: CE11_UNI CE-VLAN/EVC Map Type: Service Multiplexing with no bundling VLAN: 100 Network Failure: EVC Status: Inactive EVC Type: Point-to-Point Remote UNI shows Remote UNI Count: Configured = 1, Active = 0 Unreachable
UNI Id UNI Status Port ------CE31_UNI Unreachable Remote
114 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services
Failure Scenario: UNI Link Down
CE Access Aggregation Aggregation Access CE X UNI Link Down
EVC declared Inactive
CFM CCM CFM MEP Up (port state Down) alarm Interface Status TLV DefMACstatus IEEE alarm “isDown” EVC declared Inactive CFM to ELMI InterWorking ELMI Status message Async EVC report ELMI action: CE brings down (sub)interface 115 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services
CE Access Aggregation Aggregation Access CE X UPE11# *Apr 8 04:41:54.823: %E_CFM-6-REMOTE_MEP_UP: Continuity Check message is received from a remote MEP with mpid 3100 vlan 100 MA name customer_100_provider domain PROVIDER_DOMAIN interface status Down event code PortState.
*Apr 8 04:41:54.823: %ETHER_SERVICE-6-EVC_STATUS_CHANGED: status of EVC_P2P_100 changed to InActive
*Apr 8 04:41:57.451: %E_CFM-3-FAULT_ALARM: A fault has occurred in the network for the local MEP having mpid 1100 vlan 100 for service MA name customer_100_provider with the event code DefMACstatus.
UPE11#show ethernet cfm maintenance-point remote ------MPID Domain Name MacAddress IfSt PtSt Lvl Domain ID Ingress RDI MA Name Type Id SrvcInst EVC Name Age ------3100 PROVIDER_DOMAIN aabb.cc00.0599 Down Up 4 PROVIDER_DOMAIN Et0/1.100 - customer_100_provider Vlan 100 N/A N/A 0s Total Remote MEPs: 1 116 For your Deploying Carrier Ethernet OAM reference Ethernet Layer 2 VPN Services
CE Access Aggregation Aggregation Access CE
CE11# X
*Apr 8 04:41:54.907: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0/0.100, changed state to down
CE11#show ethernet lmi evc detail EVC_P2P_100 EVC Id: EVC_P2P_100 interface Ethernet0/0 Time since Last Full Report: 00:01:07 Ether LMI Link Status: Up UNI Status: Up UNI Id: CE11_UNI CE-VLAN/EVC Map Type: Service Multiplexing with no bundling VLAN: 100 UNI Failure: Remote UNI shows DOWN EVC Status: Inactive EVC Type: Point-to-Point Remote UNI Count: Configured = 1, Active = 0
UNI Id UNI Status Port ------
CE31_UNI Down Remote 117 Platform Support Ethernet OAM Aggregation Core Platform Support Edge Access Customer Premise
Cisco 7600 CRS-1 Catalyst 6500 ASR 9000 CRS-3 Cisco ME3400 Catalyst 4500 XR 12000 CRS-X Cisco ME3400E Cisco 880/890 Catalyst 4900 NCS 6000 Cisco ME3600X Cisco 1800/2800/3800 Cisco ONS 15454 Cisco ME3800X Cisco 1900/2900/3900 Cisco CGS 2520 Cisco ME4924 Cisco 7200/7201/7301 Cisco ME6524 Cisco MWR 2941 Catalyst 3750-ME Cisco CGR 2010 ASR 901, 920 Cisco ASR 1000 ASR 903, 903
119 Summary Carrier Ethernet OAM
• Comprehensive suite of OAM mechanisms • Link Monitoring • In-Service Monitoring • Service Turnup
• Embedded OAM capabilities in Cisco’s Metro Ethernet line of products • Carrier Ethernet ASICs, Hardware Offload to FPGA/NPU
• End to End OAM feature support for complete visibility in to Service Provider’s access network. • CPE, Access, Aggregation, Core platforms
• Standards based implementation for integration in to existing network • IEEE, ITU-T, MEF, IETF
121 Call to Action
• Visit the World of Solutions for • Cisco Campus • Walk in Labs • Technical Solution Clinics
• Meet the Engineer
• Lunch and Learn Topics
• DevNet zone related sessions Complete Your Online Session Evaluation
• Please complete your online session evaluations after each session. Complete 4 session evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt.
• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations
123 Thank you
Acronyms Acronym Acronym AIS Alarm Indication Signal MEF Metro Ethernet Forum CCM Continuity Check Message MEN Metro Ethernet Network CCMDB CCM Data Base (see CCM) MEP Maintenance Association End Point CE Customer Edge MEPID MEP Identifier (see MEP) CFM Connectivity Fault Management MHF MIP Half Function (see MIP) EFM Ethernet in the First Mile MIB Management Information Base E-LMI Ethernet LMI (see LMI) MIP Maintenance Domain Intermediate Point E-OAM Ethernet OAM (see OAM) MP Maintenance Point EVC Ethernet Virtual Connection OAM Operations, Administration and Maintenance IEEE Institute of Electrical and Electronics Engineers PDU Protocol Data Unit ITU International Telecommunication Union PE Provide Edge LBM Loopback Message RDI Remote Defect Indicator LBR Loopback Reply RFI Remote Failure Indicator LMI Local Management Interface TLV Type, Length, Value LTM Linktrace Message UNI User to Network Interface LTR Linktrace Reply UNI-C Customer side of UNI (see UNI) MA Maintenance Association UNI-N Network side of UNI (see UNI) MAID MA Identifier (see MA) VID VLAN Identifier MD Maintenance Domain VLAN Virtual LAN 126 E-OAM Documentation at Cisco.com • IOS Carrier Ethernet Configuration Guide • http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/15-2s/ce-15-2s-book.html • IOS XE Carrier Ethernet Configuration Guide • http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/ce-xe-3s-book.html • Service Diagnostics Documentation and Scripts • http://www.cisco.com/go/iossd • Link OAM Configuration Guide • http://www.cisco.com/en/US/docs/ios/cether/configuration/guide/ce_oam.html • Y.1731 Fault Management Configuration Guides • http://www.cisco.com/en/US/docs/ios/cether/configuration/guide/ce_cfm-ieee_y1731.html • Y.1731 Performance Management Configuration Guides • http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/ce-y1731-perfmon.html • Ethernet Local Management Interface (E-LMI) Configuration Guides • http://www.cisco.com/en/US/docs/ios/cether/configuration/guide/ce_elmi-pe.html (PE role) •http://www.cisco.com/en/US/docs/ios/cether/configuration/guide/ce_elmi_ps10591_TSD_Products_Configur ation_Guide_Chapter.html (CE role) 127 E-OAM Documentation at Cisco.com
• IP SLAs for Metro Ethernet • http://www.cisco.com/en/US/docs/ios/ipsla/configuration/guide/sla_metro_ethernet.html • IP SLAs LSP Health Monitor • http://www.cisco.com/en/US/docs/ios/ipsla/configuration/guide/sla_lsp_mon_autodisc.html • Cisco ME3600X/ME3800X Ethernet OAM, CFM and E-LMI Configuration Guide •http://www.cisco.com/en/US/docs/switches/metro/me3600x_3800x/software/release/15.1_2_ey/configuratio n/guide/swoam.html • Cisco ME3400E CFM, Y.1731, E-LMI and Link OAM Configuration Guide •http://www.cisco.com/en/US/docs/switches/metro/me3400e/software/release/12.2_58_se/configuration/guid e/swoam.html • Cisco ME3400 CFM, Y.1731, E-LMI and Link OAM Configuration Guides •http://www.cisco.com/en/US/docs/switches/metro/me3400/software/release/12.2_58_se/configuration/guide /swoam.html • Cisco Catalyst 3750-ME CFM, E-LMI and Link OAM Configuration Guides •http://www.cisco.com/en/US/docs/switches/metro/catalyst3750m/software/release/12.2_58_se/configuration /guide/swoam.html 128 E-OAM Documentation at Cisco.com • Cisco Catalyst 4500 CFM and OAM Configuration Guide • http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/3.1.1SG/configuration/guide/CFM.html • http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/15.02SG/configuration/guide/E_OAM.html • Cisco Catalyst 4500 Y.1731 AIS and RDI Configuration Guide • http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/3.1.1SG/configuration/guide/Y1731.html • http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/15.02SG/configuration/guide/Y1731.html • Cisco ONS 15454 ML-MR-10 CFM, E-LMI and Link OAM Configuration Guide • http://www.cisco.com/en/US/docs/optical/15000r9_0/ethernet/454/guide/45490a_eoamonmlmr.html • EoMPLS Remote Link Failure Notification via E-LMI (aka Remote Port Shutdown) • http://www.cisco.com/en/US/docs/ios/12_2sr/12_2srb/feature/guide/srbrpsdn.html • Cisco IOS Carrier Ethernet Command Reference • http://www.cisco.com/en/US/docs/ios/cether/command/reference/ce_book.html • Cisco ASR 9000 Ethernet OAM Configuration Guide •http://www.cisco.com/en/US/docs/routers/asr9000/software/asr9k_r4.1/interfaces/configuration/guide/hc41eo am.html
129 E-OAM Documentation at Cisco.com • Cisco CRS-1 Ethernet OAM Configuration Guide • http://www.cisco.com/en/US/docs/routers/crs/software/crs_r4.1/interfaces/configuration/guide/hc41eoam.html • Cisco CRS-1 Ethernet OAM Command Reference •http://www.cisco.com/en/US/docs/routers/crs/software/crs_r4.1/interfaces/command/reference/b_interfaces_ cr41crs_chapter_0110.html • Cisco XR 12000 Ethernet OAM Configuration Guide •http://www.cisco.com/en/US/docs/routers/xr12000/software/xr12k_r4.1/interfaces/configuration/guide/hc41eo am.html • Cisco XR 12000 Ethernet OAM Command Reference •http://www.cisco.com/en/US/docs/routers/xr12000/software/xr12k_r4.1/interfaces/command/reference/b_inter faces_cr41xr12k_chapter_0101.html • Cisco ASR 1000 CFM Configuration Guide • http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/ce-cfm-ieee-xe.html • http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/ce-y1731-perfmon.html • Cisco ASR 901 Ethernet OAM Configuration Guide • http://www.cisco.com/en/US/docs/wireless/asr_901/Configuration/Guide/oam.html • http://www.cisco.com/en/US/products/ps6776/products_user_guide_list.html 130 Thank you