Advanced NetFlow
Session BRKNMS-3132 The Content of This Session Is…
. Not about A level one type of presentation Introduction to IP accounting and NetFlow Marketing slides NetFlow collector details The ecosystem partners applications and mediations Many platform specific details . About New features Advanced information And a few scenarios … Assuming the NetFlow basics are known
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 2 For Your Session Abstract Reference
. This advanced session presents the latest NetFlow developments: new features, NetFlow version 9, and its standardization at the IETF. The new Flexible NetFlow feature is covered in detail. Technical details of the new features are addressed with configuration examples, show commands, tricks, and best practice advice. Scenarios such as NetFlow for security, NetFlow for application visibility, and NetFlow for capacity planning are covered. The NetFlow performance impact is also discussed, as well as the support matrix of all NetFlow features. . This session is for enterprise, service provider, and NREN experts engaged in designing, maintaining, and troubleshooting security, capacity planning, and accounting solutions. Attendees should be familiar with network management basics and should already have some understanding of NetFlow, perhaps by already having taken the introductory session.
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 3 Agenda
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 Version 5 Flow Format
Flow Key vs. Non-Key Field From/to . Packet count . Source IP address Usage . Byte count . Destination IP address
Time . Start sysUpTime . Source TCP/UDP port of Day . End sysUpTime . Destination TCP/UDP port
Application Port . Input ifIndex . Next hop address Utilization . Output ifIndex . Source AS number . . Type of service Dest. AS number . QoS . TCP flags Source prefix mask . . Protocol Dest. Prefix mask
Routing and Peering
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 5 NetFlow Cache Example 1. Create and update flows in NetFlow cache Src Src Src Dst Dst Dst Bytes/ Srclf SrclPadd Dstlf DstlPadd Protocol TOS Flgs Pkts NextHop Active Idle Port Msk AS Port Msk AS Pkt 00A Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 /24 5 00A2 /24 15 10.0.23.2 1528 1745 4 2 Fa1/0 173.100.3.2 Fa0/0 10.0.227.12 6 40 0 2491 15 /26 196 15 /24 15 10.0.23.2 740 41.5 1 00A Fa1/0 173.100.20.2 Fa0/0 10.0.227.12 11 80 10 10000 /24 180 00A1 /24 15 10.0.23.2 1428 1145.5 3 1 Fa1/0 173.100.6.2 Fa0/0 10.0.227.12 6 40 0 2210 19 /30 180 19 /24 15 10.0.23.2 1040 24.5 14
. Inactive timer expired (15 sec is default) . Active timer expired (30 min is default) 2. Expiration . NetFlow cache is full (oldest flows are expired) . RST or FIN TCP flag
Src Src Src Dst Dst Dst Bytes/ Srclf SrclPadd Dstlf DstlPadd Protocol TOS Flgs Pkts NextHop Active Idle Port Msk AS Port Msk AS Pkt Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 00A2 /24 15 10.0.23.2 1528 1800 4
3. Aggregation
4. Export version E.g., Protocol-Port Aggregation Non-aggregated flows—export version 5 or 9 Scheme Becomes 5. Transport protocol Protocol Pkts SrcPort DstPort Bytes/Pkt Export Payload 11 11000 00A2 00A2 1528 Packet (UDP, SCTP) (Flows) Aggregated Flows—Export Version 8 or 9 Header Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 NetFlow Export Version 5 and Main Cache Configuration Example
Router(config)# interface
Router(config)# ip flow-cache entries
Router(config)# ip flow-export version 5 peer-as Router(config)# ip flow-export destination 10.10.10.10 1234 Router(config)# ip flow-export source loopback 0
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 NetFlow Flow Keys on the Router
. By default, the 7 flow keys are: Source IP address, destination IP address, source port, destination port, Layer 3 protocol type, TOS byte (DSCP), input interface . The 12 NetFlow aggregations allows to reduce/change the number of flow keys Example: source prefix aggregation = source network, source interface Can be seen as a different view of the main cache . Egress NetFlow, MPLS-aware NetFlow, etc. Will specify new flow keys . Note: on the Cisco Catalyst®, we speak of the flow mask Define the flow keys
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 Flow Keys on the Cisco Catalyst 6500/7600 - The Flow Mask
Full-Interface VLAN SRC IP DST IP IP Protocol Src Port Dst Port Full VLAN SRC IP DST IP IP Protocol Src Port Dst Port Destination-Source-Interface VLAN SRC IP DST IP IP Protocol Src Port Dst Port Source-Only VLAN SRC IP DST IP IP Protocol Src Port Dst Port Destination-Only VLAN SRC IP DST IP IP Protocol Src Port Dst Port Destination-Source VLAN SRC IP DST IP IP Protocol Src Port Dst Port
Flow Keys in Orange Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 Extensibility and Flexibility Requirements Phases Approach
. Traditional NetFlow with the v5 or v8 NetFlow export New requirements: build something flexible and extensible . Phase One: NetFlow Version 9 Advantages: extensibility Integrate new technologies/data types quicker (MPLS, IPv6, BGP next hop, etc.) Exporting Integrate new aggregations quicker Process Note: for now, the template definitions are fixed . Phase Two: Flexible NetFlow Advantages: cache and export content flexibility User selection of flow keys User definition of the records Metering Process
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 NetFlow Partners in CTDP Program
Traffic Analysis
Denial of Service Billing
CS-Mars
More info: http://www.cisco.com/warp/public/732/Tech/nmp/netflow/partners/commercial/
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 NetFlow Open Source Tools
Product Name Primary Use Comment OS Cflowd Traffic Analysis No longer supported UNIX Flow-tools Collector Device Scalable UNIX Flowd Collector Device Support V9 BSD, Linux FlowScan Reporting for Flow- UNIX Tools IPFlow Traffic Analysis Support V9, IPv4, Linux, IPv6, MPLS, SCTP, FreeBSD, etc.. Solaris NetFlow Guide Reporting Tools BSD, Linux NetFlow Monitor Traffic Analysis Supports V9 UNIX Netmet Collector Device V5, support v9 Linux NTOP Security Monitoring UNIX Stager Reporting for Flow- UNIX Tools Nfdump/nfsen Traffic Analysis Supprot V5 and v9 UNIX
Different costs: implementation and customization
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 Agenda
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 NetFlow Version 9
. Version 9 is an export protocol No changes to the metering process . Version 9 is based on templates and separate flow records Templates composed of type and length Flow records composed of template ID and value Sent the template regularly (configurable), because of UDP . Support: 800, 1700, ISR (1800, 2800, 3800), ISR-G2 (1900, 2900, 3900), 2600, 3200, 3600, 3700, 6500/7600, 7200, 7300, 7500, cat6000, 7600, 10000, 12000 (IOS and IOS-XR), CRS-1, ASR 1000, ASA 5580, Nexus 7000 and Nexus 1000V . RFC3954 Cisco Systems® NetFlow Services Export Version 9 NetFlow patent: intellectual property right statement at the IETF website
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 NetFlow Version 9 Export Packet
Template 1 Template 2
Template FlowSet Data FlowSet Data FlowSet FlowSet ID #1 FlowSetFlowSet ID ID#1 #2 H E Template Template Record Record Data Data Data A Template Template Record Record Record D ID #1 ID #2 (Field (Field (Field E (Specific (Specific R Field Field Values) Values) Values) Types and Types and Lengths) Lengths)
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 NetFlow Version 9 Export Packet Options Template FlowSet Specifies the Scope: Cache, System, Template, etc. Template 3
Options Data FlowSet Template FlowSet FlowSet ID #3 H E Option Template Record Option Option A Template Data Data D ID #3 Record Record E (Specific Scope, (Field (Field R Field Types Values) Values) and Lengths)
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 Interface Name Export with NetFlow Version 9
. Example of options template FlowSet . NetFlow exports the ifIndex . Instead of the collector polling the ifName MIB variable for a specific ifIndex, the matching (ifIndex, ifName) is sent in an option data record
Router(config)# ip flow-export interface-names
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 17 NetFlow Version 9 Main Cache Configuration
router(config)# ip flow-export version [5|9] [origin-as|peer-as] [bgp-nexthop] router(config)# ip flow-export template options export-stats router(config)# ip flow-export template options timeout-rate 5 router(config)# ip flow-export template options refresh-rate 60 router(config)# ip flow-export template timeout-rate 5 router(config)# ip flow-export template refresh-rate 20 router(config)# ip flow-export destination 10.10.10.10 9996
(Options)(Options) Templates Templates Sent EverySent FiveEvery Minutes Five Minutesor Every or 20 20 Packets Packets
. Should you export from the main cache with NetFlow Version 5 or Version 9?
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 NetFlow Version 9 Aggregation Cache Configuration
router(config)# ip flow-aggregation cache bgp-nexthop-tos router(config-flow-cache)# export destination 11.11.11.11 9999 router(config-flow-cache)# export version ? 9 Version 9 export format router(config-flow-cache)# export version 9 router(config-flow-cache)# enabled
In this case, we have only version 9. Why?
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 Agenda
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 Multicast NetFlow
. Multicast NetFlow ingress One flow with the replicated number of packets/bytes
Router(config-if)# ip multicast netflow ingress . Multicast NetFlow egress One per outgoing interface, with the nonreplicated number of packets/bytes
Router(config-if)# ip multicast netflow egress . Deduced the replication factor . Display the multicast data that fails the Reverse Path Forwarding (RPF) check . No NetFlow export over multicast
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 NetFlow: Monitoring IPv6 Traffic
. Monitors the IPv6 traffic . Based on NetFlow version 9 . For both ingress and egress traffic . Non-sampled . No NetFlow export over IPv6; still IPv4 (except Nexus 7000) . All configuration is the same: replace “ip” by “ipv6” in the CLI . 12.3(7)T, 12.2(33)SXH, 12.2(33)SRB
. Beginning with Cisco IOS® Release 12.4(20)T, traditional NetFlow for IPv6 is being replaced by flexible NetFlow for IPv6
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 NetFlow Reliable Export with SCTP
. SCTP: stream control transport protocol (RFC4960) Reliable data transfer Congestion control and avoidance Multihoming support One association support for multi-streams . SCTP-PR: SCTP partially reliable (RFC3578) Three modes of reliability: reliable, partial reliable, unreliable . Advantages: (Options) templates sent reliably . Backup Options: Fail-over mode: open the backup connection when the primary fails Redundant mode: open the backup connection in advance, and already send the templates
Note: “An Introduction to SCTP”, RFC 3286 Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 NetFlow Reliable Export with SCTP
Security/Monitoring Billing
SCTP: SCTP: Partially SCTP Backup: Reliable Reliable SCTP Backup: Fail-Over Mode Redundant Mode
Main Destination- Cache Prefix Aggr.
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Agenda
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 Typical NetFlow Deployment
NetFlow for Monitoring
NetFlow for Core NetFlow for Security Traffic Matrix
Managed Services: Application Visibility ISP NetFlow for Peering
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 26 Flexible NetFlow High-Level Concepts and Advantages
. Flexible NetFlow feature allows user configurable NetFlow record formats, selecting from a collection of fields: Key, non-key, counter, timestamp . Advantages: Tailor a cache for specific applications, not covered by existing 21 NetFlow features in traditional NetFlow Different NetFlow caches: per subinterface, per direction (ingress, egress), per sampler, per … Better scalability since flow record customization for particular application reduces number of flows to monitor
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 Flexible NetFlow Multiple Monitors with Unique Key Fields
Traffic Flow Flow Monitor Monitor 1 2
Key Fields Packet 1 Non-Key Fields Key Fields Packet 1 Non-Key Fields Source IP 3.3.3.3 Packets Source IP 3.3.3.3 Packets Destination IP 2.2.2.2 Bytes Dest IP 2.2.2.2 Timestamps Source Port 23 Timestamps Input Interface Ethernet 0 Destination Oort 22078 Next Hop Address SYN Flag 0 Layer 3 Protocol TCP - 6 TOS Byte 0 Input Interface Ethernet 0
Traffic Analysis Cache Security Analysis Cache Source Dest. Source Dest. Input Protocol TOS … Pkts Source IP Dest. IP Input I/F Flag … Pkts IP IP Port Port I/F 3.3.3.3 2.2.2.2 E0 0 … 11000 3.3.3.3 2.2.2.2 23 22078 6 0 E0 … 1100
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 Flexible NetFlow Model
Interface
Monitor “C” Monitor “A” Monitor “B”
Exporter “M” Record “X” Exporter “M”
Record “Z” Exporter “N”
Record “Y”
. A single record per monitor . Potentially multiple monitors per interface . Potentially multiple exporters per monitor
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 Service Planning FNF Configuration – Example
1. Configure the Exporter Router(config)# flow exporter my-exporter Where do I want my data sent? Router(config-flow-exporter)# destination 1.1.1.1 2. Configure the Flow Record
Router(config)# flow record my-record Router(config-flowWhat-record)# data do Imatch want ipv4to meter?destination address Router(config-flow-record)# match ipv4 source address Router(config-flow-record)# collect counter bytes 3. Configure the Flow Monitor
Router(config)# flow monitor my-monitor Router(configHow -doflow I want-monitor)# to cache exporter information my-exporter Router(config-flow-monitor)# record my-record 4. Apply to an Interface
Router(config)#Which interface interface do Is3 want/0 to monitor? Router(config-if)# ip flow monitor my-monitor input
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 Flexible NetFlow User-Defined Record Configuration
Router(config)# flow record my-record Specify a Key Field Router(config-flow-record)# match Router(config-flow-record)# collect Specify a Non-Key Kield Router(config-flow-record)# match ? application Application Fields datalink Datalink (layer 2) fields flow Flow identifying fields interface Interface fields ipv4 IPv4 fields ipv6 IPv6 fields routing routing attributes transport Transport layer field
Router(config-flow-record)# collect ? application Application Fields counter Counter fields datalink Datalink (layer 2) fields flow Flow identifying fields interface Interface fields ipv4 IPv4 fields ipv6 IPv6 fields routing IPv4 routing attributes timestamp Timestamp fields transport Transport layer fields Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 31 Flexible Flow Record: Key Fields
Flow IPv4 IPv6 Sampler ID IP (Source or Payload Size IP (Source or Destination) Payload Size Direction Destination) Prefix (Source or Packet Section Prefix (Source or Packet Section Interface Destination) (Header) Destination) (Header) Input Mask (Source or Packet Section Mask (Source or Packet Section Destination) (Payload) Output Destination) (Payload) Minimum-Mask Minimum-Mask Layer 2 (Source or TTL (Source or DSCP Destination) Source Destination) Options VLAN Protocol bitmap Protocol Extension Headers Destination Fragmentation VLAN Version Traffic Class Hop-Limit Flags Fragmentation Flow Label Length Source MAC Precedence Offset address Option Header Next-header Identification DSCP Header Length Version Destination Header Length TOS MAC address Total Length Payload Length
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 Flexible Flow Record: Key Fields NEW
Routing Transport Application
src or dest AS Destination Port TCP Flag: ACK Application ID* Peer AS Source Port TCP Flag: CWR Traffic Index ICMP Code TCP Flag: ECE Multicast Forwarding ICMP Type TCP Flag: FIN Status IGMP Type* TCP Flag: PSH Replication IGP Next Hop Factor* TCP ACK Number TCP Flag: RST BGP Next Hop TCP Header Length TCP Flag: SYN RPF Check Drop* Input VRF TCP Sequence Number TCP Flag: URG Name Is-Multicast TCP Window-Size UDP Message Length NEW TCP Source Port UDP Source Port TCP Destination Port UDP Destination Port
TCP Urgent Pointer *: IPv4 Flow only
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 33 Flexible Flow Record: Non-Key Fields
Counters Timestamp IPv4 IPv4 and IPv6
sysUpTime First Total Length Total Length Bytes Packet Minimum (*) Minimum (**) Total Length Total Length Bytes Long sysUpTime First Packet Maximum (*) Maximum (**) Bytes Square Sum TTL Minimum
Bytes Square Sum Long TTL Maximum
Packets
Packets Long
. Plus any of the potential “key” fields: will be the value from the first packet in the flow
(*) IPV4_TOTAL_LEN_MIN, IPV4_TOTAL_LEN_MAX (**)IP_LENGTH_TOTAL_MIN, IP_LENGTH_TOTAL_MAX
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 34 Flow Exporter Configuration
flow exporter
New in 12.4(20)T NetFlow Exported Packets Go Through QoS, Crypto-Map, etc…
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 Transition Steps from Traditional NetFlow to Flexible NetFlow
Router(config)# flow monitor my-monitor Router(config-flow-monitor)# record netflow ipv4 ? as AS aggregation schemes as-tos AS and TOS aggregation schemes bgp-nexthop-tos BGP next-hop and TOS aggregation schemes destination-prefix Destination Prefix aggregation schemes destination-prefix-tos Destination Prefix and TOS aggregation schemes original-input Traditional IPv4 input NetFlow original-output Traditional IPv4 output NetFlow …
. First, flexible NetFlow metering process with NetFlow Version 5 flow records and Version 5 export . Second, flexible NetFlow metering process with NetFlow Version 5 flow records and Version 9 export . Third, user-defined flow records with Version 9 export
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 36 Flexible Monitor Configuration
Three Types of Cache: See Next Slides flow monitor
Collect Protocol Collect Size Distribution Statistics Distribution Statistics
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 37 Three Types of NetFlow Caches
. Normal cache (traditional NetFlow) More flexible active and inactive timers: one second minimum . Immediate cache Flow accounts for a single packet Desirable for real-time traffic monitoring, DDoS detection, logging Desirable when only very small flows are expected (ex: sampling) Caution: may result in a large amount of export data . Permanent cache To track a set of flows without expiring the flows from the cache Entire cache is periodically exported (update timer) After the cache is full (size configurable), new flows will not be monitored Uses update counters rather than delta counters
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 38 Complete Permanent Flexible NetFlow Configuration Example
. Per DSCP accounting flow record definition:
Router(config)# flow record my-dscp-record 64-Bit Router(config-flow-record)# match ipv4 dscp Counter Router(config-flow-record)# match interface input Router(config-flow-record)# collect counter bytes long Router(config-flow-record)# collect counter packets long
Router(config)# flow monitor my-dscp-monitor Router(config-flow-record)# description dscp:bytes and packets Router(config-flow-record)# record my-dscp-record Router(config-flow-record)# cache type permanent Router(config-flow-record)# cache entries 256
Router(config)# interface GigabitEthernet 0/1 Router(config)#. This replaces ip flowthe IP monitor accounting my-dscp precedence-monitorfeatureinput
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 Complete Permanent Flexible NetFlow Configuration Example
Extra Options: CSV, Table, Record
Router#show flow monitor my-dscp-monitor cache Cache type: Permanent Cache size: 256 Current entries: 0 High Watermark: 0
Flows added: 0 Updates sent ( 1800 secs) 0
IP DSCP INTF INPUT bytes long perm pkts long perm ======0x00 Gi0/1 1000 10 0x01 Gi0/1 500 5 Flow Keys in Upper Case Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 Flexible NetFlow Activation per Interface Send the “Sampler-Table” Option
Router(config-if)# ip flow monitor
For the Input or Output Traffic. Does not Determine the Flow Key
. Deterministic or random sampling
Router(config)# sampler
Need the “Match Flow Sampler” in the Record for Accuracy Determination Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 41 Useful Show Commands
. List of all possible information elements show flow exporter export-ids netflow-v9 . Template assignment show flow exporter template . High watermark in the cache show flow monitor
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 42 NetFlow Deployment Scenarios
NetFlow for Core Server Flow Monitor Traffic Matrix . Standard seven keys . Source/destination AS . IP addresses (src/dest) . BGP next hop Security Flow Monitor . Protocols . Protocol . DSCP . Ports . IP addresses . TCP flags
Managed Service Application ISP Visibility Peering Flow Monitor . IP addresses . Destination AS . Application . Source traffic index . DSCP . BGP next hop . DSCP
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 43 Flexible NetFlow Platforms, Features
. Platforms: 800, 1700, ISR (1800, 2800, 3800), ISR-G2 (1900, 2900, 3900), 2600, 2800, 3700, 3800, 7200, 7301, 12000, Nexus 7000, Nexus 1000V, CRS-1 . Beginning with Cisco IOS Release 12.4(20)T, traditional NetFlow for IPv6 is being replaced by Flexible NetFlow for IPv6 . Some traditional NetFlow features are not yet supported with flexible NetFlow: Input filters (integration with modular QoS CLI) SCTP MIB
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 44 Agenda
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 What Does a DoS Attack Look Like?
Router# show ip cache flow … SrcIf SrcIPaddress SrcP SrcAS DstIf DstIPaddress DstP DstAS Pr Pkts B/Pk 29 192.1.6.69 77 aaa 49 194.20.2.2 1308 bbb 6 1 40 29 192.1.6.222 1243 aaa 49 194.20.2.2 1774 bbb 6 1 40 29 192.1.6.108 1076 aaa 49 194.20.2.2 1869 bbb 6 1 40 29 192.1.6.159 903 aaa 49 194.20.2.2 1050 bbb 6 1 40 29 192.1.6.54 730 aaa 49 194.20.2.2 2018 bbb 6 1 40 29 192.1.6.136 559 aaa 49 194.20.2.2 1821 bbb 6 1 40 29 192.1.6.216 383 aaa 49 194.20.2.2 1516 bbb 6 1 40 29 192.1.6.111 45 aaa 49 194.20.2.2 1894 bbb 6 1 40 29 192.1.6.29 1209 aaa 49 194.20.2.2 1600 bbb 6 1 40
. Typical DoS attacks have the same (or similar) entries: Input interface, destination IP, one packet per flow, constant bytes per packet (B/Pk) . Don’t forget show ip cache verbose flow | include … . Export to a security-oriented collector: CS-MARS, Lancope, Arbor
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 46 Flexible Flow Record: Key Fields
Flow IPv4 IPv6 Sampler ID IP (Source or Payload Size IP (Source or Destination) Payload Size Direction Destination) Prefix (Source or Packet Section Prefix (Source or Packet Section Interface Destination) (Header) Destination) (Header) Input Mask (Source or Packet Section Mask (Source or Packet Section Destination) (Payload) Output Destination) (Payload) Minimum-Mask Minimum-Mask Layer 2 (Source or TTL (Source or DSCP Destination) Source Destination) Options VLAN Protocol bitmap Protocol Extension Headers Destination Fragmentation VLAN Version Traffic Class Hop-Limit Flags Fragmentation Flow Label Length Source MAC Precedence Offset address Option Header Next-header Identification DSCP Header Length Version Destination Header Length TOS MAC address Total Length Payload Length
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 Flexible Flow Record: Key Fields
Routing Transport Application src or dest AS Destination Port TCP Flag: ACK Application ID* Peer AS Source Port TCP Flag: CWR Traffic Index ICMP Code TCP Flag: ECE Multicast Forwarding ICMP Type TCP Flag: FIN Status IGMP Type* TCP Flag: PSH Replication IGP Next Hop Factor* TCP ACK Number TCP Flag: RST BGP Next Hop TCP Header Length TCP Flag: SYN RPF Check Drop* Input VRF TCP Sequence Number TCP Flag: URG Name Is-Multicast TCP Window-Size UDP Message Length
TCP Source Port UDP Source Port TCP Destination Port UDP Destination Port
TCP Urgent Pointer *: IPv4 Flow only
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 48 Flexible Flow Record: Non-Key Fields
Counters Timestamp IPv4 IPv4 and IPv6
sysUpTime First Total Length Total Length Bytes Packet Minimum (*) Minimum (**) Total Length Total Length Bytes Long sysUpTime First Packet Maximum (*) Maximum (**) Bytes Square Sum TTL Minimum
Bytes Square Sum Long TTL Maximum
Packets (*) IPV4_TOTAL_LEN_MIN, IPV4_TOTAL_LEN_MAX (**)IP_LENGTH_TOTAL_MIN, IP_LENGTH_TOTAL_MAX Packets Long
. Plus any of the potential “key” fields: will be the value from the first packet in the flow
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 49 Useful Fields for Security Monitoring
Attacks that Use Consistent Packet Size or Worms that Use Consistent Packet Size 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of Service| Total Length | Several Flows with +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ the Same Fragment | Identification |Flags| Fragment Offset | Offset: Same Packet +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Sent Over and Over | Time to Live | Protocol | Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Options | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Flow Issued From Very Large Packets or Attacks that Might Always the Same Origin Have The Same Generated Identification
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 50 Source MAC Address Example
Email Server DoS Attack Arriving from the Internet
Host A Router A
Router B NetFlow Host B Internet
Host C Router C
Router D
Report the MAC Address for Ethernet, FastEthernet, and GigabitEthernet
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 51 The Forwarding Status
Router(config)# flow record forwarding-status Router(config-flow-record)# … Router(config-flow-record)# match routing forwarding status
. Unknown (00b) . Forwarded (01b) . Dropped (10b) ACL, QoS . Consumed (11b) Destined to the router (ex: management traffic)
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 52 Packet Section Fields
. Contiguous chunk of a packet of a user configurable size, used as a key or a non-key field . Sections used for detailed traffic monitoring, DDoS attack investigation, worm detection, other security applications . Chunk defined as flow key, should be used in sampled mode with immediate aging cache . Starts at the beginning of the IPv4 header collect or match ipv4 header
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 53 Flexible NetFlow TopTalkers
. Advanced search capabilities Flow filtering: enables users to select flows based on specific values for any fields that are defined for that cache, or a predefined flow record Flow aggregation: enables users to aggregate on a subset of the key and non-key fields present in the Flows of an FNF cache Flow sorting: enables users to control how the displayed cache entries are sorted on any field present in the flows of an FNF cache and show in order or reverse order . Works with any type of flows/fields: IPv4, IPv6, L2, …
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 55 Flexible NetFlow Top Talkers Examples
. Top ten IP addresses that are sending the most packets Router# show flow monitor
Router# show flow monitor
Router# show flow monitor
TCP My Servers SYN Network Attacks 10.10.10.0/24
Router# show flow monitor
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 57 Embedded Event Manager 3.0 Flexible NetFlow Event Detector
flow record
flow exporter
flow monitor
event manager applet security-applet event nf monitor-name "
. If a flow record with TTL < 5, send a syslog message
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 58 Embedded Event Manager 3.0 Flexible NetFlow Event Detector router# show flow monitor
Flows added: 32 Flows aged: 27 - Active timeout ( 1800 secs) 0 - Inactive timeout ( 15 secs) 27 - Event aged 0 - Watermark aged 0 - Emergency aged 0
IPV4 SRC ADDR IPV4 DST ADDR IP TTL ======168.192.1.1 10.48.72.79 3
*Dec 18 2008 11:45:04.904 UTC: %HA_EM-6-LOG: security- applet: flow record with low TTL . Detect that some packets target the CPU…
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 59 NetFlow L2 and Security Monitoring (for Traditional NetFlow)
. Layer 2 IP header fields Source MAC address field from frames that are received by the NetFlow router Destination MAC address field from frames that are transmitted by the NetFlow router Received VLAN ID field (802.1q and Cisco’s ISL) Transmitted VLAN ID field (802.1q and Cisco’s ISL) . Extra Layer 3 IP header fields Time-to-live field Identification field Packet length field ICMP type and code Fragment offset . For IPv4 and IPv6
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 61 NetFlow Top Talkers (for Traditional NetFlow)
. The flows that are generating the heaviest traffic in the cache are known as the top talkers; prefer top flows . Allows flows to be sorted by either of the following criteria: By the total number of packets in each top talker By the total number of bytes in each top talker . Match criteria for the top talkers, work like a filter . The top talkers can be retrieved via the CISCO-NETFLOW- MIB (cnfTopFlowsTable) . A new separate cache Similar output of the show ip cache flow or show ip cache verbose flow command Generated on the fly Frozen for the cache-timeout value
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 62 NetFlow Top Talkers (for Traditional NetFlow)
Router(config)# ip flow-top-talkers Router(config-flow-top-talkers)# top 50 Router(config-flow-top-talkers)# sort-by
Router# show ip flow top-talkers verbose
SrcIf SrcIPaddress DstIf DstIPaddress Pr TOS Flgs Pkts Port Msk AS Port Msk AS NextHop B/Pk Active IPM: OPkts OBytes Fa1/0 10.48.71.9 Local 10.48.71.9 01 C0 10 56 0000 /24 0 0303 /24 0 0.0.0.0 56 171.0 ICMP type: 3 ICMP code: 3
Se0/0 192.1.1.97 Se0/3 192.1.1.110 01 00 00 12 0000 /30 0 0000 /30 0 192.1.1.108 1436 2.8 ICMP type: 0 ICMP code: 0
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 63 NetFlow Top Talkers (for Traditional NetFlow)
Router(config)# ip flow-top-talkers Router(config-flow-top-talkers)# top 50 Router(config-flow-top-talkers)# sort-by packets Router(config-flow-top-talkers)# cache-timeout 2000 Router(config-flow-top-talkers)# match source address 192.1.1.97/32 Router(config-flow-top-talkers)# match destination address 192.1.1.110/32
Router# show ip flow top-talkers verbose
. Available via a new separate cache or via SNMP cnfTopFlowsTable in CISCO-NETFLOW-MIB . Match criteria for the top talkers, works like a filter . Must know what we are looking for
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 64 NetFlow Top Talkers (for Traditional NetFlow)
Router(config)# ip flow-top-talkers Router(config-flow-top-talkers)# top 50 Router(config-flow-top-talkers)# sort-by packets Router(config-flow-top-talkers)# cache-timeout 2000 Router(config-flow-top-talkers)# match source address 192.1.1.97/32 Router(config-flow-top-talkers)# match destination address 192.1.1.110/32
match [[source address | destination address | nexthop address] [ip-address] [mask | /nn]] [[source port | destination port] [port-number | min port | max port | min port max port]] [[source as | destination as] as-number] [[input-interface | output-interface] interface] [tos [tos-value | dscp dscp-value | precedence precedence-value]] [protocol [protocol-number | tcp | udp]] [flow-sampler flow-sampler-name] [class-map class] [packet-range | byte-range [[min-range-number max-range-number] [min minimum-range | max maximum-range | min minimum-range max maximum-range]]] [direction]
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 65 NetFlow Dynamic Top Talkers (for Traditional NetFlow)
. Somehow similar to the top talkers But dynamic, done on the fly with show commands But does not require modifications to the router config But does not create a new cache But not available with the MIB—obviously . Even more useful than top talkers for security . show ip flow top command: show ip flow top
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 66 NetFlow Dynamic Top Talkers (for Traditional NetFlow)
. Top ten protocols currently flowing through the router:
Router# show ip flow top 10 aggregate protocol . Top ten IP addresses sending the most packets
Router# show ip flow top 10 aggregate source-address sorted-by packets . Top five destination addresses to which we’re routing most traffic from the 10.10.10.0/24 prefix Router# show ip flow top 5 aggregate destination-address match source-prefix 10.10.10.0/24
. Five VLANs that we’re sending the least bytes to: Router# show ip flow top 5 aggregate destination-vlan sorted-by bytes ascending
. Top 20 sources of one-packet flows: Router# show ip flow top 50 aggregate source-address match packets 1
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 67 Cisco Adaptive Security Appliances (ASA) 5580—NetFlow Export Version 9
. Logging in high-performance environments is nontrivial, NetFlow is replacing syslog . Flow event information can now be exported through NetFlow v9 Information about NAT modifications to the traffic Information about Flows denied by security policy Information about AAA/usernames associated with flows bidirectional flows . Provides scalable logging 10-Gbps flows, 100-k connections per second = lots of logs . Adds new NetFlow fields to represent security related parameters . NetFlow export is the logical evolution in logging technology
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 68 Agenda
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 69 Network Based Application Recognition NetFlow and NBAR Differentiation
Link Layer Header Interface NetFlow Monitors data in Layers 2 thru 4 ToS NetFlow Determines applications by port Protocol Utilizes a seven-tuple for flow IP Header Source Flow information who, what, when, IP Address where Destination NBAR IP Address Examines data from Source Layers 3 thru 7 Port Utilizes Layers 3 and 4 TCP/UDP plus packet inspection Header Destination for classification Port Stateful inspection of dynamic-port traffic Packet and byte counts
Data Deep Packet Packet (Payload) Inspection NBAR
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 70 NetFlow and NBAR Integration
. NetFlow is the de-facto mechanism to provide visibility on network utilization— who/what/where/when . Applications can no longer be identify by just L3/L4 information Application visibility is a must Example: port 80 is overloaded . NBAR (Network Based Application Recognition) Offers a Deep Packet Inspection (DPI) mechanism . NBAR is integrated in flexible NetFlow on all software based platforms 800–3800/7200/7300, ASR1000, PISA
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 71 For Your NBAR—Supported Protocols Reference
Enterprise Applications Security and Tunneling Network Mail Services Internet Citrix ICA GRE IMAP FTP PCAnywhere IPINIP POP3 Gopher Novadigm IPsec Exchange HTTP SAP L2TP Notes IRC Routing Protocols MS-PPTP SMTP Telnet BGP SFTP Directory TFTP EGP SHTTP DHCP/BOOTP NNTP EIGRP SIMAP Finger NetBIOS OSPF SIRC DNS NTP RIP SLDAP Kerberos Print Network Management SNNTP LDAP X-Windows ICMP SPOP3 Streaming Media Peer-to-Peer SNMP STELNET CU-SeeMe BitTorrent Syslog SOCKS Netshow Direct Connect RPC SSH Real Audio eDonkey/eMule NFS Voice StreamWorks FastTrack SUN-RPC H.323 VDOLive Gnutella Database RTCP RTSP KaZaA SQL*NET RTP MGCP WinMX 2.0 MS SQL Server SIP Signaling SCCP/Skinny RSVP Skype
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 72 NBAR Custom Application Examples
ip nbar custom lunar_light 8 ascii Moonbeam tcp range 2000 2999 class-map solar_system match protocol lunar_light policy-map astronomy class solar_system set ip dscp AF21 interface Serial1 service-policy output astronomy
ip nbar custom virus_home 20 hex variable scid 1 dest udp 5001 5005 class-map active-craft match protocol virus_home scid 0x15 match protocol virus_home scid 0x21 class-map passive-craft match protocol virus_home scid 0x11 match protocol virus_home scid 0x22
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 73 Flexible NetFlow—NBAR Integration Configuration Example router(config)# flow record app_record router(config-flow-record)# match ipv4 source address router(config-flow-record)# match ipv4 destination address router(config-flow-record)# match application name router(config-flow-record)# collect counter packets router(config-flow-record)# collect counter bytes router(config)# flow exporter app_collector router(config-flow-monitor)# destination
nbar = Static Applications
show flow mon
IPV4 SRC ADDR IPV4 DST ADDR APP NAME … ======10.0.1.1 10.0.1.2 nbar rtcp 10.0.1.1 10.0.1.2 nbar ssh 10.0.1.1 10.0.1.2 nbar telnet 10.0.1.1 10.0.1.2 NBAR lunar_light
NBAR = Custom Applications
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 75 Flexible NetFlow—NBAR Integration Configuration Example Classification Engine ID = 5, Selector = 48 show flow mon
IPV4 SRC ADDR IPV4 DST ADDR APP NAME … ======10.0.1.1 10.0.1.2 nbar rtcp 10.0.1.1 10.0.1.2 nbar ssh 10.0.1.1 10.0.1.2 nbar telnet 10.0.1.1 10.0.1.2 NBAR lunar_light
Classification Engine ID = 6, Selector = 125 Options Record: 5:48 rtcp Real-Time Control Protocol 6:125 lunar_light User-Defined Protocol lunar_light
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 76 Reporting Example (Plixer)
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 77 Agenda
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 78 Performance Challenge Moving Bottleneck
. Consumes a lot of CPU - Packet sampling - Metering process in hardware . Collisions in the cache - Improved the hash function - Increased the cache size . Consumes much bandwidth - Flexible flow record per … interface, per direction - Export cache type per collector - Flow sampling . CPU impact, bandwidth impact, and accuracy impact
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 79 CPU Impact NetFlow Performance Paper Tests
. Paper at www.cisco.com/go/netflow under Technical Documents . 0, 1, and 2 NetFlow data export destinations . Initial performance after enabling . v8 aggregation vs. v5, v9 performance . Full NetFlow vs. 1:100 sampled NetFlow . Tested hardware: Cisco 1841, 2600, 2811, 2851, 3640, 3745, 3845, 7200, 7301, 7500, 12000 -> will be updated soon with newest platform
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 80 CPU Impact Finding Summary
. Larger number of cache entries will have an increasing level of impact to CPU This is much more visible on the low end systems (LES) . Having multiple exporters does not add significant CPU impact . NetFlow v9 and NetFlow v5 export have similar CPU impact . Flexible NetFlow does add a slight CPU load More visible on lower end platforms However this difference is seen at large flow counts that are not expected to be seen on LES
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 81 CPU Impact NetFlow Sampling
1 K 1 K 1 K Packet Sampling (Random or Systematic)
Cache Flow Sampling … in the NetFlow Cache
Export
. Flow sampling only available on the Catalyst 6500 and 7600 . With random packet based sampling, we can evaluate the accuracy of the observed flow records
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 82 Configuring Packet Based Sampling
flow record
flow exporter
flow monitor
flow sampler
interface pos3/0 ip flow monitor
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 83 Accuracy Impact Random Packet NetFlow Sampling
. Packet Sampling for Flow Estimation Accuracy Accounting: Challenges (PLT_NZIX1, S24D00, Cisco, f=5% and Limitations, Tanja Zseby, Thomas Hirsch, Benoit Claise, PAM 2008 . Square sum of bytes available in f flexible NetFlow #Packets N #Packets
Mean Packet Size µf
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 84 Accuracy Impact NetFlow Metering in Hardware . Cisco Catalyst 6500 and Cisco 7600 supervisors Table Size Hash Efficiency Effective Size Hash Key Size
Sup2 128K 25% 32K 17 Bits
Sup720 128K 50% 64K 36 Bits
Sup720-3B 128K 90% 115K 36 Bits
Sup720-3BXL 256K 90% 230K 36 Bits
Sup32-8GE 128K 90% 115K 36 Bits
Sup32-10GE 128K 90% 115K 36 Bits
Sup720-10GE-3C 128K 90% 115K 36 Bits
Sup720-10GE-3CXL 256K 90% 230K 36 Bits
. Other metering process in hardware: . NEXUS 7000: Up to ~500K (with 95% utilization efficiency) . ASR1000: Up to ~1M cached flows . Etc. Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 85 Reducing Performance Impact CPU and Memory Impact on the Network Element, Collector, and Network . Flexible NetFlow (collect only what is really required) . Aging timers . Sampled NetFlow . Leverage distributed architectures (VIP, linecards) . Flow masks (only Cisco Catalyst 6500/7600) . Aggregation schemes (v8 on router or on collector) . Filters (router or collector) . Data compression (collector) . Increase collection bucket sizes (collector) . Place collector and router on the same LAN segment/ dedicated interface
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 86 Bandwidth Impact Case 1: Traditional NetFlow
NETFLOW-MiB: Router# show ip flow export cnfESRecordsExported Flow export is enabled Exporting flows to 1.1.1.1 (9999) Exporting using source IP address 198.198.198.11 Version 5 flow records cnfESPktsExported 29 flows exported in 4 udp datagrams 0 flows failed due to lack of export packet 0 export packets were sent up to process level 0 export packets were dropped due to no fib 0 export packets were dropped due to adjacency issues 0 export packets were dropped due to fragmentation failures 0 export packets were dropped due to encapsulation fixup failures Router# clear ip flow stats
cnfESExportRate (NETFLOW-MIB), “number of bytes exported per second” The counter for this MIB object only contain L3 bytes: Layer 2 encapsulation needs to be added
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 87 Bandwidth Impact Case 2: Flexible NetFlow
Router(config)# flow exporter
Router(config)# ip cef accounting per-prefix
CEF-MIB: cefPrefixPkts CEF-MIB: cefPrefixBytes
Router# show ip cef 1.1.1.1 1.1.1.1/32, version 9, epoch 0, attached 100 packets, 11052 bytes via Null0, 0 dependencies valid null (drop) adjacency
. The counter for this MIB object only contain L3 bytes, Layer 2 encapsulation needs to be added
. This method is also valid for traditional NetFlow
. CEF-MIB available in 12.4(20)T
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 88 Agenda
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 89 IETF: IP Flow Information Export WG (IPFIX)
. RFC3954 Cisco Systems NetFlow Services Export Version 9 . RFC3917 Requirements for IP Flow Information Export Gathers all IPFIX requirements for the IPFIX evaluation process . RFC3955 Evaluation of Candidate Protocols for IPFIX . RFC5101 Specification of the IPFIX Protocol for the Exchange of IP Traffic Flow Information . RFC5102 Information Model for IP Flow Information Export . RFC5103 “Bidirectional Flow Export using IP Flow Information Export (IPFIX)”
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 90 IETF: IP Flow Information Export WG (IPFIX)
. IPFIX protocol specifications Changes in terminology but same NetFlow Version 9 principles Improvements vs. NetFlow Version 9: SCTP-PR, security, variable length information element, IANA registration, etc. Generic streaming protocol, not flow-centric anymore Security: Threat: confidentiality, integrity, authorization Solution: DTLS on SCTP-PR . IPFIX information model Most NetFlow Version 9 information elements ID are kept Proprietary information element specification . Is IPFIX important to you?
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 91 IETF: Packet Sampling WG (PSAMP)
. PSAMP is an effort to: Specify a set of selection operations by which packets are sampled, and describe protocols by which information on sampled packets is reported to applications . Sampling and filtering techniques for IP packet selection To be compliant with PSAMP, we must implement at least one of the mechanisms: sampled NetFlow, NetFlow input filters are already implemented . PSAMP protocol specifications Agreed to use IPFIX for export protocol . Information model for packet sampling export Extension of the IPFIX information model
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 92 Agenda
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 93 Traditional NetFlow For Your Exporting Process Reference
Feature Software C6500 C7600 C12000 C10000 C4500
Version 5 12.0(1) 12.1(2)E 12.1(2)E 12.0(14)S 12.0(19)SL 12.1(13)EW
Version 8 12.0(3)T 12.2(14)SX 12.2(14)SX 12.0(6)S 12.0(19)SL 12.1(19)EW
Version 9 12.3 12.2(18)SXF 12.2(18)SXF 12.0(24)S 12.2(31)SB
Dual Export 12.2(2)T 12.2(17d)SXB12.2(17d)SXB 12.2(15)BX 12.1(19)EW
VRF Destination 12.4(4)T 12.2(33)SRA 12.0(32)S
Reliable Export 12.3(4)T
Product Manager Contact: [email protected]
Available Now Not Available Roadmap
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 94 Traditional NetFlow For Your Exporting Process Reference
Feature CRS-1 XR12000 ASR1000
Version 5 2.1
Version 8 2.1
Version 9 3.2 3.3.0 2.1
Dual Export 3.4.0 3.4.0 2.1
VRF Destination 3.2 3.3.0
Reliable Export
Available Now Not Available Roadmap
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 95 Traditional NetFlow For Your Metering Process Reference
Feature Software C6500 C7600 C12000 C10000 C4500 IPv4 12.0(1) 12.1(27b)E1 12.2(18)SXF 12.0(22)S 12.2(15)BX 12.1(13)EW IPv6 12.3(7)T 12.2(33)SXH 12.2(33)SRB Multicast 12.3 12.2(18)SXF 12.2(18)SXF BGP Next Hop 12.3 12.2(18)SXF 12.2(33)SRA 12.0(26)S 12.2(31)SB No Per Interface Yes 12.2(33)SXH 12.2(33)SRB No Sub 12.2(15)BX SUPVI Per VRF Interface Yes 12.2(33)SXH 12.2(33)SRB 12.2(17b)SX TOS Support Yes 12.2(17b)SXA Yes Yes A Min Prefix Aggr. 12.1(2)T Yes Yes MPLS Egress 12.2(28)SB with EXP MPLS Egress 12.2(2)T 12.2(33)SXJ MPLS Aware 12.3(8)T 12.2(33)SRA 12.0(24)S MPLS Label Exp. 12.2SB 12.2(33)SRB MPLS Aggregate. 12.2(31)SB
Available Now Not Available Roadmap
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 96 Traditional NetFlow For Your Metering Process Reference
Feature CRS-1 XR12000 ASR1000 IPv4 3.2.0 3.3.0 2.1 IPv6 3.5.0 3.6.0 Multicast 3.2 3.3 BGP Next Hop 3.3 3.3 2.1 Per Interface 3.3.0 3.3.0 2.1 TOS Support 3.2 3.3 2.1 Packet Sampling 3.2 3.3 2.1 Min Prefix Aggr. MPLS Egress with EXP MPLS Egress 3.2 3.5.0 MPLS Aware 3.3.1 3.5.0 MPLS Label Expo MPLS Aggregate.
Available Now Not Available Roadmap
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 97 Traditional NetFlow For Your Metering Process Reference
Feature Software C6500 C7600 C12000 C10000 C4500
Egress/Output 12.3(11)T 12.0(10)ST 12.2(31)SB NetFlow
Bridged NF 12.2(18)SXE1 12.2(18)SXE1 12.2(25)EW
Input Filters 12.3(4)T
TCP Flags 12.1(2)T 12.0(10)ST 12.2(28)SB
Mac Address 12.3(14)T
Security Exports 12.3(14)T 12.2(33)SRA
Vlan Export 12.4(4)T
Available Now Not Available Roadmap
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 98 Traditional NetFlow For Your Metering Process Reference
Feature CRS-1 XR12000 ASR1000
Egress/Output 3.2 3.3 2.1 NetFlow
Bridged NF
Input Filters
TCP Flags 3.2 3.3 2.1
Mac Address
Security Exports
Vlan Export
Available Now Not Available Roadmap
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 99 For Your Miscellaneous Features Reference
Feature Software C6500 C7600 C12000 C10000 ASR1000
NetFlow MIB 12.3(11)T 12.2(33)SXH with Top Talker
Dynamic Top 12.4(4)T Talker CLI
ISSU NetFlow 12.2(33)SRB ifIndex to 12.4(4)T Name Map
Available Now Not Available Roadmap
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 100 Traditional NetFlow For Your Metering Process, Sampled NetFlow Reference
Feature Software C6500 C7600 C12000 C10000 C4500
Systematic 12.3(11)T 12.0(11)S Sampling
Random 12.4(9)T 12.0(33)S 12.2(31)SB Sampling
Output Sampled 12.0(24)S NetFlow
Flow Sampling 12.1(13)E 12.1(13)E
Available Now Not Available Roadmap
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 101 Traditional NetFlow For Your Metering Process, Sampled NetFlow Reference
Feature CRS-1 XR12000 ASR1000
Systematic Sampling
Random 3.2 3.3 2.1 Sampling
Output Sampled 2.1 NetFlow
Flow Sampling
Available Now Not Available Roadmap
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 102 Flexible NetFlow Metering Process
Feature Cisco ISR Cisco ISR-G2 Cisco 72/73xx ASR1000
New Flexible NetFlow CLI 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Multiple User Defined Caches 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Normal Cache 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Immediate Cache 12.4(9)T 15.0(1)M 12.4(9)T Radar Permanent Cache 12.4(9)T 15.0(1)M 12.4(9)T Radar Dynamic TopNTalkers 12.4(22)T 15.0(1)M 12.4(22)T IOS XE 3.2.0S*
FNF EEM Monitor 12.4(22)T 15.0(1)M 12.4(22)T Radar
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 103 Flexible NetFlow Metering Process
Feature C6500 (SUP2T) C4500 (SUP7) Nexus 7000 Nexus1000V
New Flexible NetFlow CLI 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
Multiple User Defined Caches 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
Normal Cache 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
Immediate Cache 12.2(50)SYA IOS XE 3.1.0SG Permanent Cache 12.2(50)SYA IOS XE 3.1.0SG Dynamic TopNTalkers 12.2(50)SYA IOS XE 3.1.0SG FNF EEM Monitor Radar IOS XE 3.1.0SG
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 104 Flexible NetFlow Metering Process
Feature CRS-1 XR12000 ASR9000 C12000
New Flexible NetFlow CLI 3.2 3.3.0 3.9(1) 12.0(33)S
Multiple User Defined Caches Radar Radar 3.9(1) 12.0(33)S
Normal Cache 3.2 3.3.0 3.9(1) 12.0(33)S
Immediate Cache 3.2 3.3.0 3.9(1) 12.0(33)S Permanent Cache 3.2 3.3.0 3.9(1) 12.0(33)S Dynamic TopNTalkers Radar Radar Radar
FNF EEM Monitor
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 105 Feature Cisco ISR Cisco ISR-G2 Cisco 72/73xx ASR1000 Sampling Full Flow support 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Random Sampling 1:M 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Random Sampling N:M Activation Ingress support 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Egress support 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Per Interface 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Per Sub-Interface 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
On VRF Interface 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Per Vlan
Per Class-map 15.1(4)T 15.1(4)T 15.1(4)T IOS XE 3.3.0S*
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 106 Flexible NetFlow Sampling and Activation
Feature C6500 (SUP2T) C4500 (SUP7) Nexus 7000 Nexus1000V Sampling Full Flow support 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
Random Sampling 1:M 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
Random Sampling N:M 4.0 4.0(4)SV1 Activation Ingress support 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
Egress support 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
Per Interface 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
Per Sub-Interface 12.2(50)SYA IOS XE 3.1.0SG 4.0 N/A
On VRF Interface 12.2(50)SYA IOS XE 3.1.0SG N/A N/A
Per Vlan IOS XE 3.1.0SG
Per Class-map Radar*
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 107 Flexible NetFlow Sampling and Activation
Feature CRS-1 XR12000 ASR9000 C12000 Sampling Full Flow support 3.2 3.3.0 3.9(1) 12.0(33)S
Random Sampling 1:M 3.2 3.3.0 3.9(1) 12.0(33)S
Random Sampling N:M Activation Ingress support 3.2 3.3.0 3.9(1) 12.0(33)S
Egress support 3.2 3.3.0 3.9(1) 12.0(33)S
Per Interface 3.2 3.3.0 3.9(1) 12.0(33)S
Per Sub-Interface 3.2 3.3.0 3.9(1) 12.0(33)S
On VRF Interface 3.2 3.3.0 3.9(1) 12.0(33)S
Per Vlan
Per Class-map
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 108 Flexible NetFlow Exporter
Feature Cisco ISR Cisco ISR-G2 Cisco 72/73xx ASR1000 Exporter NetFlow v5 Export Format 12.4(22)T 15.0(1)M 12.4(22)T IOS XE 3.1.1S
NetFlow v9 Export Format 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
IPFix Export Format Radar* Radar* Radar* Radar*
IPFix Structured Data Radar* Radar* Radar* Radar* Export over UDP 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S Export over SCTP (Reliable) Radar* Radar* Radar* Radar* Export over IPv4 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
Export over IPv6 Radar* Radar* Radar* Radar*
Exporter MTU Configuration Radar* Radar* Radar* Radar*
Export in a VRF 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.2.0S
FNF QOS output features 12.4(20)T 15.0(1)M 12.4(20)T IOS XE 3.1.1S
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 109 Flexible NetFlow Exporter
Feature C6500 (SUP2T) C4500 (SUP7) Nexus 7000 Nexus1000V Exporter NetFlow v5 Export Format 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
NetFlow v9 Export Format 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
IPFix Export Format Radar* Radar* Radar* Radar*
IPFix Structured Data Radar* Radar* Radar* Radar* Export over UDP 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1 Export over SCTP (Reliable) Radar* Radar* Radar Radar Export over IPv4 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
Export over IPv6 Radar* Radar* 4.2(1)
Exporter MTU Configuration Radar* Radar* Radar* Radar*
Export in VRF 12.2(50)SYA IOS XE 3.1.0SG
FNF QOS output features 12.2(50)SYA IOS XE 3.1.0SG 4.0
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 110 Flexible NetFlow Exporter
Feature CRS-1 XR12000 ASR9000 C12000
Exporter
NetFlow v5 Export Format
NetFlow v9 Export Format 3.2 3.3.0 3.9(1) 12.0(33)S
IPFix Export Format Radar* Radar* Radar*
IPFix Structured Data Export over UDP 3.2 3.3.0 3.9(1) 12.0(33)S Export over SCTP (Reliable) Radar Radar Radar Export over IPv4 3.2 3.3.0 3.9(1) 12.0(33)S
Export over IPv6 Radar Radar Radar
Exporter MTU Configuration Radar Radar Radar
Export in a VRF 3.2 3.3.0 3.9(1) 12.0(33)S
FNF QOS output features 3.2 3.3.0 3.9(1)
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 111 Flexible NetFlow IPv4 Flows
Feature Cisco ISR Cisco ISR-G2 Cisco 72/73xx ASR1000
IPv4 Flows
IPv4 Unicast Flows 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
IPv4 Predefined Aggregations 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
IPv4 Multicast Flows 12.4(22)T 15.0(1)M 12.4(22)T Radar
IPv4 Multicast Replication Factor 12.4(22)T 15.0(1)M 12.4(22)T Radar
IPv4 Header Section Field 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
IPv4 Payload Section Field 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
UDP Fields 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
TCP Fields 12.4(9)T 15.0(1)M 12.4(9)T IOS XE 3.1.1S
SCTP Fields Radar Radar Radar Radar
Application Name (NBAR) Field 15.0(1)M 15.0(1)M 15.0(1)M IOS XE 3.1.1S
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 112 Flexible NetFlow IPv4 Flows
Feature C6500 (SUP2T) C4500 (SUP7) Nexus 7000 Nexus 1000V
IPv4 Flows
IPv4 Unicast Flows 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
IPv4 Predefined Aggregations 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
IPv4 Multicast Flows 12.2(50)SYA IOS XE 3.1.0SG Radar*
IPv4 Multicast Replication Factor 12.2(50)SYA IOS XE 3.1.0SG Radar*
IPv4 Header Section Field IOS XE 3.1.0SG
IPv4 Payload Section Field IOS XE 3.1.0SG
UDP Fields 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
TCP Fields 12.2(50)SYA IOS XE 3.1.0SG 4.0 4.0(4)SV1
SCTP Fields Radar*
Application Name (NBAR) Field
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 113 Flexible NetFlow IPv4 Flows
Feature CRS-1 XR12000 ASR9000 C12000
IPv4 Flows
IPv4 Unicast Flows 3.2 3.3.0 3.9(1) 12.0(33)S
IPv4 Predefined Aggregations 3.2 3.3.0 3.9(1) 12.0(33)S
IPv4 Multicast Flows 3.5.0 3.5.0 3.9(1) 12.0(33)S
IPv4 Multicast Flows 3.5.0 3.5.0 3.9(1) 12.0(33)S
IPv4 Header Section Field 12.0(33)S
IPv4 Payload Section Field 12.0(33)S
UDP Fields 3.2 3.3.0 3.9(1) 12.0(33)S
TCP Fields 3.2 3.3.0 3.9(1) 12.0(33)S
SCTP Fields Radar Radar Radar
Application Name (NBAR) Field
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 114 Flexible NetFlow IPv6 Flows
Feature Cisco ISR Cisco ISR-G2 Cisco 72/73xx ASR1000
IPv6 Flows
IPv6 Unicast Flows 12.4(20)T 15.0(1)M 12.4(20)T IOS XE 3.3.0S*
IPv6 Predefined Aggregations 12.4(20)T 15.0(1)M 12.4(20)T IOS XE 3.3.0S*
IPv6 Multicast Flows Radar Radar Radar Radar
IPv6 Multicast Replication Factor Radar Radar Radar Radar
IPv6 Header Section Field 12.4(20)T 15.0(1)M 12.4(20)T IOS XE 3.3.0S*
IPv6 Payload Section Field 12.4(20)T 15.0(1)M 12.4(20)T IOS XE 3.3.0S*
UDP Fields 12.4(20)T 15.0(1)M 12.4(20)T IOS XE 3.3.0S*
TCP Fields 12.4(20)T 15.0(1)M 12.4(20)T IOS XE 3.3.0S*
SCTP Fields Radar Radar Radar Radar
Application Name (NBAR) Field Radar Radar Radar IOS XE 3.3.0S*
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 115 Flexible NetFlow IPv6 Flows
Feature C6500 (SUP2T) C4500 (SUP 7) Nexus 7000 Nexus 1000V
IPv6 Flows
IPv6 Unicast Flows 12.2(50)SYA IOS XE 3.1.0SG 4.0 Radar
IPv6 Predefined Aggregations 12.2(50)SYA IOS XE 3.1.0SG 4.0 Radar
IPv6 Multicast Flows Radar Radar Radar Radar
IPv6 Multicast Replication Factor Radar Radar Radar Radar
IPv6 Header Section Field Radar
IPv6 Payload Section Field Radar
UDP Fields 12.2(50)SYA IOS XE 3.1.0SG 4.0 Radar
TCP Fields 12.2(50)SYA IOS XE 3.1.0SG 4.0 Radar
SCTP Fields Radar
Application Name (NBAR) Field
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 116 Flexible NetFlow IPv6 Flows
Feature CRS-1 XR12000 ASR9000 C12000
IPv6 Flows
IPv6 Unicast Flows 3.5.0 3.6.0 3.9(1)
IPv6 Predefined Aggregations 3.5.0 3.6.0 3.9(1)
IPv6 Multicast Flows Radar Radar Radar
IPv6 Multicast Replication Factor Radar Radar Radar
IPv6 Header Section Field Radar Radar Radar
IPv6 Payload Section Field Radar Radar Radar
UDP Fields 3.5.0 3.6.0 3.9(1)
TCP Fields 3.5.0 3.6.0 3.9(1)
SCTP Fields Radar Radar Radar
Application Name (NBAR) Field
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 117 Flexible NetFlow Layer 2, MPLS, Misc.
Feature Cisco ISR Cisco ISR-G2 Cisco 72/73xx ASR1000
Layer 2 Flows 12.4(22)T 15.0(1)M 12.4(22)T Radar
MPLS Flows Radar Radar Radar Radar
MPLS + IPv4 Flows Radar Radar Radar Radar
MPLS + IPv6 Flows Radar Radar Radar Radar
MPLS + IPv6/IPv4 Flows Radar Radar Radar Radar
Ingress VRF name Field 15.0(1)M 15.0(1)M 15.0(1)M IOS XE 3.3.0S*
Class-map Name (C3PL) Field 15.1(4)T 15.1(4)T 15.1(4)T IOS XE 3.3.0S*
4 Bytes AS Field 15.2(1)T 15.2(1)T 15.2(1)T IOS XE 3.2.0S*
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 118 Flexible NetFlow Layer 2, MPLS, Misc.
Feature C6500 (SUP2T) C4500 (SUP7) Nexus 7000 Nexus1000V
Layer 2 Flows 12.2(50)SYA IOS XE 3.1.0SG 4.2(1)
MPLS Flows Radar Radar Radar
MPLS + IPv4 Flows Radar Radar Radar
MPLS + IPv6 Flows Radar Radar Radar
MPLS + IPv6/IPv4 Flows
Ingress VRF name Field 12.2(50)SYA IOS XE 3.1.0SG
Class-map Name (C3PL) Field Radar IOS XE 3.3.0SG*
4 Bytes AS Field 12.2(50)SYA IOS XE 3.1.0SG 4.2(1) 4.2(1)
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 119 Flexible NetFlow Layer 2, MPLS, Misc.
Feature CRS-1 XR12000 ASR9000 C12000
Layer 2 Flows Radar Radar Radar
MPLS Flows 3.3.1 3.5.0 3.9(1)
MPLS + IPv4 Flows 3.3.1 3.5.0 3.9(1)
MPLS + IPv6 Flows 3.5.0 3.6.0 3.9(1)
MPLS + IPv6/IPv4 Flows 3.6.0 3.6.0
Ingress VRF name Field Radar Radar Radar
Class-map Name (C3PL) Field Radar Radar Radar
4 Bytes AS Field 3.4.0 3.4.0 3.9(1)
*: not committed yet
Presentation_IDAvailable© 2010 Now Cisco and/or its affiliates. All rights reserved.Not AvailableCisco Public Roadmap 120 Conclusion—What Did We Cover?
. Introduction . NetFlow Version 9 . Interesting Features on Traditional NetFlow . Flexible NetFlow . NetFlow for Security . NetFlow for Application Visibility . NetFlow Performance . NetFlow Standardization . Support Matrix . Appendix
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 121 NetFlow Summary and Conclusion
. NetFlow is a mature Cisco IOS feature (in Cisco IOS since 1996) . NetFlow provides input for accounting, performance, security, and billing applications . NetFlow has IETF and industry leadership . NetFlow v9 eases the exporting of additional fields . Flexible NetFlow is a major enhancement . A lot of features have been added Stay tuned for more . NetFlow export will become THE push mechanism
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 122 References
. NetFlow http://www.cisco.com/go/netflow . Cisco network accounting services Comparison of Cisco NetFlow versus other available accounting technologies http://www.cisco.com/warp/public/cc/pd/iosw/prodlit/nwact_wp.htm . Cisco IT case study http://business.cisco.com/prod/tree.taf%3Fasset_id=106882&IT=1042 52&public_view=true&kbns=1.html . A complete white paper http://www.cisco.com/univercd/cc/td/doc/cisintwk/intsolns/netflsol/ nfwhite.htm . NetFlow product manager: Jean Charles Griviaud [email protected]
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 123 Meet the Engineer
To make the most of your time at Networkers at Cisco Live 2010, schedule a Face-to-Face Meeting with a top Cisco Engineers.
Designed to provide a "big picture" perspective as well as "in-depth" technology discussions, these face-to-face meetings will provide fascinating dialogue and a wealth of valuable insights and ideas.
Visit the Meeting Centre reception desk located in the Meeting Centre in World of Solutions
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 124 Recommended Reading
. Continue your Cisco Live learning experience with further reading from Cisco Press Network Management: Accounting and Performance Strategies, ISBN 1-58705-198-2 Preview: http://www.informit.com/s tore/product.aspx?isbn= 1587051982
Available Onsite at the Cisco Company Store
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 125 NMS sessions offered (1 of 2) Session Title
BRKNMS-1030 ITIL v3 Foundation and Enhanced Telecom Operations Map (eTOM) frameworks Network Health Framework - A proactive solution for network health BRKNMS-1031 improvement Introduction to Network Performance Measurement with Cisco IOS IP Service BRKNMS-1204 Level Agent
BRKNMS-1532 Introduction to Accounting Principles with NetFlow and NBAR
BRKNMS-1640 Advanced DHCP and DNS Deployments
BRKNMS-1831 Network Performance Management: A proactive End-to-End approach
BRKNMS-1942 Managing Infrastructure as a Service (IaaS) for a Cloud environment UC Network Management: How to Ensure Your UC Services are Operating as BRKNMS-2022 Expected
BRKNMS-2025 Cisco Network Optimization
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 126 NMS sessions offered (2 of 2) Session Title
BRKNMS-2030 Onboard Automation with Cisco IOS Embedded Event Manager
BRKNMS-2031 SYSLOG Design, Methodology and Best Practices
BRKNMS-2032 DHCP and DNS for Large Scale Network Architectures and Cloud Computing Accounting and Performance Management with Network Based Application BRKNMS-2361 Recognition
BRKNMS-2658 Securely Managing Your Networks and SNMPv3
BRKNMS-2784 Automatic Configuration Deployment using CCE
BRKNMS-3021 Advanced Cisco IOS Device Instrumentation Advanced Performance Measurement for Critical IP Traffic with Cisco IOS IP BRKNMS-3043 Service Level Agreements
BRKNMS-3132 Advanced Netflow PNLNMS-3000 Implementing Effective Day-2 Network Operations Support with ITIL
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 127 Complete Your Online Session Evaluation
. Give us your feedback and you could win fabulous prizes. Winners announced daily. . Receive 20 Cisco Preferred Access points for each session evaluation you complete. . Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet Don’t forget to activate your stations throughout the Cisco Live and Networkers Virtual Convention Center. account for access to all session materials, communities, and on-demand and live activities throughout the year. Activate your account at any internet station or visit www.ciscolivevirtual.com.
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 128 Q and A
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 129 Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 130 Appendix: Traditional NetFlow ‘show ip cache flow’ router# show ip cache flow Packet Sizes IP packet size distribution (85435 total packets): 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 .000 .000 .125 .125 .250 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608 .000 .000 .000 .000 .500 .000 .000 .000 .000 .000 .000 # of Active Flows IP Flow Switching Cache, 278544 bytes 2728 active, 1368 inactive, 85310 added 463824 ager polls, 0 flow alloc failures Rates and Duration Active flows timeout in 30 minutes Inactive flows timeout in 15 seconds last clearing of statistics never Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec) ------Flows /Sec /Flow /Pkt /Sec /Flow /Flow TCP-X 2 0.0 1 1440 0.0 0.0 9.5 TCP-other 82580 11.2 1 1440 11.2 0.0 12.0 Total: 82582 11.2 Flow 1 1440Details 11.2 0.0 12.0
SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts Et0/0 132.122.25.60 Se0/0 192.168.1.1 06 9AEE 0007 1 Et0/0 139.57.220.28 Se0/0 192.168.1.1 06 708D 0007 1 Et0/0 165.172.153.65 Se0/0 192.168.1.1 06 CB46 0007 1
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 132 ‘show ip cache verbose flow’ router# show ip cache verbose flow IP packet size distribution (23597 total packets): 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608 Flow Rate .000 .000 .000 .000 1.00 .000 .000 .000 .000 .000 .000 and Duration IP Flow Switching Cache, 278544 bytes 1323 active, 2773 inactive, 23533 added ToS Byte 151644 ager polls, 0 flow alloc failures Active flows timeout in 30 minutesDestination and TCP Inactive flows timeout in 15 secondsInformation Flags last clearing of statistics never Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec) ------Flows /Sec /Flow /Pkt /Sec /Flow /Flow SourceTCP-other Mask and 22210 AS 3.1 1 1440 3.1 0.0 12.9 Total: 22210 3.1 1 1440 3.1 0.0 12.9
SrcIf SrcIPaddress DstIf DstIPaddress Pr TOS Flgs Pkts Port Msk AS Port Msk AS NextHop B/Pk Active Et0/0 216.120.112.114 Se0/0 192.168.1.1 06 00 10 1 5FA7 /0 0 0007 /0 0 0.0.0.0 1440 0.0 Et0/0 175.182.253.65 Se0/0 192.168.1.1 06 00 10 1
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 133 NetFlow Export Version 5 and Main Cache Configuration Example
Router # show ip flow export Flow export v5 is enabled for main cache Exporting flows to 10.48.71.129 (9991) Exporting using source interface Loopback0 Version 5 flow records 1303552 flows exported in 332208 udp datagrams 0 flows failed due to lack of export packet 2 export packets were sent up to process level 0 export packets were dropped due to no fib 0 export packets were dropped due to adjacency issues 0 export packets were dropped due to fragmentation failures 0 export packets were dropped due to encapsulation fixup failures 0 export packets were dropped enqueuing for the RP 0 export packets were dropped due to IPC rate limiting 0 export packets were dropped due to output drops
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 134 NetFlow Export Version 8 and Aggregation Configuration Example
Router(config)# ip flow-aggregation cache
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 135 NetFlow Export Version 8 and Aggregation Configuration Example
Router # show ip flow export … Cache for
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 136 Appendix: NetFlow for Capacity Planning The Core Traffic Matrix Traffic Engineering and Capacity Planning
Paris POP Business Critical Traffic Rome POP ISP-1 SLA Source ISP-2 Destination
Best Effort Traffic
Best Effort
London POP Munich POP
Rome Exit Point Paris Exit Point London Exit Point Munich Exit Point Rome Entry Point NA (*) …Mb/s …Mb/s …Mb/s Paris Entry Point …Mb/s NA (*) …Mb/s …Mb/s London Exit Point …Mb/s …Mb/s NA (*) …Mb/s Munich Exit Point …Mb/s …Mb/s …Mb/s NA (*)
(*) Potentially Local Exchange Traffic Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 138 Core Capacity Planning the Big Picture
1. The ability to offer SLAs is dependent upon ensuring that core network bandwidth is adequately provisioned 2. Adequate provisioning (without gross over provisioning) is dependent upon accurate core capacity planning 3. Accurate core capacity planning is dependent upon understanding the core traffic matrix and flows and mapping these to the underlying topology 4. A tool for what if scenarios
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 139 BGP Next Hop TOS Aggregation Typical Example
AS1 AS2 AS3 AS4 AS5
C PE PE C u MPLS Core u s or s t PE PE t IP Core with BGP Routes Only o o m m e PE PE e r r s PoP PoP s
Server Farm 1 Server Farm 2 Internal Traffic: PE to PE External Traffic Matrix PE to BGP AS
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 140 NetFlow BGP Next Hop TOS Aggregation Flow Keys
Key Fields (Uniquely Identifies the Flow) Additional Export Fields . Origin AS . Flows . Destination AS . Packets . Inbound Interface . Bytes . Output Interface . First SysUptime . ToS/DSCP (*) . Last SysUptime . BGP Next Hop
(*) Before Any Recoloring Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 141 Core Traffic Matrix with Flexible NetFlow
Key Fields (Uniquely Identifies the Flow) Additional Export Fields . Origin AS . Flows . Destination AS . Packets . Inbound Interface . Bytes . Output Interface . First SysUptime . ToS/DSCP (*) . Last SysUptime . BGP Next Hop
. Less flow records, less CPU impact . Potentially choose higher sampling rate for a better accuracy
(*) Before Any Recoloring Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 142 Core Traffic Matrix with Flexible NetFlow Configuration Example
flow record traffic-matrix-record match routing destination as match interface input match ipv4 dscp match routing next-hop address ipv4 bgp collect counter bytes long collect timestamp sys-uptime first collect timestamp sys-uptime last
flow monitor traffic-matrix-monitor record traffic-matrix-record Permanent? cache entries 10000 cache type normal exporter capacity-planning-collector
interface pos3/0 ip flow monitor traffic-matrix-monitor
. Export less flow records with a permanent cache . However, must know the maximum number of entries
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 143 Permanent Flexible NetFlow Configuration Using EEM + Cron + CLI
Minute (0 59) Hour (0 23) Day of the month (1 31) Month of the year (1 12) Day of the week (0 6 with 0=Sunday)
Router(config)# event manager applet periodicexport Router(config-applet)# event timer cron name "everyhour" cron-entry "0 * * * *" Router(config-applet)# action 1.0 cli command "clear flow monitor traffic-matrix-record force-export"
. Export the content the permanent cache every one hour . If the time is synchronized across routers (NTP), we have a synchronized export (snapshot)
Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 144