Hitchhiker's Guide to Troubleshooting Ipv6
Total Page:16
File Type:pdf, Size:1020Kb
BRKRST-3304 Hitchhiker’s Guide To Troubleshooting IPv6 - Advanced Nicole Wajer, @vlinder_nl Nicole Nicole Wajer Technical Solutions Architect @vlinder_nl BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 3 Cisco Spark Questions? Use Cisco Spark to communicate with the speaker after the session How 1. Find this session in the Cisco Live Mobile App 2. Click “Join the Discussion” 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public This Session…. BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5 This Session…. BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 Don’t Panic Agenda • Neighbor And Router Discovery • Addressing • IPv4 Coexistence And Transition • IPv6-centric Deployments © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Neighbor Discovery: Solicited Node Multicatscast Solicited node multicast groups: FF02::1:FF00:0000 /104 FF02::1:FF FF02::1:FF 00:0001 00:0002 FF02::1:FFAA:AAAA FF02::1:FFBB:BBBB FF02::1:FFCC:CCCC 2001:db8::0000:0002 2001:db8::0000:0001 BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 Nexus7000 not passing IPv6 traffic http://tinyurl.com/mld-on-nexus-7000 • On M1, M2 and M3 modules, you must disable IGMP optimized multicast flooding (OMF) on all VLANs that require IPv6 multicast packet forwarding. • On F2 modules, you must disable IGMP optimized multicast flooding (OMF) on all VLANs that require IPv6 packet forwarding (unicast or multicast). IPv6 neighbor discovery only functions in a VLAN with the OMF feature disabled. no ip igmp snooping optimise-multicast-flood http://www.cisco.com/en/US/docs/switches/datacenter/sw/nx-os/multicast/configuration/guide/b_multicast_chapter_0100.html#concept_4401AA5D7477469E9208FCE766906395 BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 Beware the ACL “tightening” ipv6 access-list ingress permit tcp host 2001:db8::1 eq 80 any deny ipv6 any any log permit icmp any any nd-ns implicit permit icmp any any nd-na deny ipv6 any any implicit BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 IPv6 ACL Implicit Rules • IPv6 ACLs configure like “extended named” • Matching, SRC, DST, next header • Applying the ACL uses ipv6 traffic-filter command • IPv6 ACLs have multiple implicit rules • Similar to deny ip any any ipv6 access-list IOS • IOS has 3 implicit IPv6 ACL rules permit icmp any any nd-na permit icmp any any nd-ns • NXOS has 5 implicit IPv6 ACL rules deny ipv6 any any • IOS-XE has no implicit IPv6 ACL rules ipv6 access-list NXOS permit icmp any any nd-na permit icmp any any nd-ns interface GigabitEthernet 0/2 permit icmp any any router-advertise ipv6 address 2001:db8:50:31::1/64 permit icmp any any router-solicitation ipv6 traffic-filter BLOCK-BAD in deny ipv6 any any BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 NIST guidelines for secure IPv6 deployment; RFC4890 http://csrc.nist.gov/publications/nistpubs/800-119/sp800-119.pdf http://www.ietf.org/rfc/rfc4890.txt See BRKSEC-2003 BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 Ducks in a Row • Code paths of requests/replies may differ • Multicast and Unicast processing can differ • Neighbor Solicitation contains Link-Layer address • May populate the cache without explicit request • Beware of defaults BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 Neighbor Cache State Machine • Incomplete – Pending address resolution, NS message outstanding • Reachable – Recently used mapping, Can be refreshed by ULP • Stale – Not currently communicating, waiting for next queued packet • Delay –Using stale binding, awaiting (ULP) return traffic • Probe – Sending Unicast NS to node (after Delay timer, 3x1 sec) NS No Entry Incomplete NA time expired Reachable NA ULP send packet Stale Delay Probe BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 ReachableTime: How Long Is It ? • BASE_REACHABLE_TIME • Sent in RA or taken from default • Value in milliseconds • Random(0.5 .. 1.5) * BASE_REACHABLE_TIME BASE_REACHABLE_TIME default: 30000 msec • Chosen every few hours or when BASE… changes BASE_REACHABLE_TIME BASE_REACHABLE_TIME RANDOM (0.5x .. 1.5x) 0.5x 1.5x milliseconds ReachableTime BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 Neighbor Table Maintenance Active Standby BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17 Neighbor Table Maintenance Can Burden The CPU Standby Newly active Active BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 DC ND Tuning TEST ! BASE_REAC • If FHRP is present or single gateway: increase reachable time HABLE_TIME ipv6 nd reachable-time 600000 ! 10 minutes • Pre-populate and maintain the neighbor table Expiry ipv6 nd cache expire 14400 refresh ipv6 nd na glean • Rate-limit the address resolution traffic Burst size mls rate-limit unicast cef glean 1000 10 • Start with this configuration and adjust depending on the site PPS • Wrong values can impact the neighbor resolution times! BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 IOS XR ND implementation details • New neighbor resolution: timeout 10 sec, not 3 sec. Total 3 retries • REACHABLE_TIME = 180000 ms (25% jitter), not 30000 ms (50% jitter) • DELAY is only 5 sec wait, no integration with TCP • Steady state probing (“PROBE”): • MAX_UNICAST_SOLICIT = 5, not 3 • timeout: 60 sec • Configuration to revert behavior to RFC values: RP/0/0/CPU0:ios(config)#int gigabitEthernet 0/1/0/0 RP/0/0/CPU0:ios(config-if)#ipv6 nd nud-conform RP/0/0/CPU0:ios(config-if)#commit BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 Ducks in a Row • ND has more states than ARP • Having “STALE” Neighbor Entry is ok! • Even in a connected Nespresso machine • Reachable interval is in milliseconds • Remember when adjusting • Adjust the Reachable timer up BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 Anatomy Of A Router Advertisement BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 Router Solicitations • Valid Options: Source link-layer address • The link-layer address of the sender, if known. MUST NOT be included if the Source Address is the unspecified address. Otherwise, it SHOULD be included on link layers that have addresses. BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 Router Solicitations and Neighbor Cache • If no Source Link Layer Address present • No effect • If Source Link Layer Address present • Installs/updates the entry and puts into “STALE” state • Consider for different ways of propagation of NS vs. RS • Host may get initial connectivity but not after clearing neighbor cache BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Default ACLs and Router Solicitations • None of the platform contain “default permit” for Router Solicitations • Delay with obtaining addresses • Mostly an issue with dynamic clients • Servers are less volatile BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 Router Advertisements and Battery Life The model of measurements • Three levels: • Device – level behavior • Network-wide behavior • Traffic on the network • Power consumption ~ F(number of hosts on segment, network volatility) • Two main sources of multicast traffic • IPv6 Neighbor Discovery protocol • Service Advertisements • More information on the power consumption model from the author directly: • http://tools.ietf.org/html/draft-desmouceaux-ipv6-mcast-wifi-power-usage-01 • Disclaimer: use this model as a guidance/basis only, verify your network telemetry! BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 Power Consumption On A Smartphone sleeping 10 mA awake 40 mA CPU awake 150 mA sleeping t I(t) BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 Experimental Measurements: Per Device When joining the network • At least 4 multicast packets issued (RS + 3DAD) • Possibly more than 20 (MLD, mDNS) joins Once connected • ~0.021 packets/device/second BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 Data Analysis From A Real Network (~600 nodes) • Arrival rates: exponential(λ) • Connection durations: ? • Here 600 hosts: 1/λ = 6 s (small)! • Average connection time = 55 min • Model: power multiplier is K = 1 + (0.03 + 28/Tc)*N • 27 nodes, 1 hour average connection time K = 2 (!) BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 Why Multicast Solicited RAs ? RFC4861, 6.2.6. Processing Router Solicitations In addition to sending periodic, unsolicited advertisements, a router sends advertisements in response to valid solicitations received on an advertising interface. A router MAY choose to unicast the response directly to the soliciting host's address (if the solicitation's source address is not the unspecified address), but the usual case is to multicast the response to the all-nodes group. BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31 Tcpdump On A Host In A Large WiFi Network BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 WLC Sends RAs Reliably: Can Reduce Frequency! APc47a.fe34.1cc9#show capwap mcast mgid all | begin RA MGID RA MGID Information MGID = 8341 IPv6 mc2uc Clients = 1 MGID = 8343 IPv6 mc2uc Clients = 1 APc47a.fe34.1cc9#show capwap mcast mgid id 8343 Normal Mcast Clients: Reliable Mcast Clients: Client: 14cf.929d.740c --- Qos User Priority: 3 State: ADMITTED History – Retry Pct: 0 0 0 0 Rate )500Kbps): 0 65535 65535 APc47a.fe34.1cc9# BRKRST-3304 © 2018 Cisco and/or its affiliates.