Hitchhiker's Guide to Troubleshooting Ipv6

BRKRST-3304

Hitchhiker’s Guide To Troubleshooting IPv6 - Advanced

Nicole Wajer, @vlinder_nl Nicole

Nicole Wajer Technical Solutions Architect @vlinder_nl

Questions? Use Cisco Spark to communicate with the speaker after the session

How 1. Find this session in the Cisco Live Mobile App 2. Click “Join the Discussion” 3. Install Spark or go directly to the space 4. Enter messages/questions in the space

cs.co/ciscolivebot#BRKRST-3304

• Neighbor And Router Discovery

• Addressing

• IPv4 Coexistence And Transition

• IPv6-centric Deployments

Solicited node multicast groups: FF02::1:FF00:0000 /104

FF02::1:FF

00:0001 00:0002

FF02::1:FFAA:AAAA FF02::1:FFBB:BBBB FF02::1:FFCC:CCCC

2001:db8::0000:0002 2001:db8::0000:0001

• On M1, M2 and M3 modules, you must disable IGMP optimized multicast flooding (OMF) on all VLANs that require IPv6 multicast packet forwarding.

• On F2 modules, you must disable IGMP optimized multicast flooding (OMF) on all VLANs that require IPv6 packet forwarding (unicast or multicast). IPv6 neighbor discovery only functions in a VLAN with the OMF feature disabled.

no ip igmp snooping optimise-multicast-flood

http://www.cisco.com/en/US/docs/switches/datacenter/sw/nx-os/multicast/configuration/guide/b_multicast_chapter_0100.html#concept_4401AA5D7477469E9208FCE766906395

ipv6 access-list ingress permit tcp host 2001:db8::1 eq 80 any deny ipv6 any any log

permit icmp any any nd-ns  implicit permit icmp any any nd-na

deny ipv6 any any  implicit

• IPv6 ACLs configure like “extended named” • Matching, SRC, DST, next header • Applying the ACL uses ipv6 traffic-filter command

• IPv6 ACLs have multiple implicit rules • Similar to deny ip any any ipv6 access-list IOS • IOS has 3 implicit IPv6 ACL rules permit icmp any any nd-na permit icmp any any nd-ns • NXOS has 5 implicit IPv6 ACL rules deny ipv6 any any • IOS-XE has no implicit IPv6 ACL rules ipv6 access-list NXOS permit icmp any any nd-na permit icmp any any nd-ns interface GigabitEthernet 0/2 permit icmp any any router-advertise ipv6 address 2001:db8:50:31::1/64 permit icmp any any router-solicitation ipv6 traffic-filter BLOCK-BAD in deny ipv6 any any

http://csrc.nist.gov/publications/nistpubs/800-119/sp800-119.pdf http://www.ietf.org/rfc/rfc4890.txt See BRKSEC-2003

• Code paths of requests/replies may differ • Multicast and Unicast processing can differ

• Neighbor Solicitation contains Link-Layer address • May populate the cache without explicit request

• Beware of defaults

• Incomplete – Pending address resolution, NS message outstanding

• Reachable – Recently used mapping, Can be refreshed by ULP

• Stale – Not currently communicating, waiting for next queued packet

• Delay –Using stale binding, awaiting (ULP) return traffic

• Probe – Sending Unicast NS to node (after Delay timer, 3x1 sec) NS No Entry Incomplete NA

time expired Reachable NA ULP send packet Stale Delay Probe

• BASE_REACHABLE_TIME • Sent in RA or taken from default • Value in milliseconds

• Random(0.5 .. 1.5) * BASE_REACHABLE_TIME BASE_REACHABLE_TIME default: 30000 msec • Chosen every few hours or when BASE… changes

BASE_REACHABLE_TIME BASE_REACHABLE_TIME RANDOM (0.5x .. 1.5x) 0.5x 1.5x milliseconds ReachableTime

Active

Standby

Newly active

Active

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 DC ND Tuning TEST ! BASE_REAC • If FHRP is present or single gateway: increase reachable time HABLE_TIME ipv6 nd reachable-time 600000 ! 10 minutes

• Pre-populate and maintain the neighbor table Expiry ipv6 nd cache expire 14400 refresh ipv6 nd na glean

• Rate-limit the address resolution traffic Burst size mls rate-limit unicast cef glean 1000 10

• Start with this configuration and adjust depending on the site PPS • Wrong values can impact the neighbor resolution times!

• New neighbor resolution: timeout 10 sec, not 3 sec. Total 3 retries

• REACHABLE_TIME = 180000 ms (25% jitter), not 30000 ms (50% jitter)

• DELAY is only 5 sec wait, no integration with TCP

• Steady state probing (“PROBE”): • MAX_UNICAST_SOLICIT = 5, not 3 • timeout: 60 sec

• Configuration to revert behavior to RFC values:

RP/0/0/CPU0:ios(config)#int gigabitEthernet 0/1/0/0 RP/0/0/CPU0:ios(config-if)#ipv6 nd nud-conform RP/0/0/CPU0:ios(config-if)#commit

• ND has more states than ARP

• Having “STALE” Neighbor Entry is ok! • Even in a connected Nespresso machine

• Reachable interval is in milliseconds • Remember when adjusting

• Adjust the Reachable timer up

• Valid Options: Source link-layer address • The link-layer address of the sender, if known. MUST NOT be included if the Source Address is the unspecified address. Otherwise, it SHOULD be included on link layers that have addresses.

• If no Source Link Layer Address present • No effect

• If Source Link Layer Address present • Installs/updates the entry and puts into “STALE” state

• Consider for different ways of propagation of NS vs. RS • Host may get initial connectivity but not after clearing neighbor cache

• None of the platform contain “default permit” for Router Solicitations

• Delay with obtaining addresses

• Mostly an issue with dynamic clients • Servers are less volatile

• Three levels: • Device – level behavior • Network-wide behavior • Traffic on the network

• Power consumption ~ F(number of hosts on segment, network volatility)

• Two main sources of multicast traffic • IPv6 Neighbor Discovery protocol • Service Advertisements

• More information on the power consumption model from the author directly: • http://tools.ietf.org/html/draft-desmouceaux-ipv6-mcast-wifi-power-usage-01 • Disclaimer: use this model as a guidance/basis only, verify your network telemetry!

sleeping 10 mA

awake 40 mA

CPU awake 150 mA

sleeping

t I(t)

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 Experimental Measurements: Per Device When joining the network • At least 4 multicast packets issued (RS + 3DAD) • Possibly more than 20 (MLD, mDNS)

joins

Once connected • ~0.021 packets/device/second

• Arrival rates: exponential(λ) • Connection durations: ?

• Here 600 hosts: 1/λ = 6 s (small)! • Average connection time = 55 min • Model: power multiplier is K = 1 + (0.03 + 28/Tc)*N • 27 nodes, 1 hour average connection time  K = 2 (!)

In addition to sending periodic, unsolicited advertisements, a router sends advertisements in response to valid solicitations received on an advertising interface. A router MAY choose to unicast the response directly to the soliciting host's address (if the solicitation's source address is not the unspecified address), but the usual case is to multicast the response to the all-nodes group.

APc47a.fe34.1cc9#show capwap mcast mgid all | begin RA MGID RA MGID Information MGID = 8341 IPv6 mc2uc Clients = 1 MGID = 8343 IPv6 mc2uc Clients = 1 APc47a.fe34.1cc9#show capwap mcast mgid id 8343 Normal Mcast Clients: Reliable Mcast Clients: Client: 14cf.929d.740c --- Qos User Priority: 3 State: ADMITTED History – Retry Pct: 0 0 0 0 Rate )500Kbps): 0 65535 65535 APc47a.fe34.1cc9#

IOS vs. NX-OS Default Solicited RA Behavior

• NX-OS sends unicast solicited RA packets

• Periodic RA still sent multicast as expected

• Easy (Less need for RA-throttle), but may be harder to debug (ucast vs. mcast)

• Periodic Router Advertisements: suppressed

• Solicited Router Advertisements: unicast

• Problem: maximum connection time limited by 9000 sec.

• Periodic Router Advertisements sent as Multicast

• Solicited Router Advertisements sent as Unicast

• 15.4(03)S, 15.4(02)T01

• Router Advertisements require processing • Do not blindly send them too frequently

• Router Solicitation triggers Router Advertisement • Adjusting the interval alone is not enough

• There are many tools to control the RAs • Send Solicited RA unicast • RA Throttler

• Work in progress in IETF to further improve in his are

• Neighbor And Router Discovery

• Addressing

• IPv4 Coexistence And Transition

• IPv6-centric Deployments DAD (Duplicate Address Detection)

• Neighbor Solicitation from Unspecified ( :: ) address

• At least 1 second delay

• Rfc4429 - Optimistic DAD • No delay

• Rfc7527 - Enhanced DAD • Improved loopback detection • Self-healing

Anyone with this addr ? IPv6 LL DAD NS

Router Solicitation

RtrAdv Pref; “A” “O” “M”

IPv6 g.a. DAD NS

DHCPv6 inf req

DHCPv6 reply (DNS)

DHCPv6 req

DHCPv6 reply (address) IPv6 g.a. DAD NS

Host State Post-Attachment

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64 M-, O-, A- flags: (Too) Many To Choose From ? Host State Input Behavior Host has not acquired any No RA Some OS perform addresses DHCPv6, some do not Host has not acquired any RA with M=0, O=1 Some OS acquire info addresses only if A=1 Host has acquired RA with M=0 Some OS release DHCPv6 addresses (M=1, DHCPv6 addresses A=0) immediately, some not Host has acquired RA with M=1 Some OS acquire SLAAC-only addresses DHCPv6 address (A=1, M=0) immediately, some not

https://tools.ietf.org/html/draft-ietf-v6ops-dhcpv6-slaac-problem

• Pros of using SLAAC • Cons of using SLAAC • No need to do stateful DHCP • Some stacks (iOS) are very aggressive with temp. addresses • Wide device support (Android!) • More volatility in the binding table/ND • “IPv6 way” • Address tracing is harder

Question: Would you run both SLAAC and DHCPv6 and why ?

ayourtch@mcnano:~$ ip -6 addr 1: lo: mtu 65536 inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qlen 1000 inet6 2001:470:1f13:62e:90f8:5341:15d:e733/64 scope global temporary dynamic valid_lft 601936sec preferred_lft 82936sec inet6 2001:470:1f13:62e:1d4d:4d2b:129e:13b8/64 scope global temporary deprecated dynamic valid_lft 516139sec preferred_lft 0sec inet6 2001:470:1f13:62e:bc4e:defa:819f:fb40/64 scope global temporary deprecated dynamic valid_lft 430342sec preferred_lft 0sec inet6 2001:470:1f13:62e:517:5a87:6d1c:618e/64 scope global temporary deprecated dynamic valid_lft 344544sec preferred_lft 0sec inet6 2001:470:1f13:62e:1cd:10de:7ec0:889e/64 scope global temporary deprecated dynamic valid_lft 258747sec preferred_lft 0sec inet6 2001:470:1f13:62e:11c9:c1a4:952c:d327/64 scope global temporary deprecated dynamic valid_lft 172949sec preferred_lft 0sec inet6 2001:470:1f13:62e:59f5:704b:a59a:4f13/64 scope global temporary deprecated dynamic valid_lft 87151sec preferred_lft 0sec inet6 2001:470:1f13:62e:6a5b:35ff:fed0:8d7c/64 scope global dynamic valid_lft forever preferred_lft 86307sec inet6 fe80::6a5b:35ff:fed0:8d7c/64 scope link valid_lft forever preferred_lft forever ayourtch@mcnano:~$

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 67 To SLAAC or not to SLAAC ? interface Vlan102 ip address 10.2.1.1 255.255.255.0 ipv6 address FE80::1 link-local ipv6 address 2001:db8::1/64 ipv6 nd prefix default 86400 3600 no-autoconfig ipv6 nd managed-config-flag ipv6 nd other-config-flag ipv6 nd router-preference High ipv6 nd ra mtu suppress ipv6 nd ra interval 300 ipv6 dhcp server DUALSTACK end

https://www.ietf.org/proceedings/93/slides/slides-93-intarea-5.pdf

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 69 Quiz: “No valid route for destination” – why ? R1#show run interface Gig0/1 Interface GigabitEthernet1/0 no ip address What’s the problem ? negotiation auto ipv6 address FE80::1 link-local ipv6 address 2001:DB8::1/64 R1#

R1#ping 2001:db8::2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2001:DB8::2, timeout is 2 seconds:

% No valid route for destination Success rate is 0 percent (0/1)

• Modern hosts implement optimizations • Optimistic DAD • Attempt to use old IPv6 address

• DHCPv6 – more “traditional” • Allows the control of addreses (DUID may not be known in advance) • DHCPv6-PD allows prefix allocation

• SLAAC • Device-centric model • Decentralized • No influence over the Interface ID used by a host

• Neighbor And Router Discovery

• Addressing

• IPv4 Coexistence And Transition

• IPv6-centric Deployments Dualstack: Always remember both protocols

Type “example.com” and press Enter

A? “example.com” AAAA? “example.com”

connect connect 2001:500:88:200::10 192.0.43.10

GET / HTTP/1.1 Host: example.com

User: “example.com”

getaddrinfo(“example.com”)

Connection failure Attempt IPv6 connect

Attempt IPv4 connect

Retrieve and display

Time

http://www.ietf.org/proceedings/80/slides/v6ops-11.pdf

Internet Engineering Task Force (IETF) D. Wing Request for Comments: 6555 A.Yourtchenko Category: Standards Track Cisco ISSN: 2070-1721 April 2012

Happy Eyeballs: Success with Dual-Stack Hosts

Abstract

When a server's IPv4 path and protocol are working, but the server's IPv6 path and protocol are not working, a dual-stack client application experiences significant connection delay compared to an IPv4-only client. This is undesirable because it causes the dualstack client to have a worse user experience. This document specifies requirements for algorithms that reduce this user-visible delay and provides an algorithm.

Attempt IPv6 lookup and connect

User: “example.com” Retrieve and display

Attempt IPv4 lookup and connect

~300ms

Time

• Dualstack selection service may not be deterministic

• Add two hostnames, one IPv4-only and one IPv6-only – retest with them if in doubt. dhcp-10-149-4-30:~ ayourtch$ host stdio.be stdio.be has address 188.40.136.148 stdio.be has IPv6 address 2a01:4f8:101:3245::cafe stdio.be mail is handled by 10 mail.stdio.be. dhcp-10-149-4-30:~ ayourtch$ host ipv6.stdio.be ipv6.stdio.be has IPv6 address 2a01:4f8:101:3245::cafe dhcp-10-149-4-30:~ ayourtch$ host ipv4.stdio.be ipv4.stdio.be has address 188.40.136.148 dhcp-10-149-4-30:~ ayourtch$

https://www.ripe.net/ripe/groups/tf/bcop/ipv6-troubleshooting-for-residential-isp-helpdesks

• Instrument for monitoring of both address families

• The good tooling end education are there

• There are established procedures for first-level troubleshooting • Use Them!

• Neighbor And Router Discovery

• Addressing

• IPv4 Coexistence And Transition

• IPv6-centric Deployments Do they exist, IPv6-only clients ?

Picture: http://en.wikipedia.org/wiki/File:Oftheunicorn.jpg

search for: “deploy360 t-mobile case study” Picture source: http://en.wikipedia.org/wiki/Rhinoceros

http://www.internetsociety.org/deploy360/resources/case-study-t-mobile-us-goes-ipv6-only-using-464xlat/

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 86 Because IPv6 support is so critical to ensuring your applications work across the world for every customer, we are making it an AppStore submission requirement, starting with iOS 9.” Sebastien Marineau, VP of Core OS, Apple (June 2015)

Address from IPv4 pool Map into IPv6 /96

IPv6 IPv4

IPv6 traffic IPv4 traffic

• 2001:db8:aaaa:aaaa::192.0.2.1

• 2001:db8:aaaa:aaaa::c000:201

IPv6

IPv4

Resolving Recursive Authoritative Host Resolver Name Server

IPv6

IPv4 IPv4

Andrews-MacBook-Air:~ ayourtch$ curl -v http://cs.co/6011pZiX * About to connect() to cs.co port 80 (#0) * Trying 67.192.93.178... * connected * Connected to cs.co (67.192.93.178) port 80 (#0) > GET /6011pZiX HTTP/1.1 > User-Agent: curl/7.28.1 > Host: cs.co > Accept: */* > < HTTP/1.1 301 Moved Permanently < Date: Fri, 07 Dec 2012 01:59:02 GMT < Server: Apache/2.2.3 (Red Hat) < Location: http://184.72.243.192//6011pZiX < Keep-Alive: timeout=15, max=99 < Content-Type: text/html; charset=iso-8859-1 < Content-Length: 310 < Via: 1.1 ams3-dmz-wsa-1.cisco.com:80 (WSA/x) < Connection: keep-alive < 301 Moved Permanently

Moved Permanently

The document has moved here.

Apache/2.2.3 (Red Hat) Server at cs.co Port 80

* Connection #0 to host cs.co left intact

Andrews-MacBook-Air:~ ayourtch$ curl -v http://cs.co/6011pZiX * About to connect() to cs.co port 80 (#0) * Trying 67.192.93.178... * connected * Connected to cs.co (67.192.93.178) port 80 (#0) > GET /6011pZiX HTTP/1.1 > User-Agent: curl/7.28.1 > Host: cs.co > Accept: */* Location: http://184.72.243.192//6011pZiX > < HTTP/1.1 301 Moved Permanently < Date: Fri, 07 Dec 2012 01:59:02 GMT < Server: Apache/2.2.3 (Red Hat) < Location: http://184.72.243.192//6011pZiX < Keep-Alive: timeout=15, max=99 < Content-Type: text/html; charset=iso-8859-1 < Content-Length: 310 < Via: 1.1 ams3-dmz-wsa-1.cisco.com:80 (WSA/x) < Connection: keep-alive < 301 Moved Permanently

Moved Permanently

The document has moved here.

Apache/2.2.3 (Red Hat) Server at cs.co Port 80

* Connection #0 to host cs.co left intact

Andrews-MacBook-Air:~ ayourtch$ curl -v http://cs.co/6011pZiX * About to connect() to cs.co port 80 (#0) * Trying 67.192.93.178... * connected * Connected to cs.co (67.192.93.178) port 80 (#0) > GET /6011pZiX HTTP/1.1 > User-Agent: curl/7.28.1 > Host: cs.co > Accept: */* > < HTTP/1.1 301 Moved Permanently < Date: Tue, 08 Jan 2013 00:54:25 GMT < Server: Apache/2.2.3 (Red Hat) < Location: http://ec2-184-72-243-192.compute-1.amazonaws.com//6011pZiX < Keep-Alive: timeout=15, max=99 < Content-Type: text/html; charset=iso-8859-1 < Content-Length: 338 < Via: 1.1 ams3-dmz-wsa-4.cisco.com:80 (WSA/x) < Connection: keep-alive < 301 Moved Permanently

Moved Permanently

The document has moved here.

Apache/2.2.3 (Red Hat) Server at cs.co Port 80

* Connection #0 to host cs.co left intact * Closing connection #0 Andrews-MacBook-Air:~ ayourtch$

Andrews-MacBook-Air:~ ayourtch$ curl -v http://cs.co/6011pZiX * About to connect() to cs.co port 80 (#0) * Trying 67.192.93.178... * connected * Connected to cs.co (67.192.93.178) port 80 (#0) > GET /6011pZiX HTTP/1.1 > User-Agent: curl/7.28.1 > Host: cs.co Location:> Accept: */* http://ec2-184-72-243-192.compute-1.amazonaws.com//6011pZiX > < HTTP/1.1 301 Moved Permanently < Date: Tue, 08 Jan 2013 00:54:25 GMT < Server: Apache/2.2.3 (Red Hat) < Location: http://ec2-184-72-243-192.compute-1.amazonaws.com//6011pZiX < Keep-Alive: timeout=15, max=99 < Content-Type: text/html; charset=iso-8859-1 < Content-Length: 338 < Via: 1.1 ams3-dmz-wsa-4.cisco.com:80 (WSA/x) < Connection: keep-alive < 301 Moved Permanently

Moved Permanently

The document has moved here.

Apache/2.2.3 (Red Hat) Server at cs.co Port 80

* Connection #0 to host cs.co left intact * Closing connection #0 Andrews-MacBook-Air:~ ayourtch$

IPv6 IPv4

IPv6 traffic IPv4 traffic

Appl. API Host IPv6

Function calls packets packets Android

iOS

iOS IPv4 literal synthesis API: https://www.ietf.org/mail-archive/web/v6ops/current/msg23222.html

https://developer.apple.com/news/?id=05042016a

• Different OS use different approaches for legacy apps

• Higher-level API’s provide better coexistence support

• Any new applications MUST be designed with IPv6-only/NAT64 in mind

ipv6 access-list NAT64 permit tcp 2001:DB8::/64 64:FF9B::/64 permit udp 2001:DB8::/64 64:FF9B::/64 permit icmp 2001:DB8::/64 64:FF9B::/64 ! ! nat64 v4 pool NAT64-IPv4 192.0.2.1 192.0.2.1 nat64 v6v4 list NAT64 pool NAT64-IPv4 overload !

http://docwiki.cisco.com/wiki/IPv6_only_setup_with_NAT64

Alt-Click

• IPV6-only clients • T-Mobile USA • http://www.internetsociety.org/deploy360/resources/case-study-t-mobile-us-goes-ipv6-only-using- 464xlat/ • Orange Poland • https://www.youtube.com/watch?v=Y0G5PTtZjTM (Polish language) • Telenor Norway (opt-in) • http://blog.toreanderson.no/2015/09/20/ipv6-mobile-roaming-possible-or-not.html

• IPv6-only servers • Redpill Linpro • http://blog.ipspace.net/2012/05/ipv6-only-data-center-built-by-tore.html

• Main changes are at First Hop

• Prolonged use of dualstack introduces complexity • Keep sunsetting IPv4 in mind from the start

• IPv6-only requirements from endpoint vendors pave the way to future single- stack deployments • Don’t panic!

Shameless self promotion of my own Quotes - Nicole Wajer

• https://sites.google.com/site/yartikhiy/home/ipv6book

WPA2-PSK: “cl-nat64”

5GHz only

WPA2-PSK: “cl-nat64”

5GHz only

WPA2-PSK: “cl-nat64”

5GHz only

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Tech Circle

• Meet the Engineer 1:1 meetings

• Gain Operational Experience now

• Control IPv6 traffic as you would IPv4

• “Poke” your Provider’s

• BRKIP6-2301 - Enterprise IPv6 Deployment – 31 Jan. 11:30

• LABSPG-3122 - Advanced IPv6 Routing and services lab – 31 Jan 14:00 & 1 Feb. 14:00

• BRKCOL-2020 - IPv6 in Enterprise Unified Communications Networks – 31 Jan. 16:30

• BRKCOC-2388 - Inside Cisco IT: A Tale of Two Protocols – 2 Feb. 9:00

• BRKIP6-2002 - IPv6 for the World of IoT – 2 Feb. 11:30

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 114 • Please complete your Online Complete Your Online Session Evaluations after each session Session Evaluation • Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt • All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.

Backup/bonus slides For Your Reference NDP Scaling Techniques

• ND cache sizing - ipv6 nd cache interface-limit • Need to account for link local addresses • NUD Reachable Time: ipv6 nd reachable-time WARNING • Using a FHRP, move from 30 sec (default) to 10 minutes MUST USE WITH CAUTION • Scavenge and Refresh Timer: ipv6 nd cache expire • Using a FHRP, use refresh in conjunction with NA glean • Unsolicited NA Glean: ipv6 nd na glean • Create neighbor entries from unsolicited NA’s received • Router Advertisements: ipv6 nd ra interval • IOS = 200 Sec, NXOS = 600 Sec • Router lifetime = 3x RA interval

• draft-ietf-6man-maxra • increase max router lifetime from 9000 to 65535

• draft-ietf-6man-rs-refresh-01 • client-initiated RA refresh

• RFC7559 • resilient (re)-transmission of initial RS

IPv6

LifetimeLifetime--

IPv6 RA RA Sent Every 200sec LifetimeLifetime-- +/- jitter IPv6

ip pim rp-address 172.16.10.50 ip pim ssm default

interface GigabitEthernet1 ip address 172.17.1.1 255.255.255.0 ip pim sparse mode ip igmp version 3

Outgoing interface flags: H - Hardware switched, A – Assert Timers: Uptime/Expires Interface state: Interface, Next-Hop or VCD, State/Mode

(172.17.1.20, 232.1.1.2), 00:12:36/00:02:23, flags: sTI Incoming interface: GigabitEthernet1, RPF nbr 0.0.0.0 Outgoing interface list: GigabitEthernet1.118, Forward/Sparse, 00:12:36/00:02:23

(*, 224.0.1.40), 00:24:39/00:02:53, RP 172.16.10.50, flags: SJCL Incoming interface: Null, RPF nbr 0.0.0.0 Outgoing interface list: GigabitEthernet1, Forward/Sparse, 00:24:39/00:02:53

APc47a.fe34.1cc9#show capwap mcast CAPWAP MULTICAST Multicast Group: 232.1.1.2, Source: 172.17.1.20 V1 Rpt Sent: 0; V2 Rpt Sent: 2 V3 Rpt Sent: 189; Leave Sent: 1 V1 Query Rcvd: 0; V2 Query Rcvd: 0 V3 Query Rcvd: 188; V1 Rpt Rcvd: 0 V2 Rpt Rcvd: 0; V3 Rpt Rcvd: 0 APc47a.fe34.1cc9#

APc471.fe34.1cc9#show capwap mcast mgid id 8343 rx pkts = 4 tx packets: wlan : 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 slots0 : 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 slots1 : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Normal Mcast Clients: Reliable Mcast Clients: Client: 14cf.929d.740c --- SlotId: 0 WlanId: 1 --- Qos User Priority: 3 State: ADMITTED History - Retry Pct: 0 0 0 0 Rate (500 Kbps): 0 65535 65535 65535 Client: 14cf.923c.117c --- SlotId: 0 WlanId: 1 --- Qos User Priority: 3 State: ADMITTED History - Retry Pct: 0 0 0 0 Rate (500 Kbps): 0 65535 65535 65535 APc471.fe34.1cc9##

ipv6 router eigrp 1 router-id 192.0.2.1 no shutdown

R1#conf t Enter configuration commands, one per line. End with CNTL/Z. R1(config)#int e0/1 R1(config-if)# ipv6 address 2001:DB8:CAFE::1/64 R1(config-if)#vrrp 101 ? authentication Authentication description Group specific description ip Enable Virtual Router Redundancy Protocol (VRRP) for IP preempt Enable preemption of lower priority Master priority Priority of this VRRP group shutdown Disable VRRP Configuration timers Set the VRRP timers track Event Tracking

R1(config-if)#vrrp 101

fhrp version vrrp v3

interface Ethernet0/1 no ip address ipv6 address 2001:DB8:CAFE::1/64 vrrp 101 address-family ipv6 address FE80::1 primary exit-vrrp !

Total routes: 513525 IPv4 unicast routes: 513507 IPv4 non-vrf routes: 513507 IPv4 vrf routes: 0 IPv4 Multicast routes: 3 MPLS routes: 1 IPv6 unicast routes: 5 IPv6 non-vrf routes: 5 IPv6 vrf routes: 0 IPv6 multicast routes: 3 EoM routes: 1

7600#

Default is 512K IPv4 routes

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 135 C6500/C7600 SUP720 TCAM Customization 7600# show mls cef max FIB TCAM maximum routes : ======Current :------IPv4 + MPLS - 512k (default) IPv6 + IP Multicast - 256k (default) 7600# Not 1000! Leave some space for IPv6 routes! mls cef maximum-routes ip 768

https://supportforums.cisco.com/discussion/11333356/cisco-7609-rsp720-3cxl-ge-mls-cef-maximum-routes http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html

Changing the TCAM layout requires a reboot Sup2T has a shared pool of TCAM for IPv4 and IPv6  no customization needed

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 146 Nexus 1000V flow vPath cdn-nexus1k-4# show vservice connection Actions(Act): d - drop s - reset p - permit t - passthrough r - redirect e - error n - not processed upper case - offloaded Flags: A - seen ack for syn/fin from src a - seen ack for syn/fin from dst E - tcp conn established (SasA done) F - seen fin from src f - seen fin from dst R - seen rst from src r - seen rst from dst S - seen syn from src s - seen syn from dst T - tcp conn torn down (FafA done) x - IP-fragment connection

#Port-Profile:Access_vlan1353 Node:ns1000v #Module 3 Proto SrcIP[:Port] SAct DstIP[:Port] DAct Flags Bytes icmp 192.168.37.1 192.168.37.32 p 546 icmp 192.168.37.1 192.168.37.31 p 546 tcp 192.168.37.1:1805 192.168.37.32:80 p E 1255 cdn-nexus1k-4#

> show ip Ipaddress TD Type Mode Arp Icmp Vserver State ------1) 10.48.62.6 0 NetScaler IP Active Enabled Enabled NA Enabled 2) 10.48.62.8 0 SNIP Active Enabled Enabled NA Enabled 3) 192.168.37.1 0 SNIP Active Enabled Enabled NA Enabled 4) 192.168.35.2 0 SNIP Active Enabled Enabled NA Enabled 5) 192.168.35.20 0 VIP Active Enabled Enabled Enabled Enabled >

> show ip6 IPv6 Address TD Vlan Type Scope State ------1) fe80::202:3dff:fe70:6605/64 0 1 NSIP link-local ACTIVE 2) 2001:db8:1::1/64 0 NA SNIP global ACTIVE 3) 2001:db8:1::10/128 0 NA VIP global ACTIVE Done >

> show nd6 Neighbor MAC-Address(Vlan, Interface) TD State TIME ------1) ::1 00:02:3d:70:66:05( 1, LO/1) 0 REACHABLE PERMANENT 2) fe80::202:3dff:fe70:6605 00:02:3d:70:66:05( 1, LO/1) 0 REACHABLE PERMANENT 3) 2001:db8:1::1000 00:50:56:b8:9d:4d(1351, 1/1) 0 REACHABLE 00:00:21 4) fe80::38b8:1c9:2338:e677 00:50:56:b8:9d:4d(1351, 1/1) 0 STALE 00:04:39 Done >

> show lb vserver static_VIP_vlan_1351_v6 static_VIP_vlan_1351_v6 (2001:db8:1::10.80) - HTTP Type: ADDRESS State: UP Last state change was at Thu Jan 16 08:52:56 2014 Time since last state change: 0 days, 00:04:11.900 Effective State: UP Client Idle Timeout: 180 sec Down state flush: ENABLED Disable Primary Vserver On Down : DISABLED Appflow logging: ENABLED Port Rewrite : DISABLED No. of Bound Services : 2 (Total) 2 (Active) Configured Method: LEASTCONNECTION Current Method: Round Robin, Reason: Bound service's state changed to UP Mode: IP Persistence: NONE Vserver IP and Port insertion: OFF Push: DISABLED Push VServer: Push Multi Clients: NO Push Label Rule: none L2Conn: OFF Skip Persistency: None IcmpResponse: PASSIVE New Service Startup Request Rate: 0 PER_SECOND, Increment Interval: 0 TD: 0 Mac mode Retain Vlan: DISABLED DBS_LB: DISABLED DNS64 Synth: DISABLED Bypass AAAA: NO

1) HTTP_vm-31 (192.168.37.31: 80) - HTTP State: UP Weight: 1 2) HTTP_vm-32 (192.168.37.32: 80) - HTTP State: UP Weight: 1

• Defined in RFC3315

• Multiple enhancements/additions • DHCPv6-PD, Stateless DHCPv6…

• Work In Progress: “draft-3315-bis” • DHCPv6-PD (RFC3633) • Stateless DHCPv6 extensions (RFC3736) • Interaction between the mechanisms (RFC7550)

• https://tools.ietf.org/html/draft-ietf-dhc-rfc3315bis

internet

End

- Back IPv6 Internet

IPv6

Client leg Server leg

MTU6 = 40+20+N MTU4 = 20+20+N

MTU6 MTU4

IPv6 hdr TCP hdr Data IPv4 hdr TCP hdr Data

40 bytes 20 bytes N bytes IPv6 20 bytes 20 bytes N bytes

IPv4 TCP TLS Data IPv4 TCP Data

20 20 21..35 N bytes 20 20 N bytes

MTU MTU

IPv6 TCP TLS Data IPv6 TCP Data

40 20 21..35 N bytes IPv6 40 20 N bytes IPv6 MTU4tls = MTU4 + (21..35) MTU6tls = MTU4 + (41..55) BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 159 “IPv6 requires that every link in the internet have an MTU of 1280 octets or greater. On any link that cannot convey a 1280-octet packet in one piece, link-specific fragmentation and reassembly must be provided at a layer below IPv6.” RFC2460

Data MTU=1280 2

ICMP PTB

3 4 Data Data

MTU MTU here Spot the problem! 1

Data 2 TLS Data

3 4 ICMP PTB ICMP PTB” 5

6 Data TLS Data

IPv4 has min MTU of 68, IPv6 has min MTU of 1280

• Test with different client MTUs • 1280 (Minimal IPv6 MTU, set on some tunnels) • 1480 (IPv6-in-IPv4) • 1500 (standard Ethernet)

• Keep ICMPv6 in mind when designing the network

IPv4 IPv4

asr1knat64-xtr#sh nat64 trans IPv4 tcp 72.163.4.161:80 [2610:d0:1208:cafe::48a3:4a1]:80 internet 153.16.17.82:1056 [2607:f128:42:73::2]:37897

s: 153.17.16.82:1056 End - d: 72.163.4.161:80

3 Back IPv6 Internet s: [2610:d0:1208:cafe::72.163.4.161]:80

d: [2607:f128:42:73::2]:37897 s: 72.163.4.161:80 1 5 d: 153.17.16.82:1056 s: [2607:f128:42:73::2]:37897

d: [2610:d0:1208:cafe::72.163.4.161]:80 IPv6

Client leg Server leg

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 164 Symptom: IPv6 clients can not connect %NAT64-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 1 may be exhausted

asr1knat64-xtr#show nat64 stat | beg Dynamic Dynamic Mapping Statistics v6v4 access-list NAT64 pool TEST refcount 2 pool TEST: start 153.16.17.84 end 153.16.17.84 total addresses 1, allocated 1 (100%) address exhaustion packet count 0 Limit Statistics

asr1knat64-xtr#

asr1knat64-xtr#show nat64 trans

Proto Original IPv4 Translated IPv4 Translated IPv6 Original IPv6 ------153.16.17.84 2a01:4f8:101:3245::fafa --- 192.0.2.2 2610:d0:1208:cafe::c000:202 153.16.17.84 2a01:4f8:101:3245::fafa

Total number of translations: 2

asr1knat64-xtr#

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 166 Problem: Address pool exhausted due to 1:1 NAT nat64 prefix stateful 2610:D0:1208:CAFE::/96 nat64 v4 pool TEST 153.16.17.84 153.16.17.84 nat64 v6v4 list NAT64 pool TEST overload ipv6 access-list NAT64 permit ipv6 any 2610:D0:1208:CAFE::/96

IPv6 hosts

Gig0/0/1

Gig0/0/0 IPv4 hosts

BRKRST-3304 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 167 Solution: be more specific on the NAT[46]4 ACLs! ipv6 access-list NAT64 no permit ipv6 any 2610:D0:1208:CAFE::/96 permit tcp any 2610:D0:1208:CAFE::/96 permit udp any 2610:D0:1208:CAFE::/96 permit icmp any 2610:D0:1208:CAFE::/96

IPv6 hosts

Gig0/0/1

Gig0/0/0 IPv4 hosts

Proto Original IPv4 Translated IPv4 Translated IPv6 Original IPv6 ------

tcp 192.0.2.2:80 [2610:d0:1208:cafe::c000:202]:80 153.16.17.84:1024 [2a01:4f8:101:3245::cafe]:12345 udp 192.0.2.2:53 [2610:d0:1208:cafe::c000:202]:53 153.16.17.84:512 [2a01:4f8:101:3245::cafe]:53

Total number of translations: 2

asr1knat64-xtr#

• Chrome/Firefox: use the “backup thread” mechanism, 300ms delay

• iOS / MacOS X: 25ms preference for IPv6; connect-by-name proprietary API; re- sorting by the order of received replies if using getaddrinfo()

• Windows 8: perform a connectivity check, and if does not work, change sorting order in rfc3484 getaddrinfo() call to prefer IPv4, cache the result. • http://support.microsoft.com/kb/2750841

http://docwiki.cisco.com/wiki/IPv6_only_setup_with_NAT64

Alt-Click

• IPv6-only servers • Redpill Linpro • http://blog.ipspace.net/2012/05/ipv6-only-data-center-built-by-tore.html

http://www.slideshare.net/yuyarin/janog37-ltcedecnet2015-en-57359924