Deploying 5G and LTE for Enterprise and Iot Last Mile David Mindel @Davemindel BRKSPM-2129

#CLUS Deploying 5G and LTE for Enterprise and IoT Last Mile David Mindel @davemindel BRKSPM-2129

#CLUS Cisco Webex Teams

Questions? Use Cisco Webex Teams to chat with the speaker after the session SPEAKER 1

How SPEAKER 2 1 Find this session in the Cisco Live Mobile App WEBEX TEAMS 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space DOCUMENTS 4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKSPM-2129 by the speaker until June 16, 2019.

• What are 4G and 5G Cellular Technologies?

• How Can 4G and 5G Help My Organization? • When and Where Are 4G and 5G Available? • Public and Private Networks, 5G Slices • Last Mile Access - Primary, Backup, High Availability, SDWAN • Out-of-Band Management • Use Cases (Single-Tenant, Multi-Tenant, Multi-Service, QoS, Security) • Provisioning, Programmability and Analytics

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Configuration & Troubleshooting

LTE = Long Term Evolution 3GPP - 3G/4G/5G Standards 4G = 4th Generation Cellular IETF - Internet, Virtualization Standards, etc. 5G = 5th Generation Cellular ITU - Intl Connectivity, Radio Spectrum, etc.

700 700 600 600 500 500 400 400 300 300 200 200

Milliseconds 100 100 Megabits/Second 0 0

Typical User Delay Perception in Ms Typical Per-User Throughput in Mbps

• Next Gen of Cellular Wireless: Same standards bodies, LTE Compatibility varies by SP

• 10x faster, 1/10th the Latency: Different modulation, bands, frequencies & distributed arch

• An architecture not a Frequency: Fundamentally different SP fronthaul, backhaul, core

• Improves upon 4G QoS w/Slices: Ability to condition traffic by flow, user, group, slice

• Allows for Private Networks, Plus: Public networks, with a single SIM, just like 4G

• Provides Ultra Low Latency Plus: Different egress points per slice & multiple slices per UE

• Can Combine Multiple bands: 4G, CBRS, ISM, NR (sub-6GHz, 24+GHz aka mmWave)

• NR efficiency: No guard bands, flexible framing, massive MIMO, beam forming, encoding (25%)

• Last mile access: Cellular wireless is approaching the capabilities of wireline access • Increasing use cases due to improved speed, latency, availability: 3G 4G 5G

Network Enablers: * Multiple RATs (NR, Sub-6GHz, LTE) * * Cloud-scale N/W (V-RAN, SD-WAN) * * CUPS * MEC * NW Slicing *

Multi-Gigabit Fixed Wireless Access Enhanced Mobile Broadband Low latency, High spectral efficiency/T-put eMBB Low latency, High spectral efficiency/Throughput 1ms | 5 b/s/Hz R15 Massive Machine Type Communications High density and link budget Ultra Reliable Low Latency Communications Low complexity, long battery life Low latency, Low PER

uRLLC mMTC 1ms | 10^-8 PER 10 yr Batt | 106 R15 conn./Km2 R15+

All major elements of 5G cannot be provided by a single frequency band or technology

Enterprise Charging and Policy Mobility Service Functions Control Plane Authentication and Security UPF Legal Intercept WAN Controller vRAN vCore Service Functions

UPF Fronthaul Backhaul WAN Edge Computing Internet MVNO

Service Functions

5G UPF Connected Commerce

CUPS: Control/User Plane Separation Cloud Scale Networking with Virtualized/Cloud RAN/Core and SD-WAN

IoT Core Network Control Sub DB NR Netflix Hulu Managed Video Amazon HBO GO Mobility Control Plane Sub-6GHz MBB Core Network Network Slice Service Function (Crossbar Switch) Public Cloud Apps ISM, BBRS Mobile User Plane/ Backhaul SDN Forwarder Internet LTE Caching URLLC Network MultiCast UniCast

Mobile network to scale with video using MEC Network Slicing

C-RAN

UE EPC CPRI RAN BackHaul

BBU Stack

D-RAN BBU CSR BBU Hotel Router Aggregation

Architecture UE RU Central Datacenter (PGW) IP/MPLS Core eNodeB Internet 4G 4G D-RANC-RAN&

vEPC vEPC xHaul vEPC Control plane(NG NSO V-RAN User Plane User plane Core) WAE Analytics/ Telemetry TRP/RU/RRH SR SR PCE eCPRI Fronthaul Midhaul Backhaul UE FH Access UPF Aggregation IP Core FH Agg. Central Datacenter xRAN gNB-CU gNB-CU gNB-DU CU 5G Micro mmW TRP gNB-DU MEC Internet LTE Datacenter 5G Cloud RAN 5G Cloud Architecture Small Cell Internet

E2E Network Slicing

Radio Access Frequency Antenna Building Max MIMO Carrier Channel Download Technology range Max Penetra- Modu- Antennas Aggregation Bandwidth /Upload Length tion lation in CPE * In CPE ** *** Sub-3GHz 700MHz- 125 ft at Good 256QAM 4x4 Up to 16 5-20MHz 100-500/ 2.6GHz < 2GHz sub-2GHz usually <=4 10-100m

ISM 2.4GHz 2.4GHz 75 ft Fair 256QAM 4x4 Up to 16 10-40MHz 50m-1g/ usually <=4 Shared 10-200m CBRS 3.5GHz- 50 ft Limited 256QAM 4x4 Up to 7 5-20MHz 10-300m/ 3.7 GHz usually <=4 Shared 5-50m ISM 5GHz 5.2GHz- 25 ft Poor 256QAM 4x4 Up to 16 20-100MHz 50m-1g/ 5.8GHz usually <=4 Shared 10-200m Mm Wave 24GHz- < 2 ft Very Poor 256QAM 4x4 Up to 16 400MHz 0.5g-10g/ (5G Only) 28GHz usually <=4 0.1g-5g

* As seen in currently available CPE, may change with new devices ** Carrier Aggregation: Contiguous component carriers shown, non-contiguous CCs possible *** Varies widely based on signal strength and quality, and contention Antennas and cables - different for sub-3GHz, CBRS, ISM, and NR. Deployment considerations

#CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 USER Equipment Max L1 DATA Rate Max Number Of DL Max L1 Data Rate 3GPP Category Downlink Mbit/s MIMO Layers Uplink Mbit/s Release

LTE Categories NB1 (LTE-A Pro) 0.68 1 1.0 Rel 13 M1 (LTE-A Pro) 1.0 1 1.0 0 (LTE) 1.0 1 1.0 Rel 12 LTE specifications are created by 3GPP 1 (LTE) 10.3 1 5.2 Open standards body 2 (LTE) 51.0 2 25.5 3 (LTE) 102.0 2 51.0 Rel 8 As improvements are made, 4 (LTE) 150.8 2 51.0 5 (LTE) 299.6 4 75.4 New releases of the spec are published 6 (LTE) 301.5 2 or 4 51.0 Improvements include: 7 (LTE-A) 301.5 2 or 4 102.0 Rel 10 8 (LTE-A) 2,998.6 8 1,497.8 Support for newer modulation 9 (LTE-A) 452.2 2 or 4 51.0 10 (LTE-A) 452.2 2 or 4 102.0 Using multiple carriers (CA), etc. Rel 11 11 (LTE-A) 603.0 2 or 4 51.0 Different LTE SPs support different 12 (LTE-A) 603.0 2 or 4 102.0 13 (LTE-A Pro) 391.7 2 or 4 150.8 Subsets of LTE categories 14 (LTE-A Pro) 3,917 8 9,585 Rel 12 15 (LTE-A Pro) 750 2 or 4 226 16 (LTE-A Pro) 979 2 or 4 n/a https://en.wikipedia.org/wiki/E-UTRA 17 (LTE-A Pro) 25,065 8 n/a 18 (LTE-A Pro) 1174 2 or 4 or 8 211 Rel 13/14 19 (LTE-A Pro) 1566 2 or 4 or 8 n/a 20 (LTE-A Pro) 2000 2 or 4 or 8 315 Rel 14 5G NR 20,000 2 or 4 or 8 10,000 Rel 15 #CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 4G/5G Carrier Aggregation (CA)

• CA allows up to 2x downstream bandwidth on demand

• Example: A 2nd carrier is allocated for receiving data single LTE SP, up to 32/16 radio frequencies

• Supported on Cat 4 and above, depending on SP network

• No UE (ISR) configuration required, no subscription changes

• DL: Cat4 <150 Mbps, Cat6 <300 Mbps, Cat12 <600Mbps, Cat18 1.2Gbps

• Available on ISR 1000, 4000 series (NIM-LTEA-EA) and dual radio 829

Radio S1u S5 eNodeB SGW PGW

RF Carrier 1 Public or UE RF Carrier 2 … Private RF Carrier A … Network Tunnel Tunnel Bearer

Is there CPE? • CAT18 available in commercial CPE (LTE Advanced Pro/Gigabit LTE) • 5G mobile devices (phones, MiFi) available now • 5G chipset/modem availability for enterprise devices: 2H 2019  Initially Sub-6GHz w/future NR board, or 5G SA  Later, a complete combined module, NSA

3GPPl Release 8 3GPP Release 10/11/12 3GPP Release 13 3GPP Release 15 3GPP Release 16 CAT3/CAT4 CAT6/CAT11/CAT12 (CA) CAT18/19/20 (4X4 mimo Sub 6GHz and NR 52GHz+, < power, 2011-Present 2015-Present 256 qam) 2018-Present 24-52GHz, Present Enhanced CA, IAB

• What are 4G and 5G Cellular Technologies?

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Configuration & Troubleshooting

Broadband Wireline

Deployment in Weeks Deployment in Months Lower Cost/Bit Higher Cost/Bit Fairly Consistent Perf. Highly Consistent Perf. Public Network Private Network Static IP Addresses Available No SLA Multi-Mbps Strong SLA

4G: Low SLA 5G: SLAs Expected

Deployment in Days Lowest Cost/Bit (consider usage) Performance Varies Public or Private Network 4G/5G

Apps & Remote, Temporary Services or Mobile Site 4G/LTE Wireless

Public or Private LTE Public or Private Network Wireline Network

Metro-E, Leased, ATM, Frame Private Network

VPN over Public Internet

• Provisioning of a site is needed quicker than can be accommodated by wireline access • Private wireline access (leased line, ATM/FR or Metro-E) is not available or costly • Broadband wireline Internet access is not available, or costly • A backup or parallel primary WAN connection with physical diversity is desired

• Branch (backup), billboard or kiosk (primary), ATMs or substations (primary or backup), etc. • Temporary or movable sites (sporting events, construction sites, defense, first responders) • Network on-the-move: Trains, buses, trucks, trailers, first responder vehicles, backhoes

• Flexible & secure: SPs offering public (Internet) and private network options Proven integration with wireline private networks Packets encrypted over the air (e.g. AES128) • Resilient: Diverse physical path (back hoe can disrupt Metro-E and broadband) Especially important in times of natural disaster • Low monthly cost: With high performance (data usage is a consideration) • Business priority: QoS via separate bearers over 1 LTE connection (single IP address) • Manage/Debug: Strong support for site surveys and FCAPS • Last mile access: Cellular wireless is evolving, closer to capabilities of wireline access More use cases due to speed, latency, availability: 2G 3G 4G 5G

• Immediate deployment option Disasters, temporary sites, transportation, wireline scarcity

• Out-of-Band access Replace POTS/modem, lower cost, higher performance

• SMS Out-of-band management We have an app for that

• GPS location and geofencing We have an app for that too

• Clock sync for closed systems Yes, another app for that

• LTE data usage tracking/alerting Ditto

LTE? Almost Everywhere 5G sub-6GHz? NFL cities + 5G mmWave? Varies by SP

5G mmWave…even if it’s there, can you get it? • Different radios/modems for sub-6GHz, mmWave? • Different antennas for sub-6GHz, mmWave? • Antenna placement for mmWave? • Cable length between mmWave antenna & modem? • Router placed on rooftop with/near antenna? • How to set antenna for best reception for all frequencies with router best situated/secured?

• Not At All • Backup • Primary • SDWAN Active

• What are 4G and 5G Cellular Technologies?

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Configuration & Troubleshooting

PGW 3G/4G SP Public Public Internet Public Wireless Services Public Connection

PGW Private MPLS

3G/4G HANeMo SP Private Private Wireless Services or.. Private Connection ATM/FR/TDM or.. Internet - IPsec VPN

Characteristic Public LTE Connection Private LTE Connection Remote Site Connection CPE tunnels (NAT not easily bidirectional) CPE or network-based (NeMo/PMIP) tunnels LTE WAN IP addressing Public, or RFC1918/NAT’d chosen by SP Private, chosen by enterprise Enterprise Routing Tunnel-based (e.g. OSPF inside tunnel) Tunnel based or BGP peering with SP VPN Requires CPE-based head-ends CPE-based or network-based Exposure to Internet Yes, the degree varies by SP No, unless enterprise network provides access Encryption CPE-based via IPsec CPE-based via IPsec Multi-Carrier Yes (an Internet connection) Yes (SP roaming agreements)

4G/5G PGW/UPF SDWAN, DMVPN, etc. CPE- Private Based VPN Head End Connection . Tunnel Private Network HA/LMA 4G/5G PGW/UPF Network- NeMo/PMIP BGP Based Customer Edge Private Tunnel Connection

Characteristic CPEBased VPN Network Based VPN Remote Site Connection CPE Tunnels (NAT not easily bidirectional) Network-based Tunnels (NeMo/PMIP) LTE WAN IP addressing Private, chosen by enterprise Private, chosen by enterprise Enterprise Routing Tunnel-based (e.g. OSPF inside tunnel) BGP peering with SP Capex Requires CPE-based head-ends SP Network-based (no capex) Encryption CPE-based (many options, end-to-end) Session, Network or CPE-based(GETVPN)

N/W-based tunnels: NeMo supports IPv4, PMIP supports IP v4v6, check with your SP for offerings CPE Tunnels: SDWAN often used, VPN tunnels over public and private network for consistency #CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 4G/5G + SDWAN - Considerations

SDWAN

Transport Intelligent Application Security, Independent Path Control Performance Segmentation Provider Flexibility Load Balancing Application Visibility Strong Encryption Modular Design Policy-Based Path Selection Application-Aware Threat Defense Common Operational Model Network Availability Intelligent Caching DNS/Content/Email Security

SDWAN is a solution to take advantage of multiple public/private WAN connections to a site, with a consistent deployment model

Determines the best path(s) per application type automatically or via policy, and fails over/falls back automatically

SDWAN can use 4G/5G public or private networks and is available for ISR, Meraki MX, and vEdge devices

Check with the SP regarding control plane traffic guidelines (may affect probing, polling, tunnel keepalives, routing protocol hellos)

Data usage is impacted by both user and control plane packets (probes, routing protocol, policy push, IPS signatures, ACLs …)

SDWAN is generally used without network-based VPN, as a CPE-based overlay tunnel is already present #CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 5G Network Slice

• What is it? A logical network with specific network capabilities and characteristics • Can include virtual or real, shared or dedicated 5G NFs (network functions), including RAN access • A 5G device can connect to multiple slices (different interface/IP address per slice) • Can support a single client or be multi-tenant. Can be created statically or dynamically • Why is it useful? Network characteristics, SLAs, segmentation in an efficient manner • Example: Low latency high throughput local egress for regional video surveillance • Example: Segmented, low usage, low performance high reliability for fire monitoring/alarm system • Example: Ultra-low latency for vehicle-to-vehicle communications

MIoT

eMBB

• What are 4G and 5G Cellular Technologies?

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Configuration & Troubleshooting

No backup - wireless primary access with LTE

Remote Main Branch 4G/5G SP Enterprise Wireline Network Site Mobile IP Services

Connection backup - Wireline primary, LTE backup Remote Main Site Branch 4G/5G SP Enterprise Wireline Network Mobile IP Services

Router and connection backup - higher availability can be achieved Branch Main Site Backup 4G SP Enterprise Wireline Network Mobile IP Services

Branch Primary 5G SP Enterprise Mobile IP Services

Put antennas => 2 feet apart PoE splitter SKU: PWR-UPOE-I-SPL 4G/5G Last Mile Access ISR 1101 Floor 3 Ethernet + PoE No expensive antenna cabling Increases availability, WAN + ISR Backup router is full function Out-of-Band Floor 2 Management Not a “modem” or limited router Bonus: out-of-band management

Ethernet Wireline Last Mile Access Floor 1 Cisco Internal Use 29

UE 4G/5G Last Mile Access 4G/5G Floor 3 External Interface 5G mm Wave – Antenna Distance UE – Physically separated but logically embedded …“External I/O” Floor 2 Router – Uses both WAN links based on

Ethernet Ethernet + PoE policy, performance, availability

Ethernet Wireline Last Mile Access Floor 1 Cisco Internal Use 30

• 2 options for 4G/5G high availability: • Multi-radio • Single-radio dual-SIM (dual carrier) • Multi Radio: Simultaneous send/receive on multiple LTE connections • Modular ISRs & IR829M-2LTE-EA support multiple modules/radios (a subscription/SIM for each) • Single radio/dual SIM: One LTE connection at a time, one carrier or the other • Provides for LTE high availability at a lower cost, with smaller footprint • Use product data sheets to determine the right one for your geography and carrier • Newer modems (e.g. MNA & EA) can operate in multiple carrier networks by loading different modem firmware Carrier A RAN Single radio dual-SIM Multi-radio multi-SIM

Carrier B RAN

Operator A

Y Operator B

Attach 1min Initiate FW upgrade Loss of A’s Detect 1.5 mins 10 mins 1.5 mins Modem reset with Failover Manual LTE Signal failover and attach Operator A To operator B Or EEM

1.5 mins New feature: ~15 Auto-SIM Switchover Firmware Switching and a.k.a Dynamic Carrier Cutover mins Modem reset 10X improvement over manual

Dying GASP Support: NIM-LTEA-XX, ISR 11xx Mobile Service C4321-4G#sh run | s controller Provider controller Cellular 0/1/0 … lte dyinggasp detach enable

lte dyinggasp sms send 2125559999 …….4321LTEA in Horsham lab lost power

… SMS to admin

• What are 4G and 5G Cellular Technologies?

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Configuration & Troubleshooting

POTS/Modem Challenges Cisco LTE Solution Benefits

• • Older modem technology/equipment • OOB options include an integrated LTE: Lower cost + higher means more issues & truck rolls module in ISR (LTE NIM) or 8xx/11xx performance • • POTS line monthly cost • Faster console response & downloads Faster device image downloads • • Back hoe can take out landline + POTS • Access Lists increase security Faster log/debugging uploads • • Dialup performance limits use as a • SMS can be used for LTE OOB mgmt Stronger Security (OOB over VPN) backup to WAN link or transferring • Ubiquitous access (SMS and IP) images & logs • Manage from any site, w/IP or without • Do admins still have a POTS line?

BEFORE AFTER HQ HQ

POTS WAN 4G/5G WAN Network OOB Management only OOB Management

Remote Location Remote Location

Out-of-Band Access WAN/Router Backup Business Resiliency

• Faster image/log download/upload • LTE router running router redundancy • Single low cost addition protocol, with primary router (e.g. • Access from IP and SMS significantly increases site availability HSRP) • • Stronger security Diverse path to backup (wireless) • If primary router loses WAN connection • Out-of-band access to primary router or becomes inoperative, LTE router via separate network & device. Better automatically becomes primary chance of remote recovery of primary • Primary recovers, LTE returns to backup

Combining BEFORE AFTER 2 Solutions HQ HQ With LTE

POTS WAN 4G/5G WAN Network

OOB Management only OOB Management + WAN backup

Remote Location Remote Location

8xx for out-of-band + WAN backup Primary WAN Router/Link - Any ISR

RJ45-RJ45 Rollover Cable - CAB-CON-C4K-RJ45= Smart serial to RJ45 cable – CAB-SS-RJ45=

819 1xxx/42xx/43xx/44xx /19xx/29xx/39xx ISRs Router with LTE replaces modem & phone line Any Cisco router with an aux and console port is supported LTE replaces POTS service for out-of-band access to primary Other routers with a console port and serial port should work LTE router also provides backup WAN/router function Primary router AUX (or serial) port connects to 819 console port Login to router via LTE, then console access to primary router (console access to 819 via primary router w/normal operations) Primary router debug if WAN down or router in diag mode Primary router console port connects to 819 serial port (for Can use the “smart serial to RJ45” CAB-SS-RJ45 console access to primary router via 819 if primary WAN down)

• What are 4G and 5G Cellular Technologies?

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Configuration & Troubleshooting

Private Network SP Services/Backhaul 10.1.1.0/24  10.1.2.0/24  PGW/UPF Org “A” … 10.1.1.0/24 4G/5G Org “A” VRF Private HA/LMA Network Org “A” DC NeMo/PMIP BGP Org “A” Site

10.1.2.0/24 Network-Based Tunnel Org “A” Site • Classic IPv4v6 Routing (not SDWAN) • End-to-end reachability • Without 4G/5G router running RIP, OSPF, EIGRP, BGP • Dynamic adding, changing, removing of subnets The example represents a network- • Org “A” has fully control of IP address space based tunnel/routing protocol across • No head-end needed by Org “A” to terminate tunnels a single 4G/5G connection, such as • Very high scalability NeMo (Network Mobility) or • SP provides tunnel termination as part of service PMIP (Proxy Mobile IP) • BGP peering between SP service and Org “A” network • End-to-end encryption via GETVPN or client-to-server

Public or Private Network, SDWAN, MPLS VPN, SRv6 Remote 4G/5G 4G/5G Services Branch Customer Cust A 1 Site Finance EPS Bearer Finance

Sales Sales

Single Underlay Leverages any transport, including 4G and 5G SDWAN segments to the user/policy level Supported via VXLAN, SGTs, VRFs, or VPN tunnels for multi-tenancy across an LTE connection The example represents prem-based Separation between entities, groups, etc. SDWAN , enabled by VPN tunnels, VXLAN, from remote site to data center(s) SGTs , segment routing, etc. across a Policy-based path selection (user, group, app, perf) single 4G/5G connection Compatible with LTE private network services

Private Network, Agency 1 Cust #1 Multi- 4G/5G 4G/5G Services SDWAN, MPLS VPN VRF Tenant Customer DMV Site 1 Site EPS Bearer OR DEP Agency 2 VRF Cust #2 Customer Separate VRFs 2 Site Leverages any transport, including 4G/5G APN/DNN, NeMo/PMIP, VRFs, VPNs to provide multi-tenancy across an 4G/5G connection The example represents SP-offered Complete separation between entities network-based VPN multi-VRF services, from remote site to customer data center(s) e.g. NeMo, PMIP, MPLS VPN Can support overlapping IP addresses Can also be supported with Compatible with 4G/5G private network services prem-based VPN (SGT, SDA) Check with the Service Provider for available services

Agency 1 Cust #1 Malls, 4G/5G 4G/5G Services VRF Offices, DMV Buildings Public OR Segmentation Internet DEP N/W Slices, VRFs, Agency 2 EPS Bearers VRF Cust #2

Leverages any transport, including 4G and 5G VRFs provide multi-tenancy across an LTE connection Complete separation between entities The example represents SP-offered multi- from remote site to customer data center(s) VRF services via CPE Can support overlapping IP addresses Can be augmented with 5G network slicing Compatible with LTE private network services for enhanced QoS, and security via private Check with the Service Provider for available services connection to public cloud services

Security Surveillance 4G/5G 4G/5G Services Public Internet Network ATM

EPS Bearer Video/Ad Svc Video

Cust. Care Ctr Cash Machine Customer Banking Network Separate, Isolated VRFs Isolated Separate, Care Line Separate VPNs

-Allows multiple services to be provided to one site -Applicable to many 4G/5G site types (retail, kiosk, office…) Public Network -Scalable, supports IP address overlap & strong security Single Tenant - Complimentary to existing services (no interference) - Each application service in control of its addressing Example shows a VPN per VRF -Examples: Can support multiple VRFs -ATM, retail kiosk, vending machines, branch offices across a single VPN tunnel -Vehicles, trucks, trains, mobile setups/shows

DC, Branch, Temporary or Mobile Site w/Services 4G/5G Wireless Public 4G/5G Web Clients Single IP Address/UE Management Systems Web Servers 1xx.0.0.9 Public Network Security Ops Centers IoT sensors IoT Controllers IP Cameras Internet via Metro E ID Devices Internet via Cable

Internet via TDM/Optical

• Single IP address provided to router cellular interface Public Network • Cannot publicly advertise subnets behind 4G/5G router (as w/wireline broadband) Single Tenant • Disallows use of 4G/5G as last mile for some key public network applications – Prem-based Web services: Not feasible to do port-based static NAT for access from Internet browsers/clients – IoT sensors that are polled: Not feasible to do port-based static mapping per sensor – Surveillance: Not feasible to do port-based static mapping per camera, to reach camera directly from Ops Center

Public APN Single @ 4G/5G PGW BGP NAT NAT 1xx.0.0.0/16  1xx.1.0.0/16  Web Clients Management Systems Subnet PGW LMA … 1xx.1.1.0/28 Single @ Public Network Security Ops Centers 1xx.0.0.9 4G/5G 1xx.1.1.2 BGP IoT Controllers 1xx.1.1.3 PMIP Web Servers 1xx.1.1.4 IoT sensors Network-Based Tunnel 1xx.1.1.5 IP Cameras Secure Public APN Public Network ID Devices Single Tenant

• A method to support Internet routing to IPv4 and IPv6 subnets behind a 4G/5G router • PMIP protocol, used with private network, can be used with public network • PMIP is a GRE-based tunneling protocol between a MAG (UE router) and LMA (mobility anchor) • Through the tunnel, the UE router advertises subnets behind it, and installs outbound routes • Normally the routes learned by the LMA are shared with a private enterprise network via BGP peering • This example uses PMIP with a public APN/DNN, and via BGP shares those routes to “the Internet”

45 #CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 4G: Access Point Name (APN) – 5G: Data N/W Name (DNN) • APN allows a connection to a specific PDN (Packet Data Network)

• Set by LTE modem firmware by default. Can be overwritten by OTADM

• Set automatically by the network, otherwise manually via IOS enable-mode command • 819 VZW Example: cell 0 lte profile create 3 ne01.vzwstatic none

• Verizon public APNs include: vzwinternet, ne01.vzwstatic, so01.vzwstatic, mw01.vzwstatic, we01.vzwstatic

• AT&T APNs include: i2gold, Nextgenphone, Broadband

• T-Mobile APNs: include fast.t-mobile.com, epc.tmobile.com

• APNs for private network are generally unique. Always get the appropriate APN value from the SP

Org W APN W Org W Org W PGW APN X Public or Private Org X Org X 4G/5G Carrier “X” Network(s) Radio Access 4G/5G Carrier “X” Org Y Network (RAN) Evolved Packet Core Org Y Org Z APN Y Org Z PGW Org Z APN Z

Private Network Services Remote 4G/5G 4G/5G Services Private Branch Network EPS Bearer PGW

SGW EPS Bearer PGW Internet 2 Access Point Names (APNs) 2 Packet Data Networks (PDNs) Public Cloud Services 4G/5G allows for multiple virtual interfaces across one connection/subscription Offloads Internet access from private network with less hardware/network $ Improves Internet access - shorter more efficient route (not via private Network/DC/FW/GW) Each PDN is an interface, one to public network, another to a private network Each has separate IP address (i.e. cell 0 and cell 1, cell 0/2/0 and cell 0/2/1) Pros: Efficient, powerful and flexible Considerations: Firewall/IPS, segmentation, QoS

5G Access 5G Services Private Networks Voice & Data Remote Site IoT Lighting

IoT HVAC Surveillance

IoT Fire Separate, Isolated VRFs Isolated Separate, Separate 5G Slices

- Allows multiple services to be provided to one site - Applicable to many 5G site types (retail, kiosk, office…) -Scalable, supports IP address overlap & strong security Example shows a 5G environment - Complimentary to existing services (no interference) with separate slices per function for - Each application service in control of its addressing segmentation & SLA. Appear as - Separate SLAs, network characteristics per slice separate interfaces on CPE

Video Guest WiFi Surveillance Console Video Remote LTE SP Services Private Network Surveillance Branch Main Site

Audio M2M & Enterprise Svcs Ads/News Credit card Display Transactions CC Transaction Private IP Services DSCP/QCI Markings Address NeMo or VPN Enterprise NNI DSCP/EXP Markings Tunnel

• Provide preferential treatment for applications • Integrates 4G/5G QoS with enterprise QoS Why QoS with 4G/5G? • QoS from remotes across LTE to private network • Cost-effective primary WAN option with rich services for branch, substation, ATM, Kiosk • Leverages 3GPP standards (carriers offering now) • Use case example: banks require priority for • Works with NeMo, PMIP & DMVPN, n/w & prem based ATM transactions & video surveillance over guest WiFi • SDN allows per-tunnel QoS over 4G/5G • Quick turn-up

• UE--> eNB radio channel establishment • PGW gets policy rule from PCRF for default bearer

• eNB sends the UE attach request to MME • PCRF validates UE’s service profile with HSS

• UE authenticates • PCRF installs policy rules in PGW for the bearer(s)

• MME requests info & SGW & PGW selection for the UE • Bearer set up PGW-SGW, SGW-MME

• MME--> SGW & SGW-->PGW to set-up default bearer • MME advises eNB about the bearer(s) set up

• PGW gets addtl user parms & validation from AAA • eNB sets up bearer(s) over the air with the UE Multiple Bearers share the same IP address

MME AAA HSS PCRF eNB

S6b Gx

• Bearer: an end-to-end logical channel between the LTE device (UE) and LTE anchor point (PGW)

• Bearer types: best effort, dedicated non-guaranteed bandwidth, dedicated guaranteed bandwidth

• What is “set” for QoS between the eNodeB (cell tower) and UE (LTE device)? QCI value per bearer

• QCI: QoS Class Identifier: A mechanism to provide a bearer differentiated packet forwarding treatment

• Dedicate GBR bearer for VoLTE will use a different QCI than best effort default bearer for Internet. Example:

Resource Priority Packet Delay Packet Error Example Services QCI Type Budget Loss Rate 1 GBR 2 100ms 10−2 Conversational Voice 2 GBR 4 150ms 10−3 Conversational Video (Live Streaming) 3 GBR 3 50ms 10−3 Real Time Gaming, V2X messages 4 GBR 5 300ms 10−6 Non-Conversational Video (Buffered Streaming) 5 non-GBR 1 100ms 10−6 IMS Signaling 6 non-GBR 6 300ms 10−6 Video (Buffered Streaming) TCP-Based (for example, www, email, chat, ftp, p2p, etc.) 7 non-GBR 7 100ms 10−3 Voice, Video (Live Streaming), Interactive Gaming 8 non-GBR 8 300ms 10−6 Video (Buffered Streaming) TCP-Based (for example, www, email, chat, ftp, p2p, etc.) 9 non-GBR 9 300ms 10−6 Video (Buffered Streaming) TCP-Based (for example, www, email, chat, ftp, p2p, etc.)

Expedited Forwarding Priority Priority Bearer • Some LTE carriers offer QoS on LTE VoIP Packets EF Time-sensitive applications (VoIP) • Usually tied to LTE Private Network Services CS5

• LTE QoS, a moderately priced add-on per line Assured Forwarding Priority Priority Data CS4/AF4x Priority Bearer Business mission critical data • Set choices for priority data bit rates flows including video and CS3/AF3x multi-media streams • QoS based on DSCP values, easy configuration Low Forwarding Priority • DSCP-based TFT: Integration w/enterprise QoS Best Effort Best Effort Data Best Effort Default Bearer General business data flows • Priority Data classes policed above contract rate

Modem TFT (Traffic Flow Radio S1u S5 Template) eNodeB SGW PGW DSCP 0, CS1, AF11, DSCP 0, CS1, AF11, AF12, AF13, CS2, AF12, AF13, CS2, AF21, AF22, AF23 Default Bearer Default Bearer Default Bearer AF21, AF22, AF23 DSCP CS3, AF31, DSCP CS3, AF31, AF32, AF33, CS4, Dedicated Bearer/QCI 7 Dedicated Bearer Dedicated Bearer AF32, AF33, CS4, AF41, AF42, AF43 AF41, AF42, AF43

DSCP EF, CS5 DSCP EF, CS5 Dedicated Bearer/QCI 1 Dedicated Bearer Dedicated Bearer

• 5G QoS is not based on bearers, but on flows

• Multiple flows can be in a radio bearer channel or PDU session

• Packets are classified with a QoS Flow ID (QFI)

• Each QoS FIow ID (QFI) instance has associated with it: 5QI (there are standard 5QIs, with set behaviors akin to 4G QCIs) Resource Type (GBR, delay critical GBR or non-GBR) Priority level Packet Delay Budget Packet Error Rate Averaging Window Maximum Data Burst Volume For GBR: guaranteed bit rate, max bit rate, max loss rate (UL & DL) For Non-GBR: Reflective QoS Attribute

• Classifying packets…TFT in 4G, Packet Filter in 5G

• How is traffic classified to be put into a bearer?

• When eNB sets up bearers with UE, supplies UE with a Traffic Flow Template (TFT) rec’d from MME

• TFT specifies how traffic is classified into each bearer (by 5-tuple, by DSCP, etc.)

• ISR maps to the TFT via configuration via traffic classes ( IOS class-map)

• ISR QoS policy maps a traffic class to a particular bearer (e.g. bandwidth class to dedicated NGBR bearer)

• If SP polices the bearer (max b/w limit), ISR can shape/remark that traffic class to not exceed the SLA

• Consider how UE & PGW count overhead!

QoS Policy TFT MME ISR IOS/ISR ISR LTE Port eNB

Default BE bearer & IOS Class Def Per-Bearer SGW PGW IP Network Dedicated NGBR bearer Class treatment Policy With QoS Class B1 = prec 3,4,5 + Ded NGBR B1 = QCI7 = or without Guar BW/shape=2mbps prec 3,4,5 + SLA/policer=2mbps #CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 54 SPs Offering LTE QoS - Examples

• Verizon Wireless: Offers Private Network Traffic Management via a dedicated bearer Offering in market for enterprises since 2015 Traffic receiving special treatment is determined by DSCP value Optional feature of LTE Private Network offering

• Telstra: Offers “LANES”, LTE Advanced Network for Emergency Services Planning on extended the offering to enterprise Traffic receiving special treatment is determined by DSCP value Optional feature of LTE Private Network offering

• AT&T: Differentiated serviced based on APN

• T-Mobile: Differentiated serviced based on APN

* SP services are constantly evolving. Check with your LTE carrier for latest offerings

Priority Dedicated Non-GBR

Example: IP voice strict Shape Bearer

<= V Critical Mission priority <= V bps, very Traffic important traffic => X bps, Very Important Total 4G Link Bandwidth, key business apps => Y >= X to bps. Together <= S bps S Important (SP SLA) >= Y => 25% Total Best Best Effort Bandwidth Example: Key business General Remaining Effort Best Default Bearer apps receive “better Traffic service” than general Non-Business business apps, which receive “better service” T than non-business apps Scavenger

• Traffic on dedicated bearer shaped to its service rate, reduced dropped packets T = total rate S = service rate • Full upstream bandwidth used. Default bearer not policed V = voice rate • If the bandwidth allotted to a traffic class is unused, it can be used by other classes S = V +X +Y • LTE QoS planning/configuration guide available: www.cisco.com/c/dam/en/us/td/docs/routers/access/interfaces/software/deployment/guide/mpn-lte-qos-config.pdf

Important Apps, Voice Lower Upstream Bandwidth Prioritization, Queueing Much Higher Bandwidth

LTE Backhaul Internet Cell Tower VPN Hub / Datacenter

Bandwidth Constantly Changing, but the need for QoS does not Last mile uplink speeds are slower  Avoids dropping voice/key data Accelerated encryption & forwarding  Lowers Latency/Jitter Adapt to changing upstream bandwidth  Prioritizes VoIP & Key Apps

With Best Effort LTE, no need to shape upstream. Network does not police by class

Video Console Video Guest WiFi Surveillance SP Services Remote LTE Surveillance Branch Main Site RAN Best Effort Internet Service Voice Best Effort Service

Ads Display Credit card Transactions CC Transaction Services DSCP DSCP VPN Tunnel VPN Tunnel Maintains Markings Markings Inside DSCP Markings

• Same considerations/benefits as without VPN • Integrates LTE QoS w/enterprise QoS (end-to-end DSCP) Why QoS with Best Effort 4G? • DSCP bits copied/promoted to VPN header ToS byte • Prioritizes across the slowest link (LTE upstream) • LTE network can’t change original DSCP (inside tunnel) • Same benefits as seen on asymmetric broadband • Anti-Replay ESP AH Considerations (see notes) • Well documented use case (Cisco Virtual Office)

ISR LTE Network Example: IP voice strict Priority Guarantee at at Least

<= V Critical Mission

priority <= V bps, very Traffic important traffic => X bps, Very Important Total 4G Link Bandwidth, key business apps => Y >= X Best Best Effort S

bps. Together <= T bps Default Important Bearer (75% of total upstream rate) >= Y => 25% Total Bandwidth Example: Key business General Remaining Effort Best

apps receive “better Traffic service” than general Non-Business business apps, which

receive “better service” T than non-business apps Scavenger

T = total rate • No rate limiting needed on traffic out the cell interface to enable prioritization & queueing S = service rate V = voice rate • Business critical and voice traffic recommended to be 75% or less of expected upstream bandwidth S = V +X +Y • QoS Policy dynamically adjusts to congestion and available upstream LTE bandwidth (only engages upon congestion)

• What are 4G and 5G Cellular Technologies?

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Configuration & Troubleshooting

4G LTE Cellular Network

Cisco Plug and Play Connect DNAC or MSX Platform Orchestration Plane Or Meraki Portal Or Meraki Portal

1. New unit from factory (or erase startup-config and reload) • SIM preinstalled at factory or manually inserted before power-up 2. On power up, router PNP process configures LTE w/LTE carrier’s default public LTE profile 3. Router PNP process registers to Cisco Plug & Connect Service over LTE 4. Cisco Device Connect looks up S/N, redirects unit to specified service orchestration platform ISR11XX IR 829 • e.g. enterprise’s DNAC or SP’s MSX. If needed, certs downloaded. ISR 4K 5. Orchestration VPN established to service orchestration platform. WAN template downloaded 6. Additional template(s) downloaded for VPN, SDN, NAT, ACLs, etc. (stacked templates) 7. Establish Data Plane over WAN

Network applications:

• Real-world uses for network and device features can bring to life their value

• Real-world needs met via programmability and function of ISR IOS software Geofencing, Monthly Plan Usage Alerting, Site Survey, GPS time syncs ISR clock, Show/Config via SMS, Auto-return to 4G when drops to 3G, Choose best network while roaming)

• The engine that drives these apps is a standard part of IOS & IOS XE - EEM www.cisco.com/go/eem

• The apps above are small tcl files loaded to ISR flash memory, publicly available at no charge

• These apps can be modified for an organization’s specific needs (open source, TCL) www.cisco.com/c/dam/en/us/td/docs/routers/access/interfaces/software/deployment/guide/lte_access_011414.pdf

Another powerful ISR programmability feature is IOX

• IOX is used to run applications without the need for additional servers/processors

• IOX provides for Linux apps (in containers) to run on ISR/IR dedicated core(s), separate from routing cores

• The “Site Survey via SMS" app is now part of an app allowing any command to be sent to an ISR via text message. It can provide feedback on LTE signal quality and strength, improving antenna placement and selection. When requested via SMS, the ISR responds via text with RF results. There is a short and long request. • The app is summarized in these guides: www.cisco.com/c/dam/en/us/td/docs/routers/access/interfaces/software/deployment/guide/lte_access_011414.pdf www.cisco.com/c/dam/en/us/td/docs/routers/access/interfaces/software/deployment/guide/guide_100413.pdf https://www.cisco.com/c/dam/en/us/td/docs/routers/access/interfaces/software/deployment/guide/guide_isr_lte_sms.pdf

• The tcl script and documentation can be downloaded from: https://community.cisco.com/t5/networking-documents/command-over-sms/ta-p/3340391

• App allows for using text messaging as a method to configure or show ISR status. Any “show” or configuration command can be used. The app accepts multiple commands. Common usage: “show cell 0 all”, “ping x.x.x.x”, “show ip route”, “conf t, int cell 0, no shut, end”.

• This is done by sending a text message to the phone number (MSISDN) associated with the SIM in the ISR. The app allows for white-list (specific phone #s are authorized to send commands to ISR)

• The app is detailed in the LTE Antenna Guide for ISR and CGR, available here: www.cisco.com/en/US/docs/routers/access/interfaces/software/deployment/guide/guide_100413.pdf

• LTE subscriptions provide an amount of monthly traffic as part of the rate plan. This app monitors LTE traffic during a billing month, and logs a message if a defined percentage of the allowed bytes is reached. At the end of the monthly billing cycle, the app clears the LTE counters and begin counting anew. Variables included for tuning. An option to automatically “shut” the cellular interface has been added.

• If the commandoversms app is loaded with this app, texting “usage” will return # of bytes and percentage of rate plan used for the current month

• The app is summarized in this guide, above, available here: www.cisco.com/c/dam/en/us/td/docs/routers/access/interfaces/software/deployment/guide/lte_access_011414.pdf

• The tcl script and documentation can be downloaded from https://supportforums.cisco.com/document/12098671/monitor-byte-count-over-4g-cellular-interfaces

• Automatically restores ISR Radio Access Technology (RAT) to 4G LTE if it has fallen to 3G or 2G

• This can temporarily occur due to weather conditions, interference, cellular network congestion

• Although LTE reselection is automatic, standards dictate that a device’s LTE modem be in idle mode to reselect (up to 15 seconds w/no traffic up or down). Easy for phones, not for routers

• This app can be set to run at any desired frequency and/or time-of-hour/day/week.

• This version supports a single ISR cellular interface. Note that there is a cellular service disruption of ~15-30 seconds if in 2G/3G mode and the app attempts to restore 4G operation. If the conditions still exist causing the modem to be in 2G/3G mode, the app will not force 4G.

• The tcl script can be downloaded from https://supportforums.cisco.com/document/12620516/restore-lte-service

Sep 29 14:30:09.463: %HA_EM-6-LOG: restore_lte.tcl: Service is not LTE (service = @). … … Sep 29 14:30:20.915: %HA_EM-6-LOG: restore_lte.tcl: 3 : Service successfully changed to LTE .... Service = 0 8 checking for Cellular IP address … Sep 29 17:30:03.123: %HA_EM-6-LOG: restore_lte.tcl: Currently on LTE and no changes have been made

• Location-based services are valuable in order to: Determine the physical location of a stationary device Determine location, direction, velocity, location history of a mobile device

• 5 Ways to get location: Passive GPS location via line-of-sight to the sky (GPS satellites) Active GPS location via line-of-sight to the sky (GPS satellites) Location-Based services via cellular tower triangulation (3G/4G SPs) Location services via WiFi AP triangulation and configuration Assisted GPS: location by combining data from GPS + single cell tower location (A-GPS/SUPL) in 7-10 seconds. IOS => 15.6(3)M2 16.7.2

• Various CPE supports one or more location-based Cisco ISRs support Assisted GPS and Active GPS (incl. NMEA stream) With A-GPS, if no line-of-sight, location will be reported within the area of the radius of the nearest cell tower conf t controller cell x lte gps mode standalone (or ms-based)

• This app uses GPS to log, and optionally alert (text) and/or disable LTE when an ISR is operating out of an allowed geographic area. The app has adjustments for the range of the allowed area and actions taken once out of range.

• The app is summarized in the guide available here: www.cisco.com/c/dam/en/us/td/docs/routers/access/interfaces/s oftware/deployment/guide/lte_access_011414.pdf

• The tcl script and documentation can be downloaded at https://supportforums.cisco.com/document/12150821/geo-fence

• What are 4G and 5G Cellular Technologies?

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Configuration & Troubleshooting

1921, 941 2901, 2911, 2921, 2951 3925, 3925E, 3945, 3945E CGR 1240, 1120 1240 11xx, 819, 899 4221, 4321, 4231, 4331, 807, 809, 819H, 4351, 4431, 4451 829

Secure IoT Virtual Secure Customizable Secure Collaboration IoT / Kinetic Connectivity Office Mobility Applications Rich Media Services Performance, Scalability, Availability Scalability, Performance,

Fixed 4G-LTE EHWIC-4G-LTE NIM-LTEA-EA CGM-4G-LTEA

ISR Product SKU’s LTE Modem Version Firmware Version Router OS release EHWIC-4G-LTE-V, C819(H)G-4G-V-K9 MC7750 3.5.10.6 15.5.3M4 EHIWIC-4G-LTE-A, C819(H)G-4G-A-K9, EHWIC-4G-LTE-JP MC7700 3.5.10.6 15.5.3M4 EHWIC-4G-LTE-G, C819(H)G-4G-G-K9 MC7710 3.5.10.6 15.5.3M4 EHWIC-4G-LTE-VZ, EHWIC-4G-LTE-ST, C819G- 4G-VZ-K9, C819G-4G-ST-K9, C899G-LTE-VZ- K9, C899G-LTE-ST-K9, NIM-4G-LTE-VZ, NIM-4G-LTE-ST MC7350 5.5.58.01 Classic IOS 15.6.3M4 or 16.3.2 IOS-XE EHWIC-4G-LTE-AT, C819G-4G-NA-K9, C899- LTE-NA-K9, NIM-4G-LTE-NA,, EHWIC-4G-LTE- CA MC7354 5.5.58.0 Classic IOS 15.6.3M4 or 16.3.2 IOS-XE EHWIC-4G-LTE-GB, EHWIC-4G-LTE-AU, C899- LTE-GA-K9, C89xVAG(M)-LTE-GA, C819G-4G- GA-K9, C88x-G-4G-GA-K9, NIM-4G-LTE-GA MC7304 5.5.58.0 Classic IOS 15.6.3M4 or 16.3.2 IOS-XE EHWIC-LTE-LA, NIM-LTE-LA, C819G-LTE-LA- K9, C897VAG-LTE-LA-K9, C899G-LTE-LA-K9, C819GW-LTE-LA-ZK9, C819GW-LTE-LA-QK9, C819GW-LTE-LA-NK9 MC7430 2.30.01.01 Classic IOS 15.6.3M4 or 16.3.2 IOS-XE NIM-LTEA-EA, NIM-LTEA-LA EM7455, EM7430 2.30.01.01 IOS-XE 16.6.1 C1111-xPLTExA EM7455, EM7430 2.30.01.01 IOS-XE 16.9.1 C1101-4PLTEPWD with P-LTEA-EA EM7455, EM7430 2.30.01.01 IOS-XE 16.9.1 C1109-2P with LTE WP76XX 2.18.05 IOS-XE 16.9.1 C1109-4P with P-LTEA-EA M7430/EM7455 2.30.01.01 IOS-XE 16.9.1

• What are 4G and 5G Cellular Technologies?

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Configuration

#CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 72 Basic Configuration – 819/899 Example Snippet Interface vlan1 ip address 10.1.2.1 255.255.255.0 • LAN interface, with nat and tcp ip nat inside adjust (see below) Ip tcp adjust-mss 1390 chat-script lte "" "AT!CALL" TIMEOUT 20 "OK" • Chat script (the default is ok) interface Cellular0 • Cell interface definition ip address negotiated • Must have NAT or VPN def ip nat outside • If LTE SP supports <=1500B may need to encapsulation slip set tcp adjust to avoid fragmentation ip tcp adjust-mss 1390 • Set pulse-time 1st, then enable-time dialer in-band • Always IP addr negotiated dialer idle-timeout 0 • Dialer-watch group allows always-on use dialer enable-timeout 1 (recommendation for primary and backup dialer string lte use). dialer watch-group 1 • Ip nat inside on LAN interface, ip nat outside pulse-time 0 on cell int, and nat definition, or VPN dialer watch-list 1 ip 5.6.7.8 0.0.0.0 definition that disallows any traffic towards dialer watch-list 1 delay route-check initial 60 LTE with a source IP@ other than that of the dialer watch-list 1 delay connect 1 cell interface ip nat inside source list 100 interface Cellular0 overload access-list 100 permit ip 0.0.0.0 254.255.255.255 any • NAT policy statement st line 3 • “permit” even numbers in 1 octet script dialer lte • Cell interface line definition, specifies modem InOut string to send to LTE network, cannot no exec execute commands and besides data transport input ssh connection, only SSH is permitted in/out transport output ssh

Interface GigabitEthernet0/0/1 ip address 10.1.2.1 255.255.255.0 • LAN interface, with nat and tcp ip nat inside adjust (see below) Ip tcp adjust-mss 1390 • * No Chat script needed *

interface Cellular 0/1/0 • Cell interface definition ip address negotiated • Must have NAT or VPN definition ip nat outside • If LTE SP supports <=1500B may need to encapsulation slip set tcp adjust to avoid fragmentation ip tcp adjust-mss 1390 • Set pulse-time 1st, then enable-time dialer in-band • Always IP addr negotiated dialer idle-timeout 0 • Dialer-watch group allows always-on use dialer enable-timeout 1 (recommendation for primary and backup dialer watch-group 1 use). “Dialer string” unneeded for IOS XE pulse-time 0 • Ip nat inside on LAN interface, ip nat outside on cell int, and nat definition, or VPN dialer watch-list 1 ip 5.6.7.8 0.0.0.0 definition that disallows any traffic towards dialer watch-list 1 delay route-check initial 60 LTE with a source IP@ other than that of the dialer watch-list 1 delay connect 1 cell interface ip nat inside source list 100 interface Cellular0 overload • NAT policy statement access-list 100 permit ip 0.0.0.0 254.255.255.255 any • “permit” even numbers in 1st octet

• No line definition needed for IOS XE

Interface GigabitEthernet0/0/1 ip address 10.1.2.1 255.255.255.0 ip nat inside • LAN interface, with nat and tcp Ip tcp adjust-mss 1390 adjust (see below) interface Cellular 0/1/0 ip address negotiated ip nat outside dialer watch-group 1 • 2 Cell interface definitions … • Single LTE modem but 2 cell interfaces interface Cellular 0/1/1 • Must have NAT or VPN def on each ip address negotiated • If LTE SP supports <=1500B may need to ip mobile router-service roam set tcp adjust to avoid fragmentation ip mobile router-service collocated ccoa-only dialer watch-group 2 … dialer watch-list 1 ip 5.6.7.8 0.0.0.0 dialer watch-list 1 delay route-check initial 60 dialer watch-list 1 delay connect 1 • 2 sets of dialer watch-list stanzas dialer watch-list 2 ip 5.6.7.8 0.0.0.0 • NAT policy statements for cell 0/1/0 dialer watch-list 2 delay route-check initial 60 dialer watch-list 2 delay connect 1 …

Primary WAN Router/Link - Any ISR 819 for out-of-band and WAN backup

RJ45-RJ45 Rollover Cable - CAB-CON-C4K-RJ45=

Smart serial to RJ45 cable – CAB-SS-RJ45= line con 0 interface serial 0 line con 0 login local physical-layer async login local no modem enable no ip address no modem enable encapsulation slip

line 7 transport input telnet

Remote Main Public or Private Branch Carrier A RAN Site Carrier A EPC Network

Carrier B RAN Carrier B EPC

• On ISR 4K series & ENCS (LTE-EA & LTE-LA NIMs) • Optional – Set Failover Time Delay And ISR 1K series (embedded, P-LTE-EA, P-LTE-LA) 4221(config)#controller Cellular 0/0/0 4221(config-controller)#lte failovertimer ? • Connection to 1 network at a time, but access to 2 <3-7> Configuration Timer (mins). Default value 3 mins • Switchover from primary to backup carrier ~ 3 minutes • Optional - Define the primary SIM slot • Switchover is automatic (called auto-SIM switchover) 4221(config)#controller Cellular 0/0/0 conf t, controller cell 0, lte sim primary slot 1 4221(config-controller)# lte sim primary slot 1 IOS will switch to other SIM if no LTE connection after 3 minutes • Once the secondary SIM is active, connection will remain • Behavior is the same regardless of LTE firmware unless secondary carrier connection “fails” Because the 74XX LTE modem stores multiple firmware loads Fallback can be automatically/time enabled via an EEM application And automatically loads the appropriate one for the SIM/carrier

Remote Main Public or Private Branch Carrier A RAN Site Network Carrier A EPC

Carrier B RAN Carrier B EPC

• 819 (MNA), CGM (MNA), 829 (dual radio) • Define the needed profiles (enable mode command)

• Connection to 1 network at a time, but access to 2 819#cell 0 lte profile create 1 broadband none ipv4v6 (AT&T) 819#cell 0 lte profile create 2 fast.t-mobile.com none ipv4v6 (Tmobile) • Switchover from primary to backup carrier < 3 minutes • Define the modem profile to SIM slot mapping • Switchover can be manual, automatic via IOS, via applet • Define the primary SIM slot conf t, controller cell 0, lte sim primary slot 1 819(config)#controller Cellular 0 IOS will switch to other SIM if no LTE connection after 3 minutes 819(config-controller)#lte sim data-profile 1 attach-profile 1 slot 0 • Behavior is different based on LTE model & firmware, ex: 819(config-controller)# lte sim data-profile 2 attach-profile 2 slot 1 C819G-LTE-MNA-K9: FW-MC7354-LTE-CA (North America) 819(config-controller)# lte sim primary slot 0 C819G-4G-GA-K9: FW-MC7304-LTE-GB (Europe)

interface Cellular0/2/0 policy-map PNTM-A ip address negotiated class PREC-345 policy-map SUB-CLASS-34 no ip unreachables shape average 20000000 class PREC-5 … bandwidth remaining ratio 3 priority level 1 percent 10 service-policy output PNTM-A service-policy SUB-CLASS-345 class PREC-4 class class-default bandwidth percent 50 fair-queue class PREC-3 bandwidth remaining ratio 1 bandwidth percent 40 random-detect dscp-based 1) Shape dedicated bearer traffic to 2Mbps, engage sub-class if LTE congestion up 2) Within that, Guarantee 800Kbps to Prec-3 Guarantee 1Mbps to Prec-4 Strict Priority Guarantee 200Kbps to Prec-5

• What are 4G and 5G Cellular Technologies?

• How Do I Deploy and Troubleshoot? • Models, IOS releases • Troubleshooting

Parameter Description

RSSI Received Signal Strength Indicator. Measures total received signal power. Note: It is a misnomer! It actually measures ‘Wanted Signal’ + Noise RSRP Reference Signal Receive Power. Measures reference power to calculate path loss. Since the reference power is well known, modem uses this for cell selection. Together with RSSI, this parameter calculates the noise component, etc. RSRQ Reference Signal Receive Quality. Calculated from RSSI and RSRP.

SINR Signal to Interference plus Noise Ratio. Calculated from RSSI and RSRP.

Condition RSSI RSRP RSRQ SINR

Excellent -59dBm or higher (e.g. -58) -79 dBm or higher -4 or higher 20 to 30 dB

Mid-Cell -69 dBm to -60 dBm -89 to -80 dBm -11 to -5 dB 6 dB to 20 dB

Good -79 dBm to -70 dBm -99 to -90 dBm -15 to -12 dB 0 to 5 dB

Cell-edge -80 dBm or lower -100 dBm or lower -19 to -16 dB -1 dBm or lower (e.g. -2)

This is the EARFCN (EUTRA Absolute Radio Frequency Channel Number) It provides the center frequency for UL and DL, of the TX and RX channels within a band

These are the radio parameters as described in the previous slides.

IMSI (International Mobile Subscriber Identity) is typically a 15 digit number, Modem Firmware Level obtained from the SIM card. It globally and uniquely identifies a subscriber on a Modem model network. The typical structure is as follows:

• MCC (Mobile Country Code) - 3 digits : 311 (USA) • MNC (Mobile Network Code) - 3 digits : 480 (Verizon Wireless) • MSIN (Mobile Station Identification Number) - 9 digits : 063417381

Mobile Station ISDN Number: International Mobile Station Modem Firmware Carrier Info It is the phone number of the Equipment Identity. It uniquely (e.g. VZW, ATT, Generic North subscription for this device identifies the hardware identity of America, etc. (UE) the device (modem)

1 PID Name 2 LED—EN

LED—GPS (Global Positioning 3 4 LED—WWAN System)

RSVD (reserved) port, USB 2.0 5 6 LED—SERVICE mini type B

LED—RSSI (Received Signal M1 DIV—Diversity Antenna 7 8 Strength Indicator) Connector (TNC)

M0 MAIN—Main Antenna 9 GPS Antenna Connector (SMA) 10 Connector (TNC)

*See Hardware Installation Guide*

• Confirm LTE carrier provisioned the SIM before configuration • The IMEI may be required, but often not enforced for attach • IMEI can be found on the shipping box, on the router via CLI, marketplace order • SIM can be moved to a different ISR after initial network attach • A SIM may be orderable with ISR, ICCID is available in marketplace With IMEI, allows service to be enabled prior to arrival of ISR

• Over the Air – Device Management

• Responsible for changing APN

• Could take up to 10 minutes after boot up (do not reboot during this time)

• If unsuccessful, you can manually set the APN

• Only needed for public static or wireless Private Network service

• Check your chat script • Copy and paste! • Be careful with some editors and terminal emulators inserting invisible characters • Chat script is defined once, referenced twice in IOS configuration (not in IOS XE) • IOS-XE includes chat script by default (not shown)

Cellular0/x/0 is up (spoofing), line protocol is up (spoofing) before making call A cell interface that’s Up/Up(spoofing) may not be up!

Symptom: Solution: Traffic sent over LTE does not receive reply, or connection flaps 1. NAT or VPN

Reason: 2. The LTE carrier can trace the bad IP address if needed • If packet is not sourced from the (easiest via the SIM MDN or cellular IP address, traffic cannot modem IMEI) be returned to the LTE device.

• The return packet does not have IP address of the LTE device, therefore cannot be delivered

• It is wireless: Different expectations for – availability, SLAs, throughput consistency Site surveys! Performance based on modulation/location/density Antenna options. Use the site survey SMS app Understand the differences with wireline and how to troubleshoot

• Restrictions: Some SPs have restrictions on non-user/control plane traffic Can tune routing protocol timers, IP SLA frequency, etc.

• Price plans: Ensure the monthly byte count is understood (e.g. line sharing) We have an app for alerting when approaching plan max

• LTE QoS: Wireline and Wireless QoS may differ (check the SLAs)

• SP LTE offers: Enterprise grade LTE QoS, enterprise private addressing/routing, Multi-VRF, IPv6

• www.cisco.com/go/4g www.cisco.com/c/en/us/support/interfaces-modules/high-speed-wan- interface-cards/products-installation-and-configuration-guides-list.html

#CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 89 Top Deployment Issues Issue Action IOS log error messages, can’t connect Recommended IOS and LTE modem firmware releases Can’t connect, SIM not active, APN is incorrect Check SIM in another device, ask SP for APN value IOS Configuration issues (NAT/VPN, dialer, Use the guidelines recommended by Cisco or the SP routing, MTU adjust to not fragment, etc.) Consistently low performance Checkradio reception, antenna placement, IP fragmentation Low performance in a densely populated area or at Check to see if all the bands your LTE carrier supports are certain times of day supported by the LTE module/interface (older LTE modems) Dropping from 4G to 3G, won’t reselect 4G Reception/interference, no reselection due to traffic activity, site (performance degraded) survey & lock RAT to LTE Set expectations that LTE is a shared medium. Understand Expectations of consistent/wireline performance traffic paths and anchor points Something doesn’t work…with a complex config Strip out the complexity, try a more basic config with LTE. If it that includes LTE works, try adding in functions to see where it breaks

www.cisco.com/c/dam/en/us/td/docs/routers/access/interfaces/software/de Something else ployment/guide/guide_100413.pdf pages 3-4 I got a bill for a zillion dollars for exceeding my Use the free byte count EEM app! plan’s monthly byte count!

#CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 90 Complete your online session • Please complete your session survey after each session. Your feedback evaluation is very important.

• Complete a minimum of 4 session surveys and the Overall Conference survey (starting on Thursday) to receive your Cisco Live water bottle.

• All surveys can be taken in the Cisco Live Mobile App or by logging in to the Session Catalog on ciscolive.cisco.com/us.

Cisco Live sessions will be available for viewing on demand after the event at ciscolive.cisco.com.

Demos in the Walk-in labs Cisco campus

Meet the engineer Related sessions 1:1 meetings

#CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 92 SPM Sessions Sunday, Monday, Tuesday ID Date Time Title Session type Catalog Link Sunday Sessions https://www.ciscolive.com/us/learn/ LTRSPM-2010 Sunday, June 9, 2019 02:00 PM Cisco Packet Core 5G Lab Instructor-Led Lab sessions/session- catalog.html?search=LTRSPM-2010#/ Monday Sessions https://www.ciscolive.com/us/learn/ BRKSPM-2012 Monday, June 10, 201908:00 AM 5G xHaulTransport Breakout sessions/session- catalog.html?search=BRKSPM-2012#/ https://www.ciscolive.com/us/learn/ BRKSPM-2129 Monday, June 10, 201901:00 PM Deploying 5G and LTE for Enterprise and IoT Last Mile Breakout sessions/session- catalog.html?search=BRKSPM-2129#/ https://www.ciscolive.com/us/learn/ BRKSPM-2009 Monday, June 10, 201901:00 PM Design 5G ready distributed Telco DC with Cisco ACI Breakout sessions/session- catalog.html?search=BRKSPM-2009#/ https://www.ciscolive.com/us/learn/ Monetizing the Instant Network – Connecting Enterprise and Service Product or Strategy sessions/session- PSOSPM-1102 Monday, June 10, 2019 02:30 PM Provider Domains for 5G Success Overview catalog.html?search=PSOSPM- 1102#/ https://www.ciscolive.com/us/learn/ BRKSPM-2015 Monday, June 10, 201904:00 PM Auto-deploy Virtualized RAN Breakout sessions/session- catalog.html?search=BRKSPM-2015#/ https://www.ciscolive.com/us/learn/ BRKSPM-2014 Monday, June 10, 201904:00 PM Automate your 4G/5G Mobile Packet Core Breakout sessions/session- catalog.html?search=BRKSPM-2014#/ Tuesday Sessions https://www.ciscolive.com/us/learn/ sessions/session- BRKSPM-2032 Tuesday, June 11, 201908:00 AM Defining Your 5G Architecture Breakout catalog.html?search=BRKSPM-2032#/ https://www.ciscolive.com/us/learn/ sessions/session- BRKSPM-2034 Tuesday, June 11, 201908:00 AM 5G Mobile Transport Design and Deployment Breakout catalog.html?search=BRKSPM-2034#/ https://www.ciscolive.com/us/learn/ sessions/session- BRKSPM-2071 Tuesday, June 11, 201901:00 PM 5G Technology Updates Breakout catalog.html?search=BRKSPM-2071#/ #CLUS BRKSPM-2129 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 93 SPM Sessions Wednesday and Thursday

ID Date Time Title Session Catalog Link type

Wednesday Sessions https://www.ciscolive.com/us/learn/s essions/session- BRKSPM-2017 Wednesday, June 12, 2019 08:00 AM Voice Over 5G Mobile Packet Core Breakout catalog.html?search=BRKSPM-2017#/ https://www.ciscolive.com/us/learn/s essions/session- BRKSDN-2411 Wednesday, June 12, 2019 08:00 AM NFV Performance -Challenges and Solutions Breakout catalog.html?search=BRKSDN-2411#/ Product or Strategy https://www.ciscolive.com/us/learn/s essions/session- PSOSPM-2001 Wednesday, June 12, 201902:30 PM Nuts-n-Bolt of SP Network Automation Overview catalog.html?search=PSOSPM-2001#/

https://www.ciscolive.com/us/learn/s essions/session- BRKSPM-2950 Wednesday, June 12, 2019 04:00 PM Rakuten Japan - Deploying Fully Automated Largest TelcoCloud Breakout catalog.html?search=BRKSPM-2950#/ Thursday Sessions https://www.ciscolive.com/us/learn/s essions/session- BRKSPM-2018 Thursday, June 13, 201908:00 AM 5G Ready Software Defined Test Automation@Rakuten Japan Breakout catalog.html?search=BRKSPM-2018#/ Cisco cloud native 5G architecture, products and microservices https://www.ciscolive.com/us/learn/s essions/session- BRKSPM-2019 Thursday, June 13, 201909:30 AM infrastructure Breakout catalog.html?search=BRKSPM-2019#/ https://www.ciscolive.com/us/learn/s essions/session- BRKSPM-2010 Thursday, June 13, 201910:30 AM Applying Security in a 5G World Breakout catalog.html?search=BRKSPM-2010#/ https://www.ciscolive.com/us/learn/s essions/session- BRKSPM-2031 Thursday, June 13, 201901:00 PM Bring 5G core network slices into life with automation Breakout catalog.html?search=BRKSPM-2031#/

#CLUS #CLUS How 4G/LTE Works

Logical Next Hop in Transparent EPC to Service Provider Support Mobility Network for ISR

Packet-Switched Mobile Handset Network

(Internet, MPLS VPN Service, Customer eNodeB S-GW Private Network, Other) Premises P-GW Equipment (CPE) Cellular Provider Network

Layer 3 (IP) Connectivity Between CPE and P-Gateway

EPC = Evolved Packet Core eNodeB = Cell Tower S-GW = Serving Gateway P-GW = Packet Data Network Gateway

N/W Slice Network Network Selection AUSF Rep’tory UDM UDR Exposure Function (EAP) Function Function (NSSF) (NRF) (NEF)

Binding AMF SMF PCF Support Function

Front-Hall, NG- Mid-Haul, RAN Back-Haul Public or Private UE UPF Data Network (DN) APPS N3IWF

• The Cisco definition of network slicing is expansive • In the Cisco version, selection functions are fed by orchestration and identity to implement a cross-bar that connects any access to any slice

• Each slice can flexibly contain • A complete core network with its own MME, PCRF and identity management as well as value added services • Any subset of a core network defined through anything in the subscriber session record (APN, membership in a policy class, etc.) • The definition is hierarchical/ recursive (slices can include slices)

• Slice contents are determined by a template 5G slice cross-bar concept: Get any flow from any access to the appropriate service or end-point

Technology Cellular Wireless Cable, PON, DSL, Metro-E etc. T1, T3, OCX, Metro-E, etc. Monthly Cost Varies by plan ( bytes/month included). Varies by speed & geography. 2X-10X Varies by speed, geography, and Can be Lowest cost based on rate plan cost of LTE, but there is generally no distance between locations. 10X-20X & monthly usage monthly byte count cost consideration. cost of LTE. Usage Cost Varies by subscription plan. Charge per Usage caps not common and if in place None. GB over plan, but can pool lines. may be higher than 100-300 GB/month. Where it is Available Most locations in the U.S. and most Locations in urban areas and suburbs, Locations in urban areas and suburbs, densely populated locations worldwide. varies by provider. varies by provider. Bandwidth Varies by RF conditions/SP. Varies, asymmetric or symmetric, Varies, generally symmetric, 56Kbps- Asymmetric. 5-100Mbps down, 1Mbps-1Gbps down 384Kbps-1Gbps 1Gbps down, 56Kbps-1Gbps up. 1-20Mbps up. up Throughput Varies by RF conditions and network Varies by network congestion, as some Consistent. congestion, as “last mile” is shared. Internet paths are shared. Availability (Uptime) Varies by RF quality, generally > 99.5% Varies by service, generally > 99.9%. Varies by service, generally > 99.99%. Service Level Agreement None or less stringent None or less stringent Strong, more specific Provisioning/Installation Varies , usually a few days. Easy to Varies , usually a few weeks. Moving Varies , usually => 1 month. Moving move LTE routers to new locations. existing lines requires re-provisioning. existing lines requires re-provisioning. Usage Restrictions Subscription plans may restrict non- Generally none. None. user traffic frequency, i.e. IGPs, probes Mobility Service can be moved to another None. None. location. It can also operate while in motion (trucks, buses, cars, trains). Security Varies by service, public + VPN, private Usually a public service requiring firewall Private connection does not “touch” (secure similar to private wireline) and VPN. public networks. Side-Band management SMS, even if data connection down None inherent to the connection. None. Inherent to the connection. Quality of Service (QoS) Varies by service, SLO vs SLA. Varies by service, SLO vs SLA Varies by service, generally an SLA