Cloud-Based Nokia SBC Performance Testing and Function Validation

Cloud-based Nokia SBC Performance Testing and Function Validation

September 2017

DR170831E

Miercom.com

www.miercom.com

Contents

1.0 Executive Summary ...... 3 Test Summary ...... 4 2.0 Product Overview ...... 5 3.0 How We Did It ...... 9 4.0 Performance Testing ...... 15 4.1 Signaling performance ...... 15 4.2 Media performance ...... 16 4.3 Performance Under Attack ...... 17 5.0 Security Testing ...... 20 5.1 Nessus Vulnerability Scan ...... 20 5.2 Codenomicon ...... 21 6.0 Functional Testing...... 23 6.1 SBC Resilience & High Availability...... 23 6.2 Support of Ultra-HD Voice EVS codec ...... 25 6.3 SIP Header Manipulation ...... 25 6.4 WebRTC ...... 26 7.0 Management and Orchestration Testing ...... 29 7.1 SBC VNF Cloud Orchestration ...... 29 7.2 SBC Operations, Administration and Management ...... 31 7.3 SBCs Cluster Management with NetAct ...... 38 About Miercom ...... 41 Customer Use and Evaluation ...... 41 Use of This Report ...... 41 Appendix ...... A-1

Nokia commissioned Miercom to perform an independent testing of the Cloud-based Nokia SBC and validate the product’s performance, security, functionalities and management. The Nokia SBC is a field-proven virtualized session border controller software designed to control and secure the signaling and media stream of Communication Service Provider’s (CSP) or Enterprises’ IP-communications networks. Nokia SBC operates as a Virtual Network Function (VNF) in Nokia’s end-to-end cloud-native solution or in competitive standalone deployment. The document provides a complete description of the test plan and results, using real-world network parameters. It also covers Nokia SBC VNF integration with OpenStack-based Management and Operation (MANO) framework. Key findings: • Validated signaling plane performance: Processed 126,000 concurrent signaling sessions over TLS and 48,000 concurrent VoLTE signaling sessions over IPSec on a single VM. • Validated media plane performance and high-density transcoding: Processed 77,500 concurrent media sessions on a single media plane VNF and a 4.23 MOS for AMR-WB codec. Processed 1,400 concurrent G.711µ-G.729ab transcoded calls on a single VM. • Powerful encryption/de-encryption for service security: Supported 10,600 concurrent sRTP-RTP media sessions on a single VM. • Effective denial of service/distributed denial of service (DoS/DDoS) protection: Prevented every TCP, UDP and SIP flood attacks. No calls were dropped and CPU usage remained constant for high-volume call loads. • High resiliency and availability: Showed high resiliency during failover test with 16,200 concurrent calls. All established calls were successfully transferred to other VMs. • Supports multiple services and new features: Supported simultaneous calls from access and peering, Enhanced Voice Services (EVS) codec and a Web real-time communications (WebRTC) gateway and APIs for third party to develop in-browser communications services. • Unified and easy SBC operations: Demonstrated management flexibility on three levels: SBC VNF with CloudBand Application Manager (CBAM), individual SBC with SBC web user interface (WebUI) and group of SBCs with Nokia NetAct. Based on results of our testing, the Nokia SBC offers a safe way to evolve SBCs to support key cloud architectural principles such as fully virtualized network functions and MANO. Cloud-based Nokia SBC proved impressive high performances, security and scalability potential, earning the Miercom Performance Verified certification.

Robert Smithers CEO Miercom

Performance

SIP calls over TLS (30 sec) VoLTE calls over IPSec (30 sec) • 600 rps max registration rate • 600 rps max registration rate (IMS- Signaling (MD5 authentication) AKA authentication) Section 4.1 • 800 cps max. call rate • 580 cps max call rate • 126,000 max concurrent sessions • 48,000 max concurrent sessions

Max RTP media session capacity: Max sRTP to RTP Max RTP media • 24,000 on a single VM, 4.23 MOS media session session capacity Media • 77,500 on a single media plane capacity (G.729 -G.711 Section 4.2 VNF, 4.23 MOS • 10,600 on a transcoding) • 124,000 on a dual media plane single VM, • 1,400 on a single VNF, 4.23 MOS 4.23 MOS VM, 4.3 MOS

Attacks No impact on call load for TCP SYN, UDP, SIP INVITE and SIP Registration DoS Section 4.3 and DDoS floods. Maximum of 5% CPU increase.

Security

Nessus Vulnerability scan No vulnerabilities found during local OA&M interface and remote signaling Section 5.1 interface scans. Found 7 medium-risk flaws during remote OA&M interface scan.

Codenomicon Security flaw tester passed. Ran 275,488 tests. No security flaws found. Section 5.2 Functional

Resilience & High Availability Failover and rebooting of VMs; Call rate of 120 cps, 300 ms to 2 sec failover time; Section 6.1 all failovers passed. Call rates sustained. No active calls dropped.

EVS Codec Verified EVS codec support, transcoding to AMR-WB and backwards Section 6.2 compatibility. Call rate of 1 cps with 4.22 MOS.

SIP Header Manipulation Verified SIP filtering effects on quality; 4.16 MOS was unaffected. Minimal CPU Section 6.3 increase for calls with manipulated SIP messages impacted call performance.

WebRTC Successful OPUS-G.711 connection, chat and media sharing. Custom Slack web Sections 6.4 application by third-party developer integrated with SBC WebRTC APIs.

Management and Orchestration

VNF Cloud Orchestration Verified Nokia CloudBand integration – Successful, intuitive use of CBAM GUI for Section 7.1 install; deployment; extraction; healing; updating; scaling of VNF components.

Verified VMs control on WebUI - Full visibility and control verified for charts; OA&M measurement; alarms; call tracing; backup; component/media plane status and Section 7.2 SIP screening.

Element Management System Verified NetAct integration - The centralized NetAct monitoring system provided Section 7.3 real-time visibility into every individual SBC or group of SBCs.

Cloud-based Nokia SBC

The Nokia SBC is a carrier grade, service-aware gateway which controls IP media and signaling streams across both the access and peering network boundaries, for both CSPs and enterprises. Hundreds of millions of subscribers have relied on its service and security in 11 of the top 25 mobile networks. This performance is now available in the Cloud-based Nokia SBC.

The Nokia SBC operates as a VNF, providing a cloud-based software-only SBC for deployment on any OpenStack or VMware cloud infrastructure. It helps organizations adapt to new network and technology shifts, take advantage of the speed, flexibility and efficiency of the cloud while also enabling 3GPP-standard functions migration to the cloud, including:

• Proxy-Call Session Control Function (P-CSCF) and Access Border Gateway (IMS-AGW) for end-user signaling and media connectivity to core network, • Access Transfer Control Function and Gateway (ATCF/ATGW) for seamless VoLTE call handover on a circuit-switched mobile network, • Interconnect Border Control and Gateway Function (I-BCF/I-BGF) for signaling and media connectivity to or from peering networks, • Enhanced P-CSCF function for Web Real-Time Communications i.e. audio, video and data transfer on any device with a browser and embedded IP into the context of any apps and website.

Network functions can be deployed as separate VNF instances or in combination, providing maximum configuration flexibility and faster time-to-market for new services. Each VNF instance is decomposed into VNF Components (VNFCs) that can scale independently to meet the growing control plane demands of VoLTE, VoWi-Fi, the future IoT/MTC, and ultimately the transition to 5G. As shown in the picture below, these components process all the traffic coming from the untrusted access and the peering sides.

Source: Nokia Cloud-based Nokia SBC components

Cloud-based Nokia SBC Performance Verified 5 DR170831E Copyright ©2017 Miercom 25 September 2017 The Nokia SBC communicates with the core network, which contains a registrar of allowable callers into the secured network. If registered, the call can be processed and connected inside the core. The core is the secured side of the network, an IP multimedia Subsystem (IMS) or an enterprise IP network, depending the use case (SBC for CSPs or enterprise SBC).

All functions and services are protected using an internal traffic distributor and firewall at the network edge to secure the media and signaling plane. The Nokia SBC media plane security uses packet filtering on network (L3) and transport (L4) layers of the network. Attacks on the application layer (L7) are prevented by using SIP and HTTPS filtering.

Protected services include:

• Fixed VoIP for consumer and business (cVoIP/bVoIP), • Mobile VoLTE, • Video over LTE (ViLTE), • Voice over Wi-Fi (VoWi-Fi), • Rich Communication Services (RCS), • Enterprise SIP-trunking / Cloud-PBX, • Enterprise Unified Communication and Collaboration (UC&C), • In-browser communications services.

The SBC web user interface (WebUI) provides web-based Operation, Administration and Management (OA&M) interface on individual SBC. Operations include: configuration, fault management (system status, alarms etc.), performance indicators and management, backup and troubleshooting (call tracing). Through the WebUI dashboard, control profiles can be created and security logs can be read or written.

Nokia SBC supports standard ETSI Network Functions Virtualization (NFV) architecture and interfaces to help organizations cost-effectively bolster their SBC deployments on any customer chosen OpenStack or VMware cloud infrastructure. The figure below shows the software components of the Nokia SBC on OpenStack cloud NFV solution.

Source: Nokia

Nokia SBC VNF deployment on cloud NFV

The Nokia SBC integrates with Nokia CloudBand to reduce the overall integration efforts, provide performance, resiliency, scalability and operational efficiency while also enabling a seamless transition to NFV and software-defined networking (SDN):

• CloudBand Infrastructure Software (CBIS) virtualizes and manages compute, storage, and network resources. It enables the Nokia SBC VNFs to run and ensures that they meet strict robustness, performance, and security requirements. • CloudBand Application Manager (CBAM) automates lifecycle management actions on the Nokia SBC VNFs by managing resources and applying associated workflows.

The Nokia SBC also integrates with Nokia NetAct Element Management System (EMS). The Nokia NetAct is virtualized for minimal downtime and resilience and gives one consolidated view and full visibility and control over the SBC network. NetAct also offers a uniform set of tools for radio, core and transport network management. This means reduced administration, maintenance and system integration cost.

The Nokia SBC VNF consists of several VNFCs that run on multiple VMs on the HPE c7000 server blades. The table below provides a sample configuration for a 2 million subscriber footprint.

No. No. Memory VNFC Abbrev. Description VM CPU Size Operation, Oversees all element activity; Administration and OA&M 2 4 16 GB controls all VM blades. Maintenance

Provides optional charging and local Charging iCCF 2 4 16 GB CDR storage

Layer 3-4 packet filtering and layer 7 Firewall FW SIP filtering and untrusted network 2 16 32 GB front end distributor Handles the access and peering Session Controller SC 8 32 192 GB signaling processing

Core Front End Distributes the trusted signal- CFED 2 16 64 GB Distributor sessions to core.

Diameter Front End Distributes the trusted diameter DFED 2 8 32 GB Distributor traffic

Border Gateway H.248 media gateway control BGC 2 4 16 GB Controller (server side)

System Control SCM H.248 control (client side) 4 16 32 GB Module

Packet Interface PIM Processes media traffic 16 128 256 GB Module

Media Conversion Provides software-based 12 MCM 168 224 GB Module transcoding of media sessions +2

TOTAL 54 396 880 GB

Cloud Infrastructure Setup

The test plan covered Nokia SBC combined access and peering for cloud deployment on OpenStack CloudBand Infrastructure (CBIS) and CloudBand Application Manager (CBAM).

The Virtualized Infrastructure Manager (VIM) CBIS runs on two C7000 chassis and 32 Gen9 hosts: one host is needed for under cloud VM, and three hosts for Controller with High Availability, leaving 28 hosts for compute nodes. The CBIS memory design accessed local memory quickly to handle scalable workloads. On compute nodes, CBIS used 6 vCPUs and 2G memory dedicated for hypervisor (4 vCPUs from NUMA0 and 2 vCPUS from NUMA1), that means for a Gen9 blade with 48 vCPUs, 42 vCPUs were available for SBC VMs. All computes were configured to enable single root I/O virtualization (SR-IOV) for higher virtualized media plane performance.

Compute Host Configuration Table

Server HPE BL460c Gen9

2 x Intel Xeon(R) CPU E5-2680 v3 @ 2.50GHz (12 cores) - total 24 cores – total 48 CPU HT – total 42 HT for guest vCPU

RAM DDR4, 2133 MHz, 128GB

2x Dual port NICs: Flexible LOM HP Ethernet 10GB 2-port 560FLM adapter and Network Mezzanine card: HP Ethernet 10GB 2-port 560M adapter

Integrated HDD: HP 1.2TB 12Gb/s SAS 10Krpm, RAID0 Storage HP Smart Array P244br

The VNF Manager (VNFM) CBAM managed the life-cycle of the Nokia SBC VNF. The test plan covered several LCM events (deploy, heal, scale etc.) implemented using VNF templates, VNF descriptors, Heat Orchestration Templates (HOT), Mistral workflows and Ansible playbooks.

Source: Nokia

Onboarding Nokia SBC VNF

Test Tools

Source: Miercom

Cloud-based Nokia SBC Performance Verified 10 DR170831E Copyright ©2017 Miercom 25 September 2017 • The Ixia XGS12-SD generated signaling and media loads with Transport Layer Security (TLS), VoLTE with IP Multimedia Subsystem (IMS) Authentication and Key Agreement (AKA) authentication, Real-time Transport Protocol (RTP), Secure RTP (SRTP) and transcoding. • DDoS and flood attacks traffic was generated by the Nokia TCP/IP RightTrack DoS toolkit. • The SIPp tool simulated signaling-only loads. • WebRTC call was initiated using the Nokia WebRTC API.

Lab Topology

Ixia XGS-12SD with IxLoad IMS core, SIPp, Nessus, Codenomicon, RightTrack on CBIS (Openstack Liberty cloud on HP Gen9 server blades) Source: Miercom

Nokia SBC

Unsecure Network Secure Network User Equipment Callee (UEC) User Equipment Server (UES)

Interfaces

Signal and Media Calls with TLS, VoLTE+AKA, RTP 10G

- sRTP, DoS/DDoS Generator 8 Signal Calls, RTP Ixia, SIPp, Layer 3 Nessus, CN, HP Switch Nokia IMS Right Track 4-10G 6-10G Interfaces Interfaces

Source: Miercom

Traffic Profile

Different traffic profiles were used to represent a real-world network environment:

Traffic Type Description

• Registration using MD-5 authentication • SIP over Transport Layer Security (TLS) v1.2 SIP secured by TLS • Call flow consisting of 7 messages: INVITE, 100 Trying, 180 Ringing, 200-OK, ACK, BYE, 200-OK

• Registration using IMS-AKA authentication • SIP over IPSec • Call flow consisting of 12 messages: INVITE, 100 VoLTE with AKA Trying, 183 Session Progress, PRACK, 200-OK, 180 Ringing, PRACK, 200-OK, 200-OK (INVITE), ACK, BYE, 200-OK

Source: Nokia

Standard IMS call

As shown in the picture above, in a standard IMS call (assume the calling and called parties belong to the same CSP), 1 call per session is managed on the originating SBC to connect the calling party to the network and 1 call per session is managed on the terminating SBC to connect the network to the called party.

Source: Nokia

Lab test call

During the tests, Nokia SBC managed both the calling and called party sides emulated by the load generator. That means when a call was placed, 2 calls were being handled by the SBC: 1 call per session to connect the calling party to the IMS network and 1 call per session to connect the IMS network to the called party.

• 100 cps call load from the Ixia XGS12-SD generator means the SBC handles 100 cps on the originating side and 100 cps on the terminating side, 200 cps in total. • 200 cps call rate on SBC means the load generator initiated a 100 cps call load.

Test Plan

The test plan covered four main areas:

• Performance Performance metrics for the signaling and media planes and performance under attack. • Security Find vulnerabilities that may expose the secured network to peering or attacks from untrusted network sources. Testing was performed using the Nessus Vulnerability Scanner and Codenomicon test tools. • Functionality Observe and validate four key features: SBC Resilience, SBC support of EVS VoLTE codec, SIP Header Manipulation, WebRTC Gateway and APIs. • Management and orchestration Validate the different levels of management of SBC VNF using CBAM, SBC WebUI and Nokia NetAct.

The results of the tests are presented in the following four sections.

4.1 Signaling performance

The Session Controller (SC) is one of the Nokia SBC’s virtual machines (VMs) responsible for processing calls that have passed through the firewall VM. There are 8 SC VMs per SBC VNF consisting of 4 pairs of active/stand-by VMs. The following tests have been passed against 1 SC VM pair.

This VM pair is rated to run up to 75 percent of the HPE server CPU and use 80 percent of memory during high performance handling. The CPU and memory are monitored at 400 millisecond (ms) intervals, and when thresholds are reached, alarms and overload controls are activated. Calls and SIP messages may be throttled. Additional alarms are generated as critical limits of resources are met.

4.1.1 Subscriber registration performance Description Result

Generate registrations using MD5 authentication over • Max. registration rate = 600 TLS. Verify the registration rate at which the SC VM can registrations per second (rps) successfully signal registrations. • SC CPU = 27%

Generate VoLTE registrations using IMS-AKA • Max. registration rate = 600 rps authentication over IPSec. Verify the maximum • SC CPU = 17% registration rate at which the SC VM can successfully

signal registrations.

4.1.2 Call signaling performance over TLS Description Result

Generate SIP calls over TLS with 30 sec hold time. • Call duration = 30 sec Determine the maximum call rate at Session Controller • Max. call rate = 800 cps VM resource capacity (75% CPU, 80% memory).

Generate SIP calls over TLS with 30 sec hold time. • Call duration = 30 sec Measure the SIP session capacity, i.e. maximum number • Max. concurrent sessions = 126,000 of concurrent sessions the SC VM can manage

Generate VoLTE calls using AKA authentication over • Call duration = 30 sec IPSec with 30 sec hold time. Determine the maximum • Max Call rate = 580 cps call rate at Session Controller VM resource capacity (75% CPU, 80% memory).

Generate VoLTE calls using AKA authentication over • Call duration = 30 sec IPSec with 30 sec hold time. Measure the SIP session • Max. concurrent sessions = 48,000 capacity, i.e. maximum number of concurrent sessions the SC VM can manage.

4.2 Media performance

Encrypted VoLTE signaling was generated using SIPp and Ixia to determine maximum sessions.

4.2.1 Media Calls performance with RTP Description Result

Generate media calls using AMR-WB codec and Case 1: one pair of (active/stand-by) PIM VM RTP protocol on the access and core side. (8 vCPUs) • Measure the media session capacity, i.e. • Call duration = 300 sec maximum number of concurrent media • Max. call rate = 40 cps sessions supported. • Max media session capacity = 24,000 • Determine the max. call rate • CPU = 90% • Verify quality with MOS score. • MOS = 4.23

Case 2: Single Media-Plane VNF (4 pairs of Perform these tests with different capacities, (active/stand-by) PIM VM) from smaller to larger and verify the MOS score. • Call duration = 541 sec • Max call rate = 72 cps • Max media session capacity = 77,500 • CPU = 85% • MOS = 4.23

Case 3: Dual Media-Plane VNF (8 pairs of (active/stand-by) PIM VM) • Call duration = 867 sec • Max. call rate = 71 cps • Max. media session capacity = 124,000 • CPU = 85% • MOS = 4.23

Generate media calls using G.729 codec and sRTP One pair of (active/stand-by) PIM VM (8 vCPUs) protocol on the access side and G.729 codec and with G.729 sRTP to RTP interworking: RTP protocol on the core side. Use 130 sec call • Call duration = 130 sec hold time. • Max. call rate = 40 cps • Measure the media session capacity, i.e. • Max. media session capacity = 10,600 maximum number of concurrent media • CPU = 83% sessions supported on one PIM. • MOS = 4.16 • Determine the maximum call rate • Verify quality with MOS score

4.2.3 Media transcoding performance Description Result

Generate media calls using G.729 codec and RTP Single MCM (12 vCPUs) with G.729-G.711 protocol on the access side and G.711 codec and transcoding: RTP protocol on the core side. Use 155 sec call • Call duration = 155 sec hold time. • Max. call rate = 9 cps • Measure the media session capacity, i.e. • Max. media session capacity = 1,400 maximum number of concurrent media • Call peak = 712 calls sessions supported on one MCM • CPU = 84% • Determine the maximum call rate • MOS = 4.3 • Verify quality with MOS score.

4.3 Performance Under Attack

Denial-of-Service (DoS) and Distributed (DDoS) attacks overwhelm a target with traffic, creating vulnerability. DDoS attacks have legitimate looking connections and no single attack source. To test the SBC for DoS/DDoS protection, we used a combination of packet flooding and targets. Floods of TCP, UDP and SIP packets were sent to signaling interfaces with public IP addresses.

The following results were expected: no expected noise; no impact on call load; alarms are generated and cleared; no critical issues (SegV, memory leak, core dump, switchover/failover, unexpected ASSERT, ALARM, HIGH log). A 46 percent CPU baseline was recorded with 1.1 million busy-hour call attempts and 150 second call hold time.

Generate SYN flood attack, the continuous opening of • Initial CPU = 46% new connections with SYN messages to the server, • CPU during attack = 51.3% without the follow-up ACK message to close them. • No impact on call load and system Increasing amounts of half-open connections will drain • Alarms issued system resources until DoS results; malfunction or • Firewalls reported dropped packets failure may occur. Measure resource use, system and associated measurements in the impact and prevention. WebUI  Background load = 2 million subscribers  Max calls = 95,700 IPsec sessions  Registration rate = 267 rps  Call rate = 311 cps  Attack flood rate = 600,000 pps SYN attacks

4.3.2 DDoS – Distributed TCP SYN Flood Description Result

Generate DDoS SYN flood attack, but incorporate • CPU during attack = 51% connection rate limiting. Measure resource use, system • No impact on call load and system impact and prevention. • Alarms issued  Background load = 2 million subscribers • Firewalls reported dropped packets  Max calls = 95,700 IPsec sessions and associated measurements in the  Registration rate = 267 rps WebUI  Call rate = 311 cps  Attack flood rate = 560,000 pps

4.3.3 DDoS - UDP Flood Description Result

Generate a DDoS UDP flood on a specific port • CPU during attack = 37% to cause the system failure, restart or memory loss. • No impact on call load and system Measure resource use, system impact • Alarms issued, indicating dropped and prevention. packets  Background load = 100,000 subscribers • Firewalls reported dropped packets  Max sessions = 82,000 IPSec sessions and associated measurements in the  Attack flood rate = 1.2 to 1.3 million pps with 1 WebUI GB small packets for 20 min

Generate a SIP INVITE flood using a SIP INVITE • CPU during attack = 31% message. Measure resource use, system impact • No impact on call load and system and prevention. • Alarms issued, indicating dropped packets  Background load = 100,000 subscribers • Firewalls reported dropped packets and  Max sessions = 95,700 IPSec sessions associated measurements in the WebUI  Registration rate = 267 rps  Call rate = 311 cps  Attack flood rate = 15,000 INVITE pps

4.3.5 SIP Registration Flood Description Result

Generate a SIP REGISTER flood using a SIP • CPU during attack = 24% REGISTER message. Measure resource use, system • No impact on call load and system impact and prevention. • Alarms issued, indicating dropped packets  Registered users = 2 million • Firewalls reported dropped packets and  Max sessions = 95,700 IPSec sessions associated measurements in the WebUI  Registration rate = 277 rps  Call rate = 311 cps  Attack flood rate = 15,000 REGISTER pps

Vulnerability scanning was performed for all IP addresses of different network interfaces. These vulnerabilities are categorized as low, medium, high or critical.

5.1.1 Nessus Local Scan - Guest OA&M Interface Description Result

Verify the following: A Nessus scan revealed no vulnerabilities. • Missing operating system (OS) security patches • Documentation of missing OS patches • List of open ports for each host • Post-scan LCP logs for analysis • No core dump or tenant services

5.1.2 Nessus Remote Scan - Guest OA&M Interface Description Result

Verify the following: A Nessus scan revealed: • Missing operating system (OS) • 1 High-grade vulnerability; this was reported security patches as a false positive, as it is a policy • Documentation of missing OS patches compliance summary. • List of open ports for each host • 14 Medium-grade vulnerabilities; this was • Post-scan LCP logs for analysis reduced to 7 vulnerabilities. Half were false • No core dump or tenant services positives, resolvable and resulting from policy compliance.

5.1.3 Nessus Remote Scan – Access Signaling Interface Description Result

5.2 Codenomicon

Codenomicon tools identify security flaws by sending loads of malformed protocol messages and observing system response. The system is expected to handle all stress-test conditions.

5.2.1 Codenomicon IPv4 Protocol Description Result

Verify the following: Pass; a total of 50 test cases were executed and • No unexpected noise no system flaws were found. • No impact to call load • CPI alarm generated and cleared • No critical issues (SegV, memory leak, core dump, switchover/failover, unexpected ASSERT, ALARM, HIGH log)

5.2.2 Codenomicon TCP for IPv4 Server Protocol Description Result

Verify the following: Pass; a total of 1,026 test cases were executed • No unexpected noise and no system flaws were found. • No impact to call load • CPI alarm generated and cleared • No critical issues (SegV, memory leak, core dump, switchover/failover, unexpected ASSERT, ALARM, HIGH log)

Verify the following: Pass; a total of 1,234 test cases were executed • No unexpected noise and no system flaws were found. • No impact to call load • CPI alarm generated and cleared • No critical issues (SegV, memory leak, core dump, switchover/failover, unexpected ASSERT, ALARM, HIGH log)

5.2.4 Codenomicon Diameter Client Protocol Description Result

Verify the following: Pass; a total of 273,178 test cases were executed • No unexpected noise and no system flaws were found. • No impact to call load • CPI alarm generated and cleared • No critical issues (SegV, memory leak, core dump, switchover/failover, unexpected ASSERT, ALARM, HIGH log)

6.1 SBC Resilience & High Availability

Resilience and high availability functionality is unique to each SBC. Traditional failover is accomplished using two separate SBC devices, an active SBC and a standalone SBC to pick up when failure occurs in the primary controller. Failover can occur when the primary SBC loses power, reboots, loses physical connectivity or encounters a computing crash.

In the case of the Nokia SBC, which utilizes virtual components to handle call-processing, each VM was rebooted to cause failover. High availability is expected to minimize or eliminate other VM failure, call failure and impact on stable calls.

6.1.1 Failover OA&M VM with VoLTE Load Description Result

Using a VoLTE call load, create an OA&M • Call rate = 120 cps failover by VM reboot and verify the following: • Call hold time = 135 sec • No impact to transient or stable calls. • No transient or stable calls lost

6.1.2 Failover FW with VoLTE Load Description Result

Using a VoLTE call load, create a FW failover by • Call rate = 120 cps VM reboot and verify the following: • Call hold time = 135 sec • Some transient call failure, where the • Pass; 150-300 ms failover time number of failed calls determines duration • No stable calls fail of outage • No impact on stable calls

6.1.3 Failover SC with VoLTE Load Description Result

Using a VoLTE call load, create a SC failover by • Call rate = 120 cps VM reboot and verify the following: • Call hold time = 135 sec • Some transient call failure, where the • Pass; 2 sec failover time number of failed calls determines • No stable calls fail duration of outage • No impact on stable calls

Using a VoLTE call load, create a CFED failover • Call rate = 120 cps by VM reboot and verify the following: • Call hold time = 135 sec • Some transient call failure, where the • Pass; 1 sec failover time number of failed calls determines duration • No stable calls fail of outage • No impact on stable calls

6.1.5 Failover BGC with VoLTE Load Description Result

Using a VoLTE call load, create a BGC failover • Call rate = 120 cps by VM reboot and verify the following: • Call hold time = 135 sec • Some transient call failure, where the • Pass; 500 ms failover time number of failed calls determines duration • No stable calls fail of outage • No impact on stable calls

6.1.6 Failover PIM with VoLTE Load Description Result

Using a VoLTE call load, create a PIM failover • Call rate = 102 cps by VM reboot and verify the following: • Call hold time = 135 sec • Some transient call failure, where the • Pass; 1 sec failover time number of failed calls determines duration • No stable calls fail of outage • Slight dip in MOS, but quickly recovers • No impact on stable calls

6.1.7 Failover SCM with VoLTE Load Description Result

Using a VoLTE call load, create a SCM failover • Call rate = 102 cps by VM reboot and verify the following: • Call hold time = 135 sec • Some transient call failure, where the • Pass; 2-3ms failover time number of failed calls determines duration • No stable calls fail of outage • No impact on stable calls

EVS codec supports sampling rates for narrow, wide, super-wide and full bands. It minimizes jitter and packet loss; concealing packet loss when it occurs. It is backwards compatible with Adaptive Multi-Rate Wideband (AMR-WB), a codec that supports wideband rates.

Using a VoLTE call being transcoded from EVS to AMR-WB, we observed a maintained high- quality call on the Nokia SBC. See Appendix Section 6.2 for screen captures of the transcoding process in the WebUI.

6.2.1 EVS Codec Description Result

Show a VoLTE call using EVS codec transcoded Call load: VoLTE call to AMR-WB. Call rate: 1 cps Transcoding: EVS/AMR-WB MOS: 4.22

6.3 SIP Header Manipulation

The message screening feature utilizes filters to manipulate both the SIP message and body information for interoperability and security. Filtering and manipulating SIP headers resolve protocol differences from multiple vendors and enhance security by hiding SIP topology, like IP addresses vulnerable to attacks.

Using the SC VM, we initiated calls at 40 cps to determine if the call quality could be maintained despite header filtering and manipulation. Other effects were also observed, such as the load placed on the CPU. Screen captures can be viewed in Appendix Section 6.3.

The SIP filter tested was a complex one and commonly used in VoLTE environments to strip preconditions from both the message and body. It was run against one SC VM pair to measure the MOS score impact.

6.3.1 SIP Filter to Manipulate SIP Message Header and Body Description Result

Create a SIP filter to manipulate the SIP header • Call rate: 40 cps and body. • Call duration: 130 sec • MOS: 4.16

Create a SIP filter to manipulate the SIP message • Call rate: 40 cps header and body. Verify its effect using the IMS • Call duration: 130 sec call trace tool. • MOS: 4.16 • No effect

6.3.3 SIP Filter Verified with Call Processing Signaling Performance Description Result

Create a SIP filter to manipulate the SIP message • Call rate: 40 cps header and body. Verify its effect by observing • Call duration: 130 sec any impact to call processing signaling • MOS: 4.16 performance. • 0.25% CPU increase on 100% calls

6.4 WebRTC

WebRTC technology enables HTML-5 based web browsers with Real-Time Communications capabilities via simple JavaScript APIs and without the need for plugins. WebRTC enhances the customer experience, moving away from the traditional communications experience to deliver communications contextually in web-based applications and websites. WebRTC also offers new monetization opportunities providing a suite of WebRTC APIs that developers can use to create new web applications with universal access from legacy or IP networks.

Our testing showed Nokia SBC is fully supportive of WebRTC, providing a secured transport and interworking functions with IMS. Additionally, we experienced the Nokia WebRTC in-browser communication API, in-browser collaboration API and device-switching API. Using these easy- to-use APIs, web developers can augment the multi-device service to multi-device communications via web applications that deliver voice, video, chat, and share features.

WebRTC-enabled in-browser communications

Registrations and calls can be setup and broken down using mobile or web applications on desktop devices, involved parties can chat and collaborate during the call and calls can be pushed or pulled from one device to the other.

This flexibility was demonstrated using three test cases:

6.4.1 WebRTC Call with Audio-Transcoding Description Result

A video call with audio was placed between • Client A: OPUS audio and VP8 video codec two WebRTC clients over DTLS/SRTP using • Client B: G.711 audio and VP8 video codec Nokia WebRTC ORCA client on Chrome • Connection made and media shared browser. One client uses OPUS codec and the other uses G.711. Verify successful demonstration.

A video call with audio was placed between two • Client A: OPUS audio and, VP8 video codec WebRTC clients over DTLS/SRTP using Nokia • Client B: G.711 audio and, VP8 video codec WebRTC ORCA client on Chrome browser. One • Connection made and media shared client uses OPUS codec and the other uses • Chat session started and a 78K image file was G.711. sent from Client A to Client B. • Chat session was placed • Chat session was closed and video chat • File was sent and opened resumed. • Chat session was closed • Repeated using Mozilla Firefox and observed • Verify use with different browser the same results.

6.4.3 WebRTC with Slack WebUI Description Result

Nokia’s developer partner Quobis leveraged • At user login on Slack, the WebRTC-enabled the Nokia’s WebRTC APIS to enable WebRTC Slack client is registered on IMS network on Slack, a web application for team’s • Incoming call was synchronized on Slack and communication and collaboration. With such mobile device (simultaneous ringing) integration Slack becomes a SIM-less • Call was answered on Slack WebRTC client secondary mobile device. • Call was then successfully pushed from Slack • A call was made from the legacy network UI to the mobile device. to a mobile number • After call was answered by the mobile device, • The call terminated on Slack and mobile the call was successfully pulled from Slack UI devices and moved in the browser. • The call was answered on Slack • Flexibility of communication was shown to • Call was pushed to mobile extend to any device with a browser. • Call was answered on mobile • Call pulled back on the browser

7.1 SBC VNF Cloud Orchestration

In this series of tests we observed the Nokia SBC VNFs orchestration and life cycle management using CBAM.

CloudBand Application Manager

The CBAM GUI allows the user to install and deploy, or extract the following SBC VNF components: OA&M, SC, BGC, FW, CFED, DFED, iCCF, SCM, PIM and MCM. For additional screenshots, see Appendix Section 7.1

7.1.1 Deploy Description Result

Verify CBAM operation to install, deploy or The following VNF components were extract SBC VNF components. successfully installed, deployed or extracted: OA&M, SC, DFED, BGC, FW, CFED, iCCF, SCM, PIM and MCM.

CBAM standard lifecycle operations

The CBAM standard lifecycle operations provide healing function, software upgrade and scale functions. For additional screenshots, see Appendix Section 7.1.

7.1.2 Heal Description Result

Verify CBAM Heal operation on SBC media PIM VM was successfully healed and recovered plane (PIM VM). to “in-service” using CBAM heal operation. See Appendix Section 7.1.2 for more screenshots.

7.1.3 Software Upgrade Description Result

A software update was performed for both Verify CBAM Software upgrade operation on signaling and media planes. The signaling plane both SBC signaling and media VMs. was able to evolve a database from an Verify CBAM software roll back operation on older version to a newer version using the both SBC signaling and media VMs. software upgrade.

The roll back of this software was also verified.

7.1.4 Scale Description Result

Verify CBAM procedure for scaling SBC media MCM was successfully scaled-out using CBAM plane. Add one additional pair of MCM VM scale operation. during transcoding work load. • Transcoding background call load • Call rate: 40 cps • Call duration: 130 sec • Scale-out: from 3 active (+ 1 stand-by) to 4 active VM (+ 1 stand-by)

See Appendix Section 7.1.4 for more screenshots.

7.2 SBC Operations, Administration and Management

The OA&M oversees all element activity. It controls all VM blades. The dashboard provides many display options, analysis tools and controls.

7.2.1 Performance Charts in OA&M UI Dashboard Description Result

Display the following Performance Management Charts were displayed and verified. For various (PM) charts: screenshots of the OA&M UI, see Appendix • PM data display with area zoom Section 7.2.1. • PM data display with zoom out • Restore All charts contained correct data, and the • Line chart interface was easy to navigate. • Histogram • Data review • Save as image • Charts with large PM data

Verify the following: • Successfully logged into WebUI • Login to WebUI • Used PM for measuring traffic, filterable by • Reports Scheduling many variables • Add Traffic Job • PM reports were verified as correct and • Modify Traffic Job filterable • Traffic schedules could be created based on measurement type and modified if disabled

Data was correct and easily navigated. For more screenshots, see Appendix Section 7.2.2.

The PM report allowed for measurement of all traffic, filterable by type, time, object ID, granularity, value and flag. Traffic job schedules measured groups of traffic. Groups could be based on VM, resources, policies or other components. These jobs could be created, enabled or disabled. Only disabled jobs could be modified.

Verify the following: Viewed, filtered and cleared alarms. For • Alarm view additional screenshots, see Appendix 7.2.3. • Alarm filter • Alarm meaning • Alarm clearing

Alarms were viewed and filterable by severity, time, user, event, cause and problem. Details for the alarm were available to determine the meaning for the alert. Alarms were viewable in a chart, to display type and frequency.

7.2.4 Fault Management Alarm Filter Configuration Description Result

Log into OA&M, click an alarm from the table. In All alarm functionality and visibility was the Filter tab, configure alarm setting observed and verified. For additional parameters in alarm creation page and save. screenshots, see Appendix Section 7.2.4. Verify the following: • Filter page is shown • All filters are shown on the left and Create Filter page is shown on the right • The new alarm is shown in the filter list • After clicking Choose Filter selector in the alarms page, the new filter is shown

The Alarm table showed all current alarms. These were filterable by label, type, cause and problem. They could be sorted by severity or time.

7.2.5 OA&M Call Trace Tool Description Result

Create two call trace records with trace types SIP Call trace functionality was verified. See URI and tel num. Make a call using the same Appendix Section 7.2.5 for screenshots. subscriber, then stop the call trace using SIP URI manually. Let another call trace using tel num stop automatically, when the duration time expires. Verify the following: • Two call traces can be stopped successfully and displayed in WebUI • Download one call trace • Content can be captured in call trace for the basic call

7.2.6 OA&M Backup Description Result

Confirm that multiple backup output packages All backup creation and details were observed in can be displayed in WebUI. Create multiple the WebUI, verified. backup jobs or create one job to generate multiple backup packages. Verify that all output can be displayed in WebUI.

Backup jobs were created under OA&M Tool Backup Management > Scheduling.

Backup jobs were displayed in the Scheduling list. These jobs could be enabled or disabled at any time.

Details of each backup were available.

The status of each backup was listed under Backup Management > Status.

7.2.7 OA&M Signaling Components and Inventory Description Result

Verify the status of signaling components: All components were verified and found useful • Host services for visibility of signaling components. • Service members • Components For additional screenshots, see Appendix • Diameter links Section 7.2.7. • SIP links • Network links Check system inventory.

Host services were displayed under the State tab and indicated its status, actions and other information.

Information was shown for each service network interface. The CNFG interface is shown above.

7.2.8 OA&M Media Plane Status Description Result

Open the WebUI State Management and verify Using the Node Status > Virtual Media Gateway, registered status of the MGC media plane. we verified the registered status of the MGC media plane.

Media plane status was displayed and verified as registered.

7.2.9 OA&M SIP Screening Description Result

Use the WebUI for SIP Screening related tables. The WebUI was used to view modified signaling • Check that modifications are synced to configuration and filter. signaling plane • Add filter to the filter set • Assign filter set ID to P-CSCF profile

Using Signaling > SIP Screening > SIP Filter, the signaling plane was viewed and filtered.

7.3 SBCs Cluster Management with NetAct

The Nokia NetAct is an optional web-based tool that enhances the network and SBC experience as a real-time management system, dashboard and report generator of network performance. The centralized monitoring system provides visibility into every individual SBC or group of SBCs. Data is collected from the manage networks to provide the following building blocks:

Source: Nokia NetAct building blocks

The customizable NetAct monitor detects incidents with alarm and filtering technology. Alarms can be monitored for individual SBC or group of SBCs by grouping multiple SBCs into SBC Cluster as shown below.

Source: Nokia SBCs cluster alarm monitoring

The NetAct Performance Manager tracks network performance and identifies bottlenecks with reporting tools, such as reporting suites, report creator, KPI builder, threshold and profiler and NetAct Self Monitor reporting.

Dashboards display performance data in real-time for flexible management and adjustment. Time Overlay uses timestamps to make multiple report comparisons, and any report can be grouped or filtered based on specific parameters. Also PM report can be displayed for individual SBC or group of SBCs by grouping multiple SBCs into SBC Cluster.

SBCs cluster performance monitoring

Cloud-based Nokia SBC Performance Verified 40 DR170831E Copyright ©2017 Miercom 25 September 2017 About Miercom Miercom has published hundreds of network product analyses in leading trade periodicals and other publications. Miercom’s reputation as the leading, independent product test center is undisputed. Private test services available from Miercom include competitive product analyses, as well as individual product evaluations. Miercom features comprehensive certification and test programs including: Certified Interoperable, Certified Reliable, Certified Secure and Certified Green. Products may also be evaluated under the Performance Verified program, the industry’s Customer Use and Evaluation We encourage customers to do their own product trials, as tests are based on the average environment and do not reflect every possible deployment scenario. We offer consulting services and engineering assistance for any customer who wishes to perform an on-site evaluation. Use of This Report Every effort was made to ensure the accuracy of the data contained in this report but errors and/or oversights can occur. The information documented in this report may also rely on various test tools, the accuracy of which is beyond our control. Furthermore, the document relies on certain representations by the vendors that were reasonably verified by Miercom but beyond our control to verify to 100 percent certainty.

This document is provided “as is,” by Miercom and gives no warranty, representation or undertaking, whether express or implied, and accepts no legal responsibility, whether direct or indirect, for the accuracy, completeness, usefulness or suitability of any information contained in this report.

All trademarks used in the document are owned by their respective owners. You agree not to use any trademark in or as the whole or part of your own trademarks in connection with any activities, products or services which are not ours, or in a manner which may be confusing, misleading or deceptive or in a manner that disparages us or our information, projects or developments.

© 2017 Miercom. All Rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the authors. Please email [email protected] for additional information.

6.2 Support of Enhanced Voice Services (EVS) Codec

Figure 1: Session Description

Figure 2: Call Trace List

Figure 4: Transcoding Measurement Information

Section 6.3.1 SIP Filter to Manipulate SIP Message Header and Body

Figure 5: Precondition SIP Filter

Figure 6: SIP Filtering

Figure 7: SIP Filtering with IMS Call Trace

Figure 8: Before SIP Filtering Verified with IMS Trace Tool

Section 6.3.3 SIP Filter Verified with Call Processing Signaling Performance

Figure 10: Performance Measurement Report (without filter)

7.1 VNF Cloud Orchestration

Figure 12: CBAM GUI Operation Status

Figure 14: CBAM GUI Trigger - Signal and Media Plane Application Level Back Up

Figure 15: CBAM GUI - SBC Restore for Signal and Media Planes

Figure 16: CLI for SBC Heal

Figure 17: Media PIM VM Malfunction

Figure 18: Heal Process for Media PIM VM

Figure 19: Node Status after Heal

Figure 20: Healed Media PIM VM

Figure 21: CLI for Information on Media Planes

Figure 22: Planes to Scale

Figure 24: Scaling Complete

Figure 25: Verified Scaling Finished

Figure 26: Confirmed Scale

Section 7.2.1 Performance Charts in OAM UI Dashboard

Figure 27: OAM Web User Interface

Figure 28: PM Zoom

Figure 30: Signaling Chart Configuration

Figure 31: Media Chart Configuration

Figure 32: PM Report Selection and Filter

Figure 33: Traffic Job Creation

Figure 34: Traffic Job Enable/Disable

Figure 35: Alarm Frequency Chart

Figure 36: Recovery Procedures to Clear Alarms

Section 7.2.4 Fault Management Alarm Filter Configuration

Figure 37: Alarm Filters (Left) and Parameters (Right)

Figure 38: Call Trace List

Figure 39: Call Trace Create

Figure 41: OAM Signaling Status

Figure 42: OAM Service Member Status

Figure 44: OAM Diameter Status

Figure 46: OAM Network Interface