Mobile Network Performance Benchmarking
Governorate of Dakhiliyah
Regulatory & Compliance Unit Quality of Service Department Contents
1. Background 2. Test Methodology 3. Performance Indicators Definition 4. Results 5. Conclusion
1. Background
A comprehensive field test was conducted independently by TRA to assess and benchmark the performance of Omantel and Ooredoo mobile voice and data networks in Al Dakhiliyah Governorate.
Field Survey Date & Time: 4th – 8th May 2015 from 9:00 A.M. to 10:00 P.M.
Services Tested
Network Service Technology
Omantel Voice 2G, 3G Data 3G, 4G Ooredoo Voice 2G, 3G Data 3G, 4G Test Area
Governorate Wilayat Dakhiliyah Bidbid, Samail, Izki, Nizwa (including Jabal Al Akhdar), Bahla, Al Hamra, Manah, Adam 2. Test Methodology
The following test configuration was used for measurements:
Service Technology Objective Test sequence KPIs measured Tested Mode
Omantel- Open (2G, To check network Calls of 60 sec duration with a 20 CSSR, CDR, CSR, Mobile voice 3G) accessibility, retain-ability, sec idle wait time between them to RxLev, RSCP. mobility, service integrity allow for cell reselection from 2G to and coverage 3G mode.
Omantel- Open (2G, To check data network HTTP file download from the Latency, Ping Packet Mobile data 3G, 4G) performance and service providers network and ping Success Rate, Avg. coverage test. downlink throughput, RSCP, RSRP. Ooredoo- Open (2G, To check network Calls of 60 sec duration with a 20 CSSR, CDR, CSR, Mobile voice 3G) accessibility, retain-ability, sec idle wait time between them to RxLev, RSCP. mobility service integrity allow for cell reselection from 2G to and coverage 3G mode
Ooredoo- Open (2G, To check data network HTTP file download from the Latency, Ping Packet Mobile data 3G,4G) performance and service providers network and ping Success Rate, Avg. coverage test. downlink throughput, RSCP, RSRP. 3. Key Performance Indicators Definition
Mobile voice performance was measured based on the following set of KPIs:
Call Setup Success Rate (CSSR)– This indicator is used to measure the percentage of calls successfully established without facing blockage in the network as a ratio of the total number of call attempts made to access and establish a voice call. (to check network accessibility)
Call Drop Rate (CDR) – This indicator is used to measure the percentage of calls dropped due to technical problems or coverage gaps in the service provider’s network as a ratio of the total number of calls successfully established. (to check network retain-ability)
Call Success Rate (CSR) – This indicator is used to measure the percentage of calls successfully established without facing blockage in the network as a ratio of the total number of call attempts made to access and establish a voice call and then successfully terminated from the user-end without being dropped or disconnected from the network side due to a technical irregularity. (to check service integrity)
Mobile data performance was measured based on the following set of KPIs:
Packet Latency - Packet delay, which represents the time taken for data packets to pass through the GPRS bearer in a round-trip time from the mobile to the server in the service provider’s core network and back to the mobile. (to check delay in the network)
Ping Packet Success Rate is the percentage of packets lost between designated routes in the network. It is used to indicate the loss of data packets during transmission over a telecommunications network. (to check data integrity)
HTTP Average downlink throughput - This is the average downlink throughput (rate at which data/bits are transferred to the user) experienced by a user while downloading content from the Internet. (to check download speed)
Performance Indicators Definition continued
Coverage is assessed based on the following radio parameters:
Reference Signal Received Power (RSRP) – This indicator measures the linear average of the received power on reference signal resource elements in the downlink during the drive test (to check 4G coverage).
Received Signal Code Power (RSCP) – This indicator measures the received signal code power of the pilot channel in the downlink during the drive test (to check 3G coverage).
Received Signal Level (RxLevSub) - This indicator measures the received signal strength in downlink during the drive tests (to check 2G coverage).
The following convention is used for the coverage plot.
Serving Cell RSRP (dBm) 3G CPICH RSCP (dBm) 2G RxLevSub (dBm)
Range Classification Penetration Range Range >= - 85 >= -80 >= -75 Excellent Indoor >= -95 and < -85 >=-87 and <-80 >=-82 and <-75 Good >= -105 and < -95 >=-100 and <-87 >=-95 and <-82 Fair In-Car >= 120 and < -105 <-100 <-95 Weak Outdoor Only 4. Results 4.1 Mobile Voice Coverage
Voice Coverage Distribution 100.00 90.00 80.00 70.00 60.00 50.00
PDF(%) 84.14 40.00 74.21 30.00 20.00 10.00 8.94 5.72 1.19 14.55 10.21 1.03 0.00 Omantel Ooredoo
Excellent Good Fair Weak
4.2 Mobile Voice Performance
Voice Performance Indicators 100 90 80
70
60 50 99.46 98 98.74 95 86.96 Value (%) Value 40 82.31
30
20
10 5.35 0.72 0.8 0 CSSR CDR CSR KPI
Omantel Ooredoo TRA Threshold
CSSR: Call Setup Success Rate (Higher is better) CDR: Call Drop Rate (Lower is Better)
CSR: Call Success Rate (Higher is better)
4.3 Mobile Data Download Speed
Overall Avg. Download Speed (Mbps)
30 30 5.08 5 5 4.61 100 100 0 0
Speed Distributon (Mbps)
Very Fast (5 to 100) Fast (2 to 5) Fair (0.5 to 2) Slow (0 to 0.5)
Ooredoo
Omantel
0 10 20 30 40 50 60 70 80 90 100
4.4 Mobile Data Performance
Overall Data Performance 250 100
99.95 200 99.94
99.9 150 99.85 217.83 100
99.8 Packet Latency(ms)
50 Ping Packet SuccessRate (%) 87.29 99.75 99.74
0 99.7 1
Packet Latency Omantel Packet Latency Ooredoo TRA Threshold (Packet Latency) Ping Packet Success Rate Omantel Ping Packet Success Rate Ooredoo
Packet Latency (ms): Lower is Better, Ping Packet Success Rate: Higher is Better
4.5 Mobile Data Performance Per Town/Wilayat
10 350 9 300
8
7 250
6 200 5 150
4 Packet Latency(ms) 3
Avg. Avg. Download (Mbps)Speed 100
2 50 1
0 0 Fanja Bidbid-Samail Izki Nizwa Jabal Akhdar Al Hamra Bahla Manah Adam Town/Wilayat
Avg. Downlink throughput (Mbps) Omantel Avg. Downlink throughput (Mbps) Ooredoo TRA Threshold (Packet Latency) Avg. Packet Latency (ms) Omantel Avg. Packet Latency (ms) Ooredoo
5 Conclusion
• In General, Omantel mobile network exhibits stronger voice coverage than Ooredoo throughout the Governorate of Al Dakhiliyah. • Ooredoo mobile network lacks basic voice coverage on the Jabal Akhdar access road from Birkat Al Mouz to the top of the mountain. • Ooredoo mobile network exhibits better voice performance than Omantel network with all KPIs meeting the TRA thresholds. • Omantel mobile network exhibits poor voice performance with all KPIs not-meeting the TRA thresholds. Call blocking is found to be significantly high throughout the Governorate of Al Dakhiliyah on Omantel network. • Omantel mobile data network has good LTE footprint throughout the main Wilayat centers. No LTE coverage was found on Ooredoo data network throughout Al Dakhiliyah Governorate. • Omantel mobile data network exhibits good data performance with overall high download speeds and low network delay which is within the TRA threshold. • Ooredoo mobile data network exhibits fair data performance with overall good download speeds but the network delay is extremely high and does not meet the TRA threshold in any Wilayat or Town.
Basis of results and conclusion
• The coverage information is based on the geographical drive test route used during the measurement exercise. • The exercise has been conducted independently by the Authority without sharing any prior information with the service providers about the type of tests being performed or location. • The results of the exercise are based on the data collected from the field at a certain instance of time and day; network behaviour may vary with traffic variations over time and events. • Industry standard tools and work best practices have been used during the whole exercise.