MOBILE BENCHMARK
After conducting mobile network tests in the Netherlands in 2015 and 2016,
the network testing professionals of P3 communications and connect have Groningen Leeuwarden examined the Dutch mobile networks for the third time. As all four operators Winschoten Drachten showed excellent results in the previous tests, the race is all the more Sneek Gieten exciting in 2017. How did the four candidates perform? Joure Heerenveen
Emmen
Alkmaar
Amsterdam THE 2017 Epe Naarden Hilversum Leiden Apeldoorn Enschede Waddinxveen Barneveld The Hague Utrecht Woerden Arnhem Gouda Elst P3 CONNECT Roerdam Nijmegen Gorinchem
Breda
Tilburg MOBILE Bergen op Zoom Eindhoven Venlo
Roermond
BENCHMARK Geleen IN THE NETHERLANDS Valkenburg MOBILE BENCHMARK
RESULTS IN Coming from strong results in the 2016 P3 connect Mobile Benchmark in the Nether- lands, the Dutch operators were able to main- A NUTSHELL tain their impressive results in 2017. However, we observed some interesting changes too. P3‘s network benchmarks are widely accepted for setting industry standards as well as being highly objective. The carefully designed methodology scheduled four drive- test cars to visit 17 large cities and 19 smaller T-Mobile towns in the Netherlands as well as the con Vodafone KPN Tele2 necting roads. The areas in which we tested max. 1000 Points accounted for more than 5.1 million people, or about 30 per cent of the Dutch population. All four Dutch operators achieved very good results with more than 900 out of a possible total of 1000 points. The overall winner is T-Mobile, surpassing even the 950 points threshold and thus receiving the rare grade „outstanding“. The test winner shows convin- cing results both in the data and in the voice 575 category. The latter is especially remarkable, 553 571 556 as T-Mobile was the only Dutch operator at DATA the time of testing that did not yet support max. 600 the modern Voice over LTE (VoLTE) technology. With a considerable improvement in its voice performance over the 2016 result, Vodafone ranks second and is able to outrank the market leader KPN. In the data category, VOICE max. 400 381 378 359 363 Vodafone performed a little behind KPN. However on the whole, the differences Total Score 956 931 930 919 between the competitors are small. KPN, which ranked second in 2016, is beaten Grade outstanding very good very good very good by Vodafone with a gap of only one point and ranks third. The Dutch market leader already Shown voice, data and total scores are rounded. offers VoLTE, however at the time of testing did not yet support this feature on the Samsung S7 Overall Results Voice and Data T-MobileVodafone KPNTele2 phones that were used for this benchmark. So VOICE max. 400 Points 381378 359363 the voice results of KPN are based on legacy es Drivetest 240 96%94% 91%93% telephony. Apart from long call setup times, KPN achieves very good voice results and also Towns Drivetest8095% 96%90% 88% delivers a very good data performance. RoadsDrivetest 80 94%92% 88%86% The Netherlands‘ smallest operator Tele2 DATA max. 600 Points 575553 571556 ranks last, but still with very competitive es Drivetest 360 96%92% 95%93% results. A distinct loss of points in the voice Towns Drivetest 120 95%90% 95%90% category might be explained by the dis RoadsDrivetest 120 96%94% 96%95% advantageous effects of national roaming for voice services on the T-Mobile network. TOTAL max. 1000 Points 956931 930919
2 MOBILE BENCHMARK
The Dutch mobile network market is characterised by distinct competition and a generally high level of performance. The smallest operator Tele2 cooperates with T-Mobile for 2G/3G voice but made clear progress in its LTE deployment. THE DUTCH OPERATORS
The Koninklijke PTT Neder- In 2000, the German company land N.V. emerged from the T-Mobile bought a minority of privatisation of the formerly the Dutch mobile network ope- state-owned PTT in 1998. rator Ben, which two years la- With about 8 million mobile ter was extended to a 100 per customers, KPN claims to cent acquisition. In 2003, Ben have a market share of 44 per was renamed T-Mobile cent in the Netherlands at the Netherlands, with the brand end of 2016. With these num- Vodafone Libertel B.V., the “Ben“ becoming a “no-frills“ The Swedish telecommunica- bers, the company is by far Dutch subsidiary of the inter- offer within its portfolio. In tions operator Tele2 acquired the largest Dutch mobile ope- national Vodafone Group ac- 2007, T-Mobile Netherlands the former Versatel N.V. in rator. For 2016, the company quired the formerly indepen- additionally acquired the ope- 2005. The resulting Tele2 reported a total revenue of 6.8 dent operator Libertel in 2003. rator Orange which until then Netherlands Holding N.V. is a billion Euros. Today, Vodafone is the second had belonged to France Télé- 75 per cent subsidiary of its The company focuses on largest mobile operator in the com. T-Mobile NL is a full sub- Swedish parent company. Ori- marketing its KPN brand, Netherlands with around 5.2 sidiary of the German Deutsche ginally acting as a MVNO (mo- however with Simyo, Telfort million mobile customers cor- Telekom. At the end of its fiscal bile virtual network operator), and Ortel it also has offerings responding to a mobile market year 2015, the company repor Tele2 nowadays operates its in the “no-frills“ segment. share of about 29 per cent. ted 3.7 million customers and own infrastructure and has a KPN operates 2G networks In February 2016, Vodafone a revenue of 1.4 billion Euros portfolio of fixed telephony, at 900 and 1800 MHz, 3G at Netherlands announced a which equals a mobile market data, internet and mobile tele- 900 and 2100 MHz and 4G/ merger with the cable and fibre share of about 21 per cent. phony products. At the end of LTE at 800, 1800 and 2600 operator Ziggo previously owned T-Mobile Netherlands ope- 2016 Tele2 reported about 1 MHz. The company claims a by Liberty Global. The company rates 2G mostly at 1800 MHz, million mobile subscribers 99 per cent 4G availability all now operates under the name 3G at 900 and 2100 MHz and which equals a Dutch market over the country. Where KPN VodafoneZiggo. As regulatory 4G/LTE at 900, 1800, 2100 share of roughly 6 per cent. already offers carrier aggrega- conditions required Vodafone and 2600 MHz. The company Tele2 operates its own LTE tion, the theoretical maximum to separate its former fixed-line claims a 4G coverage of more network at 800 and 2600 MHz. speed of its LTE network is business, this entity then be- than 99 per cent of the Dutch After considerable efforts to 300 Mbit/s. KPN markets came „T-Mobile Thuis“. Voda population. With carrier increase coverage, Tele2 now this configuration as “4G+“. fone‘s mobile revenues for the aggregation, T-Mobile’s 4G+ provides a 4G network to lar- Otherwise, the top LTE speed 2016/2017 fiscal year were network offers a maximum ge parts of the country, with is at 150 Mbit/s. still due at the time of writing. speed of 262.5 MBit/s. some gaps especially in the The company operates 2G west and south. The operator networks (GSM, GPRS, EDGE) even offers carrier aggrega at 900 and 1800 MHz, 3G tion with up to 225 MBit/s at (UMTS up to HSPA+) at 2100 some locations. As the com- MHz and 4G/LTE at 800, 1800 pany had to pay lower license and 2600 MHz. Vodafone fees for its 4G spectrum than claims to offer more than 95 its competitors, it can offer ag- per cent coverage with 4G (up gressive prices. Tele2 coopera- to 150 Mbit/s). In a number of tes with T-Mobile NL by loca- Dutch regions the company tion sharing as well as natio- offers “4G+“ with 225 Mbit/s. nal roaming for voice services.
3 MOBILE BENCHMARK
VOICE 40%
Call Setup Time A CLOSE LOOK AT THE Call Success Rao DUTCH NETWORKS Speech Quality For the third time in a row, P3 and connect have conducted a mobile network test in the Netherlands. How did the traditionally very reliable and powerful Dutch networks score in 2017? DATA P3 communications GmbH, In 2016 alone, P3 compiled 60% based in Aachen, Germany, is a more than 60,000 measurement world leader in mobile network hours in 65 countries on five testing. It is part of the P3 group, continents, with its test vehicles Web Page Download with over 3000 employees world covering more than one million kilo wide and a turnover of more than metres. As the de-facto industry Success Rao €300 million. P3 partners with standard, the P3 benchmarking the German telecommunications methodology focuses on custo- magazine connect, which has mer-perceived network quality Youtube Quality more than 20 years of editorial – examining both voice telephony experience and is one of the as well as data connectivity. P3‘s leading test authorities in Europe network benchmarks are widely File Up- and Download for telecommunications products accepted as an objective authority. and services. Together, P3 and connect How did the powerful Dutch have been conducting the most networks score this time? important mobile network In the previous two mobile “The Netherlands is a very competitive benchmark test in Germany benchmarks, that we conducted and innovative market and all Dutch for 15 years, extending it to in 2015 and 2016, the Dutch Austria and Switzerland in 2009. mobile operators showed very operators prove to have excellent Since 2014, P3 has also been good results. Moreover, they networks. It will be interesting to conducting network benchmarks were able to improve their per- see how they will answer future in Australia and the UK. The formance year over year. So P3 challenges like IoT readiness Netherlands and Spain followed and connect were very curious Hakan Ekmen, and autonomous driving. in 2015 and Sweden in 2016. In whether this trend would continue Managing Director of 2017, P3 and connect examine the in 2017 – and how the four can- P3 communications Dutch networks for the third time. didates would score this time. GmbH.
4 MOBILE BENCHMARK
Many users may prefer in- ternet messaging to traditio nal voice calls. But in those cases when they are talking on the phone, customers expect reliable connections and good speech quality. How do the Dutch networks VOICE fulfill these expectations? When the tests started, T-Mobile was the only Dutch operator that had not commercially launched VoLTE yet, and KPN did not yet sup- 400 of 1000 Points 600 of 1000 Points port this feature on the Samsung S7 smart- phones that were used for testing. The network- VOICE DATA wide introduction of Voice over LTE and its 400 of 1000 Points 600 of 1000 Points support on a broad range of smartphones take 96% DATA 96% time. So, the test phones were only partly able VOICE of 360 P. to use VoLTE, even if they were in areas where 94% 92% they could establish a LTE connection. 96% 96% 91% Cies of 360 P. 95% 94% of 240 P. 92% T-Mobile first with legacy technology, Voda- 93% Drivetest 93% fone shows most significant improvement91% Cies 95% of 240 P. Despite not offering VoLTE yet, T-Mobile is93% the Drivetest 93% overall voice leader. The operator thus proves 95% 95% of 120 P. that fast call setup times and excellent speech95% 96% 95% 90% quality can be achieved with legacy voice of 120 P. 96% 90% 90% 95%
technology too. But Vodafone, which has a of 80 P. Towns quota of 99.7 per cent of successful VoLTE 90% Drivetest 95% of 80 P. 88% Towns 90% calls, follows closely at only three points dis- 88% Drivetest 90% tance. Also, Vodafone shows the most signifi-
cant improvement over last year‘s result, gai- 94% 96% of 120 P.
94% 96% of 120 P. ning 15 points in the voice discipline. Tele2 92% 94% supports VoLTE in its 4G network, but outside92% 94% its LTE footprint it realises legacy voice services88% 88% Roads Roads 96% 96% of 80 P. of 80 P. Drivetest via national roaming on T-Mobile. In this sce 86% 86% Drivetest 95% 95% nario, Tele2 does not have access to the higher quality voice codecs, which may explain its lag compared to the two leading operators. Due T-Mobile Vodafone KPN Tele2 to the lack of VoLTE support on the Samsung T-Mobile Vodafone KPN Tele2 S7 test phones on the part of KPN, voice con- nections in this network were established via
circuit-switched fallback. Still, the largest Dutch Voice - DrivetestT-Mobile Vodafone KPNTele2 operator delivers very good speech quality and es OverallOverall LegacyVoLTE Overall Overall LegacyVoLTE high success ratios in all examined scenarios. Call Success Ra o (%) 99.7 99.4 99.199.8 99.7 99.6 99.5 99.7 Call Setup Time (s)3.5 3.4 4.52.4 6.03.9 5.32.4 VOICE AT A GLANCE Speech Quality (MOS-LQO) 3.53.5 3.53.6 3.63.3 3.03.6 The overall winner in the voice disci- Towns pline is T-Mobile – although this ope- Call Success Ra o (%) 99.6 99.8 99.799.8 99.5 98.7 98.8 98.5 rator is the only one in the Netherlands Call Setup Time (s)3.4 3.4 4.52.5 5.83.9 5.22.5 Speech Quality (MOS-LQO) 3.53.6 3.53.6 3.63.3 3.03.5 that has not yet introduced VoLTE. Roads Vodafone follows closely behind, while Call Success Ra o (%) 99.2 98.8 98.199.4 99.0 97.6 98.0 97.1 Tele 2 loses points due to legacy natio Call Setup Time (s)3.8 3.7 5.02.6 6.24.1 5.52.5 nal roaming. KPN offers VoLTE, but not Speech Quality (MOS-LQO) 3.43.5 3.53.5 3.53.2 2.93.5 yet on the S7 test phones. If the eva- luation had considered only legacy calls, The voice evaluation in this benchmark includes the results of both legacy and VoLTE calls in those networks where we could actually measure VoLTE. However, in order to allow additional KPN definitely would have ranked better. in-depth comparisons we have additionally specified the results of both connection types.
5 MOBILE BENCHMARK
The volume of transmitted data is steadily growing. So, all operators face the challenge to keep data rates and reaction times at a high level. Which Dutch operator manages to best DATA meet the growing demand? Good news for Dutch customers:400 of 1000 All Points 400 of 1000600 ofPoints 1000 Points 600 of 1000 Points mobile networks operating in the Nether- lands show a very high degreeVO of stabilityICE VOICE DATA DATA and performance in this benchmark. Whether our test teams drove through 96% 96% 96% 96% of 360 P. large cities, through smaller towns or on of 360 P. 94% 92% 94% the connecting roads – the success ratios 92% observed in each of these scenarios were 91% Cies 95% 91% altogetherof 240 P. remarkably high. Cies 95% of 240 P. 93% Drivetest 93% 93% In order to check the reliability and per- Drivetest 93% formance of data connectivity in the Dutch 95% 95% 95% networks, one Samsung Galaxy S7 per 95% of 120 P. operator was installed96% in our test cars. The of 120 P. 90% 96% 90% phones constantly performed90% a suite of 95%
of 80 P. Towns 90% tests including web-page downloads, file Drivetest 95% of 80 P. 88% Towns 90% downloads and uploads as well as Youtube Drivetest 88% video playbacks. For most of these checks, 90%
success ratios and94% average session times 96% of 120 P.
94% were logged. For the92% downloads and uploads 96% of 120 P. 94% 92% we also recorded the88% average throughputs. Roads 94% 96% In order to of 80 P. get an idea of the typical perfor- Drivetest 88% 86% Roads 96% 95% of 80 P. mance, we determined the minimum data 86% rates that are available in 90 per cent of the Drivetest 95% cases. To also reflect peak date rates, we additionally recorded the speeds that were T-Mobile Vodafone KPN Tele2 surpassed in 10 per cent of the cases. P3‘s approach for testing Youtube playT-Mobile- VodaDatafone in Ci esKPN - DriveteTest le2 T-Mobile Vodafone KPNTele2 back recognizes that this popular video Web-Pa c) service has recently introduced adaptive bit Success Ra o (%/%) 99.9/99.8 99.7/99.999.6/100.0 99.7/99.9 rates. This decision of the streaming provider Avg. Session Time (s/s)2.8/1.0 3.1/1.22.8/1.0 2.7/0.9 aims at a better user experience, surren- File Download (3 MB) dering pixel resolution in favour of stable Success Ra o/Avg. Session Time (%/s) 100.0/0.8100.0/1.1100.0/0.9100.0/1.0 playback. As a consequence, besides 10% faster than (kbit/s) 68926 5357174074 72072 success ratios, start times and the absence 90% faster than (kbit/s) 21101 125121699514889 of interruptions, the average value of the received video resolution became another File Upload (1 MB) important performance indicator. Success Ra o/Avg. Session Time (%/s) 99.4/0.799.7/1.2100.0/0.899.6/1.8 10% faster than (kbit/s) 24242 204452605924096 T-Mobile also leads the field in data 90% faster than (kbit/s) 7422 3558 6457 2138 Although all four operators reach very File Download (10 Seconds) high scores in the cities, T-Mobile takes Success Ra o (%) 100.099.999.999.8 a distinctive lead over its competitors. Avg. Throughput (kbit/s)77719 490536394362041 Here, KPN reaches the second rank with a 10% faster than (kbit/s) 143182 85243119424 95366 gap of only five points to T-Mobile. Tele2 comes in third in the cities. However, slightly 90% faster than (kbit/s) 30068 163482129218869 weaker values particularly in the file upload File Upload (10 Seconds) category lead to clear distance to the Success Ra o (%) 100.099.9100.099.9 results of the two candidates on the higher Avg. Throughput (kbit/s) 28117 203582341016456 ranks. With a very close gap of only one 10% faster than (kbit/s) 43440 414354243939457 point behind Tele2, Vodafone comes in last 90% faster than (kbit/s) 9213 4137 6825 2279 in the cities. This position originates in the Youtube Video loss of a couple of points especially in the web Success Ra o/Start Time (%/s) 99.9/1.499.9/1.699.9/1.699.9/1.4 browsing and file upload categories. Again, Video playouts without int ons (%) 100.0100.0100.0100.0 we must emphasize that this ranking order >> Average Video Resolu on (p)704 703701 686
6 MOBILE BENCHMARK
Data in Towns - DrivetestT-Mobile Vodafone KPNTele2 Web-Pa c) Success Ra o (%/%) 99.5/99.4 99.5/99.999.7/100.0 99.1/99.4 is achieved at very high levels of performance and Avg. Session Time (s/s)2.8/1.0 3.2/1.32.8/1.0 2.8/1.0 stability, and the overall differences between all File Download (3 MB) four networks are small. Success Ra o/Avg. Session Time (%/s) 100.0/0.7100.0/1.6100.0/0.999.7/1.2 In towns, we see an interesting ranking where T-Mobile and KPN share the same number of 10% faster than (kbit/s) 69164 488807362068027 points at the top rank, while Vodafone and Tele2 90% faster than (kbit/s) 24691 7891 1699713150 share the second position, again with an equal File Upload (1 MB) number of points. The distance to the leading duo Success Ra o/Avg. Session Time (%/s) 98.5/0.9100.0/1.4100.0/0.998.2/2.0 in this scenario is six points. 10% faster than (kbit/s) 23256 169492597421277 Basically, the same order can be observed on the 90% faster than (kbit/s) 5408 2972 4828 1924 connecting roads: Again, T-Mobile and KPN are leading the field at the same level, while Tele2 and File Download (10 Seconds) Vodafone rank second – in this scenario, however, Success Ra o (%) 100.099.7100.0100.0 with a very close distance of only two points. Avg. Throughput (kbit/s) 86323 404366212262477 All in all, T-Mobile leads the data category, follo- 10% faster than (kbit/s) 154939 79318115953 102340 wed closely by a strong KPN and at a little distance 90% faster than (kbit/s) 35167 104042178119842 by Tele2 and Vodafone. However, the fact that all File Upload (10 Seconds) four operators perform similarly well in all tested scenarios emphasises the high level of service and Success Ra o (%) 100.099.7100.099.5 performance that Dutch customers can expect. Avg. Throughput (kbit/s)24573 149002211213765 10% faster than (kbit/s) 42838 354364240236824 90% faster than (kbit/s) 6978 2631 5000 1727 DATA AT A GLANCE Youtube Video T-Mobile is also the overall winner in the Success Ra o/Start Time (%/s) 99.7/1.499.7/1.6100.0/1.6100.0/1.4 data category, but the gap over the other Video playouts without int ons (%) 100.0100.0100.0100.0 Dutch operators is small. KPN follows at Average Video Resolu on (p) 702 676700 675 close distance on a strong second rank, with Tele 2 and Vodafone falling a little be- Data on Roads - DrivetestT-Mobile Vodafone KPNTele2 hind. However, even the latter two provide Web-Pa c) very good results in the data category. Success Ra o (%/%) 99.1/99.0 99.4/99.799.3/99.699.6/99.7 Avg. Session Time (s/s)2.9/1.2 3.2/1.42.9/1.1 2.9/1.0 File Download (3 MB) Success Ra o/Avg. Session Time (%/s)100.0/0.9100.0/1.7100.0/1.2100.0/1.3 10% faster than (kbit/s) 68415 479047571066667 90% faster than (kbit/s) 17969 7334 111819926 File Upload (1 MB) Success Ra o/Avg. Session Time (%/s) 98.0/1.199.8/1.499.2/1.398.5/2.0 10% faster than (kbit/s) 23344 167542412620336 90% faster than (kbit/s) 3875 2745 3051 1618 File Download (10 Seconds) Success Ra o (%) 100.0100.0100.099.4 Avg. Throughput (kbit/s) 79001 354765736852681 10% faster than (kbit/s) 152729 71705119690 91946 90% faster than (kbit/s) 24538 8155 1465012367 File Upload (10 Seconds) Success Ra o (%) 99.699.499.899.6 Avg. Throughput (kbit/s) 24719 170861866413323 10% faster than (kbit/s) 43682 306423817933010 90% faster than (kbit/s) 4898 3418 3350 1858 Youtube Video Success Ra o/Start Time (%/s) 99.6/1.599.6/1.799.8/1.799.8/1.4 Video playouts without int ons (%) 99.8100.0100.0100.0 Average Video Resolu on (p) 699 688687 684
7 MOBILE BENCHMARK
The methodology of the P3 connect Mobile Benchmark is the result of more than 15 years of testing mobile networks, today in 65 countries. It was carefully designed to evaluate and objectively compare the performance and service quality of The Netherlands’ mobile networks from the users’ perspective. TESTING METHODOLOGY The P3 connect Mobile Benchmark acquired partly in 4G preferred to 3G in the Netherlands took place from preferred mode and partly in 4G February 27 to March 6, 2017. All preferred to 4G preferred mode. samples were collected between 8am As a consequence, the phones in and 10pm. The network tests covered most cases needed to switch (“fall larger cities, smaller towns and connec back”) to 2G or 3G when they were ting roads. The combination of test areas logged in to the LTE (so called “cir- had been carefully selected to provide cuit-switched fall back“ or CSFB). a significant series of test results cove- In order to account for typical ring the Dutch population. The areas smartphone usage scenarios during chosen for the 2017 test account for the voice tests, background data more than 5.1 million people, or about traffic was generated in a controlled 30 per cent of the total population. way through random injections of P3 conducted the tests with four small amounts of HTTP traffic. The drive-test cars, equipped with arrays voice test scores account for 40 per of twelve Samsung Galaxy S7 Cat 9 cent of the total benchmark results. Three boxes were mounted into the back and into the side windows of each measurement car in smartphones each. Eight phones were order to support twelve smartphones per car. used for the measurement of voice Data testing services, and four additional S7 cap- Data performance was measured tured the values for the data services. using four smartphones in each car – one per operator. The radio access Voice testing technology was set to LTE preferred Two smartphones per operator in mode in order to reflect the customer each car were used for the voice experience. The web tests accessed tests, setting up test calls from one ten web pages according to the widely car to another. The audio quality of recognized Alexa ranking. In addition, the transmitted speech samples was the artificial (static) “Kepler” test web evaluated using the HD-voice capable page as specified by ETSI (European and ITU (International Telecommunica- Telecommunications Standards Insti- tion Union) standardized so-called tute) for testing purposes was used. POLQA wideband algorithm (Perceptu- In order to test the data service al Objective Listening Quality Assess performance, files of 3MB and 1MB ment). All Dutch network operators for download and upload respectively offer 4G capable subscriptions. were transferred from or to a test ser- To take the high share of LTE into ver located on the Internet. In addition, account, speech samples were the peak data performance was >> DATA VOICE 60% 40%
Web Page Download Call Setup Time Each box housed four smartphones allowing the simultaneous testing of four mobile operators. Success Rao Call Success Rao Youtube Quality
File Up- and Download Speech Quality
8 MOBILE BENCHMARK
tested in uplink and downlink directions by assessing the amount of data that was transferred within a 10 seconds time period. Another discipline was the playback of Youtube videos, taking into account that Youtube dynamically adapts the video resolution to the available band- width. So, in addition to the success ratios, start times and playouts without interrup tions, the Youtube measurements also determined the average video resolution. All the tests were conducted with the best-performing mobile plan available from each operator. Data scores account for 60 per cent of the total results.
Routes and samples The test routes are shown on page 1 of this report. In the 17 big cities and 19 smaller towns indicated, the cars had to follow predefined routes. Altogether, the four test cars covered more than 7900 kilometres, of which approxima tely 3500 km led through the big cities, while 4400 km were covered in smaller towns and on connecting roads.
Performance indicators and rating The score weighting reflects both the geographical distribution of the Dutch population and the ranking of usage sce narios. Therefore, 600 of 1000 maximum points achievable were assigned to the SCORE BREAKDOWN cities – divided into 240 points reflecting the voice results and 360 points Cies 600 Towns 200 Roads 200 reflecting the data results. For the towns and the roads, 200 points each are available. In both categories, those are divided into 80 points referring to DATA the voice and 120 points referring to VOICE the data category. The charts on page 2 and page 10 of this report show the per- centage of maximum points that each operator achieved in each discipline. DATA VOICE TOWNS
DATA VOICE ROADS
CITIES
Hakan Ekmen, Managing Director of P3 commu nications GmbH and Bernd Theiss, Head of connect’s test lab, inspect the testing equipment.
9 Numbers in % of achievable maximum score
MOBILE BENCHMARK CONCLUSION All four Dutch operators score well over 900 out of 1000 points. Overall winner T-Mobile even surpasses the 950 points threshold, achieving the rare grade „outstanding“. 2015 The results of the P3 connect mobile 2016 benchmark in the Netherlands had already 2017 97 96 88 92 93 88 94 93 77 90 92 reached a very high level last year. In 2017, Operator T-Mobile Vodafone KPN Tele2 again, all four Dutch operators show a re markably high level of performance and The total score development over the last three years shows the very high level on which Dutch mobile stability. In a year-on-year comparison, Tele2 networks operate. Although there have been some shifts, the overall trend shows a constant improvement. was able to improve in all tested categories. Vodafone takes over the second rank from T-Mobile KPN, which mainly results from Vodafone‘s Vodafone KPN Tele2 distinct improvement in the voice category. max. 1000 Points If the evaluation had considered only legacy voice connections, KPN would have ranked better. When it comes to data connectivity in the cities, T-Mobile and KPN lead the field, with Tele2 and Vodafone trailing a little behind, but still scoring at a very high level. In the smaller Dutch towns, the lead of the stronger two 575 operators T-Mobile and KPN is slightly more 553 571 556 pronounced. On the connecting roads, the DATA distance between all four candidates is so max. 600 close that no operator is a bad choice. When comparing the results of this year‘s mobile network benchmark in the Netherlands to those from recent tests in other European VOICE countries, Dutch operators are among the top max. 400 381 378 359 363 tiers. Even the fourth-ranking Tele2 scores at a level that would make it a suitable candidate Total Score 956 931 930 919 for a winning position in countries with overall Grade outstanding very good very good very good weaker mobile networks like Spain or the UK. Shown voice, data and total scores are rounded. 1 2 3 4 As in 2016, the third largest Dutch With a considerable improvement In a tight competitive environment, The Netherlands’ smallest opera- operator also wins the 2017 in its voice performance, Vodafone the largest Dutch operator has to tor scores last in this benchmark. benchmark with a lead in the voice is able to outrank the market leader give way to the smaller operators, However, Tele2’s results are still and data categories and the overall KPN in this year’s benchmark. In that rank on second and third posi very competitive and deserve the grade “outstanding“. The fact that the data category, this operator tion in terms of subscriber numbers. overall grade „very good“. The T-Mobile achieves the best result still achieves a third place. However, However, KPN is beaten by Voda- loss of points in the voice category in the voice category without offe the differences are small, and es- fone with a gap of only one point. can mainly be explained by the ring VoLTE shows that legacy voice pecially on the roads and to some But the Dutch market leader still disadvantageous effects of natio- services can also offer perfect extent in smaller towns, Vodafone achieves very good results both in nal roaming for legacy voice ser- calls, when they are handled well. holds up well with its competitors. the voice and data categories. vices on the T-Mobile network.
BEST IN TEST T-MOBILE 4/2017 MOBILE BENCHMARK THE NETHERLANDS www.connect-testlab.com
10 MOBILE BENCHMARK
Objective testing will be essential to the ongoing evolution of the Dutch networks as to emerging technologies like autonomous driving and smart cities. P3 communications is preparing for these future challenges. OUTLOOK As in the years before, all four through to completely new candidates scored really well in mobile communications techno- this year‘s mobile network test in logies like the upcoming super- the Netherlands. This, however, fast 5G networks. However, 5G‘s cannot be taken for granted. approach of assigning different Upcoming improvements like network capabilities to different the further extension of 4G use cases will also require an P3 communications is coverage, the ongoing network extended approach to network constantly monitoring the technological evolution with technologies like testing.This will involve concepts development. “4G Advanced“ and the conti like advanced analytics and nuing rollout of Voice over LTE big-data technology. (VoLTE) may change this picture. P3 communications is well Extensions to the test routines prepared to accompany the fu- Readers interested in taking ture developments of the Dutch part in our continuous efforts to telecommunications market. In evaluate the performance of this context, we are already pre- mobile networks can already do CROWDSOURCED NETWORK RATING paring to include new technolo- so today – by downloading the P3 communications is increasingly focussing on aspects like the retainability of voice services, the integrity of data gies and enhancements menti- “U get“ wireless performance ra- services and “operational excellence“. An important instru- oned above in next year‘s testing. ting app. See details on the right. ment for this approach is the “U get“ app that is available Furthermore, P3 is also preparing This crowdsourcing approach under uget-app.com or via the adjoint QR code. The An- for future challenges starting will give us valuable additional droid app checks and visualises current network performance. Join the global community of users who understand their per with new communications appli- insights into the user experience sonal wireless performance, while contributing to the world’s cations like autonomous driving and operational performance of most comprehensive picture of mobile customer experience. and smart cities (see below), mobile networks in the near future.
Autonomous Driving Smart Cities
The future of transport is quickly coming upon us – one kilometre Today, 54 per cent of the world’s population lives in urban areas, loop at a time. With each new generation of car models, the auto- a proportion that is expected to increase to 66 per cent by 2050. motive industry comes closer to its vision of highly automated Mobile communications will be an essential component to delive- driving cars. However, to ensure that driverless cars can maintain ring on smart city promises. To enable smart cities to thrive and connectivity and thus optimal performance and safety, we need host successful businesses in the digital era, their technological to ensure that the technological infrastructure can manage the infrastructure must be capable of managing the increased demand increasing demand that machines will place upon it. Therefore, on network usage. Therefore, it will be a future focus point of P3 autonomous driving scenarios play an important role in P3 com- communications to determine whether cities become truly smart munications‘ concepts for the evolution of mobile network testing. by taking an even closer look at their advances in connectivity.
11