The 2020 Mobile Network Test in Spain

León Pamplona/ Vigo Iruña

Leganés

Valencia Albacete

Córdoba Murcia Cartagena Sevilla Jaén Málaga Jerez de la Almería Frontera Marbella THE 2020 Algeciras Drivetest Walktest MOBILE Roads NETWORK TEST IN SPAIN

For the sixth time, the benchmarking expert umlaut and connect magazine have conducted their authoritative While all operators in Spain strongly benchmark of the Spanish mobile worked on expanding their 4G net- networks. And, once again, we works and on establishing their first seized the opportunity to further 5G installations, we wanted to find enhance the underlying methodology. out: Which contender offers the best performance and the highest reliability of mobile voice and data services? Mobile Be nchmark KEY FINDINGS Vodafone wins the umlaut connect Mobile Benchmark Spain for the sixth time in a row. Movistar follows with a gap of only three points Vodafone is the overall winner, and leads in the Data category. Orange ranks third and Yoigo fourth. with Movistar following at a close distance while impressively umlaut‘s network benchmarks are widely improving its score compared accepted as the de-facto industry standard to our 2019 benchmark – and for being highly objective. The carefully achieving a plus of 63 points. designed methodology of our 2020 benchmark in Spain reflects umlaut‘s holistic approachto Orange ranks in the midfield with network benchmarking. Its benchmarks combine a good result, Yoigo achieves drivetests and walktests to execute detailed voice the overall grade “satisfactory“. and data measurements under controlled circumstances with a sophisticated crowdsour cing approach. This provides profound insights max. into the overall coverage of voice and data 1000 Points ES (with focus on 4G and 5G) services, real-world Vodafone Movistar User Download Speeds and the Quality of Orange

Broadband Service. The drivetests and walk- Yoigo Crowd tests allow for evaluating the maximum of the max. 200 176 166 networks‘ capabilities. Crowdsourcing reveils 174 the service quality and performance actually 169 experienced by real users. We have thoroughly weighted these components in order to give a Data realistic and authoritative assessment of the max. 480 rated networks‘ true potential and performance. 421 432 397 Even in the time of the Corona pandemic, our 391 analyses showed that the networks remained stable, though with somewhat altered customer Voice max. 320 usage patterns. When conducting this year‘s 293 289 274 175 benchmark, we have of course carefully adap- 890 ted the logistics to ensure maximum safety for Total Score 887845 735 our team members. Grade very good very good good satisfactory

VODAFONE IS THE OVERALL WINNER, Shown voice, data, crowd and total scores are rounded. MOVISTAR ACHIEVES HUGE SCORE IMPROVEMENT Vodafone managed to continue its winning streak, scoring best in the umlaut connect Mobile Bench- mark Spain for the sixth time in a row. The win is Overall Results Vodafone Movistar Orange Yoigo achieved by taking the lead in the Voice and Crowd- Voice max. 320 P. 293 289 274 175 sourcing categories. However, Movistar comes very Cities (Drivetest) 144 91% 90% 83% 54% close with a gap of only three points to Vodafone, showing a massive score improvement of 63 points Cities (Walktest) 48 99% 96% 97% 67% compared to last year‘s result – mainly gained in Towns (Drivetest) 64 97% 95% 93% 58% the Voice category. In contrast, the other three Roads (Drivetest) 64 82% 82% 76% 44% contenders could not maintain their score levels Data max. 480 P. 421 432 397 391 from the previous year and have lost points. Also, Cities (Drivetest) 216 87% 89% 82% 80% Movistar is taking the lead in the Data category. Orange falls back to the third position from Cities (Walktest) 72 93% 92% 89% 88% last year‘s second rank and achieves the grade Towns (Drivetest) 96 88% 91% 83% 85% “good“ (2019: “very good“). Yoigo ranks last, Roads (Drivetest) 96 87% 89% 78% 78% falling back from 2019‘s “good“ to the grade Crowdsourced Quality max. 200P. 176 166 174 169 “satisfactory“ this year. Still, Yoigo ranks Crowd 200 88% 83% 87% 84% second in the Crowdsourcing category, and together with Orange achieved the fastest Connect Rating max. 1000 P. 890 887 845 735 Percentages and points rounded to integer numbers. crowdsourced user Download Data Rates. For the calculation of points and totals, the accurate, unrounded values were used.

2 Mobile Be nchmark SPAIN‘S OPERATORS The three largest Spanish mobile network operators are in constant competition for subscribers and market share. After having consistently increased their LTE coverage and speeds, now all four Spanish operators have also launched 5G installations in varying degrees.

Movistar is the brand name Orange España is the brand Vodafone España has been Yoigo was the latest mobile the Spanish telecommuni- name of France Telecom‘s present on the Spanish mo- operator to enter the Spanish cations company Telefónica mobile network in Spain. bile communications market market. Founded in 2000 un- uses for the mobile network It has been operating un- since the year 2000. Then, der the name Xfera, the com- in its home market. Telefónica der this name since 2006. the British Vodafone Group pany started its actual opera- S.A. is one of the largest Previously, the network acquired Airtel Móviles tion in 2006, offering only a telco companies in the world. was known as “Amena“ – which had operated in Spain UMTS/3G network at 2100 MHz. The operator is active in this brand name lives on in since 1994. With approx. At this time, the Swedish tele- 17 countries with a total Orange Spain‘s portfolio as 14 million subscribers, communications company of 117,300 employees a low-cost offer that is only Vodafone has a share of TeliaSonera acquired the ma- and achieved worldwide available via the internet. about 23 per cent of the jority of shares and rebran- revenues of over €48 billion Also, its network serves a Spanish mobile market. ded the network as “Yoigo“. in its fiscal year 2019. number of mobile virtual This makes Vodafone the In June 2016, the former While the company intro network operators such as third largest operator in the MVNO Másmóvil bought the duced the Movistar brand MasMovil, Carrefour Móvil country following at close company. For its fiscal year in Latin American countries and others. distance behind its com- 2019, Másmóvil reported a in 2005, it has been active With approx. 16 million petitor Orange. In the fiscal record year with revenues of in Spain since the launch mobile customers, Orange year 2019, Vodafone Spain €1.7 billion and an increase of GSM services back in is the second largest Spa- achieved revenues of €4.3 of 1.4 million customers, re- 1995. Today, Movistar is nish mobile operator with billion which contri butes sulting in now 7.4 million mobile the largest mobile operator a market share of about 25 about 12 per cent to the subscribers or a market share in Spain with approx. 19 per cent. In the fiscal year whole Vodafone Group‘s of approx. 14 per cent. million subscribers, which 2019, Orange Spain repor- financial results. Yoigo had a national roa- equates to a market share ted a revenue of €5.3 billion Vodafone‘s mobile network ming agreement with of over 30 per cent. which contributed approx. in Spain offers GSM service Movistar until the end of It offers GSM service at 13 per cent to the Orange at 900 and 1800 MHz, 2016. Since January 2017, 900 and 1800 MHz, UMTS/ Group‘s total revenue. UMTS/3G at 900 and 2100 Yoigo customers freely 3G at 900 and 2100 MHz Orange Spain has deployed MHz and LTE at 800, 1800, roam in the 2G , 3G and and LTE at 800, 1800 and 2G networks at 900 and 2100 and 2600 MHz. The 4G networks of Orange at 2600 MHz. Movistar is 1800 MHz, 3G networks Vodafone 4G network in Spain locations without Yoigo supporting 4G+ carrier at 900 and 2100 MHz and supports LTE 4 carrier aggre- coverage. Yoigo operates 3G aggregation with maximum 4G at 800, 1800 and 2600 gation (4CA or “4G+“) with at 2100 MHz as well as 4G speeds reaching up to 1 Gbps. MHz. The operator claims maximum speeds of 1 Gbps. at 1800 MHz and 2100 MHz. The operator claims that its 4G network reaches Vodafone España claims to Thanks to its roaming agree- to provide 4G coverage of more than 97 per cent of offer the best LTE coverage ments, the operator claims more than 96 per cent of the Spanish population. In in Spain, reaching approx. an LTE coverage of approx. the Spanish population and 2020, Orange has launched 98 per cent of the Spanish 98 per cent of the popula- has recently switched on 5G services on 3500 MHz population. The operator tion. As the only Spanish 5G on 3500 MHz alongside in selected parts of the was the first to launch 5G in operator, Yoigo currently refarmed 1800 and 2100 five cities Madrid, Barce- Spain and now operates 5G does not support VoLTE. MHz frequencies, claiming lona, Valencia, Seville and network cells on 3500 MHz In 2020, Másmóvil has star- to provide 5G signals to 75 Malaga and plans to expand in 21 cities across the ted to deploy 5G in 15 cities. per cent of the Spanish po- it rapidly to other cities. country. Its further 5G roadmap is pulation by the end of 2020. based on a combination of own infrastructure and an agreement with Orange. 3 The network benchmarks conducted by umlaut and connect are Mobile Be nchmark widely accepted as a completely objective authority. In 2020, we present the umlaut connect Mobile Benchmark Spain for the sixth time, with a further enhanced methodology

A CLOSE LOOK AT THE SPANISH NETWORKS umlaut, based in Aachen, Germany, for more than 15 years, extending Congratulations to Vodafone is a world leader in mobile network it to other European countries since for being “Best in Test” the testing. The company was formerly 2009. As the de-facto industry sixth time in a row now. On known as P3 and changed its name standard, umlaut‘s benchmarking top, Vodafone successfully in fall 2019 in the course of restruc- methodology focuses on customer- increased their 5G foot- turing and refocusing its activities. perceived network quality. print, delivering high umlaut has over 4,500 employees, Conducting the 2020 umlaut 5G throughputs. distributed over about 50 locations connect Mobile Benchmark Spain Congratulations also to all over the world, and a turnover of was certainly challenging in the Movistar. This operator more than 400 million Euros. light of the Corona pandemic. But is following with impressive umlaut is partnering with the the drivetest and walktest teams improvements on a very international telecommunications managed to realize all planned good second position. magazine connect, which has more measurements during October 2020, than 25 years of editorial expertise covering a total of 11,540 km Hakan Ekmen, CEO and is one of the leading test autho- and approx. 25.4 per cent of the Telecommunication at umlaut rities in Europe for telecommunica Spanish population. In addition, tions products and services. Together, the score includes the results of umlaut and connect have been extensive crowdsourcing analyses conducting the most important net- considering 24 weeks from May work benchmark test in Germany to October, 2020.

DRIVETEST AND WALKTEST FACTS DATA VOICE 48 % 32 % 11.8 11,540 197,511 23,144 Web Page Download Call Setup Time million km drivetest data samples voice samples people covered Success Ratio Call Success Ratio

Youtube Quality Speech Quality

File Up-/Download MultiRAB Connectivity

CROWDSOURCING FACTS Data Rates Latency

Quality of Broadband Service 267,000 1.1 24 97.5% users providing billion weeks of built-up CROWD relevant samples samples (May to area October 2020) 20 %

4 Mobile Be nchmark VODAFONE SHOWS THE VOICE BEST VOICE Many customers use voice services less intensely than data. RESULTS, However, when actually taking or placing a phone call, they expect reliable connections. How do the Spanish mobile MOVISTAR networks manage to fulfil these expectations? RANKS SECOND, ORANGE THIRD With the ongoing prevalence of LTE provision in the Spanish mobile networks, the use of Voice over LTE (VoLTE) has become the standard AND YOIGO mode for the transmission of voice calls in most of the networks. At the time of testing, only Yoigo did still not support this modern voice stan- L AST. dard and still offered its voice service on legacy-based technology. VoLTE transmits voice calls as data packets over a 4G connection. This way, the otherwise necessary “circuit-switched fallback“, which forces smartphones to switch back to 3G or 2G in order to take or place a phone call, can be avoided. Furthermore, VoLTE codecs potentially support a wider audio bandwidth. Another advantage of realising voice connections via LTE is that VoLTE typically reduces call setup times. For the Voice rating, each drive test car and each walk test team carried one Samsung Galaxy S10 smartphone per operator. The phones in the cars called a counterpart in one of the other cars. The phones carried by the walk test teams in the cities called a stationary counterpart. In order to simulate normal smartphone usage, additional data transfers took place in the background of the test calls. In addition, we also evaluate the so-called Multirab (Multi Radio Access Bearer) Connectivity. This value denominates whether data connectivity is available during the phone calls. The Voice scores account for Vodafone 32 per cent of the total result. Movistar Orange VODAFONE AND MOVISTAR HAVE A Voic320 of 1000 Pointes Yoigo CITIES NECK AND NECK RACE IN THE DRIVETEST VOICE DRIVETESTS IN CITIES 91% As in other subdomains of this

VODAFONE year‘s benchmark, Vodafone and 90% Cities 4 P. Drivetest 14 & MOVISTAR Movistar have a neck and neck 83% of race in the voice drivetests con- 54% ducted in the larger cities. With a distance of one per cent, Vodafone takes a narrow lead. Orange 99%

ranges in the midfield, and Yoigo 96% Cities P. shows some potential for Walktest 48 97% of improvements. 67% VODAFONE AHEAD IN DRIVETEST VOICE TOWNS RESULTS IN SMALLER TOWNS 97% DRIVETEST In the smaller towns, we see a close contest of the three strong 95% Towns P. Drivetest 64 candidates Vodafone, Movistar 93% of VODAFONE and Orange. Vodafone takes the 58% overall lead in this category, with Movistar and Orange following closely, each at a distance of 82%

only 2 per cent. Yoigo brings up 82% Roads P. the rear, but similar to the other Drivetest 64 contenders scores a little stronger 76% of in smaller towns than in the cities. 44% 5 Mobile Be nchmark

FOR VOICE TELEPHONY ON THE ROADS ROADS, VODAFONE LEADS THE FIELD, VOICE RESULTS AT A GLANCE DRIVETEST FOLLOWED BY ORANGE In the drivetests conducted on Vodafone achieves the highest score in the Voice VODAFONE the connecting roads, all opera- discipline, Movistar follows closely on the second & MOVISTAR tors show somewhat increased rank. Both operators make use of the full potential of reliability issues with call suc- VoLTE/EVS and provide excellent speech quality to cess ratios dropping to a level of their customers. Orange scores behind the leading two around 95 per cent. Vodafone and contenders and ranks third in this discipline, showing very good results in the city walktests as well as in the Movistar both achieve 82 per cent drivetests conducted in smaller towns. Yoigo loses of the possible points in this cate- valuable points particularly in the Voice category. gory, with Orange following at 76 per cent. In this category, Yoigo falls behind the competition with Operator Vodafone Movistar Orange Yoigo the most distinct gap due to con- Cities (Drivetest) siderably longer call setup times Sucess Ratio (%) 98.8 99.0 97.5 98.2 and a speech quality MOS of only Call Setup Time (s) 1.6 2.6 1.7 5.9 3.0 on average. Call Setup Time P90 (s) 2.1 3.0 2.1 7.3 Speech Quality (MOS-LQO) 4.4 4.3 4.2 3.1 Speech Quality P10 (MOS-LQO) 3.7 3.6 3.4 2.5 STRONG RESULTS FOR VODAFONE, Multirab Connectivity (%) 100.0 100.0 99.9 89.1 CITIES ORANGE AND MOVISTAR IN THE VOICE Towns (Drivetest) WALKTEST WALKTESTS CONDUCTED IN SEVEN Sucess Ratio (%) 99.8 99.8 99.4 98.6 LARGER SPANISH CITIES Call Setup Time (s) 1.8 2.7 1.7 5.7 Call Setup Time P90 (s) 2.2 3.0 2.2 6.6 In the walktests which umlaut Speech Quality (MOS-LQO) 4.4 4.3 4.2 3.1 VODAFONE conducted in seven larger cities Speech Quality P10 (MOS-LQO) 3.8 3.6 3.4 2.5 (Barcelona, Bilbao, Madrid, Multirab Connectivity (%) 100.0 99.5 100.0 91.2 Málaga, Murcia, Sevilla and Za- Roads (Drivetest) ragoza), Vodafone takes the lead Sucess Ratio (%) 95.5 96.3 94.3 94.4 again and manages to achieve an Call Setup Time (s) 1.9 2.8 1.9 6.4 impressive score of 97 per cent Call Setup Time P90 (s) 2.6 3.2 2.8 8.0 of the available points. This time, Speech Quality (MOS-LQO) 4.2 4.1 4.0 3.0 the leader is closely persecuted Speech Quality P10 (MOS-LQO) 3.3 3.3 3.0 2.4 by Orange who scores with a nar- Multirab Connectivity (%) 99.8 99.7 99.8 83.9 row gap ahead of Movistar. These Cities (Walktest) three operators score very close Sucess Ratio (%) 100.0 99.9 99.8 99.7 together in this category, offering Call Setup Time (s) 1.4 2.5 1.5 5.5 an excellent voice experience to Call Setup Time P90 (s) 1.6 2.8 1.9 6.3 their customers. Yoigo scores Speech Quality (MOS-LQO) 4.5 4.3 4.3 3.0 better than in the other Voice Speech Quality P10 (MOS-LQO) 4.0 3.8 3.8 2.6 categories, but still ranks last. Multirab Connectivity (%) 100.0 100.0 100.0 93.2

6 Mobile Be nchmark MOVISTAR RANKS FIRST IN THE DATA DATA DISCIPLINE, The volume of transmitted data is growing rapidly, which VODAFONE emphasises the importance of data connectivity. Which operator in Spain manages best to keep up with the increasing demand? FOLLOWS ON THE SECOND RANK. ORANGE AND All Spanish mobile operators have worked hard on further expanding their LTE networks to provide their YOIGO RANK ON customers with the best LTE coverage as well as to deliver the highest data rates. In Spain, for many THE THIRD AND years, Vodafone, Orange and Movistar have been constantly competing to gain the lead in these areas. FOURTH POSI- Meanwhile, the smallest Spanish operator, Yoigo, has been concentrating on expan ding its 4G footprint – TIONS, SCORING and claims to have reached 98 per cent LTE coverage CLOSELY of the population thanks to its roaming agreements. At the same time, the Spanish operators have star- TOGETHER. ted their deployments of 5G (for more details, see page 10). We have honoured these advancements by fully integrating 5G measurements into our methodo logy framework: In each of the two test cars four smartphones were used for the data measurements – one per operator. In the first car, Galaxy S10 smartphones were used, set to 4G preferred mode. This device and setting can make full use of the 4 carrier aggregation (4CA), meanwhile offered by Movistar, Vodafone Orange and Vodafone. In the second car as well as in Movistar the backpacks of the walktest teams, one Galaxy Orange Dat480 of 1000 Pointas Yoigo S20+ per operator was used, set to 5G preferred mode and thus enabling 5G connectivity wherever possible. umlaut‘s testing considers fast throughputs as well 87%

as the networks‘ availability and stability. In order to P. 89% assess both typical performance and peak speeds, Cities Drivetest 216 we consider two values: the minimum data rate that 82% of is available in 90 per cent of the cases, and additio- 80% nally the peak data rate that is surpassed in 10 per cent of the cases. Web page and file downloads or 93% file uploads reward fast speeds, while the determi- nation of success ratios and assessing YouTube play 92% Cities P. Walktest outs mainly concentrate on reliability aspects. 89% of 72 88%

88%

91% Towns P. Drivetest 96 83% of 85%

87%

89% Roads P. Drivetest 96 78% of 78%

7 Mobile Be nchmark

MOVISTAR LEADS IN LARGER CITY Data Cities (Drivetest) Vodafone Movistar Orange Yoigo CITIES DATA DRIVETESTS, WITH VODAFONE Web-Page Download (Live/Static) DRIVETEST FOLLOWING AT CLOSE DISTANCE Success Ratio (%/%) 99.7/99.7 99.8/99.8 99.6/99.6 99.0/99.0 The results of our drive tests con- Overall Session Time (s) 1.1 1.1 1.3 1.4 ducted in 20 larger Spanish ci- File Download (5 MB) MOVISTAR ties (also see page 1) confirm the Success Ratio/Avg. Session Time (%/s) 99.8/2.4 100.0/2.1 99.9/3.3 99.0/5.0 broad LTE roll-out of all operators 90%/10% faster than (Mbit/s) 9.7/83.5 12.0/82.8 6.6/78.6 3.9/52.2 in urban areas. In this category, File Upload (2.5 MB) Movistar takes a narrow lead Success Ratio/Avg. Session Time (%/s) 99.6/1.8 99.9/1.8 99.7/2.3 99.8/2.9 ahead of Vodafone due to slightly 90%/10% faster than (Mbit/s) 7.2/32.9 8.0/30.2 5.8/27.9 4.0/21.8 higher success ratios and data File Download (7 Seconds) rates in most of the download and Sucess Ratio (%) 99.9 99.8 99.7 99.7 upload tests. However, Vodafone Avg. Throughput (Mbit/s) 80.4 61.0 46.1 30.4 follows at very close distance, sur- 90%/10% faster than (Mbit/s) 10.1/149.7 12.3/129.0 7.6/101.9 7.1/60.1 passing Movistar in some of the File Upload (7 Seconds) KPIs such as the average through- Sucess Ratio (%) 99.7 99.8 99.5 99.3 puts of the 7 second file downloads Avg. Throughput (Mbit/s) 29.1 28.7 25.3 18.3 and uploads. Orange and Yoigo 90%/10% faster than (Mbit/s) 7.3/52.5 8.4/47.0 6.0/46.7 4.5/33.4 fall a little behind, both still achie- Youtube Video ving overall good results in this Success Ratio/Start Time (%/s) 98.4/1.3 99.3/1.2 97.1/1.5 97.8/1.4 category. All four operators show Average Video Resolution (p) 916 914 905 911 some room for improvement when Youtube live Smartphone it comes to the reliability of YouTube Success Ratio/Start Time (%/s) 97.1/1.3 99.4/1.3 96.1/1.5 98.1/1.6 video streams. Average Video Resolution (p) 1033 1031 1017 1022

VODAFONE TAKES A NARROW LEAD IN Data Cities (Walktest) Vodafone Movistar Orange Yoigo CITIES THE LARGER CITIES DATA WALKTESTS Web-Page Download (Live/Static) WALKTEST In the results of the walktests Success Ratio (%) 99.9 99.9 100.0 99.9 conducted in Barcelona, Bilbao, Overall Session Time (s) 1.0 1.0 1.1 1.2 Madrid, Málaga, Murcia, Sevilla File Download (5 MB) VODAFONE and Zaragoza, Vodafone manages Success Ratio/Avg. Session Time (%/s) 100.0/1.3 100.0/1.6 100.0/1.9 100.0/2.7 to reclaim a narrow lead ahead 90%/10% faster than (Mbit/s) 22.2/142.6 18.9/92.1 14.8/94.2 10.0/61.7 of Movistar. But also Orange and File Upload (2.5 MB) Yoigo achieve quite high scores in Success Ratio/Avg. Session Time (%/s) 99.8/1.7 100.0/1.6 100.0/2.2 100.0/2.4 this category and rank close to- 90%/10% faster than (Mbit/s) 7.7/39.6 10.1/24.6 6.2/23.6 5.4/19.5 gether, emphasizing that pedes trians in larger Spanish cities can File Download (7 Seconds) expect a good level of quality for Sucess Ratio (%) 99.6 99.6 99.8 100.0 mobile data usage. Again, our fin- Avg. Throughput (Mbit/s) 200.0 84.1 61.4 41.5 dings show some room for impro- 90%/10% faster than (Mbit/s) 25.4/604.5 21.0/168.6 13.7/120.1 13.2/77.2 vements in the YouTube category File Upload (7 Seconds) – which applies to all four Spanish Sucess Ratio (%) 99.8 100.0 99.8 100.0 operators particularly regarding Avg. Throughput (Mbit/s) 36.2 32.8 26.8 21.2 service reliability. 90%/10% faster than (Mbit/s) 7.6/62.7 13.5/48.9 6.6/47.7 6.3/34.2 Youtube Video Success Ratio/Start Time (%/s) 99.4/1.1 99.2/1.1 99.6/1.3 99.4/1.3 Average Video Resolution (p) 919 918 913 918 Youtube live Smartphone Success Ratio/Start Time (%/s) 98.8/1.1 99.6/1.2 98.8/1.3 100.0/1.3 Average Video Resolution (p) 1040 1035 1031 1040

8 Mobile Be nchmark

MOVISTAR LEADS IN DATA DRIVE- TOWNS Data Towns (Drivetest) Vodafone Movistar Orange Yoigo TESTS IN SMALLER TOWNS, CLOSELY DRIVETEST Web-Page Download (Live/Static) FOLLOWED BY VODAFONE Success Ratio (%) 99.9 99.8 99.6 99.9 In the Data drivetests which our Overall Session Time (s) 1.1 1.1 1.3 1.4 measurement cars performed MOVISTAR File Download (5 MB) in 27 smaller Spanish towns, Success Ratio/Avg. Session Time (%/s) 100.0/2.1 100.0/2.2 100.0/3.5 100.0/3.7 Movistar once again manages 90%/10% faster than (Mbit/s) 11.5/73.8 18.1/86.1 6.2/80.9 5.6/57.0 to take the lead. Vodafone follows File Upload (2.5 MB) closely – both operators achieve Success Ratio/Avg. Session Time (%/s) 100.0/1.9 100.0/2.0 99.4/2.5 99.8/2.9 high to very high success ratios 90%/10% faster than (Mbit/s) 6.9/29.9 6.7/28.4 5.2/24.8 4.2/20.7 when downloading or uploads File Download (7 Seconds) files and also in the web browsing Sucess Ratio (%) 99.8 100.0 99.8 99.4 tests. In the Data drivetests in towns, Yoigo comes in third, and Avg. Throughput (Mbit/s) 59.2 77.9 49.9 38.2 Orange ranks last – but both con- 90%/10% faster than (Mbit/s) 14.6/115.4 19.7/157.1 6.7/112.3 12.1/68.4 tenders still score closely together File Upload (7 Seconds) and on an overall high level. Sucess Ratio (%) 99.6 99.8 99.4 99.6 Avg. Throughput (Mbit/s) 26.3 24.7 22.7 16.7 90%/10% faster than (Mbit/s) 7.7/48.7 7.4/41.7 6.1/44.5 4.2/31.0 MOVISTAR AND VODAFONE AHEAD IN Youtube Video ROADS DATA DRIVETESTS ON CONNECTING Success Ratio/Start Time (%/s) 98.9/1.2 99.8/1.2 98.0/1.5 99.4/1.4 DRIVETEST ROADS, ORANGE AND YOIGO ON A PAR Average Video Resolution (p) 919 919 909 910 It can be seen as a good sign for Youtube live Smartphone the overall LTE coverage in Spain Success Ratio/Start Time (%/s) 98.4/1.2 99.6/1.3 97.2/1.5 99.6/1.4 MOVISTAR that our test teams noted only Average Video Resolution (p) 1038 1040 1028 1029 & VODAFONE “occasional LTE availability gaps“ during their drives on 7,540 km of Data Roads (Drivetest) Vodafone Movistar Orange Yoigo connecting roads. In this category, Web-Page Download (Live/Static) Movistar is again in the lead, with Success Ratio (%) 99.2 99.4 98.3 98.0 Vodafone following at a narrow Overall Session Time (s) 1.3 1.3 1.5 1.6 distance. The gap to Orange and File Download (5 MB) Yoigo, who score on the same Success Ratio/Avg. Session Time (%/s) 99.6/3.4 99.6/2.8 99.5/4.7 99.1/5.7 level in this category, is more dis- 90%/10% faster than (Mbit/s) 6.8/67.4 8.7/74.5 4.4/70.3 3.2/58.9 tinct – to some extent exhibiting a File Upload (2.5 MB) two-class society when it comes to Success Ratio/Avg. Session Time (%/s) 98.7/2.7 99.6/2.6 98.0/3.5 98.2/4.7 data connectivity on Spanish roads. 90%/10% faster than (Mbit/s) 4.3/25.9 4.6/23.9 3.0/22.5 1.9/20.6 File Download (7 Seconds) Sucess Ratio (%) 98.6 99.5 98.9 98.4 Avg. Throughput (Mbit/s) 43.2 53.6 36.8 31.9 90%/10% faster than (Mbit/s) 7.9/91.0 10.2/116.6 5.4/83.6 5.4/70.2 File Upload (7 Seconds) Sucess Ratio (%) 99.0 99.0 97.2 96.4 Avg. Throughput (Mbit/s) 20.6 20.5 18.7 15.9 90%/10% faster than (Mbit/s) 5.1/39.6 5.1/40.2 2.9/39.9 3.0/31.2 Youtube Video Success Ratio/Start Time (%/s) 97.9/1.5 98.3/1.3 93.2/1.8 95.5/1.6 Average Video Resolution (p) 912 911 894 903 Youtube live Smartphone Success Ratio/Start Time (%/s) 96.2/1.5 97.9/1.5 91.2/1.7 92.7/1.9 Average Video Resolution (p) 1026 1024 1002 1006

9 Mobile Be nchmark

SPANISH MOBILE OPERATORS ON A GOOD WAY INDIVIDUAL ANALYSIS OF VODAFONE TO BROADEN THEIR 5G DEPLOYMENTS DATA SAMPLES PROVES 5G SPEED 5G At the time of testing, all four Spanish operators ADVANTAGES have started to deploy 5G in their networks – Due to its head start in the 5G de- however, doing so in varying degrees. ployment, Vodafone already con- VODAFONE Vodafone Spain was the first to start its 5Gservices tributed a nameable share of 5G already in mid-2019. By now, the operator has acti- samples especially to our drivetests vated 5G network cells in 21 cities across the country and walktests conducted in the using its spectrum in the 3.5 GHz band. Movistar large cities of Spain. (Telefónica) announced in September 2020 that it The tables below show com- had switched on its 5G service at some first locations parisons of the data throughputs and plans to quickly increase 5G coverage to up to achieved in the 7 second down- 75 per cent of the Spanish population. Orange has load and 7 second upload parts launched 5G services in parts of 11 cities, namely of our data tests, aggregated for Madrid incl. Pozuelo de Alarcón, Sevilla, Valencia, 4G samples and 5G samples in Málaga, Barcelona incl. Sabadell, Logroño, Pamplona, the Vodafone network. The results Vigo and Zaragoza, and has announced to increase clearly substantiate the speed them to 93 cities by the end of 2020. advantages that 5G technology Másmóvil, standing behind the Yoigo brand, has can already provide today. launched 5G services based on on a combination of its own infrastructure and a “virtual active sharing mode” Vodafone – 7s Download Data Rate Average P10 P90 Max agreement with Orange. Currently, its 5G service is Samsung S20+ 5G Samples with 4G reception available in parts of Alicante, Alcobendas, Almeria, Cities – Drivetest (Mbps) 50.8 8.8 111.9 279.6 Avila, Barcelona, Bilbao, Hospitalet de Llobregat, Cities – Walktest (Mbps) 81.3 18.7 155.9 283.4 Huesca, Jaen, Madrid, Malaga, Melilla, Orense, Sala- Samsung S20+ 5G Samples with 5G reception manca, San Sebastian, Seville, Valencia and Vitoria. Cities – Drivetest (Mbps) 380.9 58.1 689.2 887.5 For the next step, Yoigo aims at covering about Cities – Walktest (Mbps) 465.2 139.1 727.4 870.6 35 per cent of the population in about 40 cities with 5G. In the more distant future, the operator plans to Vodafone – 7s Upload Data Rate Average P10 P90 Max launch its own stand-alone 5G network. Samsung S20+ 5G Samples with 4G reception As the 5G launches of Movistar, Orange and Yoigo Cities – Drivetest (Mbps) 27.4 7.4 50.7 63.8 took place only shortly before the start of our drivetests and walktests, in this year‘s benchmark, 5G could only Cities – Walktest (Mbps) 31.5 7.0 54.3 63.6 be factored in for the Vodafone network. We will of Samsung S20+ 5G Samples with 5G reception course consider 5G for all four Spanish operators in Cities – Drivetest (Mbps) 39.0 14.2 61.5 79.4 next year‘s umlaut connect Mobile Benchmark Spain. Cities – Walktest (Mbps) 45.8 9.9 69.4 79.6

DATA RESULTS AT A GLANCE

In the Data discipline, Movistar takes the lead and outperforms the overall winner Vodafone. The race is close in the drivetests performed in the larger cities and smaller towns, and in the city walktests, Vodafone even manages to re- gain the first rank. The picture is similar on the roads: Movistar leads, Vodafone follows at a narrow distance, while Orange and Yoigo distinctly fall back. Our tests indicate an increasing 5G footprint of Vodafone. Furthermore, we observed some 4G coverage gaps on the roads and some room for improvements in YouTube reliability concerning all four Spanish operators.

10 Mobile Be nchmark VODAFONE TAKES THE LEAD CROWD IN THE CROWD 267,000 Spanish users have contributed around 1.1 billion measurement ASSESSMENT, samples between mid-May and the end of October 2020. umlaut has conducted a thorough analysis of this extensive data base, using a SLIGHTLY AHEAD further refined crowdsourcing methodology compared to previous years. OF ORANGE. YOIGO RANKS While the drivetests and walktests determine the peak THIRD AND performance of the examined networks, crowdsourcing can add important dimensions such as time, geograph y MOVISTAR or variety in devices and tariff plans – if done in the right way. A detailed description of our crowdsourcing FOURTH IN THIS methodology can be found on page 14. Based on the total population count of 46.7 million people, one of 175 DISCIPLINE. inhabitants of Spain has provided relevant samples to our crowd data. The test area of our crowdsourcing represents 97.5 per cent of the built-up area of Spain.

QUALITY OF MOVISTAR LEADS THE MARKET IN Crowd Vodafone Movistar Orange Yoigo QUALITY OF BROADBAND SERVICE BROADBAND Quality of Broadband Service Movistar takes a narrow lead in SERVICE Coverage Excellence (%) 62.3 62.7 61.9 49.5 the Quality of Broadband Service Time on Broadband (%) 94.0 95.4 94.1 93.8 category. Especially the Time on MOVISTAR Download Speed Broadband KPI which designates how often a single user had 4G or Basic Internet Class (%) 93.3 83.8 93.5 93.5 5G reception during the observation HD Video Class (%) 72.7 62.2 74.6 75.1 period is somewhat higher than for UHD Video Class (%) 14.8 13.7 13.8 13.3 the other contenders. Still, Orange Latency and Vodafone follow closely in Gaming Class (%) 56.0 61.4 50.8 41.6 this discipline which only slightly OTT Voice Class (%) 96.8 96.0 95.3 95.2 lower results. Yoigo is falling behind more distinctly, particularly due to a lower per centage in Coverage Excellence – the KPI which assesses the share of samples with 4G or 5G coverage and awards extra points if other contenders offer less or no broadband service at all in an evaluation area.

11 Mobile Be nchmark

ORANGE AND YOIGO DELIVER BEST VODAFONE AND MOVISTAR AHEAD DOWNLOAD USER DOWNLOAD SPEEDS WITH IN LATENCY METRIC, FOLLOWED BY SPEEDS VODAFONE RIGHT BEHIND THEM LATENCY ORANGE AND THEN YOIGO In our crowdsourced assessment In our examinations of Latency, ORANGE of download speeds, Orange and VODAFONE & Vodafone and Movistar are lea- & YOIGO Yoigo achieved the best results MOVISTAR ding the field, achieving the best showing a particularly high share results both in the OTT Voice of samples meeting the require- Class (roundtrip times of 100 ms ments of the speed classes Basic or less) and in the Gaming Class Internet (up to 2 Mbps) as well as (roundtrip times of up to 50 ms). HD Video (up to 5 Mbps). Orange is following at a little Vodafone follows right behind distance in this discipline, achie- them achieving some points less ving a share of 50.8 per cent of in the lower speed classes whi- the collected samples meeting le leading the field regarding the the requirements of the deman- share of samples in the most de- ding Gaming Class. manding speed class, UHD Video Yoigo is almost on a par with (up to 20 Mbps). Orange regarding the share of In our crowdsourced speed samples meeting the require- assessments, the market leader ments of the OTT Voice Class, but Movistar scores on the last rank in falls a little behind the competition the two lower speed classes, whi- in the more difficult Gaming Class. le it outranks the otherwise very strongly performing Yoigo in the fastest speed class, UHD Video – however, Vodafone and Orange still achieve slightly higher results CROWD RESULTS AT A GLANCE in this KPI. In the crowdbased score, Vodafone takes the lead, two points ahead of Orange. Yoigo and Movistar follow with a more distinct gap, with Yoigo still accomplishing a very good and Movistar achieving a good overall result in this category. However, Movistar leads the market in our Quality of Broadband Service metric, closely followed by Orange and Vodafone. When it comes to the crowdsourced assessment of Download Speeds, Orange and Yoigo achieved the best result, with Vodafone right behind them. In the Latency metric, Vodafone and Movistar scored best, followed by Orange and then Yoigo.

12 Mobile Be nchmark TESTING METHODOLOGY The methodology of the umlaut connect Mobile Benchmark is the result of more than 15 years of testing mobile networks. Today, network tests are conducted in more than 80 countries. Our methodology was carefully designed to evaluate and objectively compare the performance and service quality of mobile networks from the users’ perspective.

The umlaut connect Mobile Bench- 25.4 per cent of the total popula- mark Spain comprises of the tion of Spain. The test routes and umlaut‘s fleet of test cars is equipped results of extensive voice and data all visited cities and towns are with up-to-date test smartphones. drivetests and walktests as well shown on page 1 of this report. The phones on board are operated and as a sophisticated crowdsourcing The two drivetest cars were supervised by a unique control system. approach. equipped with arrays of Samsung Galaxy S10 and S20+ smartphones the gathered values are ranging. DRIVETESTS AND WALKTESTS for the simultaneous measurement For speech quality, we published The drivetests and walktests in of voice and data services. the P10 value (10 per cent of the Spain took place in October 2020. values are lower than the specified All samples were collected during VOICE TESTING threshold), because in this case the day, between 8.00 a.m. and One Galaxy S10 per operator in higher values are better. 10.00 p.m. The network tests each car was used for the voice In order to account for typical covered inner-city areas, outer tests, setting up test calls from smartphone-usage scenarios during metropolitan and suburban areas. one car to another. The walktest the voice tests, background data Measurements were also taken team also carried one Galaxy S10 traffic was generated in a controlled in smaller towns and cities along per operator for the voice tests. way through injection of 100 KB connecting highways. For the drive In this case, the smartphones of data traffic (HTTP downloads). tests, two cars were operated by called another Galaxy S10 as a We also evaluated the so-called two rotating teams per each car, stationary counterpart. The audio Multirab (Multi Radio Access Bearer) working on shifts. The connecting quality of the transmitted speech Connectivity. This value denomi- routes between the cities covered samples was evaluated using nates whether data connectivity about 3,800 kilometres per car – the HD-voice capable and ITU is available during the phone calls. 7,540 kilometres for both cars. In standardised so-called POLQA The Voice scores account for total, the two vehicles together wideband algorithm. All smart 32 per cent of the total results. covered about 11,540 km. phones used for the voice tests The combination of test areas were set to VoLTE preferred mode. DATA TESTING has been selected to provide re In the assessment of call setup Data performance was measured presentative test results across times we also rated the so-called by using four more smartphones the Spanish population. The areas P90 value. Such values specify the in each car – one per operator In selected for the 2020 test account threshold in a statistical distribu Car 1, this was another Galaxy S10, for 11.8 million people, or roughly tion, below which 90 per cent of set to 4G preferred mode. Car 2 and the walktest team carried one One Samsung Galaxy S10 per operator was used for the voice measurements and Galaxy S20+ per operator, set to another Galaxy S10 for half of the data measurements. In the second car and in 5G preferred mode – enabling 5G the walktest team‘s backpack a Galaxy S20+ was used and set to “5G preferred“ connectivity wherever available. For the web tests, they accessed web pages according to the widely recognised Alexa ranking. In addition, the static “Kepler” test web page as specified by ETSI (European Telecommunica- tions Standards Institute) was used. In order to test the data service performance, files of 5 MB and 2.5 MB for download and upload were transferred from or to

13 Mobile Be nchmark a test server located in the cloud. experience wherever In addition, the peak data perfor- and whenever custo- DATA VOICE mance was tested in uplink and mers use their smart- 48 % 32 % downlink directions by assessing phones. Web Page Download Call Setup Time the amount of data that was trans- Success Ratio Call Success Ratio ferred within a seven seconds QUALITY OF time period. BROADBAND SERVICE Youtube Quality Speech Quality The evaluation of YouTube play- For the assessment of File Up-/Download MultiRAB Connectivity back takes into account that network coverage, Data Rates Latency YouTube dynamically adapts umlaut applies a grid the video resolution to the avail of 2 x 2 km tiles (so Quality of Broadband Service able bandwidth. So, in addition to called evaluation areas success ratios and start times, the or EAs) over the test CROWD measurements also determined area. For each tile, a 20 % average video resolution. minimum number of All the tests were conducted users and measure- with the best-performing mobile ment values must be available. DATA RATES AND LATENCY plan available from each operator. In order to assess the Coverage Additionally, umlaut investigates Data scores account for 48 per Excellence, umlaut awards one the Data rates and Latencies that cent of the total results. point if the considered network were actually available to each provides 4G or 5G coverage in an user. The examination of these CROWDSOURCING EA. Another point is awarded to parameters is independent from Additionally, umlaut conducted a candidate for each competitor the EAs and thus concentrates crowd-based analyses of the Spa- who provides a smaller or no on the experience of each single nish networks which contribute share of broadband usage. In user. Samples which were for 20 per cent to the end result. a country with four contenders, instance obtained via WiFi or with They are based on data gathered a candidate can thus reach up to the smarthphone‘s flight mode between mid-May and the end four points per tile: one for broad- being active, are filtered from the of October 2020. band coverage and three additio- data pool before further analysis. For the collection of crowd nal ones for “beaten“ competitors. In order to take the fact into data, umlaut has integrated a The assessment then relates account that many mobile phone background diagnosis processes the obtained points to the total tariffs limit data rates, umlaut has into 800+ diverse Android apps. possible points for Coverage defined speed classes which are If one of these applications is Excellence. corresponding to particular appli- installed on the end-user’s phone In addition, we consider the Time cations: For Basic Internet, 2 Mbps and the user authorizes the back- on Broadband. It reveals how are sufficient.HD Video requires ground analysis, data collection often a single user had 4G or 5 Mbps. And for UHD Video the takes place 24/7, 365 days a 5G reception in the observati- minimum is 20 Mbps. year. Reports are generated on period – independent from the In order for a sample to count as for every hour and sent daily to EAs in which the samples were valid, a minimum amount of data umlaut‘s cloud servers. Such obtained. In order to calculate must have been transmitted within reports occupy just a small num- this, umlaut puts the number of a 15 minute period. The same ber of bytes per message and samples with 4G/5G coverage principle also applies to the do not include any per sonal user into relation to the total number of assignment of a data packet‘s data. This unique crowdsourcing all samples. Coverage Excellence latency to the according appli- technology allows umlaut to and Time on Broadband results cation-based classes: Roundtrip collect data about real-world each provide 50 per cent of the times up to 100 ms are sufficient points for the for OTT Voice, 50ms and faster

Crowdsourcing Score Model Quality of Broad- qualify a sample for Gaming. Quality of Broadband Service band Service. In the assessment, umlaut assigns Coverage Excellence 50% max. 50 P. Important: The the data rate and latency observed 50% Time on Broadband 50% max. 50 P. percentages deter- in a sample to one of these perfor Data rates (Download) mined for both pa- mance classes. Then, Basic Internet Basic Internet Class (up to 2 Mbps) 60% max. 36 P. rameters reflect the accounts for 60 per cent of the Data 30% HD Video Class (up to 5 Mbps) 30% max. 18 P. respective degrees Rate score, HD Video for 30 per cent UHD Video Class (up to 20 Mbps) 10% max. 6 P. of fulfilment. They and UHD Video for 10 per cent (see Latency (Roundtrip) do not correspond table on the left-hand side). The OTT Voice Class (up to 100 ms) 80% max. 32 P. 20% to the percentage of Latency score incorporates OTT Gaming Class (up to 50 ms) 20% max. 8 P. 4G/5G coverage of an Voice with a share of 80 per cent, Total max. 200 P. area or population. Gaming with a share of 20 per cent. 14 max. 1000 Points Mobile Be nchmark ES Vodafone Movistar Orange

Yoigo Crowd CONCLUSION 176 166 max. 200 Vodafone wins for the sixth time in a row. Movistar 174 manages to fight itself up to the second rank and 169 follows at a distance of only three points behind the Data winner, achieving an impressive score improvement max. 480 over the previous year. Orange ranks third with a good 421 432 397 result and Yoigo fourth with the grade “satisfactory.“ 391

The overall winner of the umlaut connect Mobile Voice max. 320 Benchmark Spain is Vodafone – for the sixth time in 293 289 274 175 a row. But Movistar comes very close at a distance of only three points to the winner Vodafone. This Total Score 890 887845 735 second rank is gained by a massive score improvement Grade very good very good good satisfactory of 63 points compared to last year‘s result – a large part of it due to improvements in the Voice category Shown voice, data, crowd and total scores are rounded. facilitated by the introduction of VoLTE. In contrast, the other three contenders could not maintain their score

levels from the previous year and have lost points. Also, Overall Results Vodafone Movistar Orange Yoigo Movistar is taking the lead in the Data category. Voice max. 320 P. 293 289 274 175 Orange falls back to the third rank from 2019‘s second position and achieves the grade “good“ (2019: “very good“) Cities (Drivetest) 144 91% 90% 83% 54% – but ranks second in the Crowdsourcing category, sho- Cities (Walktest) 48 99% 96% 97% 67% wing strong results in the crowdsourced user download Towns (Drivetest) 64 97% 95% 93% 58% speeds and also showing comparably short latencies. Roads (Drivetest) 64 82% 82% 76% 44% Yoigo ranks fourth, falling back from 2019‘s “good“ Data max. 480 P. 421 432 397 391 to the grade “satisfactory“ this year. This is mainly due Cities (Drivetest) 216 87% 89% 82% 80% to some shortcomings in the Voice category – in the Cities (Walktest) 72 93% 92% 89% 88% Data category Yoigo scores closely behind Orange and Towns (Drivetest) 96 88% 91% 83% 85% even outperforms the overall second-ranking Movistar Roads (Drivetest) 96 87% 89% 78% 78% in the Crowdsourcing category – achieving the fastest Crowdsourced Quality max. 200P. 176 166 174 169 crowdsourced user download speeds together with Orange. But what is most important this year: All four Spanish Crowd 200 88% 83% 87% 84% operators manage to provide mostly stable connections to Connect Rating max. 1000 P. 890 887 845 735 Percentages and points rounded to integer numbers. their users - even in today‘s particularly demanding times. For the calculation of points and totals, the accurate, unrounded values were used.

VODAFONE 11/2020 1 2 3 4

For the sixth time in a Movistar has fought its Orange ranks third with Spain‘s smallest operator row, Vodafone is the way up to the second rank an overall good result. ranks fourth, achieving winner of our Mobile following at a distance of While the operator scores the overall grade “satis- Benchmark in Spain. only three points behind in the midfield in our drive factory“. In comparison to The operator takes the the winner and showing an tests and walktests, it last year‘s results, Yoigo win due to leading in impressive score improve- shows particularly strong lost some points. Still, the the Voice and Crowd- ment of 63 points of last results in the Crowdsour- operator scores closely sourcing categories and year‘s result. Also, Movistar cing category, especially behind the third-ranking probably also by offering takes the lead in the Data due to the highest crowd- Orange in the Data cate the largest 5G footprint category and achieves the sourced user download gory and even outper- in Spain. All of this results best scores in the drive speeds in our comparison forms the overall second- in an appealing combina- tests performed in cities, and also comparably ranking Movistar in the tion for the customers. towns and on the roads. good latencies. Crowdsourcing category.