BoR (19) 255 BEREC guidelines on how to assess the effectiveness of public warning systems transmitted by different means December, 2019 BoR (19) 255 Contents Executive Summary.............................................................................................................................. 3 1. Introduction ................................................................................................................................... 5 1.1. What are Public Warning Systems? ...................................................................................................... 5 1.2. Description of the task & addressees of the Guidelines & non-binding character ................................. 5 2. BEREC’s interpretation of the scope of article 110 EECC ....................................................... 6 2.1. Legal considerations ............................................................................................................................. 6 2.1.1. Aim of article 110 EECC ............................................................................................................................. 6 2.1.2. Obligation of article 110 EECC ................................................................................................................... 6 2.1.3. Equivalence of 110(2)-PWS’ ...................................................................................................................... 6 2.1.4. Parallel roll-out of multiple ECS-PWS’ in a member state ........................................................................... 7 2.2. Systems falling under 110(1) EECC for the purpose of benchmarking ................................................. 7 2.2.1. Introduction ................................................................................................................................................ 7 2.2.2. Cell Broadcast (CB) implemented according to ETSI EU-ALERT standard ................................................. 8 2.2.3. Location Based SMS (LB-SMS) ............................................................................................................... 10 2.2.4. Automatic Voice Calling (AVC) ................................................................................................................. 11 2.2.5. Conclusion ............................................................................................................................................... 12 2.3. Systems falling under 110(2) EECC .................................................................................................... 12 2.3.1. Introduction .............................................................................................................................................. 12 2.3.2. IAS Mobile Application Based PWS.......................................................................................................... 12 3. Methodology ............................................................................................................................... 15 3.1. Criteria & sub-criteria for evaluating ECS-PWS performance ............................................................. 16 Coverage .............................................................................................................................................................. 16 3.1.1. Geographical coverage ............................................................................................................................ 16 3.1.2. Population Coverage ................................................................................................................................ 17 Capacity to reach end-users ................................................................................................................................. 17 3.1.3. Support of inbound roamers ..................................................................................................................... 17 3.1.4. Supported devices ................................................................................................................................... 17 3.1.5. Supported languages ............................................................................................................................... 18 3.1.6. Managing longer messages ..................................................................................................................... 18 3.1.7. Steps required for recipient to enable receiving warning messages .......................................................... 18 3.1.8. Accessibility for end-users with disabilities ............................................................................................... 18 3.1.9. Reliability.................................................................................................................................................. 18 3.1.10. Geographical targeting ............................................................................................................................. 19 3.1.11. Scalability ................................................................................................................................................. 19 3.2. Establishing the Benchmark ................................................................................................................ 20 3.2.1. Analysing the performance of Cell Broadcast as implemented according to ETSI EU-ALERT standard.... 20 3.2.2. Analysing the performance of Location based SMS.................................................................................. 23 3.2.3. Overview of 110(1) performance .............................................................................................................. 27 1 BoR (19) 255 3.3. Assessing the equivalence of the effectiveness of IAS-PWS .............................................................. 30 3.3.1. Analysing IAS-PWS performance ............................................................................................................. 30 3.3.2. Comparing IAS-PWS performance with 110(1)-PWS performance ........................................................... 32 Annex 1 ................................................................................................................................................ 37 Annex 2 ................................................................................................................................................ 39 Annex 3 ................................................................................................................................................ 41 2 BoR (19) 255 Executive Summary These guidelines are provided by BEREC in response to the task set in article 110(2) of the Directive (EU) 2018/1972 of the European Parliament and of the Council of 11th December 2018 establishing the European Electronic Communications Code1 (hereinafter EECC), to assist member states in assessing the effectiveness of alternative Public Warning Systems (hereinafter PWS) using means of electronic communications services (hereinafter ECS-PWS). The document has the following structure: • Chapter 1 sets out the relevant background on ECS-PWS’ and a detailed description of the task provided by article 110 EECC, focusing on BEREC’s requirement to publish guidelines on how to assess whether the effectiveness of public warning systems under article 110(2) EECC are equivalent to the effectiveness of those under article 110(1) EECC. • Chapter 2 illustrates BEREC’s interpretation of the scope of article 110 EECC, including legal considerations as well as information on the relevant ECS-PWS’ (Cell Broadcast, Location-based SMS and ECS-PWS’ using an on-device application making use of an internet access service, hereinafter IAS-PWS’). • Chapter 3 describes the methodology BEREC proposes, which is essentially a guideline of steps to conduct an assessment of the equivalence of effectiveness of ECS-PWS’. It is divided into three main sections. o Section 3.1 describes the criteria derived from the EECC the methodology proposes to assess the performance of each type ECS-PWS in order to make them comparable. It does so by describing the two main-criteria mentioned by the EECC (coverage and capacity to reach end- users) and a set of sub-criteria which can be summarised under the main-criteria, and in BEREC’s view help to substantiate them. o Section 3.2 describes how competent authorities may establish a benchmark for the assessment of the equivalence of relevant ECS-PWS falling under article 110(2) EECC (IAS- PWS). To create the benchmark BEREC proposes analysing the performance of the relevant ECS-PWS falling under article 110(1) EECC (Cell Broadcast and Location-based SMS) by assessing them against the proposed criteria and sub-criteria established in section 3.1. For the assessment by the competent authorities BEREC has compiled an initial assessment for each sub-criterion. The level of detail in the assessment of each sub-criterion varies as for some sub-criteria the performance of an ECS-PWS depends on national circumstances like the network-structure or the geographic dispersion of end-users in a Member State. o Section 3.3 then describes how competent authorities may assess their envisioned IAS-PWS against the benchmark created in section 3.2. In this section BEREC provides general information for the consideration of competent authorities when assessing the performance of their envisioned IAS-PWS against the