WORLD METEOROLOGICAL ORGANIZATION

THETHE WMOWMO STRATEGYSTRATEGY FORFOR SERVICESERVICE DELIVERYDELIVERY

……. CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I

THE WMO STRATEGY FOR SERVICE incorporating the role of users and customers DELIVERY in day-to-day operations, those customers and users are more likely to receive services 0 PURPOSE OF THIS DOCUMENT that meet their needs.

The purpose of this document is to provide a The ability of NMHSs to meet national World Meteorological Organization (WMO) service needs is put to its most critical test Strategy for Service Delivery that will assist when an extreme hydrometeorological event National Meteorological and Hydrological occurs. Even the best forecast, issued on Services (NMHSs) in the provision of time, is no defense if, for various reasons, it weather-, climate- and water-related services did not generate the desired response from to the public and decision-makers. The those at risk. In other words, the forecast had Strategy incorporates assessment of user little impact. Most of the utility of weather-, needs and the application of performance climate-, and water-related information metrics. occurs in communicating the information to users and the response of those users based While there is no prescriptive way to on information received. Ultimately, the utility provide services, the Strategy serves as a of weather-, climate-, and water-related foundation to improve service delivery by information is the degree to which it has a sharing best practices, supporting mutually beneficial impact on societal and economic agreed upon guidelines, and by increasing outcomes. If the currently available user engagement throughout the delivery information is underutilized, value can be process, recognizing the many differences in increased by improving the forecast, cultures, structures, operational practices, improving communication, and by improving resource and development levels across the decision-making process. Effective NMHSs. service delivery, then, is about providing products and services that bring utility to This Strategy, which is at once broad users and customers. yet flexible, seeks to do two things: (1) serve as a tool for evaluating current service Much has been done for service delivery practices; and, (2) serve as high- delivery by the WMO through various level guidance for developing more detailed international and regional institutions, methods and tools for better integrating users programmes and structures, such as World into the service delivery process. It is Meteorological Centres (WMCs) and adaptable to the unique needs of providers in Regional Specialized Meteorological Centres both developed and developing countries, (RSMCs), to prepare and provide products to regardless of who the users are and whether serve as a basis for NMHSs to use in the providers deliver public or commercial provision of services. Similarly, at the national products and services. The role of the WMO level, many NMHSs have focused significant Secretariat in the implementation of this efforts on improving service delivery by Strategy is to serve as facilitator and building relationships with various user coordinator. communities to better understand and respond to their needs. 1 INTRODUCTION The present Strategy seeks to build The bottom line for most government upon and institutionalize such practices to organizations is their mission. To achieve the strengthen service delivery across the entire mission, organizations need resources, but WMO by describing key strategy elements resources are often in short supply and must and activities related to a service-oriented be shared among competing organizations. culture. The Strategy focuses on This competition for scarce resources understanding the users’ value chain to gain requires NMHSs to demonstrate their value knowledge about users, the decisions they by realizing cost efficiencies while delivering must make, and how weather-, climate-, and high-quality, useful products and services. water-related information is applied to Policy-makers and the public continually minimize risk and realize benefits not only to assess the effectiveness of NMHSs based on a specific user group but also to society as a their ability to meet the service delivery standards of the nations they serve. By CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 2 whole. With this knowledge, service providers are able to develop, produce and deliver Improvements in service delivery, then, services that are useful, relevant and are a natural consequence of using QMS. responsive. NMHSs are able to measure the The WMO Strategy for Service Delivery may value of their information to society and be viewed as a supplement to the WMO continually evaluate and improve upon QMF. Even if NMHSs have no internal or services. Adopting a more collaborative external requirement to apply QMS, this approach provides everyone in the service strategy stands alone as a useful tool to delivery process – providers, users, and improve overall effectiveness of products and partners – with a clear understanding of services and customer/user satisfaction. service needs. AN EXAMPLE OF BASIC APPROACH TO LINK TO THE WMO QUALITY QMS MANAGEMENT SYSTEM (QMS) FRAMEWORK The Malaysian Meteorological Service (MMS) has implemented a process-based QMS at WMO encourages NMHSs to implement the Kuala Lumpur International Airport (KLIA) Quality Management Systems (QMS) and Forecast Center as a means of has defined a Quality Management institutionalizing effective service delivery. Framework (QMF) to provide advice on MMS implemented a QMS to improve the development and use of QMS relevant to provision of consistent products and services meteorological and hydrological that meet customer requirements; to improve organizations. The ultimate goal of a QMS is customer satisfaction through continuous to encourage and to support the continual process improvement; and to establish quality improvement of product and service quality, metrics to measure, review, and control the focusing on quality control, quality assurance forecasting processes. and quality improvement. The top management of the MMS is Quality management assesses not only responsible for the QMS processes and is the final product or service but the series of constantly upgrading its effectiveness steps or operations that occur for the final through: product or service to be produced and delivered in a manner that satisfies the  Identifying customer needs and customer. The insight gained through quality ensuring customer/client management allows NMHSs to find, fix, and satisfaction through prevent failure that might lead to a faulty questionnaires, feedback, and product or service. In the context of weather reviews; services, for example, the processes that  Regular communication with Regional make up a weather forecast and service Forecast Offices to ensure and delivery are: fulfil customer satisfaction achieved through various avenues  Data collection and analysis; like meetings, staff discussions,  Modeling for prediction; training, etc.;  Model interpretation and forecast  Determining the quality policy and production; objectives;  Dissemination of products and  Conducting management reviews; services received by users; and, and,  Understanding and use of forecasts.  Identifying and ensuring availability of resources like skilled personnel, To improve the quality of weather infrastructure, finances, training products and services, NMHSs must assess and internal audit teams. and analyze each step and sub-steps of the forecast process to determine where root problems may exist and how better to correct them. For example, QMS processes may find that a high-quality product is of marginal use because it is not received by the user in time 2 WHAT IS SERVICE DELIVERY? for decision-making. CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 3

Defining service delivery first requires a Specifically related to weather-, climate- common definition of service, which this and water-related services the following four Strategy defines as a product or activity that principles embody effective service delivery: meets the needs of a user or can be applied by a user. To be effective, services should  User engagement and feedback possess these attributes: are essential in designing and delivering effective services;  Available and timely: at time  Sharing best practices leads to and space scales that the user effective and efficient service needs; design and implementation;  Dependable and reliable:  Partnership with other delivered on time to the required international and regional user specification; organizations also engaged in  Usable: presented in user delivering services is essential in specific formats so that the client maximizing the use of weather, can fully understand; climate and water information for  Useful: to respond appropriately decision-making; and, to user needs;  The concepts and best practices  Credible: for the user to of service delivery are applied to confidently apply to decision- all WMO activities and accepted making; by the entire WMO.  Authentic: entitled to be accepted by stakeholders in the AN EXAMPLE OF COLLABORATION given decision contexts; AMONG DIFFERENT AGENCIES  Responsive and flexible: to the evolving user needs; The National Oceanic and Atmospheric  Sustainable: affordable and Administration (NOAA) and the National consistent over time; and, Science Foundation in the United States  Expandable: to be applicable to developed the Communicating Hurricane different kinds of services. Information Program (CHI) to focus on advancing the understanding by decision Service delivery, then, is a continuous, makers (e.g., emergency managers, elected cyclic process for developing and delivering officials) and the general public of hurricane user-focused services. It is further defined in outlooks, forecasts, watches, and warnings. four stages: The program illustrates how national agencies can partner to support integrated weather- • Stage 1: User Engagement - identifying society work that advances fundamental users and understanding their needs, understanding and addresses agencies’ as well as understanding the role of needs in fulfilling their missions. weather, climate, and water-related information in different sectors; 3 MOVING TOWARD A SERVICE- • Stage 2: Service Design and ORIENTED CULTURE Development - process between users, providers, suppliers, and This Strategy defines six elements and partners of creating, designing, and associated high-level activities necessary for developing services, ensuring user moving towards a more service-oriented needs are met; culture. The elements should assist providers • Stage 3: Delivery - producing, in identifying current areas of success, which disseminating, and communicating may be shared as best practices across the data, products and information (i.e., WMO, and areas where improvements are services) that are fit for purpose and needed. The elements and suggested relevant to user needs; and, activities described below serve as a • Stage 4: Evaluation and Improvement - framework to guide the development of process to collect user feedback and implementation plans that provide more performance metrics to continuously detailed processes, methodologies, and evaluate and improve upon products tools. and services. CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 4

WORKING WITH USERS IN DESIGNING related information. Users take many forms – AND DEVELOPING PRODUCTS - THE from the general public to government LEARNING THROUGH DOING (LTD) ministries, military, and private industry. Many PROJECT OF CHILE NMHSs serve customers and users in government, including disaster management, Since 2008, the Meteorological Service of agriculture, transportation, health, and Chile (DMC) has been working with the Public tourism. NMHSs may also engage with Weather Services Programme (PWSP) of intermediaries, such as the media, who WMO to implement the Learning Through represent a user group or who further Doing (LTD) project with the objective of develop products and services for end-users. enhancing Service Delivery to the fisheries, The role of the provider is to identify those agriculture and transport sectors. The Project users, including intermediaries, understand is based upon engaging users from these what they need, and how NMHSs can meet sectors with a view to determining their needs those needs, either individually or in and requirements, and to design and produce partnership with other providers and partners. improved products to meet those needs. It The evaluation of user needs is not a one- also focuses on enhancing the dissemination time requirement but a continuous and and communication channels to ensure that collaborative part of the service delivery the users access products easily. process. Multidisciplinary teams have been formed between DMC and the user sectors to steer Key Activities for Accomplishing the implementation of the project. Depending on the user group, the For example, regarding the transport provider should develop regular opportunities sector, services target the Los Libertadores to engage with users to discuss needs, and Border Complex which serves daily flow of performance. These are opportunities for the traffic between Argentina, Brazil, Paraguay, provider to better understand the user’s Uruguay and Chile. Users for meteorological business, including: their mission and goals; products and services include the public the types of decisions made on a regular transport services, tourists, freight basis; how risk is managed; and how the transportation companies, and passengers. provider’s services may contribute. The needs of each of these users are different, which requires different products Typical questions to ask of any user or and services to meet their specific needs. customer are:

The new line of products designed,  What is your mission? consists of daily weather forecasts and  How do you do it? weather warnings. The full report including all  What are your goals and how can the sectors is available at: we contribute? http://www.wmo.int/pages/prog/amp/pwsp/  How do you use our services? Activities_and_Reports_en.html  How can we make it work better?  What types of decisions do you As of 2010 the project had developed have to make? 22 new meteorological products and services;  What would help you make better improved professional and technical skills in decisions? designing and implementing products and  How do you measure success? services improvements; and enhanced dialogue and cooperation between users with Providers should facilitate the DMC, resulting in increased uptake of communication and use of weather-, climate-, meteorological products and services. and water-related information, and in some cases, provide training on specific products and services. User engagement is also a good opportunity to discuss, promote, and Strategy Element 1. Evaluate User facilitate interdisciplinary research and Needs and Decisions development efforts for user-specific products and services. At the core of effective service delivery is the user of weather-, climate- and water- CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 5

How to engage users will vary by user service providers which are not coordinated group and by country. Interactions may be with the services for airlines and air traffic formal or informal, in-person or virtual, and management. may occur through user forums, focus groups, workshops, meetings, conferences, surveys, correspondence, or face-to-face with individual users. Frequency will vary, but must be ongoing and most likely more than once a year. Engagement should include not only the users, but also partners, such as private sector organizations and the media, and other government organizations as necessary.

AERONAUTICAL METEOROLOGICAL SERVICES

Customer focus is the first and foremost of the quality management principles to be adopted by aeronautical meteorological service providers. Customer requirements are documented through relevant ICAO and national regulatory bodies, and the quality of services as perceived by the customers is monitored. The means to achieve this include verification and evaluation processes, the conduct of regular customer satisfaction surveys, liaison group meetings with representatives of the customers (e.g., pilots, dispatchers, air traffic personnel, civil aviation regulators, etc.) and visits to the operation facilities of airlines and to meteorological offices. User suggestions and feedback are formally recorded and followed up. A formal response is given to the user before a suggestion or feedback is considered closed. Specific to aviation, the liaison group meetings also provide a forum for considering and documenting agreements on local arrangements in the provision of the aeronautical meteorological services as stipulated in ICAO Annex 3 / WMO Technical Regulations [C.3.1]. This user engagement NMHSs should leverage existing WMO process also goes a long way in satisfying the guidance and tool kits (see Appendix B), as audit requirements of the Quality well as new guidance and best practices Management System and aviation safety coordinated by the WMO Secretariat to build oversight. a core set of service delivery criteria. NMHSs should develop methods and tools to Services for airports could be document and validate user needs and considered an area for improvement. These expectations and to communicate them within are not covered by ICAO regulations except the organization and to other partners as for TAFs and basic warnings, and have to be necessary. User needs should then be agreed between airport operators and service converted into requirements to be met by providers. This can lead to difficult situations existing or new products and services. when airports experiencing serious disruptions by weather elements are either User requirements should be evaluated not relying on met information at all in their to ensure that they fall within the mission of operations, or receive them from independent CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 6

NMHSs and that NMHSs have the capability TEMPLATE FOR BASIC COMPONENTS OF to meet those requirements. Evaluating user A SERVICE LEVEL AGREEMENT needs for such purposes is what this Strategy calls fit for purpose. NMHSs should not Article I. Parties evaluate user needs in isolation, but do so Describe the parties involved in the SLA collaboratively with users, providers, and partners. Fit for purpose demonstrates an Article II. Scope agreement, either implicitly or explicitly, among all involved and acknowledges some Section 2.01 Scope or all of the following: Describe the purpose and extent of the SLA

 Current and evolving user needs; Section 2.02 Assumptions  Provider capabilities, including strengths and limitations; Define any assumptions that underlie the  What services will be provided defined scope and how they will be provided;  How services will be used; Section 2.03 Goals and Objectives  Expectations of acceptable Describe what the parties are expecting to outcomes and provider accomplish with the SLA performance;  Acceptable cost or level of effort; Article III. Roles and Responsibilities and, For all parties involved in the SLA, describes  Risks inherent in applying the role of each party and the responsibilities information to decision-making. for supporting the SLA and delivering the products and services defined within

Article IV. Effective Date and Term The date the agreement is effective its duration

Article V. Delivery and Performance Describe in detail what each party is responsible for delivering and the key performance indicators to ensure compliance

Article VI. Reporting, Reviewing and Auditing Describe oversight and reporting on the agreement; when the agreement should be reviewed, and reporting points of contact

Article VII. Cost / Funding and Payment Document costs associated with the SLA, who is responsible for paying, or funding, and when payment should occur. Cost may be broken down by specific line-items, such as labor, supplies, equipment, travel, training, etc.

Article VIII. Changes and Modifications Describe the process by which changes or modifications will be made to the SLA and who is responsible for making changes

Article IX. Termination Describe terms for termination of the SLA and the process for terminating CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 7

NMHSs have limited resources and WORKING WITH THE CUSTOMER TO capacity, and therefore cannot be expected OPTIMIZE FLOOD WARNINGS to provide everything to everyone. A clear fit for purpose understood by all parties sets Flood forecasting methodology, developed by clear expectations and minimizes risk for Schröter et al1. (2008) was applied to two NMHSs while achieving the best possible small river basins in Austria and Spain. The solution for users. If appropriate, NMHSs may methodology was based on an assessment of want to explicitly outline the agreement the effectiveness and efficiency of Early reached with the user in a service level Warning Systems (EWS) for flash floods. It agreement. Agreements with other suppliers focused on the development of optimal alerts or partners may be documented in operating through the analysis of trade-offs between the level agreements. Such agreements should benefits of an increased lead time and the be prepared in such a way as to reflect the simultaneous decrease of warning reliability current scientific uncertainties associated associated with the longer lead time. with forecasting weather, climate and Determining the ability to reduce flood hydrological events. damage was based on a survey of users. The approach considered that the increase in lead Effective user engagement throughout time provided valuable opportunity for the entire service delivery process builds preparedness and prevention; whereas, the knowledge of user needs. It also builds an decease of warning reliability would cause understanding of the impact of weather-, economic loss in the case of false alarms. climate-, and water-related information on The assessment concluded that increasing protecting life and property, sustaining the lead time for flash flood events does not environment, and promoting economic produce the maximum societal benefits due to development and prosperity. This knowledge the decreasing reliability (i.e., increasing false leads to more effective products and services alarm rate). In fact, to maximize damage that are better aligned with external demands avoidance and minimize production loss due with a clear fit for purpose. to false alarms, the optimal lead time is not the longest lead time. In the Besòs basin in Strategy Element 2. Link Service Spain, for example, the optimal warning lead Development and Delivery to User Needs time was two hours. In practice, this is the time where a “watch” becomes a warning. Building knowledge of users is of marginal utility if such knowledge is not 1 Schröter, K., M. Otrowski, C. Velasco, integrated into the design, development and H.P. Nachtnebel, B. Kahl, M. Beyene, C. delivery of services. NMHSs with service- Rubin M. Gocht, 2008: Effectiveness and oriented cultures produce products and Efficiency of Early Warning Systems for services with the user at the center of the Flash-Floods (EWASE). First CRUE ERA-Net development process. This means that Common Call – Effectiveness and Efficiency NMHSs need processes and tools for of Non-structural Flood Risk Management translating requirements into tangible Measures, 132pp. Available from www.crue- products and services and then validating eranet.net that user needs and expectations are met. Linking service development and delivery to user needs necessitates an operating model that delivers forecasts and information when and how the user specifies and provides users with the necessary support. Users will have different requirements so the key is to develop an operating model that is flexible and adaptable to wide-ranging and evolving user demands. This includes workforce, systems, technical and physical infrastructures.

1 CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 8

One approach is to create a model in Key Activities for Accomplishing the form of a real or virtual co-location of meteorologists and users of weather-, NMHSs should develop and improve climate- and water-related information who upon processes and tools to document and work together to deliver products and communicate user requirements to all parties services. This approach integrates involved, including the research community, hydrometeorological information with user- developers, partners, budget and finance specific data to determine impacts on the officials, and others. Users should be brought public and industry, such as: energy grid in at various stages of the design and management; construction; flood control and development process to evaluate and test urban inundation; hospitals and health products and services to ensure that they practitioners, emergency preparedness and meet requirements and allow for optimal response; transportation; and so forth. decision-making. Meteorologists may have temporary (short- term) or permanent assignment that enables Processes should be monitored and them to work side-by-side with road evaluated. (See Strategy Element 3 for more management and maintenance specialists, details on evaluation and monitoring). public health experts, emergency responders, and others. The benefit to users is an To implement this strategy element, operational network that evolves to meet consideration should be given by the WMO to specific user needs, forecasts systems leverage existing guidance and best practices targeted to user decisions and an integrated to develop a minimum set of standards and system that aligns weather-, climate- and benchmarks for the design, development and water-related information with societal and delivery of products and services that economic impacts and user-specific integrate users throughout the process. Using information. the standards and benchmarks as a basis for evaluation, NMHSs should conduct a current PUBLIC WEATHER SERVICE PLATFORM - assessment of their service design, METEOROLOGICAL SERVICE DELIVERY development and delivery practices to identify IN THE MEGA CITY OF SHANGHAI gaps between current practices and the WMO standards. NMHSs should use The Shanghai Meteorological Bureau (SMB) structured problem solving and process of the Chinese Meteorological Administration improvement methods (see Appendix B) to (CMA) established an Integrated Public develop and implement plans to close service Weather Services (PWS) operations platform design, development, and delivery gaps. in 2009 to strengthen the integration between WMO Members are strongly encouraged to SMB, other agencies and specialized users. share results and experiences obtained The goals of the Platform are: to transform through these activities. PWS delivery into routine work by specialized duty officers; and to provide highly targeted Strategy Element 3. Evaluate and and tailored services to a variety of Monitor Service Performance and institutional, governmental, specialized users Outcomes and the public. Service delivery does not stop once the Under the direction of the Chief Service product or service has been delivered. User Officer (CSO), the Platform develops products outreach and engagement must continue to for decision-making for 26 sectors which ensure that services are received and acted include government departments, emergency upon and full benefit is achieved by the user. response agencies, the public, and weather NMHSs should have a core set of metrics to sensitive users. The daily forecasts and measure the end-to-end-to-end service warnings dissemination mechanisms for the delivery process and its outputs. Each metric PWS Platform include SMS (Short Message should only measure a specific aspect of the Service), television, radio, newspapers, process but collectively, the metrics should magazine, the Web; Basic Grid Unit enable an organization to demonstrate its management system, electronic screens, strengths and identify its areas for telephone and fax. improvement in terms of effectiveness, efficiency, impact, satisfaction, and value to its stakeholders, customers, users, partners, CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 9 and employees. Specifically, metrics should Forecast Accuracy possess the following attributes: A service-oriented culture demands use  Specific – Metrics are specific of accuracy measure from the perspective of and targeted to the area being the user, which differs from some of the measured. For example, a good accuracy measures widely applied within the metric for customer satisfaction Numerical Weather Prediction (NWP) would be direct feedback from community. A service-oriented organization customers on how they feel about should use forecast parameters which have a service or product. A poorer direct impact on users’ activities and metric would be the number of operations. Accuracy of warnings and of customer complaints because it is temperature predictions are good examples not specific nor a direct of 'service-oriented' accuracy metrics. correlation to customer Specific examples currently in use include: satisfaction and, as such, can be misleading;  Rolling average of percentage of  Measurable – Ability to collect forecast maximum and minimum data that is accurate and temperatures for today and complete; tomorrow lying within 2 degrees  Actionable – Metrics are easy to Celsius of actual values; and, understand, interpret, and act  Measure of Storm-based Tornado upon; False Alarm Rate.  Relevant – Measure only those things that are important and Customer Satisfaction relevant to an organization’s goals and objectives. A common User engagement is at the heart of a mistake is to measure everything, service delivery culture, and measurement of which is time consuming and customer, or user, satisfaction is both produces meaningless results; necessary and hugely useful in assessing  Timely – Metric data can be performance and areas for future collected when it is needed; development.  Agreed Upon – Externally-based metrics should be agreed upon by User surveys are already in widespread the NMHSs and customers, and regular use within the WMO. Surveys users, or partners. As discussed may have several levels of formality, scope under strategy element one, and standardization, ranging from frequent agreeing upon acceptable levels customer liaison visits or user workshops, to of performance is part of the bulk information gathering exercises using evaluation of user needs, or fit for standardized surveys via e-mail, the web or purpose; by telephone. Both formal and informal  Owned – Metrics should have methods for gathering user feedback are clearly identified owners. Ideally appropriate and useful. Surveys may be these owners should be undertaken at routine intervals, or following a individuals with the ability, significant weather event. Satisfaction is often influence and resources to take situational (environmental or economic) or action to ensure targets are met; influenced by public or media perceptions. and, These external factors can be minimized by  Consistent – Any two given using large and representative samples, metrics should not promote longer periods of investigation or multiple conflicting behaviors. events. Small-scale and highly-specific customer survey results are best used The following are examples of the types alongside larger survey results from which of metrics important for evaluating and statistically valid conclusions can be more monitoring service performance: easily drawn. Further, customer satisfaction results can prove important when viewed alongside accuracy metrics, highlighting differences between customer perception and CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 10 technical performance. Specific examples Australian Tsunami Warning currently in use include: Centre (JATWC) are available to emergency services and the  Telephone customer satisfaction public within 40 minutes of a surveys conducted immediately significant event in the Pacific or after a severe weather event has Indian Ocean; and, occurred or has been forecast;  Monthly measurement of  Yearly measure of customer percentage of METAR and TAF satisfaction as measured on an bulletins issued on time. external benchmarking scheme by an external assessor of public- Reach sector organizations; and,  Annual mail surveys to external As this Strategy identifies, effective users on quality of web services. services must be available, timely, and useful. Measuring the reach of services Customer Service demonstrates how well NMHSs deliver products and services that users are aware of Customer service metrics are related to and can access. In the case of public weather customer satisfaction, but tend to deal with services, there has traditionally been a monitoring the effectiveness of the processes reliance on the “push” of information to the designed to allow continuous feedback from wider public via the media – usually television users and customers, rather than the content and radio. It is necessary to measure the of the feedback itself. They can also be used effectiveness, or reach, of this to measure various aspects of the contract communication route, and the growing between NMHSs and their customers. importance of other media, such as the Web, Customer service metrics of these types tend to reach the public. Specific examples to be well-defined and can be simple to currently in use include: formulate, at least initially, though there should be regular checks for relevance and  Percentage of telephone survey the targets may need to be finely tuned to responders who affirmatively ensure they are realistic. Specific examples responded to seeing or hearing a currently in use include: warning for a specific severe weather event;  Respond to correspondence from  Number of referrals to the website all quarters within a maximum of from external sites; and, 5 working days, and answer with  Maintain full functionality of public courtesy all telephone calls within website over 99.5% of the time a maximum of 2 minutes; (three month rolling average).  95% or more of annual average of complaints answered within WORKING WITH THE USERS – KENYA 28 days; and, METEOROLOGICAL DEPARTMENT (KMD)  85% or more of annual average of all calls to be answered within The Kenya Meteorological Department 20 seconds. (KMD), through its Public Weather Services (PWS) Division serves the general public and Compliance, Timeliness & Resilience a cross section of specialized users which includes the media, the disaster community, Metrics of this kind are designed to agriculture, energy and health sectors. In measure the details of service quality away order to serve these users effectively, it has from conventional measures such as taken steps to understand their specific needs accuracy. These metrics may measure user and to organize its service delivery operations requirements, mandates, or internal to respond optimally to such needs. It has requirements for producing and accomplished this through carrying out user disseminating data and information. Specific surveys and increasing interaction with them examples currently in use include: in training workshops and through the meetings of multidisciplinary teams which  100% of Australian Tsunami have been created for the service delivery Bulletins issued from the Joint improvement. CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 11

Over the years, the scope of user Good service delivery is reliant on groups has expanded and the demand for insight into the organization’s internal new products increased. A good example is a processes. Effective and efficient internal recent request by the Kenya National processes have direct impact on the quality Examination Council (KNEC) for monthly of service delivery, the value of products and weather forecasts and weekly updates to help services, and the cost effectiveness of an them with the logistics of transporting organization’s day-to-day operations. examination papers to remote places using Measurement of an organization’s internal roads that could quickly be rendered processes should be driven largely by the impassable by heavy rains. New methods of QMS used, and the key processes defined weather dissemination options such as the therein. Specific examples currently in use RAdio InterNET (RANET) community radio include: stations have evolved too, serving areas that are highly prone to extreme weather such as  Internal and external audits to flooding and drought. The community radios review ISO9001 capabilities; and, have been very effective in issuing warnings  Periodic review of research and forecasts in local languages. activities by an external committee from the research community. KMD has also focused on public education and outreach through activities such as radio Milestones and television discussion programmes and organizing school visits to KMD facilities, in Milestones are also an internal metric order to prepare the public to respond often associated with project and programme adequately to warnings. management. Milestones measure the delivery of a product, service or system, or Impact the completion of a phase, or step, in the delivery of a product, service, or system. Measuring forecast accuracy, They should refer to specific, in-year timeliness, and reach do not tell the complete activities, with new milestones defined and story of service delivery effectiveness. agreed for the new review period. Examples Measuring the impact of a product or service include: demonstrates the value or benefit received, often measured in terms of societal or  Provide location forecasts, economic impact. Measuring impact tells observations and mountain NMHSs whether or not their products and weather hazards in local services are useful and relevant. Measuring languages by end of the fiscal impact may require a significant cultural shift year; and, within an organization because it typically  Begin deployment of next uses more subjective methods based in generation radar capability in social science. Effective impact metrics quarter XX of fiscal year YY. should be based on the input and collaboration from users and partners, Key Activities for Accomplishing including those in the social science community, such as economists and Once measures are collectively sociologists, who have expertise in identified and a methodology defined for how measuring social and economic impacts and data will be collected, NMHSs should collect human behavior. Specific examples currently baseline performance data. Baseline data in use include: informs both providers and users of current ability and capacity and serves as input when  Decrease in weather-related determining reasonable but stretch targets aviation delays; and, for future performance. Performance  Cost avoidance from measurement data should be collected and unnecessary evacuations. reviewed at regular intervals by everyone in the value chain. NMHSs should use this data to reward and promote success, as well as to Internal Processes CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 12 modify the service delivery process if performance is not meeting targets. NEW TECHNOLOGIES IN THE SERVICE OF USERS Care must be given in the design of any performance monitoring system to minimize The Hong Kong Observatory (HKO) has the number of metrics to the extent possible, evolved its service delivery by implementing and to select metrics that provide the best mobile platform and social networking measurement of service-related outcome. services in 2010. HKO developed an iPhone This is often not a simple process and the application named MyObservatory to take benefits of such measuring are best realized advantage of the iPhone’s communication if the metrics are stable over a reasonably capability and its geo-positioning function. In long period. Before implementing a system of addition to providing weather forecasts and performance metrics Members are warnings, MyObservatory automatically encouraged to review the experiences of provides the latest location-specific weather those Members who already have in place conditions, such as temperature, wind, and such monitoring. weather photos from the weather stations closest to the user. MyObservatory proved hugely popular and was on the top of free Strategy Element 4. Sustain download lists for months. HKO also began Improved Service Delivery experimenting with social networking services in 2010 by launching a Twitter service, Service delivery should continuously http://twitter.com/hkobservatory, to issue evolve, along with user demands and weather warnings and disseminate changing external drivers, such as new information. The number of HKO’s Twitter technologies or science advancements, “followers” grew from a few hundred to changing users, and evolving user capacity. thousands in a couple of months and For example, if the aviation sector improves continues to increase. By evolving their its ability to avoid weather systems, thus service delivery methods to meet changing becoming more weather resilient, the original user demands and expectations, HKO found services to the aviation sector must evolve. new, cost-effective ways to reach a greater Likewise, if a sector becomes more weather number of people. sensitive, such as the energy sector, then the services should also reflect that evolution. Evolution of services may also mean that a Strategy Element 5. Develop Skills specific product or service should be retired Needed to Sustain Service Delivery because it is no longer required by the user, or can be provided more efficiently and To achieve the elements above and effectively by another provider. succeed in user-focused service delivery, NMHSs must identify and develop the Key Activities for Accomplishing required capacity. The WMO Secretariat should also identify and develop the ability to The role of NMHSs is to ensure users are able to reap full benefit of services by facilitate and support service delivery. promoting, facilitating and coordinating Capacity includes developing the necessary improvements in interdisciplinary research, skills, processes, and technologies that observing networks, modeling, and enable, support, and sustain a service- technology. NMHSs should keep users oriented culture. Much of this Strategy has informed of new opportunities and already described needed processes and advancements – first to validate that user tools that will enable service delivery. Cross- needs continue to be met, but also to cutting across the entire Strategy and critical increase user knowledge. This can be to its success is the development and achieved through various education, outreach enhancement of workforce skills. and communication activities and should be part of ongoing user engagement described Key Strategies for Accomplishing in element one. NMHSs also have a role in institutionalizing service delivery processes internally and among partners to achieve and Efforts should be made to identify the maintain service excellence. The application necessary skills relevant to an organization’s of QMS is an effective tool for operating model and objectives and then institutionalizing processes. conduct a gap analysis to discover what skills CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 13 are lacking within the organization and how those gaps can be bridged through a 4 IMPLEMENTATION APPROACH combination of training, employee development, and recruiting. The WMO Implementing this Strategy requires Secretariat, in collaboration with the relevant more detailed action plans for developing the technical commissions, is in the process of processes, methodologies, and tools to identifying requirements for specific enable each of the strategy elements of the competencies within NMHSs and the four phases of service delivery. associated education and training needs for service delivery tasks. NMHSs must ensure The maturity and formality of service their workforce has the necessary mix of delivery among NMHSs varies significantly. Further, NMHSs operate differently due to a technical skills to meet societal demands and combination of internal and environmental user needs. Additionally, NMHSs need skills factors. For example: that enable effective service delivery. Such skills include, but are not limited to:  Some NMHSs are completely communication; customer service; government owned and offer management, problem solving; and services only to other areas of performance management. government and the public. Some are fully privatized and offer DEVELOPING SKILLS TO SUSTAIN commercial services. Many lie SERVICE DELIVERY somewhere between these extremes; The China Meteorological Administration  Some NMHSs act as data suppliers to private forecast (CMA) has made every effort to cultivate a providers, while some undertake culture of service delivery by reforming an fully commercial operations in operational-based system into a service direct competition with these oriented one. It has paid much attention to private organizations. Some play team-building, interdisciplinary research, both of these roles; outreach, application of new technology, and  Some NMHSs use their own utilization of social resources in service NWP models and forecasting and delivery. It takes special measures to production systems. Others use encourage employees to communication those supplied by external more effectively with users. CMA was organizations; and, authorized by the central government to host  Most NMHSs only provide services to their own country, a number of training courses on disaster whilst other NMHSs may offer to prevention and mitigation each year with the provide services to others. nationwide participation by nationwide city mayors. It also regularly trains the voluntary The bottom line is that a one-size-fits-all weather information deliverers at grass-root implementation approach will not be effective. level. A specific example is township leader Members need flexibility for developing their training programme which was initiated as a own unique approaches. How to implement pilot project in 2010. this Strategy within NMHSs will depend on service priorities as well as current service delivery capacity. One approach would be to Strategy Element 6. Share Best develop implementation plans that focus on Practices and Knowledge creating, growing, or sustaining a service delivery culture based on the maturity and A second cross-cutting strategy that will formality of NMHSs’ current capacity. The enable a service-oriented culture is sharing idea of creating a service delivery culture and applying best practices and knowledge may at first seem overwhelming to some across the WMO – a strategy already NMHSs and so they may wish to start highlighted earlier in this document. The incrementally by focusing on a particular WMO Secretariat should enable sharing of service area that is an organizational or practices, approaches and tools. What works governmental priority. NMHSs may want to in one country may not meet the user needs engage with the WMO Secretariat to identify in another country, but service delivery is a and implement service delivery pilot projects collaborative process where providers, that can easily demonstrate value and be partners, suppliers, and users can all learn replicated across other service areas, or even from one another. by other NMHSs. Additionally, WMO Members should seek opportunities to CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 14 transfer knowledge through advanced capacity-building approaches, such as engaging in regional partnerships and documenting best practices. All implementation approaches and plans should factor in QMS practices and processes.

APPROACH TO SERVICE DELIVERY IMPLEMENTATION IN THE TAJIK HYDROMETEOROLOGICAL SERVICE As part of the modernization of the Tajik Hydrometeorological Service, specific investments are being made to enhance service delivery. Tajik Hydromet recognizes the importance of service delivery as an element of modern meteorological service. The nascent sectors have new and emerging needs for meteorological services and in many cases, it is important for the NMS staff to have sufficient training in the user sector to be able to communicate effectively with those clients, resulting in a more collaborative approach to service delivery. The approach taken is to invest in training for both the Tajik Hydromet staff and the technical personnel from the weather-sensitive sectors in line with the four stages of the service delivery system. In the Republic of Tajikistan, the particularly important users are energy sector, agriculture and disaster reduction. For example, under this approach staff of the Emergency Management Committee (EMERCOM) local divisions will receive meteorological training to raise awareness of weather hazards and facilitate better utilization of hydrometeorological information in EMERCOM operational activities. The training will enhance EMERCOM capacity to disseminate hydrometeorological information about severe weather conditions to the regional and local branches of the Committee, and zone the country based on the probability of occurrence of hazardous hydrometeorological events.

______CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 15

Appendix A. Service Delivery Definitions Hydrometeorological Service; NHS – A National Hydrological Service. Collaborating Organization/Partner – An organization or entity (e.g., a university, a 3 Operating Level Agreement – An specialized non-government centre, a agreement among providers, suppliers, and relevant government agency) of a WMO partners detailing how a service or group of Member that provides complementary/ services would be delivered. additional weather, climate or water information to NMHSs or directly to users, 4 Product – A product is basic under terms and conditions that have been information such as observations, datasets, mutually agreed. or information that is created by an analysis or forecast process. 0 Coordinator – An organization or entity that facilitates or coordinates the 5 Providers – Individuals or entities delivery of products and services. For this that produce or acquire weather, climate or Strategy the WMO Secretariat serves in this water information or products that are then role. Working closely with Members, the supplied in support of users’ needs in this Secretariat sets standards for weather-, regard. Providers may include NMHSs, climate- and water-related products and partners, other meteorologically-relevant supporting services. This includes agencies and the private sector. This observations, data quality and Strategy focuses only on WMO NMHSs. telecommunications. The data underpinning meteorological and related products require 6 Service – A product delivered or international coordination and validation to activity that is carried out (advice, guarantee that they meet the needs of the interpretation, etc.) that meets the needs of a product generating centres. The user or that can be applied by a user. communication systems that move data and products globally are coordinated through the 7 Service Delivery – A continuous Secretariat. The assessment, and objective process for developing and delivering user- verification of products that are generated by focused services, defined by user one country and used by others may also be engagement, service design and coordinated by the Secretariat and the results development, service delivery, and evaluation shared and used in the process of improving and improvement. the quality of products for all. 8 Service Level Agreement – A 1 Fit for Purpose – results from contract between a service provider and a collaboration and dialogue among users, user or customer, it details the nature, quality, providers, suppliers, and partners and and scope of the service to be provided. Also demonstrates a clear agreement, either called a service level contract. implicitly or explicitly, among all involved. A clear fit for purpose acknowledges: 9 Users – Individuals, organizations, or intermediaries with responsibilities for  Current and evolving user needs; decisions and policies in sectors that are  Provider capabilities, including sensitive to weather, climate and water and strengths and limitations; for whom products and services are provided.  What services will be provided and If the user has paid directly for the service, how they will be provided; he/she is generally called a customer. This  How services will be used; Strategy defines users at national levels, with  Expectations of provider performance; the exception of international users in the and, aviation and shipping sectors.  Risks inherent in applying information to decision-making. ______2 NMHSs (always used in the plural) – National Meteorological Services (NMSs) and National Hydrological Services (NHSs); NMS – A National Meteorological or CBS/OPAG-PWS/ET-SPI/Doc. 3, APPENDIX I, p. 16

Appendix B. Further Reading

(1) WMO/TD-No. 1256, “Guidelines on Quality Management Procedures and Practices for Public Weather Services,” (http://www.wmo.int/pages/prog/amp/pwsp/pu blicationsguidelines_en.htm)

(2) WMO/TD-No. 1023, “Guidelines on Performance Assessment of Public Weather Services” (http://www.wmo.int/pages/prog/amp/ pwsp/publicationsguidelines_en.htm)

(3) WMO/TD-No. 1103, “Supplementary Guidelines on Performance Assessment of Public Weather Services”, (http://www.wmo.int/pages/prog/amp/ pwsp/publicationsguidelines_en.htm)

(4) For more information on QMS, see: http://www.wmo.int/ pages /prog/amp/Q MF-Web/home.html .

(5) For more information on survey designs and examples, see: http://www.wmo.int/pages/prog/amp/p wsp/surveys.htm .

______