Evidence Syntheses to Support the WHO Guidelines on Emergency Risk Communication

PROJECT REPORT

October 24th, 2016

Prepared by Emergency Preparedness, Research, Evaluation and Practice Program (EPREP) Division of Policy Translation & Leadership Development Harvard T.H. Chan School of Public Health

Prepared for Department of Communications Office of the Director-General World Health Organization

Page intentionally left blank

Harvard Chan School Project Team

Elena Savoia, MD, MPH Project Director Senior Scientist, Department of Biostatistics Deputy Director, Emergency Preparedness, Research, Evaluation & Practice Program Division of Policy Translation & Leadership Development Harvard T.H. Chan School of Public Health

Leesa Lin, MSPH Senior Project Manager Emergency Preparedness, Research, Evaluation & Practice Program Division of Policy Translation & Leadership Development Harvard T.H. Chan School of Public Health

Sarah M. Short, MPH Senior Evaluation Specialist Emergency Preparedness, Research, Evaluation & Practice Program Division of Policy Translation & Leadership Development Harvard T.H. Chan School of Public Health

Ayan Jha, MBBS, MPH Senior Evaluation Specialist Emergency Preparedness, Research, Evaluation & Practice Program Division of Policy Translation & Leadership Development Harvard T.H. Chan School of Public Health

Giorgia Argentini, MD, MSc Project Consultant for Latin America and Brazil

Noah Klein Editor Emergency Preparedness, Research, Evaluation & Practice Program Division of Policy Translation & Leadership Development Harvard T.H. Chan School of Public Health

World Health Organization Project Team

Gaya Gamhewage, MD Manager, Interventions and Guidance Infectious Hazard Management Health Emergencies Programme World Health Organization

Marsha Vanderford, PhD Director of Communications World Health Organization

Nyka Alexander, MA Communications manager (acting) Health Emergencies Programme World Health Organization

Jane Noyes, PhD Professor of Health and Social Services Research and Child Health Bangor University, Methodologist for World Health Organization

Tomas Allen Librarian World Health Organization

Report Citation:

Savoia E, Lin L, Short S, Jha A, Argentini, Klein N, Noyes J, Allen T, Alexander N, Vanderford M, and Gamhewage G. (2016) Evidence Syntheses to Support the WHO Guidelines on Emergency Risk Communication. Harvard T.H. Chan School of Public Health, USA.

Acknowledgments

We are grateful to the Guideline Development Group, comprised of external experts convened by WHO, which scoped and refined the review parameters.

We also thank Paul Bain, librarian at the Francis A. Countway Library, Harvard University, Souleymane Konate, research fellow at Harvard University for their contribution to the development of the search strategy and to Noah Klein, Harvard University for editorial support.

We are grateful to the experts in emergency risk communications that were interviewed who helped us identify additional sources of grey literature and who shared with us their experience and knowledge.

The report was developed by the Division of Policy Translation and Leadership Development under Contract Number 201400531: Evidence Syntheses to support the WHO Guidelines on Emergency Risk Communication (questions 1, 2, 6, and 8) Department of Communications, Office of the Director-General, World Health Organization.

Contact:

Leesa Lin, MSPH Senior Project Manager Emergency Preparedness, Research, Evaluation & Practice Program (EPREP) Division of Policy Translation and Leadership Development Harvard T.H. Chan School of Public Health www.hsph.harvard.edu/preparedness Phone: 617-384-9872 [email protected]

ACRONYM LIST

AHC – Acute Hemorrhagic Conjunctivitis AHI – Avian and Human Influenza APEC – Asia-Pacific Economic Cooperation CALD – Culturally and Linguistically Diverse CDC – Centers for Disease Control DEM – Disaster and Emergency Management DRIP – Disaster Risk Information Platform DRR – Disaster Risk Reduction EID – Emerging Infectious Diseases EOC – Emergency Operations Committee EM – Emergency Management EMIS – Emergency Management Information System ERC – Emergency Risk Communication FLIWAS – Flood Information and Warning System HKSARG – Hong Kong Special Administrative Region Government HPA – Health Protection Agency IM – Information Managers MBDS – Mekong Basin Disease Surveillance MECIDS – Middle East Consortium on Infectious Disease Surveillance MEDSIS – Medical Electronic Surveillance and Intelligence System MoH – Ministry of Health NGO – Non-Governmental Organization PAR – Participatory Action Research PHEP – Public Health Emergency Preparedness PIO – Public Information Officer SMS – Short Message Service SPICE – Setting, Perspective, Phenomenon of Interest, Comparison, and Evaluation VDT – Village Disaster Taskforce WHO – World Health Organization WWA – Women’s Welfare Association

TABLE OF CONTENTS

BACKGROUND...... 1 Objectives/ Questions ...... 2 METHODS ...... 3 Search methods for identification of literature ...... 3 Electronic Searches ...... 3 Articles Selection ...... 10 Quality Assessment ...... 11 Data Synthesis Methodology ...... 12 LITERATURE SEARCH RESULTS ...... 13 Evidence Quality ...... 17 Evidence Synthesis ...... 18 Prior Systematic Literature Reviews ...... 19 Question 1 ...... 20 Question 2 & Question 6 ...... 54 Question 8 ...... 85 LIMITATIONS ...... 152 CONCLUSION ...... 154 APPENDIX A. WHO SPICE Questions ...... 156 APPENDIX B. Quality Appraisal & AMSTAR Scores ...... 159 APPENDIX C. Chinese Literature Reference List ...... 160 APPENDIX D. Chinese Literature Summary ...... 162 APPENDIX E. Portuguese Literature Reference List ...... 168 APPENDIX F. Portuguese Literature Summary ...... 169 APPENDIX G. English Literature Reference List ...... 170

BACKGROUND

Risk communication is an integral part of any emergency response. In epidemics and pandemics, in humanitarian crises and natural disasters, effective risk communication allows people at risk to understand and adopt protective behaviors. It allows authorities and experts to listen to and address people’s concerns and needs so the advice they provide is relevant, trusted and acceptable.

The aim of this literature review is to support the development of guidelines in emergency risk communication (ERC) by the World Health Organization (WHO), a core public health function encompassing all types of emergencies of public health concern (natural/man- made disasters, infectious disease epidemics/pandemics, and terrorism). The purpose of the literature syntheses is to provide knowledge about building national-level capacity to integrate effective risk communication practices and structures into healthcare and response for public health emergencies. The review aims to identify evidence base and gaps relevant to the development of global recommendations for risk communication during public health emergencies. While principles, good practices, and training already exist in the field of emergency risk communication, there is no comprehensive evidence- based, up-to-date, systems-focused WHO guidance for Member States in this area. WHO has proposed the development of a new guideline on risk communication during public health and humanitarian emergencies. The proposed guideline will provide advice and direction on how countries can integrate best practices of risk communication into critical governmental and non-governmental health systems for health emergencies of various types and scopes.

To guide the literature review the WHO has developed a series of practice questions and SPICE (Setting, Perspective, Phenomenon of Interest, Comparison, and Evaluation) criteria as described in the objectives/questions section below and commissioned the Emergency Preparedness, Research, Evaluation & Practice program within the Division of Policy Translation & Leadership Development at the Harvard T.H. Chan School of Public Health to conduct the review.

OBJECTIVES/ QUESTIONS

The aim of this review is to support the development of guidelines in emergency risk communications (ERC) by reviewing the available literature related to each of the following questions posed by WHO (For more details on the SPICE breakdown for each question see Appendix A), note that the questions are not listed in sequential numbers because extracted from an original list of twelve questions:

Question 1: How can emergency risk communication best be integrated into national and international public health emergency preparedness planning and response activities?

Question 2: What are the best mechanism(s) to establish effective intra-agency, inter-agency, and/or cross-jurisdictional (such as cross-border; national with sub- national jurisdictions, etc.) information sharing for emergency risk communication?

Question 6: What are the best practices and protocols to ensure coordination of risk communication activities between responding agencies across organizations and levels of response?

Question 8: What are the best practices and most appropriate tools for gathering, analyzing, and interpreting emergency risk communication data and feedback and integrating results into emergency risk communication planning, strategy development, execution and evaluation?

METHODS

SEARCH METHODS FOR IDENTIFICATION OF LITERATURE

The literature review included searches in over twenty scientific and grey literature databases in nine languages, a review of Ministries of Health and major public health agencies webpages from Spanish, Portuguese, Italian and French speaking countries, and interviews with key experts. Our first step consisted in identifying an exhaustive list of databases (both scientific and grey literature) and websites, in order to cover a wide range of scientific, social science and general publications, aimed at providing articles and documents relevant to the four selected SPICE questions across disciplines. The search strategies were developed with support from an Information Scientist at WHO and subsequently finalized with inputs from the WHO team and a Librarian from the Francis A. Countway Library of Medicine (Harvard University). The detailed search strategies for each of these databases are listed in Tables 1, 2, 3, 4 and 5. All searches were conducted to cover the period from January 1, 2003 to February 7-9, 2016 (specific dates mentioned with individual databases below) and in nine languages (English, French, Portuguese, Spanish, Japanese, Chinese, Italian, Russian and Arabic). The basic inclusion criteria for the screening of titles and abstracts can be found below in articles selection.

ELECTRONIC SEARCHES

The detailed list of databases and corresponding search strategies for the literature review are provided below (Table 1-2). We searched electronic databases hosted by countries speaking the following languages: English, Chinese, Spanish, and Portuguese. In order to identify literature from low, middle, and high income countries, we included 9 languages in our review: English, French, Portuguese, Spanish, Japanese, Chinese, Italian, Russian, and Arabic, and interviewed experts from low and middle income countries.

ENGLISH We searched the following English language electronic databases for eligible studies for the scientific literature displayed in Table 1 and for the grey literature in Table 2:

Table 1. Search Strategy for English databases

Source Search Strategy 1. Medline (PubMed) ("Disaster Planning"[mesh] OR "Civil Defense"[majr] OR February 7, 2016 "Disasters"[mesh] OR "Disease Outbreaks"[Mesh] OR pandemic*[tiab] OR epidemic*[tiab] OR outbreak*[tiab] OR disaster*[tiab] OR emergency planning[tiab] OR emergency preparedness[tiab]) OR "preparedness"[tiab] OR "mitigation"[tiab] OR "influenza, human"[mesh] OR "ebola"[tiab] OR "terrorism"[mesh]

OR "floods"[tiab] OR "earthquake"[tiab] OR "hurricane"[tiab] OR "cyclone"[tiab] AND ("Communication"[Mesh] OR communication*[tiab]) AND ("Risk"[mesh] OR "Risk Reduction Behavior"[mesh] OR "Risk Management"[mesh] OR risk[tiab]) AND ("2003/01/01"[PDAT] : "3000/12/31"[PDAT]) 2. Cochrane Cochrane Central Register of Controlled Trials : Issue 1 of Databases 12, January 2016 February 7, 2016 Publication Year from 2003 to 2016, in Cochrane Reviews (Reviews and Protocol), Other Reviews, Trials, Methods Studies, Technology Assessments, Cochrane Groups (0) Title, Abstract, Keywords: Disaster AND communication Outbreak AND communication Epidemic AND Communication Pandemic AND Communication Preparedness AND communication Terrorism AND communication 3. PDQ-Evidence ("Disaster Planning" OR "Disease Outbreaks" OR pandemic OR February 7, 2016 epidemic OR outbreak OR disaster OR emergency planning OR emergency preparedness OR preparedness OR mitigation OR influenza OR Ebola OR terrorism OR flood OR earthquake OR hurricane OR cyclone) AND (communication) AND (Risk OR "Risk Reduction Behavior" OR "Risk Management") In: Title or Abstract Year: 2003 - 2016 4. WHO Global Health ("Disaster Planning" OR "Disease Outbreaks" OR pandemic OR Library epidemic OR outbreak OR disaster OR. emergency planning OR February 8, 2016 emergency preparedness OR preparedness OR mitigation OR influenza OR Ebola OR terrorism OR flood OR earthquake OR hurricane OR cyclone) AND (communication) AND (Risk OR "Risk Reduction Behavior" OR "Risk Management") In: Title, Abstract, Subject; EXCLUDE MEDLINE (1056 searches); 2003 -present 5. Social Sciences Search by Title, Abstract, Abstract ID & Keywords; All dates; All SSRN Research Network Networks (SSRN) 1.Disaster >> communication (search within results for Disaster February 8, 2016 search) 2. Hazard >> communication 3. Pandemic >> communication 4. Epidemic >> communication 5. Outbreak >> communication

6. Preparedness >> communication 7. Terrorism >> communication 6. Embase ('disaster planning'/exp OR 'civil defense'/exp OR 'disaster'/exp OR February 8, 2016 'epidemic'/exp OR 'pandemic'/exp OR outbreak*:ab,ti OR disaster*:ab,ti OR 'emergency planning': ab,ti OR epidemic*:ab,ti OR pandemic*:ab,ti OR 'emergency preparedness': ab,ti) AND ('interpersonal communication'/exp OR 'medical information'/exp OR communication*: ab, ti) AND ('risk'/exp OR risk: ab,ti) AND [embase]/lim AND [2003-2016]/py 7. CINAHL MH ("Disasters+" OR "Civil Defense" OR "Disease Outbreaks") OR TI February 8, 2016 (outbreak* OR epidemic* OR pandemic* OR disaster* OR "emergency planning" OR "emergency preparedness") OR AB (outbreak* OR epidemic* OR pandemic* OR disaster* OR "emergency planning" OR "emergency preparedness") AND MH ("Communication") OR TI (communication) OR AB (communication) AND MH ("Risk Management") OR TI (risk) OR AB (risk) 2003-2016 8. PsycINFO DE ("Emergency Preparedness" OR DE "Disasters" OR "Natural February 8, 2016 Disasters" OR "Epidemics" OR "Pandemics") OR TI (outbreak* OR epidemic* OR pandemic* OR disaster * OR "emergency planning" OR "emergency preparedness") OR AB (outbreak* OR epidemic* OR pandemic * OR disaster* OR "emergency planning" OR "emergency preparedness") AND DE ("Communication" OR "Electronic Communication" OR "Interpersonal Communication" OR "Communications Media" OR "Mass Media" OR "Multimedia" OR "Social Media" OR "Telecommunications Media") OR TI (communication) OR AB (communication) AND DE ("Risk Management") OR TI (risk) OR AB (risk) Dates: 2003-2016 9. Communication TX (outbreak* OR disaster* OR epidemic* OR pandemic* OR Abstracts "emergency planning" OR "emergency preparedness") AND TX (risk) February 8, 2016 10. ERIC DE ("Natural Disasters" OR "Emergency Programs" OR "Civil February 8, 2016 Defense") OR TI (outbreak* OR epidemic* OR pandemic* OR disaster* OR "emergency planning" OR "emergency preparedness") OR AB (outbreak* OR epidemic* OR pandemic* OR disaster* OR "emergency planning" OR "emergency preparedness")

AND DE ("Organizational Communication" OR "Communication (Thought Transfer)" OR "Communication Strategies" OR "Computer Mediated Communication") OR TI (communication) OR AB (communication) AND DE ("Risk" OR "Risk Management") OR TI (risk) OR AB (risk) 11. Applied Social SU.EXACT("Avalanches" OR "Cyclones" OR "Disasters" OR "Drought" Sciences Index and OR "Earthquakes" OR "Ecological disasters" OR "Famine" OR Abstracts (ASSIA) "Firestorms" OR "Floods" OR "Hurricanes" OR "Natural disasters" OR February 8, 2016 "Tornadoes" OR "Volcanoes" OR "Disaster management") OR ti (outbreak* OR epidemic* OR pandemic* OR disaster* OR "emergency planning" OR "emergency preparedness") OR ab(outbreak* OR epidemic* OR pandemic* OR disaster* OR "emergency planning" OR "emergency preparedness") AND SU.EXACT("Communication" OR "Risk communication") OR ti(communication) OR ab(communication) AND SU.EXACT("Risk communication" OR "Risk management") OR ti(risk) OR ab(risk) Dates: 2003 - 2016 12. Sociological SU.EXACT("Disasters" OR "Natural Disasters" OR "Disaster Abstracts Preparedness" OR "Epidemics") OR ti(outbreak* OR epidemic* OR February 9, 2016 pandemic* OR disaster* OR "emergency planning" OR "emergency preparedness") OR ab(outbreak* OR epidemic* OR pandemic* OR disaster* OR "emergency planning" OR "emergency preparedness") AND SU.EXACT("Organizational Communication" OR "Communication" OR "Computer Mediated Communication" OR "Publicity") OR ti(communication) OR ab(communication) AND SU.EXACT("Risk") OR ti(risk) OR ab(risk) Dates: 2003 - 2016 13. Web of Science Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, ESCI February 9, 2016 Timespan=2003-2016 TS=("outbreak*" OR "epidemic*" OR "pandemic*" OR "disaster*" OR "emergency planning" OR "emergency preparedness") AND TS=(communication) AND TS=(risk) 14. Russian Academy TX (outbreak* OR disaster* OR epidemic* OR pandemic* OR of Sciences "emergency planning" OR "emergency preparedness") Bibliographies AND February 9, 2016 TX (communication*)

Additionally, we searched the following grey literature databases.

Table 2. Search Strategy for English Grey Literature Databases

Source Search Strategy 1. Bielefeld Title search; Year: 2003-2016; Academic Books/articles/journals/reports/papers/lectures/theses/reviews/primary Search Engine data February 8, Disaster AND communication 2016 Outbreak AND communication Epidemic & Communication Pandemic & Communication Preparedness AND communication Terrorism AND communication 2. PAIS SU.EXACT("Disasters" OR "Natural Disasters" OR "Disaster Preparedness" International OR "Epidemics") OR ti(outbreak* OR epidemic* OR pandemic* OR disaster* February 8, OR "emergency planning" OR "emergency preparedness") OR ab(outbreak* 2016 OR epidemic* OR pandemic* OR disaster* OR "emergency planning" OR "emergency preparedness") AND SU.EXACT("Risk Communication" OR "Communication" OR "Computer Mediated Communication" OR "Publicity") OR ti(communication) OR ab(communication) AND SU.EXACT("Risk" OR "Risk Communication") OR ti(risk) OR ab(risk) Date: 2003-2016 3. Policy File risk communication disaster* February 9, risk communication epidemic* 2016 risk communication outbreak* risk communication "emergency planning" risk communication "emergency preparedness" subject(crisis management) keyword "risk communication"

CHINESE/MANDARIN We searched the following databases for articles published in Chinese/Mandarin:

Table 3. Search Strategy for Chinese/Mandarin Databases

Source Search Strategy China Search query：主题=(风险+危机+应急+重大灾害+埃博拉+甲型流感+SARS+非典+非 Academic 典型肺炎+公共危机+风灾+巨灾+中东呼吸综合症+地震+流感+雪+水+旱+涝+洪+ 突发 Journals 公共卫生事件+突发公共事件+风险传播+风险沟通+风险信息+危机传播+危机沟通+危 Full-text 机信息+应急传播+应急沟通)*(传播+沟通)并且关键词=(突发公共卫生事件+突发公共 Database 事件+风险传播+风险沟通+风险信息+危机传播+危机沟通+危机信息+应急传播+应急 - CNKI 沟通)-高校-血-医患-护理-金融-生态-药-信息安全-保险-浅谈-论述-概述-品牌-HIV-急诊 (China 急救个人信息并且年 National - - -sex Knowledg =(2016+2015+2014+2013+2012+2011+2010+2009+2008+2007+2006+2005+2004+ e 2003) (精确匹配),Subjects：预防医学与卫生学,感染性疾病及传染病,急救医学,公安行 Infrastruc 政工作,交通管理,社会科学理论与方法,新闻与传媒,图书情报与数字图书馆,市场研究与 ture) 信息,管理学,领导学与决策学;No sorting;

Search mode: Single-database search

PORTUGUESE We searched the following electronic databases for articles published in Portuguese:

Table 4. Search Strategy for Portuguese Databases

Source Search Strategy* LILACS/SCielo 1. NATURAL DISASTERS AND RISK COMUNICATION (DESASTRES Mirage - Fiocruz NATURAIS E COMUNICACAO) 2. EPIDEMICS/PANDEMICS AND RISK COMUNICATION (EPIDEMIAS E COMUNICACO DO RISCO) 3. MASS EVENTS AND COMUNICATION OF RISK (EVENTOS DE MASSA E COMUNICACAO DO RISCO) 4. OLIMPICS AND FOOTBALL WORLD CHAMPIONSHIP AND RISK COMUNICATION

SPANISH We searched the following databases for articles published in Spanish:

Table 5. Search Strategy for Spanish Databases

Source Search Strategy* REDYLAC/SCielo “Epidemias y pandemias y desastres naturales y grandes eventos y comunicación del riesgo desde el 2003 Liliacs y Scielo"

The Portuguese and Spanish search engines cited above did not allow for the use of a complex search string, such as the one used in Medline (Table 1). Therefore, multiple search strings had to be used, as described above (Table 4 and Table 5).

OTHER SOURCES In addition to the scientific literature and grey literature databases, we reviewed the webpages of a number of Ministries of Health and public health agencies from French, Spanish, Portuguese speaking countries and Italy, interviewed 14 key experts in different regions and countries around the globe (those interviewed are included in Table X), and reviewed the reference lists (cross-referencing) of the articles judged to be relevant to the questions of interest.

The following table provides the name and country and/or organization of the key experts we interviewed.

Table 6. Key Experts Interviewed

Name Country, Title, Organization

1 Massimo Ciotti Deputy Head of Unit, Public Health Capacity and Communication, Head of Section Country Preparedness Support, European Centre for Disease Prevention and Control 2 Mohammed Nour Public health Physician, Supreme Council of Health, Doha, Qatar 3 Chris Nelson Senior Political Scientist, RAND Corporation, USA 4 Germain Thinus Policy Officer, DE SANTE – Health Threat Unit, European Commission, Luxembourg 5 Aphaluck Bhatiasevi Technical Officer (Risk Communication), World Health Organization 6 Andrew Black Exercise Manager, Emergency Response Department, Health Protection and Medical Directorate, Public Health England 7 Andrea Bernasconi Consultant, MSF (Barcelona) 8 Mihir Bhatt Director, All India Disaster Mitigation Institute, Gujarat,India 9 Arnold Howitt Executive Director of the Ash Center, Harvard Kennedy School, Boston, MA, USA

10 Professor Virginia Murray Consultant in Global Disaster Risk Reduction, Public Health England 11 Ute Rexroth Medical Specialist, Surveillance Unit, Koch Institut, Germany 12 Silvia Bino Associate Professor of Infectious Diseases, Head, Control of Infectious Diseases Department, Institute of Public Health, Tirana, Albania 13 Paula Vasconcelos Direcção-Geral de Saúde, Ministry of Health, Lisbon, Portugal 14 Eva Benelli Journalist, Zadig Ltd. (ASSET Project), Italy

ARTICLES SELECTION

The titles and abstracts of the 1,899 identified articles were reviewed to determine eligibility for inclusion. The full text article was obtained for those articles that were determined potentially relevant by at least one of the four reviewers (SS, AJ, LL, and ES) after reading the abstract. Before reading the full text of the selected articles the reviewers met several times and reviewed a sample of articles to develop inclusion/exclusion criteria (as described below), constant consultation with the WHO team members occurred during the articles selection process. Each article considered potentially relevant (n=881) to address the WHO questions was reviewed by one reviewer, and a random sample of articles was selected for double review. All the articles (129) included as directly (71) or indirectly (58) relevant were reviewed by at least two reviewers with the exception of articles in the Chinese language for which only one reviewer was assigned to the review. In the event that there was disagreement between the reviewers, it was either resolved by discussion or when necessary, a third party was consulted. No article was classified as partial or uncertain.

In the title review phase, the reviewers specifically identified articles for inclusion that included the terms "communication" or related words (risk/ crisis communication), along with the event of interest like "pandemic,” "epidemic,” "H1N1,” "flu"/ "influenza,” "terrorism,” and natural disaster ("earthquake"/ "cyclone"/ "hurricane"). Articles were also included if the terms "risk management,” "preparedness", "preparedness exercise" or “knowledge” were included in the title. Articles were included if they addressed components of the SPICE breakdown, specific to the ‘phenomenon of interest.’ A large number of articles were marked as "unclear," and the full text was reviewed prior to exclusion.

Articles were excluded if they were editorials, opinions or commentaries without any empirical evidence, independently of the study design, if the topic was unrelated to the questions, or written in languages other than the nine languages outlined above. The flow of the literature is provided below under “literature search results”.

QUALITY ASSESSMENT

As this review included a wide range of empirical data, we utilized a number of different tools to assess the literature (see Table 7 for the type of study and quality appraisal tool). Studies were included in the review regardless of the study design or quality of the study as assessed by the use of the tools described below.

Table 7. Type of Study Design and Quality Appraisal Tool Used

Quality Appraisal Type of Study Design Link to the tool Tool Qualitative Study Critical Appraisal http://www.casp-uk.net/#!casp-tools- Design (including Skills Programme

checklists/c18f8 content Analysis) (CASP) Pluye, P., Robert, E., Cargo, M., Bartlett, G., O’Cathain, A., Griffiths, F., Boardman, F., Gagnon, M.P., & Rousseau, M.C. (2011). Mixed Methods Proposal: A mixed methods appraisal tool for Appraisal Tool Mixed Methods systematic mixed studies (MMAT) – Version reviews. Retrieved on [date] from 2011 http://mixedmethodsappraisaltoolpublic.pbw orks.com. Archived by WebCite® at http://www.webcitation.org/5tTRTc9yJ Cochrane Risk of http://handbook.cochrane.org/chapter_8/8_as RCT

Bias sessing_risk_of_bias_in_included_studies.htm http://www.bmj.com/content/suppl/2004/05 Survey/ Questionnaire Appendix E

/27/328.7451.1312.DC1 Critical Appraisal Cohort Study Skills Programme http://www.casp-uk.net/ (CASP)

Literature Reviews AMSTAR http://amstar.ca/Amstar_Checklist.php

DATA SYNTHESIS METHODOLOGY

We initially identified four methodological streams: quantitative (randomized group comparison, non-randomized group comparison and descriptive surveys), qualitative (i.e. interviews, focus groups, and textual analysis), mixed-methods (use of quantitative and qualitative methods) and case studies (description of the response to a particular emergency/incident). We synthesized the qualitative and quantitative literature by using the Best Fit Framework Synthesis described by Booth et al (2016)1 and Barnett-Page (2008)2 which involves searching for existing theories/conceptual frameworks and creating a composite framework describing findings within each theme/concept. A mixed deductive-inductive process was used to synthesize findings across studies. We started by using the SPICE components for each of the WHO questions to develop the themes and continued by adding themes derived from the literature and integrated into the framework following team discussion. Figure 1 below present the process used for the syntheses.

Figure 1: Syntheses process

Synthesized findings across Findings from Individual Individual studies based on Synthesis of findings across Studies by question ad the derived methods method themes/concepts

1 Booth, A., Noyes, J., Flemming, K., Gerhardus, A., Wahlster, P., van der Wilt, G.J., Mozygemba, K., Refolo, P., Sacchini, D., Tummers, M., and Rehfuess, E. (2016) Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions [online]. Available from: http://www.integrate- hta.eu/downloads/ 2 Barnett-Page, E. and J. Thomas. (2009). “Methods for the synthesis of qualitative research: a critical review.” BMC Medical Research Methodology 9(59).

LITERATURE SEARCH RESULTS

We identified a total of 8,182 titles and abstracts and conducted a full text review on 880 articles resulting in the inclusion of 90 articles in the evidence syntheses. Figure 2 describes the flow of the literature identified through the systematic review of the databases.

Figure 2: Flow of the Literature

Articles identified through database Articles identified through & grey literature searching additional sources

N= 8,182 N= 33 (E) (E: 5,913, C: 1,415, P: 481, S: 373)

Total Articles Screened Articles Excluded N= 8,215 N= 6,316 (E: 5,946, C: 1,415, P: 481, S: 373) (E: 5,108, C: 498, P: 382, S: 328) 23% Total Abstracts Screened after Title Review

N= 1,899 (E: 838, C: 917, P: 99, S: 45)

46%

Full Text Articles Retrieved

N= 880 (E: 463, C: 325, P: 52, S: 40)

10%

Total Articles included in the Evidence Syntheses

N= 90 (E: 71, C: 18, P: 1, S: 0)

Table 8 below reports the total number of articles identified for each question. The majority of the articles fall under question 8.

Table 8. Literature Directly Coded to the WHO Questions, by language database

Language WHO Question* Q1 Q2 /Q6 Q8 Chinese 15 2 11 English 6 21 45 Spanish 0 0 0 Portuguese 0 1 0 Total 21 24 56

In Table 9 we provide the study type/principal method used for the literature in English and Portuguese by question:

Language WHO Question*

Q1 Q2/Q6 Q8

Quantitative 0 3 23

Qualitative 0 10 19

Mixed 2 3 2 Methods

Case study 4 5 1

Total 6 21 45

In Table 10 we provide the study type/principal method used for the literature in Chinese by question:

Language WHO Question* Q1 Q2/Q6 Q8

Quantitative 5 1 4

Qualitative 6 1 4

Mixed 4 0 3 Methods

Case study 0 0 0

Total 15 2 11

In Figure 3 below we provide the distribution of countries represented in the literature included in the evidence syntheses. Thirty countries are represented by the overall evidence syntheses across the four questions, the preponderance of the articles is from the US, China (including Hong Kong and Taiwan), UK and the Netherlands. 16 out of 18 Chinese studies were conducted in China; Singapore and Taiwan each has one study; the only Portuguese article was from Brazil.

Figure 3. Distribution of countries represented in evidence syntheses*

35 29 30

25 23

20 15 10 8 5 4 3 5 2 1 1 2 1 2 1 2 2 1 2 1 2 1 1 1 1 1 1 1 1 2 1 2

NumberofArticles 0

Lao

Iran

USA

Italy

India

Israel

Brazil

Japan

Jordan

France

Turkey

Mexico

Canada

Norway

Belgium

Vietnam

Thailand

Australia

Germany

Myanmar

Indonesia

Singapore

Cambodia

Palestinian

SouthKorea

Netherlands

SierraLeone NewZealand

China HK (incl and TW) Figure 4 below shows the distribution of articles by type of emergency as represented by the literature. The majority of the articles are about pandemic influenza and other infectious diseases, followed by all hazards, terrorism, anthrax, floods and weather-related threats including hurricane and tornado.

Figure 4. Distribution of disaster type included in the evidence syntheses *

35 30 6 25 20 26 15 12 10 1 5 0 0 1 1 0 10 7 1 0 0 0 5 1 0 4 5 4 4 2 2 3 0 1

English Non English

*Terrorism includes Bioterrorism, Chemical Attack, Radiological and Nuclear Terrorism * Pandemic Influenza and Other Infectious disease includes SARS, West Nile virus, Ebola, H1N1, Avian Flu, H7N9, acute Hemorrhagic Conjunctivitis and measles outbreaks *Among the 19 non-English literature included in this review, the only article published in Portuguese studied flood and the rest were published in Chinese.

*Note: These columns are not mutually exclusive. One article could list one or more disaster types.

EVIDENCE QUALITY

The articles included in the evidence synthesis were assessed using one or more of the quality appraisal tools referenced in Table 7 above. All articles, regardless of their quality, were included in the evidence synthesis. We have provided the CERQual assessment of confidence in the evidence against each article and also for the overall synthesis of findings in subsequent question-wise tables. Table 11 is the Cochrane Risk of Bias assessment for the only two experimental studies included in our synthesis. Further, AMSTAR scores for five prior systematic reviews are provided in Appendix B.

Table 11. Quality Appraisal Scores

Articles (English) Cochrane Risk of Bias assessment (RCT) Maarten Schraagenb Baseman 2013a 2010 Random sequence generation ? ?

Allocation concealment ? (N/A) ? (N/A)

Blinding of participants and personnel + ? (N/A)

Blinding of outcome assessment + -

Incomplete outcome data: efficacy outcomes + +

Selective reporting + +

Other sources of bias - - Legends: (+) low risk of bias (-) high risk (procedure not at all mentioned) (?) unclear risk (procedure merely mentioned, not described)

a This is a sub-study in a large, multi-center, triple-blind RCT; data involves only 1 arm (King County, WA) and 528/530 (enrolled) health care providers (HCP) were included in the analysis. For a specific objective of this current paper (to assess the “alert fatigue” in HCPs due to volume of public health messages), the analysis involved only the intervention group (n=125); but other measures (like recall rate etc.) were estimated on the total sample. However, the analysis makes this paper more like an observational study. Notably, the main RCT methods are very robust. Additional note – the full trial was published in 2016 and fell outside of the search period so is not included. The full trial can be located here:

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S1935789315001391

b this is basically a controlled laboratory experiment comparing team structures

Thus, though both these studies are assessed as RCTs, in interpretation they are as good as quantitative observational studies - both are non-randomized, both have no concealed allocation. The Baseman 2013 can be treated as an observational study as per its analysis, and the Maarten Schraagen 2010 study has questionable quality criteria (- or ?) for most important parameters.

EVIDENCE SYNTHESIS

During the evidence synthesis we identified themes and sub-themes from the empirical literature for each of the four WHO questions to summarize the findings in a meaningful way. A mixed deductive-inductive process was used to synthesize findings across studies. We started by using the SPICE components for each of the WHO questions to develop the themes and continued by adding themes that were derived from the literature and integrated into the framework following team discussion.

However across all questions findings can be summarized but cannot be easily aggregated due to the fact that in most cases the articles describe practices from the field which are very context specific and there is no consistency of the specific outcomes being addressed by the individual studies or clarity on how such outcomes are defined; outcomes are to be interpreted in the context of the themes generated during the review and by which findings have been categorized. For each of the questions below, the direct literature identified through the English, Portuguese, and Chinese databases is presented in summaries and tables. Only for question 2 & 6 the Chinese literature has been integrated into the syntheses of the English and Portuguese literature because only for this question we identified similarities in the type of studies across languages. For questions 1 and 8 the syntheses of the Chinese literature is presented separately. A description of the methodological challenges encountered in the search and during the categorization and syntheses of articles across multiple languages is provided in the limitations section.

PRIOR SYSTEMATIC LITERATURE REVIEWS

Five systematic literature reviews provided indirect evidence towards the four WHO questions (AMSTAR scores can be found in Appendix B). However, while they address issues related to ERC, they do not provide enough relevant information to specifically answer any one of the questions object of this review. A brief summary of findings of each review is provided below: Fitzpatrick-Lewis (2010) aimed to identify the effectiveness of communications strategies and factors that impact communications uptake related to environmental health risks. Authors conclude that no single method of message delivery is best. Risk communication strategies that incorporate the needs of the target audience(s) with a multi- faceted delivery method are most effective at reaching the audience Ruggiero et al (2013) aimed to clarify current knowledge on the contribution of communications to crisis management of terrorism incidents. Authors conclude that terrorism crises call for fast information updates that, using a multi-channel approach, can be tailored to fit different needs and (social) media habits. Bradley et al (2014) aimed to identify, appraise, and synthesize the findings of studies on the effects of risk communications interventions during the four stages of the disaster cycle. The authors do not make specific recommendations for practice but highlight the need for high quality randomised trials and appropriately analysed cluster randomised trials in the field of disaster risk communication where these can be conducted within an appropriate research ethics framework. Lin et al (2014) aimed to characterize the scientific literature that examined issues related to communications to the public during the H1N1 pandemic. The authors conclude that trust in public officials and source of information, worry and levels of knowledge about the disease, and routine media exposure as well as information-seeking behaviors, were related to greater likelihood of adoption of recommended infection prevention practices. When addressed in communication interventions, these factors can increase the effectiveness of the response to pandemics. Bish et al (2014) aimed to review how to identify the key demographic and attitudinal determinants of three types of protective behavior during a pandemic: preventive, avoidance, and management of illness behaviors, in order to describe conceptual frameworks. The research shows that there are demographic differences in behaviour: being older, female and more educated, or non-White, is associated with a higher chance of adopting the behaviours. There is evidence that greater levels of perceived susceptibility to and perceived severity of the diseases and greater belief in the effectiveness of recommended behaviours to protect against the disease are important predictors of behaviour. There is also evidence that greater levels of state anxiety and greater trust in authorities are associated with behaviour.

QUESTION 1

QUESTION 1: How can emergency risk communication best be integrated into national and international public health emergency preparedness planning and response activities?

COUNTRY & DISASTER TYPE DISTRIBUTION– ENGLISH LITERATURE

We identified only six articles that provide examples of how ERC can be integrated into national and international PHEP planning and response. These six articles present examples of practices from the field which are relevant to the WHO questions. However, such mechanisms are presented in the context of specific countries and emergency scenarios. In the evidence syntheses section below we provide a summary of their content and summarize them by major theme/outcome.

Figure 5. Distribution of countries included in the English language evidence synthesis for Question 1.*

3.5

3 3

2.5

2 NumberofArticles 1.5 1 1 1 1 1 1 1 1

0.5

0 Cambodia China Indonesia Japan Lao Thailand UK Vietnam (including HK and TW)

*Note: These columns are not mutually exclusive. One article could list one or more countries.

Much of the literature coded as directly relevant to question 1 is focused on pandemic influenza.

Figure 6. Distribution of disaster type included in the English language evidence syntheses.*

4.5 4

4 3.5 3 2.5 2

2 NumberofArticles 1.5 1 0.5 0 Pandemic Influenza and other Infectious Diseases All Hazards

EVIDENCE SYNTHESIS The six articles identified for Q1 present examples of mechanisms that might lead to the integration of ERC functions in the leadership structure, such mechanisms can be summarized under three themes/outcomes: 1) placing ERC functions into the national leadership structure, 2) creating organizational proximity of ERC practitioners to national response leadership, and 3) developing laws, regulations, policies, and frameworks in support of ERC. A cross-cutting theme, in addition to the three cited above, resulted to be the use of training and exercises as a mechanism for testing the effectiveness of the system. Figure 7 below describes these concepts and the following sections summarize the content of the articles by theme/outcome. The following tables (Table 12 and Table 13) describe the study findings within methodological stream and across methods for the English literature.

Figure 7. Themes/outcomes derived during the evidence syntheses of Q1

• Organizational Proximity of ERC Practitioners to National Health Leadership Response

Training • Placement of ERC Communication Functions in National Leadership Structure & Exercises • Development of Laws, Regulations, Policies & Frameworks

Table 12. Synthesis of Findings within Methodological Stream and Evaluation of Certainty/Confidence – Q1

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

Placement of Yen M-Y, Wu T-SJ, Chiu AW- Acute Hemorrhagic The integration of ERC in national Moderate Minor concerns ERC Functions in H, Wong W-W, Wang P-E, et Conjunctivitis (AHC) public health emergency planning on methodology National al. Taipei’s Use of a Multi- Case Study Approach - and response activities can be and adequacy of Leadership Channel Mass Risk Survey supported by the renovation of data Structure Communication Program to Taiwan components of the leadership Rapidly Reverse an Epidemic structure when needed. The study of Highly Communicable from Yen et al (2009) describes Disease. PLoS ONE. 2009. how, following the SARS outbreak, 4(11): e7962. the Taipei City Government doi:10.1371/journal.pone.00 initiated a new public health plan 07962 using an integrated infection control system against emerging infectious diseases (EID). This new system integrated early detection of outbreaks (particularly in hospitals and schools), epidemiological investigation, and epidemiologically based public health prevention and control policies. The renovated Division of Disease Control and Prevention (Taipei’s CDC) also became the core operational unit for implementing crisis management

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

procedures and facilitating policy. These systematic upgrades allowed for quick enactment of multi-channel risk communication measures during the 2007 outbreak of conjunctivitis. Rather than rely on the media alone to convey productive messages regarding the epidemic, Taipei’s CDC was able to implement its Multi-Channel Mass Risk Communication Program to reach the public directly during critical points in the epidemic.

Placement of Hanvoravongchai P., Pandemic Influenza The integration of ERC functions in Moderate Minor concerns ERC Functions in Adisasmito W., Chau PN., Mixed Method Systematic national leadership structures can on methodology National Alexandra Conseil A., de Sa Rapid Assessment be achieved by the creation of and adequacy of Leadership J., Krumkamp R., Mounier- analytical framework: preparedness committees and data

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

Structure Jack S., Phommasack B., Review of published data preparedness frameworks within Putthasri W., Shih CS., Touch and KI interviews each country political context. S., Coker R. for the Cambodia Indonesia Lao Hanvoravongchai et al (2010) AsiaFluCap project. PDR Taiwan Thailand presents the finding of a rapid Pandemic influenza Vietnam assessment of pandemic preparedness and health preparedness in six countries. In systems challenges in Asia: all countries, the pandemic results from rapid analyses preparedness committees were in 6 Asian countries. BMC headed by the president or prime Public Health. minister or his/her representative. 2010. 10:322 In well-established health systems, pandemic preparedness is integrated within existing mechanisms, notably within the national disaster preparedness framework. In countries with a weak healthcare system, new vertical programs had to be established to manage and coordinate pandemic preparedness and response. The nature of pandemic governance also depends on the existing political context. Decentralized countries have greater challenges to deal with during both outbreaks and pandemics. In a decentralized system, like in Indonesia, the level

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

of political commitment may affect the level of investment in pandemic preparedness. Organizational Cope JR., Frost M., Richun L., General Disasters Actions should be taken to Moderate Minor concerns Proximity of ERC Xie R. Assessing Knowledge Multi-province survey of enhance the timely release of on methodology Practitioners to and Application of public health officials & information for example by and adequacy of National Health Emergency Risk interviews increasing the authority of various data Leadership Communication Principles China departments in releasing relevant Response Among Public Health information that has been Workers in reasonably verified. Cope et al China. Disaster Med Public (2014) presented findings from a Health Prep. 2014 June. 8(3): survey and interviews conducted 199–205. with public health workers in China. A prominent issue detected by the authors was the inability of some departments to release information. This restriction was noted across departments, as well as through the chain of command, starting at the local level through to the provincial and national levels. Organizational Chambers J., Barker K., Pandemic Influenza Organizational proximity of ERC Moderate Minor concerns Proximity of ERC Rouse A. Reflections on the Case study practitioners and functions to the on methodology Practitioners to UK’s approach to the 2009 UK national level leadership can be National Health swine flu pandemic: achieved by enacting responsive Leadership Conflicts between national multi-level communication Response government and the local mechanisms and by harnessing

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

management of the public local public health expertise. health response. Health & Chambers et al (2012) describe Place. 2012. 18: 737-745. that, during the response to pandemic influenza in the UK, the only formal channel of communication ‘up the chain of command’ was indirect, through the regional Health Protection Agency (HPA). The UK relied on HPA, to lead, coordinate and manage the operational response at the local level—a role for which it was ill-equipped, given that its main mission is to provide disease surveillance and epidemiological advice at the national and local levels. Development of Yen M-Y, Wu T-SJ, Chiu AW- Acute Hemorrhagic The integration of ERC in national Moderate Minor concerns Laws, H, Wong W-W, Wang P-E, et Conjunctivitis (AHC) and international public health on methodology Regulations, al. Taipei’s Use of a Multi- Case Study Approach - emergency preparedness planning and adequacy of Policies & Channel Mass Risk Survey and response activities needs to data Frameworks in Communication Program to Taiwan be contextualized to existing laws Support of ERC Rapidly Reverse an Epidemic and regulation. This article Efforts of Highly Communicable discusses how Taiwan’s Disease. PLoS ONE. 2009. Communicable Disease Act (2006) 4(11): e7962. allowed Taipei to launch a large- doi:10.1371/journal.pone.00 scale SMS campaign. This act 07962 allowed government officials to

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

override the people’s right to privacy when responding to epidemic disasters. In this case, the Taipei city government held a contract with Taiwan’s six major mobile phone companies, which committed them to six free public service messages (per year) to be sent to their users, if deemed necessary by the proper authorities. The SMS message, in response to the Acute Hemorrhagic Conjunctivitis (AHC) epidemic, was sent to 2.2 million registered mobile phone users in Taipei City. Development of Lam PY. Avian Influenza and Pandemic Influenza Sharing of intelligence and Low Significant Laws, Pandemic Influenza Case Study expertise can be achieved by the concerns on Regulations, Preparedness in Hong Kong. Hong Kong creation of agreements between methodology and Policies & Ann Acad Med Singapore. countries for a better integration adequacy of data Frameworks in 2008. 37:489-9 of ERC in international, cross- Support of ERC country planning and response Efforts efforts as. Lam et al (2008) describe the role of the Cooperation Agreement on Response Mechanism for Public Health Emergencies, signed by the Hong Kong Special Administrative

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

Region Government (HKSARG), the Mainland Ministry of Health and the Macao Health Bureau. Under this Cooperation Agreement, when a cross-boundary public health emergency occurs, the Mainland, Hong Kong and Macao will form a joint emergency response group to facilitate sharing of intelligence and expertise. Development of Cope JR., Frost M., Richun L., General Disasters The creation of amendments to Moderate Minor concerns Laws, Xie R. Assessing Knowledge Multi-province survey of existing laws may facilitate a on methodology Regulations, and Application of public health officials & better integration of ERC in the and coherence Policies & Emergency Risk interviews response. This article recommends Frameworks in Communication Principles China that amendments be made to the Support of ERC Among Public Health People’s Republic of China Efforts Workers in infectious diseases prevention and China. Disaster Med Public control law to authorize provincial Health Prep. 2014 June. 8(3): and local public health agencies 199–205. more freedom to release information. This will allow for improvements in the response time to public health emergencies by allowing agencies the freedom to readily inform their populations with critical health information. The authors also suggest that all

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

public health agencies should be required to develop a risk communication operational plan, which would provide a template from which dedicated risk communicators can work. Development of Ikeda S., Nagasaka T. An General Disasters DRIP is a promising societal Moderate Minor concerns Laws, Emergent Framework of Case Study platform of disaster risk on adequacy of Regulations, Disaster Risk Governance Japan information that works as a data Policies & towards Innovating Coping clearinghouse, collecting and Frameworks in Capability for Reducing disseminating scientific expertise Support of ERC Disaster Risks in Local on risk information from various Efforts Communities. Int. J. Disaster disaster prevention organizations, Risk Sci. 2011. 2(2): 1–9. fire brigades, and research institutions. DRIP also communicates local and experiential information from residents and civic organizations under interoperable environments linked by multilayered networks. Ikeda et al discuss this emergent framework of disaster risk governance, presented as an implementation strategy for integrated risk management that incorporates innovative local coping capabilities that reduce disaster vulnerability. The

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

framework is supported by a societal platform of disaster risk information, called DRIP, which National Research Institute for Earth Science and Disaster Prevention, Japan has developed since 2006 as a tool for promoting improved disaster risk governance. Emergency Risk Lam PY. Avian Influenza and Pandemic Influenza Exercises can be used to test ERC Moderate Minor concerns Communication Pandemic Influenza Case Study functions across countries. Lam et on methodology Training/ Preparedness in Hong Kong. Hong Kong al (2008) discuss how the Hong and adequacy of Exercises to Ann Acad Med Singapore. Kong Special Administrative data enhance ERC 2008. 37:489-9 Region Government (HKSARG) integration took part in the APEC Pandemic Response Exercise 2006, which tested communications channels for ensuring preparedness across Asia-Pacific Economic Cooperation economies in responding to emergencies. Emergency Risk Hanvoravongchai P., Pandemic Influenza Exercises can be used to test ERC Moderate Minor concerns Communication Adisasmito W., Chau PN., Mixed Method -Systematic functions across countries. on adequacy of Training/ Alexandra Conseil A., de Sa Rapid Assessment Hanvoravongchai et al (2010) note data Exercises to J., Krumkamp R., Mounier- analytical framework: that a number of simulation Review of published data enhance ERC Jack S., Phommasack B., exercises have been conducted in and KI interviews integration Putthasri W., Shih CS., Touch Cambodia Indonesia Lao all six countries it investigated. S., Coker R. for the PDR Taiwan Thailand Most of the exercises were table- AsiaFluCap project. top style, where relevant officers

Practice Citation Type of Emergency Synthesized Findings CERQual Explanation of Study Methods Assessment CERQual Country/Geographic of Confidence in Assessment the Context Evidence

Pandemic influenza Vietnam discuss and manage a hypothetical preparedness and health pandemic situation in a round- systems challenges in Asia: table manner. For example, results from rapid analyses Thailand had at least one table-top in 6 Asian countries. BMC exercise at both the central level Public Health. 2010. 10:322 and in each province. Vietnam has conducted many simulations for AHI preparedness at the national, provincial and district levels, as well as at airport and borders. There were also a few regional (multi-country) table-top exercises coordinated by the World Health Organization and one table-top exercise by the Mekong Basin Disease Surveillance Network (MBDS). Only Indonesia and Taiwan had full-scale exercises involving real field activities. Indonesia's full-scale exercise, in Bali during April, 2008, was the first of its kind in the world. Most exercises reveal that management and coordination between various players, including non-health sector players, constitutes a major weakness in preparedness.

Table 13. Synthesis of Findings among English Literature across Methodological Streams Q1

Practice Synthesis of Findings among English Literature across Reference CERQual Explanation of Methodological Streams Assessment CERQual of Confidence in Assessment the Evidence Placement of ERC Functions Renovations of existing organizational structures or Yen (2009) Moderate Evidence from in National Leadership creation of new ones are sometimes needed to achieve Hanvoravongchai (2010) both studies was Structure a better placement of ERC functions in the national individually leadership structure. As reported by Yen et al (2009), evaluated to be of following the SARS outbreak, the Taipei City Government moderate initiated a new public health plan using an integrated confidence level, infection control system against emerging infectious with minor diseases (EID). This new system integrated early concerns on detection of outbreaks (particularly in hospitals and methodology and schools), epidemiological investigation, and adequacy of data epidemiologically based public health prevention and control policies. The renovated Division of Disease Control and Prevention (Taipei’s CDC) also became the core operational unit for implementing crisis management procedures and facilitating policy. These systematic upgrades allowed Taipei’s CDC to quickly implement its Multi-Channel Mass Risk Communication Program during the 2007 outbreak of Acute Hemorrhagic Conjunctivitis. As reported by Hanvoravongchai et al (2010) the integration of national leadership into public health emergency preparedness planning and response activities in response to pandemic influenza can be seen through the participation of presidents or prime ministers on pandemic preparedness committees. In well-established health systems, pandemic preparedness is integrated within existing mechanisms, notably within the national disaster preparedness framework. In countries with a weak healthcare system, vertical programs are

Organizational Proximity of Organizational factors and restrictions need to be Cope (2014) Moderate Evidence from ERC Practitioners to National modified to facilitate timely release of information. Cope Chambers (2012) both studies was Health Leadership Response et al (2014) noted that a prominent issue in the People’s individually Republic of China was the inability of certain evaluated to be of departments to release information, a restriction that moderate was noted across departments and throughout the chain confidence level, of command, from the local level through to the with minor provincial and the national levels. The authors concerns on recommend increasing the freedom each department methodology and has to release relevant information, which has been adequacy of data reasonably verified, as a way to improve the system. The broader issue regarding restrictions on how information can be released directly ties into concerns about the timely release of information. Chambers et al (2012) described how during pandemic influenza in the UK, the only formal communication channel ‘up the chain of command’ was indirect, run through the regional Health Protection Agency (HPA) (Chambers et al 2012). The failure to allow information from the frontline to feed up the channels to national level decision making presented an issue. The UK relied on the Health Protection Agency, a quasi-independent agency, to lead, coordinate and manage the operational response at the local level—a role for which it was ill-equipped, given that its main mission is to provide disease surveillance and epidemiological advice at the national and local levels.

Development of Laws, Laws, regulations and frameworks contribute to define Yen (2009) Moderate Evidence from Regulations, Policies & the context in which ERC functions and strategies are Lam (2008) three studies was

Practice Synthesis of Findings among English Literature across Reference CERQual Explanation of Methodological Streams Assessment CERQual of Confidence in Assessment the Evidence Frameworks in Support of implemented. In some circumstances amendments may Cope (2014) individually ERC Efforts be necessary to facilitate such functions. Yen et al (2009) Ikeda (2011) evaluated to be of report on Taipei’s ability to launch a large-scale SMS moderate campaign as a direct result of Taiwan’s Communicable confidence level, Disease Act (2006). This act allowed government officials with minor to override the people’s right to privacy when concerns on responding to epidemic disasters. In this case, the Taipei methodology (Yen, city government held a contract with Taiwan’s six major Cope), adequacy of mobile phone companies, which committed them to six data (Yen, Ikeda) free public service messages (per year) to be sent to and coherence their users, if deemed necessary by the proper (Cope). authorities. Lam et al (2008) describe an important Evidence from one achievement of the Hong Kong Special Administrative study (Lam) was Region Government (HKSARG) when, in 2005, they evaluated to be of signed a Cooperation Agreement on Response low confidence Mechanism for Public Health Emergencies with the level with Mainland Ministry of Health and Macao Health Bureau. significant Under this Cooperation Agreement, when a cross- concerns on boundary public health emergency occurs, the Mainland, methodology and Hong Kong and Macao forms a joint emergency adequacy of data response group to facilitate sharing of intelligence and expertise. Cope et al (2014) recommend the need for amendments to the infectious diseases prevention and control law of the People’s Republic of China to authorize more freedom for provincial and local public health agencies to release information. This would enhance the ability of local public health agencies to release information and improve risk communications, thereby addressing the barrier in the release of critical health information, reported by many agencies. Ikeda et al (2011) describe a framework of disaster risk governance presented as an implementation strategy for integrated risk management that incorporates

Practice Synthesis of Findings among English Literature across Reference CERQual Explanation of Methodological Streams Assessment CERQual of Confidence in Assessment the Evidence innovative local coping capabilities that reduce disaster vulnerability. The framework is supported by a societal platform of disaster risk information called DRIP, developed in 2006 by the National Research Institute for Earth Science and Disaster Prevention in Japan as a tool for promoting improved disaster risk governance. DRIP is a societal platform of disaster risk information that works as a clearinghouse, collecting and disseminating scientific expertise on risk information from various disaster prevention organizations, fire brigades, and research institutions.

Emergency Risk Exercises and training can be conducted to identify Lam (2008) Moderate Evidence from Communication Training/ barriers and successes in the integration of ERC Hanvoravongchai et al both studies was Exercises to enhance ERC functions into national and international public health (2010) individually integration emergency preparedness planning and response. For evaluated to be of example, after the Hong Kong Special Administrative moderate Region Government (HKSARG) updated their pandemic confidence level, preparedness plan, they took part in the 2006 APEC with minor Pandemic Response Exercise, which tested concerns on communications channels to ensure preparedness methodology and across Asia-Pacific Economic Cooperation economies in adequacy of data response to emergencies (Lam et al 2008). Hanvoravongchai et al (2010) describes a number of simulation exercises, mostly tabletop exercises, where officers discuss and manage a hypothetical pandemic situation in a round-table manner. For example, Thailand had at least one table-top exercise at both the central level and in each province. Vietnam has conducted many simulations for AHI preparedness at national, provincial and district levels, as well as at airports and borders. There were also a few regional (multi-country) table-top exercises coordinated by the World Health Organization,

Practice Synthesis of Findings among English Literature across Reference CERQual Explanation of Methodological Streams Assessment CERQual of Confidence in Assessment the Evidence and one table-top exercise by the Mekong Basin Disease Surveillance Network (MBDS). Only Indonesia and Taiwan performed full-scale exercises involving real field activities. Indonesia's full-scale exercise, in Bali during April, 2008, was the first of its kind in the world. Most exercises reveal that management and coordination between various players, including non-health sector players, constitutes a major weakness in preparedness.

CHINESE LITERATURE A total of 15 articles were identified through the Chinese databases of literature in the Chinese language and were included as directly relevant to question 1. All articles referenced China. Of the 15 articles, 4 were based on Pandemic Influenza, 11 referenced all hazards/ general disasters and, among them, 3 were supported with a case study on an earthquake, a measles vaccination campaign, and a terrorist attack. The following tables (Table 14 and Table 15) describe the study findings within methodological stream and across methods for the Chinese literature.

Table 14. Synthesis of Findings in Chinese within Methodological Stream and Evaluation of Certainty/Confidence – Q1

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence Placement of Chen et al Qualitative-Content Chinese Government Weibo [Twitter] has Moderate Limited details on ERC Functions (2013) Analysis/ been a part of standard emergency response coding schemes and in National Governmental Weibo mechanism. When using governmental analysis Leadership [Twitter] Weibo in conducting ERC, each responding Structure agency should construct microblog messages aligned with their own roles and responsibilities. Such clarity may help the public process information received and avoid confusion. Placement of Chen et al Qualitative-Content When using governmental Weibo [Twitter] in Moderate Limited details on ERC Functions (2013) Analysis/ conducting ERC, governments should assume coding schemes and in National Governmental Weibo the roles of information disseminator, the analysis Leadership [Twitter] leader of conversation, and the monitor of Structure public opinions for feedback and intervention. Placement of Chen et al Qualitative-Content When using governmental Weibo in Moderate Limited details on ERC Functions (2014) Analysis/ conducting ERC, governments should coding schemes and in National Governmental Weibo increase interaction between the analysis Leadership [Twitter] government's Weibo and followers; improve Structure mutual communication; initiate conversations; close the gap between information needs and supply; use it as a public forum. Placement of Chen et al Qualitative-Content Chinese Government Weibo [Twitter] has Moderate Limited details on ERC Functions (2014) Analysis/ been a part of standard emergency response coding schemes and in National Governmental Weibo mechanism. To allow for immediate analysis Leadership [Twitter] response, and ensure information accuracy, It Structure is crucial to develop a database and to enhance capacity to monitor public opinions on governmental Weibo and to Integrate the

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence management of governmental Weibo into routine government Placement of Chen et al Qualitative-Content Chinese Government Weibo [Twitter] has Moderate Limited details on ERC Functions (2014) Analysis/ been a part of standard emergency response coding schemes and in National Governmental Weibo mechanism. It is crucial to offer professional analysis Leadership [Twitter] training and support for communicators in Structure charge of governmental Weibo [Twitter]. To avoid confusion, establishing an emergency response governmental Weibo [Twitter] coordination mechanism among government agencies is critically needed. Placement of Chen et al Qualitative-Content Chinese Government Weibo [Twitter] has Moderate Limited details on ERC Functions (2014) Analysis/ been a part of standard emergency response coding schemes and in National Governmental Weibo mechanism. It is crucial to implement analysis Leadership [Twitter] performance evaluation assessing its Structure effectiveness during normal times and in the preparation for, response to and recovery from public health emergencies Placement of Liu et al Qualitative-Content In control of online rumors spreading on Moderate Limited details on ERC Functions (2013) Analysis/ Weibo, government agencies need to 1. have coding schemes and in National Governmental Weibo robust regulations on rumor control; 2. analysis Leadership [Twitter] Establish an effective monitoring system; 3. Structure Proactively monitor rumors, identify sources of rumors, and intervene when necessary; 4. Communicate early and timely updates; respond to rumors immediately with correct information; build public trust in government Placement of The Quantitative- Usage Government Weibo has been a part of Moderate Limited details on ERC Functions People’s data and Qualitative- standard emergency response mechanism. coding schemes and in National Public Content Analysis/ News release and servicing have become the analysis Leadership Opinion Governmental Weibo development trend of government Structure Monitoring [Twitter] microblogs, 1) More government officials Office and should utilize Weibo as a service and Weibo communication platform. 2) Communication Data and presentation skills are key to successful

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence Center information dissemination and need to follow (2012- the current trend. 3) Adopt a "Government 2016) Service Weibo" + "Collective Management" + "Performance evaluation" three pronged mechanism to optimize the efficiency and effectiveness of government services. 4) Diversify government services offered through Weibo and build the "e-service" database to offer one-stop shop. 5) Expand and improve local government's weibo use to improve reach and impact. 6) Build a positive environment, and monitor/guide public opinions. Placement of Zeng et al Mixed-Methods/ Aided by the new media, government's ERC Moderate Limited details on ERC Functions (2015) Governmental Weibo efforts should 1) evolve away from the coding schemes and in National [Twitter] traditional top-down approach to two-way analysis Leadership exchange of ideas/opinions, and 2) Structure proactively monitor/seek public feedback and make immediate and timely adjustments to ERC strategies Placement of Zhang XE Mixed-Methods/ Weibo enabled timely monitoring of other Moderate Limited details on ERC Functions et al Governmental Weibo outbreak-relevant information, provided coding schemes and in National (2015) [Twitter] access to additional crowd-sourced analysis Leadership epidemiological information and was Structure leveraged by the local government as an interactive platform for risk communication and monitoring public sentiment on the policy response. There is potential for social networking sites to be used by public health agencies to enhance traditional communicable disease surveillance systems for the global surveillance of overseas public health threats. Social networking sites also can be used by governments for calibration of response policies and measures and for ERC.

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence In summary, 1) it may be beneficial for public health agencies to recruit and maintain a workforce of epidemiologists who are multilingual for international disease surveillance in a foreign language. 2) there is significant potential for social media surveillance to be incorporated into mainstream disease surveillance and response systems Placement of Zhou et al Qualitative-Content The Chinese Government Weibo has been an Moderate Limited details on ERC Functions (2015) Analysis/ integral part of the response during recent coding schemes and in National Governmental Weibo national and regional emergencies. It should analysis Leadership [Twitter] play a leadership role in the ERC efforts and Structure ensure: 1) Timely dissemination of the risk information improves public participation, interaction and influence that could guidance public opinion, 2) learn from best practices to improve public impact and ensure openness, transparency, and accountability, and 3) be people-oriented: understand and address the public's information needs, and fulfill social responsibility and guide public opinions Placement of Hao et al Quantitative Survey/ To integrate ERC into China’s national and Moderate Limited details on ERC Functions (2009) Capacity Building international public health emergency survey items, process, in National preparedness planning and response and analysis Leadership activities, China CDC should has its own speak Structure person. Media relations trainings and protocols are needed prior to disasters. Placement of Ma et al Quantitative Survey/ During a disease outbreak, Government must Moderate Limited details on ERC Functions (2010) Capacity Building provide the public timely update on disease survey items, process, in National characteristics and prevention methods, and analysis Leadership access to government guidelines, and Structure channels for further risk information and situational updates. Placement of Shao et al Mixed-Methods/ To integrate ERC into China’s national and Moderate Limited details on

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence ERC Functions (2014) Capacity Building international public health emergency data collection in National preparedness planning and response process, and analysis Leadership activities, China MoH has developed an Structure online training tool. After testing, researchers have identified ways to improve the training system: 1. The online training tool should tailor to users' online habits and combine search engine functionality, information exchange platform, experts’ forum in one site. 2. To encourage participation, the tool should allow for interaction between trainers and trainees and offer education credits. 3. Build case bank. 4. Have a live chat option for help and for immediate guidance from ERC experts. 5. It should be complemented with in-person trainings, which offer interaction and immediate Q&A feedback. 6. Algorithms should be used to monitor and analyze usage data in real-time, thereby optimizing user experience and quality of training. Placement of Song et al Quantitative Survey/ To integrate ERC into national and Moderate Limited details on ERC Functions (2016) Capacity Building international public health emergency survey items, process, in National preparedness planning and response and analysis Leadership activities and enhance national and local ERC Structure capacity, it is crucial to 1. Create a media monitoring position with designated roles and responsibilities at each CDC office; 2. Provide funding support and equipment; 3. Adopt a nation-wide media monitoring system, managed by central CDC; 4. Enhance media monitoring training; 5. the superior office can direct support to local CDC offices that do not have the capacity to conduct their own media monitoring activities Placement of Xie et al Qualitative/ Capacity To enhance the national and local ERC Moderate Limited details on

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence ERC Functions (2011) Building capacity, it is important to clarify roles and data collection in National responsibilities of responding agencies. In process, and analysis Leadership China, generally all agreed that the Chinese Structure MoH, especially its emergency management offices, should take the responsibility of coordination and leading ERC preparedness and response. Placement of Xie et al Qualitative/ Capacity To enhance ERC capacity building for public Moderate Limited details on ERC Functions (2011) Building health workers at all levels of governments, data collection in National appropriate training, exercises, resources and process, and analysis Leadership tools including pre-designed communication Structure supplies (such as posters, brochures, warning signs, etc.), and checklists of resources and lists. Placement of Xie et al Qualitative/ Capacity To enhance ERC capacity building for public Moderate Limited details on ERC Functions (2011) Building health workers at all levels of governments, data collection in National responding agencies should designate a point process, and analysis Leadership of contact, identify communication channels Structure and information sharing mechanism. There is also a need to improve public opinions monitoring capacity and promote better cooperation across agencies. Placement of Zhang et al Qualitative/ Capacity To enhance ERC capacity building for public Moderate Limited details on ERC Functions (2011) Building health workers at all levels of governments, it data collection in National is crucial to develop ERC guidelines for process, and analysis Leadership government spokespersons, medical and Structure public health workers, and relevant administrative agency personnel at the local level and integrate them into national and local emergency preparedness plans. Public education on ERC is also important. Placement of Zhang et al Qualitative/ Capacity To enhance ERC capacity building for public Moderate Limited details on ERC Functions (2011) Building health workers at all levels of governments, it data collection in National is crucial to offer capacity building training to process, and analysis Leadership government spokespersons, medical and

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence Structure public health workers, and and relevant administrative agency personnel at the local level and have them participate in scenario-based tabletop and functional exercises Placement of Jiang et al Quantitative/ The official public health hotline could be a Moderate Limited details on ERC Functions (2012) Official public health trusted information source that acts as a data collection in National hotline [12320] direct two-way communication channel process, and analysis Leadership between the government and the public Structure during a crisis. Communication by the public. It offered health consultation directly to the public, acted as an important emergency risk communication agent and had taken on the role to gather public reactions data. It has been highly valued by the China MoH and helped shape the ERC strategies. Placement of Wang et al Quantitative/ To develop plans and protocols to further Moderate Limited details on ERC Functions (2010) integrate the official public health hotline data collection in National into ERC infrastructure across the country to process, and analysis Leadership monitor public opinions and enhance two- Structure way communicate with the public during a health crisis. Development The Quantitative- Usage The Chinese Government Weibo has been a Moderate Limited details on of Laws, People’s data and Qualitative- part of standard emergency response coding schemes and Regulations, Public Content Analysis mechanism. News release and servicing have analysis Policies & Opinion become the development trend of Frameworks in Monitoring government Weibo, 1) More government Support of ERC Office and officials should utilize Weibo as a service and Efforts Weibo communication platform. 2) Communication Data and presentation skills are key to successful Center information dissemination and need to follow (2012- the current trend. 3) Adopt a "Government 2016) Service Weibo" + "Collective Management" + "Performance evaluation" three-pronged mechanism to optimize the efficiency and

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence effectiveness of government services. 4) Diversify government services offered through Weibo and build the "e-service" database to offer one-stop shop. 5) Expand and improve local government's weibo use to improve reach and impact. 6) Build a positive environment, and monitor/guide public opinions. Development Hao et al Quantitative Survey To integrate ERC into national and Moderate Limited details on of Laws, (2009) international public health emergency survey items, process, Regulations, preparedness planning and response and analysis Policies & activities, public health agencies such as CDC Frameworks in should have its own speakperson and media Support of ERC relations trainings and protocols need to be Efforts developed and provided to speakperson prior to disasters. Development Ma et al Quantitative Survey During a disease outbreak, Government must Moderate Limited details on of Laws, (2010) 1) update the public on disease survey items, process, Regulations, characteristics and prevention methods, 2) and analysis Policies & identify access to government guidelines, and Frameworks in 3) provide channels for further risk Support of ERC information and situational updates. Efforts Development Shao et al Mixed-Methods To integrate ERC into national and Moderate Limited details on of Laws, (2014) international public health emergency data collection Regulations, preparedness planning and response process, and analysis Policies & activities, China MoH has developed an Frameworks in online training tool for ERC personnel across Support of ERC the country. This effort is to overcome the Efforts financial and geographic barriers and disparities, yet a few key elements need to be noted: 1. The online training tool should tailor to users' online habits and combine search engine functionality, information exchange platform, experts’ forum in one

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence site. 2. To encourage participation, the tool should allow for interaction between trainers and trainees and offer education credits. 3. Build case bank. 4. Have a live chat option for help and for immediate guidance from ERC experts. 5. It should be complemented with in-person trainings, which offer interaction and immediate Q&A feedback. 6. Algorithms should be used to monitor and analyze usage data in real-time, thereby optimizing user experience and quality of training. Development Song et al Quantitative Survey To integrate ERC into national and Moderate Limited details on of Laws, (2016) international public health emergency survey items, process, Regulations, preparedness planning and response and analysis Policies & activities and enhance national and local ERC Frameworks in capacity, it will be important to 1. Create a Support of ERC media monitoring position with designated Efforts roles and responsibilities at each CDC office; 2. Provide funding support and equipment; 3. Adopt a nation-wide media monitoring system, managed by central CDC; 4. Enhance media monitoring training; 5. the superior office can direct support to local CDC offices that do not have the capacity to conduct their own media monitoring activities Development Xie et al Qualitative To enhance the national and local ERC Moderate Limited details on of Laws, (2011) capacity, it is important to clarify roles and data collection Regulations, responsibilities of responding agencies. In process, and analysis Policies & China, generally all agreed that the Chinese Frameworks in MoH, especially its emergency management Support of ERC offices, should take the responsibility of Efforts coordination and leading ERC preparedness and response. Development Xie et al Qualitative To enhance ERC capacity building for public Moderate Limited details on of Laws, (2011) health workers at all levels of governments, data collection

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence Regulations, appropriate training, exercises, resources and process, and analysis Policies & tools including pre-designed communication Frameworks in supplies (such as posters, brochures, warning Support of ERC signs, etc.), and checklists. Efforts Development Xie et al Qualitative To enhance ERC capacity building for public Moderate Limited details on of Laws, (2011) health workers at all levels of governments, data collection Regulations, responding agencies should designate a point process, and analysis Policies & of contact, identify communication channels Frameworks in and information sharing mechanism. There is Support of ERC also a need to improve public opinions Efforts monitoring capacity and promote better cooperation across agencies. Development Zhang et al Qualitative It is crucial to develop ERC guidelines for Moderate Limited details on of Laws, (2011) government spokespersons, medical and data collection Regulations, public health workers, and relevant process, and analysis Policies & administrative agency personnel at the local Frameworks in level and integrate them into national and Support of ERC local emergency preparedness plans. Efforts Population-based ERC education is also important. Development Zhang et al Qualitative It is crucial to develop and offer capacity Moderate Limited details on of Laws, (2011) building training to government data collection Regulations, spokespersons, medical and public health process, and analysis Policies & workers, and Frameworks in and relevant administrative agency personnel Support of ERC at the local level and have them participate in Efforts scenario-based tabletop and functional exercises Emergency Risk The Quantitative- Usage The Chinese Government Weibo has been a Moderate Limited details on Communication People’s data and Qualitative- part of China’s standard emergency response coding schemes and Training/ Public Content Analysis mechanism. News release and servicing have analysis Exercises to Opinion become the development trend of enhance ERC Monitoring government microblogs, 1) More government integration Office and officials should utilize Weibo as a service and

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence Weibo communication platform. 2) Communication Data and presentation skills are key to successful Center information dissemination and need to follow (2012- the current trend. 3) Adopt a "Government 2016) Service Weibo" + "Collective Management" + "Performance evaluation" three pronged mechanism to optimize the efficiency and effectiveness of government services. 4) Diversify government services offered through Weibo and build the "e-service" database to offer one-stop shop. 5) Expand and improve local government's weibo use to improve reach and impact. 6) Build a positive environment, and monitor/guide public opinions. Emergency Risk Hao et al Quantitative Survey A few steps have been identified to enhance Moderate Limited details on Communication (2009); Ma ERC training as part of the national ERC survey items, process, Training/ et al capacity building efforts: 1) provide ERC and analysis Exercises to (2010); training (including media relations trainings) enhance ERC Song et al to government spokespersons, medical and integration (2016); Xie public health workers, and et al and relevant administrative agency personnel (2011); at the national and local levels, 2) develop Zhang et al protocols for responding agencies, 3) develop (2011) appropriate training and exercises and have relevant personal and agencies participate in scenario-based tabletop and functional exercises, 4) develop ERC resources and tools including pre-designed communication supplies (such as posters, brochures, warning signs, etc.), and checklists, Emergency Risk Shao et al Mixed-Methods To integrated ERC into national and Moderate Limited details on Communication (2014) international public health emergency data collection Training/ preparedness planning and response process, and analysis Exercises to activities, China MoH has developed an

Practice Citation Study Methods/Study Synthesized finding statement CERQual Assessment Explanation of Topic of Confidence in the CERQual Assessment Evidence enhance ERC online training tool for ERC personnel across integration the country. This effort is to overcome the financial and geographic barriers and disparities, yet a few key elements need to be noted: 1. The online training tool should tailor to users' online habits and combine search engine functionality, information exchange platform, experts’ forum in one site. 2. To encourage participation, the tool should allow for interaction between trainers and trainees and offer education credits. 3. Build case bank. 4. Have a live chat option for help and for immediate guidance from ERC experts. 5. It should be complemented with in-person trainings, which offer interaction and immediate Q&A feedback. 6. Algorithms should be used to monitor and analyze usage data in real-time, thereby optimizing user experience and quality of training.

Table 15. Synthesis of Findings among Chinese Literature across Methodological Streams

Practice Synthesis of Findings among Chinese Literature across Reference CERQual Explanation of Methodological Streams Assessment CERQual of Confidence in Assessment the Evidence Placement of ERC These studies discuss how most Chinese government agencies Chen et al, 2013; Moderate Limited details on Functions in National use the microblog “Weibo,” Chinese twitter, for public Chen et al, 2014; survey items, Leadership Structure communication during normal times and public health Liu et al., 2013; process, and emergencies such as H1N1, H7N9, terrorist attacks, and the The People’s Public analysis recent Ya’An earthquake, with the goals of achieving timely, open, two-way communications, monitoring public opinion, Opinion controlling rumors, addressing public concern, and improving Monitoring Office government services. To effectively control online rumors, swift and Weibo Data response is needed by releasing official counter-messages and Center, 2012-2016; by working with the police to identify and penalize the sources Zeng et al, 2015; of rumors, and by assessing the effectiveness of these Zhang X et al, countermeasures by continuing monitoring Weibo. There might 2015; Zhang XE et be potential for social media surveillance to be incorporated into mainstream disease surveillance and response systems. al, 2015; Zhou et There has been an ongoing national effort to expand the use of al, 2015 government Weibo across the country and to establish a standardized operation, training and evaluation mechanism for Chinese government’s Weibo use and service performance. Placement of ERC These studies examine collaboration between the US CDC and Hao et al, 2009; Moderate Limited details on Functions in National China’s CDC to enhance the China’s efforts towards building Ma, 2010; Shao et survey items, Leadership Structure ERC capacity and integrating ERC into public health emergency al, 2014; Song et process, and preparedness activities. China’s CDC has conducted al, 2016; Xie et al, analysis assessments of ERC capacity and needs at local public health agencies, and identified barriers and possible solutions. 2011; Zhang et al, Training materials have been created and tested among local 2011 CDC staff. Additionally, the application of ERC was tested during the A (H1N1) Pandemic in China. Placement of ERC These studies look at 12320, the only official health hotline in Jiang et al, 2012; Moderate Limited details on Functions in National China set up by China’s MoH. 12320 has been integrated into Wang et al, 2010; survey items, Leadership Structure the emergency response system and tested as an important Zhang X et al, 2015 process, and channel of communication during the 2008 formula analysis contamination crisis, the 2008 Beijing Olympics, the 2009 A(H1N1) pandemic, and the recent measles campaign. As a direct two-way communication channel between the government and the public, 12320 was considered a trusted channel of communication by the public. It offered health consultation directly to the public, acted as an important emergency risk communications agent and had taken on the role of gathering public reaction data. It has been highly valued by China’s MoH and helped shape the ERC strategies. Due to its ease of access and two-way communication, 12320 played an integral role in ERC and public opinions monitoring during the A (H1N1) pandemic and the recent measles vaccination campaigns. Since then, efforts are being made to integrate 12320 into the

Practice Synthesis of Findings among Chinese Literature across Reference CERQual Explanation of Methodological Streams Assessment CERQual of Confidence in Assessment the Evidence existing public health response system and to develop a protocol to monitor future public reactions/opinions during routine public health activities and public health emergencies. 12320 provides services including infectious disease prevention and control, health care consultations and public health legal consultations in China’s provinces, municipalities and autonomous regions. Development of laws, Since 2012, People's Public Opinion Monitoring Office and The People’s Public Moderate Limited details on regulations, policies & Weibo Data Center have routinely published performance Opinion survey items, frameworks in evaluation and best practices on official microblogs (Weibo) of Monitoring Office process, and support of ERC efforts all levels of the Chinese government. Guidelines and and Weibo Data analysis regulations on government microblogging have been developed by central and local governments and service Center, 2012-2016 providers (e.g. 政务微博运营规范手册 [Government Microblogging Operation Guidelines] by Sina Corporation, and “Autonomous Region Government Guidelines on Microblog Use and Management” by XinJing.) Reports on government microblog use and case studies on lessons learned and best practices have been developed and are disseminated on a biannual basis. Development of laws, These studies discussed how, in collaboration with the US CDC, Hao et al, 2009; Moderate Limited details on regulations, policies & China’s CDC has been leading the country’s effort in building Ma et al, 2010; survey items, frameworks in ERC capacity and integrating ERC into public health emergency Shao et al, 2014; process, and support of ERC efforts preparedness activities. China’s CDC has conducted Song et al, 2016; analysis assessments of ERC capacity and needs at local public health Xie et al, 2011; agencies and identified barriers and possible solutions. It was Zhang et al, 2011 found that there was a demonstrated need to develop an official ERC guideline by conducting needs assessments and reviews of the laws and regulation that guide the work of ERC (Hao, 2009, Ma, 2010; Zhang, 2011). Chinese public health professionals have been developing and revising the ERC guidelines and integrating them into current emergency preparedness plans. Emergency Risk Partnered with microblog service providers such as Tencent The People’s Public Moderate Limited details on Communication and Sina Corporations, China’s People’s Public Opinion Opinion survey items, Training/ Exercises to Monitoring Office and Weibo Data Center routinely examine Monitoring Office process, and enhance ERC and publish case studies and best practices for Government and Weibo Data analysis integration Weibo use. Handbooks on critical topics such as monitoring Center, 2012-2016. public opinions and performance evaluation are also created and publicly accessible. Emergency Risk These studies discussed how China’s CDC has developed, Hao et al, 2009; Moderate Limited details on Communication assessed and verified the content of a risk communications Ma et al, 2010; survey items, Training/ Exercises to preparedness framework, laid out in the Public Health Song et al, 2016; process, and enhance ERC Emergency and Risk Communication Handbook, to enhance Shao et al, 2014; analysis integration ERC capacity and integrate ERC in the nation’s PHEP activities. Xie et al, 2011; China’s CDC is creating and refining ERC capacity building

Practice Synthesis of Findings among Chinese Literature across Reference CERQual Explanation of Methodological Streams Assessment CERQual of Confidence in Assessment the Evidence trainings (in person and/or online) offered to government Zhang et al, 2011 workers, responders and the public, and having them participate in tabletop and functional exercises. However, there remains a great need to improve media monitoring training, to enhance ERC capacity building for public health workers at all levels of governments by designing and delivering appropriate trainings, exercises, and developing tools, and to provide ERC training materials and pre-designed communication supplies (posters, brochures, warning signs, etc…), as well as checklists of ERC resources and tools.

Question 2 & Question 6

Question 2: What are the best mechanism(s) to establish effective intra-agency, interagency, and/or cross-jurisdictional (such as cross-border; national with sub-national jurisdictions, etc.) information sharing for emergency risk communication?

Question 6: What are the best practices and protocols to ensure coordination of risk communication activities between responding agencies across organizations and levels of response?

COUNTRY DISTRIBUTION GRAPH The literature from the English, Chinese, and Portuguese languages have been integrated and are presented together. Figure 8 below shows the distribution of countries represented by the literature relevant to Q2 and Q6, most studies are based in the US and the Netherlands.

Figure 8. Distribution of countries represented by the articles included in the evidence synthesis for Question 2 and 6 combined.*

8 7

7 6 5 4 3 3 3 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 NumberofArticles 0

*Note: These columns are not mutually exclusive. One article could list one or more countries.

DISASTER TYPE DISTRIBUTION GRAPH Figure 9 below describes the distribution of articles based on the type of emergency, the majority of the articles refers to general disasters/preparedness efforts, followed by floods and tornadoes.

Figure 9. Distribution of disaster types included in the evidence synthesis for Question 2 and 6 combined.*

8 7 7 6 5 4 4 3 3 2 2 2 2 2 1 1 1 1

NumberofArticles 0

*Note: These columns are not mutually exclusive. One article could list one or more disaster types.

EVIDENCE SYNTHESIS Twenty four articles describe the existence of different types of mechanisms to support information sharing and coordination between responding agencies and stakeholders. There was substantial overlap and integration of the themes identified for information sharing (question 2) and coordination of communication between responding agencies (question 6), so it was determined that the evidence syntheses for these two questions was best presented together. The following are three themes that emerged from the literature as mechanism to enhance information sharing and coordination: 1) Networks and task forces/committees, 2) Information Systems (tools and platforms), and 3) Mechanisms to facilitate local stakeholders’ engagement in risk communications.

For the evidence synthesis for Q2/Q6, the findings from the English, Chinese, and Portuguese literature have been integrated and are presented below. The following tables (Table 16 and Table 17) describe the study findings within methodological stream and across methods for the English and Chinese literature.

Figure 10. Major themes/outcomes emerged from the literature for effective information sharing and coordination between stakeholders.

Networks/Task forces/Committees

Sub-theme: Elements of functionality of networks and task-forces

Information Systems

Local Stakeholders Engagement

Table 16: Syntheses of Findings within Methodological Stream and Evaluation of Certainty/Confidence – Q2 and Q6

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

Networks to Pandemic Influenza Regional disease surveillance networks may Gresham L., Ramlawi A, Moderate Minor concerns enhance information Case study provide a useful mechanism for information Briski J, Richardson M, on adequacy of sharing Middle East sharing as described by Gresham et al (2009) Taylor T. data the Middle East Consortium on Infectious Trust across borders: Disease Surveillance (MECIDS), a regional responding to 2009 H1N1 disease surveillance network of public health influenza in the Middle experts and ministry of health officials from East. Biosecurity and Israel, the Palestinian Authority, and Jordan. Bioterrorism: Biodefense MECIDS unites public health officials of Strategy, Practice, and differing Middle Eastern nationalities and Science. Volume 7, contributes to regional health and stability by Number 4, 2009. engaging in regular cross-border information exchange, conducting regular executive board meetings, performing laboratory and risk communications training, and implementing innovative communication technology. Initially focused on food- and waterborne diseases, the partnership developed a network of laboratories, protocols for specimen collection and diagnosis of diarrheal diseases, and data sharing and notification capabilities, so as to analyze and share information on disease threats. During H1N1, the MECIDS partners agreed to prompt and coordinated border and airport screening, laboratory testing, information

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

exchange, and common communication strategies. This coordination can be largely contributed to the existence of both trust and well-exercised national and regional pandemic preparedness plans, which were initially established within this network. Networks to Pandemic Influenza Regional disease surveillance networks may Moore M, Dausey D. Moderate Minor concerns enhance information Survey provide a useful mechanism for information Response to the 2009- on methodology sharing Asia sharing. As described by Moore and Dausey H1N1 influenza pandemic and adequacy of (2009) the Mekong Basin Disease Surveillance in the Mekong Basin: data (MBDS) is a cooperating network, comprised surveys of country health of six countries - Cambodia, China (originally leaders. BMC Research just Yunnan province and, since 2008, Notes. 2011, 4:361. Guangxi Province as well), Lao People’s Democratic Republic, Myanmar, Thailand and Vietnam - which formally organized themselves in 2001 to collaborate on sub- regional infectious disease surveillance and control. During the H1N1 pandemic, MBDS health leaders perceived their pandemic responses to be effective in areas that, prior to the creation of the network, were considered problematic. Most felt that MBDS cooperation helped drive such efforts, and thus added value. Surveillance capacity within countries and surveillance information sharing across countries, a longstanding MBDS focus areas, were cited as strengths.

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

Participants noted the ability of their country level surveillance systems to exchange information efficiently within the country and the importance of MBDS for enabling timely coordinated regional response to detect disease at cross border sites and prevent the spread of the virus across countries. Task Forces and Anthrax Task forces can facilitate interaction between Chess C., Clarke L. Low Significant Committees to Interviews professionals belonging to different Facilitation of Risk concerns on facilitate USA organizations as described by Chess et al Communication During the methodology and information sharing (2007), in this case a bioterrorism task force Anthrax Attacks of 2001: adequacy of data was established in a county in New Jersey The Organizational before the anthrax attacks, and how it served Backstory. Weapons of as a mechanism to create a network of Mass Destruction. people who trusted each other during the American Journal of Public Anthrax incidents. Successful risk Health. 2007; 97:1578- communication in a health crisis is often 1583. contingent upon effective communication among agencies, and organizational and professional networks were essential to risk communications efforts. Frequent interaction among a diverse group of professionals and, in particular, between law enforcement and public health officials, facilitated the creation of consistent messages. Where law enforcement and health agencies had preexisting, cooperative relationships, there was often a commitment to compromise,

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

accommodating differing goals and methods. Task Forces and Anthrax The creation of an Emergency Operations Clarke C. and Chess C. Moderate Minor concerns Committees to Interviews Committee (EOC) can facilitate the sharing of False Alarms, Real on methodology facilitate USA information. Clarke et al (2006) describes Challenges – One and adequacy of information sharing how, during the 2001 Anthrax attacks, a University’s data university in the US organized an Emergency Communication Response Operations Committee (EOC) designed to to the 2001 Anthrax Crisis. bring together groups such as Police, EHS, Biosecurity and Occupational Health, and other departments, Bioterrorism: Biodefense to share information, pool resources, and Strategy, Practice and coordinate communication with the Science. Volume 4, university community. Collaboration involved Number 1, 2006. the pooling of resources, sharing of information, and integration of communication infrastructure to form a more organized approach to risk communications. Subtheme: Elements Wildfires Nowell et al. (2015) underlines the Nowell B, Steelman T. Low Significant of functionality of Case Study importance of interpersonal relationships, Communication under concerns on networks and t ask- USA through the use of dyadic network analysis, in Fire: The Role of methodology and forces a US based study of institutional and Embeddedness in the adequacy of data relational embeddedness in relation to the Emergence and Efficacy of response to wildfires. Responders are Disaster Response significantly more likely to interact more Communication Networks. frequently with people with whom they have Journal of Public existing relationships (familiarity/relational Administration Research embeddedness), even after controlling for and Theory. similarities in role function and stakeholder 2015;25(3):929-52. group. This suggests that efforts by

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

emergency managers and other key stakeholders to establish working relationships among responders before an incident will increase the likelihood that those responders will stay in communication with one another during an actual incident.

Subtheme: Elements Terrorism In simulated environments, network teams Schraagen JM, Veld MHit, Moderate Minor concerns of functionality of Controlled work faster and arrive at more correct De Koning L. Information on methodology networks and task- laboratory decisions than hierarchical teams as Sharing During Crisis forces experiment described by Schraagen et al (2010). Network Management in comparing team structures allow teams to exchange Hierarchical vs. Network structures information quickly, monitor each other’s Teams. Journal of Netherlands performance, and build mutual trust. Contingencies and Crisis Network teams were found to perform faster Management. than hierarchical teams, while maintaining 2010;18(2):117-27. the same level of accuracy in relatively simple environments. In relatively complex environments, network teams arrive at correct decisions more frequently than hierarchical teams.

Subtheme: Elements General Disasters The use of information manager can facilitate Bharosa N, Lee J, Janssen Moderate Minor concerns of functionality of Case study cross-agencies information sharing as M. Challenges and on methodology networks and task- Netherlands suggested by Bharosa et al (2010) in a case obstacles in sharing and and adequacy of forces study on a multi-agency disaster coordinating information data management exercise in the Port of during multi-agency Rotterdam (POR), the Netherlands in order to disaster response:

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

collect data on disaster management and Propositions from field information sharing. Unlike its American exercises. Information counterpart, the Dutch Emergency Response Systems Frontiers. Center (ERC), acts as an information center 2010;12(1):49-65. for just one relief agency. However, the presence of a central hub can act as a single point of failure, as it possesses limited knowledge about ongoing emergency response activities, resulting in uncontrolled information filtering/dissemination and a lack of direct information exchange between some information users and information sources. Managing some of the information flows between the decision-making units (COPI, ROT) and the ERC is usually the job of an Information Manager (IM), a role usually filled by a police officer, who is responsible for providing specific information services to decision-making units. Subtheme: Elements 2009 H1N1 Public Information Officers (PIOs) can Howard AF, Bush HM, Low Significant of functionality of outbreak facilitate effective information dissemination. Shapiro RM, 2nd, concerns on networks and task- Survey Howard et al (2012) describe the importance Dearinger A. adequacy of data forces USA of having a Public Information Officer (PIO) at Characteristics of Kentucky the local public health department level to local health departments facilitate effective information dissemination that influence public to health care professionals. In this analysis, health communication the absence of a PIO was associated with during times of crisis: decreased communication with those information dissemination

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

agencies external to the LHD. associated with H1N1 novel influenza. J Public Health Manag Pract. 2012;18(2):169-74. Epub 2012/01/31. Subtheme: Elements Emerging infectious Early engagement of media can enhance Holmes BJ, Henrich N, Moderate Minor concerns of functionality of diseases information dissemination. Holmes et al Hancock S, Lestou V. on methodology networks and task- Interviews (2009) presents the findings of interviews Communicating with the and adequacy of forces Canada conducted with communications directors public during health crises: data and public health officials in British Columbia Experts' experiences and on components to effective communication. opinions. Journal of Risk They describe the importance of engaging Research. 2009;12(6):793- media representatives immediately in 807. discussions about potential emerging infectious disease outbreaks, including the role media should play and how the public health community can help them fulfill that role.

Use of information Earthquake The use of information systems impacts Celik S., Crbacioglu S. Role Low Significant systems to enhance Interviews and daily timeliness in the response to earthquakes. of information in collective concerns on ERC analysis of reports Celik et al (2010) describe the response to action in dynamic disaster adequacy of data Turkey two Earthquakes in Turkey. Turkish disaster environments. Disasters experts claimed that the presence of 2010, 34(1):137-154. information systems provided timely and accurate information during the Duzce earthquake operations, whereas a lack of information systems created problems in

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

obtaining timely and accurate information for coordinated action during the Marmara earthquake operations. Consequently, the Turkish disaster management system was better prepared three months later, when the Duzce earthquake hit, and performed better in response operations with reduced difficulty in accessing and exchanging information in an accurate and timely manner, so as to facilitate coordination and search-and-rescue activities. Improvements to the information infrastructure, investments in information technology, such as a more effective use of wireless communications, and changes in behavior through individual and organizational learning increased the capacity for information searching, acquisition, and exchange. A better understanding of the importance of disaster-relevant information, along with experience of the Marmara earthquake, helped facilitate more effective use of available communication means for information sharing in response to the Duzce earthquake and assisted in achieving a faster, more coordinated response operation. Use of information All hazards Information systems need to be tailored to Seyedin SH., Jamil HR. Low Significant systems to enhance Survey and the needs of users which may differentiate Health Information and concerns on ERC interviews across roles and responsibilities in disaster Communication System for adequacy of data

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

Iran planning and response. In this article, Emergency Management recommendations are provided for the in a Developing Country, creation of an information system based on Iran. J Med Syst. 2011; the experience of interviewees. Interviewees 35:591-597. reported the need for the creation of five different information databases for emergency management (EM). These databases are: demographics; disasters; senior and line managers; hospitals and therapeutic facilities, laboratories and emergency posts, and health facilities, such as health centers. The authors noted that managers at local level had a different understanding of an EM information system compared to managers at the national level. While participants at the regional and national levels mainly saw an EM database from a preparedness perspective, the majority of managers at the local level focused on the response phase and requirements that an effective information system needs to meet at this phase, such as logging, informing and information broadcasting in response to disasters. An important issue in the EM information system is analysis of available data in order to produce sensible information for decision making and management purposes.

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

Using a website to broadcast the information of casualties, such as their photos, place of admission, and regularly updating such websites psychologically relieves and reduces the stress of families and relatives, as well as reducing unnecessary transportation and pressure on communication infrastructures. Use of information All hazards The creation of information systems for Ipe M., Raghu TS., Vinze A. Moderate Minor concerns systems to enhance Interviews surveillance and intelligence may facilitate Information intermediaries on methodology ERC USA information sharing. Ipe et al (2010) for emergency document a new public health surveillance preparedness and system, the Medical Electronic Disease response: A case study Surveillance and Intelligence System (MEDSIS) from public health. Inf Syst in Arizona (USA). MEDSIS os a web-based Front.2010; 12:67-79. application designed to enable public health agencies to detect and respond to bioterrorism events, outbreaks of infectious diseases, and other public health emergencies, and places the state agency in the role of an information intermediary. Use of information Flooding Information systems used by bordering Bharosa, N. A Case Study Moderate Minor concerns systems to enhance Case Study - countries facilitate information sharing. of Information Flows in on methodology ERC interviews, Bharosa et al. describe how, in order to Multi-Agency Emergency and adequacy of observations and improve collaboration and information Response Exercises. 2009 data document/archival sharing between relief agencies at the Dutch- January. The proceedings analysis German border, the Province of Gelderland of the 10th International Netherlands/Germa (Netherlands) and the Province of Nordrhein- Digital Government ny Westfalen (Germany) have joined forces and Research Conference.

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

organized annual cross-border multi-agency exercises on flood management since 2004. The collaboration program is labeled as VIKING, an acronym for improving information sharing in times of flood management at the Dutch-German border. During the exercise, both the Dutch agencies and the German agencies use their own proprietary information and communication systems, except for one common system: a flood management system called FLIWAS, standing for Flood Information and Warning System. FLIWAS is a high-level water control application, built as a Dutch-German cooperation, and is funded by the EU. The purpose of this application is to optimize the exchange of information during high water situations within and between water management and calamity management organizations. This application is built using generic components (e.g. communication with databases, internal and external communication, import provider, plan module and web mapping). FLIWAS plays a central role in information sharing across the Dutch-German border. Use of information Volcano eruption Common information sources across Leonard GS., Stewart C., Moderate Minor concerns systems to enhance Case study disciplines acoompained by pre-existing Wilson TM., Procter JN., on methodology

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

ERC New Zealand relationships facilutate the flow of Scott NJ., Keys HJ., Jolly information during an emergency. Leonard et GE., Wardman JB., Cronin al describe response to the 2012 Togariro SJ., McBride SK. eruption in New Zealand and describe the Integrating main science communication mechanims multidisciplinary science, used, including the communication modelling and impact data framework and the inter-agency relationships into evolving, syn-event in which the crisis hazard maps were being volcanic hazard mapping developed. The authors analyzed and communication: A communications amongst scientists (through case study from 2012 the use of a wiki technical clearinghouse), Tongariro eruption crisis, communications with emergency managers, New Zealand. Journal of communicatiosn with human health and Volcanology and agricultural agencies, communications with Geothermal Research. critical infrastructure agencies and 2014. (286): 208-232. communications with media and the public. This crisis demonstrates the value of preexisting relationships, having ready-to-go information resources, and the substantial time and resources required for communications.

Use of information Terrorism Information systems shared across agencies Kapucu N. Interagency Moderate Minor concerns systems to enhance Content Analysis & enhance response capabilities. Kapucu et al Communication Networks on methodology ERC Interviews (2006) describe barriers to effective decision During Emergencies and adequacy of USA making in response to the WTC terrorist Boundary Spanners in data attacks in New York City, and the importance Multiagency Coordination. of a multi-organizational communication and American Review of Public

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

coordination response from emergency Administration. 2006. 36 management. The New York Police (2):207-225. Department (NYPD)’s 9-1-1 operators and the Fire Department New York (FDNY) dispatch were not adequately integrated into the emergency response. Inadequate communication also hindered the Port Authority’s response. The lack of both communication and an integrated system hindered agencies’ communication and decision making. In future disasters, it is critical to analyze how responding agencies, victims, and the public will get information and help. Use of information All hazards Militello et al. discusses the challenges to Militello LG, Patterson ES, Moderate Minor concerns systems to enhance Case study coordination in the emergency operations Bowman L, Wears R. on methodology ERC USA centers (EOC). Through two county-level Information flow during exercises, it was observed that there exists an crisis management: asymmetry with regards to experience and challenges to coordination knowledge of responders across agencies, in the emergency which in turn creates coordination challenge operations center. as issues of trust and cohesion become Cognition, Technology & increasingly important during high-intensity Work. 2007;9(1):25-31. disasters. They suggested that low-cost paper-based tools like notebooks, whiteboards, and telephone books would be extremely useful tools of communication in such EOCs.

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

Use of information All To improve the disaster notification system in Chang, 2013. 台湾重大灾 Moderate Limited details on systems to enhance Hazards/Typhoon Taiwan, it is crucial to diversify the use of 害灾情查通报系统运作之 survey items and ERC Survey communication channels and devices. 经验分析 [Operations of process Taiwan the Emergency Information Management System in Taiwan]. 复旦公共行政评论 [Fudan Public Administration Review]. 2013 (2), P. 147-168. Use of information All To improve the disaster notification system in Chang, 2013. 台湾重大灾 Moderate Limited details on systems to enhance Hazards/Typhoon Taiwan, it is crucial to seek and designate 害灾情查通报系统运作之 survey items and ERC Survey local governmental resources to fund the use 经验分析 [Operations of process Taiwan of communication devices in the less the Emergency developed areas in order to address the Information Management equality gap among regions. System in Taiwan]. 复旦公共行政评论 [Fudan Public Administration Review]. 2013 (2), P. 147-168.) Use of information All To improve the disaster notification system in Chang, 2013. 台湾重大灾 Moderate Limited details on systems to enhance Hazards/Typhoon Taiwan, Chang et al found that it would be 害灾情查通报系统运作之 survey items and ERC Survey important to establish partnership between 经验分析 [Operations of process Taiwan government agencies and NGOs and the Emergency integrate the disaster information Information Management management system (such as Tsu Chi’s DRM System in Taiwan]. 复旦公 and Red Cross’ DMIS) to gather public 共行政评论 [Fudan Public opinions, coordinate and share information; Administration Review]. 2013 (2), P. 147-168.

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

Use of information All In Taiwan, the disaster situational Chang, 2013. 台湾重大灾 Moderate Limited details on systems to enhance Hazards/Typhoon information gathering and dissemination is 害灾情查通报系统运作之 survey items and ERC Survey performed by the Fire Department, the Police 经验分析 [Operations of process Taiwan Department, and the Department of Civil the Emergency Affairs, as well as being supplemented by an Information Management emergency management information system. System in Taiwan]. 复旦公 These official systems carry out cross- 共行政评论 [Fudan Public institution information sharing. To improve Administration Review]. the disaster notification system in Taiwan, it 2013 (2), P. 147-168. is crucial to identify the agency that has a better coverage, in this case, the police system, and use it to direct crisis information sharing Use of information All Taiwan has developed and implemented a Chang, 2013. 台湾重大灾 Moderate Limited details on systems to enhance Hazards/Typhoon new "Emergency Management Information 害灾情查通报系统运作之 survey items and ERC Survey System"(EMIS) for disaster information 经验分析 [Operations of process Taiwan sharing, management, and coordination the Emergency across agencies. Stakeholders suggested to Information Management improve EMIS’ capacity to be able to monitor System in Taiwan]. 复旦公 and integrate disaster information available 共行政评论 [Fudan Public on social media, offer timely update on Administration Review]. landslides, road interruption, casualty and 2013 (2), P. 147-168. share information on volunteers, relief assistance and resource available. Local stakeholders’ Natural disaster The use of information and communication Thiago Reginaldo T. Moderate Minor concerns engagement Case study technologies in the public sector is perceived Pacheco DC, Baldessar MJ, on methodology Brazil by the population as a valuable means for Benciveni Franzoni AM, offering high quality services, reliable 2013. Redução de riscos de

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

information, and for gaining insight into desastres na prática: a mechanisms of governance. participação da Defesa Thiago et al. (2013) describe the use of social Civil e do município em networks, like Twitter and Facebook (FB), by plataformas digitais de the Civil Defense (CD) of New Freiburg, Rio de redes sociais Democracia Janeiro, Brazil, as a communication channel digital e governo for alerting and preparing the population to eletronico, Capa > v. 2, n. 9 face a natural disaster. This study aimed to (2013) explore the possible benefits of expanding digital platforms, including web pages, social networks, mobile phone SMS, research engines, video games and digital TV programs, towards a larger community audience, in order to spread messages about the correct behaviors in facing different phases of disasters. Through the analysis of Twitter and Facebook use by CD to communicate with affected population during a specific event, like the flood occurred in 2011 in New Freiburg. This study examined the period from 2011- 13, so as to track digital activity since the creation of specific CD pages in social networks. Content analysis of messages provided positive feedback on enrollment and mobilization of authorities after the event. Following the flood, during the reconstructive

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

phase, the CD and municipality of New Freiburg launched and promoted several public events through social networks, in order to gather, inform, prepare and reassure both the population already affected by the flood and those still possibly at risk. The authors conclude that in a country like Brazil, where network coverage is broad and accessible, it is worth considering the potential for social networks to act as a new institutional channel of communication for health related topics. Local stakeholders’ Flooding In this study, the authors use the results of Ardalan A., Naieni KH., Moderate Minor concerns engagement Interviews interviews, conducted with experts and Kabir MJ., Zanganeh AM., on adequacy of Iran citizens, to evaluate the response to flash Keshtkar AA., Honavar data flood early warning systems in Iran.to the MR., Khodaie H.,Osooli M. authors propose the creation of a “Village Evaluation of Golestan Disaster Taskforce” (VDT), in which individual Province’s Early Warning villages act as operational units in the early System for flash floods, warning mechanism. This model would Iran, 2006-7. Int J overcome the limitations of the current Early Biometerol 2009. 53:247- Warning System by facilitating the warning 254. dissemination and communication at the grassroots level. It would not replace a high- tech early warning system, but would strengthen the existing system at the local level.

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

Local stakeholders’ All hazards In this study, the authors note that the Cole JM., Murphy BL. Rural Moderate Minor concerns engagement Interviews present literature supports the belief that hazard risk communication on adequacy of Canada disaster and emergency management (DEM) and public education: data public education should be framed for Strategic and tactical best specific target audiences. To accomplish this practices. International goal with limited resources, the authors Journal of Disaster Risk propose small rural municipalities could Reduction. 2014 (10):292- consider creating partnerships with 304 neighboring small communities, or use regional organizations, to complete these tasks. In order to deal with the problems associated with role sharing and the lack of knowledge regarding local risk perceptions and appropriate disaster risk reduction (DRR) activities, Community Emergency Management Coordinators should consider tapping into existing social networks and identifying community leaders and local organizations who can assist with the development and delivery of DEM public education programs. Local stakeholders’ Flooding Informational material that addresses the Shepherd J., Vuuren Kv., Moderate Minor concerns engagement Australia need of culturally and linguistically diverse The Brisbane flood: CALD on adequacy of Interviews/case populations should be included in centralized gatekeepers’ risk data study information sources. This study discusses communication role. how, during the 2011 Brisbane flood in Disaster Prevention and Australia, some Culturally And Linguistically Management.

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

Diverse (CALD) “gatekeepers” working at the 2014;23(4):469-483. community level encountered problems accessing emergency risk information that matched their communities’ needs, despite having access to multiple sources of information. There is a need for a well- promoted and central information source that can be immediately accessed to ensure that emergency risk information can be more effectively communicated to CALD communities. Local stakeholders’ Volcanic eruptions Empowerement of local communities in ERC Gultom DI., Joyce Z. Crisis Moderate Minor concerns engagement Interviews may be an effective strategy to enhance ERC. communication capacity on adequacy of Indonesia This study describes how local communities for disaster resilience: data can empower themselves through Community participation participation in providing, sharing, and of information providing verifying the information within their social and verifying in Indonesian networks. The Merapi Circle Information volcanic eruption. Networks (Jalin Merapi or JM) is an example Australian and New of community-based ERC sharing and Zealand Third Sector coordination. JM has developed a Research. November 2014 collaborative system, with community radio Permanent URL: stations and local communities as reliable http://apo.org.au/node/53 information sources and direct verifiers, and 138 also utilizes a community's social media capacity. The system is based on community’s charcateristics, such as local culture, levels of communication technology adaptation,

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

media pereferences, daily communication behaviors and level of trust among community’s members. Local stakeholders’ All hazard Women can play an important role in ERC> Mulyasari F., Shaw R. Role Low Significant engagement Survey This study focuses on the risk of women as risk concerns on Indonesia communications processes in Bandung communicators to methodology and Indonesia, in which women act as risk enhance disaster resilience adequacy of data communicators. The results of this study of Bandung, Indonesia. Nat show that most of the Women’s Welfare Hazards. 2013 69:2137- Association (WWA) leaders at the ward level 2160 perceive that advocating and cooperating with the ward and sub-district government in establishing early warning systems is necessary prior to a disaster taking place. Due to the unique organizational character of WWAs, where the leaders of women at the city, sub-district, and ward levels are the spouses of the mayor, the sub-district leaders, and the ward leaders respectively, the women’s leaders in Bandung have formed an effective working partnership with the local government. Interagency Major Accident To improve the response and management of Lei, 2015. 高速公路突发 Moderate Limited details on information sharing Semi-structured highway emergencies, Lei (2015) suggested to 事件组织间应急信息沟通 interview items, coordination interviews and involve both governmental agencies and 实证研究 [Empirical Study process as well as content analysis NGOs in the information sharing and on the Inter-organizational coding and China coordination mechanism Communication in the process for Case of Expressway content analysis

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

Emergency]. Management Review. 2015, Vol. 27 Issue (9): 213-220. DOI:10.14120/j.cnki.cn11- 5057/f.2015.09.019 Interagency Major Accident Develop an evidence-based, scenario-based Lei, 2015. 高速公路突发 Moderate Limited details on information sharing Semi-structured contingency plan, including communication 事件组织间应急信息沟通 interview items, coordination interviews and plans involving respective responding 实证研究 [Empirical Study process as well as content analysis agencies and case studies on various types of on the Inter-organizational coding and China scenarios, including highway emergencies Communication in the process for Case of Expressway content analysis Emergency]. Management Review. 2015, Vol. 27 Issue (9): 213-220. DOI:10.14120/j.cnki.cn11- 5057/f.2015.09.019 Interagency Major Accident To improve the response and management of Lei, 2015. 高速公路突发 Moderate Limited details on information sharing Semi-structured highway emergencies, Lei (2015) suggested to 事件组织间应急信息沟通 interview items, coordination interviews and develop communication plans that are 实证研究 [Empirical Study process as well as content analysis scenario-based and include details of how on the Inter-organizational coding and China agencies not involved in daily highway Communication in the process for operations can coordinate and share Case of Expressway content analysis information with operations agencies Emergency]. Management Review. 2015, Vol. 27 Issue (9): 213-220. DOI:10.14120/j.cnki.cn11- 5057/f.2015.09.019 Interagency Major Accident To improve the response and management of Lei, 2015. 高速公路突发 Moderate Limited details on information sharing Semi-structured highway emergencies, Lei (2015) suggested to 事件组织间应急信息沟通 interview items, coordination interviews and include communications with lay people, who 实证研究 [Empirical Study process as well as content analysis are unfamiliar with highway operations or on the Inter-organizational coding and

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence

China emergency response, in capacity building Communication in the process for training for responders Case of Expressway content analysis Emergency]. Management Review. 2015, Vol. 27 Issue (9): 213-220. DOI:10.14120/j.cnki.cn11- 5057/f.2015.09.019 Interagency Major Accident To improve the response and management of Lei, 2015. 高速公路突发 Moderate Limited details on information sharing Semi-structured highway emergencies, Lei (2015) suggested to 事件组织间应急信息沟通 interview items, coordination interviews and build a platform for emergency information 实证研究 [Empirical Study process as well as content analysis sharing and offer technical support for on the Inter-organizational coding and China various parties involved Communication in the process for Case of Expressway content analysis Emergency]. Management Review. 2015, Vol. 27 Issue (9): 213-220. DOI:10.14120/j.cnki.cn11- 5057/f.2015.09.019 Interagency Major Accident To improve the response and management of Lei, 2015. 高速公路突发 Moderate Limited details on information sharing Semi-structured highway emergencies, Lei (2015) suggested to 事件组织间应急信息沟通 interview items, coordination interviews and ensure the burden of information processing 实证研究 [Empirical Study process as well as content analysis is manageable, balanced, and within the on the Inter-organizational coding and China capacity of agency that handles it Communication in the process for Case of Expressway content analysis Emergency]. Management Review. 2015, Vol. 27 Issue (9): 213-220. DOI:10.14120/j.cnki.cn11- 5057/f.2015.09.019 Interagency Major Accident To improve the response and management of Lei, 2015. 高速公路突发 Moderate Limited details on information sharing Semi-structured highway emergencies, Lei (2015) suggested to 事件组织间应急信息沟通 interview items,

Practice Type of Emergency Synthesized Findings Citation CERQual Explanation of Study Methods Assessment CERQual Country/ of Confidence Assessment in the Geographic Context Evidence coordination interviews and incorporate mandatory time off in personnel 实证研究 [Empirical Study process as well as content analysis timetable to improve sustainability and on the Inter-organizational coding and China stability. Communication in the process for Case of Expressway content analysis Emergency]. Management Review. 2015, Vol. 27 Issue (9): 213-220. DOI:10.14120/j.cnki.cn11- 5057/f.2015.09.019

Table 17. Synthesis of Findings across Methodological Streams Q2 & Q6

Practice Synthesized Findings Citations CERQual Explanation of Assessment CERQual of Assessment Confidence in the Evidence Creation of task The creation of task forces and committees Chess (2007) Moderate Evidence from forces/committees with key stakeholders has been described as Clarke five different to enhance ERC a mechanism for improving or facilitating (2006) studies information sharing between national and Nowell (Clarke, sub-national authorities and between (2015) Schraagen, agencies. The creation of a bioterrorism task Schraagen Bharosa 2010, force at the county level in New Jersey was (2010) Bharosa 2009 described by Chess (2007) as an important Bharosa and Holmes) mechanism for enhancing risk (2010) was communications and trust among Bharosa individually representatives of different agencies during (2009) evaluated to the 2001 Anthrax incidents. Clarke and Chess Howard be of (2006) describe the utilization of an (2012) moderate Emergency Operations Committee (EOC) Holmes confidence during a university’s response to the Anthrax (2009) level with incidents. In addition a number of minor components for the effective operation of concerns on networks, taskforces, and committees were methodology identified. These include: the importance of and adequacy existing relationships between responders of data. prior to an incident (Nowell 2015), the role Evidence from of network teams, as opposed to remaining hierarchical teams, for improved decision three studies making (Schraagen 2010), and the (Chess, importance of information exchange and Nowell, distribution between decision-making units Howard) was (Bharosa 2010). They also include the use of evaluated to Information Managers (IM) for coordinating be of low information between agencies, so as to confidence reduce information demand and improve level the supply of key information (Bharosa (methodology/ 2009), the importance of a Public data Information Officer (PIO) for improved concerns), but information dissemination (Howard 2012), the same and the role media should play for effective evidence is communication in an emerging infectious reinforced by disease outbreak (Holmes 2009). Schraagen the other five et al (2010) hypothesized that network studies. teams work faster and arrive at more correct decisions than hierarchical teams Network structures. They allow teams to exchange information quickly, monitor each other’s performance, and build mutual trust. Network teams were found to perform

Practice Synthesized Findings Citations CERQual Explanation of Assessment CERQual of Assessment Confidence in the Evidence faster than hierarchical teams, while maintaining the same level of accuracy in relatively simple environments. In relatively complex environments, network teams arrive at correct decisions more frequently than hierarchical teams. Holmes et al (2009) describe the importance of engaging media representatives immediately in discussions about potential emerging infectious disease outbreaks, including the role media should play and how the public health community can help them fulfill that role.

Networks to Regional disease surveillance networks may Gresham Moderate Evidence from enhance provide a useful mechanism for information (2009) both studies information sharing. Moore was sharing Gresham et al. describes the Middle East (2011) individually Consortium on Infectious Disease evaluated to Surveillance (MECIDS), a regional disease be of surveillance network of public health experts moderate and ministry of health officials from Israel, confidence the Palestinian Authority, and Jordan. level with MECIDS unites public health officials of minor differing Middle Eastern nationalities and concerns on contributes to regional health and stability methodology by engaging in regular cross-border and/or information exchange, conducting regular adequacy of executive board meetings, performing data laboratory and risk communications training, and implementing innovative communication technology. During H1N1, the MECIDS partners agreed to prompt and coordinated border and airport screening, laboratory testing, information exchange, and common communication strategies. This coordination can be largely contributed to the existence of both trust and well- exercised national and regional pandemic preparedness plans, which were initially established within this network. Moore and Dausey describes the Mekong Basin Disease Surveillance (MBDS), a cooperating network of six countries to collaborate on sub-regional infectious disease surveillance and control - Cambodia,

Practice Synthesized Findings Citations CERQual Explanation of Assessment CERQual of Assessment Confidence in the Evidence China (originally just Yunnan province and, since 2008, Guangxi Province as well), Lao People’s Democratic Republic, Myanmar, Thailand and Vietnam. During the H1N1 pandemic, MBDS health leaders perceived their pandemic responses to be effective in areas that, prior to the creation of the network, were considered problematic. Participants noted the ability of their country level surveillance systems to exchange information efficiently within the country and the importance of MBDS for enabling timely coordinated regional response to detect disease at cross border sites and prevent the spread of the virus across countries.

Use of information Tools and platforms can be used as Celik (2010) Moderate Evidence from systems to information systems to share information. Seyedin seven enhance ERC Celik et al. (2010) describes enhancements (2011) different in the use of the communication Ipe (2010) studies (Ipe, infrastructure by comparing the response to Bharosa Bharosa 2009, two earthquakes that occurred in Turkey, (2009) Leonard, and underlines the importance of Leonard Thiago, organizational learning, as well as (2014) Kapucu, investments in information technology, to Thiago Militello and enhance capacity for the search for, (2013) Chang) was acquisition and exchange of information. Chang individually Seyedin et al. (2011) describes the types of (2013) evaluated to databases and information system in Iran Kapucu be of should include for effective communications (2006) moderate in emergency management. Ipe et al. (2010) Militello confidence describes a surveillance system used in the (2007) level with US, named the Medical Electronic minor Surveillance and Intelligence System concerns on (MEDSIS), and analyzes the role of methodology stakeholders in the exchange of information and/or through this system. Bharosa et al (2009) adequacy of reviewed the flood management system, data. Evidence named Flood Information and Warning from System (FLIWAS), for improved Dutch- remaining two German cooperation in flood scenarios. studies (Celik FLIWAS was developed to optimize the from Turkey exchange of information during threatening and Seyedin high water situations within and between from Iran) water management and calamity was evaluated to be of low

Practice Synthesized Findings Citations CERQual Explanation of Assessment CERQual of Assessment Confidence in the Evidence management organizations. Leonard et al. confidence (2014) describes different mechanisms of level communication in relation to volcanic (adequacy of eruptions in New Zealand abased on the data audience (i.e. scientists, emergency concerns). managers, media, public, etc.). Thiago et al. (2013) describes the role of social networks and mobile phones in alerting and preparing people to avoid or face natural disasters in a region of Brazil. Collaboration between government and Civil Defense in rapidly testing and developing new channels of communication was found to be successful in terms of quantity and quality of information shared both within the organization and with the target population. Based on survey results of a sample of first responders for a disastrous typhoon in Taiwan in 2009, Chang and Wang (2013) present the official information sharing and coordination operations and an emergency management information system (EMIS) in Taiwan. The authors also discuss what needs to be done, or is currently being done, to improve the system. Kapucu et al. (2006) describes how the lack of an integrated information system greatly limited the coordinated response of agencies (police, fire department and port authority) following the World Trade Center terrorist attack in New York city. In contrast to the more technical operating platforms, Militello et al (2007) suggests that low cost, paper based tools, including notebooks, whiteboards, and telephone books, provide important tools for communication and can support asymmetric knowledge in an Emergency Operations Center (EOC). Often times, EOC teams are made up of individuals with differing levels of experience in regards to crisis management, familiarity with emergency response tools and procedures, which can make it difficult to effectively utilize electronic tools without onsite support.

Practice Synthesized Findings Citations CERQual Explanation of Assessment CERQual of Assessment Confidence in the Evidence Local Engagement of local stakeholders is Ardalan Moderate Evidence from stakeholders’ important for the effectiveness of the ERC (2009) five different engagement strategy. The literature presents examples of Cole (2014) studies mechanisms for the engagement of local Shepherd (Ardalan, stakeholders in communication efforts. (2014) Cole, Ardalan et al. (2009) describe the use of Gultom Shepherd, Village Disaster Taskforces, which act as (2014) Gultom, Lei) operational units in an early warning Mulyasari was mechanism for flooding in Iran. Cole et al. (2013) individually (2014) describes the importance of using Lei (2015) evaluated to existing social networks in small be of municipalities for disaster risk reduction moderate activities. Shepherd et al. (2014) addresses confidence the issue of communicating with culturally level with diverse communities, and the need for minor centralizing resources that can be used to concerns on facilitate communication with these groups. adequacy of Gultom et al. (2014) describes the use of a data. Evidence community based communication system in from one Indonesia to facilitate the response to study volcanic eruptions. Mulyasari et al. (2013) (Mulyasari) describes the role of networks of women in was evaluated facilitating communication for all hazards in to be of low Indonesia. Lei (2015) describes the barriers confidence to effective information sharing and level (method response coordination among agencies and data during a highway emergency and proposes concerns). possible solutions.

QUESTION 8

COUNTRY DISTRIBUTION – ENGLISH LITERATURE The majority of the literature (22 studies out of 48) is from the USA, followed by literature from the UK.

Figure 11. Distribution of countries included in the English language evidence synthesis for Question 8.*

25 22

10 7 5 1 1 2 2 1 2 1 1 1 1 1 2 1 1 1

0 NumberofArticles

*Note: These columns are not mutually exclusive. One article could list one or more countries.

DISASTER TYPE DISTRIBUTION – ENGLISH LITERATURE Much of the literature coded as directly relevant to question 8 is focused on pandemic influenza, namely H1N1.

Figure 12. Distribution of disaster type included in the English language evidence syntheses.*

20 19

16 14 12 10

NumberofArticles 8 6 5 4 4 4 3 3 2 2 2 2 1 1 0

*Terrorism includes Bioterrorism, Chemical Attack, Radiological and Nuclear Terrorism *Note: These columns are not mutually exclusive. One article could list one or more disaster types.

EVIDENCE SYNTHESIS – ENGLISH LITERATURE The literature review led to the identification of 45 articles that describe population access, ability to process and act upon the information received during an emergency. By reading the articles we extrapolated the data gathering tools that were used to collect such information (i.e. surveys, focus groups, etc…). Table 18 and the summary provided below reflect the type of information that was derived by the use of each data gathering tool and does not provide information on the specific study findings because such findings are focused on the reporting of population data such as population access to information,

86 population behaviors etc. rather than on the practice or appropriateness of using a specific data gathering tool (i.e. applicability of focus groups and/or survey).

The 45 articles describe studies in which three main mechanisms for data gathering, analyzing, and interpreting risk communications data were used:

1) Community & Audience Feedback: mechanisms that are implemented to gather community feedback and engagement in the development of communication strategies. 2) Testing of messages and educational material: mechanisms used to test specific emergency communications messages or educational material 3) Tools and methods for data collection: data gathering mechanisms including media and social media monitoring, surveys, focus groups, interviews and analyses of the response to an incident (i.e. case studies)

In this case as well literature findings cannot be aggregated, the tools have been listed and categorized according to the three themes reported above as a list of practices from the field. The following table (Table 18) describes the study findings within methodological stream and across methods for the English literature.

Figure 13; data gathering tools categorized by preparedness cycle phase.

Preparation Response/Recovery (Tools for exploratory & engagement (Tools to assess and monitor the purposes in the elaboration of the effectiveness of the ERC strategy) ERC strategy)

Surveys Focus groups Focus groups Interviews Interviews

Media/social media monitoring Surveys Website monitoring Participatory approaches Case studies

Table 18 Description of use of data gathering tools per study

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) Blendon This study Yes Bioterror USA High Based on RJ. suggests the ism data from (2003) use of timely series of public surveys opinion polls can provide public health authorities with information on confidence in public officials, precautions taken, knowledge on diseases and treatment, public attitudes, and views on the likelihood of discriminatio n in post- attack services. Rapid

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) turnaround surveys can help authorities determine if the public is actually doing what is recommend ed. Driedge This study Yes H1N1 Canada High Strong r SM. uses focus methods, (2013) groups and rich data interviews to describe the reactions of segments of the Aboriginal population, which interpreted the government’ s decision of prioritizing their population during the H1N1 vaccination campaign as

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) a stigmatizatio n. The data collected showed the need to communicat e clearly how certain decisions are being made. Galarce This study Yes Water USA High Strong EM. presented emergen methods, (2012) the results of cy rich data an online survey of residents of Boston affected by a water crisis, capturing information on how acquisition, processing and information sharing differed across groups during the

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) emergency, demonstrati ng that these types of surveys may be used to describe how effectively crisis messages are conveyed to and received by different segments of the population. Massey This study Yes H1N1 Australi High Strong PD. used a a methods, (2011) participator rich data y action research approach (PAR) to understand what information Aboriginal and Torres Strait Islanders communiti

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) es in Australia wanted regarding H1N109 and how it should be received. The findings from this study show how PAR can offer a method of consultatio n with communiti es for developing effective and culturally appropriate health policies and communica tion strategies. Mebane This study No Anthrax USA Moderat Minor F. analyzed e concerns on (2001) materials methodology

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) CDC and prepared adequacy of during the data anthrax crisis that were made available to journalists and the public via CDC’s website and identified possible incongruities in emphasis between CDC information and print news. Findings show how, in this specific case, news coverage was very limited on specific recommend ations and

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) suggests how the promotion of messages would benefit from specific requests from CDC to reporters on what pieces of information to deliver, better understandi ng of how to promote messages when public health is the featured event and when it is not, and the need to ensure that all sources of public health information know how

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) to tap into the agency’s sources of information. Rubin This study No Radiologi UK Moderat Minor GJ. presented cal e concerns on (2011) interviews incident methodology conducted and with people adequacy of potentially data exposed to the 2006 London Litvinenko incident. Findings were useful to learn from the risk communicati ons efforts of public health responders, in particular the importance of helping individuals to identify their risk of exposure,

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) understand the difference between acute and chronic effects of exposure and identify best ways to deliver exposure test results (i.e. urine tests). Interviews findings also suggested how attempts at providing reassurance in the absence of specific information are likely to be counterprod uctive in any future radiological

Sugerm This study Yes Wildfires USA Moderat Minor an DE. administered e concerns on (2012) a survey adequacy of through data random digit dialing to determine residents’ exposure to, understandin g of and compliance with risk communicati ons messages during the 2007 San Diego county wildfires. Findings showed how non- technical message recall, understandin g and

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) compliance were high during the wildfires while recall, understandin g and compliance with technical messages (i.e. place air conditioning in recirculate, use of N-95 during ash cleanup) were low. Tian X. This study, . No Earthqua China Low Significant (2016) by the use ke concerns on of mixed methodology methods and (interviews coherence and surveys) assessed earthquake survivors’ knowledge, skills and practice

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) prior to and after the implement ation of health education and promotion campaigns . In this study, Yes H1N1 Netherl High Data from Willemi multiple ands multiple en van (overtime) surveys, der cross- strong Weerd sectional methods (2011) telephone surveys were used to assess the importance of government trust and risk perception with respect to vaccination acceptance Walter In this study, No H1N1 German Moderat Minor D. consecutive y e concerns on (2012) surveys were methodology used to

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) assess knowledge, attitude, and behaviors on vaccination against pandemic influenza in Germany. Information on sources of information, and information seeking behaviors were also collected. Findings showed that mistrust in the vaccines and the perceived low risk of the disease were the main barriers that contributed to the low

100

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) vaccination coverage. Wang J. In this study, Yes Water USA Low Significant (2011) in-person emergen concerns on surveys were cy methodology conducted in and the waiting adequacy of areas of data clinics and emergency departments at a large urban safety net hospital. Findings provided a snapshot of how and when various segments of the population received the boil water message during the emergency. Woien In this study, No H1N1 Norway Low Significant G. a poll was concerns on (2009) conducted methodology by randomly and

101

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) selecting adequacy of survey data participants from the telephone directory to gather opinions on the perceptions of Norwegian citizens on the level of preparednes s of Norwegian authorities to respond to the H1N1 pandemic, the information provided to the public, media coverage and perceived risk.

102

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) Yen In this study, Yes Acute Taiwan Moderat Minor (2009) a Hemorrh e concerns on questionnair agic methodology e was Conjuncti and administered vitis adequacy of to parents of (AHC) data children attending schools affected by the 2007 acute hemorrhagic conjunctivitis (AHC) in Taipei. The survey assessed source of information as well as compliance with recommend ed behaviors (i.e. home confinement of symptomatic students). In addition attack rates

103

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) and duration of the epidemic was compared between Taipei and Keelung where a specific risk communicati on strategy (use of SMS) was not implemente d. Eisenma This study Yes Hurrican USA Moderat Minor n D. conducted e e concerns on (2007) interviews adequacy of with data Hurricane Katrina evacuees. Findings demonstrate d the importance of creating disaster plans that account for the specific

104

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) obstacles encountered by vulnerable and minority communities and the important influence of extended families and social networks in communicati ons strategies. Findings also showed the importance of education on disaster terminology (i.e. category of the Hurricane) and precedent experience with distrust in authorities and the

105

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) potential role of faith based organization s in disseminatin g the message. Rundbla In this study, Yes Water UK Moderat Minor d G. a postal emergen e concerns on (2010) questionnair cy adequacy of e was used data to assess information sources and compliance with recommend ed behaviors during two water emergencies in the UK, and how compliance varied based on socio- demographic characteristi cs of the affected population.

106

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) Rubin In this study, Yes H1N1 UK Moderat Minor GJ. media e concerns on (2010) monitoring methodology and telephone surveys were implemente d in the UK to assess the influence of media on levels of worry in the community, factors predicting likely acceptance of vaccination against swine or seasonal flu, and the effects of advertising and media coverage on behavioral change during the

107

Glik D. In this Yes General USA High Strong (2013) study, prepared methods household ness (cohort preparedn study), rich ess data education received through print media was compared to household preparedn ess education received through community health workers. Findings showed that simple media- based communica tion may

108

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) be sufficient to encourage disadvanta ged households to obtain disaster supplies, however adoption of the more complex disaster family communica tion requires interperson al education. Maxwell In this No West USA Moderat Minor TA . article, Nile virus e concerns on (2003) two case and methodology studies Anthrax (1999 West Nile virus outbreak in NY and 2001

109

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) Anthrax attacks) were examined to provide informati on on the process of emergenc y communic ations and the behaviors and decision making of governme nts, intermedi aries and the public. Aburto In this study, Yes H1N1 Mexico Moderat Minor NJ. face to face e concerns on (2010) household methodology surveys were conducted during H1N1 with representati ve samples of the

110

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) population in three cities in Mexico, to determine what proportion of the population received community mitigation messages and adopted one or more community mitigation efforts. Confusion regarding messages and economic barriers to community mitigation efforts (i.e. cost of masks, soaps etc...) were sometimes reported, highlighting

111

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) the need to provide feasible community mitigation recommenda tions that minimize negative impact on economic and social functioning and to allocate resources to address logistic and functional barriers for the adoption of community mitigation measures. Basema In this study, No General USA Moderat Minor n JG. telephone prepared e concerns on (2013) surveys were ness methodology used to assess alert fatigue among

112

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) healthcare professionals towards various emergency preparednes s messages, through the use of an experimental study design. Findings showed that information delivered too frequently and/or repetitively through numerous communicati on channels may have a negative effect on the ability of health care providers to effectively recall emergency information.

113

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) Sutton J. In this study, No Floods USA Low Significant (2015) Twitter concerns on accounts relevance were examined during the 2013 Boulder, Co floods by the use of quantitative and qualitative data analysis methods. Message content and style were examined. Findings suggested that messages should be more balanced in presenting information that provides instructional messages as

114

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) well as information about the hazard. However, the extent to which such information influences the desired public health response was not tested. Gesser- In this study, No Ebola Israel Low Significant Edelsbu surveys and concerns on rg A. non- relevance (2015) probability and sampling adequacy of were used to data assess worry levels and knowledge about Ebola in Israel between healthcare workers and the general public.

115

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) Findings showed similar patterns in knowledge between these two segments of the population and specific areas in which knowledge was lacking (i.e. confusion about ways by which Ebola was transmitted). UNICEF In this study, Yes Ebola Sierra High Strong (2014) knowledge, Leone methods, attitudes, rich data practices and behaviors surveys were conducted in Sierra Leone during the 2014 Ebola outbreak, to

116

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) assess and monitor overtime acceptance of treatment recommenda tions, misconceptio ns, general knowledge about Ebola and specific knowledge about the means of prevention. Kim NH. In this study, No Radiologi Korea High Strong (2015) computer- cal methods, aided incident rich data telephone interviews were conducted in Korea to obtain views, risk perception, knowledge and confidence in existing information

117

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) sources about food safety after the Fukushima incident, and the information required to develop strategic risk communicati on plans in relation to nuclear accidents. Jehn M. In this study, No H1N1 USA High Strong (2011) a telephone methods, survey was rich data implemente d in Arizona during the H1N1 pandemic to assess H1N1 knowledge, information sources, perceptions and preparednes s, and to

118

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) identify misconceptio ns across segments of the population. Roussea In this study, No H1N1 France Low Significant u C. media and concerns on (2015) coverage Canada relevance analysis was and minor conducted concerns on simultaneou methodology sly with focus groups of call centers personnel during the H1N1 pandemic in France and Quebec, to provide information on media coverage, its relationship with risk perception, use of call centers and opinions on

119

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) credibility of voices and messages and compliance with recommend ed behaviors. Pearce In this study, Yes Radiologi UK High Strong JM. scenario cal methods, (2012) based focus incident rich data groups were used to identify perceptions and reactions to a covert radiological device from the general public. The results of the focus groups were used to design and implement a national telephone survey which

120

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) quantified intended behaviors and assessed what perceptions were correlated with these behaviors. Subsequentl y, focus groups and survey results were used to develop video and leaflet communicati on intervention s, which were then evaluated in a second round of focus groups.

121

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) Murphy In this study, Yes Flood USA High Strong MW. in-person methods, (2010) surveys were rich data implemente d after flood events with affected populations to assess how and what type of information was received and whether the right message reached specific subgroups that were more vulnerable to certain health issues. Spence In this study, Yes Hurrican USA Moderat Minor PR. surveys were e Katrina e concerns on (2007) conducted in methodology the aftermath of Hurricane Katrina to

122

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) assess differences in information sources and information seeking behaviors between disabled and non-disabled evacuees. Findings showed that disabled individuals were less likely to engage in information seeking behaviors, and that they had different interests in the type of information they were looking for.

123

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) Kuppus In this study, Yes Cyclones India Low Significant wamy S. a survey concerns on (2014) methodology adequacy of was used to data and examine the minor effectiveness concerns on of broadcast methodology and narrowcast media for disaster communicati ons. Findings indicate differences across segments of the population (i.e. age, educational background) in access to information and community response. Savoia In this study, Yes Water USA High Strong E. an online emergen methods, (2015) survey was cy rich data implemente d to assess

124

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) what factors influenced receipt of information, public compliance with the recommend ed behaviors, and opinions on the government’ s response. Data from this study show that a higher perception of risk and timely receipt of information are associated with compliance with recommend ed behaviors, underlying

125

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) the importance of releasing information to the public as quickly as possible during a crisis. This study also highlights the importance of coordinating risk communicati on activities beyond the area of the incident to assure public understandin g of what measures are recommend ed, which are not and where they need to be implemente

126

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) d. Odlum In this study, No Ebola USA Moderat Minor M. Twitter was e concerns on (2015) monitored methodology as a real- time method of Ebola outbreak surveillance, to monitor information spread, capture early epidemic detection, and examine the content of public knowledge and attitudes. Ohno K. In this study, No Radiologi Japan Low Significant 2015 questions cal concerns on and answers incident methodology were and collected adequacy of related to data inquiries from the general

127

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) public that were posted to reliable websites, including those of the government and radiation- related organization s. The analysis of such data was used to identify information al needs. Rubinst In this No H1N1 UK High Strong ein H study, methods, (2015) focus rich data groups and interviews with members of the public were conducted using uncertain and severe scenarios,

128

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) and messages were tested giving advice on vaccination and antiviral medicines. Data were thematicall y analyzed using the theoretical framework provided by the COM-B model. The benefit of using the COM-B model is that it is at the core of an approach that can identify effective strategies

129

Perko In this study, Nuclear Belgium High Strong (2013) opinion emergen methods, surveys were cy rich data used to assess receipt of information and factors associated with attention to information on protective actions. Rogers In this No Radiologi UK Low Significant (2013) study, a cal concerns on focus incident methodology group was and conducted adequacy of involving a data scenario presented in mock news broadcasts

130

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) to explore levels of public knowledge, informatio n needs, and intended behavioral reactions to an attack involving an overt radiological dispersal device (RDD), or dirty bomb. The findings from these focus groups were used to design messages for the public that could be presented in a short

131

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) leaflet. The effects of the leaflet were then tested on reactions to the same scenario in further focus groups. Boyd In this study, Yes H1N1 USA Moderat Minor CA. women from e concerns on (2013) low income methodology communities , considered priority groups for receiving the H1N1 vaccine, participated in focus groups to identify barriers and motivators to receiving the vaccination, as well as

132

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) information needs, sources, and information- seeking behaviors. Ferrante This study Yes H1N1 Italy Moderat Minor G. utilized the e concerns on (2011) ongoing methodology Italian Behavioral Risk Factor Surveillance System (PASSI) for investigating people's opinions and behaviors regarding the A/H1N1 pandemic by adding a pandemic specific module. Staes CJ In this study, No H1N1 USA Low Significant (2009) surveys of concerns on primary care relevance providers and minor were concerns on implemente methodology

133

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) d to describe communicati ons processes and information flow from public health authorities to the providers, by assessing distribution of emails, proportion of clinicians who accessed key Web sites weekly, clinicians' knowledge about recent guidance and perception about email load, primary information sources, and qualitative findings from clinician

134

Burns R In this study, Yes Wildfires Australi Moderat Minor (2010) a survey and a e concerns on focus groups methodology were used to investigate how the risk of a bushfire emergency and communicati on strategies are perceived by different community segments. Voeten This study, Yes SARS UK and Low Significant HA through the Netherl concerns on (2009) use of a ands methodology computer and assisted adequacy of phone data survey, assessed and compared the use of information sources,

135

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) knowledge, and health beliefs related to SARS and avian flu of Chinese residents of the UK and the Netherlands. McGoug This study Yes Bioterror USA Moderat Minor h M. aimed to ism e concerns on (2005) develop methodology communicati ons strategies to reach special populations in North Dakota before, during, and after a bioterrorism attack or other crisis. To achieve the study objectives, the investigators

136

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) used telephone interviews and telephone focus groups with organization s that represented special populations. Areas of inquiry included attitudes and concerns about crises, sources of information used and those identified as most credible, methods to reach people during a crisis event, and awareness of and attitudes

137

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) about the agencies and organization s that affect risk communicati ons. Quinn In this study, No Anthrax USA High Strong SC interviews methods, (2005) and focus rich data groups were conducted with postal workers employed at three facilities: Trenton, New Jersey; New York City; and Washington, DC during the Anthrax attacks. Lessons learned from this attack contribute to the growing body of

138

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) knowledge available to guide communicati ons experts and public health professional s charged with crisis and emergency risk communicati ons with the public. Henders In this study, Yes Chemical USA High Strong on JN focus groups attack methods, (2004) were used to rich data examine people's perceptions and reactions to a hypothetical terrorist attack involving a chemical agent (specifically,

139

Becker In this study, No Nuclear/ USA High Strong SM focus groups radiologi methods, (2004) were cal rich data conducted in terrorism which a multi-part, hypothetical radioactive materials terrorism situation was discussed. Twelve of the focus groups were carried out with members of the general public (drawn from a variety of ethnic backgrounds and geographic locations),

140

Community Testing of Tools and methods for data collection Does the CERQual Explanation of audience & messages, (i.e. knowledge, risk perception, behaviors, trust, etc.) study Type of Country Assessment CERQual Source feedback delivery of address emergency of Assessment (article) messages & News, websites Surveys Focus groups Case studies equity Confidence educational and social (online, by and interviews issues? in the material media telephone, in Evidence monitoring person) and four groups were composed of first responders, hospital emergency department personnel, and public health professional s. One aim of the focus groups was to elicit detailed information on people's knowledge, views, perceptions, reactions, and concerns related to a nuclear/ radiological terrorism event, and to better understand

141

142

Table 19. Synthesis of Findings across Methodological Streams Q8

Best Tools Synthesized Findings Citations CERQual Explanation and methods Assessment of CERQual for collecting of Assessment data and/or Confidence monitoring in the effectiveness Evidence of ERC Surveys Galarce (2013), Sugerman (2012), Galarce Moderate Evidence Walter (2012), Rundblad (2010), Jehn (2013) from eight (2011), Spence (2007), Savoia (2015), Sugerman studies was Kuppuswamy (2014), Ferrante (2011), (2012) individually Burns (2010), Voeten (2009) and Perko Walter evaluated to (2013) assessed how messages are (2012) be of high received, processed and acted upon by Rundblad confidence different segments of the population (2010) level, and that through online and telephone surveys. Jehn (2011) from further Baseman (2013) Staes (2009) evaluated Spence eight was of the alert fatigue among healthcare (2007) moderate professionals and flow of information Savoia confidence from public health authorities to (2015) level with healthcare providers. Kuppuswamy minor Blendon (2003) and Woien (2009) (2014) concerns on describe use of public opinion polls to Ferrante methodology assess if the public is following (2011) and/or government’s recommendations and Burns (2010) adequacy of their opinions on the government’s Voeten data. response. Willemien van der Weerd (2009) Evidence (2011) and Walter (2012) used Perko (2013) from only five longitudinal surveys repeated over time Baseman studies was of to monitor population knowledge and (2013) low behaviors. Staes (2009) confidence In-person surveys were used by Aburto Blendon level, mainly (2010) to identify barriers to compliance (2003) on account of with recommended behaviors; Murphy Woien methodology (2010) to assess when the right message (2009) of survey and reached specific subgroups most Willemien strength of vulnerable to health consequences; van der data. UNICEF (2014) to identify Weerd misconceptions during the West African (2011) Ebola epidemic; and by Wang (2011) to Aburto assess receipt of information. (2010) Murphy (2010) UNICEF (2014) Wang (2011)

Focus Groups Driedger (2003) and Quinn (2005) used Driedger High Evidence focus groups to identify communications (2003) from six challenges in specific at-risk groups. Quinn (2005) studies was In preparation to emergencies, scenario Pearce individually based focus groups were utilized for (2012) evaluated to

143

Best Tools Synthesized Findings Citations CERQual Explanation and methods Assessment of CERQual for collecting of Assessment data and/or Confidence monitoring in the effectiveness Evidence of ERC developing communications materials Boyd (2013) be of high and assessing information needs by Henderson confidence Pearce (2012), Boyd (2013), Henderson (2004) level, and that (2004), Becker (2004), Rubinstein Becker from (2015) and McGough (2005) (2004) remaining Rubinstein two was of (2015) moderate McGough confidence (2005) level with minor concerns on methodology.

Interviews Eisenman (2007) conducted interviews Eisenman Moderate Evidence to (2007) from both identify specific preparedness obstacles Rubin (2011) studies was across segments of the population; while individually Rubin (2011) studied reactions to very evaluated to specific emergency situations be of moderate confidence level, with minor concerns on methodology and adequacy of data.

Media/ Media monitoring and media coverage Rousseau Low Evidence Internet analysis paired with focus groups (2015) from one monitoring (in (Rousseau 2015, Rogers 2013) and Rogers study was of addition to population surveys (Rubin 2010) were (2013) high surveys/ focus used to assess the impact of media Rubin (2010) confidence groups) coverage on population knowledge, Sutton level, while reactions and behaviors. (2015) that from four Sutton (2015) and Odlum (2015) Odlum was analyzed social media accounts to (2015) individually determine the extent and type of Mebane evaluated to coverage of information delivered (2001) be of through this means of communication. Ohno (2015) moderate Mebane (2001) and Ohno (2015) Glik (2013) confidence monitored websites to identify Tian (2016) level. inconsistencies in the delivery of Yen (2009) Evidence messages and information needs. from other Experimental or quasi--experimental five studies methods were applied by Glik (2013), was of low Tian (2016) and Yen (2009) to study the confidence

144

Best Tools Synthesized Findings Citations CERQual Explanation and methods Assessment of CERQual for collecting of Assessment data and/or Confidence monitoring in the effectiveness Evidence of ERC impact of how messages are delivered or level, mainly that of educational campaigns and risk on account of communications strategies. methodology, relevance and/or adequacy of data.

Mixed: Kim (2015) and Gesser-Edelsburg Kim (2015) Moderate Evidence Survey + (2015) used surveys and interviews to Gesser- from Kim Interview assess information needs in preparation Edelsburg (2015) was of for an emergency, as well as the (2015) high population knowledge about a possible confidence emergency level, but that from Edelsburg (2015) was of low level due to concerns on relevance and adequacy of data

Other Maxwell (2003) used a case study Maxwell Moderate Evidence methods: approach to describe the decision- (2003) from Massey Case study, making process of government officials Massey (2011) was of participatory in relation to specific emergencies. (2011) high action Massey (2011) utilized participatory confidence action approach to integrate community level, but that feedback in emergency planning efforts. from Maxwell (2003) was of moderate level with minor concerns on methodology

Equity: Americans’ views about the likelihood of discrimination in the provision of post attack services have been investigated in the event of a bioterrorism attack. Such information is important because people who expect unfair treatment may be reluctant to seek care or comply with quarantine orders (Blendon RJ 2003). Similarly, the perception of stigmatization has been assessed in Aboriginal populations in Canada during the H1N1 pandemic (Driedger SM 2013) and data on what information Aboriginal populations and Torres Strait Islanders in Australia preferred to receive were also assessed

145

(Massey PD 2011). Different mechanisms for dissemination of information were tested, focusing on Latino communities in the US (Glik D 2014). How the information is received, processed and acted upon by different segments of the population identified by socio-demographic characteristics has been evaluated during several types of emergencies including: water crisis (Rundblad G 2010, Galarce EM 2012, Wang J 2011, Savoia E. 2015 ), wildfires (Sugeman 2012, Burns 2010), H1N1 (Willemien van der Weerd 2011, Rubin 2010, Ferrante G. 2011, Boyd 2013), SARS (Voeten HA 2009) acute hemorrhagic conjunctivitis (AHC) (Muh-Yong Yen 2009), Ebola (UNICEF 2014), flood (Murphy 2010), hurricane (Spence 2007), other weather related disasters (Kuppuswamy 2014), potential radiological incidents (Pearce JM. 2013), bioterrorism (Mc Gough), and chemical attacks (Henderson 2004). Social networks for the dissemination of information across vulnerable groups have been assessed during the response to Hurricane Katrina (Eiseman D. 2007), the economic impact of mitigation measures and the ability to adopt disseminated information on community mitigation were assessed by socioeconomic position in Mexico during the H1N1 pandemic (Aburto NJ 2010).

EVIDENCE SYNTHESIS – CHINESE LITERATURE A total of 11 articles were identified through the Chinese literature review and included as directly relevant to question 8. All articles referenced China. Of the 11 articles, five were based on Pandemic Influenza, six referenced all hazards/general disasters; among them, three were supported with a case study on an earthquake, a measles vaccination campaign, and a terrorist attack.

The following table (Table 20) describes the data gathering tools per study.

146

Table 20. Description of use of data gathering tools per study

Source Testing of messages, Tools and methods for data collection Does the Type of Country CERQual Explana (article) Community delivery of messages (i.e. knowledge, risk perception, behaviors, study address emergenc Assessm tion of audience & and educational trust, etc.) equity issues? y ent of CERQua feedback material News, websites Surveys Focus Case Confiden l and social media (online, groups studi ce in the Assess monitoring by and es Evidence ment telephone intervie , in ws person) Chen, Microblog The mission of the China Weibo has Yes – The All hazards China Moderat Limited 2013; “Weibo,” China public opinion established a served as Index showed such as e details Chen, Chinese monitoring office is public opinion a platform that there is a H1N1, on data 2014; twitter, has to scientifically and monitoring for the strong H7N9, collecti Liu, been routinely evaluate office, which Chinese association major on 2013; integrated into and report on releases governme between the accidents, process, Zeng, China’s government service quarterly, bi- nt to level of terrorist and 2015; emergency performance on yearly and survey development attacks, analysis Zhang X, response Weibo. The office annual reports public of and 2015; system as a gathers and analyzes on Weibo opinions regions/provin earthquak Zhang critical and Weibo data and monitoring and ce, size of the es. XE, central risk publishes a Weibo results. behavioral population, 2015; communication Government Service response. and the Zhou, s tool by the influence Index As of December For government 2015; Chinese biannually. The 2014, there are example, service The government. Index consists of 130,103 official during the performance. People’s indicators of microblog response Provinces Public Most Chinese dissemination and “Weibo” to the such as Opinion government impact, service accounts H7N9 Jiangsu, Monitor agencies use it performance, and registered with pandemic, Shangdong, ing for public two-way Sina Corporation via its own and Sichuan Office communication communication/inte (the largest official rank highly, and during normal raction. microblog Weibo whereas Weibo times and Lessons learned and company in accounts, Hainan, Data public health best practices of China) by the 12320 Qinghai and Center, emergencies. Weibo-based public government offices Xizang are 2012- The goals are communication, agencies (n= implemen among the 2016. to achieve including public 94164) and ted a lowest ranks. 147

Source Testing of messages, Tools and methods for data collection Does the Type of Country CERQual Explana (article) Community delivery of messages (i.e. knowledge, risk perception, behaviors, study address emergenc Assessm tion of audience & and educational trust, etc.) equity issues? y ent of CERQua feedback material News, websites Surveys Focus Case Confiden l and social media (online, groups studi ce in the Assess monitoring by and es Evidence ment telephone intervie , in ws person) operation and ongoing national evaluation effort to mechanism for establish a the Chinese standardized government’s operation and Weibo use and evaluation services’ mechanism/regu performance. lation for Chinese government’s Weibo use and services’ performance. Wang, 12320, direct Based on available 12320, the only To Yes – limited All hazards China Moderat Limited 2010; two-way science on H1N1 and official health understan by such as e details Jiang, communication the feedback hotline in China, d public capacity/reso the 2008 on data 2012; channel gathered via 12320, which offers opinion urces, there is baby collecti Zhang X, between the the only official health regarding an inequality formula on 2015. government health hotline in consultation the in the contamina process, and the public, China, China’s MoH directly to the Governme development tion crisis, and is considered a developed a series of public, acted as nt’s of 12320 the 2008 analysis trusted channel guidelines for H1N1 an important response among cities Beijing of public inquiry, as emergency risk to the and provinces. Olympics, communication well as H1N1 health communications H7N9 the 2009 by the public. It education materials. agent and had pandemic, A(H1N1) offers health These technical taken on the the 12320 pandemic, consultations materials were role of gathering offices and directly to the disseminated public reactions implemen measles public, acts as through 12320 local data. The ted a vaccinatio an important offices across the response to the survey via n emergency risk country and the H1N1 pandemic its own campaigns communication official websites. is one such official . 149

150

Summary of Chinese Literature Findings

Microblog “Weibo,” Chinese twitter (Chen, 2013; Zhang, 2015; Chen, 2014; Chen, 2015; Liu, 2013; The People's Public Opinion Monitoring Office and Weibo Data Center, 2012-2016) and 12320, the official Chinese public health hotline (Wang, 2010; Jiang, 2012), have been integrated into China’s emergency response system as a critical and central risk communications tools by the Chinese government to gather and monitor public opinions, disseminate information and evaluate government services and public communication efforts. Working with Weibo’s service provider, China’s People’s Public Opinion Monitoring Office and Weibo Data Center routinely publishes the Weibo Government Service Influence Index, the evaluation results on government service Weibo accounts, and develops training materials. Both tools are being used in routine day-to-day operations and during all type of emergencies, such as pandemic outbreaks, earthquakes, and major accidents.

Equity: Disparities were found in the development and performance of the government services Weibo and 12320. The Weibo Government Service Influence Index, jointly published by the China’s People's Public Opinion Monitoring Office and Weibo service providers, Sina and Tencent, show that there is a strong association between the level of development of the regions/province, size of the population, and the government services’ performance. Provinces such as Jiangsu, Shangdong, and Sichuan rank highly, whereas Hainan, Qinghai and Xizang are among the lowest ranked (People's Public Opinion Monitoring Office and Weibo Data Center, 2012-2016). A similar pattern is found in the development of 12320 among cities and provinces, where some cities and provinces are limited by capacity and resources to develop or expand the 12320 networks (Wang, 2010; Jiang, 2012).

151

LIMITATIONS

As a field of research, Emergency Risk Communication (ERC) is still not well-defined. Due to this fact, it is difficult to identify search terms that are sensitive enough to address all types of disciplines working in this field. Throughout our literature review process we tried to be more inclusive, and reviewed many full-text articles that were not clear after the abstract phase. We found highly relevant literature that doesn’t directly answer the WHO questions or that was non-empirical in nature. Literature that partially addresses the phenomena of interest or provided some recommendations related to the SPICE criteria was included. Overall, we found a lack of empirical literature related to WHO questions Q1, Q2 and Q6. More empirical studies were identified for Q8. However such studies presented the use of data gathering tools without proving how results from such data collection processes influenced policy or practice.

In general, the amount of literature from low- and middle- income countries is low. We attempted to address this issue by searching databases in Chinese, Portuguese, and Spanish, as well as loosening the strict definition for empirical literature to include more case studies and to reflect a broader distribution of country experiences and knowledge. As the topic of this knowledge synthesis is emergency risk communications with the public and among agencies, it is important for us to be context-sensitive and culturally aware when synthesizing, interpreting and presenting both the results of this review and the methods used to gather the data.

We would like to first acknowledge the natural challenges and limitations of cross- language, cross-cultural knowledge syntheses, where standards and methods employed in this study were essentially Western and mostly English-based, and that sometimes these methods might not have been the best approach to gather useful evidence in a non- Western, non-English context. During the review process, we found, when compared to English peer-reviewed articles, that the journal publications in the Chinese databases followed wide variations of publishing standards and have somewhat different publication focuses. Chinese publications tend to be succinct in the methodology section and elaborative about policy or program implications, with an emphasis on status report and actions taken or recommendations for future steps. This utility-driven approach might be useful in offering actionable information to practitioners on the ground in the context of China, but presents a challenge when attempting to assess the evidence of these studies. In fact, the lack of sufficient detail in methodology sections had made it impossible to look to the published Chinese literature reviews in the areas of Emergency Risk Communications, or Health Communication in general, for guidance in the development of the search strategy for the Chinese ERC literature review. To overcome this gap, we cross-referenced the available Chinese and English literature reviews, obtained inputs from the WHO China ERC experts, and identified a short list of key terms that were eventually used to build the

152 search strategy for the Chinese literature. Additionally, applicability of the quality assessment tools can sometimes be a challenge. For example, with the emergence of new media and communication tools, analyses of social media content, usage data and social networks are often employed, which speaks to the need of new evaluation methods and assessment tools to keep up with the vibrant field of emergency risk communications.

We faced similar limitations in the Spanish and Portuguese literature. In particular, we found a lack of empirical literature in these two language databases on Emergency Risk Communications. Within these fields, risk communications in Emergency Preparedness is mainly addressed by the Social Health, Technology and Communication, and Human Sciences fields; very little of this work is produced by researchers or practitioners working in public health, in addition, differences in organizational response structure, especially in Latin America, contribute to this overall finding.

Of the empirical literature we did identified, the preponderance of study designs were cross-sectional with use of surveys and interviews, as well as a limited number of case studies written with non-standardized approaches.

153

CONCLUSION

The literature lacks of empirical studies in ERC that directly address the questions posed by WHO. The nature of the studies identified for Q1, Q2 and Q6 is very context specific and refers to mechanisms, practices from the field and recommendations that can be derived from planning or response efforts implemented at the national or local level in various countries but do not provide direct evidence of their transferability to other contexts. Factors that seem to be related to the integration of ERC functions in national and international PHEP planning and response activities include renovation of components of the leadership structure when needed, modification of organizational factors and release of restrictions that may be an obstacle to timely release of information, and amendments to laws and regulations when feasible. Exercises and trainings can be conducted to identify barriers and successes in the integration of ERC functions into national and international public health emergency preparedness planning and response. Key elements to enhance information sharing and coordination across organizations include the creation of networks, task-forces and committees across disciplines, organizations and geographic areas. The functionality of information systems is also a key element for the sharing of information by tailoring information systems to the needs of the users. Engagement of local stakeholders is also important to guarantee flow of information up and down the incident command system.

In response to Q8 the literature provides 45 empirical studies in which specific data gathering tools were applied to assess how the population received, processed and acted upon the risk-communication messages during an emergency. The findings of the individual studies are not relevant to Q8 because they focus on how the population received the message or what opinions they had in a given emergency/situation. However, from such studies we can derive a list of data gathering methods that can be used to collect such information such as population surveys, opinion polls, focus groups, interviews, etc. Such studies do not provide evidence that the results from the data collection were used to improve or change the direction of ERC efforts.

The authors believe that the lack of empirical studies across all questions solicited by WHO demonstrate an overall need of research in these areas. However, an accurate identification of research gaps should be achieved by integrating the results of this review with interviews and focus groups conducted with ERC practitioners to better understand what type of evidence is needed in practice across the multitude of ERC functions. Such approach would guarantee that research is produced in the topic areas of greatest need to practice.

154

155

APPENDIX A. WHO SPICE QUESTIONS

Revised Question 1 [Health Systems: Leadership/ Governance]

Q1) How can emergency risk communication best be integrated into national and international public health emergency preparedness planning and response activities?  Setting: In the context of preparing for and responding to national and international events/emergencies with public health implications in high/ middle/ low-income countries and fragile states.  Perspective: National governments, designated national/ international agencies  Phenomena of Interest: Integration of emergency risk communication functions into national government/ international leadership for response to emergency events with public health

 Comparison: Varied placement of emergency risk communication functions in national leadership structure, organizational proximity of emergency risk communication practitioners to national health response leadership, distribution of emergency risk communication representation across national leadership sectors. Variations in how functions are integrated into national government leadership related to equity considerations such as local contextual and population characteristics.  Evaluation: Impact on public trust in event/emergency related information, on reach of information to affected populations, on coordination of information sharing, on ability of practitioners to influence decisions, on speed of information reaching the public during an event/emergency.  Time Scope: January 1, 2003 to February (7-9), 2016

Revised Question 2 [Health Systems: Information Systems]

Q2) What are the best mechanism(s) to establish effective intra-agency, inter-agency, and/or cross- jurisdictional (such as cross-border; national with sub-national jurisdictions, etc.) information sharing for emergency risk communication?  Setting: In the context of preparing for and responding to national and international events/emergencies with public health implications in high/ middle/ low-income countries and fragile states.  Perspective: National governments, designated national agencies and relevant sub-national authorities (e.g. district and local health departments), responding and implementing partners.  Phenomena of Interest: Establishment of formal, cross jurisdictional, inter-agency, and intra- agency communication linkages and mechanisms for information sharing and communication coordination; barriers and facilitators for such mechanisms and linkages.  Comparison: Varied approaches: networks, bureaucratic protocols and standard operating procedures, incident command models, other approaches to organize and facilitate connections and coordination. Variations in barriers/facilitators, linkages and coordination mechanisms related to equity considerations such as local contextual and population characteristics.

156

 Evaluation: Impact on speed and coverage of public information; release, consistency of hared information; impact on trust of hared information  Time Scope: January 1, 2003 to February (7-9), 2016

Revised Question 6 [ERC Practice: Coordination]

Q6) What are the best practices and protocols to ensure coordination of risk communication activities between responding agencies across organizations and levels of response?  Setting: In the context of preparing for and responding to national and international events/emergencies with public health implications in high/ middle/ low-income countries and fragile states.  Perspective: National governments, designated national agencies and relevant sub-national authorities (e.g. district and local health departments), responding and implementing partners  Phenomena of Interest: Mechanisms and systems for coordination of risk communication activities between national government, subnational health authorities and other responding  Comparison: Networks, standard operating procedures/joint planning, organizational structures, joint operating centers, emergency operating centers, collaborative leadership, team-building skills. Variations in coordination activities, systems, and mechanisms related to equity considerations such as local contextual and population  Evaluation: Impact on message consistency and timing, reductions in redundant action, leveraged human and financial resources.  Time Scope: January 1, 2003 to February (7-9), 2016

Revised Question 8 [ERC Practice: Analysis]

Q8) What are the best practices and most appropriate tools for gathering, analyzing, and interpreting emergency risk communication data and feedback and integrating results into emergency risk communication planning, strategy development, execution and evaluation?  Setting: In the context of preparing for and responding to national and international events/emergencies with public health implications in high/ middle/ low-income countries and fragile states.  Perspective: National governments, designated national agencies and relevant sub-national authorities (e.g. district and local health departments), responding and implementing partners, at-risk communities, stakeholders  Phenomena of Interest: Best practices for data collection, analysis, and using evidence to inform and evaluate emergency risk communication activities *  Comparison: Between types of data and data gathering, community and audience feedback, message testing, public surveys, news and social media monitoring, rapid behavioral assessments, focus groups, key informant interviews, etc. Variations in methods and systems related to equity considerations such as local contextual and populations characteristics.  Evaluation: Impact on level of integration of results into emergency risk communication plans, strategies, and tactics; speed in modifying emergency risk communication messages and materials based on results; perceptions among concerned communities that health authorities are "listening"; speed of addressing rumors, impact on public trust in health protection information, perception among at-risk communities and stakeholders about relevance of response  Time Scope: January 1, 2003 to February (7-9), 2016

157

*Note: This includes understanding how the impact of the message is monitored, how it is received by the public, and also how institutions modify their approach based on analysis of this info. This includes data gathering efforts such as monitoring media, rumors, social media, and social surveys and how the information is acted upon at the institutional level.

158

APPENDIX B. QUALITY APPRAISAL & AMSTAR SCORES

AMSTAR Score Author Pub Year Title (Total=11)

Demographic and attitudinal Bish 2010 determinants of protective 4 behaviours during a pandemic: A review The effectiveness of disaster risk Bradley 2014 communication: A systematic 6 review of intervention studies Communication about Fitzpatrick-Lewis 2010 environmental health risks: A 6 systematic review What have we learned about Lin 2014 communication inequalities 5 during the H1N1 pandemic: a systematic review of the literature Terrorism Communication: Ruggiero 2013 Characteristics and Emerging 1 Perspectives in the Scientific Literature 2002–2011

159

APPENDIX C. CHINESE LITERATURE REFERENCE LIST

1. Chang, 2013. 张四明王瑞夆. 台湾重大灾害灾情查通报系统运作之经验分析 [Operations of the Emergency Information Management System in Taiwan]. 复旦公共行政评论 [Public Management Review], 2013 (02). 2. Chen, 2013. 陈靓, 汪青云. 突发事件下的政务微博角色探析. The role of Government MicroBlogs during emergencies. 东南传播杂志 [Mass Media], 2013 (8) 3. Chen, 2014. 陈然, 谢薇. 突发事件中政务微博舆情应对的问题及对策 [Strategies of governmental microblogs' response to emergencies]. 今传媒 [Contemporary Mass Media], 2014(10): 16-17. 4. Hao, 2009. 郝爱华, 李海麟, 袁俊, 蔡文峰, 沈纪川, 景钦隆, 王鸣. 广州市疾病预防控制人员应急风险沟通需求调查 [ERC capacity and needs assessment for Guangzhou City CDC staff]. 预防医学情报杂志[Journal of Preventive Medicine] 2009 (05): 420-422. 5. Jiang, 2012. 麻疹疫苗强化免疫相关舆情监测分析 [Use of 12320 to monitor the public response on measle vaccination campaign]. 中国公共卫生 [Chinese Journal of Public Health], 2012(03). 6. Lei, 2015. 雷丽萍. Lei Liping. 高速公路突发事件组织间应急信息沟通实证研究 Empirical study on emergency response information sharing during a highway crisis. 管理评论 [Management Review]. 2015, Vol. 27, Issue (9): 213-220. 7. Liu, 2013. 刘国恒, 李灵辉, 郭汝宁, 代吉亚, 吴发好, 宋铁. 人感染 H7N9 禽流感网络谣言监测与防控策略探讨 [Monitor, prevention and control online rumors about people becoming infected with H7N9 virus.]. 华南预防医学 [South China Journal of Preventive Medicine], 2013(05). 8. Ma, 2010. 马昱, 钱玲, 佟丽, 毛群安. [MA Yu, QIAN Ling, TONG Li, MAO Qun-an] 风险沟通在我国应对甲型 H1N1 流感中的运用 [Application of risk communication in the response to the Influenza A/H1N1 outbreaks in China]. 中国健康教育 [Chinese Journal of Health Education], 2010(01). 9. 人民网舆情监测室与微博数据中心 [The People's Public Opinion Monitoring Office and Weibo Data Center], 2012-2016. 政务微博报告/政务指数微博影响力报告 Government Weibo Research Report/Government Weibo Impact Index, 2012-2016. 10. Shao, 2014. 邵安娜, 徼晓菲, 赵雯, 解瑞谦, 常慧, 杨小宝, 欧剑鸣, 肖[王乐], 张莹. 风险沟通在线培训工具包本土化开发应用效果的评价 [Localized application of a risk communication online training toolkits] 中国健康教育 [Chinese Journal of Health Education]. 2014, vol 30, (4): 340-343. 11. Song, 2016. 宋铁, 刘国恒, 金连梅, 涂文校, 方艳, 易建荣, 代吉亚. 中国疾控机构媒体监测能力建设现状 [Capacity building for media monitoring for China CDC]. 中国公共卫生 [Chinese Journal of Public Health] 2016 (6). 12. Wang, 2010. 王超, 谭枫. 12320 在甲型 H1N1 流感中风险沟通与舆情监测作用 Use of 12320, an official public health hotline, for ERC and public opinions monitoring during the A(H1N1) pandemic] 中国公共卫生 [Chinese Journal of Public Health], 2010(11).

160

13. Xie, 2011. 解瑞谦, 唐雪峰, 欧剑鸣, 阚坚力. 突发公共卫生事件风险沟通中准备工作内容的研究 Integrating risk communication preparedness framework into public health emergency preparedness. 中国健康教育 [Chinese Journal of Health Education]. 2011, vol 27, (08): 607-609. 14. Zeng 2015. 曾繁旭, 钱琪瑶. 传播链条、社会网络与公众回应社会化媒体时代的风险沟通效果研究. 新闻与写作 [News and Journalism], 2015(06). 15. Zhang, 2011. 张巍, 张继彬, 陈国永, 张刚, 李杰, 钱玲, 毛群安. 县域突发公共卫生事件大众风险沟通现状研究 Assessment of county-level public health emergencies risk communication capacity]. 医学与社会 [Medicine and Society]. 2011, vol 24, (11): 30-33. 16. Zhang XE, 2015. Emma Xuxiao Zhang, Yinping Yang, Richard Di Shang, Joseph John Pyne Simons, Boon Kiat Quek, Xiao Feng Yin, Wanhan See,a Olivia Seen Huey Oh, Khine Sein Tun Nandar, Vivienne Ruo Yun Ling, Pei Pei Chan, Zhaoxia Wang, Rick Siow Mong Goh, Lyn Jamesa and Jeannie Su Hui Tey. 利用社交網站開展疾病監測與公眾感知 Leveraging social networking sites for disease surveillance and public sensing: the case of the 2013 avian influenza A(H7N9) outbreak in China. WPSAR Vol 6, No 2, 2015 | doi: 10.5365/wpsar.2015.6.1.013. 17. Zhang X, 2015. 张晓华, 侯芊, 王卉, 蒋燕, 苏潇歌, 王东皓, 崔颖. [ZHANG Xiao-hua, HOU Qian, WANG Hui, et al. ] 人感染 H7N9 禽流感政府防控措施微博民意调查. [Public opinions on governmental measures for H7N9 avian influenza control and prevention:a survey through microblog]. 中国公共卫生, Chinese Journal of Public Health, 2015, 31(3): 296-298. 18. Zhou, 2015. 周莉, 李晓, 黄娟. 政务微博在突发事件中的信息发布及其影响 [Government Weibo Information dissemination and its impact during emergencies]. 新闻大学 [Journalism Bimonthly], 2015(02).

161

APPENDIX D. CHINESE LITERATURE SUMMARY

Reference Brief Summary 1. Chang, 2013. 张四明王 This study assesses the Taiwanese emergency information sharing 瑞夆. 台湾重大灾害灾 system's response to a severe Typhoon. In Taiwan, the disaster 情查通报系统运作之经 situational information gathering and dissemination is performed by 验分析 [Operations of the Fire Department, the Police Department, and the Department of the Emergency Civil Affairs, as well as being supplemented by an emergency Information management information system. These official systems carry out Management System in cross-institution information sharing. In order to learn from the Taiwan]. 复旦公共行政 response to Typhoon Morakot, which hit Taiwan in the summer of 评论 [Public 2009, causing the loss of over 600 lives and substantial property Management Review], damage, the authors investigate the effectiveness of the official 2013 (02). disaster information collection, the communications mechanism and operations. A survey was conducted of 159 first responders from the fire, police and civil affairs departments working at the local level in the affected area, with a focus on the disaster information data collection process, communication channels for emergency public information and warning, and their experience with the new "Emergency Management Information System"(EMIS) for disaster information sharing, management, and coordination across agencies.

2. Chen, 2013. 陈靓, 汪青 Using the response to the Ya’An earthquake as a case study, Chen et al 云. 突发事件下的政务 (2013) assessed the current role the Chinese Government Microblog, 微博角色探析. The role Weibo, plays during an emergency as a leader of public opinion, of Government disseminator of risk information and crisis updates, and monitor of MicroBlogs during government’s response. emergencies. 东南传播杂志 [Mass Media], 2013 (8)

3. Chen, 2014. 陈然, 谢薇. Chen et al (2014) analyzed the Yunnan Governmental Microblog, 突发事件中政务微博舆 Weibo, in Yunnan’s response to a terrorist attack on the Kunming Train 情应对的问题及对策 Station and explores the use and integration of governmental [Strategies of microblogs into routine public communication and emergency risk governmental communications. microblogs' response to emergencies]. 今传媒 [Contemporary Mass Media], 2014(10): 16-17.

4. Hao, 2009. 郝爱华, 李海 Hao et al (2009) of the Guangzhou City CDC Office assessed emergency 麟, 袁俊, 蔡文峰, 沈纪 risk communications training needs among local public health workers 川, 景钦隆, 王鸣. 广州 and the need to develop ERC capacity building curriculum and 市疾病预防控制人员应 materials. 急风险沟通需求调查

162

Reference Brief Summary [ERC capacity and needs assessment for Guangzhou City CDC staff]. 预防医学情报杂志[Journal of Preventive Medicine] 2009 (05): 420-422.

5. Jiang, 2012. 麻疹疫苗强 Jiang et al (2012) of the Chinese CDC described the role of 12320, a 化免疫相关舆情监测分 direct two-way communication channel between the government and 析 [Use of 12320 to the public, as a trusted channel of communication by the public during monitor the public the measles campaign. 12320 offered health consultation directly to response on measle the public, acted as an important emergency risk communications vaccination campaign]. agent and had taken on the role of gathering public reaction data. Data 中国公共卫生 [Chinese gathered through 12320 has been highly valued by the China MoH and Journal of Public Health], helped shape their ERC strategies. 2012(03).

6. Lei, 2015. 雷丽萍. Lei This study uses data from semi-structured interviews with responding Liping. 高速公路突发事 agencies, a review of official data, reports and news coverage to 件组织间应急信息沟通 investigate the coordination and information sharing mechanism 实证研究 Empirical among government agencies at different levels in the response to an study on emergency explosion on a major highway in 2009. This study identified key response information recommendations to improve the response and management of sharing during a highway highway emergencies: 1) involve both governmental agencies and crisis. 管理评论 NGOs in the information sharing and coordination mechanism, 2) [Management Review]. develop an evidence-based, scenario-based contingency plan, including 2015, Vol. 27, Issue (9): communication plans involving the respective responding agencies and 213-220. case studies on various types of scenarios, including highway emergencies, 3) Develop communication plans that are scenario-based and include details of how agencies not involved in daily highway operations can coordinate and share information with operations agencies, 4) include communications with lay people, who are unfamiliar with highway operations or emergency response, in capacity building training for responders, 5) build a platform for emergency information sharing and offer technical support for various parties involved, 6) ensure the burden of information processing is manageable, balanced, and within the capacity of agency that handles it, 7) incorporate mandatory time off into personnel timetables to improve sustainability and stability. 7. Liu, 2013. 刘国恒, 李灵 Using a public health public opinions monitoring platform developed by 辉, 郭汝宁, 代吉亚, 吴 the Guangdong CDC, Liu et al (2013) of the Guangdong CDC Office 发好, 宋铁. 人感染 analyzed rumors regarding H7N9 outbreaks in 2010 and identified H7N9 禽流感网络谣言 strategies to prevent, manage, and counter harmful rumors on social 监测与防控策略探讨 media. [Monitor, prevention

163

Reference Brief Summary and control online rumors about people becoming infected with H7N9 virus.]. 华南预防医学 [South China Journal of Preventive Medicine], 2013(05).

8. Ma, 2010. 马昱, 钱玲, 佟 Ma et al (2010), of China Health Education under the China Ministry of 丽, 毛群安. [MA Yu, Health (MoH) Public Information Office, described the application of QIAN Ling, TONG Li, emergency risk communications during the A(H1N1) pandemic in China MAO Qun-an] 风险沟通 and emphasized the importance of integrating emergency risk 在我国应对甲型 H1N1 communications in the preparedness and response to a disease 流感中的运用 outbreak through the development of communication guidelines and [Application of risk protocols, timely updates on disease characteristics and prevention communication in the methods, health promotion and public opinions monitoring, and response to the maintaining communication channels for further risk information and Influenza A/H1N1 situational updates. outbreaks in China]. 中国健康教育 [Chinese Journal of Health Education], 2010(01).

9. 人民网舆情监测室与微 The Chinese Government Weibos have been widely integrated into the 博数据中心 [The public communication system. Collaborating with Weibo service People's Public Opinion providers such as Sina and Tencent, the People’s Public Opinion Monitoring Office and Monitoring Office (2012-2016) has routinely gathered data and Weibo Data Center], conducts performance evaluations on tens of thousands of Chinese 2012-2016. 政务微博报 government Weibos [Twitter]. The People’s Public Opinion Monitoring 告/政务指数微博影响 Office and Weibo Data Center analyzes the top 100 influential 力报告 Government government Weibos and identify lessons learned for improvement of Weibo Research the use of government Weibos as a two-way communication tool Report/Government during regular times and in crises. Working with Weibo service Weibo Impact Index, providers, such as Sina and Tencent, the People's Public Opinion 2012-2016. Monitoring Office published Government Weibo Impact Index and Government Weibo Research reports, as well as developing training on effective communication strategies and identifying best practices. It also works with different levels of government agencies to promote the standardization and science of government performance evaluation on public services and communications by publishing and updating operational guidelines and protocols for Government Weibos. These efforts aim to empower this new media to play the role of facilitating open communication and improving government services. 10. Shao, 2014. 邵安娜, 徼 Shao et al (2014), of China Health Education under the China Ministry 晓菲, 赵雯, 解瑞谦, 常 of Health (MoH) Public Information Office, described the application 慧, 杨小宝, 欧剑鸣, 肖[ and appropriateness of a risk communications online training toolkit,

164

Reference Brief Summary 王乐], 张莹. 风险沟通在 CDCynergy - adapted from a US CDC toolkit, for Chinese 线培训工具包本土化开 healthcare/public health workers. 发应用效果的评价 [Localized application of a risk communication online training toolkits] 中国健康教育 [Chinese Journal of Health Education]. 2014, vol 30, (4): 340-343.

11. Song, 2016. 宋铁, 刘国 Song et al (2016) of the Guangdong Provincial CDC Office and China 恒, 金连梅, 涂文校, 方 CDC assessed the current needs for media monitoring capacity at China 艳, 易建荣, 代吉亚. 中 CDC at all levels, and identified possible solutions to meet the needs. 国疾控机构媒体监测能力建设现状 [Capacity building for media monitoring for China CDC]. 中国公共卫生 [Chinese Journal of Public Health] 2016 (6).

12. Wang, 2010. 王超, 谭枫. Wang et al (2010) of the China CDC described how 12320, the only 12320 在甲型 H1N1 流 official health hotline in China, has integrated into the emergency 感中风险沟通与舆情监 response system and was used as a two-way communication channel 测作用 Use of 12320, an that is trusted by the public during the 2008 formula contamination official public health crisis, 2008 Beijing Olympics and 2009 A(H1N1) pandemic. hotline, for ERC and public opinions monitoring during the A(H1N1) pandemic] 中国公共卫生 [Chinese Journal of Public Health], 2010(11).

13. Xie, 2011. 解瑞谦, 唐雪 As part of the national effort to integrate risk communications 峰, 欧剑鸣, 阚坚力. 突 preparedness frameworks into public health emergency preparedness, 发公共卫生事件风险沟 Xie et al (2011) of China Health Education under the China Ministry of 通中准备工作内容的研 Health (MoH) Public Information Office described the work conducted 究 Integrating risk to develop and improve the Public Health Emergency and Risk communication Communications Handbook covering five major areas: coordination, preparedness framework training and exercises, information sharing, communication tools and into public health training materials, and risk information gathering and management. emergency preparedness. 中国健康

165

Reference Brief Summary 教育 [Chinese Journal of Health Education]. 2011, vol 27, (08): 607-609. 14. Zeng 2015. 曾繁旭, 钱琪 Employing network analysis, Zeng et al (2015) assessed the 瑶. 传播链条、社会网 effectiveness of local governments’ emergency risk communications 络与公众回应社会化媒 responses to the 2014 Maoming, Guangdong PX incident using official 体时代的风险沟通效果 microblog, Weibo. By analyzing the content of the Weibo posts, Zeng et 研究. 新闻与写作 [News al (2015) evaluated how audience received, interpreted, and reacted to and Journalism], such communication efforts. 2015(06). 15. Zhang, 2011. 张巍, 张继 Through in-depth interviews with the spokespersons at the local level 彬, 陈国永, 张刚, 李杰, and focus groups with staff from various government agencies and the 钱玲, 毛群安. 县域突发 public, Zhang et al (2011) of China Health Education under the China 公共卫生事件大众风险 Ministry of Health (MoH) Public Information Office, assessed county- 沟通现状研究 level public health emergencies risk communications capacity and Assessment of county- identified solutions to better integrate emergency risk communications level public health into current emergency preparedness plans, including offering emergencies risk emergency risk communications capacity building training to local communication government workers, responders and the public and conducting capacity]. 医学与社会 tabletop and functional exercises. [Medicine and Society]. 2011, vol 24, (11): 30-33. 16. Zhang XE, 2015. Emma During an avian influenza A(H7N9) outbreak in China, Zhang XE et al Xuxiao Zhang, Yinping (2015) assessed the value and the use of Weibo for the surveillance of Yang, Richard Di Shang, disease outbreaks using two sets of data: a line listing of confirmed Joseph John Pyne cases obtained from conventional public health information channels Simons, Boon Kiat Quek, and case information from Weibo posts. Zhang XE et al revealed that Xiao Feng Yin, Wanhan Weibo enabled timely monitoring of other outbreak-relevant See,a Olivia Seen Huey information, provided access to additional crowd-sourced Oh, Khine Sein Tun epidemiological information and was leveraged by the local Nandar, Vivienne Ruo government as an interactive platform for risk communications and Yun Ling, Pei Pei Chan, monitoring public sentiment on the policy response. Weibo can be used Zhaoxia Wang, Rick Siow by public health agencies to enhance traditional communicable disease Mong Goh, Lyn Jamesa surveillance systems for the global surveillance of overseas public and Jeannie Su Hui Tey. health threats, and to calibrate response policies and measures and for 利用社交網站開展疾病 risk communications. 監測與公眾感知 Leveraging social networking sites for disease surveillance and public sensing: the case of the 2013 avian influenza A(H7N9) outbreak in China. WPSAR Vol 6, No 2, 2015 | doi:

166

Reference Brief Summary 10.5365/wpsar.2015.6.1. 013. 17. Zhang X, 2015. 张晓华, Aided by the usage data of 12320, a Chinese official public health 侯芊, 王卉, 蒋燕, 苏潇 hotline, Zhang X et al (2015) of China CDC reviewed the role 12320 歌, 王东皓, 崔颖. plays in emergency risk communications with the public during a crisis [ZHANG Xiao-hua, HOU by examining public opinions on government's prevention and control Qian, WANG Hui, et al. ] methods of H7N9, public satisfaction with information received and 人感染 H7N9 禽流感政 public confidence in the governments’ response. 府防控措施微博民意调查. [Public opinions on governmental measures for H7N9 avian influenza control and prevention:a survey through microblog]. 中国公共卫生, Chinese Journal of Public Health, 2015, 31(3): 296-298. 18. Zhou, 2015. 周莉, 李晓, Zhou et al (2015) assess the role of Government Weibo as a platform 黄娟. 政务微博在突发 for emergency risk communications and for managing public opinions 事件中的信息发布及其 during emergencies. 影响 [Government Weibo Information dissemination and its impact during emergencies]. 新闻大学 [Journalism Bimonthly], 2015(02).

167

APPENDIX E. PORTUGUESE LITERATURE REFERENCE LIST

1. Thiago Reginaldo T. Pacheco DC, Baldessar MJ, Benciveni Franzoni AM, 2013. Redução de riscos de desastres na prática: a participação da Defesa Civil e do município em plataformas digitais de redes sociais Democracia digital e governo eletronico, Capa > v. 2, n. 9 (2013)

168

APPENDIX F. PORTUGUESE LITERATURE SUMMARY

Reference Brief Summary 1. Thiago Reginaldo T. Pacheco DC, Baldessar MJ, Thiago et al. (2013) describe the use of social networks, Benciveni Franzoni AM, 2013. Redução de like Twitter and Facebook (FB), by the Civil Defense (CD) riscos de desastres na prática: a participação of New Freiburg, Rio de Janeiro, Brazil, as a da Defesa Civil e do município em plataformas communication channel for alerting and preparing the digitais de redes sociais Democracia digital e population to face a natural disaster. This study aimed governo eletronico, Capa > v. 2, n. 9 (2013) to explore the possible benefits of expanding digital platforms, including web pages, social networks, mobile phone SMS, research engines, video games and digital TV programs, towards a larger community audience, in order to spread messages about the correct behaviors in facing different phases of disasters. Through the analysis of Twitter and Facebook use by CD to communicate with affected population during a specific event, like the flood occurred in 2011 in New Freiburg. This study examined the period from 2011-13, so as to track digital activity since the creation of specific CD pages in social networks. Content analysis of messages provided positive feedback on enrollment and mobilization of authorities after the event. Following the flood, during the reconstructive phase, the CD and municipality of New Freiburg launched and promoted several public events through social networks, in order to gather, inform, prepare and reassure both the population already affected by the flood and those still possibly at risk. The authors conclude that in a country like Brazil, where network coverage is broad and accessible, it is worth considering the potential for social networks to act as a new institutional channel of communication for health related topics.

169

APPENDIX G. ENGLISH LITERATURE REFERENCE LIST

1. Aburto, N. J., E. Pevzner, et al. (2010). "Knowledge and Adoption of Community Mitigation Efforts in Mexico During the 2009 H1N1 Pandemic." American Journal of Preventive Medicine 39(5): 395-402. 2. Ardalan, A., K. Holakouie Naieni, et al. (2009). "Evaluation of Golestan Province's Early Warning System for flash floods, Iran, 2006-7." Int J Biometeorol 53(3): 247-254. 3. Baseman, J. G., D. Revere, et al. (2013) "Public health communications and alert fatigue." BMC health services research, 295 DOI: 10.1186/1472-6963-13-295. 4. Becker, S. M. (2004). "Emergency communication and information issues in terrorist events involving radioactive materials." Biosecur Bioterror 2(3): 195-207. 5. Bharosa, N., J. Lee, et al. (2010). "Challenges and obstacles in sharing and coordinating information during multi-agency disaster response: Propositions from field exercises." Information Systems Frontiers 12(1): 49-65. 6. Bharosa, N., J. Lee, et al. (2009). A case study of information flows in multi-agency emergency response exercises. Proceedings of the 10th Annual International Conference on Digital Government Research: Social Networks: Making Connections between Citizens, Data and Government. Puebla, Mexico, Digital Government Society of North America: 277-282. 7. Blendon, R. J., J. M. Benson, et al. (2003). "Using opinion surveys to track the public's response to a bioterrorist attack." J Health Commun 8 Suppl 1: 83-92; discussion 148-151. 8. Boyd, C. A., J. A. Gazmararian, et al. (2013). "Knowledge, attitudes, and behaviors of low-income women considered high priority for receiving the novel influenza A (H1N1) vaccine." Matern Child Health J 17(5): 852-861. 9. Burns, R., P. Robinson, et al. (2010). "From hypothetical scenario to tragic reality: A salutary lesson in risk communication and the Victorian 2009 bushfires." Aust N Z J Public Health 34(1): 24-31. 10. Celik, S. and S. Corbacioglu (2010). "Role of information in collective action in dynamic disaster environments." Disasters 34(1): 137-154. 11. Chambers, J., K. Barker, et al. (2012). "Reflections on the UK's approach to the 2009 swine flu pandemic: conflicts between national government and the local management of the public health response." Health Place 18(4): 737-745. 12. Chess, C. and L. Clarke (2007). "Facilitation of risk communication during the anthrax attacks of 2001: the organizational backstory." Am J Public Health 97(9): 1578-1583. 13. Clarke, C. E. and C. Chess (2006). "False alarms, real challenges--one university's communication response to the 2001 anthrax crisis." Biosecur Bioterror 4(1): 74-83. 14. Cole, J. M. and B. L. Murphy (2014). "Rural hazard risk communication and public education: Strategic and tactical best practices." International Journal of Disaster Risk Reduction 10: 292- 304. 15. Cope, J. R., M. Frost, et al. (2014). "Assessing knowledge and application of emergency risk communication principles among public health workers in China." Disaster Med Public Health Prep 8(3): 199-205. 16. Driedger, S. M., E. Cooper, et al. (2013). "Communicating risk to aboriginal peoples: first nations and Metis responses to H1N1 risk messages." PLoS One 8(8): e71106. 17. Eisenman, D. P., D. Glik, et al. (2009) "Improving Latino disaster preparedness using social networks." American journal of preventive medicine, 512-517 DOI: 10.1016/j.amepre.2009.07.022.

170

18. Ferrante, G., S. Baldissera, et al. (2011). "Surveillance of perceptions, knowledge, attitudes and behaviors of the Italian adult population (18-69 years) during the 2009-2010 A/H1N1 influenza pandemic." Eur J Epidemiol 26(3): 211-219. 19. Galarce, E. M. and K. Viswanath (2012). "Crisis communication: an inequalities perspective on the 2010 Boston water crisis." Disaster Med Public Health Prep 6(4): 349-356. 20. Gesser-Edelsburg, A., Y. Shir-Raz, et al. (2015). "What does the public know about Ebola? The public's risk perceptions regarding the current Ebola outbreak in an as-yet unaffected country." Am J Infect Control 43(7): 669-675. 21. Glik, D. C., D. P. Eisenman, et al. (2014) "Using the Precaution Adoption Process model to describe a disaster preparedness intervention among low-income Latinos." Health education research, 272-283 DOI: 10.1093/her/cyt109. 22. Gresham, L., A. Ramlawi, et al. (2009). "Trust across borders: responding to 2009 H1N1 influenza in the Middle East." Biosecur Bioterror 7(4): 399-404. 23. Gultom, D. I. and Z. Joyce (2014). Crisis communication capacity for disaster resilience: community participation of information providing and verifying in Indonesian volcanic eruption, Australian and New Zealand Third Sector Research. 24. Hanvoravongchai, P., W. Adisasmito, et al. (2010). "Pandemic influenza preparedness and health systems challenges in Asia: results from rapid analyses in 6 Asian countries." BMC Public Health 10: 322. 25. Henderson, J. N., L. C. Henderson, et al. (2004). "Chemical (VX) terrorist threat: public knowledge, attitudes, and responses." Biosecur Bioterror 2(3): 224-228. 26. Holmes, B. J., N. Henrich, et al. (2009). "Communicating with the public during health crises: Experts' experiences and opinions." Journal of Risk Research 12(6): 793-807. 27. Howard, A. F., H. M. Bush, et al. (2012). "Characteristics of Kentucky local health departments that influence public health communication during times of crisis: information dissemination associated with H1N1 novel influenza." J Public Health Manag Pract 18(2): 169-174. 28. Ikeda, S. and T. Nagasaka (2011). "Emergent Framework of Disaster Risk Governance towards Innovating Coping Capability for Reducing Disaster Risks in Local Communities." International Journal of Disaster Risk Science 2(2): 1-9. 29. Ipe, M., T. S. Raghu, et al. (2010). "Information intermediaries for emergency preparedness and response: A case study from public health." Information Systems Frontiers 12(1): 67-79. 30. Jehn, M., Y. Kim, et al. (2011). "Community knowledge, risk perception, and preparedness for the 2009 influenza A/H1N1 pandemic." J Public Health Manag Pract 17(5): 431-438. 31. Kapucu N. (2006). "Interagency Communication Networks During Emergencies Boundary Spanners in Multiagency Coordination." The American Review of Public Administration 36(2): 207-225. 32. Kim, N. H., T. J. Cho, et al. (2015). "Implications for effective food risk communication following the Fukushima nuclear accident based on a consumer survey." Food Control 50: 304-312. 33. Kuppuswamy, S. (2014). "A Perception Study on Community Response to Media Technologies for Cyclone Warning and Disaster Communication in South India." Science, Technology & Society 19(3): 399-414. 34. Lam, P. Y. (2008). "Avian influenza and pandemic influenza preparedness in Hong Kong." Ann Acad Med Singapore 37(6): 489-496. 35. Leonard, G. S., C. Stewart, et al. (2014). "Integrating multidisciplinary science, modelling and impact data into evolving, syn-event volcanic hazard mapping and communication: A case study from the 2012 Tongariro eruption crisis, New Zealand." Journal of Volcanology and Geothermal Research 286: 208-232. 36. Massey, P. D., A. Miller, et al. (2011). "Australian Aboriginal and Torres Strait Islander communities and the development of pandemic influenza containment strategies: community voices and community control." Health Policy 103(2-3): 184-190.

171

37. Maxwell, T. A. (2003). "The public need to know: emergencies, government organizations, and public information policies." Government Information Quarterly 20(3): 233. 38. McGough, M., L. L. Frank, et al. (2005). "Communicating the risks of bioterrorism and other emergencies in a diverse society: a case study of special populations in North Dakota." Biosecur Bioterror 3(3): 235-245. 39. Mebane, F., S. Temin, et al. (2003). "Communicating anthrax in 2001: a comparison of CDC information and print media accounts." J Health Commun 8 Suppl 1: 50-82; discussion 148-151. 40. Militello, L. G., E. S. Patterson, et al. (2007). "Information flow during crisis management: challenges to coordination in the emergency operations center." Cognition, Technology & Work 9(1): 25-31. 41. Moore, M. and D. J. Dausey (2011). "Response to the 2009-H1N1 influenza pandemic in the Mekong Basin: surveys of country health leaders." BMC Res Notes 4: 361. 42. Mulyasari, F. and R. Shaw (2013). "Role of women as risk communicators to enhance disaster resilience of Bandung, Indonesia." Natural Hazards 69(3): 2137-2160. 43. Murphy, M. W., S. Iqbal, et al. (2010). "Postdisaster Health Communication and Information Sources: The Iowa Flood Scenario." Disaster Medicine and Public Health Preparedness 4(2): 129-134. 44. Nowell, B. and T. Steelman (2015). "Communication under Fire: The Role of Embeddedness in the Emergence and Efficacy of Disaster Response Communication Networks." Journal of Public Administration Research and Theory 25(3): 929-952. 45. Odlum, M. and S. Yoon (2015). "What can we learn about the Ebola outbreak from tweets?" Am J Infect Control 43(6): 563-571. 46. Ohno, K. and K. Endo (2015). "Lessons learned from Fukushima Daiichi nuclear power plant accident: efficient education items of radiation safety for general public." Radiat Prot Dosimetry 165(1-4): 510-512. 47. Pearce, J. M., G. J. Rubin, et al. (2013). "Communicating with the public following radiological terrorism: results from a series of focus groups and national surveys in Britain and Germany." Prehosp Disaster Med 28(2): 110-119. 48. Perko, T., B. van Gorp, et al. (2013). "Communication in nuclear emergency preparedness: a closer look at information reception." Risk Anal 33(11): 1987-2001. 49. Quinn, S. C., T. Thomas, et al. (2005). "Postal workers' perspectives on communication during the anthrax attack." Biosecur Bioterror 3(3): 207-215. 50. Rogers, M. B., R. Amlot, et al. (2013). "The impact of communication materials on public responses to a radiological dispersal device (RDD) attack." Biosecur Bioterror 11(1): 49-58. 51. Rousseau, C., N. Moreau, et al. (2015). "Public media communications about H1N1, risk perceptions and immunization behaviours: A Quebec-France comparison." Public Underst Sci 24(2): 225-240. 52. Rubin, G., H. Potts, et al. (2010) "The impact of communications about swine flu (influenza A H1N1v) on public responses to the outbreak: results from 36 national telephone surveys in the UK (Structured abstract)." Health Technology Assessment Database, 183. 53. Rubin, G. J., R. Amlot, et al. (2011). "The London polonium incident: lessons in risk communications." Health Phys 101(5): 545-550. 54. Rubinstein, H., A. Marcu, et al. (2015). "Public preferences for vaccination and antiviral medicines under different pandemic flu outbreak scenarios." BMC Public Health 15: 190. 55. Rundblad, G., O. Knapton, et al. (2010). "Communication, perception and behaviour during a natural disaster involving a 'Do Not Drink' and a subsequent 'Boil Water' notice: a postal questionnaire study." BMC Public Health 10: 12. 56. Savoia, E., M. A. Stoto, et al. (2015). "Public response to the 2014 chemical spill in West Virginia: knowledge, opinions and behaviours." BMC Public Health 15: 790.

172

57. Schraagen, J. M., M. H. i. t. Veld, et al. (2010). "Information Sharing During Crisis Management in Hierarchical vs. Network Teams." Journal of Contingencies and Crisis Management 18(2): 117- 127. 58. Seyedin, S. H. and H. R. Jamali (2011). "Health Information and Communication System for Emergency Management in a Developing Country, Iran." Journal of Medical Systems 35(4): 591- 597. 59. Shepherd, J. and K. van Vuuren (2014). "The Brisbane flood: CALD gatekeepers' risk communication role." Disaster Prevention and Management 23(4): 469-483. 60. Spence, P. R., K. A. Lachlan, et al. (2007). "Crisis Communication, Race, and Natural Disasters." Journal of Black Studies 37(4): 539-554. 61. Staes, C. J., A. Wuthrich, et al. (2011). "Public health communication with frontline clinicians during the first wave of the 2009 influenza pandemic." J Public Health Manag Pract 17(1): 36- 44. 62. Sugerman, D. E., J. M. Keir, et al. (2012). "Emergency health risk communication during the 2007 San Diego wildfires: comprehension, compliance, and recall." J Health Commun 17(6): 698-712. 63. Sutton, J., C. League, et al. (2015). "Terse Messaging and Public Health in the Midst of Natural Disasters: The Case of the Boulder Floods." Health Communication 30(2): 135-143. 64. Tian, X., G. Zhao, et al. (2014). "Health education and promotion at the site of an emergency: experience from the Chinese Wenchuan earthquake response." Glob Health Promot. 65. UNICEF (2014) "Study on Public Knowledge, Attitudes, and Practices Relating to Ebola Virus Disease (EVD) Prevention and Medical Care in Sierra Leone." 66. van der Weerd, W., D. R. Timmermans, et al. (2011). "Monitoring the level of government trust, risk perception and intention of the general public to adopt protective measures during the influenza A (H1N1) pandemic in The Netherlands." BMC Public Health 11: 575. 67. Voeten, H. A., O. de Zwart, et al. (2009). "Sources of information and health beliefs related to SARS and avian influenza among Chinese communities in the United Kingdom and The Netherlands, compared to the general population in these countries." Int J Behav Med 16(1): 49- 57. 68. Walter, D., M. Bohmer, et al. (2012). "Risk perception and information-seeking behaviour during the 2009/10 influenza A(H1N1)pdm09 pandemic in Germany." Euro Surveill 17(13). 69. Wang, C. J., A. A. Little, et al. (2011). "Communication of urgent public health messages to urban populations: lessons from the Massachusetts water main break." Disaster Med Public Health Prep 5(3): 235-241. 70. Woien, G. and K. I. Tonsberg (2009). "Norwegians approve of the health authorities' strategy to communicate worst case pandemic scenarios." Euro Surveill 14(22). 71. Yen, M. Y., T. S. Wu, et al. (2009). "Taipei's use of a multi-channel mass risk communication program to rapidly reverse an epidemic of highly communicable disease." PLoS One 4(11): e7962.

173