The Role of Online Discussion Forums During a Public Health Emergency Jennifer Ann Cole Thesis submitted for the degree of Doctor of Philosophy Royal Holloway University of London 2017 Page 1 of 401 Page 2 of 401 Declaration of Authorship I, Jennifer Ann Cole, hereby declare that this thesis and the work presented in it is entirely my own. Where I have consulted the work of others, this is always clearly stated. Signed: Dated: Page 3 of 401 Page 4 of 401 Abstract During a public health emergency, access to professional healthcare may be constrained. Online discussion forums provide an alternative – access to a ‘Wise Crowd’ from which Collective Intelligence may emerge – but lie outside of traditional quality control structures. This thesis examines whether certain characteristics of such platforms encourage and signpost higher quality information, and what utility this may offer during a Public Health Emergency of International Concern. It synthesises results from three separate studies: interviews with Ebola witnesses during the 2014-16 West Africa outbreak; doctors’ assessments of the quality of information provided in response to questions asked on health discussion forums, including r/ebola on Reddit.com, dedicated to discussing Ebola; and an investigation of online health community moderators by non- participant observation, interviews, and access to private areas of discussion sites not accessible to general users. The research goes beyond previous work in three major ways: First, analysis of health-seeking behaviour presented in this thesis identifies three separate stages of risk perception and individual concern during an outbreak with different types of questions asked in each stage, each requiring different platform characteristics. Second, assessment of the quality of the health advice in major online forums is determined in this study according to the judgements of doctors; this differs Page 5 of 401 from previous research that used indirect assessment. Results suggests that such forums can provide signposts to good quality information. Third, the investigation of the role of teams of volunteer moderators of reddit discussion forums identifies and analyses the task-components and skill-mix required to set up and manage a discussion forum during a serious disease outbreak. This points to a larger role for technical skills and forum experience than has previously been identified. The conclusions of this thesis suggest ways in which online platforms may facilitate better health information exchange during future serious disease outbreaks. Page 6 of 401 Acknowledgements My PhD has been made possible by several individuals and organisations, all of whom deserve full acknowledgement. I have been funded in full by Royal Holloway University of London through a Reid Scholarship in Health, the Human Body and Behaviour, an innovative attempt to promote interdisciplinary research, which in this case involved the School of Biological Sciences, the Department of Computer Science, and the Department of Geography. The ability to draw on the expertise, interests and resources of all three departments has made the study stronger and more far- reaching than could have been achieved from a single disciplinary approach. The most important contribution made by the departments were of course my supervisors – Professor Chris Watkins from the Department of Computer Science along with Professor Dorothea Kleine and Dr Alasdair Pinkerton from the Department of Geography. Each has guided, critiqued and shaped my approach throughout and has helped me to hone and focus the results. I would also like to acknowledge the reddit moderators who not only gave their time to answer my questions, but also provided access to private areas of their forums beyond those accessible to general users, including statistics, data and report logs that proved invaluable to my research. So much of what they do is invisible to the reddit users who benefit from their efforts: they work incredibly hard for little reward. I would also like to acknowledge those people who have had a less direct involvement with my study, but without whom it would not have been possible. The first of these is Elizabeth Jesser Reid (1789-1866). Her legacy Page 7 of 401 provided my scholarship. More importantly, she founded Bedford College, the first higher education institution for women in Great Britain and opened the door to higher education for every woman who came after her. Secondly, I would like to acknowledge Aaron Swartz (1986-2013), the American computer programmer and activist who was an early investor in reddit and who guided its development towards the provision of news and information. A strong campaigner for open access, he took his own life while awaiting trial for allegedly downloading articles from the digital academic library JSTOR with the intention of sharing them on illegal file-sharing websites. Reddit is an incredible free and open information resource that would not be what it is without his vision, which has shaped it far more than his financial investment. Finally, I would like to acknowledge my late in-laws, Michael and Bridget Hand, who died in March 2014 shortly after my study began and whose financial legacy enabled me to switch from part-time to full-time study and, in all probability, to complete the course of study at all. Higher Education is the realm of the financially comfortable, and I would ask anyone who reads this thesis to continue to do whatever they can to ensure access is open to all. The idea that it is at present is a fallacy that should never go unchallenged. Page 8 of 401 CONTENTS I: INTRODUCTION 1.1 Introduction 1.2 Communication during public health emergencies 1.3 The research question 1.4 Justification for the research 1.5 Health information seeking online 1.6 Public Health Emergencies of International Concern 1.7 Ebola Virus Disease: 2014-16 West Africa outbreak 1.8 r/ebola: a case study 1.9 Gaps in the current understanding 1.10 Contribution to knowledge 1.11 Scope and structure of the study 1.12 Limitations of the study 1.13 Conclusions 2: THEORETICAL FRAMEWORK AND LITERATURE REVIEW 2.1 Introduction 2.2 Technology affordances 2.3 Collective Intelligence 2.4 How good is good enough? 2.5 The computer as disruptive technology 2.6 Pierre Lévy’s Theory of Collective Intelligence Page 9 of 401 2.7 The Collective Intelligence Genome 2.8 Conclusions 3: METHODOLOGY 3.1 Introduction 3.2 Phase I: Interviews with Ebola witnesses 3.3 Phase II: Quality of information in online health forums 3.4 Phase III: Observation of reddit and interviews with moderators 3.5 Summary of data collected 4: REDDIT – ‘THE FRONT PAGE OF THE INTERNET’ 4.1 Introduction 4.2 Structure of reddit 4.3 Voting on reddit 4.4 Ordering of information on reddit 4.5 Discussion 4.6 Conclusions 5: INFORMATION REQUIREMENTS DURING A PHEIC 5.1 Information requirements during a PHEIC 5.2 Questions during the 2014-16 Zaire ebolavirus outbreak 5.3 Context Shift: Far at Risk, Near at Risk, Real at Risk 5.4 Context Shift on r/ebola 5.5 Community Shift on r/ebola Page 10 of 401 5.6 Post-risk: Context and Community Shift 5.7 Discussion 5.8 Conclusions 6: INFORMATION RETURN: ANSWERS FROM REDDIT 6.1 Introduction 6.2 The relationship between trust and quality 6.3 Credibility on r/ebola 6.4 Searching for answers – the typical information-seeking journey 6.5 Quality of information on reddit health forums 6.6 Phase II-A: Investigation of information quality in online health forums (pilot study) 6.7 Phase II-B: Investigation of information quality in r/ebola 6.8 Phase III: Observation of reddit and interviews with moderators 6.9 Conclusions 7: WHO ARE THE MODERATORS AND WHAT DO THEY DO? 7.1 Introduction 7.2 Definition of a reddit moderator used in the study 7. 3 Moderatation roles 7.4 Technical skills: using the moderation tools available 7.5 Becoming a moderator 7. 6 Moderation teams 7.7 Moderation requirements and Context/Community Shift Page 11 of 401 7.8 Moderator metacommunities – reachback into the reddit crowd 7.9 Conclusions 8: HOW SUBREDDITS SCALE UP IN A PHEIC 8.1 Introduction 8.2 The PHEIC subreddits 8.3 Growth of r/ebola 8.4 Genesis of r/ebola 8.5 From r/ebola to r/zika 8.6 When do surges occur? 8.7 Discussion and conclusions 9: DISCUSSION AND CONCLUSIONS 9.1 Lessons identified from r/ebola 9.2 Limitations of the study 9.3 The information journey and dimensions of risk 9.4 Context Shift and Community Shift 9.5 Credibility and trust on Reddit 9.6 The Expert Moderator 9.7 Conclusions 10: REFERENCES Page 12 of 401 FIGURES FIG 1: Proposed model for how users judge the credibility of online information FIG 2: Technology Acceptance Model FIG 3: The Choice Framework FIG 4: The Collective Intelligence Genome FIG 5: A screenshot of the message posted on Liberia Expats Googlegroup FIG 6: An example of the message posted on reddit health forums FIG 7: A typical example of a reddit Front Page FIG 8: The Front Page contains several different elements FIG 9: A series of link posts from the subreddit r/zika_en. FIG 10: A self-post asking for advice regarding the Zika virus FIG 11: Reddit registered users can comment on posts FIG 12: Votes are cast by clicking on the up or down arrow FIG 13: Clicking on the username of poster brings up their profile FIG 14: Users can click through to subreddits using the menu bar FIG 15: Search results page showing subreddits and posts returned FIG 16: The order in which posts are displayed on reddit is determined by the hotscore algorithm FIG 17: How (r) affects