DECEMBER 2020 The challenges of online fact checking About this report Misinformation causes real harm to people’s lives, health, finances and to democracy. It is time consuming and often difficult to check. This report lays out the main challenges fact checkers around the world face in finding and checking claims which appear in online places. It also presents ideas for improving these processes, with a focus on technology and counteracting the pressures fact checkers face when working with internet companies. We welcome feedback and comments at fullfact.org/contact Full Fact 2 Carlton Gardens London SW1Y 5AA k fullfact.org/contact D @FullFact K fullfact.org Published by Full Fact, November 2020. Published under the Creative Commons Attribution-ShareAlike 4.0 International License. Contents How we gathered the data ............................................................................................................................ 4 Introduction .................................................................................................................................................................. 6 Key recommendations ......................................................................................................................................... 8 Monitoring and selection of online claims .........................................................................................15 How fact checkers find claims ..............................................................................................................................15 How fact checkers select claims ..........................................................................................................................28 Summary of main monitoring and selection challenges and possible solutions ....................................30 Researching, writing and reviewing a fact check .......................................................................32 Overview of the research, writing and review process .................................................................................32 Skills and training for fact checking online claims ..........................................................................................39 Evidence: tools, websites, and access to data and institutions ..................................................................42 Summary of main research, writing and reviewing challenges and possible solutions .......................48 Publication and distribution of online fact checks ......................................................................50 Online promotion and presentation of fact checks .........................................................................................51 Press partnerships and syndication ...................................................................................................................55 Internet companies: distributing fact checks online automatically to mass audiences .......................56 Internet shutdowns .................................................................................................................................................65 Criticism and harassment ......................................................................................................................................66 Summary of main publishing and distribution challenges and possible solutions ................................72 Fact checkers and the internet companies .......................................................................................74 Facebook and fact checkers .................................................................................................................................80 Google, YouTube and fact checkers ....................................................................................................................93 Other internet companies ..................................................................................................................................102 Summary of main challenges of working with internet companies and possible solutions.............102 Technology and automation ......................................................................................................................104 How technology has changed the fact checking environment ...............................................................104 Automated fact checking....................................................................................................................................105 Where technology can help ...............................................................................................................................107 The fact checkers developing technology to assist their work .................................................................109 Conclusion ................................................................................................................................................................ 116 Summary of challenges ....................................................................................................................................... 116 Appendix....................................................................................................................................................................118 All recommendations ...........................................................................................................................................118 CHALLENGES OF ONLINE FACT CHECKING How we gathered the data Interviewees approached for this research were chosen based on a number of factors. Foremost was the need to obtain a range of global perspectives: we wanted to hear from fact checkers in Africa, Asia Pacific, Europe, North America, Latin America and the Middle East and North Africa regions. We wanted to explore the diversity in experiences of online fact checking, so decided to approach newer as well as more established fact checkers, large and small organisations, independent organisations as well as those which are part of a larger media house, online-only fact checkers as well as those which also do political fact checking. We also wanted to interview organisations that are not Facebook partners, and at least one fact checker that is not a signatory to the International Fact-Checking Network Code. After drawing up a shortlist of fact checkers to contact against these criteria, we reached out to editors and reporters from: AFP Chequeado Factnameh PolitiFact North America Argentina Iran USA Africa Check Dubawa Fatabyyano Rappler South Africa Nigeria Jordan1 Philippines Africa Check Ellinika Hoaxes Full Fact La Silla Vacía Senegal Greece UK Colombia Animal Politico Fact Crescendo Maldita.es Teyit Mexico India, Myanmar, Sri Lanka Spain Turkey Aos Fatos Factly PesaCheck Brazil India Kenya, Tanzania, Uganda 1 Based in Jordan but serves Tunisia, Morocco, Algeria, Palestine, Syrian Arab Republic, Libya, Jordan, Saudi Arabia, United Arab Emirates, Qatar, Kuwait, Oman, Bahrain, Yemen, Iraq, Egypt, Sudan and Lebanon. fullfact.org 4 CHALLENGES OF ONLINE FACT CHECKING Semi-structured interviews were conducted over Zoom and telephone in February and March 2020, and ran to around 90 minutes for each fact checker, including for Full Fact. Themes included monitoring of online claims, research, publication, distribution, the Facebook partnership and working with internet companies more generally, and fact checkers’ use of technology. This was supplemented with unstructured calls with Full Fact’s automated fact checking team, the International Fact-Checking Network, ABC RMIT Fact Check, and Lead Stories’ Trendolizer. Transcripts and field notes were then thematically analysed and supplemented with further phone calls, emails or Slack communication. We are grateful to Facebook for funding this report, and would like to thank all the fact checkers and other colleagues who generously contributed their time to this research. fullfact.org 5 CHALLENGES OF ONLINE FACT CHECKING Introduction Online organisations have been publishing fact checks since the foundation of Snopes in 1994, but the profile of fact checking has increased significantly over the past decade. Among the many reasons for this are the proliferation of misinformation online and increased collaboration with internet companies. Fact checking organisations which partner with Facebook have gained powerful levers to identify and reduce the spread of online misinformation. Internet companies are finding new ways to display fact checks in products like news feeds and search results, bringing name recognition and large new online audiences to fact checkers. The consensus on what should be checked has evolved to include online content as well as claims from politicians. Some organisations, such as Maldita.es or Teyit, were founded specifically to check social media content, while others took longer to expand their scope and treat online claims as equally important to political speech. Alongside this, fact checkers have added open source intelligence tools and research techniques to their repertoire, such as reverse image searching and advanced social media monitoring. Criticisms of independent fact checking have also increased. A member of Facebook’s new oversight board raised a concern that “fact-checking is biased” during one of his first media outings as a new board member.1 During a recent UK Parliament Select Committee hearing, Twitter’s Head of Public Policy,