World of trade-offs: what journalists think of rating scales in fact-checking A Project presented to the Faculty of the Graduate School at the University of Missouri-Columbia _______________________________________________________ In Partial Fulfillment of the Requirements for the Degree Master of Arts _____________________________________________________ by YUE YU Professor Tom Warhover, Committee Chair Professor Scott Swafford Professor Alison Young December 2019 i © Copyright by Yue Yu 2019 All Rights Reserved ii Acknowledgments First and foremost, I would like to thank my parents, Shunxin Yu and Ying Wang, for supporting my decision coming to the U.S. for my dream in journalism two and a half years ago. My mother, who raised me by herself for more than a decade and whose yearlong salary could barely cover my tuition for the first semester, had always wanted me to stay close to home but nonetheless chose to support me and understand my career endeavors. My father, who always encouraged me to push through during the hardest times, has been a great mentor and friend for the past two years. Speaking of mentorship, I’d like to further thank all my professors at the University of Missouri who taught me invaluable skills and helped me become a better journalist. These professors include but are not limited to: Scott Swafford, Mark Horvit, David Herzog, Tom Warhover, Katherine Reed, Brett Johnson, Liz Brixey, Mike Jenner and Jeanne Abbott. I would like to express my special gratitude to Scott Swafford, who nurtured and guided me over the years with his patience and kindness; to Mark Horvit, who connected me to a wide range of opportunities and always challenged me to dig deeper, and to Tom Warhover, who offered me sharp and detailed comments on my research project. I also want to thank my academic advisor Kathy Adams for helping me whenever she can and offering me moral support at some of my lowest points. I’m also grateful for having Alison Young as my D.C. program director, who is ever understanding and caring as I struggle with my health conditions throughout the final semester. Finally, I’d like to thank all the journalists and researchers I had the pleasure of connecting with during my final semester in D.C. That includes my supporting colleagues at the Center for iii Responsive Politics where I fostered a strong set of skills covering national news from the perspective of money in politics, and all the journalists and researchers who took their time for my interviews. This professional project would not have been possible without their help. iv Table of Contents Acknowledgments ........................................................................................................................................ iii Chapter 1: Introduction ................................................................................................................................. 1 Chapter 2: Literature Review ........................................................................................................................ 3 Chapter 3: Analysis ..................................................................................................................................... 24 Appendix A: Interview Transcripts ............................................................................................................. 38 Appendix B: Activity Log .......................................................................................................................... 99 Appendix C: Personal Evaluation ............................................................................................................ 128 Appendix D: Project Proposal .................................................................................................................. 133 v vi Chapter 1: Introduction I started out at The Columbia Missourian as a local government reporter two and a half years ago. Without a clue of what the U.S. politics was like, I soon started learning about it by covering the city government in Columbia. While covering tax issues, substance abuse as well as the spring municipal election, I found myself frequently examining documents to back up politicians’ claims and sometimes asking them for proof. A year later, I changed my focus to report on issues on a larger scale. During the four months walking the halls in Jefferson City, I often took it with a grain of salt whenever lobbyists and lawmakers spoke of a piece of legislature without offering supporting evidence. It was when I was exposed to the reporting process at PolitiFact that I realized that I had been conducting fact-checks all along while covering politics at different levels. My experience at PolitiFact taught me how to spot checkable dubious claims, to track down independent sources and to defend my ratings. However, I also took notice that, more often than not, my classmates and I would struggle to reach a consensus on which rating we should assign to a claim. What kind of controversy has the rating system brought, if any? And how can we improve the rating system to further contribute to the public discourse? I chose to zoom in on the rating system because I wanted to answer those questions I had as well as to help fact-checkers brainstorm for new ideas to strengthen the current rating system and the practice of fact-checking. Research I conducted suggested clashes between fact-checkers at news outlets with a rating scale and those without, and my interviews with multiple researchers as well as professional fact- checkers further proved that point and offered more insights to the topic. While rating systems 1 generally offer readers a quick glance at the analysis behind each claim, they also invited criticism from readers who questioned the accuracy of the rating and from journalists who warned against the risk of oversimplification through assigning fixed labels to sophisticated issues. For the professional component of my project, I worked as a reporter intern at the Center for Responsive Politics, where I wrote about an array of topics including foreign and domestic lobbying, super PAC spending, political advertisement and industry spending. I learned how to write about national politics from a campaign finance perspective and advanced my skills on deadline writing and data analysis, among others. My opportunity writing about money in politics encouraged me to study the campaign finance laws at the national level, dig through documents to find otherwise overlooked information and pitch story ideas from a brand-new angle. I’ve fostered the habit of examining fact-checking websites for background information I need for certain politicians or pieces of legislature. The fact-checks often offered ample evidence and nuanced analysis that I needed while researching a certain topic. The ratings also helped me a great deal when I sought quick opinions on certain political claims when on deadline. Through this project, I’ve learned much more about both the ongoing debate among fact-checkers on the pros and cons of a rating system and the world of writing about money in politics in an exploding news cycle. Both aspects of my experience in D.C. helped prepare me to become a future political reporter who knows always to look for evidence instead of conducting “he said, she said” reporting and to explore the paper trail politicians and lobbyists leave behind. 2 Chapter 2: Literature Review Theoretical framework The fact-checking process before publication, according to Lucas Graves in his book Deciding What’s True: The Rise of Political Fact-Checking in American Journalism, speaks to “the imperative to eliminate untruth,” which means fact-checkers need to discern between facts and lies and make judgments to discard or call out misinformation (Graves, 2016). This nature of fact- checking largely fits into the framework of gatekeeping theory invented by psychologist Kurt Lewin in 1947. Lewin coined the term “gate keeper” in his series “Frontiers in Group Dynamics,” which included an analysis of the decision-making process behind food selection (Lewin, 1947). Lewin argued that there is a “gate” at work at each stage of the process of determining what kind of food would finally make it to the dinner table, and each gate is “governed either by impartial rules or by ‘gate keepers.’” (Lewin, 1947, p. 145). For instance, decisions are made over which type of food to grow and which to buy at a grocery store (p. 145). In this case, decision makers act as the “gate keepers” and play a vital role in controlling what type of food to take into consideration and what not at each “gate” (p. 145). They guard the gate to keep certain choices out and let others in according to their own standards (Lewin, 1947). In his article, Lewin (1947) mentioned that the theory of gatekeeping also applies outside of the psychology realm. “This holds not only for food channels but also for the travelling of a news item through certain communication channels in a group, for movements of goods, and the social locomotion of individuals in many organizations,” Lewin wrote (p.145). The theory was then first expanded to the field of communication studies by David Manning White in a case study in 1950 to explain how reporters and editors decide what stories to include in the newspaper. 3 Lewin (1947) further argued that it’s important to understand the “forces” influencing gatekeepers’ decisions, which Lewin said “depend partly on their ideology, that is their system of values and beliefs which determine