Editorial Judgment in an Age of Data: How Audience Analytics and Metrics Are Influencing the Placement of News Products
Total Page:16
File Type:pdf, Size:1020Kb
Editorial Judgment in an Age of Data: How Audience Analytics and Metrics are Influencing the Placement of News Products A DISSERTATION SUBMITTED TO THE FACULTY OF UNIVERSITY OF MINNESOTA BY Rodrigo Zamith IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Seth C. Lewis, Co-Adviser Marco Yzer, Co-Adviser May 2015 © 2015 Rodrigo Zamith i Acknowledgements The prominence of my name on the title page is misleading. This dissertation is not just my work. It is the product of a wonderful environment and an incredible cast of advisers, family, and friends. (These categories are far from mutually exclusive!) I should begin by acknowledging the contributions of my mother, father, brother, and sister. I would not have made it this far without their unceasing enthusiasm and optimism. A wonderful cast of friends similarly kept me sane and challenged my ideas. Anna, Brett and Kathleen, Konstantin and Emily, Liz and Brian, Meagan, and Sarah were far from the only colleagues who shared a pint with me one day and helped me interpret a finding the next, but they deserve special mention for their immense charity and good humor throughout my program. My advising team was nothing short of terrific. Dr. Fan forced me to think carefully about how to lay the foundation for my analyses well in advance of conducting them. Dr. Lewis went to tremendous lengths to help me grow as a scholar over the past four years, and I will consider myself a success if I ever come close to the high bar he sets for himself as a person and as a scholar. Dr. Watson kept me on my toes throughout my program, making sure my ego never got too inflated or deflated, all while pushing me to work harder, sharing far more of his time than a student should expect in the best of circumstances, and introducing me to a few new brews. Dr. Yzer alternated being a sounding board, a teammate, and the sort of deep thinker who gives you confidence in everything you do. In short, if any of these individuals were to be removed from this process, this dissertation would have turned out quite differently—if at all. Thank you. ii Dedication To my mother and father, who have offered me consistent and unconditional support and affection. iii Abstract In recent years, audience analytics (systems that collect and analyze digital trace data from users) and audience metrics (quantified measures of how content is consumed and interacted with) have gained currency within newsrooms, enabling them to influence editorial newsworkers‘ constructions of their audiences and, consequently, the shapes that news products take. The extent of that influence, however, remains largely unknown, with few studies examining the direct relationship between metrics and news content. The present work addresses this shortcoming by offering methodological guidance and an empirical assessment of the extent to which one particularly salient metric, page views, influences the prominence and de-selection of news items on the homepages of several news organizations. It demonstrates that algorithms can be leveraged to computationally analyze particular aesthetics of a large volume of homepages; that the ‗most viewed‘ list can serve as a useful indicator of the popularity of news items, though such lists are not always comparable across organizations and introduce important limitations; and that the effect of an item‘s popularity on its subsequent placement on the homepage is fairly muted in the process of selection, though it is greater in the process of de-selection. In short, the present research indicates that the current content-related effects of audience metrics—at least as it pertains to a particular editorial function and metric—may be overstated in the literature, and offers pathways for further studying similar relationships. iv Table of Contents Acknowledgements .............................................................................................................. i Dedication ........................................................................................................................... ii Abstract .............................................................................................................................. iii List of Tables .................................................................................................................... vii List of Figures .................................................................................................................. viii CHAPTER I: INTRODUCTION ........................................................................................ 9 Constructing Audiences ................................................................................................ 10 Environmental Challenges ............................................................................................ 12 Newswork, Analytics, and Metrics ............................................................................... 14 Gaps in the Literature and Contribution of the Dissertation ......................................... 15 CHAPTER II: QUANTIFYING AUDIENCES ............................................................... 18 The Construction of the Audience ................................................................................ 19 The Emergence of Audience Analytics ........................................................................ 23 News Media in Uncertain Times .................................................................................. 28 Audience Analytics and Metrics in the Newsroom ...................................................... 38 Synthesizing the Literature ........................................................................................... 50 CHAPTER III: COMPUTATIONALLY ANALYZING LIQUID CONTENT .............. 54 Background ................................................................................................................... 55 v Case Study .................................................................................................................... 63 Discussion ..................................................................................................................... 77 CHAPTER IV: DECIPHERING ‗MOST VIEWED‘ LISTS ........................................... 82 Background ................................................................................................................... 84 Research Questions ....................................................................................................... 88 Method .......................................................................................................................... 90 Results ........................................................................................................................... 92 Discussion ..................................................................................................................... 97 CHAPTER V: THE EFFECTS OF POPULARITY ON PROMINENCE ..................... 102 Background ................................................................................................................. 104 Research Questions and Hypotheses .......................................................................... 113 Method ........................................................................................................................ 116 Results ......................................................................................................................... 122 Discussion ................................................................................................................... 129 CHAPTER VI: CONCLUSION ..................................................................................... 137 Online Journalism and Computational Social Science ............................................... 137 Fulfilling Journalism‘s Public Service Mission .......................................................... 142 Final Remarks ............................................................................................................. 148 TABLES & FIGURES .................................................................................................... 150 vi REFERENCES ............................................................................................................... 170 vii List of Tables Table 1. List of the 50 Largest U.S. Newspaper Organizations ..................................... 150 Table 2. List of News Organizations Analyzed in Chapter V ........................................ 151 Table 3. Effect of Being Popular at Time (t) on Visibility at Time (t+1) ....................... 152 Table 4. Effect of Popularity at Time (t) on Prominence at Time (t+1) ......................... 156 viii List of Figures Figure 1. Sample Python code for evaluating pages ....................................................... 158 Figure 2. Example coding scheme for The Denver Post‘s homepage ............................ 159 Figure 3. Example coding scheme for the New York Times‘ homepage ........................ 160 Figure 4. Example coding scheme for The Star-Ledger‘s homepage ............................. 161 Figure 5. Electronic interface for verifying algorithmic coding decisions ..................... 162 Figure 6. Average rate of change for the ‗most viewed‘ lists ......................................... 163 Figure 7. Amount of time it takes news items to appear on the ‗most viewed‘ lists. ..... 164 Figure 8. Cluster of news organizations