The Challenges of Credibility in Open News Systems
Total Page:16
File Type:pdf, Size:1020Kb
UNIVERSITY OF SOUTHAMPTON FACULTY OF PHYSICAL SCIENCES AND ENGINEERING Electronics and Computer Science The challenges of credibility in open news systems by Jonathan Scott Thesis for the degree of Doctor of Philosophy March 2017 UNIVERSITY OF SOUTHAMPTON ABSTRACT FACULTY OF PHYSICAL SCIENCES AND ENGINEERING Electronics and Computer Science Doctor of Philosophy THE CHALLENGES OF CREDIBILITY IN OPEN NEWS SYSTEMS by Jonathan Scott The news media play a number of important roles in a modern democratic society. However, with the press under increasing pressure to deliver news more quickly and with fewer resources, some readers are turning to non-traditional sources of news. These sources are often open to contribution from ordinary citizens as well as journalists, allowing for a wider range of backgrounds and experiences to be represented. Alternative news sources feature different styles of writing, publishing schedules, and topical focus than traditional providers which can result in people holding different perceptions of these outlets. Due to the position of the news media in society it is important to understand how these perceptions are formed and what motivates people to engage with alternative sources of news. This thesis investigates how credibility relates to levels of openness and explores methods of increasing the credibility of open news systems. I produce a landscape of citizen participation in news and find that traditional news systems do not involve citizens to the extent that the terms used to describe them imply, and that more open systems tend to lack the structure and authority usually associated with news. An experiment is performed which shows that this `structure and authority' is related to how credible the news source is perceived to be. A graph-based community detection algorithm is then utilised to add some level of editorial control automatically to online news discussions. This algorithm is first validated to confirm that it is able to separate news discussion contributors into meaningful groups. Then an interface is designed to present the results of this algorithm, and a study is conducted to investigate the effects of using content grouping on the credibility assessments and behaviour of news readers. These studies find that though the interface presents a wider range of viewpoints than existing interfaces, presenting news discussions in this way does not result in a change of credibility. However there is some evidence that the changed interface may result in readers being exposed to different topics. This implies that credibility is linked more strongly to the source of the news report than the intrinsic qualities of the report. Contents Declaration of Authorship xiii Acknowledgements xv 1 Introduction1 1.1 Research Questions...............................4 1.2 Research Framework..............................4 1.3 Publications of this work to date.......................6 1.4 Structure of this thesis.............................6 2 Background9 2.1 Evolution of News...............................9 2.2 Defining News.................................. 15 2.3 The Role of News................................ 16 2.4 The Role of the News Editor......................... 18 2.5 Defining Citizen Journalism.......................... 20 2.6 News Values................................... 22 2.7 Criticisms of News............................... 22 2.8 Online News.................................. 30 2.9 Unequal Contribution............................. 31 2.10 Openness.................................... 32 3 An Analysis of Existing News Systems 35 3.1 Methodology.................................. 36 3.1.1 Defining Levels of Participation.................... 38 3.1.2 Producing a Spectrum of News Openness.............. 39 3.1.3 Selection of News Systems....................... 39 3.1.4 Case Studies.............................. 40 3.1.5 Building the Spectrum......................... 40 3.2 Results...................................... 43 3.3 Case Studies.................................. 44 3.3.1 Case Study 1: Your Thoughts on Hugo Chavez........... 45 3.3.2 Case Study 2: Reddit Bomb...................... 45 3.3.3 Case Study 3: My Tram Experience................. 48 3.3.4 Case Study 4: Transsexuals Should Cut It Out........... 49 3.4 Discussion.................................... 50 3.4.1 News Values.............................. 51 3.4.2 Routes Into The Mainstream..................... 51 v vi CONTENTS 3.5 Conclusions................................... 52 3.5.1 How much power is being given to citizens through online news systems?................................. 52 3.5.2 What sort of interactions exist between those systems that are more open and those that are less?.................. 53 4 Study of Credibility on The Web 55 4.1 Credibility.................................... 56 4.1.1 Models of Credibility.......................... 56 4.1.2 Types of Credibility.......................... 58 4.1.2.1 Message Credibility..................... 58 4.1.2.2 Source Credibility...................... 58 4.1.2.3 Media Credibility...................... 59 4.1.3 Spectrum of Credibility........................ 61 4.1.4 Measuring Credibility......................... 62 4.2 Methodology.................................. 63 4.3 Results...................................... 65 4.4 Discussion.................................... 68 4.5 Conclusions................................... 70 5 Study on Identifying Groups in News Communities 73 5.1 Online Voting.................................. 75 5.2 Community Detection............................. 76 5.3 Louvain Community Detection........................ 78 5.4 Methodology.................................. 79 5.4.1 Data Collection............................. 79 5.4.2 Study.................................. 81 5.5 Results...................................... 81 5.6 Discussion.................................... 82 5.7 Conclusions................................... 85 6 Study of the Effects of Novel Comment Presentation on Reading Be- haviour in Online News 87 6.1 News Comments................................ 88 6.2 Interface Design................................. 88 6.2.1 Motivations............................... 89 6.2.2 Existing Work............................. 92 6.2.3 Participatory Design.......................... 93 6.2.4 Final Designs.............................. 93 6.2.5 Design Evaluation........................... 94 6.3 Selection of Stories............................... 97 6.4 Experiment Plan................................ 98 6.5 Experimental Validation............................ 100 6.6 Results...................................... 102 6.6.1 Usability................................ 102 6.6.2 Behaviour................................ 102 6.6.3 Credibility............................... 103 6.6.4 Reasons given for ratings....................... 103 CONTENTS vii 6.6.5 Interviews................................ 104 6.6.5.1 Condition 1: Old Interface Old Algorithm......... 106 6.6.5.2 Condition 2: Old Interface New Algorithm........ 107 6.6.5.3 Condition 3: New Interface Old Algorithm........ 108 6.6.5.4 Condition 4: New Interface New Algorithm........ 109 6.6.6 Summaries Written by Participants.................. 109 6.7 Discussion.................................... 112 6.8 Conclusions................................... 115 7 Conclusions 119 7.1 Contributions.................................. 120 7.2 Research Questions............................... 120 7.2.1 How might we measure openness in terms of citizen participation in the news process?.......................... 121 7.2.2 Does a relationship exist between the openness of a system and its credibility?............................. 121 7.2.3 Will promoting a wide range of viewpoints (using viewpoint group- ings) change the exposure of readers to different viewpoints, and alter the perceived credibility of a system?.............. 122 7.3 Recommendations............................... 123 7.4 Future Work.................................. 124 7.5 Recent Developments.............................. 125 7.6 Conclusions................................... 126 A Terms References 129 B Study of Credibility on The Web Data 131 C Study of Algorithms for Identifying Sentiment Groups in News Com- munities Questions 135 D Interface Design 139 D.0.1 Iteration 1................................ 140 D.0.2 Iteration 2................................ 143 D.0.3 Iteration 3................................ 145 E Experimental validation notes 149 E.1 Participant 1.................................. 149 E.2 Participant 2.................................. 150 F Consuming Experiment Demographics 151 G Consuming Experiment Credibility Results 153 H Consuming Experiment Credibility Reasons 159 I Article and Comment Summaries 163 J Interview Transcripts 171 J.1 Condition 1 Old Interface Old Algorithm.................. 171 viii CONTENTS J.1.1 Participant1.............................. 171 J.1.2 Participant2.............................. 176 J.2 Condition 2 Old Interface New Algorithm.................. 179 J.2.1 Participant3.............................. 179 J.2.2 Participant4.............................. 184 J.3 Condition 3 New Interface Old Algorithm.................. 188 J.3.1 Participant5.............................. 188 J.3.2 Participant6.............................. 193 J.4 Condition 4 New Interface New Algorithm.................. 197 J.4.1 Participant7.............................. 197 J.4.2 Participant8.............................. 201 K Contribution