Running head: AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 1
How can automatic systems facilitate journalistic discovery of newsworthy content? An investigation into how editorial workflows can be automatically supported
Sebastian Short Snr 2031037
Master's Thesis Communication and Information Sciences Specialisation Business Communication and Digital Media
School of Humanities and Digital Sciences Tilburg University, Tilburg
Supervisor: Dr. C.W.J. van Miltenburg Second Reader: Dr. E.J Krahmer
August 2020
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 2
Abstract
This study investigated in the era of media convergence how editorial workflows can be supported by new tools. In addition, the presence of new technologies such as automated journalism and automatically written text was reasoned as a method to how such routines could be automatically supported, adding onto the emerging body of literature that has investigated the interactions between journalism, automated journalism and automatically written text. 12 interviews were conducted with editors at various news outlets, and those with knowledge of the editorial process. The results provided a current overview of sourcing routines, which consisted of using personal networks, searching online and the use of traditional sources to discover new content. In addition, they do show signs that automatic support can be facilitated by looking at what issues’ editorial teams face and how new tools can then integrate into what they currently do. Such integration however depends on factors such as quality of automatically written text and where new tools can be integrated into their current workflows. Facilitators and barriers to adoption of automated journalism are also discussed, which showed that the use of newer technologies within journalistic practice is still an ongoing and uncertain process.
Keywords: automated journalism, editorial workflows, journalism, content discovery AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 3
Contents Introduction 5 The Content Exchange 8 Societal and Scientific Relevance 8 Literature Review 9 The Role of Convergence in the Changing of the News Gathering Process 9 The ‘first wave’ of ethnographic studies. The way newsworthy content is discovered 9 The ‘second wave’ of ethnographic studies. 9 Media convergence and its implications. 10 The sourcing and discovery of content online 14 Automated journalism: can news production be automated? 16 How can journalism be automated? 17 How can automation change the newsgathering and production process? 17 Automatically written texts and readability. 18 Addressing the strengths and limitations of automated journalism. 19 Method 22 Research Design 22 Participants 22 Materials 24 Procedure 26 Analysis 27 Results 28 The current state of content discovery 28 Current editorial routines 28 Use of personal networks. 28 Searching and monitoring information online. 29 Use of traditional sources. 30 Integration of automatic support 31 Factor 1: New tools must integrate into current routines. 31 Factor 2: New tools need to address existing problems. 32 Facilitators of adoption of automated journalism 33 Editors have an open attitude towards automated journalism. 33 Editors are willing to source automatically written content. 34 Barriers of adoption of automated journalism 35 News outlets are slow to react to technological change. 35 There is a need for more technological development. 36 Editors are stubborn. 37 AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 4
News outlets must balance their available resources. 38 Discussion & Conclusion 40 The state of current sourcing routines 41 Moving towards integration of automatic support 42 Addressing the facilitators and barriers to automated journalism 44 Limitations and recommendations for future research 46 References 50 Appendix A 57 Appendix B 58 Appendix C 70 Appendix D 77 Appendix E 84 Appendix F 92 Appendix G 102 Appendix H 112 Appendix I 122 Appendix J 131 Appendix K 139 Appendix L 150 Appendix M 160 Appendix N 174 Appendix O 177 Appendix P 178 Appendix Q 187 Appendix R 220
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 5
Introduction
Journalistic workflows have changed considerably in recent years due to the media convergence of news, where outlets must digitise their content, driven by technological change (Dal Zotto & Lugmayr, 2016). Media convergence has reconfigured the ways news outlets must operate to remain financially stable (Tameling & Broesma 2013), and has led to changes in the way content is discovered, meaning that stories can be quickly produced by re- using content on newswires (Boumans, Trilling, Vliegenthart & Boomgaarden, 2018;
Welbers, 2018), or by simply finding a tweet (Broersma & Graham, 2013). One consequence on the other hand is that it has put pressure on journalists to quickly find and publish content before their competitors (Bunce, 2017), and as such it is a constantly flowing process. In a bid to be more efficient journalists turn to Wikipedia, Twitter or Google to source news
(Broersma & Graham, 2013; Kruikemeier & Lecheler, 2016). In the Netherlands, there are high levels of reliance on the use of the news agency Algemeen Nederlands Persbureau
(ANP), where outlets make use of its content to include in their own stories (Boumans,
Trilling, Vliegenthart & Boomgaarden, 2018; Welbers, 2018). Reliance on finding quicker ways to source content is reasoned because of media convergence and the economic pressures that have arisen in the journalistic world (Saridou, Spyridou & Veglis, 2017).
Changes in how news outlets operate and alternative ways of discovering content are just a few of the affect’s media convergence has had in the newsroom. Adapting to changes within newsrooms is having consequences for journalists, some even argue the quality of news is affected as many firms turn to quantity versus quality (Bunce, 2017), and debate on whether editorial and commercial interests should remain separate has resurfaced (Cornia,
Sehl & Nielsen, 2020), where news content itself has changed, with blurred lines between commercial content for advertising purposes and editorial content for readers. This is then a sign that there are now norms being adopted by both editorial and commercial managers to AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 6 keep up with changing times (Cornia, Sehl & Nielsen, 2020). Therefore, it seems that news outlets are seeking new cost-effective ways to do discover newsworthy content, the exact methods of which are more uncertain. In the world of data driven journalism (Lewis &
Westlund, 2015), it is established that tools are used in editorial rooms to monitor web traffic and track article metrics (Wang, 2017; Wu, Tandoc & Salmon, 2019a). What is then interesting to examine is another tool to add to an editors’ toolkit, something that could be used for content discovery.
One method to achieve more cost-effective journalist practices can be seen via the use of Automated Journalism, whereby Artificial Intelligence and algorithms can automate some tasks and automatically write content (Beckett, 2019). Natural Language Generation systems, where texts can be automatically written are being used increasingly more for writing news articles (Caswell & Dörr, 2018; Wu, Tandoc & Salmon, 2019b). This then provides an alternative method for producing cost-effective journalism by reducing the cost of producing content itself. This is consequently freeing up time for journalists, where some claim they can focus more on quality journalism (Anderson, 2011; Primo & Zago, 2015), whereas others claim it puts their jobs at risk (Wu, et al, 2019; Schultz & Sheffer, 2017). The presence of automated journalism further highlights how technology is further integrating into journalistic work routines (Wu, et al., 2019b), and can be seen as a new technology that news outlets are adapting to (Beckett, 2019) following years of media convergence. Arguably what this literature does not address is how such practices could be then used to solve the problem to find cost-effective ways to discover content. Based on the assumptions that consumers are generally accepting of automatically written texts (Graefe, et al., 2018; Jung, et al., 2017; Wöker & Powell, 2018), then it does not seem too big of a step for outlets then to source content that is automatically written. It could then be assumed that if a new tool were AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 7 to offer automatically written content, then this could go some way to assisting editors in discovering content more cost effectively, if editorial teams are open to it.
Such an issue then raises further questions, such as how journalists discover content currently, and how this could be supported to assist journalistic workflows, moving away from the current research that addresses and acknowledges the changes in journalistic workflows over time (Bunce, 2017; Broersma & Graham, 2013; Kruikemeier & Lecheler,
2016), it then seems logical to investigate possible solutions in the sourcing and discovery of newsworthy content. If automated journalism is seen as promising (Anderson, 2011; Primo &
Zago, 2015), it seems reasonable to assume that automatic support can be a potential solution for editors to discover new content. In addition, whether current editorial sourcing routines they feel are effective, considering the rise of automated journalism and automatically written text. Based on this, the following research questions are asked:
RQ1: What methods do news editors use to help them find content?
RQ2: To what extent can editorial workflows to find content be automatically supported?
RQ3a: What are editors’ current attitudes towards automated (journalism) support tools in newsrooms?
RQ3b: What are editors’ current attitudes towards the use of automatically written content in newsrooms?
The results of this study, with the use of 12 semi-structured interviews with editors and those with extensive knowledge of the editorial process aims to build an overview of how a content discovery platform could best match the needs of editors in their content discovery process, and the role of automated journalism within this. The results from these interviews provide an up-to-date overview of what current methods are used to discover content, and how this could be supported. The literature review then outlines how media convergence has changed content discovery routines, followed by what is currently done. It then concludes by AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 8 examining the impact of automated journalism and how this is further changing journalistic work routines.
The Content Exchange The Content Exchange (TCE) is a new Dutch start-up that aims to develop a platform where media companies, journalists and editors can share content with ease. An individual can offer text on the platform that can be reused by other outlets for a fee. TCE later plans to incorporate automatically written content into their system to create a platform where content can be accessed more cost-effectively. The goal is to produce a marketplace for content that can be easily circulated. Such a platform provides opportunity to illustrate the discussed concepts, whereby having a concrete example of what automatic support looks like can then better inform editors’ opinions on new tools.
Societal and Scientific Relevance This study was conducted in collaboration with TCE focusing on areas surrounding automated journalism which has immediate practical relevance, as it sheds light on how improve editorial workflows in terms of discovering content. The present study provides insights and improvements on editorial sourcing routines, with the benefits of being able to do so more efficiently. In a similar vein, this study adds theoretical insight into the increasing media convergence and the role of newer technologies from the perspective of those who are looking for content to be published (Bunce, 2017; Wu, et al., 2019b, Yang, 2017). This study addresses it from another angle, that of sourcing, where there is literature on how online sources have been increasingly used over time, it does not account for sourcing routines themselves. This study also adds onto the emerging literature on automated journalism (Dörr,
2016; Montal & Reich, 2016; Thurman, Dörr & Kunert, 2017; Wu, et al., 2019b) by focusing on automatically written texts within current content discovery routines, examining how it can be used as a method to achieve more cost-effective journalistic practices. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 9
Literature Review
The Role of Convergence in the Changing of the News Gathering Process
The ‘first wave’ of ethnographic studies. The way newsworthy content is discovered has changed drastically since the ‘first wave’ of ethnography studies examining newsroom practices (Epstein, 1974; Fishman, 1980; Gans, 1979; Hall, Catcher, Jefferson,
Clarke & Roberts, 1978; Gitlin, 1980; Tuchman, 1973, 1978). Previously there were routines journalists must follow to ensure that they follow organisational norms. To best adhere to organisational goals and news gathering requirements, journalists had to source new information for new stories as well as verify sources (Gans, 1979), which required constant monitoring of the ‘news net’ of other media and sources (Tuckman, 1978). Correspondents were organised into ‘news beats’, with news bureaus were being set up at different locations
(Fishman, 1980; Tuckman, 1973). This then resulted in a high dependence on relying upon official sources, who have access to their organisations and come to define news stories (Hill, et al., 1978). Such fixed structures to find content have since shifted, giving more control to the journalist.
The ‘second wave’ of ethnographic studies. The landscape for journalists and those working within the newsroom has changed with the advent of new technologies. Since the call for the ‘new wave’ of ethnographic studies to explain the new dynamics of the newsroom (Cottle, 2000), a myriad of research has examined the media convergence of journalism (Klinenberg, 2005; Saltzis & Dickinson, 2008; Saridou, et al., 2017), which broadly refers to the delivery of broadcast, radio and print media under one platform, referring to the digitisation of content (Dal Zotto & Lugmayr, 2016). Jenkins (2006) refers to it as “the flow of content across multiple media platforms, the cooperation between multiple media industries, and the migratory behaviour of media audiences”, viewing it is a process that has slowly taken place as media organisations look to publish content across multiple AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 10 digital platforms. Cottle (2000) emphasised that previously bureaucratic work routines within organisations were changing, whereby journalistic work routines were becoming more flexible, adding more control for them over production of news and granting them access to a wider range of sources (Cottle, 1999), therefore Cottle (2000) concluded that technological change was reconfiguring the way journalism was conducted as well as the routines related to sourcing.
Media convergence and its implications. Studies conducted during the period of media convergence can give insight into the phenomenon as it occurred. As far back as 2001, journalists were reporting that they were feeling cautious of the pressures of having to pick up additional technical skills due to media convergence (Duze & Bardoel, 2001). Klienenberg
(2005) observed that journalists felt their autonomy was decreasing, being asked to be more flexible across broadcast, print and online, which was creating higher job demands. This then led to them relying on the easiest access point to find information, information that is online.
Later research then found that this trend continued to rise, whereby journalists must learn new skills, have additional responsibilities and flexibility, with a consequence being higher pressure to perform. Such pressures many argue is resulting in a lower focus on quality journalism (Chada & Wells, 2016; Deuze 2008; Robinson, 2011; Ruusunoksa & Kunelius,
2007; Saltzis & Dickinson, 2008). In recent years media convergence has continued to push forward, with newsrooms fully embracing online news and social media (Lewis & Molyneux,
2018). Taken together, these studies support the notion that news outlets have had to change drastically to keep up with convergence, which highlights their need to find more efficient ways to source and find newsworthy content.
Adaptations to media convergence can further be seen in the observations of Bunce
(2017), when producing content related to financial reporting some journalists risked being openly criticised if they were not faster than their main competitor. This highlights how news AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 11 content needs to be discoverable and quickly published to stay ahead and remain competitive.
Discovering newsworthy content for journalists in the digital newsroom therefore remains a key pressure point as a determinant of their success within their jobs, thereby creating time pressures to source and discover relevant content, what arguably then does not address is then potential solutions to address such issues. While there does seem to be some adoption of tools to assist journalists in the newsroom, these tools mainly cover other functions, namely to monitor web-traffic and the reach and click-rates of news stories (Wang, 2017; Wu, et al.,
2019a).
An alternative view on changing work routines is provided by Cornia, et al., (2020), who investigated the changing relationships between commercial and editorial departments, which have traditionally been separate. The results of Cornia, et al., (2020) suggest that merging of economic and editorial interests is becoming essential to navigate the new business environment news outlets operate in. While their interviews do not specially focus on technological change, both senior editors and commercial managers expressed a need to establish new norms to best react to new changes in journalistic work routines. They concluded that editorial and commercial interests must converge, where there must be new organisational norms to adapt to business challenges, one such area being the merging of commercial and editorial content, either for generating advertising revenue or delivering a story to readers. This then shows that the adaption to convergence and technological changes is unavoidable and news outlets must now find new ways to adapt to it, regardless of their opinion, and highlights how the nature of news content itself is also changing. What this and previous research on convergence does address is how that convergence has been a process that has been adapted to, but less so how such changes can be tackled. Therefore, the use of new tooling then seems like a logical area to investigate, to see if it can fit into journalistic work routines. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 12
Many of the aforementioned studies of the newsroom base their findings on Field
Theory (Willig, Waltorp, & Hartley, 2015), a field is a categorisation of society into various spheres of actions, that must adhere to their own laws and logic to function (Bourdieu, 1993), such logics can be influenced by internal and external dynamics (Bourdieu, 2005). Of relevance is the journalistic field, which is a microcosm guided by its own laws which cannot be controlled by external factors (Bourdieu, 1998). The field can also be referred to as the journalistic game (Willig, 2013) that journalists must participate in to navigate their work environment. Furthermore, each journalist has their own outlook on their career and their perceptions of how they work, which is termed their habitus (Bourdieu, 1998). Bourdieu describes the habitus as “structuring structure” (Bourdieu, 1984, p. 170), which dictates how journalists can overcome the rules of the journalistic game (Willig, 2013). Within this habitus they then accumulate capital, which can be economic such as their salary, or cultural such as their experience and reputation (Bunce, 2017).
Subsequently, Bourdieu (2005) theorises that the journalistic field is caught within a field of power, with economic power in most situations overshadowing cultural power. Wang
(2017) expands on the journalistic field further, by incorporating the incorporation of data- driven journalism. Their model of the dimensional journalistic field theorises that journalists must grapple between qualitative and quantitative evidence. They argue that technological change has brought about more pressure to rely on quantitative data-driven sources, versus traditional methods such as interviewing sources. Subsequently, the presence of power has changed the balance of the journalistic game and is a reason behind pressures to discover content quickly (Bunce, 2017). There then appears to be opportunity for a system to automatically support such workflows, if a tool can then facilitate the discovery of new content, then it could then address some issues that journalists face. Some are turning to automatically written articles to cover news topics more cost-effectively (Graefe, 2016; AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 13
Walker, 2014). Here it could be argued now that social media and interactive websites have been widely adopted (Rudman & Bruwer, 2016), with the use of AI and algorithms on the rise, automating journalistic processes (Anderson, 2018; Butcher, 2017; Gorwa, Binns, &
Katzenbach, 2020; Lewis & Westlund, 2015) such as discovering and producing news content (Wu, et al, 2019a; Schultz & Sheffer, 2017) could now be seen as new technological change outlets are adapting to.
Wu, et al., (2019b) argue that technological firms that are pushing automation are new entrants to the journalistic field, as there is an increasing overlap of their role within the news production process. Therefore, it appears that there are more opportunities and challenges on the horizon for news outlets when it comes to automation. This can be evidenced by the prominence in exploratory qualitative literature on this topic (Graefe, Haim,
Brosius & Haarmann, 2018; Henrickson, 2018; Jung, Song, Kim, Im, & Oh, 2017; Wöker &
Powell, 2018). This then raises questions as to whether journalists are open to such automation within their sourcing routines. While this research does so far address and highlights how convergence has changed how outlets have operated over time, it does not address possible ways of making these new workflows more efficient, of which automatic support could come in. If news outlets are finding new ways to work (Cornia, et al., 2020) it then seems logical to investigate potential ways that journalistic work routines could be improved, with additional tooling to help them discover newsworthy content. Therefore, the following sections of this literature review firstly explore what is currently done in terms of finding new content, then it moves onto current role of AI in the newsroom, and how automatically written text can then create more efficient content discovery routines for journalists. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 14
The sourcing and discovery of content online A central task of journalistic work routines in the newsroom is the sourcing and gathering of newsworthy content. Traditionally this process has been highly controlled with professional routines to determine how to use sources in news stories (Gans, 1979). In modern times news outlets have increasingly relied on sourcing content online, many journalists make use of information found on Wikipedia, Twitter, and Google (Kruikemeier
& Lecheler, 2016). Many sources come from social media and entire news stories can be based on a single tweet (Broersma & Graham, 2013). Sourcing of news can be split into two categories, structured which refers to coordinated press strategies, such as press conferences and interviews and unstructured, which refers to more spontaneous observations of events such as calling on the phone or conducting background research (Kruikemeier & Lecheler,
2016). It can then be assumed that much research is then done online, as it is highly convenient. In the Netherlands, one study found that when comparing print to online outlets, almost 75% of content on online-only outlets was sourced from newswires (Boumans, et al.,
2018). It appears that the dominant online source to emerge over the years is Twitter (van
Leuven, Kruikemeier, Lecheler & Hermans, 2018; Kruikemeier & Lecheler, 2016). In their systematic literature review on online sourcing, Kruikemeier & Lecheler (2016) concluded that Twitter is one of the prominent online sources to arise but can depend on news topic.
Collectively this then demonstrates the prominence of online sourcing, in line with changing work routines due to convergence (Bunce, 2017; Chada & Wells, 2016; Deuze 2008;
Robinson, 2011).
Sariodou, et al (2017) provide a different perspective, and place emphasis on the economic pressures of media convergence on the increasing reliance of ‘pre-packaged’ online sourcing, such as PR newswires and news agencies even to the point where content is quickly produced and recycled from other websites, where outlets can quickly reproduce it at a lower cost at the expense of independent sourcing. This is termed Churnalism, whereby AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 15 such outlets must balance the demand and speed of publishing content, at the cost of quality.
Sariodou, et al (2017) highlighted that from 2013 to 2016, Churnalism has increased. The results of Boumans, et al., (2018) saw that a high amount of news agency content used by outlets was copied word-for-word, with their results also suggesting a presence of
Churnalism. This then signals that the convenience of online sourcing is prominent and supports the notion that searching online is relied upon for many journalists.
It should also not be automatically assumed however that online sourcing has replaced traditional sourcing (Kruikemeier & Lecheler, 2016), many studies conclude that online sourcing is useful and can complement traditional sourcing techniques (Deprez & van
Leuven, 2018). On the other hand, some journalists find online sourcing only useful for soft news (Moon and Hadley, 2014), whereas other journalists may heavily on online sources
(Gulyas, 2013), as websites constantly have to be updated (Thurman & Walters, 2013) which
Kruikemeier & Lecheler (2016) attribute to the time pressures of having to publish online news. Conversely, looking back at the presence of Churnalism, it does seem logical to conclude that there is some time pressure within journalistic work routines, in a similar vein to changing work routines due to convergence (Bunce, 2017; Chada & Wells, 2016; Deuze
2008; Robinson, 2011; Ruusunoksa & Kunelius, 2007; Saltzis & Dickinson, 2008) to which automatic support could come of assistance. Looking further at content discovery itself, one- stop places such as news agencies are valuable resources for news outlets (Boumans, et al.,
2018; Welbers, 2018). This could then explain why so many journalists turn to single sources such as Twitter and Wikipedia. During an information search, Wikipedia despite being perceived as less credible than other sources of information is still highly valued for its convenience and wealth of information it can provide (Neuberger, 2013). This could then be attributed to pressures of technological changes and its effects on editorial workloads (Bunce,
2017; Chada & Wells, 2016). Considering the possibility that online journalists are facing AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 16 time pressures due to having to quickly publish newsworthy content (Thurman & Walters,
2013; Kruikemeier & Lecheler, 2016), it seems logical to then connect this to media convergence.
Collectively, the literature on sourcing routines provides a view that sourcing techniques consist of a balance of offline and online information collecting, with more of the balance tipping to the latter. Such research however is describing the use of online sourcing techniques, and not necessarily solutions to make such routines more effective. Questions remain as to weather a tool could automatically support such routines and facilitate content discovery and mitigate some of the pressures of having to quickly find content (Thurman &
Walters, 2013; Kruikemeier & Lecheler, 2016). If journalists are facing time pressures, then it could be assumed that they could be open to new solutions to help them discover content.
Having another tool to add to their workflows to navigate the vast amounts of information online could have positive effects and aligns with changing organisational norms (Cornia, et al., 2020) and pressures to source newsworthy content cost-effectively (Kruikemeier &
Lecheler, 2016; Sariodou, et al., 2017). Considering the challenges described with convergence and the pressures journalists face (Bunce, 2017), it then seems logical to provide an up-to-date overview of what editors are currently doing, to add further onto how workflows can be automatically supported.
Automated journalism: can news production be automated? More recent technologies to be embraced in the journalistic world is use of Artificial
Intelligence (AI). AI is recognised as a method of replacing human cognitive tasks, and via self-learning that can automatically make decisions (Diakopoulos 2014; Latar and Nordfors
2009). It has been increasingly integrated into news outlets via the use of algorithms, data and automation (Anderson, 2018; Butcher, 2017; Lewis & Westlund, 2015, Wu, et al., 2019a). AI in journalism has been applied in areas such as automating content moderation (Gorwa, et al., AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 17
2020) and news content writing itself (Wu, et al, 2019a; Schultz & Sheffer, 2017) improving efficiency of the content creation and management process (Primo & Zago, 2015).
How can journalism be automated? One enabling application of AI is the increased uptake in news articles that are automatically written by algorithms. This concept has many names such as automated journalism (Wu, et al, 2019a), robo-journalism (Schultz
& Sheffer, 2017) and algorithmic journalism (Anderson, 2011). Carlson (2015) defines automated journalism as the automatic conversion of data into a narrative for news reports. It makes use of Natural Language Generation (NLG), which refers to automatic production of text computationally from data (Caswell & Dörr, 2018). NLG systems have most popularly been classified by Reiter and Dill (1997; 2000) who define NLG as a subfield of AI and computational linguistics that involves the creation of systems that can generate comprehensible text from data, or “non-linguistic representations of information” (Reiter &
Dale, 1997, p.1). They classified NLG into a pipeline architecture, where different modules in a linear pipeline split the task of generating text into sub-tasks. Gatt and Krahmer (2018) conducted a review on the state of NLG systems and highlight how it has evolved over the years. Gatt and Krahmer (2018) detail integrated approaches, which then relies on statistical data to learn the differences between inputs and outputs. This currently is trending in NLG systems and research, where data as an input is more relied upon for text output. An example of this was seen when an earthquake struck in Los Angeles, within 3 minutes the LA Times had already reported the incident, as they had a ‘Quakebot’, which had automatically obtained seismic data and produced a body of text about the event (Walker, 2014). Other data inputs can include numerical data and even images and video (Kulkarni, Premraj, Ordonez,
Dhar, Li, Choi, Berg, & Berg, 2013; Khan & Gotoh, 2017).
How can automation change the newsgathering and production process? The increasing inputs for data-to-text outputs is reflected in its wider range of applications, AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 18 including its adoption within the journalistic field (Wu, et al, 2019b). NLG within journalism has received much scholarly attention in recent years, which examines its impact on news production (Dörr, 2016), its adoption in the newsroom (Wu, et al. 2019b) and ethical issues such as the accountability of authorship (Montal & Reich, 2016). Examples of NLG in practice can be seen at the Associated Press, who make of NLG to automatically generate articles related to financial reporting (Graefe, 2016), as well as weather (Ramos-Soto,
Bugarin, Barro & Taboada, 2015) and sports reporting (Gong, Ren & Zhang, 2017). This is consequently freeing up more time for journalists and editors (Anderson, 2011; Primo &
Zago, 2015; Wu, et al., 2019b) which then positions it as a benefit for journalistic practice.
Much of the recent research to emerge on automated journalism assessing its use and impact is primarily exploratory. Many make use of qualitative methods to dive deeper in the relationship between journalists and readers (Dörr, 2016; Montal & Reich, 2016) and its use by journalists and their reactions (Thurman, Dörr & Kunert, 2017; Wu, et al., 2019a, 2019b).
Conversely, much of this research focuses on the quality of the text generated and its varied range of applications, rather than how people may react if an author of a text is not human
(Henrickson, 2018).
Automatically written texts and readability. Henrickson (2018) looks at NLG systems from a human perspective, making use of Actor-Network-Theory (ANT), positioning algorithms as an agent rather than a tool. Henrickson (2018) argues that as natural language is normally enough to attribute beliefs and desires to an imagined author, it can be recognised as a communicative function of a text. These blurred lines between authorship show that when reading a text, a person normally expects a person to be behind it, but how they react when an author is a robot may change the way they react to such texts. The process of automated journalism involves some or no human intervention (Carlson, 2015), looking closer at the level of human input in their definition, interviewees in Wu, et al (2019b) raised concerns AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 19 over this, even to the point of whether if it will take over journalists’ jobs. Studies that have examined NLG’s perceived readability have found that when comparing algorithmically generated to human generated content, while not being fully identical is viable enough to be used as an alternative method to write news (Haim and Graefe 2017; Graefe et al. 2018; van der Kaa and Krahmer 2014; Lewis, et al. 2019; Wolker and Powell 2018) and that readers will equally prefer a human or robot author of a text (Graefe, et al., 2018; Jung, et al., 2017;
Wöker & Powell, 2018). Graefe, et al (2018) concluded based on experimental evidence of disclosing human and robot authors of sports and financial articles to participants that there is a trade-off between credibility and readability, as automatically written articles scored higher for the former and human written texts for the latter. They also argue that when the technology further develops, this discrepancy may close. Overall, it appears that texts made with NLG is viable enough for readers, what can then also be considered is then if editors themselves would then source content from a new tool that would go on to be published.
Addressing the strengths and limitations of automated journalism. Thurman, et al (2017) reported that journalists feel that NLG’s use cannot be extended beyond reporting factual information that would normally be uneconomical to cover, such as sports with less popularity, suggesting that human-like texts are not fully achievable (Thurman, et al., 2017;
Wu, et al. 2019b). At the same time, Thurman, et al (2017) mention six limitations of automated journalism, namely (1) it relies on single data streams, (2) the quantitative data it uses gives a one-dimensional perspective, (3) it is difficult to probe the data, (4) there is not much human perspective in the texts, (5) it is required to make templates of stories where news has to be predicted in advance and (6) it is difficult to creatively interpret the data in the templating process. Wu, et al (2019b) found in their interviews that there is uncertainty leading to resistance to automation in the newsroom. There is then also managerial pressure in its implementation due to economic factors as they believe the implementation of AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 20 automated journalism within the newsroom can reduce costs and improve workflows, related to the power of economic capital (Bourdieu, 2005).
With the introduction of AI into newsrooms, it can be inferred that adaptations technological changes are continuing, whereby the journalistic field must further adapt.
Furthermore, questions arise when it comes to automatically generated texts, particularly in terms of authorship and ethics (Montal & Reich, 2016) and establishing theoretical starting points (Henrickson, 2018). Collectively, the research described here primarily addresses automatically generated text that eventually goes on to be read by consumers (Dörr, 2016;
Montal & Reich, 2016), and their reactions to it as well as the development of the technology
(Graefe, et al., 2018; Jung, et al., 2017; Wöker & Powell, 2018). Research on journalists mainly describes their reactions to its use by the news outlet itself (Thurman, Dörr & Kunert,
2017; Wu, et al. 2019b), not necessarily how editors could take advantage of it to improve their workflows. There does exist a report conducted by journalism think-tank Polis (Beckett,
2019), which surveyed news outlets on their impressions of AI in newsrooms, in their interviews there is some openness to the use of AI in the newsroom. Aside from this, there appears to be very little research on this topic, especially in terms of sourcing content that is automatically written.
If it is established that readers are generally accepting of reading automatically generated texts (Graefe, et al., 2018; Jung, et al., 2017; Wöker & Powell, 2018), there then arises questions of whether editors sourcing content will react in similar ways. Collectively, despite the limitations discussed, automated journalism does provide efficiencies in some areas of journalistic work (Anderson, 2011; Primo & Zago, 2015). It also appears that automatically written texts can be accepted by readers and managers that want to create more efficient working practices (Jung, et al., 2017; Wu, et al, 2019b). Questions then remain whether such attitudes can also be applied other areas, namely if a new content discovery tool AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 21 were to provide automatically written text and whether this would be sourced by managers.
Therefore, it seems logical to explore whether editorial attitudes towards automated journalism can be extended to account for automated journalism and automatically written texts. If a new tool such as TCE would be able to provide automatically written texts, arguably then editors would have to be open to sourcing it for it to be used, as they may have an opinion on the trade-off between credibility and readability (Graefe, et al, 2018).
Therefore, it seems like a logical area of investigation as a solution to providing more cost- effective journalism and addressing the changing work routines journalists face (Bunce,
2017; Cornia, et al., 2020).
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 22
Method
Research Design The present study made use of semi-structured interviews of news editors to determine their workflows and how their attitudes towards automated journalism. The
Grounded Theory approach (Glaser & Strauss, 1967) was used to build up a conceptual framework of editorial behaviours and how automation and new tooling fits into this.
Grounded Theory refers to the studies of concepts, where theory is developed rather than tested based on information from participants. Such information can be organised into the development of codes (quotes from the dataset that display similar themes), categories
(groups of codes that display similar characteristics) and themes. From the identified codes and themes obtained from participants experiences are then “grounded” in the data (Glaser &
Strauss, 1967; Corbin, & Strauss, 1990). From this, assumptions can then be made by developing a theoretical framework where hypotheses can be tested. Rather than an inductive or deductive approach to research, it takes an abductive approach as it can integrate unexpected findings which can be incorporated when specifying relationships between themes (Bryant & Charmaz, 2019). This then aligns with the research question as it takes a qualitative approach, which aims to identify new and unexpected findings in relation to editorial workflows, which would not be achievable compared to a quantitative approach.
Data collection consisted of 12 interviews with various editors and former editors, who had knowledge of the editorial process within the newsroom.
Participants Participants were recruited in two stages. Firstly, purposive sampling was chosen, whereby a criterion was determined within the sample population to best answer the research question (Ritchie, Lewis, Nicholls & Ormston, 2014) to best reach the sample frame. Editors, and those with a journalistic background and knowledge of the editorial room were chosen to AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 23 be best suited for the criteria of this study, as they are exposed most to the content discovery process and determine the selection of news stories to be published, or will have knowledge of how this process works. Example job titles included in the sample frame include Editor,
Editor-in-Chief and Managing Editor. Secondly, snowball sampling was conducted making use of the professional network of an employee at TCE, this then gave further access to more eligible participants (Treadwell, 2017). From here, more participants were then contacted and invited for a voluntary interview. Participants consisted of 12 editors at, or who had previously worked at medium to large news outlets, 11 being in the Netherlands and 1 in
China. A participant summary can be seen by viewing table 1. All participants were aware of automated journalism when asked.
Table 1
Anonymised participant information
Participant News topics News outlet Nationality Job title Years Transcript covered type of outlet worked in appendix within journalism A Technology Online only Dutch CEO 36 B
B Regional Broadcast/O Chinese Senior 6 C news nline reporter
C Regional Online/Print Dutch Editor-in- 25 D news Chief
D Regional Online/Print/ Dutch Publisher/ 18 E news Broadcast Founder
E Local city Online/Print Dutch CEO and 15 F news Chief Publisher F General Online/Print Dutch Head of 19 G Communic ation /Former editor and content manager G General Journalism Dutch Teacher, 9 H school/Freel Freelancer and Editor AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 24
ance/Magazi nes
H General Newswire Dutch CTO 24 I
I General Online/Print Dutch Managing 22 J Editor
J Technology Online-only Dutch Editor-in- 26 K Chief
K Local city Online/Print Dutch Editor-in- 25 L news Chief
L General Journalism Dutch Teacher 12 M school/Natio and former nal broadcast Online Editor
Materials An interview guide (refer to appendix N) was created that addressed the key themes in the same structure as the literature review. The guide used an inverted funnel structure
(Treadwell, 2017), whereby specific themes could be first addressed with room address new insights while also adhering to the core line of questioning. A pre-test with one participant with a journalistic background was conducted prior to the main interviews to check that the line of questioning was understood and feasible. Demographics were first asked as icebreakers to which more detailed questions could be asked, with the aim to build rapport to ensure that honest answers were given (Arksey & Knight, 1999). The questioning first briefly addresses their work environment and the role of technological change within it, it then moves on to their approach to finding and discovering content. Probing questions were asked to trigger participants to talk openly about some topics, namely detail-oriented and clarification probes (Patton, 1990) to ensure richer data was gathered. In addition, memory cues related to theoretical concepts were included as a memory aid for the researcher to ensure that all relevant topics were covered (Arksey & Knight, 1999), and the interviews stayed focused on the research questions. This also reduces the risk of asking loaded AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 25 questions, decreasing the occurrence of confirmation bias. Consequently, this improved reliability by ensuring there was little interviewer interference in answer giving, only to ask probing questions to stimulate richer answers. For more sensitive issues, such as commercial versus editorial interests, indirect questions were asked to avoid the risk of socially desirable answers: “In recent years, do you feel that your organisation is balancing journalistic content quality and financial performance effectively?”. Additionally, if participants did not have much to say about a question, backup questions were added to each section to push them further for more information. A live demo and a mock-up of TCE was later shown to participants (see figure 1 and appendix P), which showed the basic elements of TCE’s user interface to give interviewees an impression of what automatic support may look like.
Questions regarding its usefulness for their sourcing routines were then asked, as well as how automatically written content may fit into this. To close and to gain any more final comments, a closing question was asked as a memory cue for participants to increase the chance of obtaining more data. The following question was asked: “Considering everything we have talked about, sourcing routines, technology, AI and automated journalism, where do you see it going in the future?”.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 26
Figure 1: Screenshot of a live demo and an example of a mock-up image shown to participants
Procedure Participants were contacted and asked for an interview, which took place via a video call. Informed consent and participants’ permission to record the audio was obtained prior to the interviews taking place. The shortest interview ran for 27 minutes, with the longest being
58 minutes, with an average runtime of 35 minutes. Following demographic questions and some basic questions regarding their job and previous roles, the main questioning of the interview was conducted. Once all the main questions were covered, the concept of TCE was explained and the live demo was shown using the screen sharing functionality in the video call. The demo had basic functionalities, namely being able to search for topics, viewing news articles and using filters. After this, mock-up images showing its proposed full AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 27 functionalities were shown. Questions were then asked regarding their opinion on new tools within their current work routines. To improve external validity respondent validation was implemented, whereby participants had the opportunity to see the categories to confirm what they said reflected the correct meaning (Ritchie, et al., 2014). Therefore, throughout the interviews, clarification questions were asked if needed to ensure correctly interpreted data.
Analysis After the interviews were transcribed, an initial content analysis took place to identify all quotes of interest. Such themes then formed the basis of the underlying framework as reported in the results section. The analysis came in three stages as laid out by
Corbin, & Strauss (1990). Firstly, open coding was used to select all quotes that were of relevance to answering the research question that could be potential concepts to include, from this a coding scheme emerged of which the categorisation of data could take place. Here, events, actions and interactions form categories and sub-categories. Secondly, axial coding took place to more carefully select and categorise the data into the themes that occurred the most within the data set, to select the most prominent categories to emerge. Finally, selective coding was conducted, which unifies all key categories into one core category. This then represents the main phenomenon of the study and summarises the answer to the research questions, it had to remain more abstract to ensure that the theory generated had higher applicability to editorial workflow processes. To improve internal validity, the constant comparative method was implemented, which involves reviewing newly coded data with earlier coded data to ensure that the concepts identified are consistent (Ritchie, et al., 2014), if a theme then later does not match with other codes, then it was reviewed again and if it deviated to far from the main analysis, then it was removed. This was done in conjunction with deviant case analysis, to ensure that any outlying cases were not forced into a category
(Ritchie, et al., 2014). AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 28
Results
The current state of content discovery The results provide a snapshot into the current state of editorial workflows to discover new content within the newsroom. The interviewees provided an overview of what editors currently do in terms of sourcing and discovering new content. The main techniques to emerge were the use of journalists’ own personal network of contacts, searching and monitoring information online, and use of traditional sourcing such as press releases. Such methods can be seen by viewing Table 2. Participants expressed that finding news was a constantly flowing process, four participants were however uncertain of how it could be improved without a clear solution presented to them. Such explanation of this is detailed in the following sections.
Table 2
Sourcing methods mentioned by participants
Sourcing method identified Number of participants who mentioned using it Traditional sourcing 4 Use of personal networks 7 Social media 9 Searching online 7 Newswires 2 Current editorial routines Firstly, addressing RQ1: what editors are currently doing in terms of sourcing content. The results revealed that it consisted of a mix of methods to discover newsworthy content, both offline and online.
Use of personal networks. One major theme to occur in terms of what editors are currently doing, is their high reliance on networks of personal contacts in the areas they report in. This entails getting in contact with who they know in the region, who may lead them to a story. Such methods were expressed by seven out of the twelve participants.
Participant A expressed that who they know is the most effective way for them to find AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 29 content: “Everything we hear within our network is way more important in our heads than things that occur outside of our network. If somebody within our network would come up with a suggestion, this would lead to a story way earlier, way sooner and faster than just an email or a message, or a robot that is suggesting something”. In a similar vein, participant C expresses it as a starting point: “First of all, stories come from people that live in the neighbourhood and all the journalists in our company live in the province, and the best stories come from the people themselves”. For participant B, this was reasoned to be as part of their focus on quality journalism and getting to the real stories, coming from the people themselves. Participant B expressed that: “We almost exist for 250 years; the sources were institutional. So, it was the community, it was the politics that provided the articles. But we made a shift towards the stories of the people, the true people”. Participant E, while saying that they also use the traditional methods such as press releases expressed a similar view:
“The main sources for unique content are people in the city who knows us. We have a network; journalists have their networks and it is our main source. These are the main sources and it might also be follow-ups of articles we have already printed”. This then paints a picture that reliance on personal networks is a key starting point for many outlets and plays a central role within their current routines to discover newsworthy content.
Searching and monitoring information online. The same participants running locally based outlets mentioned that despite their insistence of using personal networks, they do also turn to searching online for information, a view that is also shared by eleven interviewees. This however comes in different forms, namely searching online or monitoring social media. Many participants used tools such as Tweetdeck or Hootsuite to keep track of what was happening on social media. Participant D explains: “We have many tools that can find news for us, like Coosto. I think 30 to 40 percent is collected by online sources, and then being checked” and participant E explaining that “we use tooling for that to make a scan of AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 30 all the content which is there”. Such tools are utilised in different ways for all participants, for monitoring the web to find something newsworthy.
Other ways to search online include Google and Reddit. Interestingly, participant G mentioned Reddit frequently, and that it was a useful place to visit to find information. When asked on how participant G used Reddit for finding information, they said: “Well mainly because it gives me some ideas of how other people are looking at subjects. So I will, I am obviously still quite young… but sometimes I heard something and I think ‘oh my god that’s weird’ or ‘that’s strange’ and go to Reddit to see how other people discuss the topic because as you are probably familiar with there’s a lot of different opinions on Reddit”. It then appears that current workflows then consist of a mix of offline and online information gathering, which requires constant monitoring of the web and their networks to stay on top of what is newsworthy.
Use of traditional sources. Aside from finding information online, some participants still make use of traditional channels, such as press releases and institutional sources. Four participants mentioned that they make of traditional sourcing techniques, it again appears to be secondary to using personal networks as participant E explains: “Well obviously, all the organisations which are sending us press releases, things like that. That is one. But the main sources for unique content are people in the city who knows us”.
Participant K adds: “The old institutions are important for us as well, like municipalities, the airport, or the bigger companies”. Participant J reports on technology news and relies on traditional sources much more: “I would say that at least 50 percent or maybe two thirds is good old-fashioned press release and so, stuff that PR people or divisions send to you. And I would say that it is not just the press release but the whole publication, from PR to journalism. That also accounts for a call you have with someone, a spokesperson from a certain company. All that kind of contact, that communication, that’s a pretty big chunk”. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 31
Integration of automatic support The second main category of the current state of supporting editorial workflows is the factors themselves that can initiate such a process. RQ2 asked whether editorial workflows can be automatically supported. The results did show some willingness from editors, this does depend on two factors as described below.
Factor 1: New tools must integrate into current routines. Firstly, a prominent factor to emerge was that having a new system to discover content should integrate into what they currently do. When shown the demo, participants were asked about how it could integrate into their current sourcing routines, seven participants expressed that integration into their other workflows was important, with participant B saying: “I think it’s useful because no one can really have enough information”¸ with participant L adding: “What I saw working at {previous company} I had several screens, I had social media, RSS, ANP, so you just skip, you look through it, and you scan, you pick up what you are interested in”. This then suggests that a system that facilitates the automatic discovery of new content could well fit into current routines, providing that it can smoothly integrate with other tools that editors are currently using.
Furthermore, a prominent sub-category affecting the integration into current routines was the system having familiarity with other tools editors use. Four respondents mentioned that the design and layout being familiar to other tools, such as Tweetdeck or ANP was highly beneficial. Participant A commented: “That helps of course it’s something that I already know, and I know how to use, so that’s clever I would say. It is also similar, if you look at my Tweetdeck then you’ll see the same with the columns of topics that I need to follow”, with participant G adding: “Yeah, I think if it looks familiar, that’s nice definitely for journalists because I have colleagues who still find it hard to work with new technology”.
Participant L had previously been involved in developing a similar tool at a previous news outlet and reported that: “When you make it simple, it is easy to use and you can browse AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 32 through it and then move on to your next screen right? That’s how {previous company} uses it in the end. When I talked to IT people, they show you everything that is possible, I was like
‘woah this is great’, but if you bring it to the floor, they don’t care. They want it to work quickly and be easy to use because every bit of more difficultly takes more time to learn and that’s the holdup”. When asked if simplicity was then key for new tools, participant L elaborated: “Make it as simple as possible, and the end of the day, we made something that looked like ANP. Because they knew that, they were familiar with it, they said ‘oh I can relate to this’”.
Factor 2: New tools need to address existing problems. Secondly, when new tools to facilitate content discovery were bought up to participants, all participants gave a mixed view overall of the system. However, two were unsure of whether it had any advantage to them. In total, four acknowledged that while the system shown to them looked promising, it had to solve a clear problem for them to see the benefits of its use, with participant A having a lot to say on this: “It is difficult to imagine a situation where things would be twice as efficient or twice as fast. If you are offered something and somebody can show me a way to be twice as efficient, then yeah that would be different”. When asked to elaborate on new tools being used within their company’s current work routines participant A added: “Of course they can be improved, but you know I think that is only human. You are satisfied with what you have, what you don’t have you don’t know. If you are of course only looking for specific topics, if you are a general journalist then that might be worthwhile, but still it is, if only you would look at your Tweetdeck or your Hootsuite, or newsfeed. You can be inspired in tons of ways, I would say. You would have ask yourself what problem do you solve with that, what you need, and sometimes it is nice to see something else, by accident or, but will this be a reason to join TCE?”. Participant K simply said after being introduced to the demo:
“Yeah I know about the idea roughly, at this stage, I don’t have an opinion yet. I mostly have AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 33 a lot of questions”. Participant K was then curious about how the demo worked and who it was aimed at.
Conversely, two of the interviewees had less of an abstract view on this, suggesting that such problems will depend on how a new system delivers and what content it offers.
Participant E expressed that a tool to find content would have to match their outlet’s needs:
“The thing that might depend on what platforms they publish on, or it would help if they know it’s a unique story. We like to have more background stories, a bit more intelligence, like NRC or like that, and if you could add something like that, you know if its our profile, then it would help”. Participant L adds an additional angle on this, saying that the type of content can also be something to address: “I make a phone call for 20 minutes and I have a new story to write, but if I want to make a video for 20 minutes, it is going to cost days. So, it is more expensive, so if that is what they want. So, if you can deliver, let’s go”. Collectively, while some participants may not know what problems they have to integrate new tools into their workflows, whether its content type or medium there are certain problems that a new tool has to address beforehand to ensure it integrates into editorial work routines.
Facilitators of adoption of automated journalism RQ3a asked editorial attitudes towards automated tooling. The results showed positive signs that automated journalism and automatically written texts could be included in the newsroom, this then depends on several factors that can facilitate this process.
Editors have an open attitude towards automated journalism. Participants suggested some willingness towards the use of automated journalism, addressing RQ3a. This viewpoint was expressed by five participants. Participant C was very open to automating certain tasks within the newsroom, expressing that: “I think that all parts of the work that could be done with automation that should be done like that. There are of course parts that are expensive, and that some point they can do more than a computer can. They are creative AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 34 and they can find those subtle human elements and because we have every year fewer of them, while there is more work to be done, they have work to the automisation (sic). I am very open and positive to it. I think in the world there are now many examples that shows us that it can work”. Participant I adds: “I think it could be beneficial, because we are working with it in a test setting and especially for the football and following the smaller leagues. But I think they have to do lots of work to get it to function well and come to a publishable text”.
However, as the above quotes demonstrate it does again depend on topic, with them expressing it is only useful for certain topics and factual information reporting. Participant E expressed more of a neutral view, saying: “I don’t really have an opinion about it, it’s more does the reader like it? That is most important for me, I guess with sports I think well, it might work, a bit. But there is also an added value of the editor, I guess. So, in the beginning it should be checked maybe, to see if the quality is good enough, and after that when all the algorithms work et cetera, I guess it could be a solution, but not the solution, because it misses the feeling which is also important in an article. Instead of only needing statistics, I guess sports journalism now, for robo-journalism would mainly be statistics”.
Editors are willing to source automatically written content. Interviewees expressed some openness to sourcing content that was automatically written, with nine expressing positive views towards sourcing it, addressing RQ3b. This shows promising signs in terms of providing automatic support to their current routines, however it does depend on some factors such as quality and source. The use of it for reporting factual news remains and it not being able to replace human writers was also frequently mentioned. Participant C when asked if they would consider sourcing automatically written content said: “Oh that wouldn’t make…it could make the difference if about what sort of content it is. Automatically written could be just small pieces in your newspaper or your site”, with participant E adding: “I don’t think it makes any difference, in the process of buying things or not buying things, AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 35 whether or not it has been written by a person or a robot. Considering that fact, that the quality of the articles should be perfectly the same”. It then appears that editors do then seem to be somewhat open to automatically written content, with one criterion being that it can maintain the same standards of content that is written by a person. Conversely, such willingness is not clear cut for all participants, and subsequently will depend on how well automatically written content can match human written content, or its originality. Participant
G commented on this by saying “I think it is all about the quality of the article. So, if it is good, I don’t see a reason why not. But I do wonder whether these types of articles show any originality. That is often expected from journalists”. Such a view was echoed by participant
B: “It could work if the quality is ok and I guess it is not easy to make the quality overall, good enough. But at the end of the day it is simple as that really good stories are shared there. It is not very good, or it does not match the standard of the news organisation, then it’s difficult, not going to work”.
Barriers of adoption of automated journalism The results also reveal that there are additional factors that are hindering the adoption of automated journalism. These can be considered as challenges that have to be overcome to ensure that if automatic support were to take place, addressing RQ3a and RQ3b, which asked whether editorial attitudes towards automation and new tools that could incorporate automatically written text.
News outlets are slow to react to technological change. The first factor that was frequently mentioned by three of the participants was that news outlets were slow to adopt to new technological changes. This was reasoned due to apprehension, or there not being a cost- effective way of implementing automated journalism within the newsroom. Participant A commented that: “Journalism hasn’t adopted a fraction of what could be useful to journalism. If you look at the field of Artificial Intelligence and of machine learning, robot- AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 36 journalism. But also, VR and AR. Those are technologies that can be way more useful for journalism and for the audience of journalism that can be used right now”. Participant L when asked about technological changes within the newsroom expressed that: “The working field has a place where mainly the focus is on the now, we got to fill this site, paper has to be filled. So, you talk innovation within media companies, its complicated, difficult, and if you look at innovation, for instance {previous company}, after I left, they started {previous company name} lab. So, they started using tools online for their news product, and that’s it.
But something as disruptive as the automated newsroom, robotics, that’s difficult to grasp for them, and slowly I think we are seeing what field can adapt to that, but its slow, like always in journalism”. This then signals that while new technologies can be promising, an organisation’s adoption of it to improve workflows can then be a slow and uncertain process.
There is a need for more technological development. Eleven out of the twelve participants operated in the Netherlands. Three participants expressed that the reason that technologies such as NLG has not entered newsrooms yet comes down to lack of algorithms that support the Dutch language. They acknowledge that a lot is being done in other markets such as the United States in the English language, however they say that in terms of using automatically written text, the technology simply needs to arrive in the Dutch market.
Participant H frequently mentioned this, saying: “There are similar things in the Netherlands, but the ones I know are almost all template-based, with some variety with synonyms and things to make it a little more dynamic, but they are not really natural language generated articles. The Netherlands has a small market anyway, so if you are promoting such technologies then it’s more difficult to get a return on investment. But to really generate text via computers in the Dutch language, you do not see that too often. I know it is being used in the English language. Our language area is too small for that currently, or of course will develop. But it takes longer than in English”. Participant J simply adds: “Here in the AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 37
Netherlands we need an algorithm that can write Dutch. I am not aware of anyone who actually…there are some experiments at {parent company} with bots, but that’s it”. This then signals while there are some things being done regarding automated journalism, the technology surrounding automated journalism and NLG may need to develop further for some to considered a to be a bigger part of their workflows. Such a view was echoed when asking participants where they see automated journalism going in the future, as many were uncertain to what extent it will become integrated within newsrooms. Addressing RQ3a, while many viewed automated journalism as promising, when asked about where it could go in the future, nine participants were unsure. Participant K commented that: “It’s still a bit vague for me how we can use it, and we are trying to find out where we can use it in our process, and well its still, I am hearing about it for years and years and now it’s still, well at the point where I can use it for this, or I can use it for that. So, I am not sure”. Participant L adds: “It is a difficult one to predict, I am looking at this from a positive point and saying, we can fill {previous company website}, NU.nl by an automated newsroom. So, for the near future, 5 to 10 years, you will see an automated newsroom, but partly filling NU.nl and
{previous company website} and after that, it remains to be seen and I hope in the meantime, we start a very good debate about robot-journalism in an automated world”.
Editors are stubborn. Another factor that may be a roadblock to adopting new tools within sourcing routines is the editors themselves, five interviewees expressed this viewpoint, explaining that a barrier in gathering content from another tool to support them could then intervene too much with their own way of working, and not wanting to change that. They mentioned that in the discovery of finding new content, some editors may be too stubborn to use new tools. When asking participant E about how new tools could integrate into their current sourcing routines, it was expressed that: “They are in a way persistent, they know what’s best, nobody should tell them how to make an article, what’s best, et cetera. So, this is AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 38 something from outside (referring to the demo), and they always like to work with their own freelancers, with their own ideas, and something from the outside. Always when we talk with someone from the outside, they kind of hold and reject it”. With participant I adding: “I know this kind of dashboard, from other companies, and my experience is that editors, that there are editors who might use it. But there is very individual choice for editors, as I already told you, everyone has his or her own workaround, and it is difficult to change those workarounds they are used to”, also adding that editors “go their own way, they know what’s best” when it comes to finding content. Furthermore, participant C, somewhat confirmed the above quotes by saying: “When I’m only a reader its ok for me when someone chooses. When I am a news organisation, I want to make the choice and I don’t want to have it done by an algorithm”.
News outlets must balance their available resources. A theme to emerge when asking participants about their current business models was that the discovery of new content then may be limited by lack of resources available to do so, such as time or finance, signalling that a new tool could fill in a resource gap for some news outlets, but it could be then held back by a news outlets’ own willingness to dedicate resources to doing so.
Participant A expressed that “The thing of course is I am only able to pay my reporters a certain amount. I am not able to pay them for doing research for like a week or two weeks and then come up with a story. That is the limit, but it is also the limit we accepted to have.
Of course, I would prefer to have all my reporters go away for two weeks and then come back with the best story ever. But it is not feasible, but they have to write one or two stories every day and I can’t afford to give them more time”. Participant L stated that this comes down to producing multiple types of content, remaining optimistic that a new tool could solve this issue: “There might be an opportunity, I know the main focus is on video. If you look at
{large media company} because they are able to write stuff, right? They have had 100 years for that. But they want more video content and the video content is expensive to make. I make AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 39 a phone call for 20 minutes and I have a new story to write, but if I want to make a video for
20 minutes, it is going to cost days”. One particular result within this category was that of participant F, who had previously worked at a regional news outlet included in the sample.
Participant F explained that “You would think they would cover the whole of {region}. But actually, if you are going to analyse their news, its mostly the area of {city}. But it is {city} because they work and live there, the people who work there. Because they live there and that is how they consume their media and therefore that is how they see the province, but it is much bigger than that”. Participant F added that many local news outlets often look to each other when trying to find new content: “If you work for the local media, then your source, you look in the local newspapers. They all look to each other actually, they all do. That is what I see and now as well. If I make a story now with a journalist, I can see three or four calls from other journalists. So, what they do is they all look at each other. It would be the best answer to say because that is how they teach it, is ‘ I am going into the community and I speak with people and I am going to find my news’, but to be honest that is mostly not how they work, because they don’t have the time to do some research or invest, or just hang out in the city and see what is going on. So mostly, newspapers are generated by what they receive”. This then signals a potential reach problem in terms of reporting on all aspects of the news within that region that a particular outlet reports within, which suggests that then maybe there is space for a new tool to fill in this gap. However, if respondents are self- reporting that they in-fact do go out within the community to find stories, then they may not realise that they have a problem.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 40
Discussion & Conclusion This study aimed to investigate the current state of editorial workflows and how they can be automatically supported, accounting for their attitudes towards automated journalism and automatically written texts. The model presented in Figure 2 is the result of the grounded theory process, can combines knowledge gained from the literature and the new insights obtained in this study of how editorial workflows related to discovering content can be supported. The current state of automatic support can then be broken down into four main categories, with tools for finding content as well as how automated journalism may fit into this picture. There are two interrelated categories generated, what editors are currently doing in terms of discovering content and the factors that could then facilitate the automatic support of this, addressing RQ1, which aimed to look at what editors currently do in terms of sourcing content and RQ2, which asked how these workflows could be automatically supported. These two categories are then both effected by two other categories – facilitators and barriers to adoption of automated journalism, which addresses RQ3a and RQ3b, which examined editors’ attitudes towards automated support tools and automatically written content.
This study has then provided an up-to-date snapshot into what editors are currently doing to find new content and how this can be currently supported and adds onto previous research examining the reactions that journalists have towards automated journalism (Dörr,
2016; Montal & Reich, 2016; Thurman, Dörr & Kunert, 2017; Wu, et al., 2019b), accounting for automatically written content and how this could integrate into their current workflows.
Previous literature has described the problems journalists have faced in the newsroom
(Graefe, et al., 2018; Henrickson, 2018; Jung, et al., 2017; Wöker & Powell, 2018), this study has then attempted to address such problems via investigating the use of automatic support with new tools to address time pressures news outlets face (Kruikemeier & Lecheler, 2016;
Sariodou, et al., 2017). By mapping out the current state of content discovery routines, and AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 41 how they can be automatically supported, this can then give an overview of what is currently done and what can be done. On the other hand, it has also highlights some barriers to adopting automated journalism, that signal that such embracing of new technologies, despite the results showing open attitudes towards automated journalism and the use of automatically written texts, is not a clear-cut process.
Figure 2: Conceptual model describing editorial workflows and where automatic support can integrate with them.
The state of current sourcing routines Current editorial work routines, consisting of a mix of methods to discover content, reporting that they rely primarily on using their personal networks to lead them to new stories, alongside searching online for information in addition to the use of traditional sources. This suggests in terms there is a mix of unstructured and structured sourcing
(Kruikemeier & Lecheler, 2016) with and aligns with the view that traditional sourcing still has its place within news outlets. The difference that emerged here was the balance of such AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 42 methods, in contrast to the heavy reliance of online sourcing reported in other studies
(Boumans, et al., 2018; Broersma & Graham, 2013). This result could be attributed to the nature of the sample, being made up of primarily locally based news outlets. The use of
Reddit highlights how journalists are seeking news ways to uncover as much information on a story as possible, to confirm facts or gain knowledge about an event. It is also worth noting that only two of the participants used a newswire, this seems to contradict other research that indicates that newswires are commonly used (Boumans, et al., 2018) this finding could then be attributed again to the nature of the sample, who do not need nation-wide news content provided on newswires. What is interesting to add is that contrast between what one outlet was reporting to have doing and what it does in reality, in terms of not covering the entire region that it claims to. Although this was the opinion of one interviewee who had worked at this outlet in the past, this does then highlight one of the pitfalls of asking questions directly to respondents, self-reported versus actual behaviour (Lavrakas, 2008). On the other hand, it does add some weight to claims that outlets are facing time pressures and may not have resources to perform (Sariodou, et al., 2017) if they cannot achieve full coverage of the region they report in, which then shows signs that support via new tools is needed.
Moving towards integration of automatic support The results also produced factors that indicate signs that automatic support can be integrated, if new tools can be successfully combined with current editorial workflows, and as such solve a problem and create more efficient work routines for editors, then it can be assumed that automatic support can be facilitated. Subsequently, the TCE demo shown to participants added a unique way to illustrate what was meant by automatic support.
Additionally, opinions on automated journalism and sourcing automatically written text seem somewhat positive but reflects established concerns such as content quality and source
(Beckett, 2019), which can be seen when compared to previous literature on automated AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 43 journalism (Thurman, Dörr & Kunert, 2017; Wu, et al. 2019b). This study therefore reiterates the feeling that many journalists feel that the quality of NLG should be there first before they consider it for adoption.
There is indication that systems should feel familiar to ensure that they can be quickly adopted into their current workflows. This suggests that if the system looks somewhat similar and can act as another destination for editors to visit aside from searching online and checking social media, improving the likelihood of adopting a new tool. However, the barriers first need to be overcome, as with many technological advances it can likely be assumed that this will take time. The current research adds onto the exploratory nature of the literature on automated journalism (Dörr, 2016; Montal & Reich, 2016; Thurman, Dörr &
Kunert, 2017; Wu, et al., 2019a, 2019b). On the other hand, combining this result with the impression that editors are open to automated journalism then shows positive signs that new tools that help them automate their work routines could then be welcomed. What is added onto this body of exploratory research is that it accounts for content discovery routines, by showing signs that editors are open to tools that use automatically written text. Conversely, as much of the literature has already described, the same challenges remain related to text quality and topic (Thurman, et al., 2017; Wu, et al. 2019b), with the results showing the same opinions related to it being only beneficial for factual news and it not quite being able to match human written texts (Graefe, et al., 2018). This echoes the limitation stated by
Thurman, et al (2017), whereby automated texts lack human perspective and reiterates the importance of machines as communicators and how they must function accurately to be accepted as authors themselves (Henrickson, 2018). Furthermore, considering the doubts that the technology is not quite there yet, this study then takes the same position as Graefe, et al.,
(2018), being that as NLG technology develops then such concerns about its effectiveness may lower. The previous literature described that automatically written text was viable AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 44 despite being fully identical to human written texts (Haim and Graefe 2017; Graefe et al.
2018; van der Kaa and Krahmer 2014; Lewis, et al. 2019; Wolker and Powell 2018). This body of literature then examines its impact for the reader, the observations made in the current study may suggest that editor behind the selection of articles may be more sensitive to the finer details of how text is formulated in terms of readability, and may not want to make any trade-offs between readability and credibility (Graefe, et al., 2018).
Addressing the facilitators and barriers to automated journalism The evidence of time pressures and related findings to outlets needing resources to adopt new technologies can be interpreted in two ways, which could be compared to the pressures to perform within newsrooms (Bunce, 2017; Kruikemeier & Lecheler, 2016).
Firstly, that editors are open to a new tool to facilitate content discovery because they need a more efficient way to find content. Secondly, they do not have the time to investigate new solutions, which is holding back their potential to create more efficient workflows.
Subsequently, it could be assumed that news outlets are focusing too much on the ‘now’ and not so much what potential future technological developments. This then presents a mixed view of automated journalism in the newsroom, more so for NLG. While the interviewees did see it as promising, in line with previous research positioning it as a benefit (Anderson, 2011;
Primo & Zago, 2015; Wu, et al., 2019b) as there are attitudes reported here that show willingness to include it in the automation of any tasks that could be automated to save time, conversely this is difficult to do so if their companies are slow to adopt such technologies.
Such observations can be compared the existence resistance to automation within the newsroom (Bunce, 2017; Wu, et al., 2019b), wanting to still use your own workaround may then indicate resistance to new tools, especially if some editors could show stubbornness towards them. Unexpected findings emerged related to automated journalism, whereby many feel the technology must develop further, while the results did indicate that editors felt that AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 45 human-like texts are not quite achievable yet (Thurman, et al., 2017; Wu, et al. 2019b), what differed from previous research was that for the Dutch market, NLG has a long way to go to be fully utilised simply because they have yet to see an algorithm that can write Dutch. This finding could then be attributed to why editors seemed sometimes uncertain to NLG, it could be reasoned that they could not comment on it any further than this and their opinions on it did not appear to be fully formed. It appears that at least with the Dutch market, NLG needs to be built-upon further and outlets must take steps to implement such technologies, it seems to be for many a waiting game to fully realise the potential of automatically written news articles.
The barriers to adoption of automated journalism displays conflict of economic versus cultural power (Bourdieu, 1998; 2005). In terms of cultural power, editors wanting to keep to their own routines and being stubborn towards new solutions can be related to their habitus and how they feel that their role should be performed. On the other hand, news outlets being slow to adopt to new technologies and experiencing time pressures indicates that economic power is present, as new technologies must be financially feasible for them to fully embrace them. Much of the research on NLG systems is focused on how the technology has developed (Haim and Graefe 2017; Gatt & Krahmer, 2018; Graefe et al. 2018; van der Kaa and Krahmer 2014; Lewis, et al. 2019; Wolker and Powell 2018) as well as the emerging body research investigating its role with the communication process (Domingo, Masip &
Meijer, 2015; Henrickson, 2018; Primo & Zago, 2015), it could be reasoned that such research is exploratory can come down to the fact the technology itself does still need to develop. This viewpoint could be compared with studies conducted in the earlier years of media convergence as it was emerging, which paints a similar picture. The recommendations of Klienenberg (2005) concluded that digital systems were in early stages of development, AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 46 with it being difficult to predict how they will develop in the future. In terms of automated journalism and NLG, such research could then be at a similar stage.
Collectively, this research has described an uncertain view of where automated journalism can go moving forward, and it could be a case of waiting to see how the technology develops for many news outlets. On the other hand, it does show signs of openness towards using adopted journalism and the use of automatically written text in the future. This is evidenced by the uncertainty that the interviewees share with automated journalism going forward and not knowing what new tools must do to improve their workflows. Looking at the conclusions of Corina, et al (2020), that commercial and editorial departments must collaborate further to adapt to new changes within news organisations.
Based on the results here, this view can be reiterated in the present study, as it appears that the adoption of new tools could then require inputs from both departments. If news outlets want to overcome their time pressure and resource problems and have room for new tools, then it seems like a logical step for all departments to collaborate to bring in new tools to improve workflows. The results here show signs that at least from an editorial perspective, there is some willingness to be open to the use of new tools and technologies. If they can more efficiently discover content, it could go some way to overcoming issues such as finding content, reaching new stories, and attaining more cost-effective journalism. For the developers of such tools, such a message then needs to be communicated, by clearly indicating such benefits new tools may have, then news outlets may become more open to such solutions.
Limitations and recommendations for future research The present research is not without its limitations, 11 of the 12 interviewees worked for Dutch news outlets, subsequently the observation that NLG technology still needs to develop could be then attributed to the fact that there is no algorithm that can write Dutch AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 47 text. It could then be inferred that if this study were to be conducted in another country where automated journalism is used more, such as the US, as the examples from the LA-Times and
AP highlight (Graefe, 2016; Walker, 2014), the results could have differed and more informed opinions on NLG could have been formed. It is difficult to fully comment on a technology that you are not exposed to yet, or in the case of the interviewees sampled here, only hearing about experiments done at their parent companies. Another limitation then concerns the sampling strategy, another explanation for the findings is that the second sampling stage used snowball sampling and sacrificed some researcher control (Treadwell,
2017) over the purposive sampling selection criterion, relying upon a network of another person could then explain by the sample was made of a majority of locally-based news outlets. Additionally, the content discovery platform used was still in early stages of development, with limited functionalities. The present study could have benefitted from showing interviewees a fully working system to ensure they can gain a better impression of what new tools to facilitate content discovery could look like.
With these limitations in mind, future research could then replicate this study, but in other countries, potentially even with use of automation already within their newsrooms. In addition, considering that technology adoption is a long process, as evidenced by the literature on media convergence (Chada & Wells, 2016; Deuze 2008; Deuze & Bardoel,
2001; Klienenberg, 2005; Robinson, 2011; Ruusunoksa & Kunelius, 2007; Saltzis &
Dickinson, 2008), future research could also benefit from a longitudinal study, collecting data at various points in time can provide insights on phenomena as they are occurring (Ritchie, et al., 2014). For example, as automated journalism technologies develop or are considered for use in newsrooms. Being able to see how workflows change over time with the introduction of new technologies could then add more insights into how automation is being used within newsrooms, with the consideration in mind that the results here indicate that editors are AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 48 currently unsure about automated journalism’s use as of yet, observing this over time could then be beneficial in examining its place in the newsroom as it develops.
An interpretation of the finding that new tools must then focus on solving a problem that editors may or may not know they have to integrate into current work routines. This then still raises questions as to what problems exactly need to be solved in order to integrate new content discovery tools into editorial workflows. The interviewees were unclear as to what exact problems they face, although based on the results here it can be speculated it can come down to time pressures and having financial resources, future research could then dive deeper into what these problems exactly consist of. For example, by giving participants the opportunity to use a new tools that have full functionalities, compared to showing them a demo as was done in this study, then more concrete conclusions can be made about what problems new tools need to solve to facilitate their content discovery processes.
This research was conducted with the showing of a live demo and mock-ups of a new content discovery tool. Based on the observations here, there are some practical implications not only on the side of those developing new tools but also for news outlets themselves. Firstly, a new content discovery tool needs to ensure that it clearly communicates where it fits into current editorial workflows, and how technological aspects such as automatically written texts can create more cost-effective journalism. This then may address the issues some interviewees had with not knowing if they had a problem that a new tool can solve, presenting it clearly then could be a solution to overcome this. Secondly, companies behind new tools should then collaborate with their potential customers. If news outlets are facing time pressures to an extent that they cannot afford to evaluate or accommodate new tools, then guidance or helping them transition through such a process can ensure that they see the benefit and subsequently foster a transition where barriers to automated journalism can slowly be taken down. The results here, namely the categories that describe the AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 49 integration and automatic support and barriers to adoption of automated journalism can be used as a guideline to anticipate issues that news outlets may have in advance. Previous literature does acknowledge the benefits of automated journalism (Anderson, 2011; Primo &
Zago, 2015; Wu, et al., 2019b), benefits of which the results here also back up, therefore it seems reasonable to assume that news outlets are becoming more open to it, also evidenced by interviewee comments that there are experiments being done within their parent companies. However, from a perspective of a new tool aiming to facilitate content discovery, with the use of automatically written text, this convincing may need to come from the developers of such tools.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 50
References Arksey, H., & Knight, P. (1999). Interviewing for Social Scientists. An Introductory Resource with Examples. Los Angeles, CA: Sage Publications. Anderson, C. (2011). Deliberative, agonistic, and algorithmic audiences: Journalism’s vision of its public in an age of audience transparency. International Journal of Communication 5: 529–547. Beckett, C. (2019). New powers, new responsibilities: A global survey of journalism and artificial intelligence. Retrieved from https://blogs.lse.ac.uk/polis/2019/11/18/new- powers-new-responsibilities/. Boumans, J., Trilling, D., Vliegenthart, R., & Boomgaarden, H. (2018). The agency makes the (online) news world go round: The impact of news agency content on print and online news. International Journal of Communication, 12(2018), 1768–1789. Bourdieu, Pierre. 1993. “Field of Power, Literary Field and Habitus.” In: The Field of Cultural Production: Essays on Art and Literature. Johnson, R. (ed). 161–175. New York: Columbia University Press. Bourdieu, P. (1998). On television. New York City, NY: The New Press. Bourdieu, P. (2005). The political field, the social science field, and the journalistic field. In: Benson, R. and Neveu, E. (eds) Bourdieu and the Journalistic Field. Cambridge: Polity. Broersma, M., & Graham, T. (2013). Twitter as a News Source: How Dutch and British Newspapers used Tweets in their News Coverage, 2007–2011. Journalism Practice, 7(4), 446-464. Bunce, M. (2017). Management and resistance in the digital newsroom. Journalism, 20(7), 890-905. doi:10.1177/1464884916688963. Caswell, D., & Dörr, K. (2018). Automated Journalism 2.0: Event-driven narratives: From simple descriptions to real stories. Journalism practice, 12(4), 477-496. doi:10.1080/17512786.2017.1320773. Case, D., & Given, L. (2016). Looking for information: A survey of research on information seeking, needs, and behaviour. Bingley, UK: Emerald. Chadha, K., & Wells, R. (2016). Journalistic responses to technological innovation in newsrooms: An exploratory study of twitter use. Digital Journalism, 4(8), 1020-1035. doi:10.1080/21670811.2015.1123100. Corbin, J., & Strauss, A. (1990). Grounded Theory Research: Procedures, Canons and Evaluative Criteria. Qualitative Sociology, 13(1), 3-21. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 51
Cornia, A., Sehl, A., & Nielsen, R. K. (2020). ‘We no longer live in a time of separation’: A comparative analysis of how editorial and commercial integration became a norm. Journalism, 21(2), 172–190. doi:10.1177/1464884918779919. Dal Zotto, C. & Lugmayr, A. (2016). Media convergence as evolutionary process. In: Lugmayr, A. & Dal Zotto, C. (eds) Media Convergence Handbook, vol.2: Firms and Users Perspectives. Heidelberg: Springer, pp. 3–16. Deprez, A., & van Leuven, S. (2018). About Pseudo Quarrels and Trustworthiness: A multi- method study of health journalism, sourcing practices and Twitter. Journalism Studies, 19(9), 1257-1274. doi:10.1080/1461670X.2016.1266910. Diakopoulos, N. (2014). Algorithmic Accountability Reporting: On the Investigation of Black Boxes. Tow Center for Digital Journalism Brief. https://academiccommons.columbia.edu/doi/10.7916/D8ZK5TW2. Diakopoulos, N. (2019). Automating the news: How algorithms are rewriting the media. Cambridge, MA. Harvard University Press. Diakopoulos, N., & Koliska, M. (2016). Algorithmic transparency in the news media. Digital Journalism, 5(7), 809-828. doi:10.1080/21670811.2016.1208053. Dörr, K. N. (2016). Mapping the field of algorithmic journalism. Digital journalism. 4(6), 700-722. doi:10.1080/21670811.2015.1096748. Domingo, D., Masip, P., & Costera Meijer, I. (2015). Tracing digital news networks: Towards an integrated framework of the dynamics of news production, circulation and use. Digital Journalism, 3(1), 53-67. doi:10.1080/21670811.2014.927996. Deuze, M & Bardoel, J. (2001). Network journalism’: Converging competencies of old and new media professionals. Australian journalism review, 23(2), 91-103. Deuze, M. (2008). The changing context of news work: Liquid journalism for a monitorial citizenry. International journal of Communication. 2(18). Epstein, E. (1974). News from Nowhere: Television and the News. New York: Vintage Books. Fishman, M. (1980). Manufacturing the News. Austin, TX: University of Texas Press. Gans, H. (1979) Deciding What’s News: A Study of CBS Evening News, NBC Nightly News, Newsweek and Time. New York: Pantheon. Gatt, A., & Krahmer, E. J. (2018). Survey of the State of the Art in Natural language Generation: Core tasks, applications and evaluation. Journal of Artificial Intelligence Research, 61(1), 65-170. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 52
Gong, J., Ren, W., & Zhang, P. (2017, May). An automatic generation method of sports news based on knowledge rules. In 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS) (499-502). doi: 10.1109/icis.2017.7960043. Gorwa, R., Binns, R., & Katzenbach, C. (2020). Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society, 7(1), 1-15. doi:10.1177/2053951719897945. Glaser, B. G., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine. Gitlin, T. (1980). The Whole World is Watching: Mass Media in the Making & Unmaking of the New Left. Berkeley, CA: University of California Press. Graefe, A. (2016). Guide to automated journalism. Available at: https://towcenter.columbia.edu/news/guide-automated-journalism (Accessed 21 March 2020). Graefe, A., Haim, M., Brosius, H., & Haarmann, B. (2018). Readers’ perception of computer- generated news: Credibility, expertise, and readability. Journalism, 19(5), 595-610. doi:10.1177/1464884916641269. Gulyas, A. (2013). The influence of professional variables on journalists’ use and views of social media: a comparative study of Finland, Germany, Sweden and the United Kingdom. Digital Journalism 1(2): 270–285. doi:10.1080/21670811.2012.744559. Hall, S., Catcher, C., Jefferson, T., Clarke, J. & Roberts, B. (1978). Policing the crisis: mugging, the state and law and order. London: Macmillan. Henrickson, L. (2018). Tool vs. agent: attributing agency to natural language generation systems. Digital Creativity, 29(2-3), 182–190. doi:10.1080/14626268.2018.1482924. Hofstetter, B., & Schoenhagen, P. (2016). When Creative Potentials are Being Undermined by Commercial Imperatives. Digital Journalism, 5(1), 44–60. doi:10.1080/21670811.2016.1155966. Jenkins, H. (2006). Convergence culture: Where old and new media collide. New York: New York. University Press. Jung, J., Song, H., Kim, Y., Im, H., & Oh, S. (2017). Intrusion of software robots into journalism: The public's and journalists' perceptions of news written by algorithms and human journalists. Computers in Human Behavior, 71, 291-298. doi:10.1016/j.chb.2017.02.022. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 53
Khan, M. U. G., & Gotoh, Y. (2017). Generating natural language tags for video information management. Machine Vision and Applications, 28(3-4), 243-265. doi:10.1007/s00138-017-0825-7. Klinenberg, E. (2005). Convergence: News production in a digital age. The Annals of the American Academy of Political and Social Science, 59(1), 48-64. Kulkarni, G., Premraj, V., Ordonez, V., Dhar, S., Li, S., Choi, Y., Berg, A. C., & Berg, T. (2013). Baby talk: Understanding and generating simple image descriptions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12), 2891-2903. Latar, N. L., & Nordfors, D. (2009). Digital Identities and Journalism Content-How Artificial Intelligence and Journalism May Co-Develop and Why Society Should Care. Innovation Journalism, 6(7), 3-47. Lavrakas, P. J. (2008). Encyclopedia of survey research methods. Thousand Oaks, CA: Sage Publications Lewis, S. C., & Molyneux, L. (2018). A decade of research on social media and journalism: Assumptions, blind spots, and a way forward. Media and Communication, 6(4), 11- 23. doi:10.17645/mac.v6i4.1562. Lewis, S., & Westlund, O. (2015). Big data and journalism: Epistemology, expertise, economics, and ethics. Digital Journalism, 3(3), 447-466. doi:10.1080/21670811.2014.976418. Lewis, S., Guzman, A., & Schmidt, T. (2019). Automation, journalism, and human–machine communication: rethinking roles and relationships of humans and machines in news. Digital Journalism,7(4), 409-427. doi:10.1080/21670811.2019.1577147. Merton, R. K., & Barber, E. (2006). The travels and adventures of serendipity: A study in sociological semantics and the sociology of science. Princeton, MA: Princeton University Press. Montal, T., & Reich, Z. (2016). I, robot. you, journalist. who is the author?: Authorship, bylines and full disclosure in automated journalism. Digital Journalism, 5(7), 829- 849. doi:10.1080/21670811.2016.1209083. Mou, Y., & Xu, K. (2017). The media inequality: Comparing the initial human-human and human-ai social interactions. Computers in Human Behavior, 72, 432-440. doi:10.1016/j.chb.2017.02.067. Newman, N., Fletcher, R., Kalogeropoulos, A., Levy, D., Rasmus Kleis and Nielsen, R. K. (2017). Reuters Institute Digital News Report 2019. Reuters Institute for the Study of Journalism, Oxford. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 54
Neuberger, C. (2013). Competition or complementarity? Journalism, social network sites, and news search engines. In Journalism and Media Convergence (pp. 119–130). Berlin, Germany: De Gruyter. Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage. Primo, A., & Zago, G. (2015). Who and what do journalism?: An actor-network perspective. Digital Journalism, 3(1), 38-52. doi:10.1080/21670811.2014.927987. Ramos-Soto, A., Bugarin, A. J., Barro, S., & Taboada, J. (2015). Linguistic Descriptions for Automatic Generation of Textual Short-Term Weather Forecasts on Real Prediction Data. IEEE Transactions on Fuzzy Systems, 23(1), 44-57. Reiter, E., & Dale, R. (1997). Building natural-language generation systems. Natural Language Engineering, 3(1), 57-87. Reiter, E., & Dale, R. (2000). Building Natural Language Generation Systems. Cambridge, UK: Cambridge University Press. Robinson, S. (2011). “Journalism as process”: The organizational implications of participatory online news. Journalism & Communication Monographs, 13(3), 137- 210. doi:10.1177/152263791101300302. Ritchie, J., Lewis, J., Nicholls, C. M., & Ormston, R. (Eds). (2014). Qualitative research practice: A guide for social science students and researchers. London, England: Sage. Rudman, R. and Bruwer, R. (2016). Defining Web 3.0: opportunities and challenges, The Electronic Library, 34(1), 132-154. doi:10.1108/EL-08-2014-0140. Ruusunoksa, L., & Kunelius, R. (2007). Professional Imagination and the Future of the Public Service Newspaper. Paper presented at the conference on the Future of Newspapers, Cardiff Centre for Journalism, Media and Cultural Studies, Cardiff University. Saridou, T., Spyridou, L., & Veglis, A. (2017). Churnalism on the Rise? Assessing convergence effects on editorial practices. Digital Journalism, 5(8), 1006–1024. doi:10.1080/21670811.2017.1342209. Schultz, B., & Sheffer, M. (2017). Newspaper trust and credibility in the age of robot reporters. Journal of Applied Journalism & Media Studies, 6(2), 339-355. doi:10.1386/ajms.6.2.339_1. Saltzis, K., & Dickinson, R. (2008). Inside the changing newsroom: journalists' responses to media convergence. Aslib Proceedings, 60(3), 216-228. doi: 10.1108/00012530810879097. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 55
Seth, C., & Logan, M. (2018). A decade of research on social media and journalism: Assumptions, blind spots, and a way forward. Media and Communication, 6(4), 11- 23. doi:10.17645/mac.v6i4.1562. Soloski, J. (1989). News reporting and professionalism: Some constraints on the reporting of news. Media Culture and Society, 11(2), 207–228. Tameling, K., & Broersma, M. J. (2013). De-converging the newsroom: strategies for newsroom change and their influence on journalism practice. The International Communication Gazette, 75(1), 19–34. Tavory, I. & Timmermans, S. (2019). Abductive analysis and grounded theory. In Bryant, A., & Charmaz, K. The SAGE Handbook of Current Developments in Grounded Theory (pp. 532-546). London, UK: SAGE Publications. doi: 10.4135/9781526485656. Thurman, N., Dörr, K., & Kunert, J. (2017). When Reporters Get Hands-on with Robo- Writing. Digital Journalism, 5(10), 1240–1259. doi:10.1080/21670811.2017.1289819. Tong J. (2017). Introduction: Digital Technology and Journalism: An International Comparative Perspective. In: Digital Technology and Journalism. Palgrave Macmillan, Cham Treadwell, D. (2017). Introducing Communication Research: Paths of Inquiry (Third Edition). Thousand Oaks, CA: Sage. Tuchman, G. (1973). Making news by doing work: Routinizing the unexpected. American Journal of Sociology 79: 110–130. Tuchman, G. (1978). Making news: A study in the social construction of reality. New York: Free Press. van Leuven, S., Kruikemeier, S., Lecheler, S., & Hermans, L. (2018). Online and Newsworthy. Digital Journalism, 6(7), 798–806. doi:10.1080/21670811.2018.1498747. Walker, A. (2014, March 19). Quakebot: An algorithm that writes the news about earthquakes. Gizmodo. Retrieved from https://gizmodo.com/quakebot-an- algorithmthat-writes-the-news-about-earth-1547182732. Wang, Q. (2017). Dimensional Field Theory. The adoption of audience metrics in the journalistic field and cross-field influences. Digital Journalism, 6(4), 472–491. doi:10.1080/21670811.2017.1397526. Willig, I. (2013). Newsroom ethnography in a field perspective. Journalism, 14(3), 372-387. doi:10.1177/1464884912442638. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 56
Willig, I., Waltorp, K., & Hartley, J. (2015). Field theory approaches to new media practices: An introduction and some theoretical considerations. Mediekultur: Journal of Media and Communication Research, 31(58), 1-1. doi:10.7146/mediekultur.v31i58.20671. Wölker, A., & Powell, T. (2018). Algorithms in the newsroom? news readers’ perceived credibility and selection of automated journalism. Journalism, (2018). doi:10.1177/1464884918757072. Welbers, K., Van Atteveldt, W., Kleinnijenhuis, J., & Ruigrok, N. (2018). A gatekeeper among gatekeepers: News agency influence in print and online newspapers in the Netherlands. Journalism Studies, 19(3), 315-333. doi:10.1080/1461670X.2016.1190663. Wu, S., Tandoc, E., & Salmon, C. (2019a). Journalism reconfigured: Assessing human– machine relations and the autonomous power of automation in news production. Journalism Studies, 20(10), 1440-145. doi:10.1080/1461670X.2018.1521299. Wu, S., Tandoc, E. C., & Salmon, C. (2019b). A Field Analysis of Journalism in the Automation Age: Understanding Journalistic Transformations and Struggles Through Structure and Agency. Digital Journalism, 7(4), 428–446. doi:10.1080/21670811.2019.1620112.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 57
Appendix A Interview Transcript Codes
[ ] Simultaneous speech
( ) Inaudible speech. Text between brackets indicates what was thought was being said.
((<)) A short pause (less than 2 seconds)
((>)) A long pause (more than 2 seconds)
{ } Name codes or identifying numbers to named persons.
.,?!;: Normal punctuation
(! !) An emphasis (text displayed in-between exclamation marks)
(( )) A clarification, for instance about the context.
=A connection between two parts of a speech act.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 58
Appendix B Interview Transcript A
Transcript template Interviewer (IR): Sebastian Short Interviewee (IE): Participant A Date: 23/04/20 Time: 10:00 Setting: Video call with living room as a background
IR: At the moment, what are your typical responsibilities? IE: At the moment it is two-fold one the one hand I am responsible for the education of journalism students, I am responsible for the Master of Journalism at the University of Groningen, where I am responsible for the skills division. We have a 50/50 skills and theory division and I am responsible for the Skills part. That is one main occupation, the other responsibly is the company Innovation Origins, I am the CEO there, ((<)) well I have to make sure the company keeps on running. I have journalism background, but in my company I hardly do any journalism, we have a separate editor-in-chief and reporters. My job is the business (side). IR: And what about your previous roles as a journalist, did you have an editorial role or similar? IE: As a journalist I have performed any role you can imagine; first intern Until editor-in-chief of several newspapers and everything in-between. Also, for different, ((referring to mediums)) I did radio, I did lots of newspapers and online. I never did TV though. IR: Ah ok, so all the mediums then! So, how many years have you been working in journalism? IE: [Well, I studied History in Utrecht = IR: Ah interesting. IE: = and that was the 1980s and even then I started as a journalist, so I think it was 1984 when I started]. IR: Ah ok, so quite a few years then, ok. Very nice! So in terms of topics you normally write about I assume it is technology, innovation, things like that. IE: [Yeah well, I have experience in all sorts of stuff, I did local journalism I did, well I was a reporter on Eastern Europe for several years = IR: Ah, interesting. IE: = I did sports journalism, so in my career I did all sorts of things. But now it is focused on innovation, yep]. IR: Great, ok super. So I guess the next question I can answer ((questions to ask)), in terms of your whole career, do you feel that technological changes impacted you work routines since you started working within journalism? IE: What kind of changes, (did you say?) IR: Like the way technology has changed. ((IE did not hear IR)) AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 59
IE: Oh technology, yep ok. I started journalism when there were no computers, I did journalism on typewriters. So, if you look at technology it really impacted that from the beginning. I think that the biggest impact though was, of course related to the arrival of the first computers was the networked way of editorial work. IR: Yeah, yeah yeah. IE: So, you could start doing work together, you could start doing work together on one story. You could share things and obviously in the 1980s and 1990s this was still, if you could compare to what you can do now its like a fundament we are still building. That was a big change, but maybe you want to focus on the more recent changes, there I would say journalism still has a long way to go. Journalism hasn’t adopted a fraction of what could be useful to journalism. If you look at the field of Artificial Intelligence and of machine learning, and what {name} is working on, robot-journalism. But also, VR and AR. Those are technologies that can be way more useful for journalism and for the audience of journalism that can be used right now. IE: So, there is a long way to go, actually for my inaugural speech five years ago, I said that there were two positive things in journalism at this moment, one is society, society needs journalism, that we perform it in the right way. The other one is technology, there is so many opportunities there for us journalists. Especially in legacy publishing houses. ((>)) Yeah, much more can be done. IR: Yeah, I agree, I have read so much literature on this topic and you can really see it has impacted it quite big. IE: Yeah and some of them are working on it. For example, if you look at the national broadcasting team at NOS. they have quite a good innovative team, and really nice things come up there. You can also for example as far as data-journalism is concerned there lots of initiatives that are quite promising. But still its well, too little I would say. IR: So it’s still yeah I guess emerging at the moment, lots to do. That was my overall impression I got from reading the literature on this topic. IE: If you get an opportunity to visit some of those newsrooms, you can see that it is not also literature, it is also real life. IR: Yeah exactly. Great so, next question. Has Innovation Origins always been online? IE: We were here now for a little more than five years. IR: Ok. IE: But online, the plan was in the first year was to also publish a weekly newspaper. And we published around 20 editions. But it was way too expensive and way too complex for us to perform that. IR: Ah I see, interesting. IE: And it didn’t bring money so that was one of the most easy decisions to take (of the) printed ambitions. ((Decision to make in regard to print)). We still do like, half a year ago we published a magazine, every year or so we publish a book which contains already published articles, so like a collection of earlier (work). That you could say is a result of the online work, so we are an online platform. IR: [Ok, so the reasons for that were, well the most cost-effective way of doing it at the time = AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 60
IE: Yep. IR: = Ok, super. Do you feel like in our online only world, is there any pressures on your job because of this? Or is it going quite well?] IE: The pressure is always on how to make money with what you are doing, we in a previous year we tried to turn this into a subscription model. We didn’t succeed, after a year we needed around 10,000 subscribers to keep going and we had 200, so quite the difference. We almost stopped the business altogether and that was then at that moment new opportunities arrived. The model that emerged at the time, was still our most important model and that is companies but more like institutes and innovation clusters et certera ((<)), pay us to write about their business. In a more marketed model, it is not content marketing what we do, we write whatever we want. It is truly independent. Still we create value. IR: So it is like a publicity platform, I guess? IE: [Yeah, but it is not publicity = IR: In the traditional sense? IE: = it is not PR, for example {company} in Eindhoven is, one of our sponsors you could say. But they don’t tell us what to write about. We decide what to write upon, there is enough trust that they ((<)), well they know what to expect. Now we have like 20 or 25 of these organisations that support us. IR: That is interesting, do you rely much on advertising or is it? IE: Nope, no advertising only incidentally. Advertising is difficult and it is possible of course. I am not against advertising. In principle I am not against advertising, but it is something we prefer not to do. (We like to keep) our platform as clean as possible, and of course it is a choice it doesn’t make things easier. IR: yeah that is true. One of these online problems, I guess? IE: Yeah well it keeps you busy every day, our main partners have difficulties themselves, so its more difficult than ever to keep things running. My job is interesting and = IR: Yeah, I can imagine. IE: = ((>)) it is not an easy one. IR: Do you feel like with these pressures you are able to find a balance between journalistic quality and financial performance? IE: Well, the thing of course is I am only able to pay my reporters a certain amount. I am not able to pay them for doing research for like a week or two weeks and then come up with a story. That is the limit, but it is also the limit we accepted to have. Of course I would prefer to have all of my reporters go away for two weeks and then come back with the best story ever. IR: Yes, indeed. IE: But it is not feasible, but they have to write like, well one or two stories every day and ((<)) I can’t afford to give them more time. [That is where it clashes more or less with the money and the quantity thing = IR: Yep, yep AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 61
IE: = But I think it is acceptable. IR: Ok great, so earlier you mentioned robo-journalism, coincidentally that is what I would like to talk about next. So, are you aware what it is, overall, what is your opinion on articles that are automatically written? IE: ((<)) I think we still, of course I can’t prove that, because maybe I have read lots of automatically written articles that I don’t know that were automatically written. But my feeling is that it is still emerging, it is still in an experimental phase. There are companies especially in the United States that are fully occupied with robot-journalism, with building the automated newsroom. But if you look at the Netherlands, well there have been some experiments, some ended successfully some others not so much, it is still in an atmosphere of experiment. IR: Yes, yep. IE: It’s not entering in any sense, it needs to grow into a mainstream or some sort of a normal way of working for (users). Where it can really take off, of course there has been some resistance in newsrooms, there still is. Some of it is understandable well, there is this feeling that the robots will take my job. IR: Yep IE: It is difficult to separate this feeling from, the search for quality from the automated newsroom. You need to improve and keep improving the machines so to say in order to get somewhere. But if you have a newsroom that is reluctant or resistant then it is difficult to improve your machine. IR: Indeed, yep, yep. IE: So that is I think the most visible handicap at this moment, at least in our country but I am not aware at this moment but maybe you are more than that. In especially in the United States, but maybe also in England there are several initiatives that are (further) than we are now. IR: Yes, in my research I have come across some examples, it depends on the outlet. I know the Associated Press for example. I think almost all of their financial reports are automatically written. IE: [And then sports is also = IR: Sports, yep. IE: = is one of the main topics I would say.] IR: Yeah, I read that too. IE: I was involved with the Google Digital News initiative for a couple of years. It ended now, but we were able to fund several initiatives also in this field. I remember one in England, indeed there was press agency, I think was AP but I am not sure. They had a very nice plan. So yeah we have a long way to go. IR: Yes, I mean that overall my impression. So in a similar vain, do you think maybe we touched this already. Do you feel that the use of Artificial Intelligence is beneficial for journalistic practice? Why or why not? IE: I do think so, I am convinced that journalism needs to and will step across the traditional boundaries of the journalistic field. AI is one obvious field that we have to cover or conquer or ((<)) however you want to say it. But there are way more fields, 10 or 15 years ago when it AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 62 became obvious that a company like Google or Facebook used algorithms in a way that was really affecting journalism, the field of journalism. We kept saying it is not our business, let them do it and let people find out that it is rubbish and let’s stay away from that. I think we ((<)) start feeling an awareness that this was not the most cleverest decision that we made. So now it maybe easier to say that yes this is something we need to, ((>)) how do you say? We need to build on ourselves. We can do that in two ways, you can try to be an AI specialist yourself, or you can join forces. I think that the latter would be well, the preferred step for this moment at least. IE: Still journalism is a profession that is well, special and different from other professions. Not any non-journalist can feel or be or act like a journalist. You need the professional journalism as well as the technology. IR: Yeah, I definitely see what you mean there. IR: Ok great, I am now going to move onto another question. So when looking to source a new story, what is the general routines that take place at Innovation Origins? Where is their starting point? IE: I think that the well, the reflex that we work on not just at Innovation Origins but any journalist is the his or her own network. Everything we hear within our network is way more important in our heads than things that occur outside of our network. We would, if somebody within our network would ((<)) come up with a suggestion, this would lead to a story way earlier, way sooner and faster than just an email or a message. Or a robot that is suggesting something. IE: That is of course, networks are important in that way it is good, and it is logical but its keeping us from finding original sources or new sources. That is something holding us back from reporting on stories that might be worth reporting on. IR: Do you start online, is their online sources? IE: Yeah, of course so ((<)) apart from our you could say ‘physical network’, people you know or relate to or contact, there is always the newswires and different from like 10 or 20 years ago is the way you could say, social media is working as an alert. So we all have our lists in Tweetdeck or Hootsuite and these lists, of course you follow hashtags and key words. So they alert us if something new happens. For example {company they have written about in the past} publishes something on social media, we see it immediately. IE: That is of course very helpful, but it is still I a first phase of being alerted in an automated way. IR: Interesting, so with that in mind, do you find the current sourcing routines easy or difficult at the moment? Are they efficient or inefficient, do you feel like they can be improved in any way? IE: Yeah of course they can be improved, but you know I think that is only human. You are satisfied with what you have, what you don’t have you don’t know. So it is difficult to imagine a situation where things would be twice as efficient or twice as fast. If you are offered something and somebody can show me a way to be twice as efficient, then yeah that would be different. IE: So it is difficult to imagine an, maybe for me it is easier, well I am from my university job I am used to following literature and doing my own research. But for an average reporter, he or she is just doing their daily job, working from a routine that is known and that feels efficient, but that might not be. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 63
IR: Yeah indeed, so is {Company of participant A} quite data driven? IE: No, I wouldn’t say that, it could be way more data driven, ((>)). What happens is with your previous question, is that you work from a routine that feels like normal, or the best you can do. So if you don’t know the better routine then stick to your old routine. IR: Yeah, yeah, indeed. Interesting. IE: That is human as well, routines are so ((>)), they are in the way of improvement. IR: Yeah, yeah. Do you feel like people have a gut feeling, or a personal drive to try and find something new? IE: Yeah, that is all over journalism. IR: Like being curious about something or? IE: Yeah and being curious is of course dependent upon your social network, the way you are raised, the money you make, the clubs you visit. It is very, maybe non-technical. IR: Yep, yep. Great, so I will try and get to the TCE part. Are you aware that nowadays, so a lot of readers kind of get personalised recommendations, I.E. they go down Facebook now and their news is tailored to them based on their browsing habits and data, so how would you feel then if such techniques were turned around from the journalistic perspective and then targeted at you to help you discover newsworthy content? IE: Yeah it would be great from a society point-of-view. But it would be quite difficult from a personal point-of-view. I have helped a couple of start-ups in journalism for trying to do exactly that, to pull people out of their bubble and show them the different interesting stories. Also technology based, but just also journalism based. IE: But it is difficult in many ways, because in most of what people say that they would love to see and read different points of view, get a better overall picture, want to find the truth wherever it is. But that is different from their daily life. Their daily life is, what every human being wants is to be right, if I read an article that proves me right that feels better than if I read an article that might be closer to real truth, but it questions my way of thinking of looking at the world. That doesn’t feel good. IE: Maybe psychologically that is the difficulty. IR: Yes, yes I see what you mean. IE: For society as a whole this would be really great, but I think it is not a thing of left or right wing, it is from both sides. There is maybe a conception of truth or a way of looking at truth that is quite personal. IR: Interesting, ok I am now going to share my screen with you and I am going to share with you the demo of TCE. ((IR then shares the screen with IR)). IR: As you can see here, this is the content exchange, as you can see. So in this search box to the left you search for a feed. This can be on any topic, I just typed in some examples here, the usual suspects, corona, technology, economy and stuff like that. These are all called feeds you can search within these feeds. You can filter them here ((pointing cursor at filter icon)). IR: The platform covers multimedia, images, audio, text. At the moment it is in its early days of development so this is an early prototype. If I click on an article ((IR clicks to open an article)) you can view it here, you have the content here and you can scroll down. You can either buy AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 64 selection or you can download the full piece, or you can just download the images for example. Then you will get a feed like this ((IR scrolls through one of the topic feeds on TCE)). IR: I can also show you some images to get a better idea of the platform ((IR Switches tabs to show mock-up images of TCE)). IR: So, they also plan to include shared feeds, so as an editor you can make a custom feed and you share it around with anyone in your editorial team, with the goal in mind that perhaps you may find something else you may have not considered before. IR: ((Switching image)) This is what a story looks like. Then you can also get personalised recommendations as you can see on the left here. Based on your browsing habits, so its kind of turning personalised recommendations on consumers around onto a journalist themselves. IR: ((Switching image)) This is the login screen. ((Switching image)) Then you have your own profile like this ((Image shows personal profile section of TCE)), and within a topic you can filter what you want. So, if you are a sports journalist you can then filter the sport you only want to see, and search within that. ((Switching image, to another mock-up of the main dashboard)) That is just a similar screenshot. ((Switching image)) Here is a better screenshot, I can zoom in, there we go ((Screenshot showing filters within feeds on main dashboard)). You can create your own filters and custom feeds, so you can tailor a topic to yourself. IE: And then I can ((<)), I see there are topics like entertainment and sport, but also nu.nl which is a platform itself. IR: Yeah indeed, so content theoretically be offered by anyone, a freelancer or a journalist at a big firm can just offer their content on this platform. It can then be purchased by anyone and reused with copyright. Well, this is another issue entirely. You can filter by content type and again down the bottom you have this recommended feed. And then also the shared feeds. IE: Well this is something that would serve the newsrooms. IR: Yes, this is for the editorial teams and anyone involved with sourcing, so I think the goal is, you are in the newsroom you are looking for a news story and you come to this dashboard as kind of a starting point. Maybe you see a recommendation here ((pointing at the personalised recommendations section of dashboard)) and you then kind of go from there and search through it really. I believe that is everything yes ((IR is reviewing mock-up images)). I’ll stop sharing my screen now ok ((IR stops screen sharing)). IE: So, what I thought it was, also from the video shown on their website is that it was meant to be a platform that will be a marketplace between a source and another platform. IR: Yes, yes, yep. IE: [This seems to be more like sourcing the editorial teams. Serves as a, what I thought it was that a freelancer could offer a story which could be distributed to other newsrooms = IR: That is the goal yep. IE: = and could be published or something, yeah ok.] IR: So its kind of a more, multiple perspectives at the moment. I have some notes on it, yeah so that is kind of the platform. Yeah you are right, it is a marketplace for content where it can be opened up, so anyone who is writing content can offer it on the platform. You can then offer either some or all of it. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 65
IR: So that is TCE so far, what is your overall impression of the system so far? Do you think it is good or bad or? IE: That’s is a lot already, I think that what it looks like now and as far as I can think about the technology behind it. This could certainly work, so I ((>)) maybe first thing before we spoke what came to my mind. This is an initiative that looks and sounds familiar to me, I have seen examples of it before. IR: Ok, interesting. IE: Yeah, the marketplace example. In fact, I have been involved with a start-up that is called Verhalenmarkt, you may have known or heard of it. They were promising the exact same. Their promise was to offer this marketplace for any journalist, freelancer or company or whatever, to be able to first publish or re-publish their articles on any platform. What they promised was to take care of the connection between the story and another publishing house. So they built algorithms and well recommendation services that would connect in an ideal way. So, if I would write about lets say a football team, then I would immediately be connected to the platforms that would be interested in football, automatically. This of course is a very simple example, but it would be way more complex. If my article would have maybe one or two lines about smart mobility, then they would connect this article to a newsroom that working on smart mobility in any way. IE: So that is what they tried to do, ((<)) apart from technology didn’t work like it should, their main problem was that they didn’t find a way to really make money out of it. Because, the first connection would be ok, but if I for example as a freelancer journalist, thanks to this platform I would be connected to the ideal newsroom for my work, I would need you only once, to make the first connection. I could go directly to this newspaper or this magazine, I wouldn’t need you anymore, and that was something they weren’t able to solve. IR: I see, yeah. IE: So that was a big problem and they were not the first to find this problem. I have been involved in a start-up accelerator, what you could call the stimuleringsfonds voor de journalistiek, a fund for journalism that was supported by the Dutch government. They had this accelerator for journalism start-ups, and we have had several examples of this marketplace like, bureaus or platforms that try to connect freelance journalists with the perfect distributor. And they all failed. IR: Hmm. IE: The difficulty was, every time not only the initiators but the people around them thought this was a brilliant idea. In essence it is a brilliant idea, because it could help freelance journalism and in the broader sense it could help society. But reality turned out to be different, in all of these examples. What is different as far as you have shown me a couple of minutes ago, is that this could also serve as an inspiration for newsrooms. If I look at this dashboard it could inspire me to write about one topic or another and use maybe fragments or paragraphs from the offered text. IE: But then it would be, the challenge of course to because if I offer my text to this TCE platform and I am not sure, well how it is used, what would still be my drive to keep on offering you my text if I know that I might be inspiring this other newsroom but what do I get back from it? IR: I see what you mean, yeah yeah. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 66
IE: And maybe you have thought of something brilliant, I don’t know. IR: Yeah I mean, I am only involved somewhat in the project, so maybe there is a masterplan. IE: So there you need to work out these connections and if you want to keep everybody happy, because there are several people who are ((<) and elements involved in the whole process and scheme to keep everybody happy. IR: Yeah indeed, I agree. So looking at the system itself, so if I was an editor going into the system and I was looking for a topic in mind and then I discovered another topic seemingly by accident, would you feel that is beneficial? You know I accidentally discover another topic I didn’t previously consider then I researched into that a bit more and it becomes a bigger idea. Do you feel like that could facilitate that process? Or do you feel like that process is beneficial, referring back to the personalised recommendations at the bottom. IE: It depends on whether you are a general journalist or a specialist. IR: True, true. IE: If you are of course only looking for specific topics, if you are a general journalist then that might be worthwhile, but ((<)) still it is, if only you would look at your Tweetdeck or your Hootsuite, yeah or newsfeed. You can be inspired in tons of ways, I would say. You would have ask yourself what problem do you solve with that, what you need, and sometimes it is nice to see something else, by accident or, but will this be a reason to join TCE or? IE: [Yeah I doubt that, as for any start-up the main question is of course what is the problem you are solving? And whose problem are you solving? = IR: Indeed, yep. IE: = And how big is this problem? Do I really need it solved?] IE: So either the result the effect or the efficiency would have to be so much better than what I have now. Or it must directly solve a problem. IR: I see what you mean yep. So referring back to the sourcing routines earlier, do you feel like this kind of dashboard could integrate with that? Or could it add value to it, or improve it any way, or not? IE: Yeah, it could, but it’s not as far as I can see now its not something that I can’t do without. As far as I can see now, you have only shown me some screenshots, I don’t know exactly how it works of course. But, there are several roles within this system, you are referring to a newsroom, editorial place at this point. Then it would certainly add something, but (if 100 would be the total view, it would be like 40 to 40). IR: I see, so it wouldn’t be like a primarily dashboard, it wouldn’t replace their Tweetdeck or their Hootsuite. IE: No exactly. It would be next to that. Next to maybe the wires and the yeah. So, yeah the main question is how needed this would be for me. If you can convince me that I cannot miss this as a tool to perform my job, then it would be good. IR: I see, I see. IE: I don’t see that yet. On the other side though, if I would be again, this freelance journalist looking for a magazines or looking for, that still might be interesting. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 67
IR: Yeah yeah, that would be interesting, it could depend on your position within the journalistic field. IE: Yep, yep. IE: Yeah and maybe the whole eco-system that is covered by TCE. IR: Yep, yep, yep. IE: Because again there are several roles visible in this platform. IR: Yes, I agree. Ok, lastly I will focus on the user interface and the design, where there any, maybe I can show you the picture again, were there any elements of the design that stood out to you? IE: It looked clean and ((>)) well user friendly I would say and it is comparable for how for example the ANP press agency is offering their content. [Also with these columns = IR: Oh, so similar to the ANP. IE = Yeah, definitely.] It is also similar, if you look at my Tweetdeck then you’ll see the same with the columns of topics that I need to follow. [That helps of course ,= IR: OH yeah yeah, the columns definitely. IE: = it’s something that I already now and I know how to use, so that’s clever I would say.] IR: So, it has a logical layout? IE: Yep. IR: Kind of invites you to navigate it a bit more? IE: Yeah. IR: [The topics look clear = IE: Yep. IR: = to use.] IE: Is it possible for me to add columns, to add topics. IR: Yeah you saw the search column in the top left? IE: Yeah. IR: If you search, type in a word, press enter and another feed will pop up next to it. IE: Ah ok yep. IR: So, if you just, you create your own feed, I just used them as an example and you can scroll down them. Then all of your feeds kind of comes up and then you have to scroll horizontally to view them all, so that’s interesting. ((referring to displayed feed topics on the demo)) IR: So, let’s see ((IR is looking at notes)). So the layouts seemed logical? Does it kind of it may be difficult to say at a quick glance but would you feel kind of persuaded to look further into it, based on the design cues? Does it invite you to see more? Is that a bit of a abstract question? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 68
IE: Of course, but for me that would depend on the topic, or the content. But the layout is helping, its not a barrier or I would say so ((<)) yeah. IR: Would you then make use of personalised recommendations, for something like really useful? IE: Yeah, but again that depends on the results. IR: I see, yep. IE: If I get useless recommendations then it would be over in a few days. IR: Yeah, indeed, indeed. IE: If it can help me, then yeah. IR: Ok, great. Super, and of course something a bit unrelated but would you or your team consider making use of sharing the content, or sharing the content yourself in the future? IE: Sharing in what way? IR: Like publishing your own content on the platform? IE: [Yeah I would, yeah that is why I contacted TCE, I asked him would that be, well I asked him to start a conversation about it = IR: Great. IE: = If it would be feasible, or whatever, yep.] IR: I’ll just have a quick glance at my questions, ok I will just close up and ask, is there anything else? In terms of everything I’ve spoken about? IE: Yeah I think that, I trust that they will build a technology that is working I would say, the real test would be in ((>)) could you create something that all sides of the eco-system, would solve a problem, and it really needs all sides. It is not only a question of solving, the problem of the freelance journalist which I think could be done quite easily. You could also have to solve the problem of the editorial room and the appliance of these editorial rooms. So there has to be a logical total and not only the separate elements. IR: I see what you mean yep, so actually just thinking of something else, in terms of automated or robo-journalism. Do you think you or your editors in general would source these kind of articles, if they knew it was automatically written? IE: I hope so, well I would try to convince them to do so, because ((<)) I would try to stress importance of the quality of the content not the (source) itself. Whether it’s a robot or a human being, or whatever. If the quality is what we are looking for, then please use it. IR: Yeah, interesting, so if an article popped up in the content exchange and it was written by a robot, it would perhaps depend on the person maybe? IE: [No no, I would say it would be judged equally = IR: Yeah? IE: = compared to the other articles. For us it would always be very important to know the source. IR: Yes, yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 69
IE: So and if it’s a robot, I would want to know how the robot works. So, and not with every single article, if there is a robot active then I would want to know what the algorithm is like or how this robot, you know, what the sources of the robot are. IR: So, transparency is kind of key there then. IE: Yep. IR: Interesting, and closing thought, where do you see this going in the future then? In terms of AI, automated journalism, and the newsroom? IE: It can only grow, there is only one way and that’s the growing importance of these robot-like sources. IR: Yep. IE: But its like with many innovations, it takes more time, some of the enthusiastic in the beginning think it will take. Because its not only a question of technology it’s a question of also and maybe more even so a question of culture. IR: That’s a good point, interesting. Ok superb, well I think that’s more than everything I need, so I’ll stop recording for now.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 70
Appendix C Interview Transcript B
Transcript template Interviewer (IR): Sebastian Short Interviewee (IE): Participant B Date: 24/04/20 Time: 11:00 Setting: Video call with living room as a background
IE: So, could you first. So you are student now but what was your occupation. IE: I was a journalist and I worked for 6 years before I restarted my education. IR: Ah ok and what company was that for? IE: It was the province TV station. IR: And that is for the province you are in? IE: Yep. IR: Ok and what kind of, is it online-only, is it also on TV, or anything else? IE: It is online and on TV. IR: Ok and what kind of news topic do you cover? IE: Quite a lot, when I first started working, I did daily news reports, like what was happening today, (mostly everything at once), like documentaries. IR: Interesting, what were your typical responsibilities? IE: I was doing the daily news reporting and mostly social areas, like something special happened in the day or policies regarding peoples’ daily lives, things like that. And for documentaries its quite diverse, but it is also mostly in the social and the cultural area. IR: Ok and then were you involved with sourcing at all? Like finding a new story? IE: Erm, finding new stories. Mostly we get our information online, because it is the fastest way to see what is happening around. IR: Ok we will come back to that later, so firstly so you have six years of experience, do you feel over that time that technological chance has impacted your work routines, and the organisation itself? IE: Yes quite a lot, when I first started we mainly focused working on the TV broadcast, after that for like 2 years I worked for, our concentration and focus has turned to the online part. So whatever we did for the reviews, we uploaded them online and that was a central part to our daily work, but we have a WeChat, its like WhatsApp and Weibo, like Twitter. We have a lot of staff working on that part and like around 14 or 15. Yeah that is a main part of our job now. So we not only shooting the daily broadcasts on TV, we also focus on trying to express our, or exchanging information online first, because it is faster than TV. IR: Yeah, I see so is it kind of, has the company transitioned more online, over the years? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 71
IE: Not only in our personal work but also the entire company, yep. IR: Interesting, so in terms of balance is it 50/50 TV online, or is it 70/30 now? IE: [I think its 50/50, because its still a TV station, so there is still a focus on TV programmes = IR: Ah ok, yep. IE: = but everything we do is now online, so we can say 50/50.] IR: Great, and so do you feel that this online world has put pressure on your job because of these changes? ((<)) On your work routines? IE: Very much because it is quite fast when you use smartphones. For example we do the TV only (reviews), we need out camera man with us and he shots a lot of (clips and reviews) that we need to edit them and then broadcast. But if you are just sending a short message, then you just use a smartphone, so that is very quick. IR: Ok interesting, and what about like the management team above you, how are they coping, if you know? IE: I think the online part has become an evaluation of our work, so they mostly they are the same content, but they need to calculate, evaluate whether you have done this part, or how well you are doing in the online part. Its kind of a evaluation. IR: So is your firm quite data-driven, as in do you use audience metrics to determine the success of a news story? So like page views and how many people click on a story and things like that. IE: Yeah we had that. IR: Was it quite big part of your routine? IE: Yeah it was. IR: Ok, have you heard of automated journalism? IE: ((<)) Sorry, automated? IR: Automated journalism. IE: ((Thinking about answer)) IR: Or robot-journalism. So basically one of the more recent technological developments in the newsroom has been this rise of automated journalism. What that is basically there are news outlets now that increasingly using algorithms, for example do you know to Associated Press in the United States? IE: Yep. IR: All of their financial reports are written automatically, IE they get the financial data from the markets and then the system automatically writes a story and publishes it online, and another example is the LA Times, in 2014 there was an earthquake in Los Angeles and within three minutes they reported the story because they had this quakebot and they detected the seismic data, so the earthquake happening, took that data and text and published it on the website within three minutes of it happening. So its kind of the use of Artificial Intelligence and data and Algorithms to automatically write stories. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 72
IE: Oh yeah we had that kind of not really into the practical area, we mostly see that kind of experiments. IR: So it was implemented in your firm? IE: We had it before yep, so somehow I can say it is kind of a debasement, so people trust these kind of artificially made articles, some people put their trust into people. So its kind of a debasement, so its not really a practical use. We had that, the technology part yes. IR: So you also, what kind of articles did you automate? IE: I think it was, there were examples, there seems to be also in the financial part, we used a lot of figures and compared to an article written by people I think its mostly contains the same information, yeah its almost the same. The audience just have different opinions. IR: Did you disclose whether the articles were written by robots in the articles themselves or? IE: I think if people have basic information it doesn’t make any difference. IR: But did the readers know it was written by a robot? IE: Sorry? IR: Did you tell people like, by the way this article was written by a robot? IE: [Yes because its kind of an experiment done by another (media) = IR: Oh I see, yes. IE: = And they say yes this article was written by a robot and not a human.] IR: Ah ok interesting that is good to know. So what is your kind of your overall opinion on this? Is it beneficial or not? IE: I think its beneficial if you just need to use some figures or basic information, like time, date and place. I think that is enough. And if you want some in-depth reporting like, investigation or like that then that part then people are irreplaceable. IR: Yep, yep, interesting. So if you were to write a news story and would source an article that was automatically written? If you were planning to write a news story about, something that came up recently in the news, would you write? IE: (That depends on where) the artificial article comes from, a reliable media or a reliable source. IR: Interesting, so now I want to look at the sourcing and discovery of content, so how do you or your company in general how do they go about discovering news stories, what is their starting point, what do they do find a new story? IE: Well, during my past years one of the most bothering work was to discover news, yeah actually there is no real news, because when you see something happening, somebody else has already uploaded something or posted something. So we spend a lot of time online searching for using some keywords like a place or the areas, yeah we spend a lot of time searching online. IR: Do you search on social media too? IE: Yep. IR: There is like a social network equivalent to Twitter right in China? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 73
IE: Weibo? IR: Weibo, yeah, yep. Is that used quite a lot? IE: Yeah, quite a lot because its posted by individuals, for example if I pass by some accident or something I can post a picture and the news agencies will find my post. IR: Is there like a news agency in China, like the associated press? Kind of a source of or place where all people go to get news. Like the associated press, a newswire or press agency something like that. IE: No I don’t think so, we work in separated companies but for example for special events or activities we have such gatherings for all of the news reporters, journalists gather together and share information. Actually we work independently. IR: Yeah, ok so in terms of their sourcing process is it, do they find it easy or difficult? Are there any challenges involved with that? IE: The biggest challenge is you can never discover anything new, (real new). That is the biggest problem, and that still we can discover (known by real people), so we try to, or otherwise we can dig into it deeper. Sometimes people post a few sentences and a few pictures and we can just go to interview them and discover the stories behind them. IR: Ah ok, interesting. So for the second part I am going to share my screen. ((IR shares screen with IE)) IR: Can you see yourself? IE: Yeah. IR: Great, I am going to show you some images, the company I am working with is called, oh I am getting a call from someone else, he’ll have to wait. So the company I am working with is called the content exchange. And basically its this new start-up that basically want to produce a marketplace for content and the aim is produce an online platform. ((IR addresses technical difficulties)). IR: I am going to show you the dashboard. Here we go, here we go, ok this is the dashboard. So essentially what happens is you search on the left column here, any topic you like. And then it comes up here as a feed, in a column so here is just some examples ((Referring to feeds displayed already on the screen)). You can get an overview of all of the content, all of the stories within that topic. So these are placed by other people, either freelance journalists or other people, journalists or other outlets. Anyone can offer their content on the marketplace, so its kind of for freelancers and things like that. And it also covers text, audio and video so it is multimedia. And you see along here you have a recommended feed so you also get personalised recommendations, and within a category you can refine it so if you want sports you can then pick a type of sport that you want to view. ((IR switches through images)) You have a profile like this, a login screen and yeah you can view a story like this. You can either purchase the whole text or a parts of it or just the images depending on what you want and what you need. Then you can also share feeds within editorial teams, like this. So if you if you are an editorial team you can also create custom feeds. You can also get personalised recommendation based on your usage patterns or what other people similar to you are sourcing to. Of course the main challenge is to try and discover content that is you new and not reported on yet, they is what they have got in mind. IR: So I will leave this one on screen, so what is your overall impression of this system? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 74
IE: Well I think, my question is if I want to join this editorial team, do they have an identification for you as a journalist or reporter? IR: Yes you make your own account and then you just get started. Its for any news outlet or any freelance journalist, so I mean every newsroom has a team right dedicated to sourcing articles, so its kind of. IE: Do the journalists or reporters need to upload some proof, like our press card our something? To confirm that. IR: That’s a very good question actually, I will pass that on. IE: Because for me, I think that this idea is quite useful if I can use it to discover things and share my own news. But the only thing that I am concerned about is whether it is reliable, because you always need to, because the internet is not that trustworthy than the real world sometimes. IR: Ah, ok. IE: So we need to confirm sometimes, = IR: So the source is important, ok. IE: = everyone is posting everything online, posting the real news instead of the so-called fake news. IR: Yeah that is an interesting point IE: There if there is a confirmation for those reporters or journalists, I think that would be perfect. IR: Ok, interesting. And so if one of these articles for example was automatically written by an algorithm, would you still consider using it? IE: ((<)) I think that depends on the topic, a simple one something like weather reports or report for some basic figures I think is trustworthy. I think its concerning about a social issue or something that people have different perspectives on I might choose, ((<)) articles written by people. IR: Ok that is an interesting point, and looking at the design itself and the user interface what is your impression of that? Does it make sense? IE: I think its clear. IR: Do you like the columns, like this? ((Pointing to image on shared screen)) IE: Yes. IR: You then scroll horizontally, depending on how many topics you have. Does it make sense? I mean, so I know online, a lot of users on social media for example, news articles are personalised based on usage data and stuff like that. So looking at the recommended feed here, how do you feel about this, having a personalised recommendation based on user data or of those similar to you or what they are looking at, do you think that is a useful feature? IE: Yep, I think so. IR: Do you feel like you would still be able to, if there was something there like an interesting article that you haven’t previously considered, would you become interested in it? So you AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 75 scrolling through these feeds and on the left here you see something, “oh that’s interesting”, not what initially thought of but could be interesting, does it sound something that you? IE: I guess I would first try to discover something with some keywords, its pretty great if big data can provide you something more. IR: That is interesting, great, great. So considering what you said earlier about your routine, do you feel like this could fit into your routines in terms of discovering content? IE: We had something similar, because China is quite large geographically so we can hardly go anywhere ourselves in the province, we have 13 cities. So we have kind of a similar website, basically the TV stations from each city will upload some useful information, and (we are at every city in the province), so we can discover what is happening in the other cities. IR: And what is that called? IE: ((<)) It’s a company website its called Litchi Cloud, litchi is a symbol of our company and what ((inaudible)) and the cloud, as we upload to the cloud and share. IR: Yep, yep, interesting ok, how does that look like, does it also use columns? IE: No IR: Or is it just like a search engine? IE: Like there are 5 or 10 pieces of information per city. IR: Yep, yep, so with that in mind, do you like a platform like the content exchange can add on to that? Or do you feel its not really a replacement for that system? IE: [I think its useful because we, no one can really have enough information = IR: Oh yeah that’s a good point, I see what you mean. IE: = (exchange is important).] IR: Yeah ok, so having a broad range of topics is quite beneficial to maximising discoverability. So, would there be anything else you would look for in this kind of system, I mean does it solve some of the problems you mentioned earlier in terms of finding something before its newsworthy? IE: I think its (somewhat) helpful, there is no such platform that is gathering all the information, so we need to search on social media A, social media B, social media C, still. If there is such kind of information where all the agencies can just exchange information, that would be very helpful. IR: Great, ok I will wrap things up, so going back to this and automated journalism and just technological change in general, where do you see it going in the future, for journalists and news in general? IE: Well I think journalists is, this job is somehow being replaced by technology and replaced by (ordinary people) because in the past, there were only news reporters for the TV papers, for the TV stations that there might be no other ways for the audience to gather information, and now with the help of the internet everyone can get information, everyone can be a journalist him or herself. So I think the future of journalism personally I would say would go deep with some essential issues, also we need some professional editing, for example the coronavirus is a world issue now, so a lot of journalists and reporters are reporting news and relevant news from different texts, so because I was a journalist and now I am not, and if I were in Wuhan or AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 76 somewhere very special I think my previous work experience can help me to share my own information a bit more professional than the other people. IR: yep yep. IE: Because we need to, for example if I want to tell something on the internet, I will, for me I will also always focus on the source, the time and the basic information very clearly. Some people might share unconfirmed information, but for me I would only do something I am quite convinced about. IR: What about more automatically written news in the future? IE: I think so, for example every government is reporting the daily figures of the infected people, whatsoever, so I definitely think it can be replaced by. IR: Super, I think that is everything, thank you very much. I am going to stop the recording.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 77
Appendix D Interview Transcript C
Transcript template Interviewer (IR): Sebastian Short Interviewee (IE): Participant C Date: 24/04/20 Time: 9:30 Setting: Video call with living room as a background
IE: So first could you please tell me what your occupation is? IR: I am the chief editor of {current company IE works for}, a local newspaper, the online part is important as the paper, the print part. IR: Indeed yep. IE: Especially in these times I guess, in print every we reach about 400,000 to half million people in the region. IR: Oh wow! Big reach, so the {company} is not just online, are you on other platforms too, do you have a print operation or radio, anything like that? IE: We have a site and an app, but on those platforms ((<)) it is more and more video podcasting, so we don’t need to have a radio station anymore. IR: Yes indeed, that’s a point. So how long have you been working within journalism? IE: 25 years almost. IR: So have you always been at the Brabants Dagsblad, or other places? IE: No I started at the Gelderlander, in the east Nijmegen and then I worked as a political reporter in The Hague, I have been for 15 years at the Eindhoven Dagsblad as an editor (in the evening) and 9 years as the newsroom manager, and 3 and a half years now I am at {current company}, as the Chief Editor. IR: Perfect, so how many years have you been in journalism for? IE: I have for 25 years been in journalism. IR: 25 in journalism, ok. IE: The first few, about six writing and some years a bit in between and about 15 years I am managing I am afraid. IR: All right, so not thinking about just perhaps not your job now but your previous career over the years, how do you feel like technological changes have impacted your work routines, since you started working within journalism? IE: Those changes have been enormous, when I started even there wasn’t a mobile phone, ((IR and IE laugh)) and the last few years smartphones have changed everything dramatically, which means that reporters are making videos with their phones, but also thing got really really fast, everything goes fast. But still, the creation part is in the hands of journalists. IR: Yep, yep yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 78
IE: [The next steps is that more and more will be done by the computer, starting by using very simple and short parts of text concerning for instance simple football matches = IR: Oh yeah, yeah. IE: = that are not enough for the big story.] IE: Also, I was in Denmark and the newspapers who were noticed readers who were really fond of prices of houses on the market the cheap houses, the buying of houses, and they have an automated system that provides their readers all transactions on the market. And it is not at the hand of a reporter or consumed in it, it is fully done by computers. IR: Interesting, interesting. With that point in mind, that is actually something else I am talking about today, I am looking at automated journalism in general. What is kind of your opinion on that at the moment? IE: I think that all parts of the work that could be done with automation that should be done like that. There are of course parts that are expensive, and that some point they can do more than a computer can. They are creative and they can find those subtle human elements. And because we have every year fewer of them, while there is more work to be done, they have work to the automisation. I am very open and positive to it. I think in the world there are now many examples that shows us that it can work. IR: I have read quite a few examples of it yeah. Like the LA Times had a Quakebot, because there was earthquake in Los Angeles and within 3 minutes the LA Times had reported it because they had a quakebot, that detected seismic data and it automatically wrote an article about and published it. So yeah it has been used quite creatively I think, so that is interesting to hear. So do you use it at all, at {Company name}. IE: We don’t have the tools yet and I know that we are part of {parent company}, they are trying to make steps on it, and it will be within a years. IR: Interesting. If you are sourcing a new article, would you be happy sourcing an article that would be automatically written? IE: Oh yes, sure. You know for some, (publishing the news) is based on data, which is almost every news. There are just a few ways that you can write it down. Either you are a human, or you are a computer. And those sort of passages, its not a place where you make a big difference for your readers, so if it helps me to make better use of the reporters that we have, I will be happy to use it. I don’t think that you should use it to replace your reporters. IR: Yeah I would say that is a good point. So finding a balance between the two? IE: Yep. It can help you to make better use of the reporters you have, to have them write things that are more valuable to your readers. IR: Oh yeah definitely that’s a good point. IE: So, not to fire them. ((Both IR and IE laugh)) IR: Exactly, that is music to their ears. So going back to the {current company} itself, has it always been online? IE: Can you talk a little louder. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 79
IR: Yeah, sorry. So going back to the {current company} itself, has it always been, how long as it been online-only for? IE: We were among the first local newspapers that had a website. IR: Oh, really? Ok. IE: Its about 24 years ago, Eindhoven Dagsblad 25 years ago was the first. {Current company} was the second, for a long time we filled it, but was a bit like fishing in an empty sea, the market wasn’t ready for it. Ok, there were people who wanted to read it, they certainly didn’t want to pay it and advertising really was small. So the only good reason to do it was to pace up with the technique to be relevant. IR: Yes, yes indeed, and so because I know the issue of generating revenue online is a challenge nowadays, do you feel there is a good balance between journalistic content and effective financial performance? If I may ask that question. IE: Between? IR: Like the quality of journalism and the financial performance so far. IE: Yeah sure you may ask. For a long time, their wasn’t. For a long time the only thing that counted online was the reach. How many browsers do you have on your site? But things are changing rapidly, its about one year. Many newspapers and {current company} are having this paywall. Three years ago people thought no one will ever be willing to pay for something online, but then came Spotify, then came Netflix, people were used to paying for online content. About half a year ago, we really started to focus on making stories people want to pay for. And no longer all those accidents and fires, you know journalistic small things that are less important. Because this is not the content people pay for. They want to read it, but not pay it. You can then be a member, what then people pay for is quality journalism. IR: Oh yes, yep. IE: For us that is a good thing, because quality journalism is what we get in the newspaper. So these things go in parallel together now, so we are really focusing on making good things. So this year I think we will have some few thousand digital subscribers. IR: Oh wow. IE: That is really something, I told you I am 25 years in journalism, these 25 years our circulation was declining, declining, declining, at times 5 percent a year. But now, for the first time we have more subscribers than a year ago. So we are growing and of course digital subscribers pay less than print subscribers, it’s a group that’s growing, you don’t have costs from delivering and printing. So for us it’s a good time, its good news. IR: That’s great, good to hear. IE: The best part is all about quality journalism. IR: Yeah that is nice. Yeah it is finding a balance nowadays, I have heard mixed things about that. That’s really good yeah. IR: I would like to move on now to the sourcing routines, so when looking to source a new story, what is kind of your routine, or starting point for new information, like where do you begin, what’s the process when looking to start a new story? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 80
IE: One of our strong points as a local newspaper is that we know a lot of people and we know a lot of local sources. ((<)) During that time and we almost exist for 250 years, that the sources were institutional, so it was the community, it was the politics that provided the articles. But we made a shift towards the stories of the people, the true people. So nowadays it’s a bit of institutional journalism, human stories. The part we are not very good at and that is data journalism. We (don’t use) data, sometimes because they are not there, but also because it still isn’t in our routine very good. IR: Ok, so do you rely much sourcing stories online, or because its more local its more, in person? IE: It’s a lot in person. A lot of stories coming out of the towns and the villages. People who have (a job) for example, or people always want to read what happens in their city. IR: Yeah, yep. IE: But there is also a lot of politics, crime involved. But we don’t have a lot of data analysis. IR: Ok, interesting. Do you use the ANP at all? Or is that too broad? IE: Well they’re not very local. IR: I see, I see. IE: We’re working together with the Algemeen Dagblad, that is the national and international news, and they make a big part of our print edition. IR: Ok, interesting. IE: ANP isn’t very local at all. IR: I see, yep yep. That makes sense. Ok, so we move on to part 2, so I am just going to share my screen ((IR Shares screen on video call with IE)). IR: There we go, ok. Can you see my screen? IE: Yes, I can see it. IR: Great, so I’ll zoom in a bit ((Showing dashboard mock-up images)). So this is the content exchange, this is the start-up I have been doing my thesis in collaboration with. Essentially it is an online marketplace for content, it can be not just text it can also be video, audio and images too. So what you are seeing on this feed on the left side, you can search for a topic and then it comes up as a feed like this. These are just some examples, so then you have feeds down here, and here you get personalised recommendations, based on your browsing habits. ((IR switches image)) and here you can purchase the whole text, or just the images to reuse in your outlet. I guess the goal here, anyone can offer content on the platform, so if you are a freelancer or independent you can actually make money from your content. ((IR switches image)) So you can also have shared feeds so an editorial team for example you can share with someone else. You can make a custom feed with certain filters and things like that. ((IR switches image)) This is the login screen. ((IR switches image)) This is the login screen, you have your own profile like this. So within sports can filter the type of sport you want to know more about, and the source itself. ((IR switches image)) This is more examples, some more examples here, just with the filters, ((IR switches image)) and here again you get personalised recommendations based on what’s trending and your own user data like that. So that is the Content Exchange, so far its… IE: So this is not directly for readers, this is for news organisations. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 81
IR: Yep it is for organisations, yep. So I know that news a lot nowadays is kind of for readers is, the kind of stories they see on their newsfeed is based on data and this turning that around on an editorial team. So what is your overall impression of this system so far? IE: ((<)) The proof of the pudding is in the eating and the quality of journalism that is profiled by it. ((<)) What I see is that ((<)) most news organisations try to create content that makes them unique and that makes the difference for them. IR: Ok. IE: You have those things that you need to have it, but you don’t have to make it yourself, but because everyone has this sort of news, you don’t make the difference with it. IR: I see, yep. I see. IE: And that could be problem when you make a platform like this, the things you put on it is, everyone can buy and use it. Nevertheless, if it is really good and its simply really good, it could work. But if, its really good because there is a really good reporter on it then you always have the opportunity that news organisations will buy someone away. (So it would take you into service.) I think its complicated to make it work. IR: Yeah I see what you mean, so if there was like a {local region that outlet reports on} feed, and there were people that were, for example local freelancers that were offering local stories about the area, Would you find that beneficial? IE: It could work. IR: If there’s perhaps something you hadn’t seen before? IE: It could work if the quality is ok and I guess it is not easy to make the quality overall, good enough. But at the end of the day it is simple as that really good stories are shared there. It is not very good, or it does not match the standard of the news organisation, then its difficult, not going to work. IE: We had some projects, its groups of journalists, freelance. Mainly investigative journalists, and its quite difficult to make a match, because a news organisation has something of its own culture, its own habits, its way of doing things and there should be a match. Sometimes it is not easy to, accomplish that. IR: Yeah, I see what you mean. And if some of the articles you came across were automatically written, would still consider them? IE: Oh that wouldn’t make, it could make the difference if about what sort of content it is. Automatically written could be just small pieces in your newspaper or your site. More in your newspaper, on the website small isn’t very useful because, there is not much space on your site or app. Everybody thinks that newspapers hasn’t much space, and online is never ending, but no your site, people don’t scroll 2 metres down. IR: Yeah, that’s an interesting point. IE: (it more stops there). Newspapers are the place (where you can put small in use), it could be useful, but only for small things I guess. IR: Ok, interesting and in terms of from more of a design perspective, does the layout of the system make sense to you, does it look ok, its it easy to navigate?
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 82
IE: At first glance it does. IR: And it seems like the columns and things like that, does it seem logical?
IE: Yep. IR: Ok, great. IE: (More of a content planning system, yep.) IR: So, going back to more sourcing routines in terms of discovering content, do you find that sometimes, when searching for one story you accidently stumble across another one, that becomes more relevant. For example you have something in mind, then accidently, or you come across something else that is very interesting and eventually becomes a story. Because that is an interesting idea. IE: Sorry I didn’t understand your question exactly. IR: ((<)) Let me go back a step, so the system uses personalised recommendations, which you feel that’s useful for you? Based on what you or similar users are.. IE: It could, but for there’s a difference. When I’m only a reader its ok for me when someone chooses. When I am a news organisation, I want to make the choice and I don’t want to have it done by an algorithm. IR: Yeah, I see. IE: That’s just a program, you know its less work to find things you like. So maybe the second thought is ok it could work, it could work. IR: So if the system, but this is still in work I think it aims to try and recommend articles, because you have to recommend something that is up and coming. ((Connection issues with video call)) IR: Oh sorry let me say that again, so the aim is to eventually to try and as is priority for journalists is to kind of get a story before everyone and as its emerging. So if there was a recommendation that was a new story that hasn’t been reported on much yet, and it was upcoming and you researched a bit more on that topic and it was quite a relevant topic for you. Would that work? IE: Yeah it could work, and because at the end of the day it is still, is the story good or not? That’s an important question, on top of this could it be exclusive for us? So if everyone can use it, I would say, well we make just a short story of it. We don’t make the difference with it. IR: So there is kind of a focus on quality and relevance? IE: Yep. IR: Ok so just to finish things off, I will ask, in terms of automated journalism, artificial intelligence and technology in general, where do you see it going in the future for journalism? IE: I think the next within a few years is to have the small news, like football matches maybe also the short police news, crime news, car accidents, things like that can be done automatically. Its not such a big step, it happens everywhere. But I think that the top stories will always be done by reporters, why? Because your reader wants to identify with it and maybe at first glance, AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 83 they won’t see the difference, and maybe they won’t identify with a computer. But that will take some years. But for now it is good to see the face of someone. IR: Yep, yep. IE: Yesterday I was watching an episode of House of Cards on Netflix and in it was a fragment from CNN which has a beautiful reporter whose name is Wolf Blitzer, one of my favourites. If he is doing an item its always good for me. You always want to feel a bit, and for that it is good to know who is a writer of an article. IR: Yeah, I see what you mean. And also, one final question just to check, if there any other comments regarding a system like the Content Exchange, do you think it’s a viable idea? IE: Its difficult to say, also because on the local markets you wont find many customers. We are a big company, with other newspapers, and you have one which is on local radio, {company name}. (It many not fit I guess) and there is not much money on the market. I am not saying it can’t work, but it will be really hard to make a living out of it. IR: Yeah, local is bit of a different challenge, ok. IE: On a national level there have been some things like novum, which was a press agency. It’s a tough market. IR: Yes, yes I understand. All right super, I will stop recording now because that is all, thanks again for your time.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 84
Appendix E Interview Transcript D
Transcript template Interviewer (IR): Sebastian Short Interviewee (IE): Participant D Date: 24/04/20 Time: 13:00 Setting: Video call with living room as a background IR: We are beginning, so thanks a lot for your time. So firstly could you please tell me what your occupation is? IE: I’ve done a study in journalism, HBO study. And at the same time I was studying, HEAO, that’s administrative and economic studies. That was a study that started 25 years ago, and it was called ‘journalism and management’. So you studied journalism and some economic courses. IR: So right now you are the chief editor at {current company}, ok what kind of roles have you had before this? IE: Before this I had the same role at a different regional station, in the south of Holland, before that I was Editor-in-Chief at ‘Kassa’, which was a consumer programme on television which had a website and a radio programme, and before that I worked in several different jobs in national broadcast. IR: Interesting, so how many years have you been working within journalism? IE: 25 now. IR: Great, so the {current company} is not just online, right? You also have radio station and a TV platform? IE: Mmmhmm. Mostly importantly it is online, but namely our TV is second best. IR: Ok great, so referring back to your past 25 years in journalism, how do you feel that technological change has impacted your work routines? ((IE did not hear question, IR repeats question)) IE: ((IE chuckles)) Everything, well it changed everything. Not only the way we distribute our news but its also the way that we collect the news. Its mainly online, when I started in journalism it was making a phone call and going out on the street to make your report, but that has changed dramatically. I think the biggest change is the way that, the public is consuming the news. We don’t have the only right on news anymore because the right on the news is not with the journalists anymore. IR: Yeah, yeah that’s a good point, yep. IR: And so, with that in mind, in terms of {current company}, how is the balance between online, radio and TV, is it equally shared? IE: No, no, no, absolutely not. {current company} is a company that is mainly online and has had its biggest successes online, so mainly the news has been gathered and distributed first and AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 85 foremost, online. And radio and Television are consumers of the online strips, so everything that has been online comes to radio and Television afterwards. IR: I see, so was it always online, has it existed before? IE: No, I think {current company} was one of the first regional companies to be online and that is why it is very successful now online. Last month we had 3 million visits, in one month, that’s extremely, much. But I think it is the result of being online in an early stage and that’s what {current company} has done. IR: Interesting, interesting, so do you feel like over the years this transition to the online world we live in now has put pressure on your job, on your role? IE: Not pressure, but it has changed it dramatically, but not pressure. I think maybe it has become easier to do the work we do, because 20 years ago you needed a network to get your news and now there are so many online sources and there are so many data that can be collected to make news of. I think it has made the job easier than 20 to 25 years ago. IR: Just to quickly mention, in terms of news topics you cover, what kind of topics do you cover? Is it just in general news you cover? IE: Its general news concerning the whole province, so that’s a problem because what’s a province and what’s interesting in the province, that’s. You talked to {another participant} and he is covering just a small portion of the province and I think that is easier to make news, we cover the whole province. IR: So, do you feel like then, is there a good balance between quality and financial performance at the moment? IE: ((>)) Specify the question for me please. IR: So, I have spoken to others and it depends on who you speak to, but sometimes the moving to online news, some people feel that the quality of articles is going down in some areas with the prevalence of clickbait and stuff like that. Do you feel that the online world, has put any pressure…because I know that the business models have changed. IE: I think that’s true, because we are in very strong competition between the news titles, like the one from {competitor} and that makes it even more important to act very very quickly, and be very correct and have all of your facts and figures in order. But on the other hand, digital news has changed the way maybe television can work too. Because, television and radio have always been in a broadcasting company, a base for the news. So now the base for the news is online journalism and they use it to make television and radio, and that has shifted 180 degrees. IR: Ah ok, so in terms of business models, it impacts how financially successful some outlets are? IE: Yeah but we are a public broadcaster so the financial consequences for us other than a newspaper like {competitor} are having way less impact than commercial organisations. IR: Ok, that’s interesting. So a more recent technological development that is finding its way within the journalistic world now is automated journalism, I don’t know if you have heard of that? IE: Yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 86
IR: So like using data to automatically generate articles, for example the Associated Press, all of their financial reports for example are all automatically written, based on the data in the markets. I am wondering what your opinion is on that? IE: Well we could discuss it for half an hour, but in short I am a real believer in the principle of robot journalism. ((<)) Why is that? Because the biggest broadcast in Holland NOS has stopped an experiment, saying that it doesn’t bring us anything, its only useful in financial information and sports information, and sports information is being bought by a robot in way that people can see it is robotised news and they said, well if this is collected from certain facts the audience can find the facts themselves and they don’t need a robot to make a good text out of it. IR: I see, yeah. IE: And I think that is the main problem of robotised news, some facts you are searching, you can find the facts, and you don’t need it in a good language. If you want to read a good article, then you need a human person to write the article. So I am worried about the future of robot- journalism. IR: Do you think its use will increase, or do you see it fading away? IE: ((<)) I don’t know, I don’t know. Because maybe everyday I am reading robotised articles which I am not aware of that they are written by a robot, I don’t know. I am not sure if it has the future, I am really not sure. It’s an experiment that has been started in the United States many years ago already and it wasn’t too successful there, as my information goes. Maybe you have different information. So I am not too enthusiastic about it. IR: Ok, that is an interesting perspective. IE: Well, it depends on what you categorise as robot-journalism because any of the stories that we have like for instance on teletext, are made by robots, because no human being comes in- between but that’s what I call journalism, what I call collecting facts and figures. IR: Yeah I see what you mean an example I found was, in Los Angeles there was an earthquake in 2014 and the LA Times had what is called a Quakebot, so basically it detected the seismic data from the earthquake and it automatically wrote a text and then it published the story about the earthquake within 3 minutes after it happened. So yeah it guess it depends on the story and what kind of, where the data is coming from. IE: That is exactly what I can mean because I think that the audience has grown up, has become so mature, I don’t think they need a story from the New York Times or whatever. They can find their sources and if they where to find their sources they can find the website where the earthquake is being measured and they don’t need a newspaper to make some really nice sentences about this earthquake. IR: I see, yeah that’s a good point. IE: That’s where I think the problem is with robot-journalism. IR: Yeah, I have read similar views, that is interesting. So I’ll move on now to sourcing routines. So for you and {current organisation] in general, for when looking to source a new story, what is your routine? Where do you start and what do you do to try and find a new story? IE: First of all, stories come from people that live in the neighbourhood and all the journalists in our company live in the province. And the best stories come from the people themselves, but I have to admit 30 maybe 40 percent is also from the internet. Twitter is a very, very main source AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 87 for us. We have many tools that can find news for us, like Coosto, which I think you are acquainted with that, so I think 30 to 40 percent is collected by online sources, and then being checked. But the rest, the best stories we have are from the persons, the journalists themselves. IR: I see, do you use something like the ANP, or is that too national? IE: Its national yes, but we do have an account with ANP yep. But it’s mainly used for the photos they have because, in our province we have it first then ANP has it. IR: Yeah, yeah interesting, and with that routine in mind, is it, do you find it easy or difficult? Is there any problems with that? IE: To find stories you mean? IR: Yep. IE: Nooo, noo we live in the province with the most news in the whole of the Netherlands I think, so that has never been a problem. The problem we have now is to find the best people to make it for us, the workers at our station. Some years ago there was a lack of jobs, now there’s a lack of people to fill the jobs. IR: Yeah that has kind of flipped around. So is {current organisation} quite data driven? Do you use metrics like page views and click-throughs to measure your success? IE: Absolutely, absolutely. I think we use the same system, but in a different way that {competitor} uses it, its called content insights and its shows not only how many times an article has been viewed, but also the loyalty, engagement, et cetera, et cetera for the audience. So that is how we measure how the audience likes our news. Despite journalists who don’t want to know that their story isn’t good enough, we are data driven because the stories people don’t read or don’t get loyalty to, we skip them more and more often. IR: And that’ quite a discussion at {current company} because that’s what the journalists say, if we are only driven by what the journalists wants, you can give them a real good steak and some French Fries, if the audience only wants French Fries, are you going to stop serving the steak? And the answer we have is no, we try to serve the steak, but if they don’t want the steak you have to prepare it better. IR: Ah ok, that is a good metaphor. Are they any personal factors that can start a story, like curiosity, being curious about something or? IE: That what’s I call the personal stories. The journalists come with those stories because of their personal internet in things. IR: [I see yep, interesting. Ok, great so I = IE: I am very curious about your demo. IR = yes, that is my next step, so I am going to share my screen.] ((IR shares screen on video with IE)) IR: Ok, can you see my screen? IE: It’s still loading, yep. IR: Ok can you see the dashboard? IE: Yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 88
IE: Well I don’t see a dashboard I see a white page and on top of it is… IR: Yep ((referring to the blank TCE dashboard before any feeds are generated)) IR: So this is the Content Exchange, let me type in something here. ((IR experiences technical difficulties with generating feeds, so IR switches to the mock-up images)) IR: ((Shows the first dashboard image)) So, you have the first search box on the left, this is just some placeholders here. Each feed is like a column and you can see all of the stories related to that topic. And then you also you can have personalised recommendations along the side based on various sources of data like your usage data and what’s trending and things like that. IR: So I’ll get some images up ((IR switches image)), so you can see within a topic you can refine it a bit more. So if you are looking for sports you can specify what sport you want, you can search within the feed too. ((IR switches image)) You can create custom feeds, let me continue here. Then you have a recommended feed page, so you can open the article up, you can either buy the whole text, part of the text or just the images. This platform also covers multimedia, so video, audio and images too. ((IR switches image)) So you have a profile like this. ((IR Swithes image)) and there’s the login screen. ((IR switches image)) You can also share feeds within your team. So an editorial team can share custom feeds with each other to be more in sync. IR: That is basically the rundown of the platform, so the goal is kind of to have a marketplace for content, so anyone who’s writing an article can upload an article onto the system, and anyone like an editor or a publisher can purchase it for a fee. You would have to ask the founder {name} a bit more about it, but so far its early days in terms of development. IE: Help me out with one thing. Is this an platform for sharing articles and sending them to people, or is this a dashboard helping journalists to find the news? IR: Yeah it’s a content sharing platform, for example you could be an editor, it is a dashboard and editors and people who are finding content try and. IE: Ok because now I am seeing sources of Dutch news. I mentioned the dashboard of Coosto, which we are using a lot, which does exactly what you are showing now, within Twitter and all of the news sources. But I wanted to make sure, is this a platform to sell news, to news companies or is it a platform to find news within the open sources the internet has. IR: It’s a platform to sell news, so for example a freelancer could offer his content on the platform, then an outlet could buy it for use on their website for example. IE: Very interesting. IR: So what is overall your impression of this content exchange, is it similar to what you use already? IE: Sorry? IE: Is it similar to what you use currently? IE: No we don’t use that, because all the content we make is made by ourselves, so we don’t buy, well of course we buy content but that is from freelancers that come inside our company, or send it us so its not on another platform. Its very interesting, really very interesting, I have some questions that come to mind immediately. IR: Yeah, go ahead. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 89
IE: First of all, copyright. I know that it is a competitive market and I am not sure if all of the editors would be so honest to buy a story, or maybe write the story over or make it themselves. IR: Yeah maybe that is one of the issues they discussed in detail is the issue of copyright. I think they are thinking of that as we speak, it is something to consider. IE: At the ANP, that’s the national press agency, they have news, a newspaper collects the news. They have one phone call with the same source then its their news and not the ANP again. IE: So that’s one problem. The second is, is there not already a platform like this? I don’t know. IR: Also a good question yeah. I have heard from others that there have been similar start-ups in the past that have not been so successful. So I am not sure what they have in mind, it is still in development I think, I am curious to how they will differentiate themselves. IE: Further, I haven’t seen a place for audio and video, and audio and video is so important nowadays. IR: Yeah I think that will come later, in the test environment. IE: Ok. IR: So, it is planned, but so far you can only view news articles and images. But definitely it will come. IE: The last one is, is the Dutch market big enough to need a platform like this? To make it grow. IR: [Also a very good question, I think they are looking internationally already = IE: Ok IR: = to the US and the UK, but early days. But of course with the current situation we are in, with the coronavirus it is difficult to reach people. So, yeah, it’s definitely something they are considering.] IE: Because if you look at the province, is one of the best covered provinces, when you look at journalism, the {parent company} that owns {regional outlets} and they understand they already have 190 journalists in the province. We, {current company}, we have 150 to 200 people working within the province so, {province} is one of the best covered in terms of journalism. But, in this best covered province, there is only two competitors that’s {competitor}, the newspapers and we. So, if you put that to the test of this platform you are presenting, it would not get a real chance to get alive because either we buy the story or {competitor} buys the story, and that’s so, do you need a big platform for that? IE: But ok, it sounds negative, and I am not trying to be negative but I am just trying to be critical, is the Dutch market big enough, for a platform like this? IR: It is a very valid concern and I feel like eventually they will try and look elsewhere, in bigger markets. I know in the UK for example, they have more varied range of news outlets with differing opinions and stances, yeah who knows. It could be very interesting to see. IE: But on the other hand, if you should have presented for selling news, but for finding news, on which you have many, many sources that we don’t even know. A perfect way to search these sources, that could be very interesting. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 90
IR: Yeah indeed, that has come up a lot too in my previous interviews, you know the sourcing, where are these articles coming from, is there a confirmed source? So that’s some feedback I will pass on. IE: Don’t let them make you crazy with all the feedback and all the criticism because I really love the idea. I love the innovation that comes out of this idea really, but its not meant negatively. IR: [Oh yeah, you have got to iron out the kinks before you launch = IE: yep, yep. IR: = yeah, definitely.] IR: So, if one of these articles was automatically written by a robot, would you still consider using some of that content? IE: ((<)) I don’t think it makes any difference, in the process of buying things or not buying things, whether or not it has been written by a person or a robot. Considering that fact, that the quality of the articles should be perfectly the same. IR: Yep. IE: So the question before is, are articles written by robots as good as written by a human person. IR: Yeah, that comes up quite a lot. IE: Yep. IR: The more richer texts. IE: Yep. IR: I think the technology is slowly catching up with that. So, looking at the personalised recommendations do you feel like that’s a useful feature? IE: Yep, absolutely, absolutely. Finding our news online, we depend on tools like Coosto, which is so perfect for us because it knows what kind of news we are looking for, and it gives us recommendations of where to find the news or even finds news that we haven’t discovered yet so, yeah personal recommendations are very good. IR: So sometimes you can stumble upon something that you haven’t previously considered? IE: Yep. IR: Ok interesting. IE: Coosto works with very, very clever algorithms that’s learning by seeing what we do with the news so that is very important for us. IR: Ok, great. And so in terms of the interface and the design, what is your first impression of it, is it logical? IE: It is very good, because most of the tools that professionals use and journalists use don’t have such a good interface. They don’t have such a good look. So it was very good looking, I am very positive about it. IR: Did the columns make sense? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 91
IE: Yep, perfect. IR: I think the goal is, you have a lot of feeds you can scroll horizontally through them all, and you can scroll down them and search within them, things like that. ((IE asks IR to demonstrate the demo again, but again it does not work)) IE: Is this also a platform meant for selling pictures? IR: yes, yep. Multimedia. IE: Yes, because you will really have to consider the rules of copyright and author rights. It is so complex. IR: [Oh yeah, the legal issues are going to be, something else = IE: the hardest ones. IR = entirely, yep.] I think you are right there, it came up when I first met the team. I think they have got it in the planning. So I have to ask, this is something unrelated, I don’t know if you know the name {TCE employee}, he has previously worked at {previous companies}. But he came up with the idea and they need test outlets to offer their content on the platform and he wanted me to check with you if you were possibly interested in sourcing some content? IE: I am very interested in testing, we really love to test because we are a really online and modern company. As I told we are very big online and we’re the first company to use metrics and data-driven journalism within regional broadcasting. But, sending content is not possible because we are a public broadcaster, so everything we have made has been paid by your and my money and we can’t sell it again. IR: Ah ok, I will bear that in mind. I will just wrap up with a question. IE: But I am really interested in getting a demo account, so I can show it to my colleagues, when it’s finished. IR: I’ll send you an email of {TCE employee} and you can get in contact. IE: Yep, yep. That would be perfect. IR: Sounds good, so just to finish things off, so in terms of everything we’ve talked about, in terms of AI, robots and journalistic change and sourcing news. Where do you see it going in the future? IE: I think that in professional journalism, the personal touch and the human factor is becoming more and more increasingly important. In an online world where everybody has news, brings news, and there is a lot of fake news, to attract people to come to a website, more and more. We as professional journalists will be working on bringing the real news. And I think more and more we will be recognised as the ones who can be trusted. So I think that in producing our news online, metrics, robot-journalism and all tools that we have will become more and more important. But in producing it, in making it, we finally need some really good journalists to make what we produce for the audience. IR: Great, I think that is a good conclusion, I will stop recording now, thank you very much.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 92
Appendix F Interview Transcript E
Interviewer (IR): Sebastian Short Interviewee (IE): Participant E Date: 28/04/20 Time: 15:00 Setting: Video call with living room as a background
IR: Ok, so first I will just start with some basic questions. So what is your occupation? IE: I am a publisher and one of the persons who started the local newspaper {current company}. I have a background in online marketing and also I have a company on social media and content marketing. IR: Oh interesting, interesting. IE: Yep, (so lets start with {current company}). IR: Yep, yep, so do you cover just {region} as a city or {region} as a province? IE: As a city. IR: Great, so what are your typical responsibilities at the moment, in terms of {current company}? IE: Well, mainly the financial part, the sales part, and the online part. So everything which has to do with technique, is my responsibility. IR: Ok, great. So in terms of topics you write about you kind of write about more general news in Utrecht? IE: Yeah exactly, its general news. But what we write about as well, is city development, because there is a lot of city development in {region} at the moment, Sustainability, because {region} is also very keen on developing on the kind of things in the city for sustainability. We have a green city, how do you call that? Mayor and the parties are very green in that case. IR: Ah ok, so there’s a lot of sustainability comes up? IE: Yep. IR: Ok, interesting. Ok great, so moving onto technology then. Oh, one more thing, how long have you been working in journalism, or how long has this been going on for? IE: This is going for 10 years, so that’s {current company}, before I used to work for a national newspaper, then I worked there for something about 4 years, and before I worked in magazines, such as {magazine name}. That was about 3 years I guess, so I have been a long time working actually. But never as a journalist, but always in the commercial side. IR: In what side sorry? IE: Always on the commercial side. IR: Ah ok interesting. A different perspective, ok great. So how many years have you been in this business, in this industry? IE: Something about, 18 years I guess then? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 93
IR: [Fantastic = IR: So, quite a long time. IE: = so, since you started working within it, how do you feel that technological change has impacted your work routines?] IR: My work routines, it made it easier because we all have things like Slack or like what we’re doing now, video conferencing. Yeah, all of this tooling, is also there for sending the bills for instance. It made it all easier and faster, more accessible. So yep that’s say 10 years, about 10 years, which makes it easier and easier. IE: It’s also how we started, because we started as an online newspaper instead of a print newspaper, and a few years ago we started with a printed issue, which is the other way around normally. IR: Oh, interesting, interesting. IE: Basically the journalism is mainly the same, its only the techniques which are developed. If you are looking at journalism, if you look at for instance (delegate) methods, or a lot of money methods. Yeah which are, mainly digital methods which they using. That developed journalism, but journalism itself does not change that much, because its all the same kind of questions which are, their underlying values, which remain. IR: So what was behind the decision to go back to print? IE: Because we still believe in print, it is not ((<)), (a goal at itself). We still believe in print because, during the weekend it’s nice to have some paper in your hands, and to relax with a cup of coffee, only for the background stories and not for the news. Yeah, so there will not be news in it, but only background stories related to the news, the most newspapers still use it for the news, so that’s then you only get old people who like that. And even the old people read their news online first, instead of the newspaper, so what’s the added value there? IR: Yeah, I see. IE: But for print for background stories, we still see added value. IR: Ok, so more for the more casual reader? IE: Yep, yep. IR: Interesting, and do you feel like, in general the moving of news online has put pressure on your job, or on your firm at all, over the years? IE: With the online news? IR: Yep. IE: No not really because we started that way, because when it was our first business model, and then we started print and that added extra value. So no, its on the contrary. IR: Yeah, indeed. So do you feel like you balance content quality and financial performance effectively at the moment? IE: Yeah, that’s always an issue ((<)), we try to get them to make news, but we like to bring more background stories, and/or research. That’s a problem because ((<)) there’s enough funding at AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 94 the moment, it’s a bit harder but it still survives, so ((<)) but we don’t have a enough money to do a good enough background research. IR: Ah ok, yep. IE: And that’s something which is missing. But I think that’s a problem for everybody in journalism, accept maybe for the national newspapers. Everybody is trying to see what they could do about it. But still so far I think nobody has an answer. IR: Yeah I have heard that a lot, I have spoken to several local news outlets, so yeah similar story. ((IE switches from audio only mode to video call mode)) IR: Yeah, so and are you are of automated journalism? Robo-journalism as many have named it. IE: [Yeah, yeah, yeah I heard about it, before in sports journalism = IR: Yeah, that’s quite common. IE: = And also because the project of Tilburg, where I heard about it before, but also because Tilburg has worked with it right? IR: Yes yes I think my supervisor is working on this very subject. Making it more, you know rich. IR: So, what is your opinion on automatically written news? IE: I don’t really have an opinion about it, its more does the reader like it? That is most important for me, I guess with sports I think well, it might work, a bit. But there is also an added value of the editor, I guess. So, in the beginning it should be checked maybe, to see if the quality is good enough, and after that when all the algorithms work et cetera, I guess it could be a solution, but not the solutions, because it misses the feeling which is also important in an article. Instead of only needing statistics, I guess sports journalism now, for robo-journalism would mainly be statistics. IR: Yep, I guess the data it draws from varies by topic. IE: Yep, yep, so it might work and it might be a start of developing better articles in the near future because maybe even there are (challenges) to be solved, but it needs to start somewhere. But I also think, at least how do you call it? The eindredacteur, do you know that word? Let’s say the Editor-in-Chief always has a final verdict, or always has something to add extra. So, some supervision is always needed, I guess. IR: So for the topics you write about, maybe its not quite there yet, you feel? IE: Maybe on sports journalism, because what we do now we have a very small section of sports journalism, we only follow the main hockey fields, field hockey is quite a big sport in the Netherlands. We follow our top team, and we follow football, FC {regional team}, the top team, and that’s about it. In these articles you need a feeling and what’s happening, so you need the editor, for extra added value. Maybe, in the what is it called? The wijken, do you know that word? De wijk? [The region, the very local = IR: Oh I see yep. IE: = Suburbs, lets say suburbs. Maybe we could use it there, because more football clubs et cetera, people like to read about it also about the scores also, more statistics. So, there’s a AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 95 purpose for it, yeah and there it might be useful and that is what we would like to give it a try there. IR: Definitely, makes sense. Ok so in terms of sourcing and finding new content, what is your company’s routine in relation to that? How do you go about sourcing and finding new stories? IE: New stories? Well obviously, all the organisations which are sending us press releases, things like that. That’s one. But the main sources for unique content are people in the city who knows us. We have a network, journalists have their networks and its our main source. These are the main sources and it might also be follow-ups of articles we have already printed. IR: Do you search online at all? Or is it? IE: Yeah its also online, so Facebook might also be a good one, we have lot of people respond on, for instance but also on our articles and that might also lead to new content. IR: Interesting. IE: They give tips, and everything like that. IR: Ah I see yep, and do you feel like your sourcing routines are effective at the moment, or efficient? IE: It’s always hard to say. I am not the editor, so I am not the one who’s doing it. And on the other hand, well we use from our content and social media agency, we use (tooling) for that to make a scan of all the content which is there. Its called, I don’t know if you know these tooling its monitoring tooling, for instance we have Meltwater, but you also have (inaudible) and Coosto. IR: Oh yeah, yeah. IE: You know these kind of tools? IR: Yeah I know the ones you mean. IE: Yeah, and I would say that editors might use that as well, but they don’t. Because they think that they know best and tooling does not. I think that is a missed opportunity. IR: Yeah, yeah I see what you mean. All right, well lets slide into part two shall we. IE: Yeah, yep ((IR shares screen on video call with IE and displays mock-up dashboard image)) IR: Ok can you see the picture on my screen? IE: Yep. IR: Great, so this is the Content Exchange, basically it’s a brand new start-up, it launched a few weeks ago, in collaboration with, well my supervisor is helping out. Essentially in a nutshell it aims to be a market place for content, and it’s a platform where companies, journalists and editors can share content. Anyone can offer content on the platform. For example if you are a freelancer on the platform you put it online, and anyone can purchase it and reuse for their own use. IR: So what you are seeing here is the main dashboard, so you search in this column to the left here, for a topic and then you have these feeds, and you can search for as many as you like, and AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 96 you can scroll horizontally. You can scroll down these feeds and view all of the articles you want. ((IR switches image)) IE: Can I ask I question? IR: Yep, go ahead. IE: For instance I saw nieuw.nl a lot right? But that’s already a published article then? IR: Yes, this is something I have been asked about. IE: Or is that just a mock-up? IR: I think for now it is just a mock, yeah so I think literally anyone can offer content. Whether you are a publishing house or a freelancer yourself. I think its still in the development phase at the moment, it’s all early days. In terms of how is offering the content? It’s all to be confirmed on that one. IE: Maybe its also just, use it if you like. Because we provide content for Nieuw.nl, we provide the local content for Nieuw.nl for {region}. So actually we are doing kind of a similar thing then, but only for {region} and it comes from {current company}, there are no other providers. IR: I see. IE: Maybe people who give tips or et cetera, but its strange if already content is published somewhere, but you are offering it yet again, if you see what I mean. IR: Yeah, I guess it is more for the biggest, most obvious target group would be a freelancer who doesn’t have an audience, and they can then place their content on this platform and then it can be seen by lots of big outlets and be distributed. Then it generate some money from that, so yeah the people behind this idea maybe have some other plans in mind but for now that’s kind of as far as I know. IR: So you can download everything, its not just text you can also download images, audio and video. Here you can see an example article, you can download just the images, or just parts of the text, if you see a quote you can just pull a quote from there and use it in your article. ((IR switches image)) You can also have shared feeds, so you made an account, feeds are like the ones is showed you earlier, those columns. You can make a custom feed and for example share it within your editorial team, for example if you are an Editor-in-Chief. ((IR switches image)) This is the main login screen. ((IR Switches image)) You have a profile like this. ((IR switches image)) And then within a topic you can filter it even more, for example if you are writing about sport you can then filter for a specific type of sport, if that is what you need. You can see to the left, you get a personalised recommended feed, based on browsing habits and what is trending and stuff like that. So you get personalised recommendations. ((IR switches image)) This is a similar image, you can kind of filter and search here and things like that, also you have country and source. IE: This is to be offered to publishers right? That’s the main? IR: Yeah its kind of to bridge a gap between those writing content and those needing content, is my overall understanding yep. IE: Its not the public? That’s what I mean. IR: [Yeah, yep that’s it, so journalists and editorial teams for example = IE: Ah yep exactly. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 97
IR: = they can increase their reach, that is kind of the end-goal. IR: So here you have a personalised recommendation feed, like that. So I think that’s all the images yep. That is the overall dashboard. So yeah that is the Content Exchange so far, it is very early days. ((IR stops sharing screen with IE)) IR: What is your overall impression of that system? IE: It looks simple and it looks good, that’s an impression, and but I also have like the questions I was just asking. IR: Yep, of course. IE: For who is it? I don’t understand why the publishers are mentioned everywhere, because it should be, I think it might be more of a success if you offer unique content. IR: Yes, yes I see what you mean. IE: Because if I read the article at ((IE lists local publishers that cover region that IE reports on)), which are the local publishers in {region}, then there is no added value for me, it is not unique, its not one of a kind. IR: I see. IE: And that’s everybody what like to have, their own unique content. So that is one important issue, and the other issue I guess is well, editors are pretty how do you call that? They go their own way, they know what’s best. IR: I see, yeah. IE: Yeah how do you call it? I forgot the word. Eigenwijs, you know the word? IR: Not yet. ((Chuckles)) IE: Not yet? I don’t why I don’t get the word, because I know this word, I will look it up. ((IR searches on computer)) IE: Stubborn! IR: Yeah, resistant, stubborn. IE: Yeah they are (in a way) persistent, they know what’s best, nobody should tell them, how to make an article, what’s best, et cetera. So this is something from outside, and they always like to work with their own freelancers, with their own ideas, and something from the outside. Always when we talk with someone from the outside, they kind of (hold) and reject it. IR: Ah I see, so they want to have the final say? IE: [“Not invented here”, kind of = IR: It wants to be original? IE = exactly.] IR: Yep, so maybe then perhaps you feel like the personalised recommendations could help? Because it could recommend something that has just started to emerge and perhaps they might see something that they might have not seen before? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 98
IE: Yeah, recommendations, not really these type of people, the editor are not really much open in my opinion, for things like that. Recommendations are more for travel websites, et cetera, or maybe even if these are known names, people they know then a recommendation helps. IR: Ah I see. IE: But if it is people they don’t know, I am not sure if that would be the thing. The thing that might depend on what platforms they publish on, or it would help if they know it’s a unique story, or it would help if for some kind of reason, it would help if we have for instance, AD they have a lot of news about accidents that happened. We call the 112 in the news, when there’s police involved, they publish it. And for us we like to have more background stories, a bit more intelligence, like NRC or like that, and if you could add something like that, you know if its yeah kind of our profile, then it would help, I guess. IR: So if it was tailored to you and it made sense? IE: Yeah, (perhaps) tailored, because we like to make, we call it quality news. But I guess the AD would also say, yeah we made quality news, but we say, now it’s a bit more (pulp) news. IR: Yeah so I guess the algorithm for these recommendations need to be refined and tested. IE: Yeah I guess so, but these insights I think, which is important to know if you build a platform like that and how editors pick their stories. IR: Indeed, indeed. IE: At least for {current company} that is the case. IR: Yep, yep. And referring back to the sourcing routines you mentioned earlier, do you feel like this kind of dashboard or system can fit into that? IE: Yeah, the thing is it should be part of their routine. IR: Yep, yep. IE: It’s pretty hard to become part of their routine. They should ((<)), they might look at it once or twice, but if there’s nothing there which they could use, they’re not going to use, they are not going to look at it again, because its not part of the routine. IR: So it kind of needs to work, make a good first impression? IE: Straight away. ((Laughs)) You have got one or two chances. IR: Yeah, make a good first impression, from the get go. IE Yeah, and then its gone otherwise I said, “now, we tried that, there’s nothing on it, so we don’t try it again, I enough have too, so I am so busy. IR: Yep, that makes sense, and if one of these articles were automatically written by a robot, would you then still consider using it? IE: Yeah, like I mentioned, I would not at this moment. If I relate it to your platform, it would not be the way to, ((<)) get to know it. Because robo-journalism looks like an experiment, it needs to be developed and when its developed, it could be part of the platform. But since its not well known yet, and not proven yet, I would start by just offering it separate as a unique project or something. IR: Like an experiment at the moment? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 99
IE: Exactly. Would you like to use it? And could you help us to improve it? Something like that. So then the journalist is more involved. IR: I see, yep. Ok. In terms of the design and layout, did it make sense, did it feel logical? IE: Yeah, yeah I feel like it did. The topics were quite easy. IR: The columns and things like that? IE: Yeah, yeah, that’s something which is, I think so yep. IR: I think the idea is quite similar to something like, Tweetdeck where you have lists of columns and its easy to use if its familiar. IE: To navigate, you have a lot of topics, but yeah it seems easy. UX, did its work. ((IR and IE laugh)) IR: Yeah, definitely. Do you have any other comments regarding that, the concept in general or the design and layout? IE: No, not really. With you know ((<)), I guess its proven, it works everywhere, its not spectacular but does it need to be spectacular? It should work and that’s the main function, and it does, so, yeah looks good. IR: Great, and there’s just one last thing referring back to sourcing do you think that maybe some reporters have a personal drive, that sometimes influences their discovery of new articles? Things like do they have like curiosity? Or can be become curious about a topic and that kind of develops into a story, if you know what I mean? IE: You mean our own journalists, right? IR: Yeah. IE: Yeah, definitely they have their own interest because we obvious we have, like I told you about city development et cetera, we have that from the newspaper, kind of guidelines: this is what we should stand for, this is what we are. But journalists have themselves as well, we are experimenting with now with research journalism, and our journalist is looking at topics which he finds interesting and that has got to do with curiosity. But it depends a bit, because there are two types of journalist I guess. The curious one, these are the good ones. But I found out, and we have one, we don’t have a lot of journalists so its not like they are all lazy. ((IR laughs)) But we had another one, but he was more the lazy one who is just thinking, ok I put the news online, I do this everyday, just find as quickly as I can as many articles et cetera, and that’s it. And they are not really open, to I think new things, I found that journalists are pretty conservative, also in new techniques, adapting new techniques, but also on not really too open minded to other people because they know like I told you, they think they know best. IR: Yeah, exactly, but do you feel like a platform like TCE would help, or would they still be a bit too stubborn? IE: I think that might be the case. ((IE laughs)) IR: Yeah? Interesting. IE: It really depends on what kind of article, because if you the correspondence, you know that one for us, that type of article would be very interesting. But if you start offering, sports news or AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 100 the ((<)) main news which you see everywhere, yeah (inaudible). So unique should be the key word, I guess. IR: Yes, so unique and emerging. IE: Yeah. IR: Ok, interesting. IE: There might also be for other platforms, for instance if you have these cities where, we call them ‘112enters’, so that’s people who chasing the accidents and the police to try and make the first pictures. That is not really our style and not really for our newspaper, but you have newspapers which thrive on that. So it might work there, but if you are offering with that kind of news first, we would be gone, so that’s the first impression you make. IR: Yeah definitely, first impressions is key. IE: Yeah and maybe it is good to have a kind of a ‘wizard’, I don’t know to before you go to the platform to try and ((<)) fill in the wizard, and see what is important for your newspaper. IR: Oh, so like a pre-questionnaire of what you want to see? IE: Yeah exactly, so you will have the right content to show to you. IR: Yeah, that’s a good idea, great. Ok super I’ll just have a final closing question. In terms of everything we have talked about today, about digital news, AI, automated journalism, where do you see it going in the future? IE: That’s a pretty difficult question and a very open question. IR: Indeed. ((both IR and IE laugh)) IE: ((>)) If you look at things like AI, I think the filter bubbles which are already here now, by Facebook, things like that, might even go further. That’s what might happen to AI, but on the other hand, (until so far) Facebook has already been there for a long time. People still appreciate, they are being surprised with content they would not look for normally, but still read. IR: Yes, exactly. IE: I know NU.nl made the platform, so it might only present which you like or which you follow all the time, and then they found out later this is not the way forward, then the filter bubble is there and people do not like the filter bubble, in the end. IR: Yes, yes. IE: But it depends a bit on what kind of public, because on Facebook you see the other way around. IR: So you should also present, content they might not have thought of, to try and burst this bubble? IE: Yeah, you should be surprised because that is also the task of media to show people all kind of angles in the news. IR: I agree, yep, yep. That’s interesting. IE: Yeah but I think it’s a hard question. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 101
IR: Yeah, it is difficult to tell, its an open question. All right, superb, well I think that is everything from me, so I will thank you for your time and stop recording.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 102
Appendix G Interview Transcript F
Interviewer (IR): Sebastian Short Interviewee (IE): Participant A Date: 30/04/20 Time: 10:00 Setting: Video call with living room as a background IE: Great, so thanks for your time. So could you tell me first what your occupation is and how you are involved with the Content Exchange? IE: Ok, so my name is {name}, I am involved by the Content Exchange because I know {TCE employee} very well. We worked together in the past, I was chief editor at the press agency in the Netherlands, the ANP. IR: Oh really? Ok. IE: But first in the video department, and he was a video publisher for a very big news agency. NU.nl, a website, and so we always had contact, also at the other publishers because I worked with much content and I worked with press agency, I got loads of experience in this type of business, and I also know lots of journalists, I know lots of publishers so I hope I can help {employee} and his team, with this nice product. IR: So you have already seen the demo then? IE: Yeah, yeah I have seen the demo yep. IR: So my questions will firstly be more related to my research question for my thesis, we will cover technology and the newsroom and stuff like that. Then we will move onto the Content Exchange, with previous interviewees I have been sharing my screen to show the demo, so I’ll skip that part. IE: Oh, you can do if you want to so then I know exactly what you’re talking about. IR: Yeah a little refresher, its just like the dashboard and things like that. IE: Yeah, yeah the dashboard. IR: If you want a refresher, I can also do that. And then I’ll ask you some questions and that will be all. So you said you worked at the ANP, was there anywhere else you worked at in the past? IE: Yeah at the {previous company}, and at the moment I am head of communication in {current company}. IR: Oh interesting. IE: We’re all into the coronavirus at the moment. IR: Oh yeah it must be quite challenging at the moment, I can imagine. IE: (I am working with) journalists as well, but now on the other side. Now I am working as a press officer. IR: Oh yeah from the other side of things. IE: Yeah. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 103
IR: Giving them news instead? IE: Yeah, it’s a very funny a very good experience to be honest. IR: Yeah? Interesting, it must be different to flip the other side of the coin. So how many years have you been working within journalism? IE: I have been working in the press agency for 10 years and 3 years in the {previous company}. IR: Ok great, and I guess {previous company} is general news, in terms of topics? IE: Yep, I was the director of content for all the local news and so therefore we had to work with lots of journalists who had to fill in their content for their newspapers, every day. IR: So, all kinds of topics? IE: Yeah, very small news, so local new yep. IR: Great, so considering your whole career, in that time do you feel that technological changes have impacted your work, since you started working within journalism? IE: Yes, yes, very much, because all it became very much quicker, so in the beginning of my career it would take an hour upload a video in come kind of a platform, really an hour. And the (telephone cables), you know? And when I stopped working at the press agency, we all started working with live video with packages we carried around, with a camera and everything together. So instead of working with a camera man and a sound person and a journalist, who did the interviews. You were (all at one) with a live exchange on your back, to send signals to the internet. So you had many, many changes. Also for newspapers, also they are much quicker, and different types of media of course. So instead of working with, making a video you are also working with making an audio clip at the moment, you are making a news message for the internet. So we are working very differently than compared to when I started. IR: Interesting, were the outlets online-only? Or were they, also broadcast? IE: Also broadcast, yep. At the press agency you make news and you distribute it to the media in the Netherlands, so they can all use your clips, you photo, your text message, everything. IR: And in terms of changes regarding online-only news nowadays, do you feel that has impacted the news quite a lot? IE: Yeah because it has become much quicker, so in the beginning all the media made loads of content. Now you see two types of media, that’s a my opinion. You see a type which is only producing loads of news messages and the type of media that try to be more (talkative), make bigger pictures, try to give more background of the story. So you see two different kinds of media, of what you see now. IR: interesting, interesting. And in the organisations you worked at, more specifically the {previous company}, were they able to balance journalistic content quality and financial performance effectively? IE: What do you mean? IR: As in, some outlets have pressure because of online-only, they have pressure, you know advertising, people don’t want to pay for news, and then some firms don’t have time anymore to try and write, to try and research a story. Some people are saying that the scales are tipping in having quick content, you know trying to find a new business model in these online times. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 104
IE: There are so big, {previous company} still have lots of resources to make (qualitative) journalism, but they also opened a very big department for branded content. So they have got an area, but I was also the manager of the branded content department at the press agency. We also started organising branded content over there, because we saw that publishers, we didn’t make enough money to keep the news alive. IR: Ah ok I see, changing business models. IE: Yep IR: New areas to, yep interesting. IE: Branded content, that is what they do. IR: So branded content as in? IE: At the agency we made content for companies, which they could share with journalists and we pushed their content over the channels at the press agency. With one button you hit all of the publishers in the Netherlands and at {previous company}, they make branded content, they distribute it to their own platforms, because they are so big because they have got so many media in the Netherlands, they have got a lot of eyeballs. IR: Ah yeah, I see, I see. IR: So, I am going to move onto more recent technological change, so are you aware of, it has many names, automated journalism, robot-journalism, things like that? IE: Yep. I read about it, I don’t work with it. NU.nl says they are doing automatic journalism and they automatically re-blog content from people who want to send them a message, that’s what I read, how it works, but I don’t work with it. IR: Yeah I see, so what is your overall impression of that, do you think its beneficial or not? Does it have consequences in the newsroom? IE: ((>)) Yeah, yeah, yeah. I am also very interested in the ‘deepfake’ technique. IR: Oh yeah, yeah that’s something else. IE: It’s very interesting, because everybody can send you images, and you are not sure if they are true. That kind of thing. You’re a news agency and you are receiving footage you have to make sure that it is true and that is also, ((<)) that’s an interesting topic. And you are talking about robot-journalism on the news floor. ((>)) I don’t know really, I think it can help, but I don’t think it can make the unique content, I would like to read. IR: Yeah, at the moment for example its primarily used, I know for example the Associated Press, in the US = IE: Yep. IR = yeah I think, all of their financial reports for example are generated by robots. So, I think for more richer stories, maybe that’s still developing. The sentiment I have heard from past interviews is, yeah its emerging and for the simpler stories where you just report basic data, that is there.] IE: Yeah, the who, what, why, how, when. You, know? There’s a fire? There’s a fire, where? How? What? And when? That’s ok, but if you want to add something ((>)), if you want to add some special content then it is not going to work. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 105
IR: Yeah, I see, I see, yep. IE: It can be very useful, with a quick scan. What I don’t like is every newsroom think its relevant to have a liveblog, especially now in the corona crisis. IR: I have seen it a lot, yeah. IE: [I don’t like it, the liveblog I think its, = IR: Not a fan? IE: = I think it is way too much and it doesn’t give any kind of structure in the news. It is just, I don’t how to say but I know how to express it. They do it like blurrggggh, you get all the news.] IR: Like, overload? IE: Yeah, its (guilty). I am trying to find something and I am scrolling until the end, like come on. When they are doing an interview in court, a robot could be useful for that kind of stuff as well. IR: Oh yeah, good point. IE: Because maybe there are people, not me, who like to read it, I would rather read the summary at the end, and the interesting stuff. That because I don’t have the time, to follow it live. IR: So you would rather read it when the event is over? IE: Yeah, yeah. For football it is also interesting, for robots I think. IR: Yeah, that’s another area, Definitely. IE: So that kind of stuff its good to have it, yep. IR: So if you were working as a journalist, would you source an article that is automatically written? IE: Sorry, your source? IR: Would you source an article that is automatically written? IE: Ah yeah, of course. I would use it for my unique piece, yep definitely. IR: And why would that be? IE: I would use the automatically generated content because then I would save me time, because the robot made the script, and until it’s over I get the script and then that makes my unique story and that’s how I would work with it. IR: Indeed, indeed. And again depending on the topic? IE: ((>)) No, if its about factual information, it could be any topic. IR: That’s a keyword, ok great. IE: Well say factual information, because maybe I am wrong. I said factual information, but when for example a press agency says its doing factual information as well. That is what they do. Can we say ok we don’t use press agencies, we just put robots in the news agency. I think that is how it is going to work, because you still have to do research work. IR: Yes, yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 106
IE: So, there is slightly a nuance in factual. IR: Yeah I see what you mean, another example maybe you have already heard of it. But in the US, the LA Times, there was an earthquake in Los Angeles, the LA times had this quakebot. Within 3 minutes they had already reported this story, as this bot had detected the seismic data of the earthquake and generated the text and published the article, within 3 minutes. So the data could come from literally anywhere. IE: yep, yep. IR: So, that’s another example, so yeah all areas. IE: Yep, you have for example if you are working in politics, at this moment everybody is calling all these politicians for their quotes. What if they would send their information to a bot, and a bot creates a message, would it work? Maybe not, yeah could be interesting yeah. IR: Yeah, that is something else to think about, yeah, yep. IE: But really I have been called about, something calling 20 times a day with the same question for corona, for example. How many patients, how many patients in the hospital, how many patients are in intensive care. IR: Yeah. IE: If a robot scans all of this information, and I would just simply send it to one source, and its gone. It would be very, very efficient. IR: That’s a good point yeah, it could help with sourcing, and in the past, speaking of that, what was kind of your sourcing routine? How did go about discovering and finding news to write about? IE: What you do is, it depends on what kind of news you are looking for. If you work for the local media, then your source, you look in the local newspapers. They all look to each other actually, they all do. That is what I see and now as well. If I make a story now with a journalist, I can see 3 or 4 calls from other journalists. So what they do is they all look at each other. It would be very ((<)), the best answer to say because that is how they teach it, is “yeah I am going into the community and I speak with people and I am going to find my news”, but to be honest that is mostly not how they work, because they don’t have the time to do some research or invest, or just hang out in the city and see what is going on. So mostly, newspapers are generated by what they receive. IR: I see, yep, yep. IE: So if they receive tips, messages, what they see in other newspapers. That is how it works, yep. IR: And do people sometimes search online for stuff? IE: Sorry? IR: Do people also search online for news? IE: Yeah, definitely, all the time. Online and on social media, Facebook, Twitter, yeah. That is how I make my news at this moment as well ((laughs)). IR: Yes, indeed. Convenient, so at the ANP was it primarily big firms or small firms that were coming to you? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 107
IE: Big firms, yep and the news mostly the people who came to us were the usual politics, police, you know all that kind of stuff. {previous company}, it was the local soccer club, more local parties who came to us. IR: So for the more local news, people kind of used their own networks to discover it? IE: Yeah, yeah their own networks. IR: I have spoken to some others from {other participants} and that is generally what I hear. IE: Yep, I work with {company}, at this moment. Yeah what I see that it is not doing a good job, that is my opinion. IR: hmm. IE: No because it is hard for me to reach them, because they are so focused on the Eindhoven area, they are in. Do you know {province} a little bit? IR: Yeah, yep. IE: Yeah? Ok, you would suggest or you would think they would cover the whole of {region}. But actually, if you are going to analyse their news, its mostly the area of {city}. But it is {city} because they work and live there, the people who work there. IR: Oh, I see. Interesting. IE: Yep, very bad. I am not very keen on {company}. I have worked their as well, yeah it was 20 years ago, and I still know the people there, they are very kind and friendly. But, very {city} oriented. IR: So why do you think that is, so do you feel like they can’t reach the rest of the area? Or is it convenient to stay around? IE: Yeah, because they live there and that’s how they consume their media and therefore that is how they see the province, but it is much bigger than that. IR: Almost like a bubble. IE: Yep, we’re living in a bubble really, yep. In Amsterdam they are also living in a bubble. IR: Yeah true, especially also yep. IE: Yep. IR: That is an interesting point. I did speak to {company} the other day. So it ((<)) adds another angle to it. IR: Normally at this point I would show you the demo of The Content Exchange. Do you want to see it again just to? IE: Yeah, then it may be easier to talk. IR: Yeah ok, I will share my screen. ((IR shares screen with IE)) IR: Can you see my screen? IE: Yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 108
IR: Perfect, so I will start with this one. I will just zoom in a bit, there we go. So here is the dashboard. IE: It is good that you are showing me these pictures, because these are presentations and in the demo it doesn’t look like this. IR: Yeah, the demo is a bit simplistic. IE: So it is good you are showing this. IR: Yeah, so far I have shown both to interviewees. It is good to show them both, so they get the full picture. IE: Yep. IR: Yeah, so oh and they don’t have the personalised recommendations on the side, or the feeds at the top. So I will just cycle though these ((IR switches through images simultaneously to talking)), and you have the story like this. And then you have the shared feeds section, and the custom feeds, login screen, profile section, oh and then the filters within the topics. So, if you are interested in a particular sport you can select your favourite sport you want to write about. This is a similar view, and here is a better view of the filters. Then you have this recommended feed dashboard, maybe you know more than I do, do you know anything about this, recommended feed? IE: Yeah, yeah, yep. Based on your profile they are giving you recommended feeds. IR: Ok, but just your profile information? IE: We have Topics in the Netherlands, Topics, do you know it? IR: What sorry? IE: Topics, its called Topics. I am going to show it to you, one moment. ((IE grabs phone, shows it on screen and opens the ‘Topics’ application. IE: When you have an account for one these newspapers from {previous company}, you can swipe ((IE scrolls down on phone, showing a news application which is organised by news topic)) IR: Oh like that, yep I see. IE: This is not the topic, wait. ((Both wait for application to load)) Then I have got all the news, see how it is called? IR: Oh I see it yep, politics, Netherlands ((IR is reading from topic list on application)). IE: It is interesting you here, because here you can follow all your topics from the (top) group. So what they do, they say ok you have for example the {newspaper} or you can read the {newspaper}. You can also read, they understand you can everything they have but if you want to have one newspaper from us. They centralise all this information in topics, and it is interesting to see, because they also have feeds, and if you are working with it, then you are going to see how many, many, many feeds you have after working with it for well, a few months. Its crazy, I’ve got so many feeds. IR: Yeah, yeah I see what you mean, ah ok that’s good to know. ((IE shows list of feeds on screen)) AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 109
IE: These are my feeds for example, its not (interesting) but it gives you an overview of all my feeds. Its still not ready, that’s because when I see an interesting article I am going to click on it and follow the feed, and in the end I have got so many feeds that I am drowning every day when I log on to this. I am like “oh my god, I have to do something about my feeds”. IR: Information overload. IE: Yeah, yeah. Then that is just what I (inaudible). IR: Oh ok, well that’s good to know. So is that commonly used within the field then, do all journalists have these topics? IE: This is for consumers. IR: Ooh I see, yeah that’s. IE: They want me as a consumer to buy one of their newspapers, and they give me for free all the other newspapers. IR: Ok, interesting. IE: If you want to read all of these newspapers everyday they say ok you can select a few topics that we categories the news for you in those kind of topics. But what I see in the beginning is that a got a few topics, I got 4 topics for example, and then it was interesting because I saw that topic explained in {differing newspapers}, a saw a bigger interview in a local newspaper and then it was nice because you’ve got different views for one news story. But what you see when you are using it for a few months, you add more, and more, and more topics. But that is going to be a very big, content mess. IR: ((Chuckles)) Yep, yep. IE: Its different because I know the Content Exchange is making content for journalists, but still I think it’s a very big challenge to see how you are going to categorise all these topics and to make sure you are going to be relevant in these kind of topics, so that (users) are not drowning in everything they are going to send to them. IR: Yeah that is my next question, do you think it will work, if you spin this around for editorial teams or journalists who use this in the Content Exchange? IE: Yeah I think it will work, but we have to test it, we have to be very careful with how many topics we add and how many sub-topic we add. Because its going to be a very, very big mess like my phone. IR: Yes, indeed and I guess you need like a broad range of topics, do you think it will be beneficial to offer perhaps topics they haven’t considered before, that they might discover accidentally? IE: Definitely, also maybe it’s a good idea to send them push notifications about new topics, or other interesting topics. Maybe push notifications via email, so you can trigger people to go to the Content Exchange and see what’s going on out there. IR: Yeah, like stimulate their search. IE: Yeah, definitely. IR: Interesting. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 110
IE: And so it is interesting, for example to make some kind of basic profile so its like: “hi guys it’s nice to see you here on the platform, you are a sports journalist, this might be interesting for you” and then give them all kinds of interesting topics. Yeah, well that’s what you do, but sport is an easy one. ((>)) Yeah what’s an, I don’t know, foreign news for example could be very different, but yep. Something you really have to think about. IR: So like more, broader topics or generic topics. IE: Yeah, yeah. IR: Like, finance or something. People might need a bit of stimulation maybe. IE: Yep. IR: Ok, that is interesting, and so what is your overall impression of the Content Exchange, so far? IE: I am very enthusiastic about the Content Exchange ((<)), yeah I am very enthusiastic about the platform, it worked very quick and importing a feed. Every time I am surprised by the speed of the feed, so that’s pretty good. We’re still working on some stuff, like does it receive photo, video and text in one, column or well how do you expose it. We’re still discussing those type of things, but yeah in general I am very enthusiastic about it. IR: Great, and in terms of the user interface and design, does it make sense to you? IE: Yep, yep. IR: Does it feel familiar, is it similar to other platforms you’ve used? IE: Yep, its kind of similar to other platforms. I have seen many logos, in the demo you do not see that many logos, as on your slides you showed me. Which is messy for me, which is what I said to {TCE employee}, I said I see loads of pictures and icons. They are not very relevant because I for example I would like to know if I would pay for something, very nice for me to see if what we still have in our, ((<)), what’s the range? How many can we use, and is it ok for me to download this or does someone else have to do it? For me as a journalist that would be nice to know, because I would be working and if I have to ask somebody else to download a video so I can use it, that wouldn’t be very easy to work with.
IE: It has to be quick and easy. I’m the editor who has to download, I have to edit the video and it would be helpful if I could download it as well. But still I know as a newsroom you would like to have grip on how many you have to pay for the content of course. I would like to have a grip on the budget, so that would be something you have to think about, how to do it. IR: So something like within user profiles that give you a financial overview maybe? IE: Yeah, that would be great yeah. Is it important for me as a journalist? No, because I don’t know the total, maybe they are already working on it. Well, it would be my team who is using this platform, and everybody is going to download one type of video, how much money will we have to pay? And do you organise it, how do you get a grip? IR: yep, yep, that’s true, that’s a good point. But in terms of the design itself, the first impression is kind of positive or negative? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 111
IE: Very positive, very positive. I also like the option that you can choose to see text only, only video or photo only. So you can customise it for all types of content, on all types of topics, everything. That’s very good. IR: Interesting, great. So far so good I would say, yep. Ok so I think that’s my main line of questioning, I will just close up and ask you. In terms of everything we have talked about, in regards to this and robot-journalism and things like that, where do you see it going in the future? Do you see it increasing or? IE: Robot journalism? IR: Or just in general technology in journalism, artificial intelligence. IE: Definitely, yeah definitely. And now you are going to ask me why, because the who society is going to change, it is changing already. Its not the same, and its going to be more and more digital information, but also statistics are going to be more and more relevant. IR: Yeah, definitely. IE: There it is very important that we have got good statistics as well. Yeah, so for me it is a no brainer that it is going to be something. IE: But how it will, what it looks like, that’s the question. IR: Yeah, that’s unclear you can’t see that at the moment. But where you worked previously were the firms quite data driven, did they use metrics? IE: The {previous company}, no. In the beginning they were all data driven, but data is going to be a very big part of their business, yep. IR: But in terms of robots, do you think they will take over eventually or do they just kind of have a supportive role? IE: Yeah, they will help you, they won’t take over. IR: Ok, interesting. All right, well that is all from, thanks a lot again and I will stop the recording.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 112
Appendix H Interview Transcript G
Interviewer (IR): Sebastian Short Interviewee (IE): Participant A Date: 23/04/20 Time: 10:00 Setting: Video call with living room as a background
IE: Ok, so firstly could you tell me your occupation? IR: Yeah, so I have multiple occupations actually. I lecture about 2 to 3 days a week at the applied university, for journalism and then next to I work as a journalist and as an editor, as a freelance journalist and editor for multiple platforms. Some of which are actually very journalistic where others are more or less meant as a, I don’t know what the English term is. ((<)) They are more focused on companies to give knowledge that they need to do their jobs so that they need, so the job I am doing is similar to that of a journalist but it is less focused, less critical or focused on the democratic process. IR: Oh so its like business info? IE: For example, one of the platforms I work for is focused on people who work in publishing, and then when for example who will come out with a new feature to create e-books. I will interview and write about that. But I will only focus on the functionalities that are practical for people who work in publishing, not necessarily on any privacy issues whatsoever. So it was kind of a mixed job. IR: Ah ok, interesting. So what is the company called, if I may ask? IE: Its called {company name} IR: {company name}, ok. IE: I can send the link you a link to the platform if you want to? IR: Yeah that would be interesting yep. So what are the other companies you worked for? IE: Well there is a whole list. One of my most important companies I work for at this point is {company name} in the Netherlands. It also has journalistic, a sort of part. Where we interview not only journalists but also companies who for instance criticise journalists, or write about journalists, or deal with journalists. So this is actually journalistic work, I also work for {company name}, which is focused on equality and global sharing, so this is pure journalism. IR: Ah yeah I have heard of that yep. And so how many years have you been doing this for then? IE: ((<)) I think about 6 or 7 as a freelance journalist and then longer if I count my internships, then it would be like 9 or 10. IR: 9 or 10, ok great. And so what of I guess at the moment, what kind of topics are you involved with in general, in terms of like news topics? IE: Mainly journalism and public discourse and that type of thing, privacy, so the hard topics that journalism generally deals with, then publishing for the publishing platform I am writing on. And just generally I get a lot of requests for if I would like to write on a certain topic and then sometimes these are topics I haven’t really written on before, but because journalism is my AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 113 profession its more about the form then it is about the topic sometimes, so a lot of articles I am writing about now is focused on music festivals and the manner in which covid-19 has impacted those and the manner in which they will use new technologies to move on in the future. IR: Ah ok, so kind of broad range of general news? IE: Yeah. IR: Ok, interesting. So in the 9 to 10 years you have been within journalism, how do you feel that technological change has impacted your work since you started? IE: Well since I haven’t been in journalism that long, I know 9 years is obviously quite long but when I started doing journalism a lot of technological changes had already impacted journalism and starting to impact journalism. IR: True, true. IE: So, for me it was very natural to do things online, find candidates to interview, find topics, find sources. Well for a lot of colleagues, this was not natural at all. So I sort of noticed how much had changed by looking at the people, and the strangeness they experienced. So that’s the main thing that has changed, other things that have, well these are details. IE: Things that have been changing is that when I started, a lot of services were free, so I think for instance all tools to scrape information and other information resources, tools to visualise information, tools for publishing. And right now, I believe almost everything is paid, or it is provided by companies like Google and you are probably familiar with, there are a lot of sort of challenges with working with these companies, such as how do they use your information, what other models they have to make money and all sorts of other stuff. So I think that is one of the major changes that I have to deal with, almost on a daily basis at this point. IR: So, changing business models, or how they make money? IE: Yeah, more change in the manner in which you have to collaborate with third parties. IR: I see, yep, yep. IE: So (when I started), I just made an article for a platform, and they published it, and that was it. But now I have to use all sorts of third parties to find information, to deal with my audience, so I think that is one of the major changes, and then I am thinking about other changes…I don’t, know. IR: So these companies you have been working for, or are working for right now. Were they always online, or did they transition from being offline to online? IE: Most of them were already online when I started working for them. But the internet was very much and this is actually this is a change I should have mentioned for your last question, I didn’t think about it. But when I started working for them, the internet was very much a secondary platform, so if you were a good journalist your articles would be published in print, and if your articles were just…ok, sort of, they were for the website. So this was a clear divide, and it was also a divide that would be recognised by interview candidates, so if I called someday and said I am writing an article for the website, ((<)) they were more often say “oh I don’t have time”, but if you would say I am writing an article for the print version of this magazine, they would be more inclined to work with you. IE: Ok, that’s interesting. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 114
IR: And this has changed, so I get almost not entirely but I get almost paid as much money for online articles as I get for print articles. So this is a significant change, because in payment there used to be quite large differences. A lot of my interview candidates take the website just as serious and sometimes even more serious than the magazines I write for. But this is a major shift. IR: Yeah, I see, yeah. Do you feel like outlets are able to balance financial performance and journalistic quality effectively? IE: That’s a nice question, I am not sure if I can answer that because I am obviously on the other side of the system, so I am not quite sure if they are. But I also don’t have any reasons to think they would not be. IR: Ok, do you feel that you have enough time and resources to write the story you want to write? IE: No. IR: Really? IE: I believe I have that has been a problem, maybe even since the 80s or 90s in journalism. IR: Yeah. IE: It has been a problem as long as I work as a journalist, which is one the reasons why I am a lecturer, so I (read) a lot of teaching in journalism, but it is also incredibly hard but also impossible to be a full time journalist, and live a comfortable life. IR: Yeah, yeah. I have heard that a few times yep. IE: Yeah, so for some of my articles I get, I get about 200 euros per article for online articles at least. For an average piece, say between 600 and 1000 words, and I can write an article in say 1 to 2 days, but I wouldn’t be satisfied with the piece. But since my name is under it, then a week writing this article. In those cases 200 euros is ((<)), yeah you cannot depend on that. IR: Yeah that doesn’t quite cut it, yeah, yeah. I see what you mean. So moving on now to more recent technological changes, are you aware of well, it has many names like robot-journalism and automated journalism. IE: Yeah, that is actually one of my specialties. IR: Ah, so yes. ((Laughs)) Ok. IE: [My specialities are online and innovative journalism, so robot-journalism is very much erm =. IR: Oh perfect, well you are the right person to talk to then. IE: (one of my focuses too).] IR: You are the right person to talk to then. So what is your overall opinion on it so far? IE: {!Oh god!} Well if we just think about just as (narrowing it) as an AI way to employ robot journalism. I think it could be great. If we would have some sort of AI that would scrape social media, for say sentiments with regards to certain topics, great. But I do think again, we do have to be aware of the third parties we would cooperate with because as far as I am concerned, no AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 115 truly journalistic organisation so far has managed to create a bot that ((<)) is doing a sufficient job. IR: Ok. IE: So maybe you have some new information that I am not aware of, ((<)) so some, I think in general I think it could be a great idea but I have some concerns with regard to the parties we would have to deal with in order to get the data we need. Or the functionalities we need, so yeah. IR: What kind of functionalities are you referring to? IE: Sorry? IR: What kind of functionalities are you referring to? IE: Well I am not quite sure, but maybe cloud functionalities or sort of the algorithms that you wouldn’t write ourselves. I have seen a lot of times, where we are trying to develop something and either Google or Amazon, or one of these companies would come and say “Oh look, we have already have like 80% of the thing that you are doing, you will take it and build your system on it, and then you will have exactly what you want”.(Then you have that one), but then your data is not secured and sometimes you cannot explain to your users what actually forms the basis of these systems and what’s actually going on. So I think those are my main concerns. IR: Ok, that’s interesting. Because I know right now they are used for more factual stories like, I know the Associated Press, all their financial reports financial reports are doing by robots = IE: Yeah. IR: = things like that. Football games, sports is quite popular with it. IE: Yeah, they have this tool that {colleague} and {organisation}, this football tool. You have seen it probably? Because of {colleague}. IR: Yep, yep yep. IE: Yeah so I think for these types of small applications, its fine, its great. IR: Ok, ok. Yeah that’s pretty much the sentiment I am hearing a lot at the moment, it is interesting to know. So in terms of like, well maybe not yours but in general, what is your sourcing routine, if you want to write a new article and discover a new story? How do you go about that process? IE: So I think a lot of it depends on the matter of which I begin to write a story because I think about 70% of the stories I write, I write because news platforms ask me to. IR: Yep. IE: So in that sense, they already give me a lot of resources before I start writing. So they will simply: “Oh this and this will happen in two weeks” or “we have heard this and this and that, through this and that person, can you write something about it?”. In that case, I already have the information I need to get started. IR: Interesting. IE: In the other cases, I often just read, see something on Reddit, on Facebook, on some platforms, sometimes even in real life. Can you imagine that? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 116
IR: Yeah ((Both IR and IE chuckle)) IE: ((sarcastically)) So weird when that happens. But ((<)) so I already have some sort of a starting point, then I will go to a platform that I think might be interested in the story and then I tell them, I heard this or I saw that. Then they will pay me to write the story, or they just tell me to, I don’t know, forget about it. And after I have done that, it still depends on where the initial information came from, or I would most likely go back to the first source, to gather more information, and then I will use the internet to find more (and get contact for interviews). IR: I see, yeah. So how do you use Reddit? I am kind of curious now. IE: Sorry? IR: How do you use Reddit? I am kind of curious. IE: Well mainly because it gives me some ideas of how other people are looking at subjects. So I will, I am obviously still quite young, but sometimes I heard something and I think “oh my god that’s weird” or “that’s strange” and go to Reddit to see how other people discuss the topic because as you are probably familiar with there’s a lot of different opinions on Reddit. IR: Very, yeah, yeah I am aware of it, yeah. IE: So I use it to see what kind of perspectives people use to look at a certain topic and it gives me a lot of familiarity. A lot of ideas. IR: Interesting. IE: Instead of my own perspective. IR: Yeah because I notice sometimes on comments on news articles people mention more sources on the same story. IE: Sorry, what? IR: I notice sometimes on comments for a news story some people comment and provide another source that provides another angle on that same story for example. IE: Yeah, and I think that’s very nice and also because, well I am obviously looking at things from a Dutch perspective and a female perspective, and a high educated perspective. And then I think oh that’s such a dumb idea when I heard something in public, and a lot of politicians then agree or disagree and then I look on Reddit and then I see that some people have opinions that I would not have imagined. IR: Oh interesting. IE: And that is actually nice. IR: That’s actually very smart, yeah. Because there’s so many comments, how many comments are posted on Reddit a day? Millions, hundreds of millions. IE: They are also quite amusing, what people sometimes say about (it) IR: Exactly, yeah that’s very true. You can get stuck in a hole there. IE: Yeah, the black hole of Reddit. IR: Exactly, yeah. IE: Its one of the best places to find a lot of opinions fast. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 117
IR: Yep, yep. IE: Facebook groups obviously work as well but then some are hidden, yeah so and the Facebook search doesn’t work quite as easily as the Reddit search in my opinion. So Reddit is often a good starting point if you want to know what people think about something. IR: Interesting, ok and would you or do you think people in general would consider sourcing an article that was automatically written? Or sourcing text that was written by a robot? IE: ((<)) Err, I don’t know. I think that is something that is something a coordinator at an editorial office would probably have an answer to. I don’t knows it’s a bit of an odd question to answer. I think it is all about the quality of the article. IR: Yep. IE: So if it is good, I don’t see a reason why not. But I do wonder whether these type of articles show any originality. That is often expected from journalists. IR: Mmmhmm. IE: It all depends on the topic I think. IR: Oh yep the topic is important I think, yep. IE: Yep. IR: Ok, interesting. All right I am going to move on to part 2. I am going to share my screen. ((IR shares screen on video call with IE)) IR: Can you now see my screen? IE: Yeah, I can. IR: Great. So first I’ll do the live demo. ((IR has TCE demo displayed on screen)) So this is it, this is the Content Exchange. Of course its in an early development phase at the moment, so the pictures later will have a bit of a detailed design interface. This is basically it. Not every news topic comes up at the moment, because they still need more content to be fed into it. But this is the basic interface, like if I type in for example “Eindhoven” ((IR adds a new feed to the demo)), press enter, ((<)) and then it comes up here as another feed, then it comes as a list. You can scroll down, and then you can view the story, and you can either purchase the whole text or some of the text, or the images. It’s also multimedia, meaning that it also does images, but eventually it will incorporate video and audio in addition to that. Alongside here you have all of your feeds, I guess now I will move onto the pictures. ((IR switches from demo to mock-up images)) IR: So this is a more detailed image of the dashboard, and you see on the left here you have personalised recommendations, topics. So if you into a certain, well whatever you want, economy, politics, it will recommend you stories related to that and possibly, other options you may not have considered before. Based on your usage data. I will just cycle through these. ((IR switches through images)) Then you also have shared feeds, so for example within an editorial team they can create their own custom feeds and then share them within their teams, so everyone is on the same page. And then there is the login screen. You have your own profile like this. And then within a topic you can filter it even further and search within it. For example if you are into sports you can then select a particular sport you want to write about, for example. This is another image of the dashboard, and here is a better view of the filters, and custom feeds. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 118
IR: At the moment it is just those columns that are available at the moment, and if I can get to a bit later on. Then you have this, recommended feeds like this, that are kind of tailored. Its like you know in a news app you can follow particular topics? IE: Yeah. IR: It’s kind of like that but for a journalist I guess is the best way to describe it. So it kind of essentially aims to be a marketplace for content, I think, what I hear a lot from the team is that it can connect journalists to editorial teams and publishers, so they can discover content more easily. For example a freelancer could offer an article on this platform, and then it can be reached by thousands of outlets who may want to reuse their content for example. And of course as you are probably thinking, there are issues that become, yeah. Business models and payment, things like that. But for now, this is kind of the er ((<)), this is where it is at, at the moment. So that is the Content Exchange, I will stop sharing my screen. ((IR stops sharing screen with IE)) IE: Could you say something about the manner in which AI is incorporated in it? IR: Well, at the moment, its not there yet. But I think they eventually want to start offering automatically generated articles and offering them on the platform, as a more efficient way to deliver the news. But that is something to do in the future, yep. IE: Ok, and the recommendations are these generated based on (the users themselves)? IR: Yeah, it’s a good question. I don’t know too much at the moment, but as far as I know it is based on usage data and usage data of similar users to you. IE: Ok. IR: So it is supposed to suggest you topics, like an ((<)). So it suggests to you a list of topics perhaps, a broader range of topics that are not that similar so you discover something else that you may not have thought of before. That’s kind of the goal. Yeah that’s still in development at the moment. IE: Its very important to think about the way in which these recommendations are generated. IR: Definitely, yeah. They have a guy working on it, doing all the coding, so what they have in store for that, we will see. So at the moment they are just kind of nailing the basic feeds and the content. So yeah, I am curious to see how that will works, because I get that question quite a lot. IE: Yeah, because your personal life is obviously very different than your, sort of business life. In the sense that the feeds I am interested in personally like cats, it’s not something I would write about as a journalist. IR: Yep, yep. That’s a very good point. So yeah, how that works, the algorithm for that is something they will, presumably they might have something in mind already. We’ll see, we’ll see. So what is your overall impression of this system, as it is at the moment? IE: Yeah it looks nice. IR: Yeah? IE: Yeah it reminds me of Tweet deck and I know a lot of journalists are familiar with Tweetdeck. So that’s great I think. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 119
IR: So you feel like the columns and things like that, if its familiar then its kind of, easy to use if its, yeah a familiar face, so to speak. IE: Yeah, I think if it looks familiar, that’s nice definitely for journalists because I have colleagues who still find it hard to work with new technology. IR: Yep, yep. IE: I think that’s nice, I think that’s great. IR: And in general, the user interface does it make sense to you? Is there kind of anything that looked a bit off, or? IE: [Well I haven’t obviously been able to interact with it = IR: Yeah, that’s the other thing. IE: = But from this perspective, for me at this point, it looks fine. IR: Based on the quick demo, yeah, yeah. So if this were eventually to incorporate automatically written articles, would that benefit the system? Would people source those articles? IE: Yeah, its still a very hard question to answer I think. Definitely (still a problem). I think for instance people who write about sports, I know the example you just mentioned through {name}, who write about sports. I think this is great. But at other editorial offices, this might not work, I would also be concerned about journalistic platforms who want to have an exclusive article. IR: Yeah, I see, yep. IE: So you have to be able to take the sort of take stuff out of the platform, and take that off for everybody. IR: I see, I see, yep. So it would kind of depend on the person and what that person is looking for. IE: Yeah exactly, and as far as I am concerned, a lot of platforms deal with these kind of things in a lot of different ways, and as I mentioned before, if I believe that the quality is good, if it is almost you read an article that was written by a human journalist, then of course its ok. But a lot of platforms are looking for perspectives that are just a little bit different, than perhaps the ones that are already there. So as you mentioned, we have to feed data into the platform to generate the right kind of topics. So I think that a lot of platforms are looking for stuff that can not found by data. Because it has to be (different), than what…do you understand? IR: So, it has to be like fresh and undiscovered? IE: Yeah exactly. IR: And if its online, its arguably already discovered. IE: Yeah, yeah. IR: Ah ok, that’s an interesting point. IE: Yeah so for covid-19, this is crazy because you see that, I even saw some content about how golfers are dealing with covid-19. People apparently don’t know what to write about anymore, that hasn’t been written. IR: Yeah, I see. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 120
IE: So, yeah. IR: Yep, yep. So maybe a topic popularity is something erm…yeah the more popular the topic gets, the less is to write about it anymore. IE: Yeah exactly, and for topics like football. Yeah you can’t really come up with 100 creative perspectives on one football match. IR: Yeah, yeah. IE: There was somebody who won, and that’s it, or yeah you could write something about the trainer, and about the supporters, and about some partners they have. But there are topics I believe that have just some possible perspectives, and then there are topics who have most, well almost endless. IR: Yeah, that’s an interesting point. So ok that’s great. I will just close up by asking a final question, by saying in terms of everything we have talked about, like the digital newsroom, artificial intelligence and robots, where do you see that going in the future? In the context of journalism? IE: Yeah that’s a difficult one, especially because I think the influence of Google and Facebook and stuff is making things quite difficult for a lot of journalists. IR: Oh yeah, yep. IE: So, I am afraid that journalism will just be something for sort of a small group of people, and that one journalism will stay for a general group of people like, televised journals and stuff like that. But that will probably be government sponsored as it is already at this point. IR: Yep, yep. IE: And the real quality journalism will probably stay for just, I don’t know, business, professors, like just a small group of people. I think the rest of the information will be provided by Google, by Wikipedia, by those types of platforms. So, yeah I think it is going to be difficult, but on the other hand, I don’t know if it was easy before. IR: How do you mean? IE: Well, if I read about the history of journalism, I believe that the true quality journalism has always been for sort of a niche group of people. IR: Oh yeah, yeah. IE: So I think we are wrongly believing that the journalism was for everybody, is now all of sudden becoming for just a small… do you understand? IR: Yeah I see what you mean, yep. IE: Yeah, so I think we are afraid, and I think we are wrongly… we are right to be afraid of the influence of some big tech companies, but some of the challenges we are identifying at this point are not necessarily new I think. IR: Yeah, that makes a lot of sense. Its just the technology has changed. IE: Yeah. IR: Its changed processes but not necessarily the way journalism is done? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 121
IE: Yeah I think that’s right. I have find a better way to tell the Google part though. IR: Yeah, I have read about that yeah. IE: Yeah, so Google is the interface for all the news at this point, or at least the importance interface as far as I know. The platforms that I write for get 60 to 70 percent of their readers through Google. IR: Yep, yep. IE: Well I think that is something that is going to have major consequences for journalism in the future. IR: I see. What kind of consequences like tech taking away from the website, or? IE: Yeah maybe, or a lot of news is now already shown in the search feeds, so as a reader you don’t necessarily have to go to that platform anymore. Then I see for instance the Google journalism initiative, data visualisation tools and the publishing tools at Google. So I think at some point maybe Google might incorporate all the news organisations. Instead of just letting them just having their own platforms and sending traffic towards these platforms, and I do understand because it is not Google’s job to provide the news companies with traffic. But yeah I do think that maybe that’s the direction we are going to. IR: Ok that’s interesting, and do you feel like maybe a platform like the Content Exchange, could kind of squeeze in to yours or anyone’s’ current sourcing routines? Would it improve it, or hinder it, or? IE: Hmm. It might be a nice way, if you write a story to find a platform that would want to have it. I think that might be nice, but on the other hand I already have a lot of platforms I am working for, I am not necessarily looking for other platforms that I can publish for. IR: Ok. IE: So, maybe it would be more interesting for journalists who are trying to find a new platform to write for, or maybe young journalists who are still looking for platforms to write for. IR: Or perhaps to borrow some content from it, to write into your own articles? For example, because you can also purchase a little bit of text. IE: But it would be dependent on how much they would to have to pay. Well if I make 200 euros for an article, even this 200 euros is not enough for that piece. Or it is not for my (time). I would probably not want to pay more than 1 or 2 euros. IR: Yeah, I see what you mean. IE: Yeah and it would have to be exclusive because none of the people I write for would want me to use content that somebody else might be using too. IR: Yep, yep. I see what you mean. IE: So it would be very difficult. IR: Ok, good points, good points. All right I think is everything from me, so I will stop the recording.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 122
Appendix I Interview Transcript H
Interviewer (IR): Sebastian Short Interviewee (IE): Participant H Date: 30/04/20 Time: 15:00 Setting: Video call with living room as a background
IE: Ok, so firstly could you tell me what your occupation is? IR: Ok, I am CTO with {current company}. I am partially responsible for innovation as well. As a CTO at {current company}, we have different businesses. So we have the traditional press agency business, where we as a wholesaler in news, provide news in text, photos, audio and video, to basically almost all media in the Netherlands. We also have a media monitoring company, we deliver information to businesses, corporates and government, it works to track specific persons, or topics in the news, and we do that for them. We have a press release business, and we have a big photo business. We not only have our own photographers, but we also have, or represent photo agencies from, well a variety of agencies worldwide. IR: Interesting, interesting. It is quite varied, and what kind of background do you have yourself? IE: Ok I studied informatics and telematics, and I graduated mid-90s. After I worked in a small company, and I became a shareholder. What we did there was we had some content exchange, where we (had) content from publishers, we sold that content to publishers, primarily those publishers were consumer oriented. We sold that content to the B2B market. IR: Ok, interesting. IE: And that is still what we are doing, and we call that media monitoring. We did that, starting in 96 I guess, and we (published) with financial newspapers, like Financial Times, Wall Street Journal, ((<)) South China Morning Post, (for an reminder), we have in the Netherlands the financieel dagblad. All those financial newspapers, to banks like ABN Amro, or Public Standard Chartered and many other organisations. IR: Interesting, so you are quite involved in the technological side of journalism then? IE: Yep, yep. IR: Ok, that’s great. Interesting, all right so in general then, looking at technological change, how do you feel, maybe within a journalistic context, or a content distribution context, how do you technology has changed over time? IE: Yes of course it changed a lot. It gives more power to the consumer, and (perhaps) like publishers, if they are not uniquely positioned, like the New York Times or the Financial Times, you know big markets. Then they tend to have problems, surviving and the struggle because content can go directly from journalists to consumer for example, via social media or via other means than (professional) publishers for example. So, there was a lot of disruption the last 20 years. IR: Yep, yep, yep. IE: And 5 years, because of the capabilities and possibilities appearing, and I think both the market developed, so if you look at the B2B market, professionally written content by AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 123 journalists is a developed market and it wasn’t 20 or 25 years ago. At the same time, the consumer market eroded and declined because less and less consumers are paying for content. So content is, basically paid for by advertisement. IR: Yes, yep. IE: Now journalists, during the corona crisis it is quite clear that is not sustainable in such a crisis. IR: Yeah, definitely, that is a very interesting point. IE: So, yeah a lot of things happened, a lot of disruptive things happened and it’s still moving and a lot of things will happen in the future as well. IR: Yeah, definitely, definitely, and do you feel like then, maybe news outlets you have seen or just in general. Is there a balance between journalistic quality and financial performance? Considering people are less willing to pay for… IE: You know what you also see, before the corona crisis but what you know see confirmed is that a lot of consumers are willing to take a subscription because they want trustworthy content and (they are willing) pay for it. So they are seeing an increase in subscribers, however, not enough generally to compensate the advertisement market, that almost collapsed. IR: I see, so subscription models are kind of leading the way at the moment? Or increasing? IE: Yeah. IR: Interesting, that’s a good point, and looking at more recent technological changes. I am assuming you have heard of automated journalism, robot-journalism, things like that? IE: Yep. IR: What is your overall impression of that? What do you think of articles that are automatically written? IE: Yeah well, automatically written I don’t see that too much happening in the Netherlands, to be honest. IR: Ok, interesting. IE: And it depends on how you define automatically written. Yes? IR: Yes, yes, yep. IE: So, basically you have Natural Language Generation, and you have the more template-based approach. What you see in the Dutch market, generally the template-based approach, and of course templates can be assisted or generated using robots by getting enough examples from previous archived stories. IR: Yes, yep. IE: But to really generate text via computers in the Dutch language, you do not see that too often. I know it’s being used in the English language. IR: Yes, quite a lot in the US, yep. I have seen it already, for like financial reporting, sports, I seen has come up quite a lot. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 124
IE: So, there are similar things in the Netherlands, but the ones I know are almost all template- based, with some variety with synonyms and things to make it a little more dynamic, but they are not really natural language generated articles. IR: Oh it’s something different. IE: Yep. Our language area is too small for that currently, or of course will develop. IR: Yep, yep. IE: But it takes longer, than in English. IR: Yep, so do you think that is because, everything so far has been developed in English and Dutch is yet to catch up? IE: Yeah, yeah, yeah, and of course the Netherlands has a small market anyway, so if you’re promoting such technologies then it’s more difficult to get a return on investment. IR: Yeah, I see what you mean, that’s a good point, and has {current company} out of curiosity tried it out at all? IE: Yes, we are doing some things depending on how you define robo-journalism. IR: Yep. IE: We are for a couple of years, is we automatically crawl content to try to automatically select events out of them, so we have robots that crawl the entire web, the Dutch language part of the internet, and those robots when they detect that its an event, they will collect that and that way we make sure that our events database is partially automatically filled. IR: Oh ok, that’s interesting. So kind of scraping content for something? IE: Scraping content and trying to find out whether it’s an event or not, because a lot of news is made out of events. You have breaking news of course, expected news, but there is even more expected news. There is a press conference or that there is a football match, well not now but. IR: Coming up? Yeah I see yeah. So you can kind of see something coming, that could be newsworthy? IE: Yeah, you can of course automatically apply metadata, you can assign tasks to the sports department, you can announce to customers, what is going to happen. We partially use robots for that. IR: Interesting. But in terms of Natural Language Generation, not so much? IE: No, in terms of Natural Language Generation, we are currently doing nothing. IR: Ok. IE: But in terms of template-based, yes we are doing some things. For example, I am responsible for the election service, where we deliver the preliminary election results at the day that we are going to vote. And I deliver data, comparing this election to the previous election. IR: Oh yep. IE: But of course you can easily create a template that will make a story out if that. IR: I see, yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 125
IE: So, we experimented with that. For the rest, no not really. IR: Interesting. IE: Not Natural Language Generation or written articles, not too much. IR: So its more, data journalism at the moment? IE: Yep, yep. True. IR: Interesting. IE: And of course you have for example, we have a financial newswire and at the financial newswire we have a specific rhythm that companies, they publish their quarterly, yearly, or half-yearly results. IR: yes, yep. IE: And we use templates for that, so that the editor only has to fill in some really basic stuff. Of course that could be automated in the near future, but we did not do it yet. IR: Ok, so do you think if there more NLG systems in place, considering that the ANP is providing content would you feel that news outlets would buy an article that was automatically written? IE: I think, of course first you need to achieve, or you need to determine what you need to achieve with the automated content. I think that our job generally as a press agency is to fact check, (to filter by), to check sources and to make sure that what is written by our editors, is 100 percent true. It’s quite difficult currently to do that job via robots. What you do of course is if you have specific data sets, for example that can regionalise the data from the bureau of statistics, the central bureau of statistics in the Netherlands. IR: Yep. IE: They publish each month a lot of statistics, and human assisted we could make something and show something, and then the machine could automatically generate regional angles, yeah? So we do one region manually, and then the machine will take similar data from a different region and then will (join the other region). IR: Ok, interesting. So yeah, creative uses of data. IE: (Platform data as well.) IR: Yep, I see that is interesting. So looking at more sourcing routines, do {current companies} customers, do they primarily rely on {current company} or do they feel like they use a mix of sources to find a new story? IE: They use a mix of sources, I think they tend to trust our content completely. One of our roles in this landscape is that they know that they can trust our content, and they don’t need to check that themselves. But we do not cover each topic, or we do not have an opinion, or we do not have opinions from experts for example. So what the media, the customer of us can do on top of our content is make their own angle and make their own story out of it. IR: Yes I see what you mean. So you have like a generic text and it can be repurposed to match their needs. IE: Yep, yep. IR: Ok, that’s interesting. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 126
IE: [And we are writing articles, but we call them ‘particles’, IR: ((Chuckles)) Ok, yeah. IE: = because they are even used as a raw material and then being offered, and our customers add their own flavour and then (information) to it.] IR: Ok, that is interesting, very interesting. Ok right I am going to now share my screen to quickly show you this platform I am working with. Of course I am only just a student on the thesis, so I don’t know too much about it. But I can, give a basic explanation. Lets share my screen. ((IR shares screen with IE, showing TCE mock-up images)) IR: Can you now see my screen? IE: Yep, I see it. IR: So this, I’ll zoom in. This is the Content Exchange, so what I can tell you is that it aims to be a marketplace for content, the most obvious example is a freelance journalist can offer their content here, and can be published, or purchased, or reused by other outlets who may need it for their own websites. Here on the left you search for a topic and then appears a column and here you have all the individual feeds. You can search about almost anything. Then you see along the left here, you also have personalised recommendations, this will be based on, I mean this is to be confirmed but it will be based on user data, or user data or similar users within the platform. IR: I will cycle through the photos ((IR switches between mock-up images simultaneously to speech)), so you can view article, again it is multimedia so audio, images and text. You can either purchase some text, or just the images. You can also share feeds within editorial teams, and create custom ones depending on what you need. This is the login screen. You have your own profile. And then within a particular topic, for example if you want sports you can pick the particular part you want, like so. This is a similar picture. Here is a more detailed image of the filters. And then again you have this recommended dashboard, I have been asked a lot about how this will work. To be honest, I am not 100 percent sure, but I think it will just use, it will be some kind of algorithm or some usage data to try and personalise recommendations that are most relevant, or you know I think you have to show similar and dissimilar topics to people, because they might source something that they might discover by accident or they might not have thought of previously. So, one last look at the dashboard like this. So that is kind of what there is so far, I will come back to the chat now. ((IR stops sharing screen with IE)) IR: So, looking at this Content Exchange, what is kind of your overall impression? Is there a place for it in this technological world we now live in? IE: Can I share my screen first? IR: Yes, please do. ((IE shares screen with IR)) IE: I will show you something that we have. IR: I see it yep. ((IR is displaying and showing a similar dashboard on screen)) AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 127
IE: You know the content exchange is something from {name} and some other person. IR: Yes, I have been in contact with {name}, yep. IE: Ok, can you see what I share? IR: I can see it yep. IE: It looks quite similar I guess. IR: Very, with the columns yep, yep. IE: So I’ll show you, here I have tabs, like in a spreadsheet, they do have content, several content and I can open an article, a story for example. IR: Oh yeah. IE: And its not only, this is {current company’s system}, because this is {current company} tab. And now I go to my newspaper tab, I’ll show you here, I have content from Financieel Dagblad. IR: Oh yeah. IE: Or I have content from = IR: de Telegraaf, yep. IE = or I have content from ((IE navigates system)), from websites. IR: Mmmhmm. IE: Or even, I don’t know if I can show you, well I can even show you. I have content from, well let’s search for “Rutte”. This is a (radio and TV monitor) and here I can show content, and I search and I can only get it by searching. IR: Oh yep, yep. I see. IE: For the name of our prime minister. IR: Yep, yep. IE: This is from, I’ll hover over it. ((IR plays audio clip from system.)) IR: Oh yeah. IE: This is from the radio for example. IR: Ok, yeah, yeah. IE: I will go back to our… ((IE plays the same audio clip)). So basically, we have also what we call kind of a content exchange. IR: Indeed, indeed, hmm, hmm. IE: (I will, stop), sharing my screen. ((IE stops sharing screen with IR)) IR: Yeah so, interesting really isn’t it? IE: Yep, and that’s because we are in media monitoring so we have a database, that has a lot of content from publishers. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 128
IR: Mmmhmm. IE: From ourselves, but also from corporates. They publish content on their website, and we try to collect that all, of course we also pay them license fees. If its publishers, an authorised licensed content, and we sell that to B2B. I wonder, the Content Exchange currently is more ((<)) focusing on selling to media. IR: Yeah, yeah. I am not really sure what their plans are, but yeah early days, early days, yep. IE: Yep, yep. IR: Yep. I think its maybe something for freelancers is my first thought, but yep, yep. IE: [It is interesting of course because what I told you, how I started was that = IR: I was thinking, yeah. IE = it was the middleman. The middle between the journalist, the editor, and the consumer. And that is weaker and if you have a Content Exchange, of course there is, some way of thinking in, well now perhaps the Content Exchange is doing that from freelancer to publisher, yeah? IR: Yeah, yeah, yeah. IE: But of course, companies like (Blendl have tried to put that their own publishers in). IR: Oh really? I have heard that there have been many start-ups in the past, with a similar idea, yeah. So, it’s interesting to see how it will work out. IE: Yep. What is your relation to the Content Exchange? IR: Sorry? IE: What is your relation to the Content Exchange? IR: My thesis supervisor knows {name}, so via that I was able to do a thesis topic based on this. IE: Ah yep, ok. IR: [So I am not too involved, I am only here to, yeah to interview people and get some insights, = IR: Yep. = to what they think about it, so yeah.] IE: It’s interesting of course and basically if you look at, what we are doing, not for text but for phots for example. We typically are a photo content exchange, because the photographers are primarily freelance photographers. We represent a lot of freelance photographers, more than 500. IR: Yep. IE: We have their content, photos, in our database. IR: Yep. IE: And we fill that to media. IR: Yes, indeed. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 129
IE: So, that’s a content exchange. IR: Indeed, yeah, literally. The name is there. That’s interesting. So do you feel like there’s place for something like this? Or? IE: I don’t know, I think there is always a niche market, (for organising things). The Dutch market is highly consolidated, so if you look at the market, the media market in the Netherlands, its so consolidated that… IR: Yeah that’s true. IE: I am not sure whether there is a real, viable place for a new platform. IR: Yeah, I see what you mean. IE: [It could be, of course if they get the support from one of the big players, then they have = IR: Yep, yep. IE: = yep.] IR: I guess it is up to, yeah it depends on their strategy, but it remains to be seen. IE: Yep, yep. IR: But in terms of the design and layout, it does feel very familiar? IE: It looks nice. IR: It looks nice? IE: I think it is the best way to do it, because it looks how we are doing it, so I guess we (get a little tired), as we have been doing it a little longer. IR: Indeed. IE: I think if you think about how to present things, then you sooner or later end up with the column approach. IR: Yeah, that’s true. IE: And profiles, then you can do things like that. IR: Whatever seems logical, yep, yep. Then what do you think of these personalised recommendations, is that something that could work? Or are editors’ kind of too stubborn to have something pushed to them, they have to find something themselves. IE: Well, we looked at personalisation in the past, but I think for a B2B market, and we are (primarily able for that.) IR: Yep, yep. IE: But of course, it is not personalised, but its tailored to the customer, yes? IR: Yep, yep. IE: [And, personalised for media, well it could work perhaps. Depending on who is using your Content Exchange. But I think generally in the first phase, generally the users will look at news in many directions and not… = AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 130
IR: Yeah, yep. IE: Perhaps I am wrong, and I don’t have enough information, I don’t know. IR: Yeah, that is very true, that is very true. Ok interesting comments, I will just close up then. Going back to the technological aspects, like in terms of everything we have talked about, the online newsrooms, technology, AI, NLG, automated journalism in general. Where do you see that going in the future? IE: Well I think that for many years, it will increase, it will not replace journalists but it will empower them to do more, for an ever higher quality, to get rid of tedious, boring work. IR: Yep, yep. IE: That’s for years to come, its more machine-assisted, and not completely automated journalism. IR: Yeah, so not fully automated, but more assisted journalism. IE: Yep. IR: Interesting, ok. Well I believe that is everything from my side, so I will stop the recording.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 131
Appendix J Interview Transcript I
Interviewer (IR): Sebastian Short Interviewee (IE): Participant I Date: 01/05/20 Time: 13:00 Setting: Video call with living room as a background
IE: So I will get straight into the questions. So firstly could you state what your occupation is? IR: You mean by job title or? IR: Yes, yep. IE: My name is {name} and I am the Managing Editor of {current company}, and also of the regional titles, which we work together with. IR: Mmmhmm. IE: And I am also deputy Editor-in-Chief. But my main responsibilities is well everything digital, in a digital section. I work on that especially. IR: Oh interesting, and what kind of roles have you had in the past? IE: So yeah, sort of the same, but in a lower managing functions. I have been, well I started my career as a reporter, first of all in Friesland in the northern part of the Netherlands and after a few years in Amsterdam, where I still live. After 5 or so years I started in a leading position, so I have been there for a long time, and yeah well, I like it a lot. IR: So how many years have you been within journalism? IE: Well, my age is 48. I think I have been now for more than 20 years in Amsterdam. I guess, 22 years. IR: 22 years? IE: 22 to 25, something like that. IR: Ok great, super. So, I am just trying to get a sense of the organisational structure, so where you work owns the local newspapers? Is that correct? IE: {No I started my career in Friesland as a correspondent of the national news agency, {name}. After a few years I started working for the national news agency in Amsterdam, that is also where I know {name} from = IR: Ah, I see. Yeah. IE: who worked for {news agency}. Nowadays I work for {current company}, a newspaper from origin. But nowadays also one of the biggest news organisations in the Netherlands with a website and an app that reaches a few million people everyday. IR: So in terms of news topics you cover, is it kind of a bit of everything? Or is there anything in particular you cover? ((>)) What kind of news topics do you normally write about? IE: Yeah, the mainstream news. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 132
IR: Ok. IE: Mostly everything, from sports, politics, from yeah, police news also I work a lot with my video colleagues nowadays. But yeah I have mostly, I don’t know if you translate {company name} is like the main newspaper, we are a popular news title so we cover everything. IR: Yeah I have heard of it, yep, yep. Ok great, so looking back at your career in general, how do you feel that technological change has impacted your work routines since you started working within journalism? IE: Yes but, for me its personally its hasn’t changed very much because of, also I started my career at the news agency, and you know then there wasn’t that much journalism on the internet, or there was at the news agencies, always that deadline, every minute of the day. So it was very much part of my work route to publish as soon as possible. IR: Yes. IE: And it always has been, when I started at {current company}, I worked a few years for only the newspaper. For me it was very strict, because I had one deadline in 24 hours, I wasn’t used to that. And nowadays well, there’s a deadline every minute, so the fastest… IR: Yeah that’s it. So do you feel like the news being online has contributed to that, is a reason for that? Or is it just in general that’s how its always been. IE: Well, it contributes to that because people get used to it, to get their information very fast. If you know, well for example the fire at Notre Dame, you go to your news app and if it is not fast enough then you go to another news app and you want to see what’s going on. IR: Yeah that’s true. So moving onto more recent technological changes, I am assuming you are aware of automated journalism, robot-journalism? It has many names. IE: Yep. IR: What is overall impression of this? IE: I think it might help journalists because of the, well you can signal new movements. Especially for now as for when, you start looking when corona virus was mentioned first of all, and when it first explodes. And I guess you can see it can its going to explode across the world when you ((<)), should study the internet, from say December until now. But we were much more aware, we should have done that real time, then we yep. IR: I see, but in terms of like text, news articles that are automatically written, do you feel that can be beneficial or not to the newsroom? IE: [Yeah I think it could be beneficial, because we are working with it in a test setting = IR: Oh interesting. IE: = and especially for the football and following the smaller leagues. But I think they have to do lots of work to get it to function well, and come to a publishable text.] IR: Yep, yep, yep. So its kind of an experiment at the moment? IE: Yeah, its an experiment, and you need to have the right data, also of course. IR: That’s a good point. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 133
IE: That’s a reason why we started with sports, because that is the kind of journalism with a lot data. IR: Yeah, I have heard from others that is a common topic that is being used for automation, yep. So going back to {current company}, do you feel like there is kind of a good balance between journalistic quality and financial performance? IE: How do you mean? IR: As in, do you feel like you or your team have enough time to write what they want, if you know what I mean? IE: Yeah, mostly. I have enough, I have a feeling we have enough time, enough financial possibilities to do so. IR: Are you primarily using advertising for revenue, or is there any other? IE: Yeah advertising and subscription. IR: Oh, and subscription yep. IE: Yep, and ((>)), well nowadays especially in these times of coronavirus, you see that the advertising is very difficult, because a lot of companies are not advertising at the moment. But subscriptions are doing very well, (we are pleased to have those sort of incomes). IR: Yeah, I have heard that from a couple of other people too. IE: Yeah. IR: They have said the same thing, that’s interesting how that works, good for you guys I guess, yeah. And so, looking now more at sourcing routines, how do you or the editorial team in general, how do they go about sourcing new articles. How do they go about sourcing new content, that could become a new story? IE: Yeah, well we start everyday with a news meeting, with all the editors. Some of them are really, well they have their contacts and come to stories like that. Other ones use data (from it) or use social media to find topics and see what is going on. So yeah it depends on which editor you (are working with). But it is a mix for those things. IR: And what kind of social media do you look on? IE: Well, Twitter is still a big thing for news organisations, I don’t think for the main public, but also Facebook, Instagram. Some of the younger people use Reddit for it. IR: Yeah, that’s quite useful I hear, interesting. And do people just kind of search online for information too? Or do they go straight to social media? IE: Noo, I also think they are online also, and we also use tools like for example, (crowdtangle), you know that? It’s a tool, you can monetise Facebook, and you see especially what’s doing well on Facebook, so you can see the topics people are reading. IR: Yeah, interesting, and do you use something to monitor Twitter with, like Tweetdeck or anything like that? Or Hootsuite? ((IE changes location due to too much background noise)) IR: So, I was just wondering do you use anything to monitor Twitter with, like Tweetdeck or Hootsuite, anything like that? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 134
IE: Yeah, well I personally use Tweetdeck and I think most of my colleagues also do, ((>)) and Hootsuite I used that in the past, but I that was a long time ago. But there is not a main way to do those things, every editor has their own, kind of workaround he or she prefers to work with. IR: Yeah, I see, so it kind of differs on the editor? IE: Mmmhmm. IR: And what is kind of the overall balance between say, using their network versus using their online sources? IE: Oh yeah it depends who you are talking to, but it is kind of 50/50. IR: Ok, interesting, interesting. With that in mind, I am going to now share my screen. I am going to show you the Content Exchange. Has {name} told you about it at all? IR: Yeah, just only highlights of it, so. ((IR shares screen with IE, showing TCE demo)) IR: Ok, can you see my screen? IE: I see your screen yes. IR: Perfect, so I will just tell you what I know about it too. So, the Content Exchange was launched very recently, its basically aims to be marketplace for content. IE: Mmmhmm. IR: And so you have a dashboard like this and an account, and you see here I have 3 feeds, and if I ((IR types “Eindhoven” into the search box to generate a new feed))…its in very early stages of development so if I search a topic here on the left, then a feed comes up here. And then you can scroll down, then you can select an article. You can, oh that’s a long text, you can either download some of the text, or all of the text, or this one does not have images, but you can do just the images. It eventually aims to be multimedia, so you can also do audio, video and things like that. You can search within the feeds, I think also eventually their will be filters within the feeds, but that is a feature that hasn’t arrived yet. So, to give you a better idea I have some more mock-ups here. ((IR switches to mock-up images and switches through them simultaneously to talking.)) IR: I will just zoom in here. So this is more what the dashboard will look like in the future. So you can see here, you have on the left side here you have recommended feeds, so the aim here is to kind of produce personalised recommendations, based on either your usage data or users similar to you, their usage patterns. On the topics that you may have not considered yet, that’s the plan. I think they are kind of still developing the algorithm for it, so the exact details of how that may work, is TBC. IR: So, here’s the article again. You can also have shared feeds, so for example if you are within an editorial team you can create a custom feed and share them within the organisation, so everyone is on the same page. And then, that’s the login screen. That’s the profile page, and then you can see within a topic you can select what you want. For example, if you are looking for sports then you can specifically specify a sport you want to write about. And then, that’s another view of the dashboard. And then you have your own feeds here again, that’s just a another perspective on that one. And here it is again, with the filters, and you see here you have again AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 135 the personalised recommendations, that can come from a variety of sources based on, yeah what you are interested in, or what you may be interested in, that’s the key point there. IR: So, one more last look at the dashboard ((IR switches back to the demo)), as you can see it is very early days yet, but something is there. This is the demo account I have, so I will come back now I still stop sharing my screen. ((IR stops sharing screen with IE)) IR: So I guess one obvious example is freelance journalists can offer content on the platform, and it can be immediately reached by outlets who want to use it. And I guess the goal is for editorial teams and news outlets, or publishers to try and more easily find content that is online, from a variety of sources, depending on the feeds they have. So that is kind of the Content Exchange, maybe {name} can tell you a bit more about it in detail in the future, but that is where it is so far. IR: So, with that in mind, and what you have seen, what is kind of your overall impression of this system like this? This kind of dashboard? IE: Yeah, well I know this kind of dashboard, from other (companies). And my experience is that editors, that there are editors who might use it. But is there very individual choice for editors, as I already told you, everyone has his or her own workaround, and it is difficult to change those workarounds they are used to. IR: Yep, ok. IE: Especially now on a daily basis, you know they have other tools. You know sometimes you introduce a new tool, everyone is enthusiastic about it but in the end, its not that, well they don’t use it or nearly use it and not because they don’t want to use it. They are used to their own workaround and they… IR: Yeah, I see, I see. IE: Yeah. So it has to be very attractive and very useful, to give a benefit when, if they really are going to use it on a daily basis. IR: Yep, I see. So for it to integrate into someone’s routine, it would have to stand out, or do something incredible. IE: Mmmhmm. IR: Ok, interesting, and in terms of the design and layout, did it seem… did it make sense to you, did it feel logical? IE: Yes it makes sense, because you get a good overview and what I saw in these few moments is that you can personalise it and that is important. IR: That’s handy, yep. IE: Yeah, because (all the) sports editors want to see very different topics for instance, than political editors. IR: Yes, indeed, indeed. IE: Yeah. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 136
IR: Yeah the filters are a key feature, yep. In terms of personalised recommendations, because you mention everyone has their own routine, do you feel like that is a useful feature or, an interesting idea? IE: Yeah, I think it is a useful feature, we have in our own publishing system, we have used, how is it called? Its (Smartocto), its not the same as this tool, but you publish a story, it manages, well how does it perform on Facebook or is it still not on Facebook, and it gives you a very personal advice to do things with your article, and I think that works out well, because you think “Oh! This makes my story a bigger…”. IR: But if this system recommended a story to someone, do you think that person would then go onto use that story, or would they think I need to find this myself? IE: I don’t think he or she is going to use it. IR: Ok. IE: But I think she’s going to read it, and then do something by themself. Well, how do you say? Well in journalism this is sort of not done, to use someone else’s story. ((Connection drops out, so there is a pause in the conversation)) IE: I think the journalist will use it, I think he will use it more as advise and not will use the story itself, but to start to bring their own story, or to investigate a topic. IR: And if a story or a text was automatically written by a robot, do you think they would, or you would still consider using that? Like, sourcing a text that was written by a robot. IE: Yeah, I think so. (There was) a study at the University of Tilburg where a sports desk, they did some interesting stories about the psychological effects on automatically written articles. Our confidence is, well their was not much difference between, when it was written by a journalist or when it was written by a robot, or computer or how you may call it. IR: So, actually sourcing an article that is not written by a robot is not much of a big step? IE: No, I don’t think so. IR: Interesting, yeah I think I…it sounds familiar this study, yeah. That’s the impression I get, yeah interesting. But, does it fell familiar like with the columns and stuff? I know a lot of people said its kind of similar to Tweetdeck, because you have the columns and the scrolling. Do you feel like it is beneficial to have it familiar to other platforms? IE: Well, I think so because we again, there is your routine where those journalists, where, well it is similar to the routines they are used to. IR: Yeah, I see, I see. In terms of, going back to the sourcing routines themselves, is there any like, personal factors that drives people to find new stories? For example being curious? Does that play a large role, or do people reach out to their networks and then get a story? IE: Oh I think it is so different between journalists, the way they work. The example, while sports reporters, they go to, for example an Ajax reporter, he needs his contacts and he has to be on the training field and walk around at the club to get to his information. But, well when you are a health reporter, you read a lot in all kinds of magazines and yeah, well you have contacts at universities and those kind of things. But much more working from behind a desk. IR: Yeah that is an interesting point. The topic and the starting point is key. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 137
IR: Ok, interesting. Are there any other comments, just in general about the content exchange? Like, about the business model, or the design or anything else like that? IE: No, I think it has to be easy, most journalists well are digital savvy but not that digital savvy that they can, work with a lot of interfaces and all that kind of stuff. And yeah I think you should have a test panel with it, (see how it progresses), and that is mostly the best way to see what is useful for them, or what do they miss? IR: Definitely, yep. IE: Yep. IR: Yeah that is a, I think they are with contact with some people, they have some testers. IE: Yeah. IR: So, I think that is the next step. IE: Yeah, also for us when we are design the web or the app, it is always very useful to get feedback from the users. Because you know you own product too well. IR: Yes, exactly, yeah. IE: To see improvements, users see. IR: Yeah, because you know it so well and you push it to users and they, see something else. IE: Find a black spot. IR: Yeah, exactly, yeah. Interesting, ok great I think that is the main line of questioning covered, so I will just conclude by asking, in terms of everything we have talked about, in terms of technological change, Artificial Intelligence, Automated Journalism, where do you kind of see that going in the future? For journalism? IE: Well I hope that we can put our journalists into the work where they as humans are very good in. So everything what you can do automatically, it should be done automatically. So everything what repeats, should be worked with artificial intelligence, or whatever, and you can use your creativity and your knowledge to do the things, well you have a really, special gift as a journalist, so I hope this will find a way through. IR: It will increase, do you feel like it will stay within factual topics, or will it expand, will it kind of stay as like a compliment? IE: Well I think it will expand, but it has to start with a lot of basic work, as I told you I handled sports journalism, but also in financial journalism. Journalism where there is a lot of data, well you can use those techniques, but I think there will be a, when it works well, there will be those texts, well I don’t know at this moment but I am sure they will be those texts. IR: Yeah, yeah, that’s interesting. So wait and see at the moment. IE: Yeah. IR: Hmm, yeah that’s a sentiment that… IE: You have that experiment with that author who wrote a book, he wrote it together with a computer, so (there was robot). Then I guess this is a step, well it was not perfect but it will be become reality. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 138
IR: Yeah, it could get better, it’s…the technology is catching up. IE: Yeah, yeah. IR: Interesting, yep. It’s a sentiment shared by most people I spoke to, yeah. Its like a yeah, going to increase. IE: There are some journalists who are also afraid, for (technology rising up.) IR: Yeah, yeah I have heard that too. IE: Well I am more curious than afraid. IR: Yeah, yeah. IE: I think you can be afraid of your job and then say “ooh I don’t want to work with it because it takes my job”, but then other people will investigate it and (use) their resources and it will happen anyway. You can better be part of it than. IR: That’s true, embrace it. IE: Yeah. IR: All right, perfect, well thanks for your time, I will stop the recording now.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 139
Appendix K Interview Transcript J
Interviewer (IR): Sebastian Short Interviewee (IE): Participant J Date: 01/05/20 Time: 15:00 Setting: Video call with living room as a background
IR: So firstly could you state what your occupation is? IE: Yeah, my name is {name} I am the Editor-in-Chief of {current company} and yep, I have been for the past 15 years, I was the creator of {current company}. IR: Oh, interesting ok, and what other roles have you had before that? Or have you always been at {current company} IE: No I studied at the University of Amsterdam, communication science. Then I sort of rolled into journalism, I thought it was more fun to practice journalism than to study it, perhaps. I started out at a lifestyle magazine and at that first job I sort of, I started an electronic newsletter and such a method, yeah maybe I was one of the first internet journalists in the Netherlands. IR: Interesting, wow. IE: Back in 1994. IR: Yep, yep. IE: So from there I went onto an internet provide with…, they had an editorial team so from {company} and {other company} I made there to plan an internet… there’s a bio on my website, my personal website, by the way if you want to. IR: Ok, I will check that out. Great. IE: Anyways, so in 2005 I started {current company}, I did it with a publisher and as of 2015 {current company] is part of {parent company}. IR: Yep, ok. Interesting. IE: Which brings us to the current situation. IR: Interesting, so roughly how many years have you been in journalism? IE: Well, since 1994. IR: Ok, great, and since you started working within it, how do you feel that technology has changed your work routines? IE: Well, perhaps for myself it has not changed compared to someone who is traditionally schooled in journalism, because for me journalism started with all the technology. And I was one of the first internet journalists in the Netherlands so, I don’t know any better, in a way. IR: Indeed. IE: But I have seen in those 25 years, I have seen it transition or develop in technologies that has been used in journalism, and you know I started out with very simple plain text newsletters, to AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 140
(new style sheet) driven, how do you call them again? Those websites that change to platform you are using, you are watching it on? Dynamic. IR: Yeah, yeah the dynamic yep. IE: Those two style sheets, dynamic driven websites and yeah so, I have watched the web blog arrive, obviously the mobile apps. IR: Yep. IE: Podcasting, you name it, basically everything. IR: Yeah, yeah. So was {current company} itself, was it always online? IE: [Yep, yep, in the beginning we also had a magazine, = IR: Oh, you did? Ok. IE: an old-fashioned magazine that has been around for roughly 10 years. IR: Mmmhmm, mmmhmm. IE: It doesn’t exist anymore nowadays, so {current company} is currently mostly digital, and television and events. We got a website, we have got an app, all sorts of media profiles. We have a podcast, we have a television show, with {parent company}, we do {event}, a big tech fest. So yeah, many feathers. IR: And you write primarily about technology right? IE: Yep, I think our biggest competitor in the Netherlands is {competitor name}, I would place us, if you have a scale between hardcore technology and lifestyle, then we are more towards lifestyle than {competitor}, who are just a bit more towards the hardcore tech side. IR: Oh, ok so it is more towards like consumer tech? IE: Those two are the biggest titles in technology in the Netherlands. IR: And is it more consumer tech or? IE: For {current company} yes, primarily (our biggest scene). We consider ourselves sort of pre- testers for our audience, so = IR: Ah ok. IE: = we test technology, we write about it and we also have a buyer’s guide.] IR: Ah, ok. IE: So, we sort of thing is tech-minded people. IR: Interesting, I read something similar called, I don’t know if you have heard of The Verge, in the US? IE: Oh yep. IR: Yeah so it is kind of like that really? IE: Yeah, or perhaps an example that has been around for a bit more is Wired. IR: Yes, yep, Wired. Yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 141
IE: They also started out with a magazine and a website, same as {current company}. Yeah the first that came from a group of people that did Engadget before that. IR: Yep. IE: Actually one of them is an acquaintance of mine, {name} he lives in Amsterdam here as well. IR: Oh really? IE: Yeah, so it is basically The Verge meets Marcus Brownlee. IR: Yep. IE: Yeah, so I think {competitor} is the biggest in terms of web traffic, but {current company} is the biggest in terms of YouTube traffic and Instagram and Facebook. IR: Ok. IE: Yep. IR: Ok, great so looking at more recent technological change, I assuming you know about automated journalism and robot-journalism, like articles written by algorithms? IE: Yep. IR: What is your overall impression of that? IE: Well I would say that, there is a part and maybe a larger part than we can imagine right now, or we hoped for, I think can actually be written by smart algorithms, or robot journalists, it depends on how you call them. But because in the end it is, a lot of the content put out by news sites, news outlets, it is in a way almost mechanical. IR: Yeah, yeah. IE: So, you know. You got to source, there is a lot of institutional sources, sort of news flowing in, press releases, you name it. So that can be, yeah up to a point automated. IR: Yeah, yep. IE: And I would say where you transcend, this basic level of information ((<)) presentation, so relate to the realms of opinion, or context, you know I would say there is still room for people. IR: Ok, so for example you couldn’t automatically write a product review for example? IE: Well a review actually in my opinion is opinionated. IR: Yeah, there you go, yep. IE: Well so a basic news article, mostly consisting of facts. IR: Yeah. IE: That has been written because there was a press release. It basically, so stating facts. That can be automated. IR: Yep, yep. IE; As soon as you enter the realms of opinion, or expertise, or context, then that I think there is still room for us. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 142
IR: So, do you think its beneficial, or not for journalism at the moment? IE: The existence of robots? IR: Yeah, yep IE: I said say that I think it is, the actual output by robots compared to the output by human journalists is still, well its very small. It may have been my perspective from the way I use content myself. IR: Yep, yep. IE: I wouldn’t say it is significant yet, but I think it can be. IR: Ok, yep. Some example off the top of my head are financial reporting, and sports reporting. I know it is used quite a lot in football matches for example. So yeah, those factual events are, I guess it is quite useful for. But yeah once you get over to… IE: Especially you know when there is a time issues and there is long, for example in the financial sector, time is money, and when your team has scored a goal, you want to know as soon as possible. IR: Yes exactly. IE: So yeah, I can see there is room there, like you said when its concerning factual journalism, yep. IR: Ok, yep that’s. I have heard about that a lot yep, yep. So moving onto your sourcing routines, how do you go about discovering new content or souring new, like finding something that could become a new news story? IE: Sure, well I would say that at least 50 percent or maybe two third is good old fashioned press release and so, stuff that PR people or divisions send to you. And I would say that it is not just the press release but the whole publication, from PR to journalism. That also accounts for a call you have with someone, a spokesperson from a certain company. All that kind of contact, that communication, that’s a pretty big chunk. I think the second biggest chunk would be what others do, but put it very simple. IR: Yep. IE: I think in the end, everybody, journalists included has their own web of trust. IR: Yep, yep, yep. IE: They have their own circle of sources that they like, that they trust and that they regard highly. IR: And what kind of sources could that include? IE: Well for example you mentioned one already, The Verge is one. There is also, yeah, well also here in Europe there are certain product releases, or spec releases, or information that is being released under embargos. IR: Yep, yep, I have heard about that. IE: Americans have it sooner than we have. IR: Ah ok, yep, yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 143
IE: [So, at that point they became a regular source for us = IR: Ah, ok. IE: = but when the company itself would send us that information.] IR: Mmmhmm. IE: Yeah, so what others do ((>)), it sounds perhaps a bit weird, it is not like we are you know copying things, we are not sort of you know, ((>)) spying on, how do you call it? In Dutch you say spieken. Well, anyway it is not about copying what they are doing, it is just, you’re widening your own reference pallet. IR: Yeah, I see what you mean. IE: So you have got your own sort of line of opinion making, and here there is a bit of news, or there is a new product, you get it or you read it, and you create your own opinion about it. And you make all kinds of associations, so your expertise kicks in because you have been doing this for a long time already, you can remember “hey didn’t company X do almost the same a couple of years ago”, or you know that kind of thing. So you are processing the news, or the product and in that process, other journalistic sources for example, like The Verge, can play a part in that. It is not just The Verge of course. IR: There is a myriad of outlets, yeah. IE: Yeah, I would say I have an A-list, I would say 10 sources. The Verge is on there, so for my field of interest, technology. You got The Verge, Wired, Dave Lee, Marcus Brownlee. There is also the communication between the team members at {current company}, because we are all experts in this field. But for certain, sort of sub-specialties and certain typical unique points of view, and certain even biases. Not biases, but perhaps, favourables. IR: Yeah. IE: Favourites. IR: Favourite brands. IE: So and all that together, and even perhaps some user comments already. IR: Oh yeah. IE: Some response beneath a certain video, or reading an article or. All that information combined, leads you to your own opinion about something. IR: I see, yep, yep. Do you search online at all? Do you use social media? Or is it kind of different for tech news? IE: Both. Yeah well its mainly online. IE: That’s true, yep. I don’t watch television to get opinionated about tech related news. Well perhaps one or two shows that can have, that can be, for example here in the Netherlands you have Tegenlicht, a news documentary series by the VPO. That can be informative, or bring you a certain new perspective on things. But it is mainly on the internet, so itself, the news sites, the web blogs, social media profiles, perhaps well in my own case not many influencers. With influencers I mean people on Instagram, just taking nice pictures and not particularly being knowledgeable about things. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 144
IR: I see, yep. IE: So, for example the names I mentioned, Dave Lee, Marcus Brownlee. I consider them critics not influencers I consider them critics. IR: Interesting. IE: So, yeah what was the question again? ((said jokingly)) IR: ((Chuckles)). So like yeah just in general your routine is just kind of researching and seeing what’s out there, and forming your own opinion on a story you are writing about? IE: Yeah, yeah, so in the process there is a phase of information gathering, from a direct source, other journalists, your colleagues, your own historical knowledge of things and so once this gathering phase is finished, you start constructing your own story. So you are laying the groundworks and you are building up from there. And obviously the third and final phase to get it out, with video or an article, or you name it, yep. IR: Interesting, all right. So with that in mind I am now going to share my screen, and show you what {name} has been working on. ((IR shares screen with IE, displaying TCE demo)) IR: Can you see my screen now? IE: Yep. Handsome devil – oh sorry. ((Both IR and IE laugh)) IR: Perfect, so this is the Content Exchange, this is a demo at the moment so it is kind of a basic interface, but I also have some screenshots later. So first this is the main dashboard. You have columns for each story. IE: The logo needs work man. IR: The logo? IE: Yep. IR: You think so? Ok. ((Both IR and IE laugh)) IR: Yeah, I’ll feed that back to them. IR: So, for example. I am not sure if technology, because they need more content into this system ((IR types in ‘technology’ into the search box to attempt to generate a feed’)). Let’s just try and see what happens. Yeah, technology is empty at the moment. I will just remove that for now. IR: [So, essentially = IE: Well that’s unfortunate. IR: = yeah. I think they need for the demo, to like feed into it.] IR: But, let’s see what they so with that. So I am not sure how much {name} has told you about this, but basically it aims to be a central marketplace for content. For example off the top of my head, a freelancer can offer an article on this system, and then it can be purchased and reused by other outlets, or editorial teams, or publishers, what have you, for a modest fee. For so example you can view this article, you can view the text. You can either purchase some of it, or all of it, depending on what you need. You can also just purchase the image, I think later it is also going AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 145 to use video and audio. So its going to become multimedia, so along the side you have the feeds and the columns here. You can search inside the feeds, and things like that. So I will show the images here, because they give you a better idea. ((IR switches from TCE demo to mock-up images. IR then shows all images while talking)) IR: So, this is the dashboard, a bit more of a detailed view of what they see it looking like. You see alongside on the left column here, they have the recommended feed. I think their aim is to have personalised recommendations, based on some algorithm, TBC on that one. I think it is going to be based on the usage data and the usage data of similar users. It will kind of push topics that you may find interesting, that are similar and dissimilar, so you kind of discover something that you might have not previously considered. So, I will just show you these images. There is the article again, you also have shared feeds. So, for example within an editorial team you can create a custom feed for a particular aspect of a news topic and share it within an editorial team or you can share it with other users. And then that is the login screen. You have your own profile, like this. And then for example, you can filter within a topic, so for example if you into sports you can select the specific sport you want to write about. And then, that’s another image. And here is a more detailed view of the filters, also by country. And then we carry onto the end here, and then you have this recommended feed again, of personalised recommendations from a variety of sources. For example you could have your top 10 list coming up here, if they include that in the algorithm. IR: So, that is kind of The Content Exchange at the moment, of course it is early days, of course this demo is I think, yeah it doesn’t have all the features they have yet. I think it is going to launch sometime in June. So that’s kind of where they are, so far. So I will stop sharing my screen. ((IR stops sharing screen with IE)) IR: Ok, so looking at The Content Exchange and the dashboard. What is kind of your overall impression of this system? IE: Well it’s a very first impression obviously. IR: Yeah, indeed. IE: You know, like 20 seconds ((Both IR and IE chuckle)). Yeah I know about the idea roughly, yeah at this stage, I don’t have an opinion yet. I mostly have a lot of questions. IR: Ok. Go ahead. IE: So, in the end I understand the basic premise. It is similar to syndication of content. IR: Yes, exactly. IE: But so, who are the partners in this system? How do you enter? What are the pricing schemes? So I have a lot of questions actually. IR: Indeed, indeed. IE: And maybe the most important question right now is ((>)), why are you asking me? What capacity do you see me, do you need to be asking me these questions? As a potential customer? As a potential partner? Or both? Or? IR: Well, more from the perspective of, you are an Editor-in-Chief and I am assuming you know you like an editorial team, and they’re all, well they have their routines for researching and AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 146 finding content themselves. Could something like this, for example if it offered the right content, could that fit in? That is the goal, at the moment. Of course the content there is, they are testing with other content at the moment. But in the future it might include more, you know tech content from a range of sources, depending on… IE: Sure, well one thing that comes to mind is that, I would say you have two types of content. IR: Yep. IE: First, going back to our introduction, I would say you have got hard news, hard tech news, so very factual, and you have got tech opinion. Opinionated news, you can have a very long discussion on whether that is actually news, opinionated content, you could call it that. IR: Yep, yep. IE: So, and I goes for actually for every…it goes for sports, for economics, for technology, all kinds of sections so to speak. But I would say there is a big difference there, and so for hard news that is the kind of content that will be offered, through the main players. IR: Mmmhmm. Yep, yep. IE: So, NU.nl, RTL, NOS, Telegraaf, so the big newspapers. Those are the players there. IR: Yep. IE: The other customers I would see are, medium-sized publications, like regional newspapers. IR: Ok, interesting. IE: Yeah because ((<)) if RTL doesn’t have a certain kind of article, what would convince them to buy an article, Dutch is a very small language, so everybody knows all the other players. So what would persuade RTL to actually buy an article from lets say NU.nl, and publish it. While there is also already ANP, yeah ANP ((saying first the Dutch letters and then the English letters)). (They could use that). IR: Yeah, yeah, yeah, yeah. But what if like a freelancer offered like a new article that was on something that they had seen, like a local event that only they had reported for example. IR: Sure, sure I can imagine smaller players being added to this system. But then, the vetting of those smaller players would be very crucial. IR: Yes, yep. I see what you mean. Yeah the questions remain about business models, and how the financial side, and also the legal side of things too. So that’s something they are working out. IE: I can imagine that when a certain medium joins and is a partner in this system, it’s a system that can give, and that from which can you can take. IR: Yep, yep. IE: So, you contribute your content to this system, and you can use others, the content of others. IR: Yeah, literally exchanging content. IE: Well, there you go. I am not sure whether either one should be, obligatory. I am not sure if it, it really depends on an exchange like this, it can still go in all kinds of ways. It could be an exchange for big players trying to cut budgets and sort of cut corners and get content on the cheap. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 147
IR: Oh yep. IE: But then I would say that the number of players is so small, that they probably won’t do it, because well they are too proud or something = IR: Yeah I have heard that a lot. IE: or (each other), I don’t know.] IR: = So that’s one scenario. Another scenario is that you try to connect through this exchange, big and smaller players.] IR: Yep, yep. IE: Then you, because like I mentioned you got a big issue about vetting, for smaller players. A third scenario could be, ((>)) well what’s the third here. There is a third. I have to sort of…phrase it. ((>)) Well, for the time being let’s stick to the first two, but what is this exchange, what does it want to be? Does it want to be sort of a new ANP? Or sort of for (natural) articles and pictures, maybe like other agencies like Hollandse hoogte or internationally of course, with what’s the big picture bank called again? It starts with a G. IR: Oh, um. I know the one you mean, yep. IE: Well anyway, those kinds of companies. IR: [But = IE: It’s hard. IR: = I see what you mean yeah, but looking more at the design and user interface, do you feel it makes sense? Does it seem logical and navigable?] IE: Well a couple of things, so a minor detail, but something that draws my eye, the logo, there is still sort of a better version I would say. Then, I find all the logos from the sources very distracting because they are full colour. It would distract me from the actual content. IR: Yep, yep. IE: I would consider perhaps doing that on black and white, or something, or grey tones. IR: Mmhmm, mmhmm. Yeah, I have heard that before, yep. IE: [Yeah because now it almost seems like an RSS reader = IR: Yes, yep. IE: = for the end user]. IE: And, ((>)) yeah its really hard to say something really sensible at this stage, because I am, so the concept is still evolving, and you are asking me about interface. IR: Yeah, yeah, yeah. I am not expecting like a, yeah you know. IE: So, the second one follows from the first, and that is not done yet. IR: And if some of these articles were automatically written, do you think you or anyone in general would still source them? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 148
IE: Yeah, actually I think I would source them sooner than getting content from my biggest competitor. IR: Oh really? And why is that? IE: Well because then in that case, if I publish it my competitor doesn’t feel “hey look at that man, there he is buying and publishing our stuff”. IR: Oh yeah I see what you mean, interesting. Like a way to stand out, and what about… IE: If it is written by a bot, actually I would say it would enlarge the USP of the Content Exchange. IR: There you go, yep. IE: If they could develop really cutting edge algorithms that none of the players, yeah the current large publishing players have, or can develop. IR: Yeah, there you go, a technological advantage. IE: Yep. IR: That’s an interesting point actually, something they, I am not sure if they are. I think they are considering it, so we will see how that plays out. IE: I have about five minutes left. IR: Ok no problem, just quickly then about the algorithms, do personalised recommendations, do you feel that would work for an editor? Or do they feel that they have their own routine that they are stuck and they don’t want..? IE: It depends, there is a question prior again to this one, and that depends on the scalability, and what kind of scalability is the Content Exchange aiming for? IR: Yes, I see. IE: If this goes too big, you can’t solve it with people, just like YouTube can’t curate all of their videos, (that pass through their system). IR: Yeah that’s a good comparison, yep, yep. Because the recommendations you get on YouTube are really strange, I find. You watch one tutorial video on how to, fix your doorframe for example and then you get a massive list of doorframe tutorials. So… IE: (They need to figure something out) IR: So, yeah that is something to work out. And, ok so I will kind of wrap things up, by just asking in terms of everything we have talked about, in terms of artificial intelligence, automated journalism and sourcing, where do you see it going in the future? Do you think it will continue to rise, or? IE: Continue to rise? IR: Or would you, where do you see automated going in the future? IE: Well, it probably goes ((<)), it will probably grow but it will probably make bigger leaps when there are breakthroughs with algorithms, for this particular practice. IR: I see, yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 149
IE: For example, here in the Netherlands we need an algorithm that can write Dutch. IR: Yes, yep. IE: Simple, simple. IR: I have heard that a lot. IE: Yeah, but I am not aware of anyone who actually, there are some experiments at {parent company} with bots, but that’s it. IR: So it is still experimental at the moment, at {parent company}? IE: Yep. IR: So yeah it is just seeing how the technology catches up, with what people need. IE: Yep. IR: Ok, well thanks a lot I will stop recording now.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 150
Appendix L Interview Transcript K
Interviewer (IR): Sebastian Short Interviewee (IE): Participant K Date: 23/04/20 Time: 10:00 Setting: Video call with living room as a background
IR: All right. So firstly could you please state your occupation. IE: I am what you would call Editor-in-Chief in English, of {current company}, we have a website. So I have been here for the last 10 years now. IR: And what other kind of jobs have you had before that? IE: I was, mostly a journalist, long before that, that was 25 years ago. I have done everything, I was a (musician) once as well. IR: Oh wow. IE: But, most of the times I have been a journalist, freelance and in {current company}. IR: Ah ok, and at the moment then for {current company}, what kind of topics do you cover? Is it just general news for {city} or? IE: Yeah, everything you need to know, in the {city} region, so that’s economics, public worlds, restaurants, ((>)) everything. IR: Yeah, yeah, yeah. Ok great, so looking back just in general over the past 25 years you have been working within journalism, how do you feel that technological change has impacted your work routines? IE: Honestly? IR: Yeah. IE: Yeah of course because like 25 years ago, we had to invent everything we were planning to do on the internet. We were still mainly a paper driven organisation. IR: Yeah, yeah. IE: And also, we made it very old school, fashioned. Like you made the pages, they were still, you still how do you call it? Cut and glued together, (to feature) and then to print. So two things on the paper production side, everything has changed. IR: Mmmhmm, mmmhmm. IE: Apart from, everything we print now we read from out screens, so everything in-between has gone, in 25 years’ time. And of course, we have had the digital revolution, like any other company. IR: Yeah, yeah. So is {current company} primarily online now? Do you still have newspaper going? Like a print version? IE: Yeah. We still have a circulation of, between lets say around 70,000. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 151
IR: Ah ok. IE: That is quite a lot in a small region, a region that is 750,000 inhabitants big. So, 70,000 is still a very big newspaper. IR: Yeah that is pretty decent. IE: Online, unique visitors a day varying between 250,000 to 300,000 a day. IR: Ok that’s interesting. IR: Yeah. IR: Ok great, so do you feel like moving of news online put any pressure on your role at all? In terms of adapting to it? IE: On my role as an Editor-in-Chief or as a news bringer? IR: As your role, as an Editor-in-Chief. IE: Yes of course, because you know like 15 or 20 years ago everybody said the newspaper would be dead around now. But we are still printing them, so that’s one concern, we have to keep it alive for the next 20 years. IR: Yeah, yep. IE: Or maybe 10, or, who can tell? IR: Yeah difficult to tell, yep. IE: And at the same time we have to invent the newsroom, to make all the digital content. So some of it is the same, in paper. But under the digital, the public has different demands, like video and… IR: Oh yeah, yeah. IE: So that, sometimes you are in a difficult position. You have to make the old product, and the old product has to be finished every day, so you would, well you could wait a day or decide if it is big, or small that day. On paper you have so many pages you have to fill them each day, and around 12 O’clock it has to go to print. So, you know, sometimes they are two different worlds that you have to bring together every day, so sometimes it is difficult, sometimes it is easy. IR: Interesting, and do you feel like if I can ask this. Is there a good balance between journalistic quality and financial performance? IE: Yes. IR: Yeah? IE: Very much so this time, because of the whole corona crisis, people know how to find us, and we sell more digital subscriptions at the moment. And people, of course they feel that have, they want to read about sports, and games as well, most of the demand is on, what do you call it? More serious journalism. IR: I see, I see. Yeah I have heard that from a lot of people I spoke to already, they said exactly the same, the digital subscription part is really growing at the moment. IE: Yeah, very fast. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 152
IR: Interesting, yeah. IE: Yeah. It’s a different public, the question is, can we keep them? Like we can keep the print subscriptions. Print subscribers are people that have had a subscription for like 20 years, so they’re very loyal. IR: [And we don’t know that about the online subscriptions, yet. So we still have to reinvent how we keep them with us = IR: Yeah and keep them, retain them. IE: = for periods of time.] IR: Interesting, yeah that’s sounds quite good. It is interesting to see how people react to this, in terms of news, to the crisis at the moment. IE: Yep. IR: Yeah, really interesting, yep. And so moving onto more recent technological change, I am assuming you may have heard of, like it has many names, but like automated journalism, or robot-journalism, things like that? IE: Yep. IR: What is your overall impression of that? IE: Well, I didn’t understand the last sentence. IR: What is your overall impression of robot-journalism, or automated journalism at the moment? IE: So its very useful I think. So, why not? You know? Yeah if I can use the good journalists to do the good journalistic work, and have everything automated that can be automated, why not? You know? So like, you know sports, ((>)) well the numbers, sports numbers. Like ((>)), the agendas, you can have it automated by robots, why not, so I am not against it, not at all. IR: Interesting, interesting, and so you do feel like its use will increase in the future? IE: If I could? IR: Do feel like its use will increase in the future? Or will it stay within basic stories? IE: Its still a bit vague for me how we can use it, and we are trying to find out where we can use it in our process, and well its still, I am hearing about it for years and years and now its still, well at the point where I can use it for this, or I can use it for that. So, I am not sure. IR: So, waiting for it to catch up? IE: Yeah. IR: Has {current company} done any experiments with it at all? Or thought about it? IE: Not {current company} itself, but the firm we are in {parent company} is experimenting with robot-journalism, yes. IR: Interesting. IE: Yep, yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 153
IR: And do you know anything about that, or is it still kind of early days? IE: Its early days, but hopeful. IR: Ok, its promising. IE: You know for the, what you would call, the chores you could, like in sports or in agenda. IR: So, all the factual information? IE: Yep, and of course robots can be useful, but with helping to write your story as well. So not only that it can totally take over your job, but it could help you with writing your story. IR: Yep, yep. IE: Some journalists are better in finding stuff out there than they are in writing so, a robot could help. IR: Yeah, indeed, indeed. One example I can mention in Los Angeles in 2014 I think, there was an earthquake. Within 4 minutes they had reported the story, because they had was is called a ‘Quakebot’, so basically it detected the seismic data of the earthquake, produced a text of it automatically and published it online, all within 4 minutes. So you can see how, yeah different sources of data can be used to, you know produce a text. IE: Yep. IR: So, it’s really interesting to see. IE: Why not? Why not? Yep, yep. IR: I have heard from other participants that for more, richer stories and opinion pieces it may not be quite there yet, do you agree with that? IE: Well of course yes, but you have to use your brain to come to the right questions, to come… everything starts with the right (amazement), when you are amazed about something, or you are struck by something and you think how, how could this be? I don’t see a robot doing that, so as long as there is a robot to be programmed to think like we can, we still have to use real journalists. IR: Yep. IE: But you know, traffic news, earthquakes, storms, you can use robots, why not? IR: Yeah I see what you mean. IE: Yep. IR: And looking more at sourcing routines. For you and maybe the editorial team in general, what is your sourcing routine, how do you discover content, how do you find something that becomes a new story? IE: For us as a regional paper, the main ((>)), the biggest sources are our network. So people you know, people you speak to, people we see. The region we write about, we live in, so I go the store, I go to the library, I go to the ((<)) swimming pool, I go to the sports club. I see and find and hear and speak people. IR: Yep, yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 154
IE: For us, it is important to have a lot of people in the region, who speak to people, like a virus you know? We have like 80 journalists, and we have 100 what we call correspondents, and they are eyes and ears in the region. IR: I see. IE: So, that’s the main source. IR: Do you search online for stories at all, or? IE: Well, it can start with trying to find stories online. Although, social media, its different now. IR: Yes. IE: Search engines and search methods, they can see what is happening, where online. So, we have like a {city}, {city} in the search. We can see “oh there’s a lot of traffic going on about {city airport}, so sometimes that, is the main source, but most of the times you just start by hearing something. The old institutions are important for us as well, like municipalities, ((<)) {city} airport, or the bigger companies like Phillips and ASML. But they put out news, and they report on this as well. IR: Interesting. IE: That is where a story can start as well. IR: Yep, yep, and so do you use any tools to keep track of this? Like I know for Twitter you can use Tweedeck or there’s that other, like a monitoring tool to keep track of everything. IE: Yeah. IR: Yep. IE: Yeah, we use them as much as possible. IR: Yep, yep, interesting. IE: So we have a digital desk, and they keep track of everything that is going on in the region, and then when something stands out, they warn the news editors, and they, you know? IR: I see, yep. So would you say then that {current company} is quite data driven? IE: No. IR: Ok. IE: Maybe not enough. IR: Ok. That’s interesting, so is that because a lot of the stories come from your networks rather than online sources? IE: ((<)) Err, yes. I think, yeah, and we are still a bit driven by the old institutions as I told you. IR: Mmmhmm, mmmhmm. IE: I think we are not enough data driven, we could do more there. IR: Ok, in terms of like measuring how successful stories are in terms of reach and pageviews and engagement, is that something you do? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 155
IE: Yeah, of course, constantly and during the day, so we can adjust headlines, or we can adjust the picture, or the story. Or we put it on a different position on the site, and try and find out, or we can post it on Facebook, depending on what kind of story it is. So, stories with people in it, then we publish on Facebook, stories with numbers, it’s for the site, you know? IR: Ah I see, yep. IE: What kind of stories will you publish where, we are still trying to find out what works best, but we, yep, yep. We measure everything, constantly, what do you read, and how long, and when did they, if they read this what do they read as well? Did they take a digital subscription, what were the 10 stories they read before the subscription? IR: Oh wow, yep. IE: When they read, do they have a subscription? [So yeah, because = IR: It all helps. IE: The more we know, we more we can do.] IR: Exactly, yep very true. All right, perfect, I am now going to share my screen, I will move onto the demo part. ((IR shares screen with IE, displaying TCE demo)) IR: Can you see a dashboard? IE: Yes, I can. IR: Perfect. IE: Corona, Sports, Brabant ((Listing topic feeds currently shown on screen)) IR: Yep. So this is The Content Exchange, at this point at the moment it is in its development phase, so it is kind of changing all the time. The goal of The Content Exchange is to be a marketplace for content, its going to be multimedia. So, you can actually see here you can filter by images, text, audio and video. The most obvious example that pops into my head is for example a freelancer can offer their article on this platform, it can then be bought and reused by any editorial team or publisher to be reused in their own outlets. And you can see you have these feeds here with the columns, and with these stories. For example I think I can type Eindhoven ((IR types ‘Eindhoven’ into the search bar, generating a feed next to the currently displayed ones)), press enter, and then it comes up here like this, and then you can scroll down and see the stories. IR: And so, you click on a story here, you can view it, I will get a text article up. Yeah, so you can either buy some of the text, or just part of the text, or just the images depending on what you want, so what you are looking for. And you can also search within the feeds here. I will now move onto some images that show you a more detailed view of what it might look like in the future. ((IR Switches from demo to mock-up images)) IR: So you can see here, a more feeds, on the left here, you can get personalised recommendations. So the goal is to kind to have personalised recommendations based on, which maybe to do with your usage data, and similar users to what their doing, to kind of try and bring you topics you may have not considered yet. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 156
IE: Yeah. IR: So, moving onto other images. That’s another example of the story. You can also have shared feeds, you can create custom feeds and share them, for example if you are in an editorial team, you can create your own feed that has the most relevant news for you, then you can share it within your team so they can also view it. Going back, that is the login screen. You have profile like this. And then so, within a topic you can actually filter it, for example if you are into sports, you can select the particular sport you are interested in. Then next, yep this is kind of a similar interface here again the profiles, and your feeds. And then here is a more detailed view of the filters, also by country and source. And then here again is, you have your own recommended feed here within your dashboard, so I will just show you this one last time so you get a good look. ((IR switches briefly back to TCE dashboard)) So you have all the columns here, scroll down, and then yep early days, so I guess, well it may launch in June, but we will see how it goes. But yeah, so far so good. So that is The Content Exchange so far, I will stop sharing my screen and come back. ((IR stops sharing screen with IE)) IR: So I will just check the description. IE: So, who will upload stories there, just everyone? If I was a freelance journalists, could I just upload? IR: Yep, in theory everyone can upload a story. IE: You don’t need to be a member, or? IR: No, in theory you just sign up for an account and anyone can offer content on the platform. So its kind of open for everyone, with a journalistic background and content to offer. IE: Ok. IR: So what is your overall impression of this Content Exchange, so far? I know its very brief. IE: I have a couple of questions still. IR: Yes, go ahead. IE: If I am a journalist I would upload there, but would I, and then depending on how much it would sell, I would get my money? Or? IR: Yes, yes exactly. So people can buy content, I think it also as a feature, it is performance based, so the better it goes, the more money you get. And so, what’s trending and things like that. So yeah you get paid for the content and how many times it gets reused, yep. IE: And supposedly I would have a company like, lets keep it simple, PSV. I would get someone to post a lot of positive content on the platform. IR: Mmm, I see where you are going. IE: I would pay, as PSV for a journalist who would post on the platform and I would be on the platform and think “oh this is a nice story”. How do I know where it comes from? IR: The source is very important, a few people have already said this to me, so that is something else they have to tackle. I think, maybe there is something involved with the assignment process, yeah I only know so much. I feel like the guy I am contact with, he is you know starting this company, he will probably know a lot more about this. I know there are lots of issues, such AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 157 as like, the best way to sell content, the legal issue with copyright and things like that, is the Dutch market alone to sustain their success. So, there are many things they have to figure out in terms of that, and the source of where the stories are coming from. IE: Of course the Dutch market is big enough. IR: Yeah? IE: Because we have like, 20 newspapers, so why not you know? IR: Ok, that’s a good perspective yeah. IE: That… of course it is big enough, but the problem is, how do I know if I want to buy from you? Is it legal, you know? Is it checked? From who, who gave it to you? IR: Exactly yep, the legal issues in terms of accountability and copyright are something they are tackling at the moment. At the moment yeah, it’s early days at the moment, so they are going to have to that into account, definitely. IE: Because, just I don’t know if you know how it works, but I am in the end, I am responsible for everything that comes in the newspaper, and on the site. Because the legal issues are growing and growing and growing, so a bigger problem everyday. So, I am a bit concerned on that part, always, always. That is my biggest question always: where does it come from? Where are your sources, you know? IR: Yes, yeah they haven’t mentioned how that will work. Maybe, there will be something incorporated into the profiles, where you view the profile of the person who wrote the story, that is just off the top of my head something they might do. IE: Ok, ok. IR: So, yep its developing. IE: The way it looks, that will be fantastic, yeah of course. It would be something, we could run on all day and just check, is there anything relevant for us that we can get from the platform. IR: Do you feel like, in terms of design and layout, it looks quite good? IE: Yeah, very useful and very simple. IR: Like, simple to use? IE: Yeah. IR: And the columns make sense? IE: Yes, very much so. IR: Interesting, ok great. IE: So, it is very easy to understand how it works, right off the bat, so that’s great. IR: Great, great, and so if one of these articles for example were automatically written by a robot, would you still consider sourcing it? IE: Yes, if I could know, so, yep. I am not against robot-journalism, I have to know. IR: Great, so knowing is a key part. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 158
IE: Yep. IR: And so, looking back to the routines you described earlier, do you feel like a dashboard like this could integrate into what you currently do? IE: Yes. IR: Yeah? IE: Yeah, why not? Yes, it could be one of the sources for stories for us, why not? IR: Mmmhmm, mmmhmm, great. Were there any particular features that stood out to you? IE: The simplicity. IR: Yeah? Great. Well so far so good, perfect, perfect, and are there any other general comments about it all? IE: No, well I would have to use it. IR: Yes indeed. IE: I would give you better comments I think. IR: Ok great, well that’s kind of the main questioning. So I will wrap up now and ask you one final question. In terms of everything we have talked about, in terms of technology, journalism, artificial intelligence and automated journalism, where do you see that going in the future? IE: I think a lot of the things we do can be automated, like in the long run. Everything that has to do with the weather, a lot of people want to know about the weather, to know about traffic, want to know about closures of roads, a lot of people want to know about agendas, what can I do this weekend, where can I go? That can be automated, like the real deep stories and the news that can help you with important decisions in your life, that has to be made by people, because it all starts with the good questions. IR: Yes, yep. IE: So, and you have to have some intelligence there, so maybe in the long run robots can do that for us as well, but I don’t see that happening in the near future. IR: so you don’t think eventually, the robots won’t take over everyone’s jobs? IE: No, not everything. I think, I don’t know. I don’t know if it would be a bad thing, you know? IR: Yeah that’s a valid point. IE: Can I trust a robot? But you know, can I trust a journalist? That’s the same question, not always, so… IR: Yeah, that’s a good point. IE: Yeah. IR: There was actually… IE: If we could all go out fishing in the park everyday and the robots would do our jobs, I would very much be in favour for that. IR: Life wouldn’t be so bad then, yeah. ((Both IR and IE laugh)) AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 159
IR: Yeah, so yeah that’s a good closing statement. Thanks a lot for your time, I will stop the recording now.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 160
Appendix M Interview Transcript L
Interviewer (IR): Sebastian Short Interviewee (IE): Participant L Date: 07/05/20 Time: 13:00 Setting: Video call with living room as a background
IR: Ok firstly, could you tell your current occupation? IE: I am a teacher at {current organisation}, I worked for {previous company} for 12 years. IR: Oh yeah? Interesting. IE: I left there in 2013, and then I got caught up in innovation and entrepreneurship within journalism, within {current organisation}. Well, that’s my current occupation actually, as a teacher, lecturer. IR: I spoke to {name}, so your also like a tutor in innovation and things like that? IE: {name}? IR: Yep, yep. IE: Yeah, she’s my colleague right? Yep. IR: So, you do the same kind of thing as {name}? IE: Exactly, {name} is more technical, I am more strategic. IR: Ah I see, ok. Interesting. So how many years have you been working within journalism? So you were at {previous company} for 12 years? IE: Yeah. IR: Was there anything before that, or? IE: No, it was my second year from school and I started there. IR: Ah no problem. IE: And I stayed for 12 years. IR: Yeah, nice, nice. All right… IE: I started teaching in 2011, I did both, {previous company} and {current company} and then in 2013 I got this great opportunity to develop entrepreneurship and innovation within {current company}, so I thought, well, that’s interesting. IR: Yeah, yeah, very cool. So within those 12 plus years, how do you feel that technological change has impacted journalistic work routines? IE: Again, you just faded away a bit, sorry. IR: Oh, I will say that again. How do you feel that within those 12 years and up to now, how technological changes has impacted journalistic work routines? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 161
IE: Well at my time at {previous company}, and even afterwards looking around, you still see journalism doing all the things they did, journalists doing the things they did. Making their own phone calls, writing text, finding their own pictures, and now putting it online, where it used to be a newspaper for instance. ((<)) There were only in the editorial rooms, where you could use new kind of tools to make everything more efficient right? IR: Yeah, yeah, yep. IE: But the handywork didn’t change, I had to pick up the phone and get in touch with sources I wanted to speak to. IR: Yep. IE: And that’s still, still a thing. Of course there are many more opportunities online, the techniques, what I saw was all these new tools developing, which weren’t actually for journalism. IE: [They were usually for marketing, or advertising, or commercial, and you saw for journalism used those tools like, animation videos and = IR: Oh, yeah, yeah. IE: = those things, trying to embed them in their stories, timelines, you know all those things?] IR: Yeah, I see what you mean. IE: Yeah, that was starting around I think 2011, 12, 13. So that was the main leap I think, and was hesitant, because there weren’t many people on the floor, who knew how to use it, right? IR: Yeah exactly. IE: And even the students we had, we weren’t teaching them that. So, well you saw after 2013/14 you saw students starting to learn this, and starting to use it in the workplace. IR: Starting to catch up a bit with the technology. IE: Exactly. IR: So, what kind of role did you have within {previous company}? IE: I worked for radio, I started out on radio education at {educational institution}, so I came there for {radio stations}, where I did, I developed some formats and, for {radio station} I did mainly writing for the hosts. And in (due) time we started {website}, we already had a site, but NU.nl has already becoming big, it became our main competitor, so we wanted to do something like that. So we started growing, we had our own site, we had it well developed and I worked for that a lot in writing. IR: Ah ok interesting. IE: We did try to for instance, making everything more efficient, you had these RSS feeds right? IR: Yeah, yeah. IE: And nobody was following them, and in the meantime organisations were putting their recent, their latest news on RSS. So we thought can we build something to bring them together in one field, and pick up everything. So that’s one thing I developed with the back office of {previous company}. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 162
IR: Oh interesting, ok. IE: Yep. IR: Do you feel like there was a good balance between journalistic quality and financial performance? IE: Well at {current company} we didn’t have to worry about finances, because we are public, right? IR: Oh yeah of course. IE: We had a budget, and of course there were periods where you know the government would cut back, but we remain big enough right? We keep doing what we wanted. IR: But you never, you felt you had enough to write what you wanted to write? IE: We had all the time in the world. IR: Ah ok interesting, yeah. IE: No but it was, it wasn’t that much pressure, of course you had to keep the site running, and you had to keep the news on the radio running. IR: Yep, yep, yep. IE: But we all accepted that there were moments where you had to work very hard, and all those moments where everyone was sitting back and saying nothing is happening, right? IR: Yep. IE: It was off and on. IR: Yeah, I see, I see. Yep. And turning more now to more recent technological change, I am assuming you know about, well it has many names, but automated journalism, and robot- journalism, that kind of thing? IE: Yep. IR: What is your overall impression on that? IE: Its going to happen. IR: Yeah? ((IR and IE both chuckle)) IR: I have heard that a lot yep. IE: It’s going to happen, I mean and the good thing is that we are trying to find out how it works now, but what’s {colleague} is trying right? IR: Yep, yep. IE: Trying to get a second project for this, to find out what works, and what doesn’t and also very important to take the work field with us. IR: Yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 163
IE: Because I know the working field has a place where mainly the focus is on the now, we got to fill this site, paper has to be filled. So you talk innovation within media companies, its complicated, difficult, and if you look at innovation, for instance {previous company}, after I left they started {previous company name} lab. So they started using tools online for their news product, and that’s it. But something as disruptive as the automated newsroom, robotics, that’s difficult to grasp for them, so I am happy with what {colleague} is doing, and slowly I think we are seeing what field can adapt to that, but its slowly, like always in journalism. IR: Yeah, yeah that’s true, and so do you feel like at the moment, I know because I have spoken to a lot of people and they work for {companies} and big companies like that, at the moment it is quite experimental, would you say the same? IE: It is, yep. It is because if you talk to {large media company}, maybe they have mentioned this, but {previous company} has a different story. But let me take {large media company} right? I have spoken with {big media company} a lot about innovation, we have tried a lot of projects with them, you sit there and in the first conversation there is always after 3 minutes “Ok how is this going to pay for itself”, right? IR: Yeah, I see. IE: That is something that is right there on the table immediately and that is always what they are trying to find. I am going to do it, but when I can make money with it, and if it is going to cost more than it pays, then I am not going to do it at the moment commercially. That is the choice that is always in the back of their minds, so they are willing to try, but they are not willing to invest a lot. IR: I see, yeah. IE: So they will wait it out, you know? And they will see, ok is this really going to fly? Is this going to work? How is going to work? And at the end of the day, am I going to make more money with it? IR: Yeah, yeah, like does it have a profit margin? IE: Exactly. IR: Yeah, yeah. So what kind of projects were tried out then? Just testing information or something? IE: At the moment we are running what is called a (brunes Brabant), its ending in two months. But it has been an accelerator, it started at {current company}, where one is doing exactly the same the company you mentioned. IR: Oh yeah. IE: they are trying right? They are not…We have seen about 10 start-ups developing concepts together with {large media company}. IR: Oh, ok. IE: This has been like, bringing efficiency to the work floor and like, a news chain. Maybe you have heard of it? But also, building new products, new news products, such as WhatsApp news, audio… They have tried it themselves, {large media company} but they failed hopelessly. IR: Oh. IE: But again, am I making money on it? Or not? No, lets skip it. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 164
IR: Yeah that is it right? IE: So I see start-ups who have tried to do all kinds of products and projects, and develop a concept. Gamification of news, but at the end of the day it is always difficult to get a structural contract with persons, right? So like we’ll buy one, we’ll buy two, and that’s it and we will see how it goes. Oh it has 1000 clicks? Ok I am interested maybe, you know? IR: Yeah, yeah, yeah. IE: That’s difficult, that’s difficult, but I would say to the start-ups, that is something you have to cope with. You have to convince them you are going to make money with it, otherwise do not bother. You might have a beautiful new product, which you believe in. You have to convince them. IR: Yep, yep. IE: And, we have done about 10, 12 of these projects. I helped them with one project they did themselves, they did a follow up on the WhatsApp news, and like I told you they failed hopelessly, because if you see what they tried was, to take one of the editors out of the newsroom, an editor who didn’t feel the room to innovate because you always knew this has to get us money. IR: Yeah, yeah, yeah. IE: And so, it didn’t quite work. IR: Yeah, I see. Its trail and error at the moment. IE: Yeah but a lot of errors and not learning from the errors. IR: That’s it, yeah that’s it. IE: [I don’t mind making errors, I mean that is all over the place. But if you don’t learn from them, then that is = IR: Yeah then you don’t move forward. IE = not innovation.] Its stupid. IR: Yeah, but if I turn to more now like sourcing routines, maybe this question is maybe referring to {previous company} or just editorial rooms in general. How do they or you go about finding new stories? What is their starting point and where to they turn to, to get a new story? IE: Have you talked to, you have talked to a lot of journalists? IR: Yeah. IE: Or have you talked to management? IR: I have talked to a lot of Editor-in-Chiefs, a lot of founders of the companies they are in, but they were previously journalist, so yeah, it’s a mixed bag. IE: Because if I look at {previous company} it was ANP, which was hugely important. IR: Ah, yep, yep. IE: But they did all the work for us, lets put it that way. IR: Great. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 165
IE: But a lot of the work for us, right? IR: Yep, yep. IE: But then you had your own network, sources you know, sources that bel ((meaning via phone)) you stuff, during that time we working there, of course social media become a thing. IR: Yep, yep. IE: A source of news, and we of course like, RSS was a sort of news source. It was out there but we didn’t put it together, at the end we did. So, that’s actually we way we get new stories. IR: And then do you feel like those routines were as efficient as they could be? Or did you feel that they were effective? IE: Well it kept the site and the radio going, right? IR: Yeah, yeah, yeah. IE: There is enough to bring, but are those the right stories? And that’s a different question, because, and that’s a question that every newsroom asks for themselves. If you look at the correspondent, they have a totally different view on news, right? IR: Yeah, indeed. IE: Are they bringing the right stories? Or does {previous company website} bring the right stories? It depends, I have a personal view on that, I mean, but so you have this ‘snack news’ on NU.nl, a bit on {previous company website], but do they invest a lot of time in investing in journalism? Very little. Is that the way to go? It feels right, but is it something that {previous company} thinks is important? IR: Yep. IE: So that’s a thing, that’s a different question. IR: It is yep. IE: If you look at your audience, right? What do they need, what do they want? IR: Yes, exactly. IE: And, if then you look at the numbers. NU.nl, {previous company} they are doing great. They have excellent ratings. IR: Indeed yep. IE: So then you could say, they are doing great! They are doing the right thing, they are making the right stories. Does the discussion end there? I don’t think so. IR: ((Chuckles)) Yeah. I see what you mean. But do you feel like now there is quite a reliance on, because I know a lot of sourcing nowadays is done online, do you feel like there is a reliance on that? Or does it again depend on the outlet or audience? IE: Yeah, the last one, it depends on, yep. You can maybe put it in a negative way, right? IR: Yep. IE: You can do your online research and your audience can be, ((<)) are satisfied with that, so you can get away with it. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 166
IR: Yeah I see, yep. IE: So you don’t have to the extra mile, to get the better story, because they are ok. They find you anyway, and they click at stories anyway. So we’re good, money is rolling in, advertisement is going great. IR: Yeah, pushing your luck, yeah. IE: Right? IR: Yeah I see yeah. IE: So, it’s a choice to go the extra mile, maybe like the (how the spending is) doing, so also a different way for paying for journalism, right? Because you are a member, and it gives them the opportunity to give them the time to build a story. IR: Yep, yep. IE: But is it better than {previous company website}, that’s anyone’s opinion, right? IR: Yeah, exactly. Yeah it remains to be seen, yeah that’s a good point. IR: Great, so I think I will now share my screen. IE: Ok. IR: And I will show you this, new start-up dashboard. ((IR shares screen with IE, displaying TCE dashboard)) IR: Ok, so this is the main dashboard for The Content Exchange, as far as I know, it essentially aims to be a marketplace for content, the most obvious case I can think of in my mind is for example a freelancer can offer their content on their content on this platform and it can be reused by other outlets, giving them an audience. So it is connecting journalists, publishers and editorial rooms together, to try and provide a marketplace for content. IR: So this is the main dashboard. There are feeds here, and you scroll down them like this. So, for example I type in Eindhoven because that is one of the ones that works at the moment ((IR types in ‘Eindhoven’ into the search box to display a new feed on the dashboard)). Press enter, and it comes up along the side here, you can scroll down. And you can view the article like this, you can view the text. You can either purchase either the whole text or just a part of the text. For example, if you want a quote. It is also going to be multimedia, of course it is still in early days of development, but it will also eventually include audio and video. But at the moment you can just see the text stories, things like that. So here is the main dashboard, within here you can filter by media type. You can search within this feed, and you can also create filters. I will switch over to the pictures here because they give a bit more of a detailed view. ((IR switches from the demo to the mock-up images)) IR: So this is what they are aiming to make it look like, so along the left here you have a personalised feed. You have personalised recommendations based on either, well to be confirmed but based on your usage data, or similar users usage data to suggest you topics that you may not have considered yet. So I know that the filter bubble is quite a problem within the news, go I guess it is trying to tackle that at the moment. ((IR switches through images while talking)). So, that is another news item. You can also create shared feeds, so the goal is you can create a custom feed with the particular aspects of a topic you want. You can share it within an editorial team, for example if you created this feed and its relevant to your team, you can share AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 167 it with others to save yourself time. And going back, there is a login screen like this. You have your own profile within it. And then here you can see for example, if you want a specific sport, you can then filter within the feed for the sport that you want, so you have more relevant news. That is more examples of the dashboard and here is a more detailed view of the filters. You can also do it by country, if I keep going, all very similar. Yeah you can create custom feeds again, and then here again you have the overview of the personalised recommendations. IR: Yeah so, I will go back to this quickly ((IR goes back to displaying the demo)), so it is in the early stages of development at the moment, but yeah here it is. It is quite a basic dashboard at the moment, but I think it will launch sometime in June depending on development and testing, things like that. So is there anything else I need to tell you? ((IR checks interview guide)) Yeah, so essentially it acts like a marketplace for content, so I will come back to you. ((IR stops sharing screen with IE)) IR: So that is The Content Exchange, so far. So, far I know it is a very brief glance, but what is your overall impression on a dashboard like this? IE: One thing for sure, all these news companies are always looking for new content. IE: So if you give it on a golden platter, and make the price right then, there is an option. IR: Yes, indeed. IE: And I know this because we have had several conversations with several media outlets about this in the past. With also start-ups trying to do the same thing. IR: I see, yep. IE: So, I haven’t seen an example that has been working so far. IR: Ok. IE: So, there might be an opportunity, I know the main focus is on video. IR: Ok. IE: If you look at {large media company} because they are able to write stuff, right? IR: Yeah, yeah, yeah. IE: They have had 100 years for that, right? IR: Yep, indeed, yep. IE: But they want more video content. IR: Ah, I see yep. IE: And the video content is expensive to make. IR: Ah, yep. IE: I make a phone call for 20 minutes and I have a new story to write, but if I want to make a video for 20 minutes, it is going to cost days. IR: Exactly, yep. IE: So, it is more expensive, so if that is what they want. So if you can deliver, lets go. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 168
IR: ((Chuckles)) Yeah, that’s it right. If you solve a problem, then theoretically anything is possible, yep. IE: Yep. IR: And so, because I have heard, I have spoken to many other people and they say things like: how are you going to make money? Who is this for? How is it going to work legally? Because there is issues with things like copyright. IE: Yep. IR: So, you have said you have seen similar ideas, what went wrong with those other platforms? Or what did they do? IE: Some where trying to tackle the problems you just said. IR: Yep, yep. IE: And, ((<)) so they need a whole lot of help with figuring out copyright, but also the ethics of things, right? IR: Yeah, I see. IE: What am I? Am I in the middle? Am I just something passing through, or do I have a responsibility in what I am passing? And that becomes completed quick. IR: Yep. I see. IE: Because when you talked to your customers, they will just say “Give it to me fast, give it to me cheap”. But I do not want mistakes, if you send my something and I put it online, an hour later I have troubles with copyright, or with privacy violations, we’re done, right? IR: Yeah, then it’s your reputation on the line. IE: So its one to build a platform like this, right? Where you try… a platform works like taking away problems and just giving it, handing it over to me. IR: Yep, yep. IE: Right? So I am working in the newsroom, I am looking for a new story to post, I am looking on this platform of yours. And I say “ok this looks interesting, I am going to copy it, and I am going to post it”, and so that means, and you can do that technically, and that’s not a problem. I think you can build that. IR: Yep. IE: So you are helping a newsroom to post new stories, great. But all the troubles, like privacy and you know, you have to solve, because it is not my problem anymore. What you send me is ok, it is checked by you. IR: Yep, yep, yep. IE: And if you look at that part, that is the difficult part. IR: Yes, I agree, yep. IE: Because I have seen examples where they focused on social media video, right? IR: Yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 169
IE: Which is probably, it’s a choice, it’s probably cheaper because everybody makes social media videos, posts them online. You can talk to people and say “can I take your video?”, I am going to pay you a small fee for it, I am going to use it in a news story, right? IR: Yep, yep, yep. IE: So that could be an interesting one, to get more content in a cheap way. IR: That’s a good point, yep. IE: And then the trouble starts, because who is in the video? Where was it made? Is it really showing what we’re telling its showing? IR: Yes. IE: And me in a newsroom getting this video, I do not want to ask myself these questions. I want to use video, I want an outline with it and, post it. ((IE uses a hand sweeping hand gesture to signal posting the video online)). IR: Yeah, you just want to use it, no questions asked. IE: Exactly, and that’s, so the middle part is hugely…there’s a big responsibility. IR: Yeah, yeah. I definitely… IE: And that makes it complicated. IR: Yes, that is where it gets sticky, yep. Well yeah I would have to see what they do with that. But do you if this dashboard works, considering the editorial routines you described earlier, that it could fit into that? In some shape or form? IE: Sure, if you could make it easy for them. IR: Yep. It would like compliment what they currently have? IE: Exactly. The more they have to choose from, the easier it is from them, because it used to be quite easy, ANP was, is the big deliverer, still if you look at and {previous company}, but also {large media company}, but they have better networks. IR: Yep, yep. IE: Because they have little journalists on the ground right? The newsroom are reproducing what is happening outside, and ANP is a huge source for that. The newspapers have better, are more out there, right? They use their network and people to talk to sources more often. But adding extra content to the sources they already have, is interesting for them. IR: Oh, ok. IE: Because it gives them more options, more to choose from, and maybe even to start new projects because they have new content. IR: Yep. IE: Which sometimes is more interesting for somebody for 18 years, than somebody working for 80 years, right? It can be more diverse, and at the end of the day, what you put online and the traffic you generate, that’s what this is about, right? IR: Yeah. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 170
IE: So for {previous company} it is about being relevant, people come to us because they get the story if they want to know. In the {large media company}, what they put online, they have to generate traffic, to sell advertisement and to earn money. IR: Yep, yep. So you don’t feel people would be overwhelmed by having too many tools? IE: No, I don’t think so. I have never felt it that way. What I saw working at {previous company} I had several screens, I had social media, RSS, ANP, so you just skip, you look through it, and you scan, you pick up what you are interested in. IR: Yeah, I see. IE: You know, you make it the easiest if you have a platform of where I can look, and I can just copy and paste right? IR: Yep. IE: I like the story, I know it’s checked, its printed, lets put it out there, let’s post it and I can move onto the next story. IR: Yep. IE: Or this one needs more work, I have to make some phone calls, that’s going to take me half an hour, which is more expensive. IR: Yep, yep, I see, and if the stories on The Content Exchange, some of them were for example automatically written by robots, do you think people would consider to use them? IE: Well learning that people, the audience, for the audience it is more and more difficult to see which one is written by software and which one is written by a journalist. So, again if you ask the audience when they are satisfied, they are satisfied. If you ask the journalist, it probably has a date, so “well I am important here in the process”, and some of my colleagues are afraid I think, of losing their jobs. IR: Oh really? IE: So, there are multiple players here, right? IR: Yeah, I see, yep, yep. So do you feel like then editors would take a second look if sourcing an article that was automatically written? Or would they embrace it? IE: They’re going to embrace it because its cheaper. IR: Ah ok. Yep, I see, and in terms in looking at the, well it’s a very quick glance, but the user interface and the design and layout, was there anything that stood out to you? Elements of the design? IE: Well, the thing is I have developed a certain platform like this for RSS. IR: Oh you did? IE: I found out, yeah it was this totally new platform. I found out within weeks that there was too many options. IR: Ah ok, yeah, yeah, yeah. IE: So, make it simple, because when you make it simple, it is easy to use and you can browse through it and then move on to your next screen right? That’s how {previous company} uses it AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 171 in the end. So we have a really simple way, when I talked to IT people, they show you everything that is possible, I had like “wooah this is great”, but if you bring it to the floor, they don’t care. They want it to work quick, easy to use because every bit of more difficultly takes more time to learn and that’s the hold up. IR: Yeah, I see. IE: Because I am there to do one thing: put journalists story out there right? IR: Exactly. IE: It is nice it is helping me, but it has to help me. You don’t have to complicate things. IR: So it has to have a clean interface that has to invite people to use it? IE: Yep, make it as simple as possible, and the end of the day, we made something that looked like ANP. Because they knew that, they were familiar with it, they said “oh I can relate to this”. IR: Yeah, that’s it right. IE: And gradually we put some more options in it, but we took them out just as quick when we found out, this is not what they wanted or they are not going to use it. So, it really helps when it looks like something from somebody already knows. IR: Yeah, I have heard that a lot actually. For example the columns on the interface is quite familiar for a lot of people. IE: Yep. IR: Having features like that is just easy to transition onto. So what kind of features then were removed? IE: We had all these options to make your own feed, right? And, but they didn’t know where to start, so they had a long list of themes, of topics and behind the topics there were relevant organisations, that were putting out these RSS feeds, right? IR: Oh yeah, yeah. IE: So, we had maybe, I don’t know maybe 20, 30 topics, right? Then you think ok, which ones are relevant for me? Well, all of them? Because I look at all topics, that’s my job. Or should I focus? And, so that was something we had to guide them with, so at the end of the day I took it over from them and I actually did interviews with them. IR: Oh yeah. IE: So, then you find out that they sometimes they don’t really know what they’re focusing on. So, you have to find out together, how better it could add into the programme and say “ok, this is what I heard, so you are going to get these and these feeds”. IR: Yeah, yep. IE: “Ok, this sounds logical, can I get back to my work now?” Yeah you can, you know? IR: All about, yeah fitting in. IE: So it could work from our side, we did that. We had a lot of trouble, but that was maybe more our side, developing. IR: Yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 172
IE: But I used journalists to think with me, because eventually they have to use it. But, we had a lot of organisations, but organisations don’t uniquely fit one box usually right? So we got into trouble in connecting which one goes in which box, right? IR: Yeah, I see. IE: So, its very complex. It’s the same thing with tags. You start adding tags to stories and then suddenly you say “what a minute this is not adding up, because we are, (using too many tags)”. So you know that is what happened on the backside of the programme. IR: Yep, yep, yep. IE: So we actually, fortunately we were in the position to just throw away the backside, and rebuild it quickly, so that was a problem. We thought about, the journalists adding tags, forget it. We didn’t do that, so we left that out, so all these options to make it more personal at the end the day, we took it out. At the end of the day I had conversations with them, and I made it specific for them, so they didn’t have to change anything with them, anything anymore. IE: And we also gave them the option to put extra feeds in, or add extra feeds, extra organisations, nobody ever used it. So, at the end of the day, we had one person responsible who checked, and just asked around, what organisations are important to you? We did that on a regular basis so we could update the programme, right? IR: Yep, yep, yep, yep. Ok, so simple is key then? IE: Absolutely! IR: Yeah, ok that’s the takeaway. All right, great, well that is covering my main line of questioning, I guess I will sum up by asking: in terms of everything we have talked about, AI, automated journalism and technology in the newsroom. Where do you see that going in the future? IE: It is going to replace part of our work. Yeah, I don’t know how far it is going to go, I am not sure. IR: Yeah, yeah. IE: It’s a difficult one to predict, I am looking at this from a positive point and saying, we can fill {previous company website}, NU.nl by an automated newsroom. IR: Yep. IE: And give journalism time to build new correspondence. IR: Yep. IE: Right? You that is maybe something in 5 to 10 years. After that I am not sure, I think software will be able to write stories like a correspondent is doing right now. IR: Oh really, you think so? IE: That’s something I see developing, I am predicting here right now. IR: Yeah, yeah, yeah. IE: So what does that mean for journalism right? What does kind of role do you have? IR: That’s it, that’s the open question. Yep. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 173
IE: So, for the near future, 5 to 10 years, you will see an automated newsroom, but partly filling NU.nl and {previous company website} and after that… IR: Remains to be seen. IE: Remains to be seen and I hope in the meantime, we start a very good debate about robot- journalism in an automated world. IR: Yes, yep. Yeah, in a world taken over by robots! IR: Exactly. IE: Yeah interesting, yeah. All right perfect, that is all from me, so I will now stop the recording.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 174
Appendix N Interview Guide
Note: all questions that are not numbered are backup questions in case participants did not provide full answers.
Introduction and icebreakers 1. What is your occupation? • If currently not a journalist: What role within journalism did you have previously? 2. What are/were the typical responsibilities of your job role within your company? 3. How long have you been working in journalism for? 4. What news topic(s) do/did you normally write about? The digital newsroom 5. Do you feel that technological changes have impacted your work routine since you started working within journalism? • Sourcing online • Pressure to perform • Metrics to measure performance • Changing business models Do you feel like the moving of news online has put pressure on your job? • In terms of sourcing content. 6. In recent years, do you feel that your organisation is balancing journalistic content quality and financial performance effectively? • Cultural power vs economic power • Economic versus commercial interests • Actions towards sourcing content? • Establishing new norms between editorial and commercial departments? • Churnalism
Automated journalism and AI 7. Are you aware of what “automated journalism” is? • If YES = is it used in your workplace at all? o Have you sourced articles that were automatically written? • If NO = explain it briefly to them o News articles that are automatically written by algorithms. It makes use of Natural Language Generation (NLG), which refers to automatic production of text computationally from data. o Examples: AP for financial data reporting, and the LA Times Quakebot. 8. What is your opinion on articles that are automatically written? AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 175
• Positive/Negative? • Efficient method? • Risk of job loss? • Loss of quality? • Improvement or hinderance to workflow? Do you think that there are any benefits or consequences of sourcing automatically generated news articles? 9. Would you source an article from a newswire or other platform that had been written by an algorithm? • Depends on type of article/topic? Do you feel that the use of Artificial Intelligence is beneficial for journalistic practice, why or why not? • Algorithmic content automation • Data-driven news • Automated journalism
Discovery and sourcing of content online/personalisation of content 10. When looking to source a new story, what is your routine to look for new information? • Where is the starting point? • Online? • Offline? • Sources? o ANP/newswires? o Firsthand data collection? o Social media? o Depends on topic? • Structured or unstructured?
11. How easy or difficult do you find your current content sourcing process? What factors are there for someone to turn to sourcing content online? (refer to previous answers describing workplace) • Time pressures • Convenience • Most efficient way to do so?
SHOW MOCKUP/REFER TO FEEDBACK SESSIONS – Describe TCE if needed The Content Exchange (TCE) is a new Dutch start-up that aims to develop a platform where media companies, journalists and editors can share content with ease. A content writer can offer text on the platform that can be reused by other outlets for a fee. TCE plans to at some point incorporate automatically written content into their platform. The goal is to produce a marketplace for content that can be circulated more cost-efficiently. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 176
12. What is your overall impression of this system? (Can refer back to answers on their described sourcing routine) • Does it improve/hinder your current routines • Would it integrate with described issues (or lack thereof) 13. If you were to use TCE, are there any elements of its design you feel would benefit your search? • Personalised recommendations dashboard o Describe what they are • Tagging • Selection via topics
Finishing up 14. Considering everything we have talked about, sourcing routines, technology, AI and automated journalism, where do you see it going in the future?
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 177
Appendix O Informed Consent Form
Informed consent form for interview data collection Thank you for your participation and cooperation for this study. The aim of this study is to investigate editorial workflows in terms of how newsworthy content is discovered. If you are happy to participate then please complete and sign the form below, please type an “X” in the box to confirm that you agree with all the statements below. If you do not agree to participating in this study, please leave this form blank and inform the researcher and your participation will be withdrawn.
• I confirm that I have read and understood the information sheet dated [xx/xx/2020] and have had the opportunity to ask questions. • I understand that my participation is voluntary and that there are no negative consequences if I want to opt out during the interview. • I understand that I am free to decline to answer any of questions in this interview. • I understand that my answers will be kept confidential. Moreover, my name or any other personal information will not be linked in the results section of the thesis. • I agree to have an audio tape recording during this interview. I understand that this recording will only be used for this paper, which I will not be personally identified with. • I also understand that no one else aside from the researcher and his supervisors, Dr. E.H Krahmer and Dr. C.W.J van Miltenberg will have access to the original recording. • I agree that my anonymised data will be stored for approximately 5 months until the thesis has been completed and a final grade has been received. I agree to take part in this interview and have read the above statements: Name of the participant: Date:
Signature: (Digital signature feature removed)
Interviewer name: Sebastian Short Date: xx/xx/20 Signature:
______If you still have questions after the interview or you are interest to have the paper when it is published, you can send an email to ]
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 178
Appendix P Screenshot of TCE demo and mock-up images shown to participants
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 179
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 180
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 181
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 182
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 183
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 184
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 185
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 186
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 187
Appendix Q Open coding
Question Quotes
How do you feel that Journalism hasn’t adopted a fraction of what could be useful to technological change has journalism. If you look at the field of Artificial Intelligence and of impacted your work machine learning, and what {name} is working on, robot-journalism. routine? But also, VR and AR. Those are technologies that can be way more useful for journalism and for the audience of journalism that can be used right now.
So, there is a long way to go, actually for my inaugural speech five years ago, I said that there were two positive things in journalism at this moment, one is society, society needs journalism, that we perform it in the right way. The other one is technology, there is so many opportunities there for us journalists. Especially in legacy publishing houses. ((>)) Yeah, much more can be done.
[The next steps is that more and more will be done by the computer, starting by using very simple and short parts of text concerning for instance simple football matches = that are not enough for the big story.]
Basically the journalism is mainly the same, its only the techniques which are developed. If you are looking at journalism, if you look at for instance (delegate) methods, or a lot of money methods. Yeah which are, mainly digital methods which they using. That developed journalism, but journalism itself does not change that much, because its all the same kind of questions which are, their underlying values, which remain.
Things that have been changing is that when I started, a lot of services were free, so I think for instance all tools to scrape information and other information resources, tools to visualise information, tools for publishing. And right now, I believe almost everything is paid, or it is provided by companies like Google and you are probably familiar with, there are a lot of sort of challenges with working with these companies, such as how do they use your information, what other models they have to make money and all sorts of other stuff. So I think that is one of the major changes that I have to deal with, almost on a daily basis at this point.
So (when I started), I just made an article for a platform, and they published it, and that was it. But now I have to use all sorts of third parties to find information, to deal with my audience, so I think that is one of the major changes,
And 5 years, because of the capabilities and possibilities appearing, AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 188
and I think both the market developed, so if you look at the B2B market, professionally written content by journalists is a developed market and it wasn’t 20 or 25 years ago. At the same time, the consumer market eroded and declined because less and less consumers are paying for content. So content is, basically paid for by advertisement.
Yes of course, because you know like 15 or 20 years ago everybody said the newspaper would be dead around now. But we are still printing them, so that’s one concern, we have to keep it alive for the next 20 years. And at the same time we have to invent the newsroom, to make all the digital content. So some of it is the same, in paper. But under the digital, the public has different demands, like video
Well at my time at {previous company}, and even afterwards looking around, you still see journalism doing all the things they did, journalists doing the things they did. Making their own phone calls, writing text, finding their own pictures, and now putting it online, where it used to be a newspaper for instance. ((<)) There were only in the editorial rooms, where you could use new kind of tools to make everything more efficient right?
It is, yep. It is because if you talk to {large media company}, maybe they have mentioned this, but {previous company} has a different story. But let me take {large media company} right? I have spoken with {big media company} a lot about innovation, we have tried a lot of projects with them, you sit there and in the first conversation there is always after 3 minutes “Ok how is this going to pay for itself”, right?
That is something that is right there on the table immediately and that is always what they are trying to find. I am going to do it, but when I can make money with it, and if it is going to cost more than it pays, then I am not going to do it at the moment commercially. That is the choice that is always in the back of their minds, so they are willing to try, but they are not willing to invest a lot.
So I see start-ups who have tried to do all kinds of products and projects, and develop a concept. Gamification of news, but at the end of the day it is always difficult to get a structural contract with persons, right? So like we’ll buy one, we’ll buy two, and that’s it and we will see how it goes. Oh it has 1000 clicks? Ok I am interested maybe, you know?
That’s difficult, that’s difficult, but I would say to the start-ups, that is something you have to cope with. You have to convince them you are going to make money with it, otherwise do not bother. You might have a beautiful new product, which you believe in. You have to convince them.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 189
And, we have done about 10, 12 of these projects. I helped them with one project they did themselves, they did a follow up on the WhatsApp news, and like I told you they failed hopelessly, because if you see what they tried was, to take one of the editors out of the newsroom, an editor who didn’t feel the room to innovate because you always knew this has to get us money.
But online, the plan was in the first year was to also publish a weekly newspaper. And we published around 20 editions. But it was way too expensive and way too complex for us to perform that.
In recent years, do you The pressure is always on how to make money with what you are feel that your organisation doing, we in a previous year we tried to turn this into a subscription is balancing journalistic model. We didn’t succeed, after a year we needed around 10,000 content quality and subscribers to keep going and we had 200, so quite the difference. financial performance We almost stopped the business altogether and that was then at that effectively? moment new opportunities arrived. The model that emerged at the time, was still our most important model and that is companies but more like institutes and innovation clusters et certera ((<)), pay us to write about their business.
Well, the thing of course is I am only able to pay my reporters a certain amount. I am not able to pay them for doing research for like a week or two weeks and then come up with a story. That is the limit, but it is also the limit we accepted to have. Of course I would prefer to have all of my reporters go away for two weeks and then come back with the best story ever.
But it is not feasible, but they have to write like, well one or two stories every day and ((<)) I can’t afford to give them more time.
Yeah sure you may ask. For a long time, their wasn’t. For a long time the only thing that counted online was the reach. How many browsers do you have on your site? But things are changing rapidly, its about one year. Many newspapers and {current company} are having this paywall. Three years ago people thought no one will ever be willing to pay for something online, but then came Spotify, then came Netflix, people were used to paying for online content. About half a year ago, we really started to focus on making stories people want to pay for. And no longer all those accidents and fires, you know journalistic small things that are less important. Because this is not the content people pay for. They want to read it, but not pay it. You can then be a member, what then people pay for is quality journalism.
For us that is a good thing, because quality journalism is what we get in the newspaper. So these things go in parallel together now, so we are really focusing on making good things. So this year I think we will have some few thousand digital subscribers.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 190
That is really something, I told you I am 25 years in journalism, these 25 years our circulation was declining, declining, declining, at times 5 percent a year. But now, for the first time we have more subscribers than a year ago. So we are growing and of course digital subscribers pay less than print subscribers, it’s a group that’s growing, you don’t have costs from delivering and printing. So for us it’s a good time, its good news.
Not pressure, but it has changed it dramatically, but not pressure. I think maybe it has become easier to do the work we do, because 20 years ago you needed a network to get your news and now there are so many online sources and there are so many data that can be collected to make news of. I think it has made the job easier than 20 to 25 years ago.
Its general news concerning the whole province, so that’s a problem because what’s a province and what’s interesting in the province, that’s. You talked to {another participant} and he is covering just a small portion of the province and I think that is easier to make news, we cover the whole province.
I think that’s true, because we are in very strong competition between the news titles, like the one from {competitor} and that makes it even more important to act very very quickly, and be very correct and have all of your facts and figures in order.
it has been a problem as long as I work as a journalist, which is one the reasons why I am a lecturer, so I (read) a lot of teaching in journalism, but it is also incredibly hard but also impossible to be a full time journalist, and live a comfortable life.
Yeah, so for some of my articles I get, I get about 200 euros per article for online articles at least. For an average piece, say between 600 and 1000 words, and I can write an article in say 1 to 2 days, but I wouldn’t be satisfied with the piece. But since my name is under it, then a week writing this article. In those cases 200 euros is ((<)), yeah you cannot depend on that.
Yeah so for covid-19, this is crazy because you see that, I even saw some content about how golfers are dealing with covid-19. People apparently don’t know what to write about anymore, that hasn’t been written.
Yeah exactly, and for topics like football. Yeah you can’t really come up with 100 creative perspectives on one football match.
There was somebody who won, and that’s it, or yeah you could write something about the trainer, and about the supporters, and about some partners they have. But there are topics I believe that have just some possible perspectives, and then there are topics who have most, AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 191
well almost endless.
You know what you also see, before the corona crisis but what you know see confirmed is that a lot of consumers are willing to take a subscription because they want trustworthy content and (they are willing) pay for it. So they are seeing an increase in subscribers, however, not enough generally to compensate the advertisement market, that almost collapsed.
Yep, and ((>)), well nowadays especially in these times of coronavirus, you see that the advertising is very difficult, because a lot of companies are not advertising at the moment. But subscriptions are doing very well, (we are pleased to have those sort of incomes).
Very much so this time, because of the whole corona crisis, people know how to find us, and we sell more digital subscriptions at the moment. And people, of course they feel that have, they want to read about sports, and games as well, most of the demand is on, what do you call it? More serious journalism.
Yeah. It’s a different public, the question is, can we keep them? Like we can keep the print subscriptions. Print subscribers are people that have had a subscription for like 20 years, so they’re very loyal.
[And we don’t know that about the online subscriptions, yet. So we still have to reinvent how we keep them with us for periods of time.] Are you aware of what There are companies especially in the United States that are fully automated journalism is? occupied with robot-journalism, with building the automated What is your overall newsroom. But if you look at the Netherlands, well there have been opinion on this? some experiments, some ended successfully some others not so much, it is still in an atmosphere of experiment.
It is difficult to separate this feeling from, the search for quality from the automated newsroom. You need to improve and keep improving the machines so to say in order to get somewhere. But if you have a newsroom that is reluctant or resistant then it is difficult to improve your machine.
I do think so, I am convinced that journalism needs to and will step across the traditional boundaries of the journalistic field. AI is one obvious field that we have to cover or conquer or ((<)) however you want to say it.
But there are way more fields, 10 or 15 years ago when it became obvious that a company like Google or Facebook used algorithms in a way that was really affecting journalism, the field of journalism. We kept saying it is not our business, let them do it and let people find out that it is rubbish and let’s stay away from that. I think we ((<)) start feeling an awareness that this was not the most cleverest decision that we made. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 192
Still journalism is a profession that is well, special and different from other professions. Not any non-journalist can feel or be or act like a journalist. You need the professional journalism as well as the technology.
We don’t have the tools yet and I know that we are part of {parent company}, they are trying to make steps on it, and it will be within a years.
Oh yes, sure. You know for some, (publishing the news) is based on data, which is almost every news. There are just a few ways that you can write it down. Either you are a human, or you are a computer. And those sort of passages, its not a place where you make a big difference for your readers, so if it helps me to make better use of the reporters that we have, I will be happy to use it. I don’t think that you should use it to replace your reporters.
Yep. It can help you to make better use of the reporters you have, to have them write things that are more valuable to your readers.
Oh yeah we had that kind of not really into the practical area, we mostly see that kind of experiments.
We had it before yep, so somehow I can say it is kind of a debasement, so people trust these kind of artificially made articles, some people put their trust into people. So its kind of a debasement, so its not really a practical use. We had that, the technology part yes.
I think it was, there were examples, there seems to be also in the financial part, we used a lot of figures and compared to an article written by people I think its mostly contains the same information, yeah its almost the same. The audience just have different opinions.
(That depends on where) the artificial article comes from, a reliable media or a reliable source.
Well, during my past years one of the most bothering work was to discover news, yeah actually there is no real news, because when you see something happening, somebody else has already uploaded something or posted something. So we spend a lot of time online searching for using some keywords like a place or the areas, yeah we spend a lot of time searching online.
((<)) I think that depends on the topic, a simple one something like weather reports or report for some basic figures I think is trustworthy. I think its concerning about a social issue or something that people have different perspectives on I might choose, ((<)) articles written by people. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 193
I think that all parts of the work that could be done with automation that should be done like that. There are of course parts that are expensive, and that some point they can do more than a computer can. They are creative and they can find those subtle human elements. And because we have every year fewer of them, while there is more work to be done, they have work to the automisation. I am very open and positive to it. I think in the world there are now many examples that shows us that it can work.
Well we could discuss it for half an hour, but in short I am a real believer in the principle of robot journalism. ((<)) Why is that? Because the biggest broadcast in Holland NOS has stopped an experiment, saying that it doesn’t bring us anything, its only useful in financial information and sports information, and sports information is being bought by a robot in way that people can see it is robotised news and they said, well if this is collected from certain facts the audience can find the facts themselves and they don’t need a robot to make a good text out of it.
That is exactly what I can mean because I think that the audience has grown up, has become so mature, I don’t think they need a story from the New York Times or whatever. They can find their sources and if they where to find their sources they can find the website where the earthquake is being measured and they don’t need a newspaper to make some really nice sentences about this earthquake.
I don’t really have an opinion about it, its more does the reader like it? That is most important for me, I guess with sports I think well, it might work, a bit. But there is also an added value of the editor, I guess. So, in the beginning it should be checked maybe, to see if the quality is good enough, and after that when all the algorithms work et cetera, I guess it could be a solution, but not the solutions, because it misses the feeling which is also important in an article. Instead of only needing statistics, I guess sports journalism now, for robo- journalism would mainly be statistics.
Maybe on sports journalism, because what we do now we have a very small section of sports journalism, we only follow the main hockey fields, field hockey is quite a big sport in the Netherlands. We follow our top team, and we follow football, FC {regional team}, the top team, and that’s about it. In these articles you need a feeling and what’s happening, so you need the editor, for extra added value. Maybe, in the what is it called? The wijken, do you know that word? De wijk? [The region, the very local = Suburbs, lets say suburbs. Maybe we could use it there, because more football clubs et cetera, people like to read about it also about the scores also, more statistics. So, there’s a purpose for it, yeah and there it might be useful and that is what we would like to give it a try there
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 194
Yep. I read about it, I don’t work with it. NU.nl says they are doing automatic journalism and they automatically re-blog content from people who want to send them a message, that’s what I read, how it works, but I don’t work with it.
And you are talking about robot-journalism on the news floor. ((>)) I don’t know really, I think it can help, but I don’t think it can make the unique content, I would like to read.
Yeah, the who, what, why, how, when. You, know? There’s a fire? There’s a fire, where? How? What? And when? That’s ok, but if you want to add something ((>)), if you want to add some special content then it is not going to work.
Well say factual information, because maybe I am wrong. I said factual information, but when for example a press agency says its doing factual information as well. That is what they do. Can we say ok we don’t use press agencies, we just put robots in the news agency. I think that is how it is going to work, because you still have to do research work.
Well if we just think about just as (narrowing it) as an AI way to employ robot journalism. I think it could be great. If we would have some sort of AI that would scrape social media, for say sentiments with regards to certain topics, great. But I do think again, we do have to be aware of the third parties we would cooperate with because as far as I am concerned, no truly journalistic organisation so far has managed to create a bot that ((<)) is doing a sufficient job.
Well I am not quite sure, but maybe cloud functionalities or sort of the algorithms that you wouldn’t write ourselves. I have seen a lot of times, where we are trying to develop something and either Google or Amazon, or one of these companies would come and say “Oh look, we have already have like 80% of the thing that you are doing, you will take it and build your system on it, and then you will have exactly what you want”.(Then you have that one), but then your data is not secured and sometimes you cannot explain to your users what actually forms the basis of these systems and what’s actually going on. So I think those are my main concerns.
Yeah so I think for these types of small applications, its fine, its great.
So, basically you have Natural Language Generation, and you have the more template-based approach. What you see in the Dutch market, generally the template-based approach, and of course templates can be assisted or generated using robots by getting enough examples from previous archived stories.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 195
But to really generate text via computers in the Dutch language, you do not see that too often. I know it’s being used in the English language. Yep. Our language area is too small for that currently, or of course will develop. But it takes longer, than in English.
So, there are similar things in the Netherlands, but the ones I know are almost all template-based, with some variety with synonyms and things to make it a little more dynamic, but they are not really natural language generated articles.
Yeah, yeah, yeah, and of course the Netherlands has a small market anyway, so if you’re promoting such technologies then it’s more difficult to get a return on investment.
We are for a couple of years, is we automatically crawl content to try to automatically select events out of them, so we have robots that crawl the entire web, the Dutch language part of the internet, and those robots when they detect that its an event, they will collect that and that way we make sure that our events database is partially automatically filled.
Scraping content and trying to find out whether it’s an event or not, because a lot of news is made out of events. You have breaking news of course, expected news, but there is even more expected news. There is a press conference or that there is a football match, well not now but.
But in terms of template-based, yes we are doing some things. For example, I am responsible for the election service, where we deliver the preliminary election results at the day that we are going to vote. And I deliver data, comparing this election to the previous election.
And of course you have for example, we have a financial newswire and at the financial newswire we have a specific rhythm that companies, they publish their quarterly, yearly, or half-yearly results and we use templates for that, so that the editor only has to fill in some really basic stuff. Of course that could be automated in the near future, but we did not do it yet.
I think, of course first you need to achieve, or you need to determine what you need to achieve with the automated content. I think that our job generally as a press agency is to fact check, (to filter by), to check sources and to make sure that what is written by our editors, is 100 percent true. It’s quite difficult currently to do that job via robots. What you do of course is if you have specific data sets, for example that can regionalise the data from the bureau of statistics, the central bureau of statistics in the Netherlands. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 196
They publish each month a lot of statistics, and human assisted we could make something and show something, and then the machine could automatically generate regional angles, yeah? So we do one region manually, and then the machine will take similar data from a different region and then will (join the other region).
[Yeah I think it could be beneficial, because we are working with it in a test setting and especially for the football and following the smaller leagues. But I think they have to do lots of work to get it to function well, and come to a publishable text.]
Yeah, its an experiment, and you need to have the right data, also of course.
That’s a reason why we started with sports, because that is the kind of journalism with a lot data.
So, you know. You got to source, there is a lot of institutional sources, sort of news flowing in, press releases, you name it. So that can be, yeah up to a point automated And I would say where you transcend, this basic level of information ((<)) presentation, so relate to the realms of opinion, or context, you know I would say there is still room for people.
Well I would say that, there is a part and maybe a larger part than we can imagine right now, or we hoped for, I think can actually be written by smart algorithms, or robot journalists, it depends on how you call them. But because in the end it is, a lot of the content put out by news sites, news outlets, it is in a way almost mechanical.
I said say that I think it is, the actual output by robots compared to the output by human journalists is still, well its very small. It may have been my perspective from the way I use content myself.
And I would say where you transcend, this basic level of information ((<)) presentation, so relate to the realms of opinion, or context, you know I would say there is still room for people.
So its very useful I think. So, why not? You know? Yeah if I can use the good journalists to do the good journalistic work, and have everything automated that can be automated, why not? You know? So like, you know sports, ((>)) well the numbers, sports numbers. Like ((>)), the agendas, you can have it automated by robots, why not, so I am not against it, not at all.
Its still a bit vague for me how we can use it, and we are trying to find out where we can use it in our process, and well its still, I am AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 197
hearing about it for years and years and now its still, well at the point where I can use it for this, or I can use it for that. So, I am not sure.
You know for the, what you would call, the chores you could, like in sports or in agenda.
Yep, and of course robots can be useful, but with helping to write your story as well. So not only that it can totally take over your job, but it could help you with writing your story.
It’s going to happen, I mean and the good thing is that we are trying to find out how it works now,
Because I know the working field has a place where mainly the focus is on the now, we got to fill this site, paper has to be filled. So you talk innovation within media companies, its complicated, difficult, and if you look at innovation, for instance {previous company}, after I left they started {previous company name} lab. So they started using tools online for their news product, and that’s it. But something as disruptive as the automated newsroom, robotics, that’s difficult to grasp for them, so I am happy with what {colleague} is doing, and slowly I think we are seeing what field can adapt to that, but its slowly, like always in journalism. When looking to source a That is of course, networks are important in that way it is good, and it new story, what is your is logical but its keeping us from finding original sources or new routine to look for new sources. That is something holding us back from reporting on stories information? that might be worth reporting on.
No, I wouldn’t say that, it could be way more data driven, ((>)). What happens is with your previous question, is that you work from a routine that feels like normal, or the best you can do. So if you don’t know the better routine then stick to your old routine.
Yeah and being curious is of course dependent upon your social network, the way you are raised, the money you make, the clubs you visit. It is very, maybe non-technical.
I think that the well, the reflex that we work on not just at Innovation Origins but any journalist is the his or her own network. Everything we hear within our network is way more important in our heads than things that occur outside of our network. We would, if somebody within our network would ((<)) come up with a suggestion, this would lead to a story way earlier, way sooner and faster than just an email or a message. Or a robot that is suggesting something.
Yeah, of course so ((<)) apart from our you could say ‘physical network’, people you know or relate to or contact, there is always the newswires and different from like 10 or 20 years ago is the way you could say, social media is working as an alert. So we all have our lists in Tweetdeck or Hootsuite and these lists, of course you follow AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 198
hashtags and key words. So they alert us if something new happens. For example {company they have written about in the past} publishes something on social media, we see it immediately.
So it is difficult to imagine a situation where things would be twice as efficient or twice as fast. If you are offered something and somebody can show me a way to be twice as efficient, then yeah that would be different.
The biggest challenge is you can never discover anything new, (real new). That is the biggest problem, and that still we can discover (known by real people), so we try to, or otherwise we can dig into it deeper. Sometimes people post a few sentences and a few pictures and we can just go to interview them and discover the stories behind them.
We had something similar, because China is quite large geographically so we can hardly go anywhere ourselves in the province, we have 13 cities. So we have kind of a similar website, basically the TV stations from each city will upload some useful information, and (we are at every city in the province), so we can discover what is happening in the other cities.
One of our strong points as a local newspaper is that we know a lot of people and we know a lot of local sources. ((<)) During that time and we almost exist for 250 years, that the sources were institutional, so it was the community, it was the politics that provided the articles. But we made a shift towards the stories of the people, the true people. So nowadays it’s a bit of institutional journalism, human stories. The part we are not very good at and that is data journalism. We (don’t use) data, sometimes because they are not there, but also because it still isn’t in our routine very good.
It’s a lot in person. A lot of stories coming out of the towns and the villages. People who have (a job) for example, or people always want to read what happens in their city.
We’re working together with the Algemeen Dagblad, that is the national and international news, and they make a big part of our print edition.
First of all, stories come from people that live in the neighbourhood and all the journalists in our company live in the province. And the best stories come from the people themselves, but I have to admit 30 maybe 40 percent is also from the internet. Twitter is a very, very main source for us. We have many tools that can find news for us, like Coosto, which I think you are acquainted with that, so I think 30 to 40 percent is collected by online sources, and then being checked. But the rest, the best stories we have are from the persons, the journalists themselves. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 199
we do have an account with ANP yep. But it’s mainly used for the photos they have because, in our province we have it first then ANP has it.
Absolutely, absolutely. I think we use the same system, but in a different way that {competitor} uses it, its called content insights and its shows not only how many times an article has been viewed, but also the loyalty, engagement, et cetera, et cetera for the audience. So that is how we measure how the audience likes our news. Despite journalists who don’t want to know that their story isn’t good enough, we are data driven because the stories people don’t read or don’t get loyalty to, we skip them more and more often.
New stories? Well obviously, all the organisations which are sending us press releases, things like that. That’s one. But the main sources for unique content are people in the city who knows us. We have a network, journalists have their networks and its our main source. These are the main sources and it might also be follow-ups of articles we have already printed.
It’s always hard to say. I am not the editor, so I am not the one who’s doing it. And on the other hand, well we use from our content and social media agency, we use (tooling) for that to make a scan of all the content which is there. Its called, I don’t know if you know these tooling its monitoring tooling, for instance we have Meltwater, but you also have (inaudible) and Coosto.
What you do is, it depends on what kind of news you are looking for. If you work for the local media, then your source, you look in the local newspapers. They all look to each other actually, they all do. That is what I see and now as well. If I make a story now with a journalist, I can see 3 or 4 calls from other journalists. So what they do is they all look at each other. It would be very ((<)), the best answer to say because that is how they teach it, is “yeah I am going into the community and I speak with people and I am going to find my news”, but to be honest that is mostly not how they work, because they don’t have the time to do some research or invest, or just hang out in the city and see what is going on. So mostly, newspapers are generated by what they receive.
Yeah, definitely, all the time. Online and on social media, Facebook, Twitter, yeah. That is how I make my news at this moment as well ((laughs)).
Yep, I work with {company}, at this moment. Yeah what I see that it is not doing a good job, that is my opinion.
Yeah? Ok, you would suggest or you would think they would cover the whole of {region}. But actually, if you are going to analyse their AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 200
news, its mostly the area of {city}. But it is {city} because they work and live there, the people who work there.
Yep, very bad. I am not very keen on {company}. I have worked their as well, yeah it was 20 years ago, and I still know the people there, they are very kind and friendly. But, very {city} oriented.
Yeah, because they live there and that’s how they consume their media and therefore that is how they see the province, but it is much bigger than that.
Well mainly because it gives me some ideas of how other people are looking at subjects. So I will, I am obviously still quite young, but sometimes I heard something and I think “oh my god that’s weird” or “that’s strange” and go to Reddit to see how other people discuss the topic because as you are probably familiar with there’s a lot of different opinions on Reddit.
They use a mix of sources, I think they tend to trust our content completely. One of our roles in this landscape is that they know that they can trust our content, and they don’t need to check that themselves. But we do not cover each topic, or we do not have an opinion, or we do not have opinions from experts for example. So what the media, the customer of us can do on top of our content is make their own angle and make their own story out of it.
Yeah, well we start everyday with a news meeting, with all the editors. Some of them are really, well they have their contacts and come to stories like that. Other ones use data (from it) or use social media to find topics and see what is going on. So yeah it depends on which editor you (are working with). But it is a mix for those things.
Noo, I also think they are online also, and we also use tools like for example, (crowdtangle), you know that? It’s a tool, you can monetise Facebook, and you see especially what’s doing well on Facebook, so you can see the topics people are reading.
Well, Twitter is still a big thing for news organisations, I don’t think for the main public, but also Facebook, Instagram. Some of the younger people use Reddit for it.
Yeah, well I personally use Tweetdeck and I think most of my colleagues also do, ((>)) and Hootsuite I used that in the past, but I that was a long time ago. But there is not a main way to do those things, every editor has their own, kind of workaround he or she prefers to work with.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 201
Yeah, I would say I have an A-list, I would say 10 sources. The Verge is on there, so for my field of interest, technology. You got The Verge, Wired, Dave Lee, Marcus Brownlee. There is also the communication between the team members at {current company}, because we are all experts in this field. But for certain, sort of sub- specialties and certain typical unique points of view, and certain even biases. Not biases, but perhaps, favourables.
Sure, well I would say that at least 50 percent or maybe two third is good old fashioned press release and so, stuff that PR people or divisions send to you. And I would say that it is not just the press release but the whole publication, from PR to journalism. That also accounts for a call you have with someone, a spokesperson from a certain company. All that kind of contact, that communication, that’s a pretty big chunk. I think the second biggest chunk would be what others do, but put it very simple.
I think in the end, everybody, journalists included has their own web of trust. They have their own circle of sources that they like, that they trust and that they regard highly.
Well for example you mentioned one already, The Verge is one. There is also, yeah, well also here in Europe there are certain product releases, or spec releases, or information that is being released under embargos.
Yeah, so what others do ((>)), it sounds perhaps a bit weird, it is not like we are you know copying things, we are not sort of you know, ((>)) spying on, how do you call it? In Dutch you say spieken. Well, anyway it is not about copying what they are doing, it is just, you’re widening your own reference pallet.
So you have got your own sort of line of opinion making, and here there is a bit of news, or there is a new product, you get it or you read it, and you create your own opinion about it. And you make all kinds of associations, so your expertise kicks in because you have been doing this for a long time already, you can remember “hey didn’t company X do almost the same a couple of years ago”, or you know that kind of thing. So you are processing the news, or the product and in that process, other journalistic sources for example, like The Verge, can play a part in that. It is not just The Verge of course.
Yeah, yeah, so in the process there is a phase of information gathering, from a direct source, other journalists, your colleagues, your own historical knowledge of things and so once this gathering phase is finished, you start constructing your own story. So you are laying the groundworks and you are building up from there. And obviously the third and final phase to get it out, with video or an article, or you name it, yep.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 202
Well of course yes, but you have to use your brain to come to the right questions, to come… everything starts with the right (amazement), when you are amazed about something, or you are struck by something and you think how, how could this be? I don’t see a robot doing that, so as long as there is a robot to be programmed to think like we can, we still have to use real journalists.
For us as a regional paper, the main ((>)), the biggest sources are our network. So people you know, people you speak to, people we see. The region we write about, we live in, so I go the store, I go to the library, I go to the ((<)) swimming pool, I go to the sports club. I see and find and hear and speak people.
Search engines and search methods, they can see what is happening, where online. So, we have like a {city}, {city} in the search. We can see “oh there’s a lot of traffic going on about {city airport}, so sometimes that, is the main source, but most of the times you just start by hearing something. The old institutions are important for us as well, like municipalities, ((<)) {city} airport, or the bigger companies like Phillips and ASML. But they put out news, and they report on this as well.
So we have a digital desk, and they keep track of everything that is going on in the region, and then when something stands out, they warn the news editors
I think we are not enough data driven, we could do more there.
What kind of stories will you publish where, we are still trying to find out what works best, but we, yep, yep. We measure everything, constantly, what do you read, and how long, and when did they, if they read this what do they read as well? Did they take a digital subscription, what were the 10 stories they read before the subscription?
Because if I look at {previous company} it was ANP, which was hugely important.
But then you had your own network, sources you know, sources that bel ((meaning via phone)) you stuff, during that time we working there, of course social media become a thing.
A source of news, and we of course like, RSS was a sort of news source. It was out there but we didn’t put it together, at the end we did. So, that’s actually we way we get new stories.
Are they bringing the right stories? Or does {previous company website} bring the right stories? It depends, I have a personal view on that, I mean, but so you have this ‘snack news’ on NU.nl, a bit on {previous company website], but do they invest a lot of time in AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 203
investing in journalism? Very little. Is that the way to go? It feels right, but is it something that {previous company} thinks is important?
You can do your online research and your audience can be, ((<)) are satisfied with that, so you can get away with it.
So you don’t have to the extra mile, to get the better story, because they are ok. They find you anyway, and they click at stories anyway. So we’re good, money is rolling in, advertisement is going great.
it’s a choice to go the extra mile, maybe like the (how the spending is) doing, so also a different way for paying for journalism, right? Because you are a member, and it gives them the opportunity to give them the time to build a story.
How easy or difficult do Yeah of course they can be improved, but you know I think that is you find your current only human. You are satisfied with what you have, what you don’t content sourcing process? have you don’t know.
Nooo, noo we live in the province with the most news in the whole of the Netherlands I think, so that has never been a problem. The problem we have now is to find the best people to make it for us, the workers at our station. Some years ago there was a lack of jobs, now there’s a lack of people to fill the jobs.
Yeah, that’s always an issue ((<)), we try to get them to make news, but we like to bring more background stories, and/or research. That’s a problem because ((<)) there’s enough funding at the moment, it’s a bit harder but it still survives, so ((<)) but we don’t have a enough money to do a good enough background research.
And that’s something which is missing. But I think that’s a problem for everybody in journalism, accept maybe for the national newspapers. Everybody is trying to see what they could do about it. But still so far I think nobody has an answer. What is your overall That’s is a lot already, I think that what it looks like now and as far as impression of this system? I can think about the technology behind it. This could certainly work, so I ((>)) maybe first thing before we spoke what came to my mind. This is an initiative that looks and sounds familiar to me, I have seen examples of it before.
So that is what they tried to do, ((<)) apart from technology didn’t work like it should, their main problem was that they didn’t find a way to really make money out of it. Because, the first connection would be ok, but if I for example as a freelancer journalist, thanks to this platform I would be connected to the ideal newsroom for my work, I would need you only once, to make the first connection. I could go directly to this newspaper or this magazine, I wouldn’t need you anymore, and that was something they weren’t able to solve. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 204
So that was a big problem and they were not the first to find this problem. I have been involved in a start-up accelerator, what you could call the stimuleringsfonds voor de journalistiek, a fund for journalism that was supported by the Dutch government. They had this accelerator for journalism start-ups, and we have had several examples of this marketplace like, bureaus or platforms that try to connect freelance journalists with the perfect distributor. And they all failed.
What is different as far as you have shown me a couple of minutes ago, is that this could also serve as an inspiration for newsrooms. If I look at this dashboard it could inspire me to write about one topic or another and use maybe fragments or paragraphs from the offered text.
So there you need to work out these connections and if you want to keep everybody happy, because there are several people who are ((<) and elements involved in the whole process and scheme to keep everybody happy.
If you are of course only looking for specific topics, if you are a general journalist then that might be worthwhile, but ((<)) still it is, if only you would look at your Tweetdeck or your Hootsuite, yeah or newsfeed. You can be inspired in tons of ways, I would say. You would have ask yourself what problem do you solve with that, what you need, and sometimes it is nice to see something else, by accident or, but will this be a reason to join TCE or?
Yeah, it could, but it’s not as far as I can see now its not something that I can’t do without. As far as I can see now, you have only shown me some screenshots, I don’t know exactly how it works of course. But, there are several roles within this system, you are referring to a newsroom, editorial place at this point. Then it would certainly add something, but (if 100 would be the total view, it would be like 40 to 40).
Yeah I think that, I trust that they will build a technology that is working I would say, the real test would be in ((>)) could you create something that all sides of the eco-system, would solve a problem, and it really needs all sides. It is not only a question of solving, the problem of the freelance journalist which I think could be done quite easily. You could also have to solve the problem of the editorial room and the appliance of these editorial rooms. So there has to be a logical total and not only the separate elements.
No exactly. It would be next to that. Next to maybe the wires and the yeah. So, yeah the main question is how needed this would be for me. If you can convince me that I cannot miss this as a tool to perform my job, then it would be good.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 205
[I think its useful because we, no one can really have enough information (exchange is important).]
I think its (somewhat) helpful, there is no such platform that is gathering all the information, so we need to search on social media A, social media B, social media C, still. If there is such kind of information where all the agencies can just exchange information, that would be very helpful.
((<)) The proof of the pudding is in the eating and the quality of journalism that is profiled by it. ((<)) What I see is that ((<)) most news organisations try to create content that makes them unique and that makes the difference for them.
You have those things that you need to have it, but you don’t have to make it yourself, but because everyone has this sort of news, you don’t make the difference with it.
And that could be problem when you make a platform like this, the things you put on it is, everyone can buy and use it. Nevertheless, if it is really good and its simply really good, it could work. But if, its really good because there is a really good reporter on it then you always have the opportunity that news organisations will buy someone away. (So it would take you into service.) I think its complicated to make it work.
It could work if the quality is ok and I guess it is not easy to make the quality overall, good enough. But at the end of the day it is simple as that really good stories are shared there. It is not very good, or it does not match the standard of the news organisation, then its difficult, not going to work.
Because if you look at the province, is one of the best covered provinces, when you look at journalism, the {parent company} that owns {regional outlets} and they understand they already have 190 journalists in the province. We, {current company}, we have 150 to 200 people working within the province so, {province} is one of the best covered in terms of journalism. But, in this best covered province, there is only two competitors that’s {competitor}, the newspapers and we. So, if you put that to the test of this platform you are presenting, it would not get a real chance to get alive because either we buy the story or {competitor} buys the story, and that’s so, do you need a big platform for that?
Yep, absolutely, absolutely. Finding our news online, we depend on tools like Coosto, which is so perfect for us because it knows what kind of news we are looking for, and it gives us recommendations of where to find the news or even finds news that we haven’t discovered yet so, yeah personal recommendations are very good. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 206
Coosto works with very, very clever algorithms that’s learning by seeing what we do with the news so that is very important for us.
But on the other hand, if you should have presented for selling news, but for finding news, on which you have many, many sources that we don’t even know. A perfect way to search these sources, that could be very interesting.
Because if I read the article at ((IE lists local publishers that cover region that IE reports on)), which are the local publishers in {region}, then there is no added value for me, it is not unique, its not one of a kind.
And that’s everybody what like to have, their own unique content. So that is one important issue, and the other issue I guess is well, editors are pretty how do you call that? They go their own way, they know what’s best.
It’s pretty hard to become part of their routine. They should ((<)), they might look at it once or twice, but if there’s nothing there which they could use, they’re not going to use, they are not going to look at it again, because its not part of the routine.
Yeah, and then its gone otherwise I said, “now, we tried that, there’s nothing on it, so we don’t try it again, I enough have too, so I am so busy.
Maybe its also just, use it if you like. Because we provide content for Nieuw.nl, we provide the local content for Nieuw.nl for {region}. So actually we are doing kind of a similar thing then, but only for {region} and it comes from {current company}, there are no other providers.
Yeah they are (in a way) persistent, they know what’s best, nobody should tell them, how to make an article, what’s best, et cetera. So this is something from outside, and they always like to work with their own freelancers, with their own ideas, and something from the outside. Always when we talk with someone from the outside, they kind of (hold) and reject it.
Yeah, recommendations, not really these type of people, the editor are not really much open in my opinion, for things like that. Recommendations are more for travel websites, et cetera, or maybe even if these are known names, people they know then a recommendation helps.
But if it is people they don’t know, I am not sure if that would be the thing. The thing that might depend on what platforms they publish on, or it would help if they know it’s a unique story, or it would help AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 207
if for some kind of reason, it would help if we have for instance, AD they have a lot of news about accidents that happened. We call the 112 in the news, when there’s police involved, they publish it. And for us we like to have more background stories, a bit more intelligence, like NRC or like that, and if you could add something like that, you know if its yeah kind of our profile, then it would help, I guess.
Yeah it reminds me of Tweet deck and I know a lot of journalists are familiar with Tweetdeck. So that’s great I think.
Yeah, I think if it looks familiar, that’s nice definitely for journalists because I have colleagues who still find it hard to work with new technology.
Yeah, its still a very hard question to answer I think. Definitely (still a problem). I think for instance people who write about sports, I know the example you just mentioned through {name}, who write about sports. I think this is great. But at other editorial offices, this might not work, I would also be concerned about journalistic platforms who want to have an exclusive article.
Yeah exactly, and as far as I am concerned, a lot of platforms deal with these kind of things in a lot of different ways, and as I mentioned before, if I believe that the quality is good, if it is almost you read an article that was written by a human journalist, then of course its ok. But a lot of platforms are looking for perspectives that are just a little bit different, than perhaps the ones that are already there. So as you mentioned, we have to feed data into the platform to generate the right kind of topics. So I think that a lot of platforms are looking for stuff that can not found by data.
Hmm. It might be a nice way, if you write a story to find a platform that would want to have it. I think that might be nice, but on the other hand I already have a lot of platforms I am working for, I am not necessarily looking for other platforms that I can publish for.
So, maybe it would be more interesting for journalists who are trying to find a new platform to write for, or maybe young journalists who are still looking for platforms to write for.
Yeah and it would have to be exclusive because none of the people I write for would want me to use content that somebody else might be using too.
It is interesting of course because what I told you, how I started was that it was the middleman. The middle between the journalist, the editor, and the consumer. And that is weaker and if you have a Content Exchange, of course there is, some way of thinking in, well AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 208
now perhaps the Content Exchange is doing that from freelancer to publisher, yeah?
I think if you think about how to present things, then you sooner or later end up with the column approach.
[And, personalised for media, well it could work perhaps. Depending on who is using your Content Exchange. But I think generally in the first phase, generally the users will look at news in many directions
Yeah, well I know this kind of dashboard, from other (companies). And my experience is that editors, that there are editors who might use it. But is there very individual choice for editors, as I already told you, everyone has his or her own workaround, and it is difficult to change those workarounds they are used to.
Yeah. So it has to be very attractive and very useful, to give a benefit when, if they really are going to use it on a daily basis.
But I think she’s going to read it, and then do something by themself. Well, how do you say? Well in journalism this is sort of not done, to use someone else’s story.
Yeah I know about the idea roughly, yeah at this stage, I don’t have an opinion yet. I mostly have a lot of questions.
The other customers I would see are, medium-sized publications, like regional newspapers. Yeah because ((<)) if RTL doesn’t have a certain kind of article, what would convince them to buy an article, Dutch is a very small language, so everybody knows all the other players. So what would persuade RTL to actually buy an article from lets say NU.nl, and publish it. While there is also already ANP, yeah ANP ((saying first the Dutch letters and then the English letters)). (They could use that).
Sure, sure I can imagine smaller players being added to this system. But then, the vetting of those smaller players would be very crucial.
Well, there you go. I am not sure whether either one should be, obligatory. I am not sure if it, it really depends on an exchange like this, it can still go in all kinds of ways. It could be an exchange for big players trying to cut budgets and sort of cut corners and get content on the cheap.
But then I would say that the number of players is so small, that they probably won’t do it, because well they are too proud or something. So that’s one scenario. Another scenario is that you try to connect through this exchange, big and smaller players.]
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 209
Then you, because like I mentioned you got a big issue about vetting, for smaller players. A third scenario could be, ((>)) well what’s the third here. There is a third. I have to sort of…phrase it. ((>)) Well, for the time being let’s stick to the first two, but what is this exchange, what does it want to be? Does it want to be sort of a new ANP? Or sort of for (natural) articles and pictures, maybe like other agencies like Hollandse hoogte or internationally of course
If this goes too big, you can’t solve it with people, just like YouTube can’t curate all of their videos
And supposedly I would have a company like, lets keep it simple, PSV. I would get someone to post a lot of positive content on the platform.
I would pay, as PSV for a journalist who would post on the platform and I would be on the platform and think “oh this is a nice story”. How do I know where it comes from?
That… of course it is big enough, but the problem is, how do I know if I want to buy from you? Is it legal, you know? Is it checked? From who, who gave it to you?
Because, just I don’t know if you know how it works, but I am in the end, I am responsible for everything that comes in the newspaper, and on the site. Because the legal issues are growing and growing and growing, so a bigger problem everyday. So, I am a bit concerned on that part, always, always. That is my biggest question always: where does it come from? Where are your sources, you know?
I make a phone call for 20 minutes and I have a new story to write, but if I want to make a video for 20 minutes, it is going to cost days.
So, it is more expensive, so if that is what they want. So if you can deliver, lets go.
So, there might be an opportunity, I know the main focus is on video. If you look at {large media company} because they are able to write stuff, right? They have had 100 years for that, right? But they want more video content And the video content is expensive to make.
Because when you talked to your customers, they will just say “Give it to me fast, give it to me cheap”. But I do not want mistakes, if you send my something and I put it online, an hour later I have troubles with copyright, or with privacy violations, we’re done, right?
o its one to build a platform like this, right? Where you try… a platform works like taking away problems and just giving it, handing it over to me. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 210
Right? So I am working in the newsroom, I am looking for a new story to post, I am looking on this platform of yours. And I say “ok this looks interesting, I am going to copy it, and I am going to post it”, and so that means, and you can do that technically, and that’s not a problem. I think you can build that.
you are helping a newsroom to post new stories, great. But all the troubles, like privacy and you know, you have to solve, because it is not my problem anymore. What you send me is ok, it is checked by you.
Which is probably, it’s a choice, it’s probably cheaper because everybody makes social media videos, posts them online. You can talk to people and say “can I take your video?”, I am going to pay you a small fee for it, I am going to use it in a news story, right?
And then the trouble starts, because who is in the video? Where was it made? Is it really showing what we’re telling its showing?
And me in a newsroom getting this video, I do not want to ask myself these questions. I want to use video, I want an outline with it and, post it. ((IE uses a hand sweeping hand gesture to signal posting the video online)).
Which sometimes is more interesting for somebody for 18 years, than somebody working for 80 years, right? It can be more diverse, and at the end of the day, what you put online and the traffic you generate, that’s what this is about, right?
So for {previous company} it is about being relevant, people come to us because they get the story if they want to know. In the {large media company}, what they put online, they have to generate traffic, to sell advertisement and to earn money.
, I don’t think so. I have never felt it that way. What I saw working at {previous company} I had several screens, I had social media, RSS, ANP, so you just skip, you look through it, and you scan, you pick up what you are interested in. If you were to use TCE, Because again there are several roles visible in this platform. are there any elements of its design you feel would It looked clean and ((>)) well user friendly I would say and it is benefit your search? comparable for how for example the ANP press agency is offering their content.
It is also similar, if you look at my Tweetdeck then you’ll see the same with the columns of topics that I need to follow.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 211
Of course, but for me that would depend on the topic, or the content. But the layout is helping, its not a barrier or I would say so ((<)) yeah.
That helps of course it’s something that I already now and I know how to use, so that’s clever I would say.
It is very good, because most of the tools that professionals use and journalists use don’t have such a good interface. They don’t have such a good look. So it was very good looking, I am very positive about it.
Its different because I know the Content Exchange is making content for journalists, but still I think it’s a very big challenge to see how you are going to categorise all these topics and to make sure you are going to be relevant in these kind of topics, so that (users) are not drowning in everything they are going to send to them.
Yeah I think it will work, but we have to test it, we have to be very careful with how many topics we add and how many sub-topic we add. Because its going to be a very, very big mess like my phone.
Definitely, also maybe it’s a good idea to send them push notifications about new topics, or other interesting topics. Maybe push notifications via email, so you can trigger people to go to the Content Exchange and see what’s going on out there.
And so it is interesting, for example to make some kind of basic profile so its like: “hi guys it’s nice to see you here on the platform, you are a sports journalist, this might be interesting for you” and then give them all kinds of interesting topics. Yeah, well that’s what you do, but sport is an easy one. ((>)) Yeah what’s an, I don’t know, foreign news for example could be very different, but yep. Something you really have to think about.
I am very enthusiastic about the Content Exchange ((<)), yeah I am very enthusiastic about the platform, it worked very quick and importing a feed. Every time I am surprised by the speed of the feed, so that’s pretty good. We’re still working on some stuff, like does it receive photo, video and text in one, column or well how do you expose it. We’re still discussing those type of things, but yeah in general I am very enthusiastic about it.
No, not really. With you know ((<)), I guess its proven, it works everywhere, its not spectacular but does it need to be spectacular? It should work and that’s the main function, and it does, so, yeah looks good.
It has to be quick and easy. I’m the editor who has to download, I have to edit the video and it would be helpful if I could download it AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 212
as well. But still I know as a newsroom you would like to have grip on how many you have to pay for the content of course. I would like to have a grip on the budget, so that would be something you have to think about, how to do it.
So, it is very easy to understand how it works, right off the bat, so that’s great.
The way it looks, that will be fantastic, yeah of course. It would be something, we could run on all day and just check, is there anything relevant for us that we can get from the platform.
So, make it simple, because when you make it simple, it is easy to use and you can browse through it and then move on to your next screen right? That’s how {previous company} uses it in the end. So we have a really simple way, when I talked to IT people, they show you everything that is possible, I had like “wooah this is great”, but if you bring it to the floor, they don’t care. They want it to work quick, easy to use because every bit of more difficultly takes more time to learn and that’s the hold up.
It is nice it is helping me, but it has to help me. You don’t have to complicate things.
Yep, make it as simple as possible, and the end of the day, we made something that looked like ANP. Because they knew that, they were familiar with it, they said “oh I can relate to this”.
And gradually we put some more options in it, but we took them out just as quick when we found out, this is not what they wanted or they are not going to use it. So, it really helps when it looks like something from somebody already knows.
We had all these options to make your own feed, right? And, but they didn’t know where to start, so they had a long list of themes, of topics and behind the topics there were relevant organisations, that were putting out these RSS feeds, right?
So, we had maybe, I don’t know maybe 20, 30 topics, right? Then you think ok, which ones are relevant for me? Well, all of them? Because I look at all topics, that’s my job. Or should I focus? And, so that was something we had to guide them with, so at the end of the day I took it over from them and I actually did interviews with them.
So, then you find out that they sometimes they don’t really know what they’re focusing on. So, you have to find out together, how better it could add into the programme and say “ok, this is what I heard, so you are going to get these and these feeds”.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 213
“Ok, this sounds logical, can I get back to my work now?” Yeah you can, you know?
So it could work from our side, we did that. We had a lot of trouble, but that was maybe more our side, developing.
So, its very complex. It’s the same thing with tags. You start adding tags to stories and then suddenly you say “what a minute this is not adding up, because we are, (using too many tags)”. So you know that is what happened on the backside of the programme.
So we actually, fortunately we were in the position to just throw away the backside, and rebuild it quickly, so that was a problem. We thought about, the journalists adding tags, forget it. We didn’t do that, so we left that out, so all these options to make it more personal at the end the day, we took it out. At the end of the day I had conversations with them, and I made it specific for them, so they didn’t have to change anything with them, anything anymore.
And we also gave them the option to put extra feeds in, or add extra feeds, extra organisations, nobody ever used it. So, at the end of the day, we had one person responsible who checked, and just asked around, what organisations are important to you? We did that on a regular basis so we could update the programme, right? If these articles in this I hope so, well I would try to convince them to do so, because ((<)) I system were automatically would try to stress importance of the quality of the content not the written, would you still (source) itself. Whether it’s a robot or a human being, or whatever. If consider sourcing them? the quality is what we are looking for, then please use it.
No no, I would say it would be judged equally compared to the other articles. For us it would always be very important to know the source. Oh that wouldn’t make, it could make the difference if about what sort of content it is. Automatically written could be just small pieces in your newspaper or your site. More in your newspaper, on the website small isn’t very useful because, there is not much space on your site or app. Everybody thinks that newspapers hasn’t much space, and online is never ending, but no your site, people don’t scroll 2 metres down. it could be useful, but only for small things I guess.
It could, but for there’s a difference. When I’m only a reader its ok for me when someone chooses. When I am a news organisation, I want to make the choice and I don’t want to have it done by an algorithm.
Yeah it could work, and because at the end of the day it is still, is the story good or not? That’s an important question, on top of this could it be exclusive for us? So if everyone can use it, I would say, well we make just a short story of it. We don’t make the difference with it.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 214
I don’t think it makes any difference, in the process of buying things or not buying things, whether or not it has been written by a person or a robot. Considering that fact, that the quality of the articles should be perfectly the same.
Yeah, like I mentioned, I would not at this moment. If I relate it to your platform, it would not be the way to, ((<)) get to know it. Because robo-journalism looks like an experiment, it needs to be developed and when its developed, it could be part of the platform. But since its not well known yet, and not proven yet, I would start by just offering it separate as a unique project or something.
It really depends on what kind of article, because if you the correspondence, you know that one for us, that type of article would be very interesting. But if you start offering, sports news or the ((<)) main news which you see everywhere, yeah (inaudible). So unique should be the key word, I guess.
Ah yeah, of course. I would use it for my unique piece, yep definitely.
I would use the automatically generated content because then I would save me time, because the robot made the script, and until it’s over I get the script and then that makes my unique story and that’s how I would work with it.
I think it is all about the quality of the article.
So if it is good, I don’t see a reason why not. But I do wonder whether these type of articles show any originality. That is often expected from journalists.
I think the journalist will use it, I think he will use it more as advise and not will use the story itself, but to start to bring their own story, or to investigate a topic.
Yeah, I think so. (There was) a study at the University of Tilburg where a sports desk, they did some interesting stories about the psychological effects on automatically written articles. Our confidence is, well their was not much difference between, when it was written by a journalist or when it was written by a robot, or computer or how you may call it.
Well, I think so because we again, there is your routine where those journalists, where, well it is similar to the routines they are used to.
No, I think it has to be easy, most journalists well are digital savvy but not that digital savvy that they can, work with a lot of interfaces and all that kind of stuff.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 215
Yeah, actually I think I would source them sooner than getting content from my biggest competitor.
Well because then in that case, if I publish it my competitor doesn’t feel “hey look at that man, there he is buying and publishing our stuff”.
If it is written by a bot, actually I would say it would enlarge the USP of the Content Exchange.
Yes, if I could know, so, yep. I am not against robot-journalism, I have to know.
Yeah, why not? Yes, it could be one of the sources for stories for us, why not?
Well learning that people, the audience, for the audience it is more and more difficult to see which one is written by software and which one is written by a journalist. So, again if you ask the audience when they are satisfied, they are satisfied. If you ask the journalist, it probably has a date, so “well I am important here in the process”, and some of my colleagues are afraid I think, of losing their jobs. They’re going to embrace it because its cheaper.
Considering every we It can only grow, there is only one way and that’s the growing have talked about so far, importance of these robot-like sources. technology in the newsroom, AI and But its like with many innovations, it takes more time, some of the automated journalism. enthusiastic in the beginning think it will take. Because its not only a Where do you see this question of technology it’s a question of also and maybe more even going in the future? so a question of culture.
So and if it’s a robot, I would want to know how the robot works. So, and not with every single article, if there is a robot active then I would want to know what the algorithm is like or how this robot, you know, what the sources of the robot are.
Well I think journalists is, this job is somehow being replaced by technology and replaced by (ordinary people) because in the past, there were only news reporters for the TV papers, for the TV stations that there might be no other ways for the audience to gather information, and now with the help of the internet everyone can get information, everyone can be a journalist him or herself. So I think the future of journalism personally I would say would go deep with some essential issues, also we need some professional editing, for example the coronavirus is a world issue now, so a lot of journalists and reporters are reporting news and relevant news from different texts, so because I was a journalist and now I am not, and if I were in Wuhan or somewhere very special I think my previous work experience can help me to share my own information a bit more AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 216
professional than the other people.
I think so, for example every government is reporting the daily figures of the infected people, whatsoever, so I definitely think it can be replaced by.
: I think the next within a few years is to have the small news, like football matches maybe also the short police news, crime news, car accidents, things like that can be done automatically. Its not such a big step, it happens everywhere. But I think that the top stories will always be done by reporters, why? Because your reader wants to identify with it and maybe at first glance, they won’t see the difference, and maybe they won’t identify with a computer. But that will take some years. But for now it is good to see the face of someone.
Yesterday I was watching an episode of House of Cards on Netflix and in it was a fragment from CNN which has a beautiful reporter whose name is Wolf Blitzer, one of my favourites. If he is doing an item its always good for me. You always want to feel a bit, and for that it is good to know who is a writer of an article.
Its difficult to say, also because on the local markets you wont find many customers. We are a big company, with other newspapers, and you have one which is on local radio, {company name}. (It many not fit I guess) and there is not much money on the market. I am not saying it can’t work, but it will be really hard to make a living out of it.
On a national level there have been some things like novum, which was a press agency. It’s a tough market.
I think that in professional journalism, the personal touch and the human factor is becoming more and more increasingly important. In an online world where everybody has news, brings news, and there is a lot of fake news, to attract people to come to a website, more and more. We as professional journalists will be working on bringing the real news. And I think more and more we will be recognised as the ones who can be trusted. So I think that in producing our news online, metrics, robot-journalism and all tools that we have will become more and more important. But in producing it, in making it, we finally need some really good journalists to make what we produce for the audience.
I don’t know, I don’t know. Because maybe everyday I am reading robotised articles which I am not aware of that they are written by a robot, I don’t know. I am not sure if it has the future, I am really not sure. It’s an experiment that has been started in the United States many years ago already and it wasn’t too successful there, as my AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 217
information goes. Maybe you have different information. So I am not too enthusiastic about it. Definitely, yeah definitely. And now you are going to ask me why, because the who society is going to change, it is changing already. Its not the same, and its going to be more and more digital information, but also statistics are going to be more and more relevant.
If you look at things like AI, I think the filter bubbles which are already here now, by Facebook, things like that, might even go further. That’s what might happen to AI, but on the other hand, (until so far) Facebook has already been there for a long time. People still appreciate, they are being surprised with content they would not look for normally, but still read.
I know NU.nl made the platform, so it might only present which you like or which you follow all the time, and then they found out later this is not the way forward, then the filter bubble is there and people do not like the filter bubble, in the end.
Yeah maybe, or a lot of news is now already shown in the search feeds, so as a reader you don’t necessarily have to go to that platform anymore. Then I see for instance the Google journalism initiative, data visualisation tools and the publishing tools at Google. So I think at some point maybe Google might incorporate all the news organisations. Instead of just letting them just having their own platforms and sending traffic towards these platforms, and I do understand because it is not Google’s job to provide the news companies with traffic. But yeah I do think that maybe that’s the direction we are going to.
And the real quality journalism will probably stay for just, I don’t know, business, professors, like just a small group of people. I think the rest of the information will be provided by Google, by Wikipedia, by those types of platforms. So, yeah I think it is going to be difficult, but on the other hand, I don’t know if it was easy before.
So, I am afraid that journalism will just be something for sort of a small group of people, and that one journalism will stay for a general group of people like, televised journals and stuff like that. But that will probably be government sponsored as it is already at this point.
Well I think that for many years, it will increase, it will not replace journalists but it will empower them to do more, for an ever higher quality, to get rid of tedious, boring work.
That’s for years to come, its more machine-assisted, and not completely automated journalism.
Well I hope that we can put our journalists into the work where they as humans are very good in. So everything what you can do AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 218
automatically, it should be done automatically. So everything what repeats, should be worked with artificial intelligence, or whatever, and you can use your creativity and your knowledge to do the things, well you have a really, special gift as a journalist, so I hope this will find a way through.
Well I think it will expand, but it has to start with a lot of basic work, as I told you I handled sports journalism, but also in financial journalism. Journalism where there is a lot of data, well you can use those techniques, but I think there will be a, when it works well, there will be those texts, well I don’t know at this moment but I am sure they will be those texts.
You have that experiment with that author who wrote a book, he wrote it together with a computer, so (there was robot). Then I guess this is a step, well it was not perfect but it will be become reality.
I think you can be afraid of your job and then say “ooh I don’t want to work with it because it takes my job”, but then other people will investigate it and (use) their resources and it will happen anyway. You can better be part of it than.
Well, it probably goes ((<)), it will probably grow but it will probably make bigger leaps when there are breakthroughs with algorithms, for this particular practice.
For example, here in the Netherlands we need an algorithm that can write Dutch.
Yeah, but I am not aware of anyone who actually, there are some experiments at {parent company} with bots, but that’s it.
I think a lot of the things we do can be automated, like in the long run. Everything that has to do with the weather, a lot of people want to know about the weather, to know about traffic, want to know about closures of roads, a lot of people want to know about agendas, what can I do this weekend, where can I go? That can be automated, like the real deep stories and the news that can help you with important decisions in your life, that has to be made by people, because it all starts with the good questions. Can I trust a robot? But you know, can I trust a journalist? That’s the same question, not always
It’s a difficult one to predict, I am looking at this from a positive point and saying, we can fill {previous company website}, NU.nl by an automated newsroom.
Right? You that is maybe something in 5 to 10 years. After that I am not sure, I think software will be able to write stories like a correspondent is doing right now. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 219
So, for the near future, 5 to 10 years, you will see an automated newsroom, but partly filling NU.nl and {previous company website} and after that…
Remains to be seen and I hope in the meantime, we start a very good debate about robot-journalism in an automated world.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 220
Appendix R Axial coding Code Description Quotes
Use of personal Gathering information Everything we hear within our network is way networks for a new story via their more important in our heads than things that own networks is a key occur outside of our network. We would, if source of information, somebody within our network would ((<)) come especially for local news up with a suggestion, this would lead to a story outlets, way earlier, way sooner and faster than just an email or a message. Or a robot that is suggesting something.
That is of course, networks are important in that way it is good, and it is logical but its keeping us from finding original sources or new sources. That is something holding us back from reporting on stories that might be worth reporting on.
One of our strong points as a local newspaper is that we know a lot of people and we know a lot of local sources. ((<)) During that time and we almost exist for 250 years, that the sources were institutional, so it was the community, it was the politics that provided the articles. But we made a shift towards the stories of the people, the true people. So nowadays it’s a bit of institutional journalism, human stories. The part we are not very good at and that is data journalism. We (don’t use) data, sometimes because they are not there, but also because it still isn’t in our routine very good.
First of all, stories come from people that live in the neighbourhood and all the journalists in our company live in the province. And the best stories come from the people themselves,
New stories? Well obviously, all the organisations which are sending us press releases, things like that. That’s one. But the main sources for unique content are people in the city who knows us. We have a network, journalists have their networks and its our main source. These are the main sources and it might also be follow-ups of articles we have already printed.
What you do is, it depends on what kind of news you are looking for. If you work for the local media, then your source, you look in the local newspapers. They all look to each other actually, they all do. That is what I see and now as well. If AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 221
I make a story now with a journalist, I can see 3 or 4 calls from other journalists. So what they do is they all look at each other. It would be very ((<)), the best answer to say because that is how they teach it, is “yeah I am going into the community and I speak with people and I am going to find my news”, but to be honest that is mostly not how they work, because they don’t have the time to do some research or invest, or just hang out in the city and see what is going on. So mostly, newspapers are generated by what they receive.
Yeah, well we start everyday with a news meeting, with all the editors. Some of them are really, well they have their contacts and come to stories like that. Other ones use data (from it) or use social media to find topics and see what is going on. So yeah it depends on which editor you (are working with). But it is a mix for those things.
For us as a regional paper, the main ((>)), the biggest sources are our network. So people you know, people you speak to, people we see. The region we write about, we live in, so I go the store, I go to the library, I go to the ((<)) swimming pool, I go to the sports club. I see and find and hear and speak people.
But then you had your own network, sources you know, sources that bel ((meaning via phone)) you stuff, during that time we working there, of course social media become a thing.
Searching and Many outlets search The biggest challenge is you can never discover monitoring online and monitor online, anything new, (real new). That is the biggest sometimes with the problem, and that still we can discover (known assistance of by real people), so we try to, or otherwise we can dig into it deeper. Sometimes people post a few monitoring tools. sentences and a few pictures and we can just go to interview them and discover the stories behind them.
We had something similar, because China is quite large geographically so we can hardly go anywhere ourselves in the province, we have 13 cities. So we have kind of a similar website, basically the TV stations from each city will upload some useful information, and (we are at every city in the province), so we can discover what is happening in the other cities. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 222
Noo, I also think they are online also, and we also use tools like for example, (crowdtangle), you know that? It’s a tool, you can monetise Facebook, and you see especially what’s doing well on Facebook, so you can see the topics people are reading.
Well mainly because it gives me some ideas of how other people are looking at subjects. So I will, I am obviously still quite young, but sometimes I heard something and I think “oh my god that’s weird” or “that’s strange” and go to Reddit to see how other people discuss the topic because as you are probably familiar with there’s a lot of different opinions on Reddit.
I have to admit 30 maybe 40 percent is also from the internet. Twitter is a very, very main source for us. We have many tools that can find news for us, like Coosto, which I think you are acquainted with that, so I think 30 to 40 percent is collected by online sources, and then being checked. But the rest, the best stories we have are from the persons, the journalists themselves.
we do have an account with ANP yep. But it’s mainly used for the photos they have because, in our province we have it first then ANP has it.
Absolutely, absolutely. I think we use the same system, but in a different way that {competitor} uses it, its called content insights and its shows not only how many times an article has been viewed, but also the loyalty, engagement, et cetera, et cetera for the audience. So that is how we measure how the audience likes our news. Despite journalists who don’t want to know that their story isn’t good enough, we are data driven because the stories people don’t read or don’t get loyalty to, we skip them more and more often.
Yep, absolutely, absolutely. Finding our news online, we depend on tools like Coosto, which is so perfect for us because it knows what kind of news we are looking for, and it gives us recommendations of where to find the news or even finds news that we haven’t discovered yet so, yeah personal recommendations are very good.
Coosto works with very, very clever algorithms that’s learning by seeing what we do with the news so that is very important for us.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 223
It’s always hard to say. I am not the editor, so I am not the one who’s doing it. And on the other hand, well we use from our content and social media agency, we use (tooling) for that to make a scan of all the content which is there. Its called, I don’t know if you know these tooling its monitoring tooling, for instance we have Meltwater, but you also have (inaudible) and Coosto.
Yeah, well I personally use Tweetdeck and I think most of my colleagues also do, ((>)) and Hootsuite I used that in the past, but I that was a long time ago. But there is not a main way to do those things, every editor has their own, kind of workaround he or she prefers to work with.
Use of traditional Some outlets still make So, you know. You got to source, there is a lot of sources use of traditional institutional sources, sort of news flowing in, sources, but less so than press releases, you name it. other methods Well obviously, all the organisations which are sending us press releases, things like that. That is one. But the main sources for unique content are people in the city who knows us.
The old institutions are important for us as well, like municipalities, the airport, or the bigger companies
I would say that at least 50 percent or maybe two thirds is good old-fashioned press release and so, stuff that PR people or divisions send to you. And I would say that it is not just the press release but the whole publication, from PR to journalism. That also accounts for a call you have with someone, a spokesperson from a certain company. All that kind of contact, that communication, that’s a pretty big chunk
New tools must Some felt that despite What is different as far as you have shown me a integrate into using multiple tools couple of minutes ago, is that this could also current routines already, another serve as an inspiration for newsrooms. If I look platform where they can at this dashboard it could inspire me to write browse to discover more about one topic or another and use maybe content could fit into fragments or paragraphs from the offered text. their existing routines. [I think it’s useful because we, no one can really have enough information (exchange is important).] AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 224
I think its (somewhat) helpful, there is no such platform that is gathering all the information, so we need to search on social media A, social media B, social media C, still. If there is such kind of information where all the agencies can just exchange information, that would be very helpful.
, I don’t think so. I have never felt it that way. What I saw working at {previous company} I had several screens, I had social media, RSS, ANP, so you just skip, you look through it, and you scan, you pick up what you are interested in.
Yeah. So it has to be very attractive and very useful, to give a benefit when, if they really are going to use it on a daily basis.
That helps of course it’s something that I already now and I know how to use, so that’s clever I would say.
It looked clean and ((>)) well user friendly I would say and it is comparable for how for example the ANP press agency is offering their content.
It is also similar, if you look at my Tweetdeck then you’ll see the same with the columns of topics that I need to follow.
Yeah it reminds me of Tweet deck and I know a lot of journalists are familiar with Tweetdeck. So that’s great I think.
Yeah, I think if it looks familiar, that’s nice definitely for journalists because I have colleagues who still find it hard to work with new technology.
I think if you think about how to present things, then you sooner or later end up with the column approach.
So, make it simple, because when you make it simple, it is easy to use and you can browse through it and then move on to your next screen right? That’s how {previous company} uses it in the end. So we have a really simple way, when I talked to IT people, they show you everything that is possible, I was like “wooah this is great”, but if you bring it to the floor, they don’t care. They want it to work quick, easy to use because AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 225
every bit of more difficultly takes more time to learn and that’s the holdup.
It is nice it is helping me, but it has to help me. You don’t have to complicate things.
Yep, make it as simple as possible, and the end of the day, we made something that looked like ANP. Because they knew that, they were familiar with it, they said “oh I can relate to this”.
And gradually we put some more options in it, but we took them out just as quick when we found out, this is not what they wanted or they are not going to use it. So, it really helps when it looks like something from somebody already knows.
We had all these options to make your own feed, right? And, but they didn’t know where to start, so they had a long list of themes, of topics and behind the topics there were relevant organisations, that were putting out these RSS feeds, right?
So, we had maybe, I don’t know maybe 20, 30 topics, right? Then you think ok, which ones are relevant for me? Well, all of them? Because I look at all topics, that’s my job. Or should I focus? And, so that was something we had to guide them with, so at the end of the day I took it over from them and I actually did interviews with them.
New tools need to Participant were unsure So it is difficult to imagine a situation where address problems of weather another things would be twice as efficient or twice as dashboard could fast. If you are offered something and somebody improve their routines, if can show me a way to be twice as efficient, then they were presented yeah that would be different. with a platform that improved their current Yeah of course they can be improved, but you routines that worked, know I think that is only human. You are then they would be open satisfied with what you have, what you don’t to it. have you don’t know.
If you are of course only looking for specific topics, if you are a general journalist then that might be worthwhile, but ((<)) still it is, if only you would look at your Tweetdeck or your Hootsuite, yeah or newsfeed. You can be inspired in tons of ways, I would say. You AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 226
would have ask yourself what problem do you solve with that, what you need, and sometimes it is nice to see something else, by accident or, but will this be a reason to join TCE or?
Yeah I think that, I trust that they will build a technology that is working I would say, the real test would be in ((>)) could you create something that all sides of the eco-system, would solve a problem, and it really needs all sides. It is not only a question of solving, the problem of the freelance journalist which I think could be done quite easily. You could also have to solve the problem of the editorial room and the appliance of these editorial rooms. So there has to be a logical total and not only the separate elements.
No exactly. It would be next to that. Next to maybe the wires and the yeah. So, yeah the main question is how needed this would be for me. If you can convince me that I cannot miss this as a tool to perform my job, then it would be good.
There might also be for other platforms, for instance if you have these cities where, we call them ‘112enters’, so that’s people who chasing the accidents and the police to try and make the first pictures. That is not really our style and not really for our newspaper, but you have newspapers which thrive on that. So it might work there, but if you are offering with that kind of news first, we would be gone, so that’s the first impression you make.
But if it is people they don’t know, I am not sure if that would be the thing. The thing that might depend on what platforms they publish on, or it would help if they know it’s a unique story, or it would help if for some kind of reason, it would help if we have for instance, AD they have a lot of news about accidents that happened. We call the 112 in the news, when there’s police involved, they publish it. And for us we like to have more background stories, a bit more intelligence, like NRC or like that, and if you could add something like that, you know if its yeah kind of our profile, then it would help, I guess.
Especially now on a daily basis, you know they have other tools. You know sometimes you introduce a new tool, everyone is enthusiastic about it but in the end, its not that, well they don’t use it or nearly use it and not because they AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 227
don’t want to use it. They are used to their own workaround
Yeah I know about the idea roughly, yeah at this stage, I don’t have an opinion yet. I mostly have a lot of questions.
I make a phone call for 20 minutes and I have a new story to write, but if I want to make a video for 20 minutes, it is going to cost days.
So, it is more expensive, so if that is what they want. So if you can deliver, lets go.
Editors have an I think that all parts of the work that could be open attitude done with automation that should be done towards automated like that. There are of course parts that are journalism expensive, and that some point they can do more than a computer can. They are creative and they can find those subtle human elements and because we have every year fewer of them, while there is more work to be done, they have work to the automisation (sic). I am very open and positive to it. I think in the world there are now many examples that shows us that it can work.
I think it could be beneficial, because we are working with it in a test setting and especially for the football and following the smaller leagues. But I think they have to do lots of work to get it to function well and come to a publishable text.
I don’t really have an opinion about it, it’s more does the reader like it? That is most important for me, I guess with sports I think well, it might work, a bit. But there is also an added value of the editor, I guess. So, in the beginning it should be checked maybe, to see if the quality is good enough, and after that when all the algorithms work et cetera, I guess it could be a solution, but not the solutions, because it misses the feeling which is also important in an article. Instead of only needing statistics, I guess sports journalism now, for robo-journalism would mainly be statistics.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 228
Well I would say that, there is a part and maybe a larger part than we can imagine right now, or we hoped for, I think can actually be written by smart algorithms, or robot journalists, it depends on how you call them. But because in the end it is, a lot of the content put out by news sites, news outlets, it is in a way almost mechanical.
I said say that I think it is, the actual output by robots compared to the output by human journalists is still, well its very small. It may have been my perspective from the way I use content myself.
So its very useful I think. So, why not? You know? Yeah if I can use the good journalists to do the good journalistic work, and have everything automated that can be automated, why not? You know? So like, you know sports, ((>)) well the numbers, sports numbers. Like ((>)), the agendas, you can have it automated by robots, why not, so I am not against it, not at all.
Well we could discuss it for half an hour, but in short I am a real believer in the principle of robot journalism. ((<)) Why is that? Because the biggest broadcast in Holland NOS has stopped an experiment, saying that it doesn’t bring us anything, its only useful in financial information and sports information, and sports information is being bought by a robot in way that people can see it is robotised news and they said, well if this is collected from certain facts the audience can find the facts themselves and they don’t need a robot to make a good text out of it.
Editors are willing Participants I hope so, well I would try to convince them to to source demonstrated openness do so, because ((<)) I would try to stress automatically overall to source articles importance of the quality of the content not the written content that were automatically (source) itself. Whether it’s a robot or a human written. However being, or whatever. If the quality is what we are concerns about topic looking for, then please use it. and source remain. No no, I would say it would be judged equally compared to the other articles. For us it would always be very important to know the source.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 229
Oh that wouldn’t make, it could make the difference if about what sort of content it is. Automatically written could be just small pieces in your newspaper or your site. More in your newspaper, on the website small isn’t very useful because, there is not much space on your site or app. Everybody thinks that newspapers hasn’t much space, and online is never ending, but no your site, people don’t scroll 2 metres down. ), it could be useful, but only for small things I guess.
It could work if the quality is ok and I guess it is not easy to make the quality overall, good enough. But at the end of the day it is simple as that really good stories are shared there. It is not very good, or it does not match the standard of the news organisation, then it’s difficult, not going to work.
Ah yeah, of course. I would use it for my unique piece, yep definitely.
I would use the automatically generated content because then I would save me time, because the robot made the script, and until it’s over I get the script and then that makes my unique story and that’s how I would work with it.
I don’t think it makes any difference, in the process of buying things or not buying things, whether or not it has been written by a person or a robot. Considering that fact, that the quality of the articles should be perfectly the same.
I think it is all about the quality of the article.
So if it is good, I don’t see a reason why not. But I do wonder whether these types of articles show any originality. That is often expected from journalists.
I think the journalist will use it, I think he will use it more as advise and not will use the story itself, but to start to bring their own story, or to investigate a topic.
Yeah, I think so. (There was) a study at the University of Tilburg where a sports desk, they did some interesting stories about the psychological effects on automatically written articles. Our confidence is, well their was not much difference between, when it was written by AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 230
a journalist or when it was written by a robot, or computer or how you may call it.
Well, I think so because we again, there is your routine where those journalists, where, well it is similar to the routines they are used to.
Yeah, actually I think I would source them sooner than getting content from my biggest competitor.
Well because then in that case, if I publish it my competitor doesn’t feel “hey look at that man, there he is buying and publishing our stuff”.
If it is written by a bot, actually I would say it would enlarge the USP of the Content Exchange.
Yes, if I could know, so, yep. I am not against robot-journalism, I have to know.
Yeah, why not? Yes, it could be one of the sources for stories for us, why not?
Well learning that people, the audience, for the audience it is more and more difficult to see which one is written by software and which one is written by a journalist. So, again if you ask the audience when they are satisfied, they are satisfied. If you ask the journalist, it probably has a date, so “well I am important here in the process”, and some of my colleagues are afraid I think, of losing their jobs. They’re going to embrace it because its cheaper.
News outlets are More recent Journalism hasn’t adopted a fraction of what slow to react to technological changes could be useful to journalism. If you look at the technological within the newsroom are field of Artificial Intelligence and of machine change slow to be adopted learning, and what {name} is working on, robot- within the newsroom. journalism. But also, VR and AR. Those are technologies that can be way more useful for journalism and for the audience of journalism that can be used right now.
So, there is a long way to go, actually for my inaugural speech five years ago, I said that there were two positive things in journalism at this moment, one is society, society needs journalism, that we perform it in the right way. The other one is technology, there is so many opportunities there for us journalists. Especially in legacy publishing houses. ((>)) Yeah, much more can be done. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 231
Yeah and some of them are working on it. For example, if you look at the national broadcasting team at NOS. they have quite a good innovative team, and really nice things come up there. You can also for example as far as data-journalism is concerned there lots of initiatives that are quite promising. But still its well, too little I would say.
Yes of course, because you know like 15 or 20 years ago everybody said the newspaper would be dead around now. But we are still printing them, so that’s one concern, we have to keep it alive for the next 20 years. And at the same time we have to invent the newsroom, to make all the digital content. So some of it is the same, in paper. But under the digital, the public has different demands, like video
Well at my time at {previous company}, and even afterwards looking around, you still see journalism doing all the things they did, journalists doing the things they did. Making their own phone calls, writing text, finding their own pictures, and now putting it online, where it used to be a newspaper for instance. ((<)) There were only in the editorial rooms, where you could use new kind of tools to make everything more efficient right?
I do think so, I am convinced that journalism needs to and will step across the traditional boundaries of the journalistic field. AI is one obvious field that we have to cover or conquer or ((<)) however you want to say it.
But there are way more fields, 10 or 15 years ago when it became obvious that a company like Google or Facebook used algorithms in a way that was really affecting journalism, the field of journalism. We kept saying it is not our business, let them do it and let people find out that it is rubbish and let’s stay away from that. I think we ((<)) start feeling an awareness that this was not the most cleverest decision that we made.
Because I know the working field has a place where mainly the focus is on the now, we got to fill this site, paper has to be filled. So you talk innovation within media companies, its complicated, difficult, and if you look at innovation, for instance {previous company}, after I left they started {previous company name} lab. So they started using tools online for AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 232
their news product, and that’s it. But something as disruptive as the automated newsroom, robotics, that’s difficult to grasp for them, so I am happy with what {colleague} is doing, and slowly I think we are seeing what field can adapt to that, but its slowly, like always in journalism.
There is a need for Participants seemed There are companies especially in the United more uncertain about where States that are fully occupied with robot- technological automated journalism journalism, with building the automated development could go, or felt that newsroom. But if you look at the Netherlands, the technology had to well there have been some experiments, some ended successfully some others not so much, it is be developed further still in an atmosphere of experiment. to fully use it. So, there are similar things in the Netherlands, but the ones I know are almost all template- based, with some variety with synonyms and things to make it a little more dynamic, but they are not really natural language generated articles.
Yeah, yeah, yeah, and of course the Netherlands has a small market anyway, so if you’re promoting such technologies then it’s more difficult to get a return on investment.
But to really generate text via computers in the Dutch language, you do not see that too often. I know it’s being used in the English language. Yep. Our language area is too small for that currently, or of course will develop. But it takes longer, than in English.
For example, here in the Netherlands we need an algorithm that can write Dutch.
Yeah, but I am not aware of anyone who actually, there are some experiments at {parent company} with bots, but that’s it.
I don’t know, I don’t know. Because maybe everyday I am reading robotised articles which I am not aware of that they are written by a robot, I don’t know. I am not sure if it has the future, I am really not sure. It’s an experiment that has been started in the United States many years ago already and it wasn’t too successful there, as my information goes. Maybe you have different information. So I am not too enthusiastic about it.
And you are talking about robot-journalism on the news floor. ((>)) I don’t know really, I think AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 233
it can help, but I don’t think it can make the unique content, I would like to read.
It’s still a bit vague for me how we can use it, and we are trying to find out where we can use it in our process, and well its still, I am hearing about it for years and years and now its still, well at the point where I can use it for this, or I can use it for that. So, I am not sure.
Yeah, like I mentioned, I would not at this moment. If I relate it to your platform, it would not be the way to, ((<)) get to know it. Because robo-journalism looks like an experiment, it needs to be developed and when its developed, it could be part of the platform. But since its not well known yet, and not proven yet, I would start by just offering it separate as a unique project or something.
It can only grow, there is only one way and that’s the growing importance of these robot-like sources.
But its like with many innovations, it takes more time, some of the enthusiastic in the beginning think it will take. Because its not only a question of technology it’s a question of also and maybe more even so a question of culture.
Its difficult to say, also because on the local markets you wont find many customers. We are a big company, with other newspapers, and you have one which is on local radio, {company name}. (It many not fit I guess) and there is not much money on the market. I am not saying it can’t work, but it will be really hard to make a living out of it.
I don’t know, I don’t know. Because maybe everyday I am reading robotised articles which I am not aware of that they are written by a robot, I don’t know. I am not sure if it has the future, I am really not sure. It’s an experiment that has been started in the United States many years ago already and it wasn’t too successful there, as my information goes. Maybe you have different information. So I am not too enthusiastic about it.
Well, it probably goes ((<)), it will probably grow but it will probably make bigger leaps when there are breakthroughs with algorithms, for this particular practice.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 234
It’s a difficult one to predict, I am looking at this from a positive point and saying, we can fill {previous company website}, NU.nl by an automated newsroom.
Right? You that is maybe something in 5 to 10 years. After that I am not sure, I think software will be able to write stories like a correspondent is doing right now.
So, for the near future, 5 to 10 years, you will see an automated newsroom, but partly filling NU.nl and {previous company website} and after that…
Remains to be seen and I hope in the meantime, we start a very good debate about robot- journalism in an automated world.
Editors are Editors want to think in the end, everybody, journalists included stubborn determine content has their own web of trust. They have their own themselves and may be circle of sources that they like, that they trust and too stubborn to have that they regard highly. content pushed to them, instead of being Yeah they are (in a way) persistent, they know automatically reco what’s best, nobody should tell them, how to mmended. make an article, what’s best, et cetera. So this is something from outside, and they always like to work with their own freelancers, with their own ideas, and something from the outside. Always when we talk with someone from the outside, they kind of (hold) and reject it.
Yeah, recommendations, not really these type of people, the editor are not really much open in my opinion, for things like that. Recommendations are more for travel websites, et cetera, or maybe even if these are known names, people they know then a recommendation helps.
And that’s everybody what like to have, their own unique content. So that is one important issue, and the other issue I guess is well, editors are pretty how do you call that? They go their own way, they know what’s best.
It’s pretty hard to become part of their routine. They should ((<)), they might look at it once or twice, but if there’s nothing there which they could use, they’re not going to use, they are not going to look at it again, because its not part of the routine.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 235
Yeah, well I know this kind of dashboard, from other (companies). And my experience is that editors, that there are editors who might use it. But is there very individual choice for editors, as I already told you, everyone has his or her own workaround, and it is difficult to change those workarounds they are used to.
But then I would say that the number of players is so small, that they probably won’t do it, because well they are too proud or something.
It could, but for there’s a difference. When I’m only a reader its ok for me when someone chooses. When I am a news organisation, I want to make the choice and I don’t want to have it done by an algorithm.
News outlets must Outlets may feel time Well, the thing of course is I am only able to pay balance their or resource pressures, my reporters a certain amount. I am not able to available preventing them from pay them for doing research for like a week or resources being able to two weeks and then come up with a story. That is the limit, but it is also the limit we accepted to implement new tools. have.
Of course I would prefer to have all of my reporters go away for two weeks and then come back with the best story ever. But it is not feasible, but they have to write like, well one or two stories every day and ((<)) I can’t afford to give them more time.
Yeah, that’s always an issue ((<)), we try to get them to make news, but we like to bring more background stories, and/or research. That’s a problem because ((<)) there’s enough funding at the moment, it’s a bit harder but it still survives, so ((<)) but we don’t have a enough money to do a good enough background research.
And that’s something which is missing. But I think that’s a problem for everybody in journalism, accept maybe for the national newspapers. Everybody is trying to see what they could do about it. But still so far I think nobody has an answer
Yeah? Ok, you would suggest or you would think they would cover the whole of {region}. But actually, if you are going to analyse their news, its mostly the area of {city}. But it is {city} because they work and live there, the people who work there. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 236
Yep, very bad. I am not very keen on {company}. I have worked their as well, yeah it was 20 years ago, and I still know the people there, they are very kind and friendly. But, very {city} oriented.
Yeah, because they live there and that’s how they consume their media and therefore that is how they see the province, but it is much bigger than that.
What you do is, it depends on what kind of news you are looking for. If you work for the local media, then your source, you look in the local newspapers. They all look to each other actually, they all do. That is what I see and now as well. If I make a story now with a journalist, I can see 3 or 4 calls from other journalists. So what they do is they all look at each other. It would be very ((<)), the best answer to say because that is how they teach it, is “yeah I am going into the community and I speak with people and I am going to find my news”, but to be honest that is mostly not how they work, because they don’t have the time to do some research or invest, or just hang out in the city and see what is going on. So mostly, newspapers are generated by what they receive.
So, there might be an opportunity, I know the main focus is on video. If you look at {large media company} because they are able to write stuff, right? They have had 100 years for that, right? But they want more video content And the video content is expensive to make. I make a phone call for 20 minutes and I have a new story to write, but if I want to make a video for 20 minutes, it is going to cost days. So, it is more expensive, so if that is what they want. So if you can deliver, lets go.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 237
Automated Many participants’ Yeah, its an experiment, and you need to have journalism organisations were part the right data, also of course. experiments of larger media groups. These media groups We don’t have the tools yet and I know that we were mostly are part of {parent company}, they are trying to experimenting with make steps on it, and it will be within a years. automated journalism, with mixed results Oh yeah we had that kind of not really into the practical area, we mostly see that kind of experiments.
I think it was, there were examples, there seems to be also in the financial part, we used a lot of figures and compared to an article written by people I think its mostly contains the same information, yeah its almost the same. The audience just have different opinions. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 238
Automated news Automatically written I think that depends on the topic, a simple one depends on news news can only be used something like weather reports or report for topic for certain, factual news some basic figures I think is trustworthy. I think topics. its concerning about a social issue or something that people have different perspectives on I might choose, articles written by people.
[The next steps is that more and more will be done by the computer, starting by using very simple and short parts of text concerning for instance simple football matches = that are not enough for the big story.]
I don’t really have an opinion about it, its more does the reader like it? That is most important for me, I guess with sports I think well, it might work, a bit. But there is also an added value of the editor, I guess. So, in the beginning it should be checked maybe, to see if the quality is good enough, and after that when all the algorithms work et cetera, I guess it could be a solution, but not the solutions, because it misses the feeling which is also important in an article. Instead of only needing statistics, I guess sports journalism now, for robo-journalism would mainly be statistics.
Maybe on sports journalism, because what we do now we have a very small section of sports journalism, we only follow the main hockey fields, field hockey is quite a big sport in the Netherlands. We follow our top team, and we follow football, FC {regional team}, the top team, and that’s about it. In these articles you need a feeling and what’s happening, so you need the editor, for extra added value. Maybe, in the what is it called? The wijken, do you know that word? De wijk? [The region, the very local = Suburbs, lets say suburbs. Maybe we could use it there, because more football clubs et cetera, people like to read about it also about the scores also, more statistics. So, there’s a purpose for it, yeah and there it might be useful and that is what we would like to give it a try there
It really depends on what kind of article, because if you the correspondence, you know that one for us, that type of article would be very interesting. But if you start offering, sports news or the ((<)) main news which you see everywhere, yeah (inaudible). So unique should be the key word, I guess.
AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 239
Yeah so I think for these types of small applications, its fine, its great.
Yeah, its still a very hard question to answer I think. Definitely (still a problem). I think for instance people who write about sports, I know the example you just mentioned through {name}, who write about sports. I think this is great. But at other editorial offices, this might not work, I would also be concerned about journalistic platforms who want to have an exclusive article.
That’s a reason why we started with sports, because that is the kind of journalism with a lot data.
[Yeah I think it could be beneficial, because we are working with it in a test setting and especially for the football and following the smaller leagues. But I think they have to do lots of work to get it to function well, and come to a publishable text.]
Of course, but for me that would depend on the topic, or the content. But the layout is helping, its not a barrier or I would say so ((<)) yeah.
: I think the next within a few years is to have the small news, like football matches maybe also the short police news, crime news, car accidents, things like that can be done automatically. Its not such a big step, it happens everywhere. But I think that the top stories will always be done by reporters, why? Because your reader wants to identify with it and maybe at first glance, they won’t see the difference, and maybe they won’t identify with a computer. But that will take some years. But for now it is good to see the face of someone.
So, you know. You got to source, there is a lot of institutional sources, sort of news flowing in, press releases, you name it. So that can be, yeah up to a point automated, And I would say where you transcend, this basic level of information ((<)) presentation, so relate to the realms of opinion, or context, you know I would say there is still room for people. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 240
Robot authors Participants feel that Still journalism is a profession that is well, cannot replace automated journalism special and different from other professions. Not human ones. can not replace humans any non-journalist can feel or be or act like a entirely and plays more journalist. You need the professional journalism of a support role to as well as the technology. make their work routines more efficient. I think its beneficial if you just need to use some figures or basic information, like time, date and place.
I think that is enough. And if you want some in- depth reporting like, investigation or like that then that part then people are irreplaceable.
Yep. It can help you to make better use of the reporters you have, to have them write things that are more valuable to your readers.
Oh yes, sure. You know for some, (publishing the news) is based on data, which is almost every news. There are just a few ways that you can write it down. Either you are a human, or you are a computer. And those sort of passages, its not a place where you make a big difference for your readers, so if it helps me to make better use of the reporters that we have, I will be happy to use it. I don’t think that you should use it to replace your reporters.
I think that all parts of the work that could be done with automation that should be done like that. There are of course parts that are expensive, and that some point they can do more than a computer can. They are creative and they can find those subtle human elements. And because we have every year fewer of them, while there is more work to be done, they have work to the automisation. I am very open and positive to it. I think in the world there are now many examples that shows us that it can work.
Yeah, the who, what, why, how, when. You, know? There’s a fire? There’s a fire, where? How? What? And when? That’s ok, but if you want to add something ((>)), if you want to add some special content then it is not going to work.
Well say factual information, because maybe I am wrong. I said factual information, but when for example a press agency says its doing factual information as well. That is what they do. Can we say ok we don’t use press agencies, we just put robots in the news agency. I think that is how it is going to work, because you still have to do research work. AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 241
Well we could discuss it for half an hour, but in short I am a real believer in the principle of robot journalism. ((<)) Why is that? Because the biggest broadcast in Holland NOS has stopped an experiment, saying that it doesn’t bring us anything, its only useful in financial information and sports information, and sports information is being bought by a robot in way that people can see it is robotised news and they said, well if this is collected from certain facts the audience can find the facts themselves and they don’t need a robot to make a good text out of it.
And I would say where you transcend, this basic level of information ((<)) presentation, so relate to the realms of opinion, or context, you know I would say there is still room for people.
Yep, and of course robots can be useful, but with helping to write your story as well. So not only that it can totally take over your job, but it could help you with writing your story.
Well of course yes, but you have to use your brain to come to the right questions, to come… everything starts with the right (amazement), when you are amazed about something, or you are struck by something and you think how, how could this be? I don’t see a robot doing that, so as long as there is a robot to be programmed to think like we can, we still have to use real journalists.
Well I think that for many years, it will increase, it will not replace journalists but it will empower them to do more, for an ever higher quality, to get rid of tedious, boring work.
Well I hope that we can put our journalists into the work where they as humans are very good in. So everything what you can do automatically, it should be done automatically. So everything what repeats, should be worked with artificial intelligence, or whatever, and you can use your creativity and your knowledge to do the things, well you have a really, special gift as a journalist, so I hope this will find a way through.
Well I think it will expand, but it has to start with a lot of basic work, as I told you I handled sports journalism, but also in financial journalism. Journalism where there is a lot of data, well you can use those techniques, but I think there will be a, when it works well, there AUTOMATIC DISCOVERY OF NEWSWORTHY CONTENT 242
will be those texts, well I don’t know at this moment but I am sure they will be those texts.
I think you can be afraid of your job and then say “ooh I don’t want to work with it because it takes my job”, but then other people will investigate it and (use) their resources and it will happen anyway. You can better be part of it