Handbook of Quantitative Science and Technology Research

1 Scholarly Twitter metrics

Stefanie Haustein [email protected] 1 School for Information Studies, University of Ottawa, Ottawa, (Canada) 2 Centre interuniversitaire de recherche sur la science et la technologie (CIRST), Université du Québec à Montréal, Montréal (Canada)

Abstract Twitter has arguably been the most popular among the data sources that form the basis of so-called altmetrics. Tweets to scholarly documents have been heralded as both early indicators of citations as well as measures of societal impact. This chapter provides an overview of Twitter activity as the basis for scholarly metrics from a critical point of view and equally describes the potential and limitations of scholarly Twitter metrics. By reviewing the literature on Twitter in scholarly communication and analyzing 24 million tweets linking to scholarly documents, it aims to provide a basic understanding of what tweets can and cannot measure in the context of research evaluation. Going beyond the limited explanatory power of low correlations between tweets and citations, this chapter considers what types of scholarly documents are popular on Twitter, and how, when and by whom they are diffused in order to understand what tweets to scholarly documents measure. Although this chapter is not able to solve the problems associated with the creation of meaningful metrics from social media, it highlights particular issues and aims to provide the basis for advanced scholarly Twitter metrics.

1.1 Introduction Enabled by the digital revolution, the open access and open science movement, big data and the success of social media have shaken up the scholarly metrics landscape. Academic careers are no longer shaped only by peer-reviewed papers, citation impact and impact factors, university managers and funders now also want to know how researchers perform on social media and how much their work has impacted society at large.

Bibliometricians have started to adapt to the policy pull and technology push and expanded their repertoire of scholarly metrics to capture output and impact beyond the ivory tower, so far that some speak about a scientometric revolution (Bornmann, 2016). Metrics are no longer restricted to formal parts of communication but expand beyond the borders of the scholarly community (Cronin, 2013a). Similarly to how the Science Citation Index formed the field of bibliometric research and research evaluation, the altmetrics, or more precisely the social-media metrics landscape is being heavily shaped―if not entirely driven―by the availability of data, in particularly via Automated Programming Interfaces (APIs) (Haustein, 2016; Priem, 2014).

Twitter has arguably been at the epicenter of the earthquake that has shaken up the scholarly metrics landscape. The majority of altmetrics research has either focused on or included Twitter (see Sugimoto, Work, Larivière, & Haustein (2017) for a review of the literature). Following a general definition of scholarly metrics which include activity on social media (Haustein, 2016), scholarly Twitter metrics are defined as “indicators based on recorded events of acts [on Twitter] related to scholarly documents […] or scholarly agents […]” (Haustein, 2016, p. 416).

Although findings of an early study had suggested that tweets were a good early indicator of citations for papers published in the Journal of Medical Internet Research (Eysenbach, 2011), the generalizability of this claim was refuted by low correlations reported by more representative studies (Costas, Zahedi, & Wouters, 2015a; Haustein, Peters, Sugimoto, Thelwall, & Larivière, 2014). Low

1 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research correlations between tweets and citations did, however, spark hopes that Twitter activity was able to reflect impact on users and use beyond citing authorsa new type of previously unmeasurable impact, possibly on society at large. Twitter’s popularity in the altmetrics realm has essentially been caused by two factors, which are both heavily influenced by technology and the data push and policy pull described above:

a) a significant number of scholarly articles are shared on Twitter, producing a measurable signal; b) Twitter is a social media platform created for and used by a wide and general user base, which theoretically has the potential to measure impact on society at large.

As tweets represent an “unprecedented opportunity to study human communication and social networks” (G. Miller, 2011, p. 1814), Twitter is being used to analyze a variety of social phenomena. Centering on either the message (i.e., the tweet, its content and associated metadata) or social connections (i.e., the network of follower-followee relations), tweets have been used to show discussions during upcoming elections, how people communicate during natural disasters, political upheaval, cultural events and conferences and have even been used to predict elections outcomes and the stock market (Rogers, 2014; Weller, Bruns, Burgess, Mahrt, & Puschmann, 2014).

While only a small share of academics use Twitter for scholarly communication (Rowlands, Nicholas, Russell, Canty, & Watkinson, 2011; Van Noorden, 2014) and to diffuse scientific publications (Priem & Costello, 2010), more than one fifth of recent journal articles are being tweeted (Haustein, Costas, & Larivière, 2015), which adumbrates that it is non-academics who engage with scholarly publications on Twitter. At this point, social media-based indicators have flourished rather as vanity measuresculminating in a tongue-in-cheek metric called the Kardashian Index (Hall, 2014)than as validated indicators of societal impact. Even though altmetrics have left their mark on the scholarly publishing and metrics landscape (Desrochers et al., 2018), they have not (yet) established themselves within the reward system of science, where citations remain the only hard currency:

Neither Twitter mentions nor Facebook ‘likes’ are, for now at any rate, accepted currencies in the academic marketplace; you are not going to get promoted for having been liked a lot, though it may well boost your ego. (Cronin, 2013b, p. 1523)

Still, almost all big publishing houses now report some form of article-level metric based on social media activity, including tweets. Despite the lack of validation and a clear definition regarding the type of impact measured, the number of tweets are thus already used as scholarly metrics “in the wild” (Priem, Piwowar, & Hemminger, 2012).

This chapter aims to contribute to the understanding of Twitter and Twitter-based metrics with a particular focus on their potential and limitations when applied as scholarly metrics. To provide some context for the meaning of scholarly impact measures derived from tweets and Twitter activity, this chapter describes Twitter’s role in scholarly communication. It depicts how Twitter is used in academia and how scholarly contents are diffused and discussed in tweets. The chapter provides an overview of the literature of Twitter use by the scholarly community and scholarly output on Twitter. The latter part is supported by empirical results based on an analysis of 24 million tweets mentioning scientific papers captured by the data provider Altmetric.com. Both the review of the relevant literature and the patterns extracted from the Twitter data are intended to contribute to the understanding of what type of scholarly contents are diffused on Twitter, who is diffusing them, when and how. This will help to assess Twitter metrics as valid impact indicators and to interpret their meaning.

2 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

1.2 Twitter in scholarly communication Twitter launched in 2006 as a public instant messaging service and evolved from an urban lifestyle social network, where users would update their friends about what they were doing, to a platform for communicating news and events used by 500 million users worldwide, or 23% of US adults online (Duggan, Ellison, Lampe, Lenhart, & Madden, 2015). Although other microblogging platforms (e.g., Sina Weibo, tumblr, Plurk) exist, tweeting has become a synonym (and preferred term) for microblogging and Twitter the most popular service. Twitter constraints microposts to a maximum length of 140 characters, a restriction that originates from the 160 character limit of text messages. Users can follow each other and create user lists to manage the updates they receive from other Twitter users. Similarly to regular blogs, microblogs are ordered sequentially in reverse chronological order and, due to their brevity, usually appear more frequently (Puschmann, 2014), making Twitter the “most dynamic and concise form of information exchange on social media” (Grajales, Sheps, Ho, Novak- Lauscher, & Eysenbach, 2014, p. 5). While the brevity of tweets is seen as a restriction by some, others perceive it as a particular advantage:

The brevity of messages allows [tweets] to be produced, consumed, and shared without a significant amount of effort, allowing a fastpaced conversational environment to emerge. (boyd, Golder, & Lotan, 2010, p. 10)

Tweets have three major specific affordances, which facilitate communication on the platform: retweets (RTs), user mentions (@mentions) and hashtags (keywords following #). All of these functions originated within the Twitter user base and were eventually adopted by Twitter, representing a co-creation of functions by users and developers. Twitter provides three main levels of communication: interpersonal communication on the micro level, meso-level exchanges of people who are directly connected through their network of followers and followees, and hashtag-centered macro- level communication which enables exchanges among all Twitter users with common interests (Bruns & Moe, 2014).

In academia Twitter is used to disseminate and discuss scholarly outputs and other relevant information; maintain collaborations or find new ones; as a virtual “water cooler” (Veletsianos, 2012, p. 347) for social networking with colleagues; to increase student participation in teaching; as a backchannel at scientific conferences to foster discussions among conference attendees and those who participate remotely; as well as to increase visibility and reach wider audiences (Nentwich, 2011; Osterrieder, 2013; Pearce, Weller, Scanlon, & Ashleigh, 2010; Van Noorden, 2014; Veletsianos, 2012; Zhao & Rosson, 2009).

1.2.1 Twitter uptake Twitter is used by various stakeholders in the scholarly community, including individual researchers and academics, journals and publishers, universities and other academic institutions, as well as at scholarly conferences. From the perspective of using Twitter activity as the basis for scholarly metrics, it is essential to know Twitter uptake in academia, as it informs about biases and differences between disciplines and other user demographics, which may have a direct effect on derived metrics.

1.2.1.1 Scholars on Twitter In the scholarly context, Twitter use by academics lags behind its uptake among the general public. Although the majority of researchers are aware of the platform, most do not make use of it in a professional context, giving it the reputation of a hype medium in academia (Carpenter, Wetheridge, Tanner, & Smith, 2012; Gu & Widén-Wulff, 2011; Pscheida, Albrecht, Herbst, Minet, & Köhler, 2013; Van Noorden, 2014). A certain reluctance in academia to use Twitter might be caused by its perception as a shallow medium that is used to communicate “pointless babble” (Kelly, 2009, p. 5) rather than informative content (Rogers, 2014). Described as “phatic” (V. Miller, 2008, p. 396), Twitter is less about what people tweet rather than how they are connected. 3 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Reported Twitter uptake shows extreme variations depending on user demographics―in particular disciplinary orientation―and when a study was conducted; it usually stays behind use of other social media. For example, a survey among 2,414 researchers conducted in 2010 demonstrated that while more than three quarters used social media, less than one fifth were on Twitter (Rowlands et al., 2011). A more recent study conducted by Nature also showed that Twitter was among the least used social media platforms in academia: while almost half of 3,027 science and engineering researchers regularly used ResearchGate, only 13% regularly visited Twitter (Van Noorden, 2014). At almost one quarter of regular users, Twitter uptake was higher among the 482 social science and humanities scholars participating in the same survey (Van Noorden, 2014).

Depending on the sample and when the survey was conducted, Twitter uptake varied heavily between a few percent to more than one third of surveyed scholars using Twitter, which calls the representativeness of findings into question. Moreover, surveys vary in terms of whether or not they differentiate between general Twitter uptake, Twitter use for scholarly communication and professional purposes or active vs. passive use, which further complicates comparison and generalization of findings regarding Twitter uptake in academia. Twitter represented the social media tool with the highest difference between awareness and use. Although known by 97% of university staff in Germany, as few as 15% used Twitter and 10% used in a professional context (Pscheida et al., 2013). A similar use-to-awareness ratio was found by other studies, for example, at Finnish universities (Gu & Widén-Wulff, 2011) or researchers surveyed by Nature (Van Noorden, 2014), and academic staff in Germany (Weller, Dornstädter, Freimanis, Klein, & Perez, 2010).

Most studies found self-reported Twitter use in academia at around 15%; an uptake of 13-16% was reported for surveys based on 215 health services and policy researchers (Grande et al., 2014), 454 geographers (Wilson & Starkweather, 2014), 1,058 UK academic staff (Tenopir, Volentine, & King, 2013) and 3,027 scientists and engineers (Van Noorden, 2014), while uptake was lower (7-10%) for academics in Germany (Weller et al., 2010) and the UK (Procter et al., 2010; Rowlands et al., 2011). Although 18% used it, Twitter was the least popular social media tool of 345 European scholars (Ponte & Simon, 2011). The highest Twitter use was reported for a survey of 126 Finnish university staff at 23% (Gu & Widén-Wulff, 2011), 1,910 professors at US universities at 32%23% for professional purposes(Bowman, 2015b), 382 urologists attending a conference at 36% (Loeb et al., 2014) and 71 participants of the 2010 Science & Technology Indicators (STI) conference in Leiden at 44% (Haustein, Peters, Bar-Ilan, et al., 2014).

As an alternative to determining Twitter uptake through self-reported use in surveys, studies have also assessed the extent of scholarly microblogging based on Twitter activity of scholars. Identifying scholars on Twitter is challenging, as the 160-character Twitter bio and the provided user name are often the only basis for identification. Most studies thus search for Twitter users based on a list of names of academics (Darling, Shiffman, Côté, & Drew, 2013; Desai et al., 2012; Desai, Patwardhan, & Coore, 2014; Priem, Costello, & Dzuba, 2012; Work, Haustein, Bowman, & Larivière, 2015) or apply snowball sampling starting from a set of known scholars on Twitter (Chretien, Azar, & Kind, 2011; Holmberg, Bowman, Haustein, & Peters, 2014; Holmberg & Thelwall, 2014). Searching for a list of 8,038 US and UK university staff, Priem, Costello, and Dzuba (2012) found Twitter accounts for 2.5% of them. Although the authors admit that their study underestimated Twitter use, it reflected that the microblogging platform is not popular in academia and thus confirms finding by most surveys. Investigating Twitter use in the scientometric community, Bar-Ilan et al. (2012) found Twitter profiles for 9 of the 57 presenters at the 2010 STI conference.

Other than searching for known scholars on Twitter, some studies try to extract information from Twitter to identify scholarly users. The most common approach is to classify users based on searching for specific words in the Twitter bio. Retrieving users whose Twitter bio contained words such as 4 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research university, PhD or professor, Barthel et al. (2015) identified scientists’ Twitter accounts with a precision of 88%. False positives contained university accounts or those of non-academic staff at research institutions. Recall cannot be determined in such studies as the number of false negatives, that is the scientists on Twitter who do not include any of the queried keywords in their self-descriptions, remains unknown.

Altmetric also applies a keyword-based approach to categorize Twitter users as scientists, science communicators (journalists, bloggers, editors), practitioners (doctors, other healthcare professionals) and members of the general public. It should be noted that Altmetric’s general public category includes all users that can not be classified as belonging to any of the other three groups and is therefore not a good indicator of how much an article has been tweeted by members of the general public. An obvious limitation of the keyword approach is that it is unable to capture scholars who do not identify themselves as such or who do not use the terminology or language covered by the list of keywords. However, many scholars seem to reveal their professional personas on Twitter. Ninety percent of doctoral students funded by the Canadian Social Sciences and Humanities Research Council (SSHRC) identified as academics on Twitter (Work et al., 2015), 87% of surveyed US university professors claimed to mention both their professional title and place of work in their Twitter profiles (Bowman, 2015b) and 78% of Twitter users who self-identified as a physicians used their full names (Chretien et al., 2011). This willingness to reveal their scholarly identities on Twitter suggests that scholars make use of the microblogging platform in a professional context at least to some extent.

Ke, Ahn, and Sugimoto (2017) took advantage of crowdsourced Twitter lists to identify scholars. Based on a method introduced by Sharma, Ghosh, Benevenuto, Ganguly, and Gummadi (2012), they identified scientists on Twitter with an approach based on membership in scientific Twitter user lists. Other studies have estimated Twitter activity by scholars by analyzing users who engage with scholarly content on Twitter. Hadgu and Jäschke (2014) applied machine learning to automatically identify scholars on Twitter based on a training set of users whose tweets contained a computer science conference hashtag, while others selected users who have tweeted scientific papers (Alperin, 2015b; Haustein, Bowman, & Costas, 2015a; Haustein & Costas, 2015b; Holmberg & Thelwall, 2014; Tsou, Bowman, Ghazinejad, & Sugimoto, 2015). Since the latter type of studies focuses on categorizing who is tweeting about scholarly contents rather than estimating Twitter uptake in academia, these studies are discussed in more detail below.

1.2.1.2 Scientific conferences Twitter does particularly well in fostering communication among people participating in shared experiences (Rogers, 2014), which may be why tweeting at scholarly conferences has been one of the earliest and most popular uses of Twitter in academia. Almost every scientific conference today has a specific hashtag to connect attendees and those interested but not able to attend in person, thus expanding the conference audience to include remote participants (Bonetta, 2009; Sopan, Rey, Butler, & Shneiderman, 2012; Weller, Dröge, & Puschmann, 2011). Apart from increasing the visibility of presentations, tweeting at scientific conferences has introduced another level of communication, creating backchannel discussions online among participants complementing presentations and discussions which take place at the meeting. Conference tweets usually directly refer to presentations and discussions during sessions and sometimes summarize key take-away points (Chaudhry, Glode, Gillman, & Miller, 2012; McKendrick, Cumming, & Lee, 2012; Mishori, Levy, & Donvan, 2014). Other motivations to tweet at a scientific conferences were to share information and learn about discussions in parallel sessions, networking with others and feeling a sense of connectedness, as well as note-taking (McKendrick et al., 2012). A significant number of tweets associated with two medical conference were uninformative or promotional (Cochran, Kao, Gusani, Suliburk, & Nwomeh, 2014; Desai et al., 2012).

5 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Due to the ease of collecting tweets with a particular hashtag, as well as Twitter’s relative popularity in the context of scientific conferences, there are countless studies analyzing scholarly Twitter use based on tweets with conference hashtags (Chaudhry et al., 2012; Cochran et al., 2014; Ferguson et al., 2014; Hawkins, Duszak, & Rawson, 2014; Hawkins et al., 2014; Jalali & Wood, 2013; Letierce, Passant, Breslin, & Decker, 2010; McKendrick et al., 2012; Mishori et al., 2014; Reinhardt, Ebner, Beham, & Costa, 2009; Sopan et al., 2012; Weller et al., 2011).

Similar to the overall uptake among scholars, Twitter activity at conferences differs among disciplines as well as individual conferences and increased over the years. Overall, only a small share of conference participants contributed to discussions on Twitter: Less than 2% of attendees of the American Society of Nephrology's 2011 conference (Desai et al., 2012) and less than 3% of participants of the 2012 Winter Scientific Meeting of the Association of Anesthetists of Great Britain and Ireland tweeted (McKendrick et al., 2012). Another medical conference in 2013 reported higher Twitter engagement, as 13% of conference attendees tweeted using the conference hashtag (Mishori et al., 2014). Longitudinal studies also observed an increase in Twitter activity at conferences over the years (Chaudhry et al., 2012; Hawkins et al., 2014; Mishori et al., 2014). For example, 2% of conference participants tweeted at the 2010 Annual Meeting of the American Society of Clinical Oncology, while 5% contributed to conference tweets in 2011. Similarly, the number of tweets nearly doubled from 4,456 tweets in the first to 8,188 in the following year (Chaudhry et al., 2012). A similar increase was observed for the 2011 and 2012 annual meetings of the Radiological Society of North America (Hawkins et al., 2014). Conference-related discussions on Twitter are not restricted to in-person attendees. In fact, at some conferences the majority of Twitter users only participate remotely (Sopan et al., 2012).

Just as with other social media and information in general, tweeting activity is usually heavily skewed with a few users contributing the majority of tweets at conferences (Chaudhry et al., 2012; Cochran et al., 2014; Mishori et al., 2014). Tweeting about a conference has been shown to lead to an increase in the number of followers regardless of attending in-person or remotely. Follower counts grew particularly for speakers and in-person attendees, while the number of followers grew least for remote participants (Sopan et al., 2012). Most organizers of scientific conferences embrace the potential of increasing visibility and outreach and thus encourage tweeting through a conference-specific hashtag. Some also specifically display conference-related tweets in real time and thus make tweeting activity visual to participants who are not on Twitter (Ferguson et al., 2014; Jalali & Wood, 2013; Sopan et al., 2012; Weller et al., 2011).

1.2.1.3 Journals and publishers Twitter’s technological features afford direct connections and two-way conversations between users changing what was traditionally known as a unidirectional sender-audience relationship. Opposed to traditional publishing and mass media, Twitter has given rise to personal publics of audiences (Schmidt, 2014). This direct link between the sender and receiver has changed the relationship with audiences; for example, musicians use Twitter to market their own brand and respond to @replies from fans to seek out in-person interactions (Baym, 2014). TV audiences turn Twitter into a virtual lounge room when they connect with other users discussing TV events in real time (Harrington, 2014). Similarly, discussions of scientific publications can now happen publically, when readers share their opinions on Twitter. A specific use case are Twitter journal clubs, an adaption of small-group in-person journal clubs that are common particular in the medical sciences (Leung, Siassakos, & Khan, 2015; Mehta & Flickinger, 2014; Thangasamy et al., 2014; Topf & Hiremath, 2015; Whitburn, Walshe, & Sleeman, 2015). Twitter journal clubs are used to discuss and review recent publications and educate researchers and practitioners; in the medical sciences they also have the advantage over their offline predecessors to directly involve patients (Mehta & Flickinger, 2014). Often these journal clubs are initiated or at least supported by journals to promote their publications. A journal club initiated by a

6 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research gynecology journal showed that discussing papers and making them freely available has boosted their Altmetric scores (Leung et al., 2015). Twitter journal clubs also motivated authors of discussed papers to create Twitter accounts (Thangasamy et al., 2014).

With journals and authors on Twitter, readers can get in touch directly and involve them in discussions using @mentions, tearing cracks in the wall of traditional gatekeeping, as “Twitter makes it possible to directly connect journal readers at various stages of training with authors and editors” (Mehta & Flickinger, 2014, p. 1317). Many journals and publishers have started to use Twitter as a marketing instrument to increase online visibility and promote published contents. These accounts can be used to create a personalized audience relationship and to foster interaction among readers. Similar to the mix of professional and personal interactions by academics on Twitter, the lines between scholarly communication and marketing campaign are blurred for accounts maintained by journals and publishers. Almost half of the 25 general medicine journals with the highest impact factor in 2010 had a Twitter presence (Kamel Boulos & Anderson, 2012), while Twitter uptake was lower for other sets of journals: 24% of 33 urology journals (Nason et al., 2015), 2 of the top 10 ophthalmology journals (Micieli & Micieli, 2012), 16% of 100 Web of Science (WoS) journals (Kortelainen & Katvala, 2012) and 14% of 102 journals specialized in dermatology (Amir et al., 2014) maintained an account. As most of these studies focused on the top journals according to the journal impact factor, Twitter uptake might be biased towards high-impact journals and slightly lower when including others. The variation suggests similar differences between disciplines as observed for Twitter uptake by individual scholars and conferences.

While most journal accounts are used to share articles and news (Zedda & Barbaro, 2015) and often tweet the article title (Friedrich, Bowman, Stock, & Haustein, 2015; Thelwall, Tsou, Weingart, Holmberg, & Haustein, 2013), some journals have incorporated tweeting into the formal communication process. In addition to regular abstracts, they ask authors to write so-called tweetable abstracts that meet the 140-character restrictions, which are used to attract readers on Twitter (Darling et al., 2013). Twitter even interfered with the journal’s role in scholarly communication, when a genomics paper was criticized and corrected results posted in a tweet, leading to a conflict with the authors of the criticized paper (Woolston, 2015).

Even if a journal is not represented by a proper Twitter account, it is likely that its publisher is. Zedda and Barbaro (2015) found that Twitter adoption was particularly common among 76 publishers in science, technology and medicine; 89% had official Twitter accounts, exceeding the presence on any other social media platform, and 74% had embedded tweet buttons that allowed readers to directly share publications on Twitter. Promotion of publications by publishers seem to be welcomed by authors, as a survey by Nature Publishing Group revealed that almost one fifth of authors would consider it a very valuable service if publishers promoted papers using marketing and social media (Nature Publishing Group, 2015).

As shown in the analysis of Twitter accounts diffusing scientific articles below, accounts maintained by journals and publishing houses are responsible for a significant amount of tweets mentioning scientific articles. Once Twitter metrics are being used to evaluate journal impact, these types of self- tweets might be considered as a type of gaming in a manner similar to journal self-citations and citation cartels to boost the impact factor (Seglen, 1997; Van Noorden, 2013). With publishers invested in the success of their journals, tweet cartels and tweet stacking in analogy to their citation equivalents are easily conceivable and even easier to implement. While the WoS excludes journals from the Journal Citation Reports, which have been caught increasing their impact factors artificially, companies like Altmetric.com and Plum Analytics do not (yet) intervene in such self-promotional activity.

7 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

1.2.1.4 Universities and academic libraries Scholarly institutions are affected by Twitter’s impact on academia on two levels: they exploit the microblogging platform to increase their visibility (and that of their members) and provide guidelines and recommendations for their members to navigate the new communication space. The Association of American University Professors updated their report on Academic Freedom and Electronic Communication (Association of American University Professors, 2013) in reaction to a university rescinding a tenure-track job offer to an English scholar who had made an anti-Semitic comment on Twitter (Herman, 2014). The updated report emphasized that professors enjoy academic freedom even when they comment on social media and particularly addressed the blurring of boundaries between private and professional opinions on social media. It stressed how, in this new context, comments are particularly prone to be misunderstood and misinterpreted, as they are often taken out of context:

Electronic communications can be altered, or presented selectively, such that they are decontextualized and take on implicit meanings different from their author’s original intent. With the advent of social media such concerns about the widespread circulation and compromised integrity of communications that in print might have been essentially private have only multiplied further. (Association of American University Professors, 2013, p. 42)

The report further recommends that universities and other academic institutions, along with their staff, develop policies that address the use of social media. In general, academic institutions lack specific social media guidelines or address social media in policies. Although more and more institutions adopt specific policies (Pasquini & Evangelopoulos, 2015), only half of US doctorate-granting universities had a social media policy, while rates were even lower for other universities and colleges. At the same time, Twitter was specifically mentioned in more than 80% of policies (Pomerantz, Hank, & Sugimoto, 2015).

The majority of university Twitter accounts apply a so-called megaphone model of communication, where news and information concerning the institution are broadcasted following a traditional communication model (Kimmons, Veletsianos, & Woodward, 2017; Linvill, McGee, & Hicks, 2012). Universities use Twitter for public relations, dissemination of news and events, as well as recruitment (Kimmons et al., 2017). Ninety-four percent of 474 US university admission officers reported that their institution had a Twitter account (Barnes & Lescault, 2013) and 96% of the websites of 100 US colleges linked to Twitter (Greenwood, 2012). On the departmental level, it was less common to be represented with an organizational account, as only 8% of 183 US radiology departments had a Twitter presence (Prabhu & Rosenkrantz, 2015). Twitter was also commonly used for faculty development at medical schools (Cahn, Benjamin, & Shanahan, 2013). Analyzing the Twitter activity of 29 Israeli universities and colleges, Forkosh-Baruch and Hershkovitz (2012) found significant differences between both types of institutions. Colleges were more likely than universities to post social tweets. While almost half of the tweets by universities focused on research conducted elsewhere, colleges focused more on reporting the work by its own researchers.

Twitter was also frequently used by academic libraries as a marketing instrument, to communicate with patrons, to announce new resources and promote services (Boateng & Quan Liu, 2014; Hussain, 2015; Shulman, Yep, & Tomé, 2015). On par with Facebook at a 63% adoption rate, Twitter was the most commonly used social media platform among 38 surveyed academic libraries from different countries (Chu & Du, 2013), while all of the 100 US university libraries analyzed by Boateng and Quan Liu (2014) maintained a Twitter presences. In Canadian academic libraries Twitter adoption was lower at 47% (Verishagen & Hank, 2014).

8 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

1.2.2 Twitter use Apart from Twitter uptake, scholarly Twitter metrics are further influenced by how Twitter is used. Academics use social media to share information, for impression management and to increase their visibility online, to network and establish a presence across platforms, to request and offer help, expand learning opportunities, or simply to be social (Veletsianos, 2012). Twitter specifically was used mostly to tweet work-related content, discover peers working on similar research, follow research-related discussions and get recommendations for papers (Van Noorden, 2014).

One central motivation for scholars to tweet is to communicate and explain their work to lay people. As many science communicators are active on Twitter, they help to bridge the gap between the scholarly community and the general public. Science communicators were the largest user group of 518 Twitter users mentioned in tweets by 32 astrophysicists (Holmberg et al., 2014). An evolutionary biology professor valued Twitter to communicate his work to the general public:

Twitter and regular blogging are more effective than anything else I do to publicize a paper, which was really surprising to me […]. If you do it right, Twitter is an effective way of telling people about your work. (Bonetta, 2009, p. 453)

However, most researchers still preferred traditional media over Twitter to promote their research (Wilkinson & Weitkamp, 2013).

Even when identifying professionally on Twitter, a large share of tweets by scholars are not related to their work or academia in general (Bowman, 2015b; Haustein, Bowman, Holmberg, Peters, & Larivière, 2014; Mou, 2014; Pscheida et al., 2013; Van Noorden, 2014). The Nature survey found that 21% of scientists who used Twitter regularly did not use it professionally and 28% said that they never posted content about their work (Van Noorden, 2014). Bowman (2015a, 2015b) reported that, while 29% of American university professors used Twitter strictly in a professional way and 42% used it for both for personal and professional reasons, the vast majority of tweets were coded as personal (78%) rather than professional (19%). Again, large variations can be observed between disciplines as well as individual Twitter users (Chretien et al., 2011; Holmberg & Thelwall, 2014; Loeb et al., 2014; Mou, 2014; Priem, Costello, et al., 2012; Work et al., 2015). Examining more than half a million tweets from 447 researchers, Holmberg and Thelwall (2014) found that less than 4% of tweets were classified as scholarly communication and results varied between disciplines ranging from less than 1% for sociology up to 34% for biochemistry. A study on emergency physicians’ tweeting behavior found that 49% of their tweets were related to health or medical issues, 21% were personal, 12% self-promotional and 3% considered unprofessional as they contained profanity, were discriminatory or violated patient privacy (Chretien et al., 2011). In a sample of tweets by funded doctoral students in the social sciences and the humanities in Canada, 4% of tweets were related to their thesis, 21% to the discipline and 5% to academia in general, while 70% of tweets were coded as non-academic (Work et al., 2015). Personal use also prevailed among 382 urologists (Loeb et al., 2014).

These findings highlight that even when scholars identify professionally on Twitter and use the platform for scholarly purposes, many tweets will be irrelevant to scholarly communication and should thus be excluded from a scholarly indicator perspective:

The lack of a dividing line between scientists and non-scientists, as well as the great variety of topics that even scientists tweet about mean that Twitter is not comparable to the orderly world of science publishing, where every piece of information is assumed to be relevant. Instead, a typical user’s timeline is likely to be populated both by scholarly content and personal remarks, more or less side by side. (Puschmann, 2014, p. 98)

9 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Due to their brevity and the fact that when analyzing tweets they are often taken out of context, categorizing tweet content is as difficult as it is to classify Twitter users (Bowman, 2015b). Distinguishing between scientific and non-scientific tweets is especially challenging (Holmberg & Thelwall, 2014).

Large variations can also be found between individual tweeters in terms of how often they tweet. A group of astrophysicists analyzed by Haustein et al. (2014) tweeted, on average, between 0 and 58 times per day. Tweets to scientific papers have been shown to peak shortly after their publication and decay rapidly within just a few days. For example, 80% of arXiv submissions received the largest number of tweets the day after they were published (Shuai, Pepe, & Bollen, 2012). Similarly tweeted half-life was 0 days for papers published in the Journal of Medical Internet Research (Eysenbach, 2011) and 39% of a sample of tweets linking to a scholarly document referred to those published within one week before (Priem & Costello, 2010). Determining the delay between publication and first tweet as well as half-lives on Twitter is, however, challenging due to the ambiguity of publication dates (Haustein, Bowman, & Costas, 2015b).

Since not all Twitter use culminates in a tweet, a large share of activity remains invisible and thus unmeasurable. In fact, passive use prevailed among UK doctoral students on Twitter (Carpenter et al., 2012) and seems to be common for scholarly use of social media in general. While most academics access and view information, only a minority actively contributes by creating content on social media (Procter et al., 2010; Tenopir et al., 2013); less than 2% of 1,078 UK researchers surveyed actively contributed daily (Tenopir et al., 2013). 1.2.2.1 Tweeting links The most frequent use of Twitter among researchers in higher education was to share information, resources or media (Veletsianos, 2012). A survey among US university professors revealed that embedding URLs was the most commonly used Twitter affordance. Half of the survey participants claimed to tweet links either sometimes, mostly or always (Bowman, 2015b). Links are a common way to send more information than 140 characters would fit. Addressing the length limitation, a scholar explained:

It is a double-edged sword. The majority of my tweets are pointers to other resources, so there is a headline—an enticement in other words—and a link to the resource. You don’t need more than 140 characters for that. (Bonetta, 2009, p. 453)

Weller and Puschmann (2011) refer to links in tweets as “external citations” (Weller & Puschmann, 2011, p. 2). Studies about scholars on Twitter show that they make frequent use of tweeting URLs, as the share of tweets with links exceeds that observed for general Twitter users (boyd et al., 2010; Harrington, 2014). About one third of 68,232 tweets sent by 37 astrophysicists (Haustein, Bowman, Holmberg, et al., 2014) and 38% of 22,258 tweets posted by Canadian social sciences and humanities doctoral students contained links (Work et al., 2015). Tweeting links was even more popular among scholars studied by Weller and Puschmann (2011) and emergency physicians analyzed by Chretien et al. (2011), as respectively 55% and 58% of their tweets contained URLs. Links were much more common when a sample of 445 US professors tweeted professionally: 69% of professional tweets contained URLs, while only 15% of personal tweets did (Bowman, 2015b). Tweets with the #www2010 and #mla09 conference hashtags linked to a website in 40% and 27% of the cases, repeating each unique URL less than three times (Weller et al., 2011).

Priem and Costello (2010) found that 6% of a sample of 2,322 of tweets by academics containing a URL mentioned a scholarly publication, 52% of which were first-order and 48% second-order links (i.e., via another website) to the document. Similarly, Holmberg and Thelwall (2014) found that scholarly tweets frequently contained a link to scholarly publications via a blog post about the paper.

10 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

First-order links were significantly more likely to refer to open access articles (Priem & Costello, 2010). Tweets containing the #iswc2009 conference hashtag linked to applications (e.g., online services or research projects; 31%), the conference website (21%), blog posts (12%), slideshows (12%) and publications (9%) (Letierce et al., 2010). Blogs were the most common linked resources in conference tweets analyzed by Weller and colleagues (2011), while news websites were a frequent link destination of tweets sent by Canadian doctoral students, even when discussing scholarly topics (Work et al., 2015).

When linking to scholarly papers, tweets often contained the paper title and rarely expressed any recommendation or sentiment (Friedrich, Bowman, Stock, et al., 2015; Thelwall et al., 2013). The great majority of tweeted articles were published very recently (Eysenbach, 2011; Holmberg & Thelwall, 2014; Priem & Costello, 2010). According to surveys asking for motivations to use social media, finding relevant publications and staying up-to-date with the literature was found to be a frequent, yet passive, use of Twitter (Van Noorden, 2014). A Columbia university professor in biology and chemistry describes how they used Twitter to be alerted about the literature:

Sometimes four or five people I follow will mention a paper that I did not come across and I will look it up. I think I am much more up to date on science literature since I started following Twitter. (Bonetta, 2009, p. 452)

A study by Tenopir, Volentine, and King (2013) found that academics on Twitter read more scholarly publications, which seems to confirm the use of Twitter as a publication alert service. At the same time, as the analyses by Priem and Costello (2010) and Letierce et al. (2010) show, the share of tweets linking to academic papers is low, suggesting a rather passive use: scholars follow links to tweeted articles but do only infrequently distribute them themselves. This suggests that a significant part of Twitter use cannot be captured by scholarly Twitter metrics. 1.2.2.2 Retweets Retweets represent a specific form of diffusing information, as users forward messages sent by others. As such, they do not represent an original contribution by the retweeting user. Since retweets directly quote another users text, they can be seen as “internal citations” (Weller & Puschmann, 2011, p. 3) on Twitter. An analysis of retweets demonstrates how information circulates within a specific user community (Paßmann, Boeschoten, & Schäfer, 2014). A common disclaimer that retweets do not equal endorsements adapted from early Twitter use by journalists, emphasizes that tweets are forwarded to increase information diffusion. Once a frequent part of Twitter bios, the disclaimer has now been established as common sense and is no longer needed (Warzel, 2014).

As sharing information is one of the main motivations for scholarly Twitter use, retweeting is likely to be common among tweeting academics. Conference participants interviewed by Letierce and colleagues retweeted “tweets that are close to their interest or tweets that speak about their own work or research project” (Letierce et al., 2010, p. 7). Studies showed that retweeting is less common than other affordances used by scholars on Twitter but exceed expectations of a random sample of tweets in 2009, in which as few as 3% were retweets (boyd et al., 2010). Between 15% and 20% of tweets sent at scientific conferences were retweets (Letierce et al., 2010); similarly, 15% of 68,232 tweets by a group of 37 astrophysicists were retweets (Haustein, Bowman, Holmberg, et al., 2014). Retweeting was more common among 28 academics analyzed by Priem and Costello (2010), as 40% of tweets were retweets. Similarly, 37% of 43,176 tweets by Canadian doctoral students were retweets, while 10% of their tweets were retweeted (Work et al., 2015). At the same time, the share of retweets at a radiology conference was 60% (Hawkins et al., 2014). Being asked about Twitter affordance use, 14% of US professors said that they mostly or always and 34% that they sometimes retweeted (Bowman, 2015b). These professors were more than twice as likely to retweet when their tweets were classified

11 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research as professional rather than personal (Bowman, 2015b). These differences again demonstrate that tweeting behavior differs depending on who is tweeting and it what context.

The majority of retweets sent by scholars from ten disciplines contained links, while conversational tweets (i.e., @mentions) were less likely to contain links.

This clearly shows that researchers […] frequently share web content and forward information and content they have received from people they follow on Twitter, while links are not that often shared in conversations. (Holmberg & Thelwall, 2014, p. 1035)

Links to papers were significantly less likely to be retweeted: while 19% of tweets with links to scholarly publications were retweeted, the retweet rate was twice as high in the overall sample of tweets analyzed by Priem and Costello (2010). This is in contrast to a random sample of more than 200,000 tweets, over half of which contained a URL (boyd et al., 2010). A random sample of 270 tweets linking to scientific journal articles found that many were modified retweets of tweets originating from the journal’s own Twitter account (Thelwall et al., 2013).

From the perspective of scholarly metrics, a distinction should arguably be made between tweets and retweets, as the latter reflects a rather passive act of information sharing (Holmberg, 2014). Although with each retweet the visibility of the tweet and the information it contains (e.g., the link to a publication) increases, retweets represent diffusion of information rather than impact. As retweeting requires even less effortas little as one click since the implementation of the retweet buttonthan composing an original tweet, Twitter metrics should distinguish between tweets and retweets to reflect these different levels of user engagement (Haustein, Bowman, & Costas, 2016). 1.2.2.3 Followers, @mentions and @replies Scholars on Twitter actively seek new connections and connect others (Veletsianos, 2012). Twitter is built in a way that information is spread via user networks. This means that users receive updates from those they follow and diffuse their messages to those they are followed by, creating a personal public (Schmidt, 2014). Selecting who to follow and who one is followed by are thus essential to communicating on Twitter. Twitter users build a reputation based on both their number of followers and followees. A parallel can be drawn to authors’ citation identity and citation image based on which authors one cites and is cited by (White, 2000, 2001).

The purpose of expanding one’s social network and finding peers was apparent in a study of 632 emergency physicians on Twitter: those who included work-related information in their Twitter bios had more followers and the most influential users in the network were connected to at least 50 other emergency physicians (Lulic & Kovic, 2013). Similarly, conference participants on Twitter saw a significant increase in their number of followers, particularly when they were also presenting (Sopan et al., 2012). The average number of followers of 260 physicians analyzed by Chretien et al. (2011) was 17,217 with a median of 1,426, indicating the typically skewed distribution of followers among Twitter users.

In addition to broadcasting one’s message via retweets on a meso-level of communication, Twitter users can also directly address users with @mentions or @replies on an interpersonal level. Both replies and mentions thus represent a particular type of tweet that focuses on conversation rather than broadcasting (Honeycutt & Herring, 2009). Like most Twitter affordances, these types of tweets were also developed by the user base before being implemented by Twitter. While @replies happen in response to a tweet and are only visible on the timeline of the tweeter who sent the original tweet, @mentions refer to tweets that contains another user’s Twitter handle, which triggers a notification to inform them about being mentioned (Bruns & Moe, 2014). About one third of tweets in 2009 included another user name (boyd et al., 2010). A random sample of tweets without an @mention were mostly

12 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research about the tweeting user’s experience, while those with an @mention were more likely about the addressee. In fact, more than 90% of @mentions functioned to address another user, while 5% worked as a reference (Honeycutt & Herring, 2009).

The great majority of tweets sent by social sciences and humanities doctoral students referred to other Twitter users, as 72% of the 43,176 tweets sent contained other user names (Work et al., 2015). Conversational tweets were also popular among a group of tweeting astrophysicists. Of the 68,232 tweets 46% were @replies or @mentions (61% including RTs), making it the most frequently used Twitter affordance. Conversational tweets were particularly common among those who tweeted regularly or frequently (Haustein, Bowman, Holmberg, et al., 2014). Most of these mentions referred to science communicators (24%), other astrophysicists (22%) or organizations (13%) on Twitter (Holmberg et al., 2014).

Conversational tweets hardly contained links (Holmberg & Thelwall, 2014), and among tweets linking to publications, only 8% were @replies (Priem & Costello, 2010). Opposed to hashtag use, retweeting and embedding URLs, @mentions were the only affordance that were less likely to occur in professors’ professional tweets (56%) than those identified as personal (67%), which suggests that when professors discuss their work, they are less likely to address or reference other users directly than when they tweet about private matters. However, mentioning other users was still more common in their professional tweets than retweeting and using hashtags (Bowman, 2015b). 1.2.2.4 Hashtags Similar to retweets and @mentions, hashtags are a user-driven Twitter affordance. Hashtags are keywords following the #-sign, which facilitate connections between users interested in the same topics. Conversations revolving around hashtags represent the macro layer of Twitter communication (Bruns & Moe, 2014). Holmberg and colleagues suggest that “hashtags may resemble the traditional function of metadata by enhancing the description and retrievability of documents” (Holmberg et al., 2014, p. 3).

Hashtag use seem to be less common than that of other Twitter affordances among academics. Sixty- one percent of surveyed American university professors declared that they rarely or never used a hashtag (Bowman, 2015b). This might be because scholars are either less familiar with this Twitter- specific affordance or they do not wish to expand conversations beyond their personal publics defined by their follower networks. Although actual hashtag use was low by the US professors analyzed by Bowman (2015b), they were more likely to use hashtags in their tweets identified as professional (28%) than those coded as personal (17%).

An early large-scale study found that among a random sample of 720,000 tweets, only 5% contained a hashtag (boyd et al., 2010). Almost one quarter of tweets by astrophysicists contained a hashtag (Haustein, Bowman, Holmberg, et al., 2014), but hashtag use varied significantly among different clusters of the follower network (Holmberg et al., 2014). The same share of hashtags (25%) was found for tweets of SSHRC funded doctoral students; on average each hashtag was mentioned 2.8 times (Work et al., 2015). It is problematic to infer hashtag use from most other studies on scholarly tweets, as data collection itself is often based on a specific hashtag, such as a conference hashtag (e.g., Letierce et al., 2010; Weller et al., 2011).

1.2.3 Reluctance against and negative consequences of using Twitter When using tweets as the basis to measure scholarly impact of any sorts, it is essential to consider who is not on Twitter and why academics might be reluctant to join the microblogging platform. Twitter is a platform where “content is not king” (V. Miller, 2008, p. 395) and has been perceived as “shallow media, in the sense that it favors the present, popular and the ephemeral” (Rogers, 2014, p. xiv). Early Twitter studies which identified the majority of tweets to be “pointless babble” (Kelly, 2009, p. 5) or

13 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

“daily chatter” (Java, Song, Finin, & Tseng, 2007, p. 62) casted doubt on the value of Twitter as a meaningful communication medium (Rogers, 2014). This reduction to banal content has led many in academia to consider tweeting a waste of time and therefore rejecting Twitter as a means of scholarly communication (Gerber, 2012). Particularly the 140-character limit of tweets has many scholars doubt Twitter’s usefulness for research. This may be why Twitter is one of the best known and at the same time least used social media platforms in the scholarly community (Carpenter et al., 2012; Gerber, 2012; Gu & Widén-Wulff, 2011; Pscheida et al., 2013; Van Noorden, 2014).

The adoption of new technologies is often met with functional and psychological barriers. Lack of time and skills as well as a negative perception of platforms have been identified as barriers to use social media in academia (Donelan, 2016). Reluctance often stems from the notion that tweeting wastes precious time and introduces challenges that come with the blurred boundaries of professional and personal communication. This mix of professional and personal identities on Twitter specifically has been revealed by many studies and has been identified as a potential reason not to use Twitter in academia. Even though in the general public, Twitter uptake is higher among young adults, it is often early career researchers who are more reluctant to tweet about their work (Bulger et al., 2011; Coverdale, 2011). Young academics feel the highest pressure to publish in high-impact journals and limit their time spent on social media, and feel more vulnerable when publicly exposing their ideas, particularly to uncertain audiences (Coverdale, 2011; Harley, Acord, Earl-Novell, Lawrence, & King, 2010; Mccrea, 2011; Veletsianos, 2013). An academic interviewed about the future of scholarly communication expressed their concern regarding the use of Facebook and Twitter for work purposes:

There’s this research group in my area and, for some reason, they’re really into Facebook. So they want to do a lot of discussions on Facebook and that type of thing. But I really just don’t have time. It’s like Twittering. I just can’t…There needs to be a little bit of space where I can actually think about something. And I think for some people, they’re just wired in such a way that they like that constancy, and they are also able to actually say something intelligent quickly. And I’m not like that. I have to be a little bit more deliberate and think about things a little bit more. And so I can’t Twitter…I need some time to reason… (Harley et al., 2010, p. 97)

However, early-career researchers were often more likely to find social media useful in the context of scholarly communication and collaboration (Grande et al., 2014; Gruzd, Staves, & Wilk, 2012; Tenopir et al., 2013). Bowman (2015b) identified a u-shaped relationship between Twitter use and academic experience: US professors seven to nine years into their academic careers were more likely to use Twitter compared to those with fewer and more years of experience. A humanities scholar addressed the issue of blurred boundaries on Twitter:

I think it can be distracting, especially to grad students, when they’re trying to navigate, when they’re needing to learn, adopt, and use these new technologies, but at the same time learn to discriminate among technologies that are more for social things but are being used in the name of research. The lines are too blurry. (Bulger et al., 2011, p. 59)

The tension that arises between scholars who take to Twitter and those who are reluctant to discuss scholarly matters on social media is demonstrated by an incident where a genomics paper published in PNAS was criticized on Twitter for flaws in study design and analysis casting doubt on its conclusions. The tweet sent by a genetics researcher at the University of Chicago included charts and tables from a re-analysis of the PNAS paper’s data, which he published in the open access and open-peer review journal F1000Research. The tweet and re-analysis provoked many responses on Twitter and in the comment section of the F1000Research paper, some of which demanded the retraction of the PNAS paper (Woolston, 2015), while the PNAS authors accused the critic of violating the norms of science by taking to Twitter.

14 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

The clashing of personal and professional and the fuzzy boundaries between the two has also lead to severe negative consequences for scholars. Tweets by faculty have caused outrages among students, other faculty members, university administration and the public at large. Identifying professionally on Twitter has affected the academic careers of some scholars and, in a certain case, controversial tweets have provoked death threads (Rothschild & Unglesbee, 2013). Tweeting had serious effects on the career of a tenured University of New Mexico psychology professor, who had fat-shamed students on Twitter: “Dear obese PhD applicants: if you didn’t have the willpower to stop eating carbs, you won’t have the willpower to do a dissertation #truth.” The professor was asked to apologize and had to undergo sensitivity training, while his work was monitored and he was banned from working on the graduate students admission committee for the rest of his career (Bennett-Smith, 2013; Ingeno, 2013; Pomerantz et al., 2015).

In another incident, an English professor lost his tenure-track position offer from the University of Illinois at Urbana-Champaign due to a tweet that was interpreted as anti-Semitic (Herman, 2014). In response to withdrawal of the job offer by the University of Illinois, the American Association University Professors updated their report on academic freedom to reinstate that academic freedom applies to comments made on social media (Association of American University Professors, 2013).

1.3 Scholarly output on Twitter As this chapter discusses scholarly Twitter metrics, it focuses on how scientific papers are diffused via tweets. Although, as described above, only a small amount of scholars’ tweeting activity involves linking to publications (Letierce et al., 2010; Priem & Costello, 2010), tweets to journal articles represents one of―if not the―most popular altmetric. This might be due to the significance of publications in peer-reviewed journals in scholarly communication as well as the ease at which tweets that mention or link to document identifiers (e.g., the Digital Object Identifier DOI) can be retrieved. Another reason that Twitter-based altmetrics―and, in fact, all altmetrics―gravitate towards journal articles is that they aim to complement existing bibliometric measures, which reduce scholarly output in a similar manner.

In the following, the altmetrics literature is reviewed to provide an overview of currently used scholarly Twitter metrics for journal articles. As the majority of available studies only scratch the surface of what can potentially be extracted from Twitter activity, the literature review is complemented by an analysis of tweets collected by Altmetric.com. This analysis goes beyond tweet counts and correlations with citations and aims to reflect the What, How, When, Where and Who of scholarly publications shared on Twitter. This includes what types of documents are tweeted, how Twitter affordances such as hashtags, retweets and @mentions, are used to share them, when and where articles are tweeted and who is tweeting them. Although this chapter is not able to provide solutions to problems associated with creating meaningful metrics from tweets, particular limitations and pitfalls are highlighted, while demonstrating the potential of available data. Together with the findings on Twitter use described above, the chapter tries to provide context to help the interpretation of different metrics. This chapter thus aims at improving the understanding of what Twitter-based scholarly metrics can and cannot reflect.

1.3.1 Data and indicators The analysis of tweets that mention scholarly documents is based on Twitter data collected by Altmetric.com until June 2016, which contains 24.3 million tweets mentioning 3.9 unique documents. Altmetric started systematically collecting online mentions of scholarly publications in 2012 and is a particularly valuable data source for tracking Twitter activity related to scholarly output, as it continuously stores tweets that mention scholarly publications with a DOI. Through accessing tweets through the Twitter API firehose, Altmetric circumvents the usual issues of Twitter data collection that researchers are confronted with when using the freely available Twitter APIs. While the Twitter Streaming API limits access to a random sample of 1% of tweets, the Representational State Transfer 15 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

(REST) API is rate-limited and the Search API restricts access to only the most recent tweets relevant to a particular query (Gaffney & Puschmann, 2014). As Altmetric started data collection in 2012, Twitter activity is incomplete for documents published earlier.

Table 1 Scholarly Twitter metrics associated with scholarly documents

Type of Scholarly Twitter Description metric metric Twitter coverage Percentage of documents with at least one tweet Number of tweets Sum of total number of tweets Tweets Twitter density Mean number of tweets per document Twitter intensity Mean number of tweets per tweeted document

Share of retweets Percentage of tweets that were retweets Retweets Retweet density / Mean number of retweets per document / tweeted intensity document

Number of users Unique number of users associated with a document Mean number of users per document / tweeted User density / intensity Users document Mean number of Mean of the number of followers of users tweeting a followers document

Hashtag coverage Percentage of documents with at least one hashtag Unique number of hashtags associated with a Number of hashtags document Hashtags Hashtag frequency Sum of total number of hashtag occurrences Share of hashtags Percentage of tweets with at least one hashtag Hashtag density / Mean number of hashtags per document / tweeted intensity document

@mention coverage Percentage of documents mentioning a user name Number of mentioned Unique number of users mentioned in tweets users associated with a document @mentions @mention frequency Sum of total number of @mentions @mention density / Mean number of @mentions per document / tweeted intensity document

Tweet span Number of days between first and last tweet Number of days between publication of a document Time Tweet delay and its first tweet Twitter half-life Number of days until 50% of all tweets have appeared

Altmetric’s Twitter data is matched to bibliographic information from WoS using the DOI. This match between document metadata and tweets affords the possibility to determine the amount of scholarly output that does and does not get tweeted. The link to WoS data also provides access to cleaner and extended metadata of tweeted documents, such as the publication year, journal, authors and their affiliations, and a classification system of scientific disciplines. At the same time, the match of the two databases also excludes tweets to publications not indexed in WoS and thus comes with the known restriction and biases of WoS coverage. This is why the following analysis describes results for two datasets, containing, respectively all 24.3 million tweets covered by Altmetric (dataset A), and 3.9 million tweets mentioning documents with a DOI, covered by WoS and published in 2015 (dataset B). The number of unique documents in dataset A is based on Altmetric ID. As Altmetric.com’s metadata is based on multiple sources, one publication might be treated as two documents, particularly if it has

16 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research multiple versions, such as a journal article on the publisher’s website and a preprint on arXiv. Similar duplications are possible but less likely in dataset B, which is based on unique identifiers and cleaner metadata in WoS.

In the following, a set of descriptive indicators are used based on tweets to scholarly documents and associated metadata. Table 1 provides an overview of each metric. As described above, the focus here is on journal articles but the metrics can, nevertheless, be applied to any scholarly document or other research object such as scholarly agents including “individual scholars, research groups, departments, universities, funding organizations and others entities acting within the scholarly community” (Haustein, Bowman, & Costas, 2016, p. 376). Similar to most bibliometrics, the metrics described in Table 1 can be applied to any aggregated set of documents, such as all documents relevant to a certain topic, published in the same journal, by the same author, institution, country or in a specific language.

1.3.2 What scholarly output is tweeted? As shown above, only a small share of tweets sent by scholars actually link to scholarly output. Tweets linking to blogs or other websites are often more frequent than those linking to scientific publications. At the same time, the great majority of altmetric studies on Twitter focus on peer-reviewed scientific journal articles with a DOI, which represent one of the main limitations of currently captured altmetrics (Alperin, 2013; Haustein, 2016; Haustein, Sugimoto, & Larivière, 2015; Taylor, 2013).

In comparison to other common altmetrics, Twitter is the platform which exhibits the second largest activity related to scientific papers, following the social reference management platform Mendeley (Costas, Zahedi, & Wouters, 2015b; Haustein, Costas, et al., 2015). The disciplinary differences in Twitter uptake described above are equally visible, and likely resulting in, large difference in Twitter coverage between scientific disciplines. In the majority of studies, usually between 10% to 30% of selected documents were mentioned on Twitter at least once (Alhoori & Furuta, 2014; Andersen & Haustein, 2015; Costas et al., 2015a; Hammarfelt, 2014; Haustein, Costas, et al., 2015; Haustein, Larivière, Thelwall, et al., 2014; Haustein, Peters, Sugimoto, et al., 2014; Priem, Piwowar, et al., 2012). As the majority of tweets linking to scholarly papers occurs immediately after publication (Eysenbach, 2011; Shuai et al., 2012) and Twitter activity increased annually, Twitter coverage increases by year of publication, with the most recent papers being more likely to be tweeted and older papers hardly getting shared on Twitter (Haustein, Peters, Sugimoto, et al., 2014). For example, while more than half of 2015 PLOS papers were tweeted at least once (Barthel et al., 2015), Twitter coverage was at 12% for those published in 2012 (Priem, Piwowar, et al., 2012). Similarly, 13% of 2011(Costas et al., 2015a), 16% of 2011-2013 (Robinson-García, Torres-Salinas, Zahedi, & Costas, 2014) and 22% of 2012 WoS documents (Haustein, Costas, et al., 2015) had received at least one tweet, while Twitter coverage increased to 36% for WoS 2015 papers in dataset B (see Table 2 below).

Coverage varied between disciplines and journals, but also between databases and geographic regions. Twitter is blocked in countries like Iran and China, which reflects on the visibility of their authors and papers. For example, the share of tweeted papers was low for papers published by authors from Iran (Maleki, 2014). Such geographical biases affect Twitter visibility and need to be taken into account when comparing Twitter impact of documents, authors and institutions from different countries. For example, as few as 6% of Brazilian documents published 2013 and indexed in SciELO (Alperin, 2015a) and 2% of a sample of Iranian papers covered by WoS had been tweeted (Maleki, 2014). On the contrary, at 21% Twitter coverage of Swedish publications seems to be more in line with general findings (Hammarfelt, 2014). As the sample of Iranian publication included documents published between 1997 and 2012 and 98% of tweeted papers were published between 2010 and 2012, both geographical and publication date biases influence Twitter coverage.

Particularly high coverage was found for arXiv submissions (Haustein, Bowman, Macaluso, Sugimoto, & Larivière, 2014; Shuai et al., 2012). This high activity was, however, not caused by high Twitter 17 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research uptake in the physics, mathematics and computer science communities, but created by Twitter bots (Haustein, Bowman, Holmberg, et al., 2016). The extend of such automated rather than human activity and its implications for the meaning of scholarly Twitter metrics are further discussed below.

While Twitter coverage describes the extent to which a set of documents gets diffused on Twitter, Twitter density (i.e., the mean number of tweets per paper) reflects the average tweeting activity per document. In general, each paper receives less than one tweet on average, with large variations between disciplines (see below). As Twitter density is influenced by Twitter coverage and thus particularly low when only a small share of papers gets tweeted, Twitter intensity reflects the average tweeting activity for tweeted papers only, excluding non-tweeted papers (Haustein, Costas, et al., 2015). The number of tweets per paper is usually heavily skewed, much more than citations. For example, 63% of tweeted biomedical papers were only tweeted once (Haustein, Peters, Sugimoto, et al., 2014).

Figure 1 Number of tweets and users per tweeted document for datasets A and B. A similar distribution can be observed for the two datasets described above (Figure 1). The 24.3 million tweets in dataset A mentioned 3.9 documents (based on Altmetric ID), 43% of which were tweeted once, 19% twice, 10% three times, while only 4% of documents received more than 25 tweets. The document with the most Twitter activity was mentioned 35,135 times by 144 users, with one user tweeting 34,797 times. This highlights that the number of distinct users per document might be a better proxy of diffusion than the total number of tweets.

Tweets in dataset A were sent by 2.6 million users and link to 3.9 million documents, amounting to a Twitter intensity of 6.2 tweets per tweeted paper. The number of users is based on unique Twitter handles included as Author ID on Source in Altmetric’s Twitter data. Since the data does not include Twitter’s unique user ID, users might be counted more than once if they changed their Twitter handle. Very likely caused by the 140-character limitation, less than 1% of tweets link to more than one document. As shown in Figure 1, the distribution of number of users per document is even more skewed with 47% of all documents tweeted by a single user only. The distribution for WoS 2015 papers (dataset B) is similar but slightly less skewed for documents (based on WoS identifier) mentioned in one tweet and by one user only; 35% and 38% of documents were tweeted once or by one user, respectively. The total number of 3.9 million tweets were sent by 601,290 users mentioning 548,841 documents, which corresponds to a Twitter coverage of 36%, intensity of 7.2 and a density of 2.6 tweets per document. 1.3.2.1 Disciplines and journals Twitter activity varies among disciplines and even journals of the same field. As there is no gatekeeping in Twitter, these variations might not be entirely due to actual impact of a particular journal but can be heavily influenced by individuals and marketing strategies of publishing houses or other stakeholders. Despite these variations, studies have shown that multidisciplinary and biomedical journals as well as social science publications are particularly visible on Twitter, while the so-called hard sciences are tweeted about less (Alhoori & Furuta, 2014; Andersen & Haustein, 2015; Barthel et al., 2015; Costas et al., 2015b; Fenner, 2013; Hammarfelt, 2014; Haustein, Costas, et al., 2015; 18 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Haustein, Larivière, Thelwall, Amyot, & Peters, 2014; Haustein, Peters, Sugimoto, et al., 2014; Knight, 2014; Priem, Piwowar, et al., 2012).

Previous findings are corroborated by the analysis of discipline and journal-based tweeting activity in dataset B. As shown in Table 2, over one third of 2015 documents in WoS have been tweeted, which represents a significant increase compared to papers published in the previous years. Twitter coverage shows large variation between disciplines from 59% of publications in Biomedical Research, Health and Psychology to less than 10% in Mathematics and Engineering & Technology. This upholds previous findings that Twitter activity is particularly elevated around publications from the biomedical and social sciences (Costas et al., 2015a; Haustein, Costas, et al., 2015).

Table 2 Dataset B: Twitter activity for WoS articles published in 2015.

Papers 2015 Tweeted papers Tweets Users

Discipline N N N N

DOI

Twitter

coverage coverage

Density Intensity % RTs Intensity All disciplines 2,014,977 76% 548,841 36% 3,960,431 2.6 7.2 50% 601,290 6.6

Biology 124,402 73% 33,945 37% 226,575 2.5 6.7 54% 52,235 4.3 Biomedical Res. 226,011 84% 112,470 59% 1,025,061 5.4 9.1 50% 229,851 4.5 Chemistry 158,929 90% 32,829 23% 81,739 0.6 2.5 34% 13,860 5.9 Clinical Medicine 663,481 64% 223,641 52% 1,784,438 4.2 8.0 51% 288,226 6.2 Earth & Space 96,792 91% 25,616 29% 136,732 1.6 5.3 51% 42,641 3.2

Engineering Engr. & Tech. 253,020 88% 18,441 8% 50,577 0.2 2.7 33% 17,235 2.9

Natural Sciences & Sciences Natural Mathematics 51,240 85% 2,890 7% 17,441 0.4 6.0 43% 7,826 2.2 Physics 128,766 93% 16,783 14% 55,295 0.5 3.3 30% 14,822 3.7

Arts 18,995 19% 566 15% 1,962 0.5 3.5 39% 1,134 1.7 Health 51,535 74% 22,662 59% 191,530 5.0 8.5 52% 60,090 3.2 Humanities 76,998 39% 4,601 15% 19,150 0.6 4.2 52% 9,784 2.0 Prof. Fields 54,109 72% 14,760 38% 93,936 2.4 6.4 47% 41,449 2.3

& Humanities Psychology 40,657 78% 18,735 59% 145,767 4.6 7.8 51% 50,974 2.9

Social Sciences Sciences Social Social Sciences 70,042 76% 20,902 39% 135,193 2.5 6.5 53% 51,288 2.6

It should be emphasized that Twitter activity in the Arts and Humanities cannot be generalized as DOIs are not commonly used in these disciplines. While in general 76% of all documents had a DOI, as few as 19% of all WoS-indexed journal articles in the Arts were linked to this unique identifier. DOI use does not only differ between disciplines but also by country or language of publication, which is why results may be biased in these regards as wells. Considering National Science Foundation (NSF) specialties with more than half of its papers having a DOI, Twitter coverage was highest in Parasitology (78%), Allergy (76%) and Tropical Medicine (70%) and lowest in Metals & Metallurgy (1%), Miscellaneous Mathematics (2%), Mechanical Engineering (2%) and General Mathematics (3%). With more than 10 tweets per document, Twitter density was highest in General & Internal Medicine (13.5) and Miscellaneous Clinical Medicine (12.3).

At a Twitter coverage of 100%, 198 of 9,340 tweeted journals had all of their documents diffused on Twitter, which strongly suggests a systematic and automated diffusion, possibly by a dedicated account maintained by the journal or publisher (see below). Among journals with at least 100 papers with a DOI in 2015, JAMA, and Biotechnology Advances had the highest Twitter density at 115.4 and 113.2 tweets per paper, respectively. With more than 10,000 unique Twitter users, PLOS ONE, BMJ, Nature, Science, PNAS, NEMJ, JAMA, Lancet, Scientific Report, Nature Communications, JAMA Internal Medicine, PLOS Biology, British Journal of Sports Medicine, Cell, Biotechnology Advances and BMJ Open were tweeted by the largest audience on Twitter (Table 3). These journals reflect large 19 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

multidisciplinary and biomedical journals with large readership, which have been previously identified as highly tweeted, possibly because they exhibit a particular relevance to people’s everyday lives (Costas et al., 2015b). Almost all of these journals maintain official Twitter accounts appearing among the three most tweeting users, which suggests that Twitter has become an important platform for journals and publishers to promote their contents. The number of unique Twitter users, in particular for the general science journals seems to be low in comparison to global readership. For example, while Nature claimed to have 3 million unique visitors per month (Nature Publishing Group, 2012), as few as 42,365 Twitter users mentioned a 2015 paper. However, numbers seem to be in line with print circulation (Nature Publishing Group, 2010).

Table 3 Twitter metrics for journals with the largest Twitter audience based on number of users (≥10,000 users) including three most active users. Journal accounts are highlighted in bold.

Tweets Users

Top 3 most active users Journal N N (according to number of tweets;

Twitter Twitter

%RTs

coverage

Density official journal accounts are marked in bold)

Intensity

PLOS ONE 70% 148,494 5.2 7.4 47% 59,210 aprendedorweb, uranus_2, PLOSONE BMJ 91% 149,874 45.3 49.7 66% 57,622 bmj_latest, bookapharmacist, npceorg Nature 100% 85,523 89.0 89.3 56% 42,365 Doyle_Media, randomshandom, TheRichardDoyle Science 99% 78,141 70.0 70.8 67% 39,225 sciencemagazine, PedroArtino, rkeyserling PNAS 89% 72,312 19.9 22.4 54% 36,496 abbrabot, EcoEvoJournals, uranus_2 NEMJ 99% 74,263 75.2 75.9 62% 33,142 medicineupdate, NEJM, JebSource JAMA 95% 68,420 115.4 121.1 64% 31,018 JAMA_current, robarobberlover, ehlJAMA Lancet 98% 45,752 69.9 71.6 65% 25,673 medicineupdate, darmtag, TheLancet Scientific Reports 56% 41,177 4.6 8.3 50% 23,395 SciReports, uranus_2, geomatlab Nature communications 78% 39,540 11.8 15.1 52% 21,274 NatureComms, Kochi_Study, PatrickGoymer JAMAInternalMed, JAMA_current, JAMA Internal Medicine 91% 48,882 99.8 110.1 69% 20,197 GeriatriaINNSZ PLOS Biology 99% 24,481 90.0 91.3 70% 12,878 PLOSBiology, PLOS, nebiogroup BJ of Sports Medicine 97% 26,121 75.7 77.7 69% 11,618 BJSM_BMJ, exerciseworks, SportScienceNI Cell 96% 23,519 38.2 39.9 57% 11,081 Brianxbio, CellCellPress, topbiopapers Biotechnology Advances 55% 14,823 113.2 205.9 67% 10,331 robinsnewswire, GrowKudos, ElsevierBiotech BMJ Open 76% 16,882 11.5 15.1 60% 10,090 BMJ_Open, LS_Medical, SCPHRP

Going beyond peer-reviewed journals indexed in WoS, other sources (based on Altmetric metadata) of scholarly documents are also frequently shared on Twitter. In fact, the largest number of tweeted documents in dataset A came from arXiv: a total of 319,411 arXiv submissions were tweeted 1.1 million times by 110,134 users. As shown in Figure 2, the number of tweeted documents and unique users derivate particularly for the most popular sources. Although arXiv, PLOS ONE and SSRN are the most popular platforms according to the number of tweeted documents, Nature, The Conversation and PLOS ONE are tweeted by the largest number of users.

The majority of popular sources are peer-reviewed journals indexed in WoS, which also lead the ranking in dataset B (see above). Apart from these, links to repositories for documents (arXiv, SSRN, bioRxiv) or data (figshare, Dryad) as well as to website like The Conversation or ClinicalTrials.gov are also frequently tweeted. It should be mentioned that the document metadata in Altmetric is based on a variety of sources and thus shows some inconsistencies. For example, the source is unknown for 6% of tweeted documents and 4% of tweets in dataset A, and some sources appear in various spellings. Therefore, the number of distinct sources (49,379) or journal IDs (28,457) represents an overestimation of different sources. These inconsistencies can also be found in the publication year and other document metadata, which are essential to characterize what kind of scholarly output has been diffused on Twitter.

20 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Figure 2 Number of tweeted documents and unique users per source in dataset A (≥100 tweeted documents). 1.3.2.2 Document characteristics Besides disciplinary and topical differences, studies have also focused on determining what type of documents are popular on Twitter. Papers with particular high Twitter visibility often had humorous or entertaining contents rather than scientific merit (Haustein, Peters, Sugimoto, et al., 2014; Neylon, 2014). A study that coded title characteristics of 200 highly tweeted papers found that 16 included a cultural reference (i.e., proverbs, idioms, fictional characters, music) and 13 were humorous or light (Didegah, Bowman, Bowman, & Hartley, 2016). Bornmann (2014) reported that among papers recommended on F1000, those labeled as good for teaching were frequently tweeted. Andersen and Haustein (2015) found that meta-analysis and systematic reviews received significantly more tweets than other medical study types.

Marking a clear distinction from citation patterns, particularly high Twitter activity was found for document types that are usually considered uncitable (Haustein, Costas, et al., 2015). For example, Twitter coverage for news items was twice as high as that for all documents. Twitter density was highest for news items (3.0), editorial material (1.6) and reviews (1.4), by far exceeding the overall average of 0.8 tweets per document. The success of these document types on Twitter suggests that “documents that focus on topical subjects, debates and opinions, which are probably presented in simpler and less technical language, are more likely to appear and become popular on Twitter” (Haustein, Costas, et al., 2015, p. 8). Publications with shorter titles, fewer pages and references tend to receive more tweets, while the opposite tendencies are usually observed for citations (Haustein, Costas, et al., 2015). 1.3.2.3 Similarity to other types of usage Since altmetrics were proposed as an alternative or complementary to traditional bibliometric indicators, most studies analyze in how far these new impact metrics correlate with citations. The motivation behind correlation studies lies in determining whether tweeting patterns are comparable to citing behavior. Positive correlations between citation and tweets would indicate that tweets measure something similar to but much earlier than citations, making tweet-based indicators predictors of impact on the scholarly community (Eysenbach, 2011). Early studies argued that low or negative correlation coefficients would indicate a different type of impact than that on citing authors, possibly impact on society in general.

21 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

The first study analyzing the relationship between tweets and citations found a significant association between highly tweeted and highly cited papers published in the Journal of Medical Internet Research, as highly tweeted publications identified 75% of those which were later highly cited (Eysenbach, 2011). However, this claim was based on 55 papers in a journal which itself maintained a strong Twitter presence. For a set of 4,606 arXiv submissions, tweets were a better predictor of early citations than downloads (Shuai et al., 2012). The generalizability of the finding that tweets were early indicators of citation impact was later refuted by a large-scale analysis based on 1.3 million documents, which found overall low correlations between tweets and citations (Haustein, Peters, Sugimoto, et al., 2014).

Correlations between tweets and citations vary between datasets due to particular differences between disciplines or journals but are overall low between 0.1 and 0.2 (Barthel et al., 2015; Costas, Zahedi, & Wouters, 2015a; Haustein, Costas, et al., 2015a; Haustein, Peters, Sugimoto, et al., 2014; Priem, Piwowar, et al., 2012). It should be noted that correlations are affected by low Twitter coverage and thus differ whether untweeted papers are excluded or included from the analysis (Haustein, Costas, et al., 2015). Instead of replacing citations as a faster and better filter of relevant publications, as was suggested in the altmetrics manifesto (Priem, Taraborelli, Groth, & Neylon, 2010), the number of tweets seem to mirror visibility on other social media platforms, in particular Facebook, rather than visibility within the community of citing authors (Barthel et al., 2015; Haustein, Costas, et al., 2015a). If nothing else, the difference between tweet and citation counts as reflected in low correlations might be due to the fundamental difference between the act of citing and tweeting (Haustein, Bowman, & Costas, 2016).

A moderate negative correlation was found comparing publication output and tweeting activity of a group of astrophysicists on Twitter, suggesting that researchers who tweet a lot focus their efforts on communication and outreach rather than publishing peer-reviewed articles (Haustein, Bowman, Holmberg, et al., 2014). This inverse relationship between a researchers’ standing in the scholarly community and their visibility on Twitter has led to the so-called Kardashian index, a tongue-in-cheek indicator that reveals that those who tweet more publish less and vice versa (Hall, 2014).

Rather than correlating tweets and citations Allen, Stanton, Di Pietro and Mosley (2013) aimed to measure the effect of promoting articles on social media (including Twitter) on usage statistics. Comparing the number of views, downloads and citations of randomly selected articles published in PLOS ONE before and after promoting them on social media, article views and downloads increased significantly but citations one year after publication and social media metrics did not.

1.3.3 How is scholarly output tweeted? Not the least due to the evaluation community’s focus on counts, altmetrics research has focused much less on tweet content than on correlations and other quantitative measures. Among those looking at tweet content, the focus has been on the analysis of Twitter specific affordance use (Bowman, 2015b; Weller et al., 2011). This includes in particular the use and analysis of hashtags, retweets and @mentions, which are further described below.

Analyzing 270 tweets linking to journal articles, Thelwall et al. (2013) found that 42% contained the title of the article, 41% summarized it briefly and 7% mentioned the author. As few as 5% explicitly expressed interest in the article. While sentiment was absent in the great majority of tweets, 4% of tweets were positive and none negative. Similarly a large-scale study that automatically identified sentiments in tweets using SentiStrength found that the majority of tweets were neutral and that, if sentiment was expressed, it was positive rather than negative (Friedrich, 2015). Based on 192,832 tweeted WoS documents published in 2012, 11% of 487,610 tweets were positive, 7% negative and 82% did not express any sentiment after removing the article title words from tweets. Tweets linking to chemistry papers were the least likely to express sentiments (Friedrich, 2015; Friedrich, Bowman, & Haustein, 2015). 22 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

1.3.3.1 Retweets and @mentions As described above, retweets and @mentions represent a particular form of conversational tweets, which seemed to enjoy particular popularity among academic Twitter use. Half of the 4 million tweets linking to 2015 WoS papers were retweets (Table 2), which suggests that a significant amount of tweeting activity reflects information diffusion that does not involve much engagement. Compared to the studies investigating retweet use among general Twitter users (boyd et al., 2010) and academics (Haustein, Bowman, Holmberg, et al., 2014; Letierce et al., 2010; Priem & Costello, 2010; Work et al., 2015), the share of retweets among tweeted journal articles is rather high. The percentage of retweets tends to be lowest in disciplines with low Twitter coverage, which suggests that users in disciplines with low Twitter uptake do not use it as much for information diffusion, possibly because they are not as well connected. In 32 of 120 NSF specialties with DOI coverage above 50%, retweets exceed original tweets (Table 2): retweeting was particularly common in Miscellaneous Zoology, General & Internal Medicine, Miscellaneous Clinical Medicine and Ecology with retweet rates above 60% and low in Solid State Physics, Inorganic & Nuclear Chemistry, Chemical Physics and Applied Chemistry with less than 20%. 1.3.3.2 Hashtags Thirty-one percent of the 24.3 million tweets captured by Altmetric until June 2016 contained a hashtag, which is comparable to other studies on hashtag use by academics (Haustein, Bowman, Holmberg, et al., 2014; Work et al., 2015) but far higher than the 5% among a random sample of tweets in 2009 (boyd et al., 2010). 401,287 unique hashtags were mentioned 12.6 million times, which amounts to an average occurrence of 31 per unique term. 105,705 unique hashtags were used in tweets linking to 2015 WoS papers. While 33% tweets contained at hashtag, 46% of all articles were described with at least one hashtag. Each hashtag was mentioned on average 21 times for a total hashtag frequency of 2.2 million. Hashtag frequency is extremely skewed, as 3% and 6% of hashtags are responsible for 80% of hashtag occurrences in dataset A and B, respectively. For example, the most popular hashtag in dataset A was used 162,754 times (1.3% of all occurrences), while 169,992 hashtags only occurred once. Figure 3 demonstrates on a log-log scale the number of tweets hashtags were mentioned in, as well as the number of distinct users mentioning each hashtag. While in general, a linear relationship can be found between the number of occurrences and users, a few popular hashtags are tweeted only by limited number of users, indicating a smaller community.

The most popular hashtags in dataset A were #science (Table 4), 1.3% of hashtag occurrence), #cancer (0.9%), #physics (0.8%), #openaccess, #health (0.7%), #paper, #oa and #research (0.5% each). The occurrence of #oa as well as #openaccess among the most frequent hashtags reflects the known heterogeneity of folksonomies and the need for tag gardening when trying to analyze topics (Peters, 2009). WoS 2015 papers (dataset B) were most frequently tagged as #cancer (1.0%), #health, #openaccess, #science (0.9%), #FOAMed, #Diabetes, #ornithology and #Psychiatry (0.6%). The order changes, when considering the number of unique users instead of tweets per hashtag. Among hashtags that occurred at least 1,000 times, the largest discrepancy between the number of tweets and users can be observed for #genomeregulation (1,924 tweet; 10 users), #eprompt (2,281; 17) and #cryptocurrency (4,515; 38), which were, on average, tweeted more than 100 times by the same users. On the contrary, the user-hashtag ratio was lowest for #Fit (4,818; 4,743), #StandWithPP (Stand with Planned Parenthood; 1,060; 972), #dataviz (1,010; 912), #coffee (1,517; 1,246), and #PWSYN (title of popular science book The Patient Will See You Now; 1,017; 834), which indicates a widespread adoption among Twitter users. Accordingly, these hashtags are more general and less scientific.

23 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Figure 3 Number of users and number of tweets per hashtag for all tweets captured by Altmetric (A) and for tweets to 2015 WoS papers (B).

Table 4 Hashtag statistics for datasets A and B with most frequent hashtags based on number of tweets.

Most frequent Statistics of hashtags hashtag frequency

Number Number tweets of Number users of per Users hashtag of Number documents Number journals of span Tweet Mean 31 17 1 11 233 #science Standard deviation 617 237 4 209 459 #cancer Minimum 1 1 1 1 0 #physics Maximum 162,754 65,334 1,425 54,845 1,825 #openaccess n/a 99th percentile 395 229 6 129 1,746 #health Dataset A Dataset #paper 90th percentile 20 15 2 8 1,036 #oa 75th percentile 5 4 1 2 182 50th percentile 2 2 1 1 0

Mean 21 13 1 7 4 102 #cancer Standard deviation 222 111 2 73 19 192 #health Minimum 1 1 1 1 1 0 #openaccess Maximum 21,122 12,056 274 9,765 1,684 1,723 #science 99th percentile 311 186 5 92 49 831 #FOAMed Dataset B Dataset #Diabetes 90th percentile 20 15 2 7 6 381 #ornithology 75th percentile 6 5 1 2 2 130 50th percentile 2 2 1 1 1 0

24 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Table 4 shows hashtag-based stats for both datasets. As to be expected, hashtag frequency and the number of unique terms is greater for dataset A, as it covers all documents in Altmetric and the whole timespan, while dataset B is restricted to WoS documents published in 2015. On average, each hashtag occurred in 21 tweets, was used by 13 users to tag 7 documents and 4 journals indexed in WoS. As shown by the percentiles, hashtag occurrence is extremely skewed. On the individual level, the number of users, documents and journals associated with a hashtag can provide information as to how general and widespread a hashtag is, or how specific and relevant to only a small group of users. The timespan, that is, the number of days between the first and last occurrence of a hashtag, indicates its topicality or timeless relevance. For example, among hashtags that occurred at least 1,000 times, #diet, #water and #nutrition were used during the course of more than four years to describe 2015 documents, while #XmasBMJ lasted only 73 days. The first tweet linking to a 2015 WoS paper with the #diet hashtag appeared 17 November 2011. The discrepancy between tweet date and publication date can be described by the lag between online date and journal issue date (see Haustein et al. (2015b) for an analysis of the publication date problematic).

1.3.4 When is scholarly output tweeted? Tweet activity related to scholarly documents has been shown to occur shortly after publication and disappear within a few days (Eysenbach, 2011; Shuai et al., 2012). Tweeted half-lives and delay between publication and first tweet can thus be measured in hours rather than days. This short-lived attention also points to Twitter being used to diffuse new papers instead of discussing them intensely. It is, however, challenging to accurately calculate delay and decay for all publications in WoS as the publication date of the journal does not sufficiently represent when a publication was actually available. Even with the more accurate article-level information of online dates there are issues to determine the actual date of publication, as demonstrated by tweets mentioning articles before they were supposed to be published (Haustein, Bowman, et al., 2015b). Due to the inaccuracy of available publication dates, tweet delay and tweeted half-lives are not computed.

Figure 4 Number of tweets per week day per year and mean of the weekly percentage per year. As shown in Figure 4, there are clear differences between weekdays and weekends, reflecting patterns of the work week, which has also been shown to have an effect on journal submissions (Cabanac & Hartley, 2013) and download patterns (Wang et al., 2012). During the week, tweeting activity increases from Monday (14% of tweets) to peak Wednesday (18%) and decrease again towards the weekend. Twitter users tweet, on average, 23% more about scholarly documents on a Wednesday and 41% less on a Sunday. Figure 4 also shows the different magnitude of Twitter activity among years, as well as an overall increase throughout each year. While a general increase from January to December can be observed for each of the four years of tweets, the general trend also reflects the academic year: activity is higher in spring and fall and drops slightly in summer and particularly during the winter break during the last two and first weeks of each year. Considering that Twitter activity often climaxes the day of

25 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research or day after publication, the season and weekday of publication might influence a document’s visibility on Twitter (Haustein, Bowman, et al., 2015b). Zhang and Paxson identified Twitter bots based on tweeting patterns that were too regular to be human (Zhang & Paxson, 2011).

1.3.5 Where is scholarly output tweeted? Twitter provides the possibility to geotag each tweet with precise latitude-longitude information of a user’s current location. However, since this function is not activated by default, it is only rarely used. Less than 5% of tweets contain geo coordinates (Graham, Hale, & Gaffney, 2014; Severo, Giraud, & Pecout, 2015), which is why geotags of tweets are not a reliable source to determine where scholarly output is tweeted. Another data source to determine the geographic distribution of Twitter users is to analyze the location information provided in the Twitter bio. However, since the profile location is usually not automatically generated but freely edited by users, it cannot be used without extensive data cleaning. Takhteyec, Gruzd and Wellman (2012) showed that 8% of a sample of 3,360 Twitter profiles contained specific latitude-longitude information, 57% named a location and 20% a country, while 15% used fictional places (e.g., Hogwarts) or too general descriptions to determine the users’ whereabouts. Using this profile information, Altmetric is able to determine location information for two-thirds of its tweets. It becomes apparent that in many cases information is not accurate enough to determine the exact location, as remote locations in the UK and Kansas are among the most frequent tweet locations (Haustein & Costas, 2015a).

Table 5 Top 10 countries by number of users for dataset A and B.

Dataset A Dataset B Documents Tweets Users Documents Tweets Users Number of 3,903,064 24,343,105 2,622,117 548,841 3,960,431 601,290 unique items Some country 71% 58% 57% 70% 58% 58% information Missing country 69% 42% 44% 77% 42% 42% information US 33.0% 19.8% 20.1% US 36.2% 19.5% 19.4%

GB 21.6% 11.0% 8.3% GB 27.4% 12.3% 10.6%

by CA 7.9% 2.9% 3.1% CA 9.3% 3.1% 3.4% JP 4.3% 1.8% 2.3% ES 9.6% 3.3% 2.7%

users AU 6.5% 2.9% 2.3% AU 7.6% 2.5% 2.4% ES 7.5% 2.8% 2.2% JP 4.4% 1.5% 2.0% FR 5.3% 1.5% 1.4% NL 4.0% 1.0% 1.3%

number of IN 2.6% 0.7% 1.1% FR 7.9% 1.6% 1.2%

Top 10 countries NL 3.5% 1.0% 1.1% DE 5.6% 1.1% 0.9% DE 4.1% 1.0% 0.9% IN 3.4% 0.7% 0.9%

Due to these limitations, the analysis of where users tweet scholarly documents is restricted to the country level (Table 5). Altmetric provides location information for 57% of users, 58% of tweets and 71% of documents (dataset A) and 58% of both users and tweets and 70% of documents for the WoS subset (dataset B). Users from the US are overrepresented as 20% of tweets are sent by Twitter users with an identified location in the US, which is followed by the UK (8%), Canada (3%), Japan, Australia and Spain (2% each). A similar distribution can be observed for the 2015 WoS articles (dataset B), as the top 10 countries by number of users stay the same, although the UK, the Netherlands, Spain, Canada and to a lesser extent Australia and Germany gain in percentage of users, while India, France, Japan and the US lose in comparison to dataset A. While altmetrics have been marketed as democratizers of science evaluation in the terms of having the potential to correct for biases created by WoS and other US and English-centric journal databases, these results show that, when it comes to Twitter, known biases persist or are even intensified on social media.

26 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

1.3.6 Who tweets scholarly output? One of the main motivations to consider tweets as an altmetric indicator is that Twitter is used by the general public and thus, at least theoretically, offers insight into how non-academics engage with scholarly output. In order to separate tweets by the public from those sent by members of the scholarly community, Twitter users have to be identified and classified as such. 1.3.6.1 Identifying users who tweet scholarly content One of the main challenges of determining the type of impact reflected by tweets to scientific papers is to identify who is tweeting. While Mendeley provides certain standardized user demographics such as academic status, discipline or country for users associated with a paper, the classification of tweets by user type is restricted to Twitter bios. These self-descriptions are 160-character texts, which provide users with the space to present themselves to other users of the microblogging platform.

Applying a codebook to determine who tweets scientific papers based on Twitter username, bio and photo, a sample of 2,000 accounts tweeting links to articles published in Nature, PLOS ONE, PNAS and Science, Tsou et al. (2015) found that almost one quarter of accounts were maintained by an organization. Among these were mainly non-profits (42%), corporations (29%) and universities (13%), while many were also classified as news, media or outreach institutions (19%). Among the 1,520 accounts identified as individuals, two thirds were male. One third of the users were identified as having a PhD and 12% as students. This amounts to almost half of all identified individuals having completed or pursuing a doctorate degree, which stands in glaring contrast to about 1% of the US population with a PhD (Tsou et al., 2015), strongly suggesting that it is the scholarly community rather than the general public who tweets links to scientific papers.

Applying a similar codebook to a random sample of 800 accounts tweeting 2012 WoS papers, 68% of accounts were maintained by an individual, 21% by an organization, while 12% could not be identified (Haustein et al., 2016), corroborating the findings by Tsou et al. (2015). Among individuals, 47% used professional terms (e.g., doctor, MD, photographer) to describe themselves, 22% identified as researchers (e.g., scientist, professor, postdoc), 13% as science communicators (e.g., writer, author, journalist, blogger) and 7% as students (e.g., grad student, PhD candidate). Reflecting the blurred boundaries between personal and professional communication, many individuals used words from more than one of these categories to describe themselves. For example, 8% of accounts were classified as researchers and professionals and 5% as professionals and science communicators (Haustein et al., 2016). Science communicators were also the largest group of Twitter users mentioned by astrophysicists (Holmberg et al., 2014). Although labor-intensive and based on little more than 160- character self-descriptions, the above studies show that it is feasible to extract members of academia from users tweeting scholarly documents. Keyword-based searches can be applied to identify scholars in larger samples, but are limited by either low recall or low precision depending on the particular query (Barthel et al., 2015).

It is considerably more challenging to identify members of the general public. Although many Twitter bios contain terms depicting personal lives (e.g., father, wife, yoga lover), the presence of these terms does not necessarily mean that accounts are maintained by non-academics, because scholars often describe themselves in both personal and professional manner on Twitter (Bowman, 2015b; Haustein & Costas, 2015b; Work et al., 2015). Similarly, it is challenging to distinguish members of the public based on an indeterminate list of terms of non-academic professions (e.g., consultant, photographer), especially when also considering accounts in languages other than English. Even the comparably straight-forward identification of a researcher who strictly identifies as such on Twitter, becomes problematic when they shared a paper out of private interests. For example, a tweet by a physicist might actually reflect engagement by the public rather than scholarly communication, if they tweeted about a cancer study as a member of a patient group rather than in their academic role.

27 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

An alternative to classifying users based on publicly available information, is to approach them directly and ask them who they are. Alperin (2015b) pioneered such a survey method on Twitter, which with the help of an automated Twitter account, asked users who had tweeted a Scielo Brazil paper, whether they were affiliated with a university. Such a direct approach might also be helpful to determine the motivation of a user to tweet a specific paper, helping to quantify and distinguish different types of tweets, such as endorsement or critical discusion, diffusion or self-promotion. Author self-citations or self-tweets accounted for 7% of a sample of 270 tweets (Thelwall et al., 2013).

Table 6 Number of followers, tweets, tweet span, tweets per day and tweeting activity per week for the most active users in dataset A (≥20,000 tweets per user).

Twitter handle Number of Tweets Tweet Tweets Tweeting activity per week followers span per day

blackphysicists 12,914 92,583 1,826 50.7 MathPaper 1,889 73,239 1,086 67.4 anestesiaf 1,428 63,953 1,713 37.3 oceanologia 389 62,585 1,353 46.3 UIST_Papers20XX 35 50,211 1,360 36.9 UIST_Papers19XX 6 49,838 1,359 36.7 hiv_insight 14,328 48,714 1,822 26.7 russfeed 2,127 44,463 577 77.1 uranus_2 2,519 42,053 1,790 23.5 Immunol_papers 477 40,721 639 63.7 psych2evidence 513 40,658 410 99.2 InorganicNews 1,044 36,869 1,657 22.3 arXiv_trend 166 33,272 457 72.8 hlth_literacy 5,582 32,337 1,823 17.7 AstroPHYPapers 3,427 31,154 1,086 28.7 cirugiaf 406 30,692 1,710 17.9 ThihaSwe_dr 601 29,357 1,164 25.2 semantic_bot 0 28,908 28 1032.4 CondensedPapers 530 27,603 912 30.3 libroazuln 128 24,994 1,613 15.5 rnomics 1,520 24,905 1,821 13.7 PhysicsPaper 476 22,894 1,086 21.1 epigen_papers 788 21,916 739 29.7

The 24.3 million tweets captured by Altmetric were sent by 2.6 million users. Looking at the most active users who tweeted more than 1,000 times during the whole period covered by Altmetric (Table 6), the presence of accounts automatically diffusing scholarly articles on Twitter becomes apparent (Haustein, Bowman, Holmberg, et al., 2016). In fact, 15 of the 19 most productive accounts in Table 6 with more than 25,000 tweets self-identified bots (see below). 1.3.6.2 Classifying users by Twitter activity Instead of classifying users according to their self-descriptions, accounts can also be grouped based on their activity. Dividing Twitter accounts into three groups of top 1%, 9% and 90% of users (according to number of tweets) helps to distinguish lead and highly active users from less active ones (Bruns & Stieglitz, 2014). This classification provides insights into tweeting behavior of different types of users. Separating the 601,290 users in dataset B by number of tweets linking to a 2015 WoS article, 6,016 lead users, 54,535 highly active and 540,739 least active users can be identified. Lead users contributed between 84 and 19,973 tweets, a median of 149 tweets per users, had on average 935 followers (median; mean=3,862) and tweeted the 2015 papers during an average tweet span of 598 days. Highly 28 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research active users contributed between 9 and 83 tweets (median=16), had less followers (median=442.5; mean=2,136) and shorter tweet spans (mean=388 days), while least active users tweeted up to eight times (median=1), had 212 followers and were active for a period of 58 days.

Lead (top 1% of users), highly active (9%) and least active (90%) users contributed 43%, 31% and 25% of tweets to the entire set of 2015 WoS papers, respectively (Figure 5). Interestingly, these percentages differ among NSF disciplines with least active users overrepresented among those tweeting literature from Professional Fields, Social Sciences, Psychology and Earth and Space. On the contrary, lead users were overrepresented in Chemistry, Physics, Mathematics and Engineering & Technology, which were the fields exhibiting the lowest Twitter coverage, density and number of unique users (Table 2). Assuming that the general public is least active when it comes to tweeting about scholarly papers, they are more likely to engage with articles published in journals from the Professional Fields and Social Sciences and less likely to tweet Chemical papers. The high presence of lead users in Chemistry and Physics might, at least partly, be caused by accounts promoting these papers automatically, such as @blackphysicists and @MathPaper, which were the two most active accounts in dataset A, tweeting 51 and 67 scholarly documents per day (Table 6).

Figure 5 Percentage of tweets from lead users (1%), highly active users (9%) and least active users (90%) per discipline (dataset B). 1.3.6.3 Twitter bots Automated Twitter accounts have become prevalent on Twitter (Mowbray, 2014). About one fifth of tweets sent during the 2016 presidential election were estimated to be sent by bots (Bessi & Ferrara, 2016) and almost one quarter of tweets in 2009 came from accounts tweeting more than 150 times per day. Ferrara et al. (2016, p. 96) define social bots as “a computer algorithm that automatically produces and interacts with humans on social media, trying to emulate and possibly alter their behavior”. Automated Twitter accounts can be further distinguished between useful bots and antisocial or spambots (Ferrara et al., 2016; Mowbray, 2014).

Bots are also infiltrating academic Twitter. Among a random sample of 800 Twitter accounts captured by Altmetric, 8% seemed completely and 5% partially automated (Haustein et al., 2016), while automated accounts who self-identified as such were responsible for 9% of tweets to arXiv submissions (Haustein, Bowman, Holmberg, et al., 2016). Shuai et al. (2012) even removed half of the tweets to a sample of tweets to documents on arXiv, as they were created by bots. Regardless of whether Twitter accounts that automatically tweet scientific papers are considered useful or spam, it is safe to say that their automated tweets do not reflect impact. In the context of altmetrics and tweets to scientific papers, bot activity thus needs to be at least identified, if not entirely removed in an impact assessment. Although spammers and excessive self-promotion was identified as a challenge by Altmetric, they still considered gaming a rare and easy to identify threat in 2013 (Liu & Adie, 2013). 29 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Twitter’s terms of service specifically address automation. While prohibiting spam, Twitter encourages automated tweets if they “broadcast helpful information” (Twitter, 2017). However, what is considered spam continuously evolves as users apply “new tricks and tactics” (Twitter, n.d.) to adapt to or circumvent Twitter rules. Twitter bots can be identified based on specific regularities in their tweeting behavior, such as the frequent and repetitive use of the same hashtags, URLs, tweet format and content, regular temporal activity, as well as the follower/friends ratio, @mentions to non- followers or account suspensions (Mowbray, 2014). Sixteen percent of Twitter accounts were identified as automated based on not-uniform-enough or too-uniform tweeting patterns to be stemming from a human (Zhang & Paxson, 2011). Social network indicators based on the Twitter follower and friend network are considered more robust measures, as they are harder to influence (Yang, Harkreader, & Gu, 2011). The BotOrNot algorithm additionally considers linguistic features and tweet sentiment to detect automated Twitter accounts (Davis, Varol, Ferrara, Flammini, & Menczer, 2016). However, as these spamming measures get known and integrated by Twitter to block spam accounts, bots employ more sophisticated algorithms to avoid getting caught, resulting in an arms race between those who create and those who seek to identify Twitter spam.

Figure 6 Query used to identify automated Twitter accounts.

Figure 7 6 Query used to identify publisher accounts.

Analyzing the most active users who have tweeted at least 1,000 times (dataset A), a keyword-based query searching the Twitter bio as well as user name and handle revealed that among 2,043 accounts 248 identified themselves as automated (Figure 6) and 305 as journal or publisher accounts (Figure 7). These make up 30% and 11% of the tweets sent by the 2,043 most active users (Table 7), which correspond to 7% and 3% of the entire 24.3 million tweets to scholarly documents in dataset A. The median number of followers is significantly lower than other accounts (Table 7) and as few as 6% of the 1.8 million tweets sent by the 248 accounts contain @mentions. If scholarly bots do mention other users, they often seem to reference journals such as @hiv_insight, which frequently mentions @PLoSMedicine, @STI_BMJ and @JAMA_current. Other than social bots in the general Twittersphere, scholarly bots seem to not try to emulate human behavior or game the system. They rather resemble RSS feeds tweeting the paper title and a link, often specifying what type of information they diffuse. Some even provide instructions to create similar feeds. For example, the Twitter bio of @asthma_papers reads “RSS feed for #asthma papers in #Pubmed. Create a feed of your own using instructions here: https://github.com/roblanf/phypapers”.

30 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Table 7 Twitter metrics for self-identified bots, journal and publisher accounts and other accounts based on users with at least 1,000 tweets (dataset A).

Most active Twitter accounts Followers Tweets Tweets Tweet (≥1,000 tweets, n=2,043) per day span median 212 3,479 5.1 845 Self-identified bots mean 1,014 7,339 14.4 923 n=248 std dev 2,781 10,390 67.0 477 30% of tweets min 0 1,001 0.6 28 max 25,003 73,239 1,032.4 1,823 median 3,199 1,670 1.2 1,647 Journal and mean 21,475 2,249 1.7 1,484 publisher accounts std dev 116,124 1,874 1.6 369 n=305 min 3 1,001 0.6 122 11% of tweets max 1,448,649 19,256 14.7 1,822 median 1,535 1,599 1.4 1,388 Other accounts mean 5,236 2,408 2.9 1,278 n=1,490 std dev 14,091 3,670 9.3 496 59% of tweets min 0 1,000 0.6 8 max 228,224 92,583 297.3 1,826

Considering that scholars use Twitter to diffuse information and stay aware of relevant literature, automated accounts might be considered useful. However, bots have shown to be harmful to society, when they are used to influence public opinion and behavior such as political opinions or elections (Bessi & Ferrara, 2016; Metaxas & Mustafaraj, 2012; Ratkiewicz, Conover, Meiss, Gonçalves, Flammini, et al., 2011; Ratkiewicz, Conover, Meiss, Gonçalves, Patil, et al., 2011) or manipulation of the stock market (Ferrara et al., 2016). If Twitter impact became part of the scholarly reward system, Twitter bots might be able to similarly influence opinions or shape outcomes of certain research metrics.

1.4 Conclusion and outlook This chapter provided an overview of the use of Twitter in scholarly communication. By demonstrating who uses Twitter in academia and for what reasons, what types of scholarly outputs are diffused how, where and when, it aimed to add context and help to interpret any scholarly metrics derived from this and similar types of social media activity.

Research evaluators and managers were particularly excited at the prospect of an easily accessible data source that would be able to capture traces of the societal impact of research. However, perhaps unsurprisingly, the majority of tweets stem from stakeholders in academia rather than from members of the general public, which indicates that the majority of tweets to scientific papers are more likely to reflect scholarly communication rather than societal impact. At the same time, Twitter uptake in academia lacks behind Twitter user by the general public. Twitter activity is influenced by geographical and disciplinary biases and publication date. Known biases towards US and UK sources persist, rather than democratizing scholarly communication and the reward system of science.

The majority of tweets linking to scientific articles appear shortly after their publication; tweeting half- lives can be measured in hours rather than days. Moreover, one can observe weekday as well as seasonal patterns with Twitter activity peaking Wednesdays and in the fall and plummeting during the weekend and holiday season. Journal and publisher accounts as well as Twitter bots contribute significantly to tweeting activity linked to academic papers, which suggests that a significant extent of tweeting activity serves promotional purposes or is automated, which does neither reflect societal nor scholarly impact. A large share of tweets contains hashtags and mention either the title or a short summary of the paper they referred to. Half of all articles linking to 2015 WoS papers were retweets 31 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research and the majority contained no sentiments. These tweeting characteristics emphasize particular low engagement of users linking to journal articles. The main motivation for researchers to use Twitter is information diffusion, networking and to stay up-to-date with the literature. However, the sheer brevity of tweets makes intense discussions the exception rather than the rule on Twitter.

As citation behavior and motivations to cite or not cite certain sources are biased and influenced by many factors other than a paper’s significance, not every citation represents impact. However, each scholarly author is bound by scholarly norms to participate in the citation process. In some rare cases, where scholars tweet corrections to publications or journals provide tweetable abstracts and organize journal clubs, Twitter has started to be integrated in or even replace certain functions of formal journal publishing. However, in most fields tweeting does not yet play an important role in scholarly communication. In most disciplines, Twitter uptake is low and the platform is only used in a passive or infrequent manner, or tweets reflect only a part of informal scholarly communication, such as conference chatter. Moreover, Twitter uptake varies between disciplines, countries, journals, as well as individuals and can be easily influenced and manipulated. The presence of automated Twitter accounts which promote certain contents becomes particularly problematic when tweet counts become the basis for measures of impact.

This is not to say that Twitter should be completely disregarded as a data source for scholarly metrics. Rather, the microblogging platform should be approached critically in terms of what kind of use and user populations it captures. By reviewing the role of Twitter in scholarly communication and analyzing tweets linking to scholarly documents in depths and beyond crude counts, this chapter attepmted to provide the basis for more sophisticated and well-balanced approaches to scholarly Twitter metrics.

References Alhoori, H., & Furuta, R. (2014). Do altmetrics follow the crowd or does the crowd follow altmetrics? In Digital Libraries (JCDL), 2014 IEEE/ACM Joint Conference on (pp. 375–378). IEEE. Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6970193 Allen, H. G., Stanton, T. R., Di Pietro, F., & Moseley, G. L. (2013). Social media release increases dissemination of original articles in the clinical pain sciences. PloS One, 8(7), e68914–e68914. https://doi.org/10.1371/journal.pone.0068914 Alperin, J. P. (2013). Ask not what altmetrics can do for you, but what altmetrics can do for developing countries. Bulletin of the American Society for Information Science and Technology, 39(4), 18–21. Alperin, J. P. (2015a). Geographic variation in social media metrics: an analysis of Latin American journal articles. Aslib Journal of Information Management, 67(3), 289–304. https://doi.org/10.1108/AJIM-12-2014- 0176 Alperin, J. P. (2015b). Moving beyond counts: A method for surveying Twitter users. In altmetrics15: 5 years in, what do we know? Amsterdam, The Netherlands. Retrieved from http://altmetrics.org/wp- content/uploads/2015/09/altmetrics15_paper_3.pdf Amir, M., Sampson, B. P., Endly, D., Tamai, J. M., Henley, J., Brewer, A. C., … Dellavalle, R. P. (2014). Social networking sites: emerging and essential tools for communication in dermatology. JAMA Dermatology, 150(1), 56–60. https://doi.org/10.1001/jamadermatol.2013.6340 Andersen, J. P., & Haustein, S. (2015). Influence of study type on Twitter activity for medical research papers. In Proceedings of the 15th International Society of Scientometrics and Informetrics Conference (pp. 26–36). Istanbul, Turkey. https://arxiv.org/abs/1507.00154 Association of American University Professors. (2013). Academic Freedom and Electronic Communications. Retrieved from https://www.aaup.org/file/Academic%20Freedom%20%26%20Electronic%20Communications.pdf Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars ’ visibility on the social Web. In Proceedings of the 17th International Conference on Science and Technology Indicators Montreal Canada 58 Sept 2012 (Vol. 52900, pp. 98–109). Montreal. http://arxiv.org/abs/1205.5611

32 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Barnes, N. G., & Lescault, A. M. (2013). College presidents out-blog and out-tweet corporate CEO’s. UMass Center for Marketing Research. Retrieved from http://www.umassd.edu/cmr/socialmediaresearch/collegepresidentsoutblog/ Barthel, S., Tönnies, S., Köhncke, B., Siehndel, P., & Balke, W.-T. (2015). What does Twitter Measure? Influence of Diverse User Groups in Altmetrics. In Proceedings of the 15th ACM/IEEE-CE on Joint Conference on Digital Libraries (pp. 119–128). Knoxville, Tennessee, USA: ACM Press. https://doi.org/10.1145/2756406.2756913 Baym, M. (2014). The Perils and Pleasures of Tweeting with Fans. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 221–236). New York: Peter Lang. Bennett-Smith, M. (2013, June 4). Geoffrey Miller, Visiting NYU Professor, Slammed For Fat-Shaming Obese PhD Applicants. Retrieved April 14, 2016, from http://www.huffingtonpost.com/2013/06/04/geoffrey-miller-fat-shaming-nyu-phd_n_3385641.html Bessi, A., & Ferrara, E. (2016). Social bots distort the 2016 U.S. Presidential election online discussion. First Monday, 21(11). https://doi.org/10.5210/fm.v21i11.7090 Boateng, F., & Quan Liu, Y. (2014). Web 2.0 applications’ usage and trends in top US academic libraries. Library Hi Tech, 32(1), 120–138. https://doi.org/10.1108/LHT-07-2013-0093 Bonetta, L. (2009). Should you be tweeting? Cell, 139(3), 452–453. https://doi.org/10.1016/j.cell.2009.10.017 Bornmann, L. (2014). Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime. Journal of Informetrics, 8(4), 935–950. https://doi.org/10.1016/j.joi.2014.09.007 Bornmann, L. (2016). Scientific Revolution in Scientometrics: the Broadening of Impact from Citation to Societal. In C. R. Sugimoto (Ed.), Theories of Informetrics and Scholarly Communication (pp. 347–359). Berlin: de Gruyter Mouton. Bowman, T. D. (2015a). Differences in personal and professional tweets of scholars. Aslib Journal of Information Management, 67(3), 356–371. https://doi.org/10.1108/AJIM-12-2014-0180 Bowman, T. D. (2015b, July). Investigating the use of affordances and framing techniques by scholars to manage personal and professional impressions on Twitter (Dissertation). Indiana University, Bloomington, IN, USA. Retrieved from http://www.tdbowman.com/pdf/2015_07_TDBowman_Dissertation.pdf boyd, D., Golder, S., & Lotan, G. (2010). Tweet, Tweet, Retweet: Conversational Aspects of Retweeting on Twitter (pp. 1–10). IEEE. https://doi.org/10.1109/HICSS.2010.412 Bruns, A., & Moe, H. (2014). Structural Layers of Communication on Twitter. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 15–28). New York: Peter Lang. Bruns, A., & Stieglitz, S. (2014). Metrics for Understanding Communication on Twitter. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 69–82). New York: Peter Lang. Bulger, M. E., Meyer, E. T., De la Flor, G., Terras, M., Wyatt, S., Jirotka, M., … Madsen, C. M. (2011). Reinventing research? Information practices in the humanities (Research Information Network). Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1859267 Cabanac, G., & Hartley, J. (2013). Issues of work-life balance among JASIST authors and editors: Work-Life Balance Issues Among JASIST Authors and Editors. Journal of the American Society for Information Science and Technology, 64(10), 2182–2186. https://doi.org/10.1002/asi.22888 Cahn, P. S., Benjamin, E. J., & Shanahan, C. W. (2013). “Uncrunching” time: medical schools’ use of social media for faculty development. Medical Education Online, 1, 1–6. https://doi.org/10.3402/meo.v18i0.20995 Carpenter, J., Wetheridge, L., Tanner, S., & Smith, N. (2012). Researchers of Tomorrow. Retrieved from http://www.jisc.ac.uk/publications/reports/2012/researchers-of-tomorrow Chaudhry, B. A., Glode, L. M., Gillman, M., & Miller, R. S. (2012). Trends in Twitter Use by Physicians at the American Society of Clinical Oncology Annual Meeting , 2010 and 2011. Journal of Oncology Practice, 8(3), 173–178. https://doi.org/10.1200/JOP.2011.000483 Chretien, K. C., Azar, J., & Kind, T. (2011). Physicians on Twitter. JAMA, 305(6), 566–568. https://doi.org/10.1001/jama.2011.68 Chu, S. K.-W., & Du, H. S. (2013). Social networking tools for academic libraries. Journal of Librarianship and Information Science, 45(1), 64–75. https://doi.org/10.1177/0961000611434361 Cochran, A., Kao, L. S., Gusani, N. J., Suliburk, J. W., & Nwomeh, B. C. (2014). Use of Twitter to document the 2013 Academic Surgical Congress. The Journal of Surgical Research, 190(1), 36–40. https://doi.org/10.1016/j.jss.2014.02.029 Costas, R., Zahedi, Z., & Wouters, P. (2015a). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019. https://doi.org/10.1002/asi.23309 33 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Costas, R., Zahedi, Z., & Wouters, P. (2015b). The thematic orientation of publications mentioned on social media: Large-scale disciplinary comparison of social media metrics with citations. Aslib Journal of Information Management, 67(3), 260–288. https://doi.org/10.1108/AJIM-12-2014-0173 Coverdale, A. (2011). Negotiating Doctoral Practices and Academic Identities Through the Adoption and Use of Social and Participative Media. In Proceedings of the European Conference on eLearning 2011 (p. 909). Brighton, UK. Cronin, B. (2013a). Metrics à la mode. Journal of the American Society for Information Science and Technology, 64(6), 1091–1091. https://doi.org/10.1002/asi.22989 Cronin, B. (2013b). The evolving indicator space (iSpace). Journal of the American Society for Information Science and Technology, 64(8), 1523–1525. https://doi.org/10.1002/asi.23041 Darling, E. S., Shiffman, D., Côté, I., & Drew, J. A. (2013). The role of Twitter in the life cycle of a scientific publication. Ideas in Ecology and Evolution, 6, 32–43. https://doi.org/10.7287/peerj.preprints.16v1 Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016). BotOrNot: A System to Evaluate Social Bots (pp. 273–274). ACM Press. https://doi.org/10.1145/2872518.2889302 Desai, T., Patwardhan, M., & Coore, H. (2014). Factors that contribute to social media influence within an Internal Medicine Twitter learning community. F1000Research, 3, 120–120. https://doi.org/10.12688/f1000research.4283.1 Desai, T., Shariff, A., Shariff, A., Kats, M., Fang, X., Christiano, C., & Ferris, M. (2012). Tweeting the Meeting : An In-Depth Analysis of Twitter Activity at Kidney Week 2011. PLoS ONE, 7(7), 1–9. https://doi.org/10.1371/journal.pone.0040253 Desrochers, N., Paul-Hus, A., Haustein, S., Costas, R., Mongeon, P., Quan-Haase, A., Bowman, T.D., Pecoskie, J., Tsou, A., & Larivière, V. (2018). Authorship, citations, acknowledgments and visibility in social media: Symbolic capital in the multifaceted reward system of science. Social Science Information, 1–26. https://doi.org/10.1177/0539018417752089 Didegah, F., Bowman, T. D., Bowman, S., & Hartley, J. (2016). Comparing the characteristics of highly cited titles and highly alted titles. In 21st International Conference on Science and Technology Indicators (pp. 1190–1195). Valencia, Spain. Retrieved from http://ocs.editorial.upv.es/index.php/STI2016/STI2016/paper/viewFile/4543/2327 Donelan, H. (2016). Social media for professional development and networking opportunities in academia. Journal of Further and Higher Education, 40(5), 706–729. https://doi.org/10.1080/0309877X.2015.1014321 Duggan, M., Ellison, N. B., Lampe, C., Lenhart, A., & Madden, M. (2015, January 9). Demographics of Key Social Networking Platforms. Retrieved from http://www.pewinternet.org/2015/01/09/demographics-of-key- social-networking-platforms-2/ Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13(4), e123–e123. https://doi.org/10.2196/jmir.2012 Fenner, M. (2013). What can article-level metrics do for you? PLoS Biology, 11(10), e1001687–e1001687. https://doi.org/10.1371/journal.pbio.1001687 Ferguson, C., Inglis, S. C., Newton, P. J., Cripps, P. J. S., Macdonald, P. S., & Davidson, P. M. (2014). Social media: A tool to spread information: A case study analysis of Twitter conversation at the Cardiac Society of Australia & New Zealand 61st Annual Scientific Meeting 2013. Collegian, 21(2), 89–93. https://doi.org/10.1016/j.colegn.2014.03.002 Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016). The rise of social bots. Communications of the ACM, 59(7), 96–104. https://doi.org/10.1145/2818717 Forkosh-Baruch, A., & Hershkovitz, A. (2012). A case study of Israeli higher-education institutes sharing scholarly information with the community via social networks. The Internet and Higher Education, 15(1), 58–68. https://doi.org/10.1016/j.iheduc.2011.08.003 Friedrich, N. (2015, September 3). Applying sentiment analysis for tweets linking to scientific papers (Bachelor’s thesis). Heinrich-Heine-Universität Düsseldorf, Düsseldorf. Friedrich, N., Bowman, T. D., & Haustein, S. (2015). Do tweets to scientific articles contain positive or negative sentiments? In altmetrics15: 5 years in, what do we know? Amsterdam, The Netherlands. Retrieved from http://altmetrics.org/altmetrics15/friedrich/ Friedrich, N., Bowman, T. D., Stock, W. G., & Haustein, S. (2015). Adapting sentiment analysis for tweets linking to scientific papers. In Proceedings of the 15th International Society of Scientometrics and Informetrics Conference (pp. 107–108). Istanbul, Turkey. https://arxiv.org/abs/1507.01967 Gaffney, D., & Puschmann, C. (2014). Data Collection on Twitter. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 55–67). New York: Peter Lang. 34 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Gerber, A. (2012). Online trends from the first German trend study on science communication. In Science and the Internet (pp. 13–18). Düsseldorf University Press. Graham, M., Hale, S. A., & Gaffney, D. (2014). Where in the World Are You? Geolocation and Language Identification in Twitter. The Professional Geographer, 66(4), 568–578. https://doi.org/10.1080/00330124.2014.907699 Grajales, F. J., Sheps, S., Ho, K., Novak-Lauscher, H., & Eysenbach, G. (2014). Social media: a review and tutorial of applications in medicine and health care. Journal of Medical Internet Research, 16(2), e13–e13. https://doi.org/10.2196/jmir.2912 Grande, D., Gollust, S. E., Pany, M., Seymour, J., Goss, A., Kilaru, A., & Meisel, Z. (2014). Translating Research For Health Policy: Researchers’ Perceptions And Use Of Social Media. Health Affairs, 33(7), 1278–1285. https://doi.org/10.1377/hlthaff.2014.0300 Greenwood, G. (2012). Examining the Presence of Social Media on University Web Sites. Journal of College Admission, 216, 24–28. Gruzd, A., Staves, K., & Wilk, A. (2012). Connected scholars: Examining the role of social media in research practices of faculty using the UTAUT model. Computers in Human Behavior, 28(6), 2340–2350. https://doi.org/10.1016/j.chb.2012.07.004 Gu, F., & Widén-Wulff, G. (2011). Scholarly communication and possible changes in the context of social media. The Electronic Library, 29(6), 762–776. https://doi.org/10.1108/02640471111187999 Hadgu, A. T., & Jäschke, R. (2014). Identifying and analyzing researchers on twitter (pp. 23–32). ACM Press. https://doi.org/10.1145/2615569.2615676 Hall, N. (2014). The Kardashian index: a measure of discrepant social media profile for scientists. Genome Biology, 15(7), 424–424. https://doi.org/10.1186/s13059-014-0424-0 Hammarfelt, B. (2014). Using altmetrics for assessing research impact in the humanities. Scientometrics, 101, 1419–1430. https://doi.org/10.1007/s11192-014-1261-3 Harley, D., Acord, S. K., Earl-Novell, S., Lawrence, S., & King, C. J. (2010). Assessing the future landscape of scholarly communication: an exploration of faculty values and needs in seven disciplines. Berkeley: The Center for Studies in Higher Education, Univ Of California Press. Retrieved from http://escholarship.org/uc/item/15x7385g Harrington, S. (2014). Tweeting about the Telly: Live TV, Audiences and Social Media. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 237–247). New York: Peter Lang. Haustein, S. (2016). Grand challenges in altmetrics: heterogeneity, data quality and dependencies. Scientometrics. https://doi.org/10.1007/s11192-016-1910-9 Haustein, S., Bowman, T. D., & Costas, R. (2015a). “Communities of attention” around scientific publications: who is tweeting about scientific papers? Presented at the Social Media & Society 2015 International Conference, Toronto, Canada. Haustein, S., Bowman, T. D., & Costas, R. (2015b). When is an article actually published? An analysis of online availability, publication, and indexation dates. In Proceedings of the 15th International Society of Scientometrics and Informetrics Conference (pp. 1170–1179). Istanbul, Turkey. https://arxiv.org/abs/1505.00796 Haustein, S., Bowman, T. D., & Costas, R. (2016). Interpreting “altmetrics”: viewing acts on social media through the lens of citation and social theories. In C. R. Sugimoto (Ed.), Theories of Informetrics and Scholarly Communication. A Festschrift in Honor of Blaise Cronin (pp. 372–405). Berlin: De Gruyter. http://arxiv.org/abs/1502.05701 Haustein, S., Bowman, T. D., Holmberg, K., Peters, I., & Larivière, V. (2014). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Journal of Information Management, 66(3), 279–296. https://doi.org/10.1108/AJIM-09-2013-0081 Haustein, S., Bowman, T. D., Holmberg, K., Tsou, A., Sugimoto, C. R., & Larivière, V. (2016). Tweets as impact indicators: Examining the implications of automated “bot” accounts on Twitter. Journal of the Association for Information Science and Technology, 67(1), 232–238. https://doi.org/10.1002/asi.23456 Haustein, S., Bowman, T. D., Macaluso, B., Sugimoto, C. R., & Larivière, V. (2014). Measuring Twitter activity of arXiv e-prints and published papers. In altmetrics14: expanding impacts and metrics, Workshop at Web Science Conference 2014. https://doi.org/10.6084/m9.figshare.1041514%20 Haustein, S., & Costas, R. (2015a). Determining Twitter audiences: Geolocation and number of followers. In altmetrics15: 5 years in, what do we know? Amsterdam, The Netherlands. Retrieved from http://altmetrics.org/altmetrics15/haustein/

35 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Haustein, S., & Costas, R. (2015b). Identifying Twitter audiences: who is tweeting about scientific papers? Presented at the Metrics 2015 ASIS&T SIG/MET Workshop. Retrieved from https://www.asist.org/SIG/SIGMET/wp-content/uploads/2015/10/sigmet2015_paper_11.pdf Haustein, S., Costas, R., & Larivière, V. (2015). Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLoS ONE, 10(3), e0120495. https://doi.org/10.1371/journal.pone.0120495 Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (2014). Tweets vs. Mendeley readers: How do these two social media metrics differ? It - Information Technology, 56(5), 207–215. https://doi.org/10.1515/itit-2014-1048 Haustein, S., Minik, V., Brinson, D., Hayes, E., Costas, R., & Sugimoto, C. R. (2016). Identifying Twitter user communities in the context of altmetrics. In 3rd Altmetrics Conference 3:AM. Bucharest, Romania. Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145–1163. https://doi.org/10.1007/s11192-013-1221-3 Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Larivière, V. (2014). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Science and Technology, 65(4), 656–669. https://doi.org/10.1002/asi.23101 Haustein, S., Sugimoto, C. R., & Larivière, V. (2015). Guest editorial: social media in scholarly communication. Aslib Journal of Information Management, 67(3). https://doi.org/10.1108/AJIM-03-2015- 0047 Hawkins, C. M., Duszak, R., & Rawson, J. V. (2014). Social media in radiology: early trends in Twitter microblogging at radiology’s largest international meeting. Journal of the American College of Radiology : JACR, 11(4), 387–90. https://doi.org/10.1016/j.jacr.2013.07.015 Herman, B. (2014, September 5). Steven Salaita Twitter Scandal: University Offers Settlement, But Free Speech Questions Linger. International Business Times. Retrieved from http://www.ibtimes.com/steven- salaita-twitter-scandal-university-offers-settlement-free-speech-questions-linger-1678854 Holmberg, K. (2014). The impact of retweeting. Figshare. Retrieved from http://dx.doi.org/10.6084/m9.figshare.1037555 Holmberg, K., Bowman, T. D., Haustein, S., & Peters, I. (2014). Astrophysicists’ conversational connections on twitter. PloS One, 9(8), e106086–e106086. https://doi.org/10.1371/journal.pone.0106086 Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics, 101(2), 1027–1042. https://doi.org/10.1007/s11192-014-1229-3 Honeycutt, C., & Herring, S. C. (2009). Beyond Microblogging: Conversation and Collaboration via Twitter (pp. 1–10). IEEE. https://doi.org/10.1109/HICSS.2009.89 Hussain, A. (2015). Adoption of Web 2.0 in library associations in the presence of social media. Program, 49(2), 151–169. https://doi.org/10.1108/PROG-02-2013-0007 Ingeno, L. (2013, June 4). Fat-Shaming in Academe. Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/2013/06/04/outrage-over-professors-twitter-post-obese-students Jalali, A., & Wood, T. (2013). Analyzing Online Impact of Canadian Conference of Medical Education through Tweets. Education in Medicine Journal, 5(3). https://doi.org/10.5959/eimj.v5i3.162 Java, A., Song, X., Finin, T., & Tseng, B. (2007). Why we twitter: understanding microblogging usage and communities (pp. 56–65). ACM Press. https://doi.org/10.1145/1348549.1348556 Kamel Boulos, M. N., & Anderson, P. F. (2012). Preliminary survey of leading general medicine journals’ use of Facebook and Twitter. Journal of the Canadian Health Libraries Association, 33(2), 38–47. Ke, Q., Ahn, Y.-Y., & Sugimoto, C. R. (2017). A systematic identification and analysis of scientists on Twitter. PLOS ONE, 12(4), e0175368. https://doi.org/10.1371/journal.pone.0175368 Kelly, R. (2009). Twitter study (White Paper). San Antonio, TX: Pear Analytics. Retrieved from http://pearanalytics.com/wp-content/uploads/2012/12/Twitter-Study-August-2009.pdf Kimmons, R., Veletsianos, G., & Woodward, S. (2017). Institutional Uses of Twitter in U.S. Higher Education. Innovative Higher Education, 42(2), 97–111. https://doi.org/10.1007/s10755-016-9375-6 Knight, S. R. (2014). Social media and online attention as an early measure of the impact of research in solid organ transplantation. Transplantation, 98(5), 490–6. https://doi.org/10.1097/TP.0000000000000307 Kortelainen, T., & Katvala, M. (2012). “Everything is plentiful—Except attention”. Attention data of scientific journals on social web tools. Journal of Informetrics, 6(4), 661–668. https://doi.org/10.1016/j.joi.2012.06.004 Letierce, J., Passant, A., Breslin, J., & Decker, S. (2010). Understanding how Twitter is used to spread scientific messages. Retrieved from http://journal.webscience.org/314/ 36 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Leung, E. Y. L., Siassakos, D., & Khan, K. S. (2015). Journal Club via social media: authors take note of the impact of #BlueJC. BJOG: An International Journal of Obstetrics & Gynaecology, 122(8), 1042–1044. https://doi.org/10.1111/1471-0528.13440 Linvill, D. L., McGee, S. E., & Hicks, L. K. (2012). Colleges’ and universities’ use of Twitter: A content analysis. Public Relations Review, 38(4), 636–638. https://doi.org/10.1016/j.pubrev.2012.05.010 Liu, J., & Adie, E. (2013). Five Challenges in Altmetrics : A Toolmaker’ s Perspective. Bulletin of Associatioon for Infomation Science and Technology, 39(4), 31–34. Loeb, S., Bayne, C. E., Frey, C., Davies, B. J., Averch, T. D., Woo, H. H., … Eggener, S. E. (2014). Use of social media in urology: data from the American Urological Association (AUA). BJU International, 113(6), 993–998. https://doi.org/10.1111/bju.12586 Lulic, I., & Kovic, I. (2013). Analysis of emergency physicians’ Twitter accounts. Emergency Medicine Journal : EMJ, 30(5), 371–6. https://doi.org/10.1136/emermed-2012-201132 Maleki, A. (2014). Networking in science tweets linking to scholarly articles of Iran. Presented at the SIG/MET Workshop, ASIS&T 2014 Annual Meeting, Seattle. Retrieved from http://www.asis.org/SIG/SIGMET/data/uploads/sigmet2014/maleki.pdf Mccrea, P. (2011). Reflections on Academic Blogging as a Vehicle for Professional Development. In Proceedings of the European Conference on eLearning (p. 997). Brighton, UK. McKendrick, D. R., Cumming, G. P., & Lee, A. J. (2012). Increased Use of Twitter at a Medical Conference: A Report and a Review of the Educational Opportunities. Journal of Medical Internet Research, 14(6), e176. https://doi.org/10.2196/jmir.2144 Mehta, N., & Flickinger, T. (2014). The Times They Are A-Changin’: Academia, Social Media and the JGIM Twitter Journal Club. Journal of General Internal Medicine, 29(10), 1317–1318. https://doi.org/10.1007/s11606-014-2976-9 Metaxas, P. T., & Mustafaraj, E. (2012). Social Media and the Elections. Science, 338(6106), 472–473. https://doi.org/10.1126/science.1230456 Micieli, R., & Micieli, J. a. (2012). Twitter as a tool for ophthalmologists. Canadian Journal of Ophthalmology. Journal Canadien D’ophtalmologie, 47(5), 410–3. https://doi.org/10.1016/j.jcjo.2012.05.005 Miller, G. (2011). Social Scientists Wade Into The Tweet Stream. Science, 333(6051), 1814–15. https://doi.org/10.1126/science.333.6051.1814 Miller, V. (2008). New Media, Networking and Phatic Culture. Convergence: The International Journal of Research into New Media Technologies, 14(4), 387–400. https://doi.org/10.1177/1354856508094659 Mishori, R., Levy, B., & Donvan, B. (2014). Twitter Use at a Family Medicine Conference : Analyzing #STFM13. Family Medicine, 46(8). Mou, Y. (2014). Presenting professorship on social media: from content and strategy to evaluation. Chinese Journal of Communication, 7(4), 389–408. https://doi.org/10.1080/17544750.2014.938669 Mowbray, M. (2014). Automated Twitter Accounts. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 183–94). New York: Peter Lang. Nason, G. J., O’Kelly, F., Kelly, M. E., Phelan, N., Manecksha, R. P., Lawrentschuk, N., & Murphy, D. G. (2015). The emerging use of Twitter by urological journals: Use of Twitter by urological journals. BJU International, 115(3), 486–490. https://doi.org/10.1111/bju.12840 Nature Publishing Group (2010). Media Options. Retrieved from: https://web.archive.org/web/20110513233735/http://www.nature.com/advertising/resources/pdf/2010nature mediakit.pdf?prod_code=NATURE&prod=Nature Nature Publishing Group. (2012). Announcement: A new iPad app for Nature readers. Nature, 492(7428), 154–154. https://doi.org/10.1038/492154a Nature Publishing Group. (2015). Author Insights 2015 survey. Figshare. Nentwich, M. (2011). Das Web 2.0 in der wissenschaftlichen Praxis. In Digitale Wissenschaftskommunikation–Formate und ihre Nutzung (pp. 35–53). Giessen. Retrieved from http://geb.uni-giessen.de/geb/volltexte/2011/8227/pdf/DigitaleWissenschaftskommunikation_2011.pdf Neylon, C. (2014, October 3). Altmetrics: What are they good for? Retrieved January 10, 2015, from http://blogs.plos.org/opens/2014/10/03/altmetrics-what-are-they-good-for/ Osterrieder, A. (2013). The value and use of social media as communication tool in the plant sciences. Plant Methods, 9(1), 26–26. https://doi.org/10.1186/1746-4811-9-26 Pasquini, L. A., & Evangelopoulos, N. (2015). Organizational identity, meaning, and values: analysis of social media guideline and policy documents (pp. 1–5). ACM Press. https://doi.org/10.1145/2789187.2789198

37 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Paßmann, J., Boeschoten, T., & Schäfer, M. T. (2014). The gift of the gab. Retweet cartels and gift economies on Twitter. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 331–344). New York: Peter Lang. Pearce, N., Weller, M., Scanlon, E., & Ashleigh, M. (2010). Digital Scholarship Considered: How New Technologies Could Transform Academic Work. In Education, 16(1), 33–44. Peters, I. (2009). Folksonomies indexing and retrieval in Web 2.0. Berlin, Allemagne: De Gruyter/Saur. Pomerantz, J., Hank, C., & Sugimoto, C. R. (2015). The State of Social Media Policies in Higher Education. PLOS ONE. https://doi.org/10(5): e0127485. doi:10.1371 Ponte, D., & Simon, J. (2011). Scholarly Communication 2.0: Exploring Researchers’ Opinions on Web 2.0 for Scientific Knowledge Creation, Evaluation and Dissemination. Serials Review, 37(3), 149–156. https://doi.org/10.1016/j.serrev.2011.06.002 Prabhu, V., & Rosenkrantz, A. B. (2015). Enriched Audience Engagement Through Twitter: Should More Academic Radiology Departments Seize the Opportunity? Journal of the American College of Radiology, 12(7), 756–759. https://doi.org/10.1016/j.jacr.2015.02.016 Priem, J. (2014). Altmetrics. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: harnessing multidimensional indicators of performance (pp. 263–287). Cambridge, MA: MIT Press. Priem, J., Costello, K., & Dzuba, T. (2012). Prevalence and use of Twitter among scholars. Presented at the Metrics 2011 Symposium on Informetric and Scientometric Research. https://doi.org/http://dx.doi.org/10.6084/m9.figshare.104629 Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Proceedings of the American Society for Information Science and Technology, 47(1), 1–4. https://doi.org/10.1002/meet.14504701201 Priem, J., Piwowar, H., & Hemminger, B. M. (2012). Altmetrics in the wild: Using social media to explore scholarly impact. arXiv Print, 1–17. Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010, October 26). Altmetrics: A manifesto. Retrieved from http://altmetrics.org/manifesto/ Procter, R. N., Williams, R., Stewart, J., Poschen, M., Snee, H., Voss, A., & Asgari-Targhi, M. (2010). If you build it, will they come? How researchers perceive and use Web 2.0. London, UK: Research Network Information. Retrieved from http://wrap.warwick.ac.uk/56246 Pscheida, D., Albrecht, S., Herbst, S., Minet, C., & Köhler, T. (2013). Nutzung von Social Media und onlinebasierten Anwendungen in der Wissenschaft. ZBW–Deutsche Zentralbibliothek für Wirtschaftswissenschaften–Leibniz-Informationszentrum Wirtschaft. Retrieved from http://www.qucosa.de/fileadmin/data/qucosa/documents/13296/Science20_Datenreport_2013_PDF_A.pdf Puschmann, C. (2014). Opening Science. Opening Science, 89–106. https://doi.org/10.1007/978-3-319-00026-8 Ratkiewicz, J., Conover, M. D., Meiss, M., Gonçalves, B., Flammini, A., & Menczer, F. (2011). Detecting and tracking political abuse in social media. In Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media (pp. 297–304). Ratkiewicz, J., Conover, M., Meiss, M., Gonçalves, B., Patil, S., Flammini, A., & Menczer, F. (2011). Truthy: mapping the spread of astroturf in microblog streams (p. 249). ACM Press. https://doi.org/10.1145/1963192.1963301 Reinhardt, W., Ebner, M., Beham, G., & Costa, C. (2009). How People are using Twitter during Conferences. In Creativity and Innovation Competencies on the Web, Proceeding of 5th EduMedia conference (pp. 145– 156). Salzburg: Hornung-Prähauser, V., Luckmann, M. (Ed.). Retrieved from http://lamp.tu- graz.ac.at/~i203/ebner/publication/09_edumedia.pdf Robinson-García, N., Torres-Salinas, D., Zahedi, Z., & Costas, R. (2014). New data, new possibilities: Exploring the insides of Altmetric.com. El Profesional de La Información, 23(4), 359–366. https://doi.org/10.3145/epi.2014.jul.03 Rogers, R. (2014). Debanalising Twitter. The Transformation of an Object of Study. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. ix–xxvi). New York: Peter Lang. Rothschild, S., & Unglesbee, B. (2013, September). Kansas University professor receiving death threats over NRA tweet. The Dispatch. Retrieved from http://www.shawneedispatch.com/news/2013/sep/24/kansas- university-professor-receiving-death-threat/ Rowlands, I., Nicholas, D., Russell, B., Canty, N., & Watkinson, A. (2011). Social media use in the research workflow. Learned Publishing, 24(3), 183–195. https://doi.org/10.1087/20110306 Schmidt, J.-H. (2014). Twitter and the Rise of Personal Publics. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 3–12). New York: Peter Lang.

38 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Seglen, P. O. (1997). Citations and journal impact factors: questionable indicators of research quality. Allergy, 52(11), 1050–1056. https://doi.org/10.1111/j.1398-9995.1997.tb00175.x Severo, M., Giraud, T., & Pecout, H. (2015). Twitter data for urban policy making: an analysis on four European cities. In C. Levallois (Ed.), Handbook of Twitter for Research. EMLYON. Sharma, N. K., Ghosh, S., Benevenuto, F., Ganguly, N., & Gummadi, K. (2012). Inferring who-is-who in the Twitter social network (p. 55). ACM Press. https://doi.org/10.1145/2342549.2342563 Shuai, X., Pepe, A., & Bollen, J. (2012). How the scientific community reacts to newly submitted preprints: article downloads, Twitter mentions, and citations. PloS One, 7(11), e47523–e47523. https://doi.org/10.1371/journal.pone.0047523 Shulman, J., Yep, J., & Tomé, D. (2015). Leveraging the Power of a Twitter Network for Library Promotion. The Journal of Academic Librarianship, 41(2), 178–185. https://doi.org/10.1016/j.acalib.2014.12.004 Sopan, A., Rey, P. J., Butler, B., & Shneiderman, B. (2012). Monitoring Academic Conferences: Real-Time Visualization and Retrospective Analysis of Backchannel Conversations. 2012 International Conference on Social Informatics, (SocialInformatics), 62–69. https://doi.org/10.1109/SocialInformatics.2012.20 Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: a review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037– 2062. https://doi.org/10.1002/asi.23833 Takhteyev, Y., Gruzd, A., & Wellman, B. (2012). Geography of Twitter networks. Social Networks, 34(1), 73–81. https://doi.org/10.1016/j.socnet.2011.05.006 Taylor, M. (2013). Towards a common model of citation : some thoughts on merging altmetrics and bibliometrics. Research Trends, (35), 1–6. Tenopir, C., Volentine, R., & King, D. W. (2013). Social media and scholarly reading. Online Information Review, 37(2), 193–216. https://doi.org/10.1108/OIR-04-2012-0062 Thangasamy, I. A., Leveridge, M., Davies, B. J., Finelli, A., Stork, B., & Woo, H. H. (2014). International Urology Journal Club via Twitter: 12-Month Experience. European Urology, 66(1), 112–117. https://doi.org/10.1016/j.eururo.2014.01.034 Thelwall, M., Tsou, A., Weingart, S., Holmberg, K., & Haustein, S. (2013). Tweeting Links to Academic Articles. Cybermetrics: International Journal of Scientometrics, Informetrics and Bibliometrics, 17(1), 1–8. Topf, J. M., & Hiremath, S. (2015). Social media, medicine and the modern journal club. International Review of Psychiatry, 27(2), 147–154. https://doi.org/10.3109/09540261.2014.998991 Tsou, A., Bowman, T. D., Ghazinejad, A., & Sugimoto, C. R. (2015). Who tweets about science? In Proceedings of the 2015 International Society for Scientometrics and Informetrics (pp. 95–100). Istanbul, Turkey. Retrieved from https://pdfs.semanticscholar.org/81fe/8b63188cf25648a7c592bc6b5457fee3c101.pdf?_ga=1.184338726.126 4550827.1478885332 Twitter. (2017, April 6). Automation rules. Retrieved from https://support.twitter.com/articles/76915 Twitter. (n.d.). The Twitter Rules. Retrieved from https://support.twitter.com/articles/18311 Van Noorden, R. (2013). Brazilian citation scheme outed. Nature, 500(7464), 510–511. https://doi.org/10.1038/500510a Van Noorden, R. (2014). Online collaboration: Scientists and the social network. Nature, 512(7513), 126–129. https://doi.org/10.1038/512126a Veletsianos, G. (2012). Higher education scholars’ participation and practices on Twitter. Journal of Computer Assisted Learning, 28(4), 336–349. https://doi.org/10.1111/j.1365-2729.2011.00449.x Veletsianos, G. (2013). Open practices and identity: Evidence from researchers and educators’ social media participation. British Journal of Educational Technology, 44(4), 639–651. https://doi.org/10.1111/bjet.12052 Verishagen, N., & Hank, C. (2014). Are there birds in the library? The extent of Twitter adoption and use by Canadian academic libraries. First Monday, 19(11). https://doi.org/10.5210/fm.v19i11.4945 Wang, X., Xu, S., Peng, L., Wang, Z., Wang, C., Zhang, C., & Wang, X. (2012). Exploring scientists’ working timetable: Do scientists often work overtime? Journal of Informetrics, 6(4), 655–660. https://doi.org/10.1016/j.joi.2012.07.003 Warzel, C. (2014, April 15). Meet The Man Behind Twitter’s Most Infamous Phrase. BuzzFeed News. Retrieved from https://www.buzzfeed.com/charliewarzel/meet-the-man-behind-twitters-most-infamous- phrase?utm_term=.jwWokRoZw#.fnElNylG4 Weller, K., Bruns, A., Burgess, J., Mahrt, M., & Puschmann, C. (2014). Twitter and Society: An Introduction. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. xxiv– xxxviii). New York: Peter Lang.

39 Glänzel, Moed, Schmoch, & Thelwall (Eds.) Handbook of Quantitative Science and Technology Research

Weller, K., Dornstädter, R., Freimanis, R., Klein, R. N., & Perez, M. (2010). Social software in academia: Three studies on users’ acceptance of Web 2.0 Services. Retrieved from http://journal.webscience.org/360/2/websci10_submission_62.pdf Weller, K., Dröge, E., & Puschmann, C. (2011). Citation Analysis in Twitter : Approaches for Defining and Measuring Information Flows within Tweets during Scientific Conferences. In Proceedings of the ESWC2011 Workshop on “Making Sense of Microposts”: Big things come in small packages (Vol. 718, pp. 1–12). Heraklion, Greece. Retrieved from http://ceur-ws.org/Vol-718/msm2011_proceedings.pdf Weller, K., & Puschmann, C. (2011). Twitter for Scientific Communication : How Can Citations / References be Identified and Measured ? In Proceedings of the ACM WebSci’11. Koblenz, Germany. Whitburn, T., Walshe, C., & Sleeman, K. E. (2015). International Palliative Care Journal Club on Twitter: Experience so far. BMJ Supportive & Palliative Care, 5(1), 120.2-120. https://doi.org/10.1136/bmjspcare- 2014-000838.48 White, H. D. (2000). Towards ego-centered citation analysis. In B. Cronin & H. B. Atkins (Eds.), The Web of Knowledge: A Festschrift in Honor of Eugene Garfield (pp. 475–496). Medford, NJ: Information Today. White, H. D. (2001). Authors as citers over time. Journal of the American Society for Information Science and Technology, 52(2), 87–108. https://doi.org/10.1002/1097-4571(2000)9999:9999<::AID-ASI1542>3.0.CO;2-T Wilkinson, C., & Weitkamp, E. (2013). A case study in serendipity: environmental researchers use of traditional and social media for dissemination. PloS One, 8(12), e84339–e84339. https://doi.org/10.1371/journal.pone.0084339 Wilson, M. W., & Starkweather, S. (2014). Web Presence of Academic Geographers: A Generational Divide? The Professional Geographer, 66(1), 73–81. https://doi.org/10.1080/00330124.2013.765290 Woolston, C. (2015). Potential flaws in genomics paper scrutinized on Twitter. Nature, 7553(521), 397. https://doi.org/10.1038/521397f Work, S., Haustein, S., Bowman, T. D., & Larivière, V. (2015). Social Media in Scholarly Communication. A Review of the Literature and Empirical Analysis of Twitter Use by SSHRC Doctoral Award Recipients (Study commissioned by the Social Sciences and Humanities Research Council). Yang, C., Harkreader, R. C., & Gu, G. (2011). Die Free or Live Hard? Empirical Evaluation and New Design for Fighting Evolving Twitter Spammers. In R. Sommer, D. Balzarotti, & G. Maier (Eds.), Recent Advances in Intrusion Detection (Vol. 6961, pp. 318–337). Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-23644-0_17 Zedda, M., & Barbaro, A. (2015). Adoption of web 2.0 tools among STM publishers. How social are scientific journals? Journal of the EAHIL, 11(1), 9–12. Zhang, C. M., & Paxson, V. (2011). Detecting and analyzing automated activity on Twitter. In Proceedings of the 12th International Conference on Passive and Active Measurement PAM’11 (pp. 102–111). Berlin, Heidelberg: Springer. Retrieved from http://www.icir.org/vern/papers/pam11.autotwit.pdf Zhao, D., & Rosson, M. B. (2009). How and why people Twitter: the role that micro-blogging plays in informal communication at work (p. 243). ACM Press. https://doi.org/10.1145/1531674.1531710

40 Glänzel, Moed, Schmoch, & Thelwall (Eds.)