sign in sign up
<<
Home , Altmetric, Altmetrics, Citation index, CiteScore, Essay, Faculty of 1000

University of Arkansas, Fayetteville ScholarWorks@UARK Libraries Faculty Publications and University Libraries Presentations

2017 Measure for Measure: Beth Juhl University of Arkansas, Fayetteville, [email protected]

Follow this and additional works at: https://scholarworks.uark.edu/libpub

Recommended Juhl, Beth, "Measure for Measure: Altmetrics" (2017). University Libraries Faculty Publications and Presentations. 31. https://scholarworks.uark.edu/libpub/31

This is brought to you for free and by the University Libraries at ScholarWorks@UARK. It has been accepted for inclusion in University Libraries Faculty Publications and Presentations by an authorized administrator of ScholarWorks@UARK. For more information, please contact [email protected]. Bibliographic Essay

Measure for Measure: Altmetrics

By Beth Juhl

nless you are a professional bibliometrician, the proliferation separate company, Analytics. The citation indexes and measures developed of information and statistics about scholarly publications by the predecessor Institute for Scientific Information (ISI), founded in 1960 by and the burgeoning set of tools to capture and analyze those Garfield, were acquired by Thomson in 1992 to become Thomson ISI.6 This toolkit data can be confounding. This “Cambrian explosion of continues to form the basis for librarians’ collection decisions, as it has informed the metrics”1 is recent—a fast-moving, visibly evolving system, but one that has evaluation of faculty tenure and promotion U dossiers and is still misguidedly consulted already left a few fossils in its wake. The taxonomy, too, of this new era is by many to compare journals from different disciplines, disregarding different citation still being developed. As a shorthand term, “altmetrics” encompasses a range and publication patterns. New tools and methods to analyze citation of measures from the extension of traditional (with article- data emerged throughout the 2000s. In 2004, launched a new database, level metrics or ALMs) to ’s mentions, likes, tweets, and shares. , poised to be a direct competitor to . also appeared This essay offers a snapshot of the transforming landscape of new metrics: on the scene in 2004. All three services offered large, cross-disciplinary article search embarking from a brief discussion of measure that was itself designed to filter engines. At the same time, each could also the seminal works and tools for tracing the proliferating number of scientific be mined for citation analysis at the author, scholarly impact, and moving on to survey publications for quality and content article, journal, subject area, and (in the 4 some of the key publications addressing relevant to scientific researchers. Garfield case of Web of Science and Scopus), the the need for—and use of—an alternative has reiterated in numerous interviews and institutional level. Physicist Jorge E. Hirsch set of metrics. The essay then identifies articles that the Journal , (Univ. of California San Diego) proposed and describes the new measurement tools which measures the number of articles cited the h-index in 2005 as a measure of the total currently available to institutions and in individual journals, should never be used number of papers published by a scholar individuals. Cited works at the end of to assess the relative importance of particular and the times those papers have been cited the essay point out those tools and web articles, the stature of specific scholars, by others.7 The h-index can be applied to resources that are open-access and those that or the research rankings of organizations entire journals as well as authors, but like require a paid subscription. (departments, labs, or other research the Journal Impact Factor, the standard institutions).5 Yet to some, numbers seem so h-index score should not be compared across “The Number That’s scientific, unbiased, and straightforward that incommensurable disciplines with different many researchers and their host of assessors citation patterns. Nascent cross-disciplinary 2 Devouring Science” cannot seem to resist their allure. The indexes Google Scholar and Elsevier’s Scopus Journal Impact Factor and its trademarked quickly adopted the h-index, as did Web of The search for alternative metrics can companion measures—the Immediacy Science. Later in the decade, bioinformatics be traced to the growing discontent with Index® and the Cited Half-Life®—are central professors Carl Bergstrom and Jevin West the use and misuse of the Journal Impact to the (JCR) and (both, Univ. of Washington) proposed the Factor developed by Eugene Garfield3—a Web of Science citation-index databases ®, another journal-level metric published for many years by Thomson based on examination of five years of Journal Beth Juhl is Web Services Librarian at the University of Reuters, whose property Citation Reports data. Their lab went on Arkansas Libraries. division is now, since October 2016, a to produce the Article Influence® score to

June 2017 CHOICE 1 BIBLIOGRAPHIC ESSAY

rank individual articles by frequency of more interactive means to engage with archive reflects the immediate flurry citation. Both not-for-profit and commercial authors’ recent publications and ideas. of discussion, linking, and sharing of the publishers explored new ways to help their Traditional needed to expand altmetrics manifesto. Traditional metrics were authors and visualize citation to encompass webometrics,9 measuring too slow to relay the story of this new idea. patterns and relative influence. Open-access the sharing and linking of research in a publisher PLOS (known variously as PLOS networked environment. As every academic The altmetrics proposition emerged at a or the Public Library of Science) introduced field began to adopt social media for time when the open-access movement had its Article-Level Metrics (ALMs) in 2009, not scholarly communication, tweets and likes moved past its own manifestos and toward only tracking views on PLOS but also on joined the mountain of saves, shares, links, implementation. The two initiatives are the PubMed Central full-text platform (and and downloads that might be mined as intertwined and in many ways mutually the larger PubMed database). In 2015, the indicators of influence and impact. beneficial, as open, freely accessible articles National Institutes of Health introduced may be more widely shared and discussed, In October 2010, Jason Priem (then a leading to more appearances in the venues a beta version of a new bibliometric 12 tool, iCite. Leveraging PubMed article graduate at the Univ. of North measured by altmetrics. New measures metadata, iCite presents citation reports for Carolina-Chapel Hill, now head of the logically would allow various scholarly individual articles, authors, departments, firmImpactstory , which he cofounded with societies, publishers, and funders to show labs, or institutions. Users can upload up Heather Piwowar) and his colleagues posted the reach, immediacy, and impact of their to 1,000 PubMed IDs (PMIDs) to view “Altmetrics: A Manifesto.” They appropriated new publication models. Launched at the data about to those articles from a Twitter hashtag for its name and claimed Annual Meeting of the American Society other publications indexed in PubMed. the domain name altmetrics.org (although for Cell in 2012, the San Francisco Even as this article went to press, Elsevier the URL today is mostly a placeholder for Declaration on Research Assessment (DORA) announced a new journal-level metric, the essay). The neologism coined by Priem outlines the limitations of the Journal CiteScore, based on Scopus database metadata (with Dario Taraborelli, Paul Groth, and Impact Factor, and its authors have called covering 20,000-plus journals, offered ) served as both a summary for shifting away from journal-level to as a more encompassing, freely available of the widespread restiveness with traditional article-level metrics, as well as examining alternative to the Journal Impact Factor. The bibliometrics and as a call to action to build other plausible output measures such as CiteScore suite of metrics covers all articles better tools. Like Garfield’s 1955 proposal for research data sets released, the number and communications in a journal, not just citation indexes (as a means to “eliminate the and quality of trained, and the research articles analyzed by the Journal uncritical citation of fraudulent, incomplete, funding awarded. As documented in the or obsolete data,” with the goal “to establish DORA News Archives, the declaration also Citation Reports.“Simple metrics tend to 10 count what is easily counted, such as articles the history of an idea”), altmetrics is all demanded open access to citation data for and citations in established journals, rather about filters, albeit different ones. The external analysis. That same year, the Public than what is most valuable or enduring.”8 manifestants characterized traditional filters Library of Science—the open-access journal such as citation counting, the Journal Impact publisher of PLOS Biology, PLOS ONE, However innovative and revealing these Factor, and the peer-review process as dated, and other similarly titled PLOS journals— emerging journal and article metrics may narrow, and threatened with becoming launched a specialized PLOS Collections: have been, many felt they fell frustratingly “swamped,” and they sought new ways to Altmetrics website with articles drawn short in presenting a full picture of scholarly “reflect and transmit scholarly impact” while from the site’s archives and blogs. Soon influence and impact. First, the scientific sifting through the exploding volume of after, the National Information Standards peer-review and publication process academic .11 Rejecting journal-level Organization (or NISO) received a grant in remained cumbersome, inefficient, and slow, measures and proprietary data sources in 2013 from the Alfred P. Sloan Foundation to and it was not adequate in keeping up with favor of article-based metrics drawn from study the promise of altmetrics, as described the increasing pace of research. Second, relatively open data sets such as citation- and in NISO Alternative Assessment Metrics citation-analysis tools for document-management platforms (Altmetrics) Initiative, and released in 2016 were inadequate or nonexistent, even with and Zotero, the altmetrics proponents their recommended practices in Outputs the of also sought methods to evaluate emerging of the NISO Alternative Assessment Project. to the Web of Science suite of databases, formats leading up to publication, including The validity and scalability of altmetrics largely leaving humanities scholarship out raw research data, blogs, comments, and are still hotly contested, questioning what of the frame. Finally, the extant citation other informal—but more immediate— to measure, how to measure it, and how to tools did not address the developing body communications. Although Priem and his address problems of bias toward more recent of published items preserved on the web, colleagues did not propose what form these publications, and countering the perhaps or explain how research publications new tools would take, the cogency of their inevitable claims of emphasis on narcissistic might make (and be shown to have made) rallying cry could be seen in the citation utterances or glib ephemera. But it appears an impact outside of academe. Even in counts for their manifesto on Web of Science for now that alternative metrics have found journal-based disciplines, the growth of and Google Scholar Citations, each of which a place at the scholarly communications web forums changed the way scholars trace only a trickle of formal citations to table, as evidenced by their adoption by responded to research, offering faster, the piece before 2013. Meanwhile, the publishers, funders, and .

2 CHOICE June 2017 BIBLIOGRAPHIC ESSAY

and CEO of ) and William Gunn We’re British,” and the forthright “Sick of Getting a Grip— (formerly at Mendeley, and now directing Impact Factors.” While Scholarly Metrics Handbooks and Guides scholarly communications at Elsevier). looks backward and only touches briefly on newer measures such as the h-index Greg Tananbaum’s Article-Level Metrics: as applied in Google Scholar Citations, Delivering on its subtitle, Meaningful A SPARC Primer, posted on the SPARC Metrics: A 21st-Century Librarian’s Cronin and Sugimoto’s volume with newly (Scholarly and Academic contributed essays, Beyond Bibliometrics: Guide to Bibliometrics, Altmetrics, and Resources Coalition) website of the same Research Impact, by Robin Chin Roemer Harnessing Multidimensional Indicators name, compares journal and emerging of Scholarly Impact, includes a section on (Univ. of Washington) and Rachel article metrics, offering observations on Borchardt (American Univ.), offers a altmetrics and developing fields such as the potential applications and limitations. intriguing “academic genealogy.” comprehensive overview of tools and One can find examples of how library staff methodologies as well as practical advice are integrating altmetrics into instruction National academies, funding agencies, and for librarians hoping to raise awareness and outreach programs, along with government oversight groups are searching of the promise and the pitfalls of metrics sample library guides, reports, and service for meaningful methodologies to assess on their campuses. Though aimed at from Association of Research the research contributions of institutions academic librarians, Meaningful Metrics Libraries (ARL) members in Ruth Lewis and individuals. As mentioned above, is a highly readable and thorough guide and coauthors’ Scholarly Output Assessment the NISO Alternative Metrics Initiative for any interested reader. The authors Activities, part of ARL’s long-standing launched in the US in 2013. In the UK, a previously produced a freely available “SPEC Kit” series. And some of the 2015 report commissioned by the Higher “Altmetrics” basic overview in Library clearest, most straightforward introductions Funding Council, The Metric Technology Reports covering much of the to altmetrics can be found in academic Tide, by James Wilsdon et al., surveyed the same territory, and they recommended librarians’ freely accessible library guides current landscape, reviewed types of metrics websites and readings in two earlier, brief (e.g., from the University of Pittsburgh and available, and discussed the challenges facing essays also published by the Association of Utrecht University), or ACRL’s pertinent consumers of those new metrics. The Metric College and Research Libraries (ACRL), LibGuide called Scholarly Communication Tide led to the formation of an organization “From Bibliometrics to Altmetrics” and Toolkit. Vendors, too, supply practical guides and website, Responsible Metrics, which has “Keeping Up with ... Altmetrics.” for advocates planning workshops and doing not been updated in some time; sadly, their In the UK, several authors affiliated with outreach, or using altmetrics in collection proposal for a “Bad Metric” award does the Statistical Cybermetrics Research , for example Altmetrics for not seem to have been realized, although Group at the University of Wolverhampton Librarians: 100+ Tips, Tricks, and Examples, a Twitter hashtag, #ResMetrics, carries produced monographs touching on both by Stacey Konkiel and coauthors. on the conversation. In Europe, SURF, a traditional bibliometrics and alternative collaborative organization for technology in measures. Of these, Kim Holmberg’s Stepping Back— higher education, produced the provocatively Altmetrics for Information Professionals: Past, titled Users, Narcissism and Control, by Paul Present and Future provides a slim but well- Research and Wouters and Rodrigo Costas, which presents documented survey of the development a hopeful look at new technologies but and potential of the new metrics, while Assessment concludes that new measures fail to normalize David Stuart’s Web Metrics for Library and data across disciplines and need to be more Information Professionals travels farther afield Blaise Cronin (former editor in chief transparent about their data sources and to address link analysis, full-text queries, of the Journal of the American Society for depth of coverage. analysis of web logs, and big data. Michael Information Science & Technology) and Thelwall, who leads the Wolverhampton his coauthors and colleagues at Indiana Keeping Up—Current research group, has published two works University and around the globe continue aimed at social scientists and information to contribute important research articles on Developments professionals in publisher Morgan developing measures of scholarly impact. and Claypool’s “Synthesis Lectures on Think an anthology of previously published As in most developing fields, published Information Concepts, Retrieval, and articles on bibliometrics sounds as dry research on altmetrics can mainly be found Services” ebook series: Introduction to as day-old toast? Try a taste of Scholarly in the journal literature. A remarkable range Webometrics: Quantitative Web Research Metrics under the Microscope, ed. by Blaise of discipline-specific journals have offered for the Social Sciences and Web Indicators Cronin and Cassidy R. Sugimoto—a hefty articles addressing the potential and pitfalls for Research Evaluation: A Practical Guide. tome that appears completely daunting but of new metrics for researchers in particular Andy Tattersall (Univ. of Sheffield) has tempts with some of the most accessible, fields of study, from Academic Emergency edited Altmetrics: A Practical Guide for thoughtful essays from the last few decades, Medicine,13 to ,14 to the Journal of Librarians, Researchers, and Academics, with including contributors’ clever titles such Wildlife Management.15 The key research essays contributed from such leaders in the as “Bibliometrics as Weapons of Mass articles are found in journals addressed to altmetrics movement as Euan Adie (founder Citation,” “No Citation Analyses, Please, practitioners and information professionals

June 2017 CHOICE 3 BIBLIOGRAPHIC ESSAY

studying bibliometrics and scholarly to “3:am” in Bucharest in 2016. Through Web of Science (or may have one already communication. The Journal of the American the conference site, one can access blog created for them), and add or correct Society for Information Science and Technology posts, presentations, and streaming video of associated citations, research interests, or offers many timely studies of, for example, selected sessions. biographical information. Authors can the correlation (or lack of correlation) of register for or claim a Researcher ID even if altmetrics scores and the Journal Impact their institution does not have a subscription Factor, or analysis of readership levels in Altmetrics Websites to the Web of Science. Elsevier’s Scopus also various disciplines. The companion Bulletin and Tools offers a unique researcher ID, although users of the American Society for Information cannot manage their own identities and can Science and Technology has devoted a special Librarians live to classify and it only request corrections. issue to “Altmetrics: What, Why and is helpful to organize the new metrics Google Scholar Citations, introduced in Where?” Key journals Research Evaluation into categories—a sort of who-to-whom 2012, allows scholars to create a personal (Oxford) and (Springer) matrix of interacting variables—defining profile that can be shared publicly and primarily address the sciences, while measurables (saves, tweets, posts, shares, displayed in search results, or kept Evidence Based Library and Information likes, views, downloads, etc.) and the private. Authors can add or edit their Practice, an open-access journal published sources of who or what is being assessed list of publications, either by retrieving by the University of Alberta, offers practical (single articles or other publications, them from Google Scholar or by adding advice for academic librarians, including individual authors, journals, or publishing them manually. The citations profile page a research-methods feature treating both platforms, or organizations such as labs, provides a metric snapshot for all the bibliometrics and altmetrics. academic departments, universities, publications associated with the author, etc.). Additionally, it is useful to know But, as the altmetrics proselytizers including total number of citations to which websites and tools individuals can themselves would observe, peer-reviewed these works from across Google Scholar, access freely themselves, and which are journal literature isn’t able to keep up with for total citations, the h-index for that available only to institutional subscribers. the rapidly developing marketplace of new author (a formula for the number of Unfortunately, the artificially neat lines of measures. Following the Twitter hashtag papers published and the number of times any organizing matrix soon begin to blur #altmetrics (note—not #altmetric, which cited), and the i10 index, a derivative of and dissolve, as some tools, such as peer belongs to a company that will be described the h-index that examines papers cited networks like Mendeley or Academia, act below) all but guarantees a deluge of news, more than ten times within the Google as both source data and the measure itself, observation, and opinion. For exposure to Scholar database. Though one can link to while other tools, such as Altmetric, offer more nuanced ruminations, try The Scholarly a public profile page, it is difficult to reuse limited free access to individuals and a more Kitchen blog from the Society for Scholarly the profile identifier on other websites or fully functioning product for purchase by Publishing. Featuring contributions from an platforms. impressive salon of authors, publishers, and institutions, funding agencies, or publishers. librarians, this resource regularly serves up What follows is a list of the main resources ORCID—the Open Researcher and perceptive insights, particularly from Rick organized by basic function, with occasional Contributor ID—launched in 2012, Anderson (Univ. of Utah), Todd Carpenter hopscotching across categories. provided the interoperability that Google (NISO), and others who have viewed Scholar lacked and is a nonproprietary from many angles. alternative to the Researcher ID. One’s The site also offers an archived “Metrics and Identities ORCID provides each registered scholar Altmetrics” collection, a regular podcast with a unique alphanumeric code and is series that features Jason Priem interviewed Just as the disaggregating power of now commonly used on curricula vitae, by Stewart Wills (“Altmetrics, Today and the web drove the need for the Digital in scholarly publishing workflows, and Tomorrow”), and an “Ask the Chefs” Object Identifier or DOI as a means to as a registration service that can link roundtable discussion led by Ann Michaels locate specific versions of articles, chapters, various accounts together across the web. on the role of social media in publishing. , journals, or other publications As an open-source tool, ORCID can be A number of commercial providers also on distinct platforms,16 various unique integrated into a variety of web services, host substantial blogs that go beyond mere identifiers for individual authors help to pull including altmetrics tools such as Altmetric, marketing, including an Altmetric Blog at together a unified portrait of an author’s Impactstory, and Plum Analytics. In a nod to https://www.altmetric.com/blog/ and the publications and affiliations and allow the ubiquity of ORCID, Thomson Reuters’ Plum Analytics Blog at http://plumanalytics. automated systems, databases, and websites Researcher ID and the Scopus Author com/interact/blog/. Since 2014, The to link related items to the correct “Joan Identifier can now pull citations from or Altmetrics Conference has attracted scholars, Smith.” For many years, the Researcher ID®, push citations to linked ORCID accounts, publishers, librarians, and vendors who Thomson-Reuters’ proprietary identifier so that publication and personal details do convene annually in various European used by the Web of Science database, was not have to be entered in multiple places. cities—from the “1:am” first altmetrics the leading tool for author disambiguation. Individuals can register for an ORCID meeting in London, “2:am” in Amsterdam, Users can sign up for a Researcher ID in identifier for free; institutions may join at

4 CHOICE June 2017 BIBLIOGRAPHIC ESSAY

several different levels of membership in and modules such as a recently launched RG Score, which is based on interactions order to be able to curate faculty accounts datasets repository, thanks in part to the of other ResearchGate users with one’s own and to integrate ORCID information into development support of its new owner, contributions, discussions, and answers campus reporting systems. Elsevier. Mendeley not only offers fully to research questions; institutions or featured reference-management software academic departments can also be assigned Approaching identities from a different complete with word-processor integrations an RG Score. As with so many other new Crossref angle, the organization (previously and shared collections, but also supports a metrics, the score is intriguing but, being styled CrossRef) launched an open registry robust collaboration environment that links based on a self-selected community of of funder identities in 2013 to essentially like-minded groups, claiming more than five varying depth by discipline, it is difficult normalize funders’ names for better cross- million users worldwide. A personal Stats to know exactly what such numbers mean Funder Data Crossref platform reporting; its — dashboard displays the number of citations outside of their own network. Meta-Data Search (known also as FundRef) to one’s work (and the h-index measure, from of more than 14,000 registered entities Scopus), plus views or readers of one’s articles SSRN, the Social Science Research Network is now accessible via the site’s metasearch (from ScienceDirect and Mendeley databases). (with more than 2 million users when it DataCite engine. , a membership organization The major altmetrics services, including was acquired in May 2016 by Elsevier), was supported by an alphabet soup of public Impactstory, Altmetric, and Plum Analytics, established in 1994—long before either and private institutions from more than all harvest and report user activity on the Academia or ResearchGate—and it offers a twenty countries—and coordinated with Mendeley platform. rigorously organized repository for social the International DOI Foundation and other sciences , conference papers, and metadata initiatives—serves as a registry and Like Mendeley, Academia and ResearchGate other publications. Participation in SSRN is DOI clearinghouse for research data sets, were also founded in 2008 but they free though the site also sells subscriptions an important function complementary to marked a distinct break from the world of to subject portals and offers pass-throughs publication and personal IDs. bibliographic reference managers. Instead, to some publishers for full-text article access. each serves more as a paper repository and Posted rankings offer top downloads by hub for making scholarly connections. subject area, author, and organization. SSRN Boasting more than 45 million users had a particularly high participation rate in Scholarly Networks worldwide, Academia (previously named some areas, such as business and law, but “We see only what we know,” as Johann Academia.edu) offers user profiles and its acquisition by Elsevier caused a storm Wolfgang von Goethe observed.17 To subject tags based on article uploads and of concern among open-access advocates. many librarians, new web-based reference- manually created cited references. By While SSRN is still freely available, new management tools such as CiteULike, Zotero, assigning tags to one’s collected articles and initiatives such as SocArXiv Preprints offer a Papers, or Mendeley looked like services choosing to follow particular scholars, one fully noncommercial alternative for social they already knew, offering a way to collect, builds a personal landing page that supplies scientists to archive their published works in organize, and format the cited references related articles of potential interest. Users earlier stages, from working papers presented in one’s . At first, these have access to real-time analytics showing at conferences to final preprints. SocArXiv appeared to be only web-based alternatives to profile and document views; alerts emails has been developed in partnership with the traditional, commercially produced citation- notify users of new visits or mentions in Center for ’s Open Science manager tools like EndNote, ProCite, Reference recently uploaded papers. Framework, an initiative that promises to help Manager, or RefWorks. But the new open- manage the workflow of scholarship from the source platforms soon evolved beyond simply Academia has no particular subject inception of a research project to publication, managing bibliographic references into affiliation, whereas ResearchGate, which with an analytics section documenting usage. collaborative systems for sharing, discussion, claims more than 12 million users, focuses and . While not primarily on the sciences. Like the other tools, the Beyond bibliographic products, figshare, a designed to convey the stature of authors free profile setup on ResearchGate walks part of the family (owned or the reach of their publications, each tool users through a list of possible name by Macmillan, along with ReadCube and preserves source data that can be measured in variations and suggests colleagues that Altmetric), similarly offers repository terms of saves, shares, and downloads. one may wish to follow based on common space for research data. Individuals may research interests or co-citations. An join for free and upload both public and CiteULike and Mendeley were developed by interesting organizing feature is the ability private research materials and see real-time UK academics, and both launched in 2008. to group articles or other publications information on views and downloads. CiteULike is a browser-based bookmark tool into projects and to post information Institutional subscriptions allow universities whose design has remained fairly stable since about current projects underway. Personal to host all their research data in one place, its original release, allowing users to cite and statistics present profile views, reads or to promote their research outputs by making save references to their personal libraries, article views, and citations; statistics are them publically available with stable DOIs, share those libraries, and connect with others driven by the full-text availability of papers and to view insights from usage analytics who save the same items. Mendeley, on the uploaded to the system. ResearchGate offers at the item, researcher, department, or other hand, continues to add new features researchers their h-index value as well as an institution level.

June 2017 CHOICE 5 BIBLIOGRAPHIC ESSAY

Open Peer Review peer review site offers “a measurable EBSCO after its three-year ownership. indicator of a researcher’s expertise and “Rather than portending something Some measures of impact are difficult contributions” and provides a means for amiss in the altmetrics space,” Todd to express numerically but nonetheless editors and publishers to identify expert Carpenter wrote in The Scholarly Kitchen, carry weight within research communities. reviewers. The site allows reviewers to rate “this deal appears to signal a developing Open peer-review initiatives—both pre- and one another and presents ranked lists of understanding of where altmetrics sit in post-publication—have varying goals: to reviewers by field, by reviewer institution, academia and who is most interested in accelerate the path to publication, allow or country. Altmetric incorporates them and why.” Carpenter suggests that more voices to contribute to assessment, metrics into its Attention Score, described the deal situates Plum Analytics more and acknowledge the contributions of below. Another site, F1000: Faculty of successfully in the suite of assessment tools reviewers themselves. Several of the services 1000 and its F1000Prime component, and publishing integrations already offered mentioned above offer some form of goes beyond expert peer review to provide by Elsevier, though he also voices concerns nontraditional peer review, from informal article recommendations from researchers that this acquisition might lead to walling discussions (such as various Mendeley in the fields of biology and medicine. off altmetrics source data as publishers forums) to explicit post-publication review While the core product (launched in align with specific providers. (as offered by the new Open Review tool 2002) is a subscription service, its F1000 from ResearchGate). Research: F1000 Faculty Reviews site PLOS ONE, through its comments commissions review articles on the life For Institutions functionality, has since inception allowed sciences as part of a new open-access registered users to discuss articles published publishing initiative. The subscriber Founded by medical genetics researcher in their journals. The National Institutes of portion provides article rankings that Euan Adie in 2011, the London-based Health’s PubMed Commons is more rigorous are updated daily, including a “Top 10 company Altmetric (and its website suite in that it only allows comments from Hidden Jewels” list for less high-profile of tools) provides services to institutions, authors who have articles in the PubMed specialty journals. publishers, and funders as well as individuals. database. PubMed Commons also offers Altmetric generates an Attention Score membership to journal-club discussion for a publication by factoring a growing groups based on specialty research areas. Getting It Together— collection of sources, including mainstream PubPeer is another independent journal-club Reporting Tools news reports, blogs, mentions, site that allows anonymous review, but it has saves to online reference managers (e.g., Mendeley), reviews and recommendations been embroiled in scrapes related to suspect With the multitude of venues on post-publication peer-review sites such research data and article retractions. Peerage documenting research impact to choose as Publons and F1000, blog posts, social of Science aims to transform pre-publication from, one could easily devote so much media mentions, and posts to YouTube or review by providing a free platform where time and energy to monitoring one’s authors can submit their work and solicit discussion sites such as Reddit. The Altmetric scholarly reputation that there is no Support portal (and Knowledge Base, in its comment from peers (volunteer scientist- time left to actually publish research. reviewers who are vetted by virtue of their Solutions section) provides a chronology and Fortunately, several startups have stepped complete list of sources. The Attention Score publication records in ranking international into the marketplace with services that outlets—ironically, the journals indexed in is computed from number of mentions, the aggregate different measures and provide type of source where the mentions occur (for Web of Science). Conceived in 2010 by a reporting tools at the article, individual Peerage of Science example, standard news outlets rank more group of Finnish scientists, scholar, department, or organizational has developed productive partnerships with highly than blogs), and the author of the level. Publishers and content providers both for-profit and open-access publishers, mention (with prominent persons in the field have licensed altmetrics reports as means including Springer, BioMedCentral, and ranking more highly). The Attention Score to enrich their journal sites or database Brill, who benefit from timely reviews is then presented graphically as the Altmetric platforms. At institutions whose libraries from self-selected (and thus, presumably, donut, a colorful representation of the license the Summon discovery service motivated) reviewers. Those reviewers categories where impact could be measured. from Ex Libris or Elsevier’s Scopus, for themselves benefit by being able to choose Through its Altmetric Explorer web apps example, users can observe the signature what they review and then gain recognition and various Badge products, the company Altmetric donut. Those who have used for their contributions. provides real-time metrics to institutions, EBSCO’s version of the nursing database publishers, and funders. Altmetric provides Publons goes even farther to put peer CINAHL can witness Plum Analytics in the Badge service free of charge to universities reviewers into the spotlight. Named action. These new ventures have quickly for use in institutional repositories, and as a joking reference to the “smallest moved from nascent startups to valuable makes research output data available on publishable unit” and boasting more than properties, with Altmetric becoming part of request to academic librarians. Altmetric 100,000 registered reviewers who have Macmillan’s Digital Science family in 2012, Badges for Books were introduced in 2016 uploaded more than 500,000 reviews, and as this essay goes to press in early 2017, to track book mentions and cites based on the New Zealand and UK-based Publons Elsevier has acquired Plum Analytics from ISBNs or book- and -level DOIs.

6 CHOICE June 2017 BIBLIOGRAPHIC ESSAY

PlumX metrics from Plum Analytics do representation from SPARC, Impactstory takes Researchers create a Kudos account, add not compute a numeric score, but instead a third path to reporting, preferring to help their publications or import them from report on more than sixty different types of individuals “tell stories” about their research ORCID, and then have access to existing research output—so-called “artifacts,” and rather than offering numerical scores or metrics as well as tools to maximize the the activities related to those artifacts—in graphs. Impactstory provides simple badges, visibility of each item (“increase publication five categories: usage (e.g., downloads, such as one’s “greatest hit” publication performance”) by allowing authors to plays, or WorldCat holdings), captures (based on saves or shares) or the percentage explain their research output (“What’s (exports and saves, bookmarks, subscribers), of one’s publications that are open access. it about?” Why is it important?”), link mentions (blog posts, reviews, links), Such achievements are based on Impactstory to research data sets or supplemental social media (likes, tweets, shares), and categories of buzz (the volume of discussion information, and promote their work citations (from Crossref, PubMed, Scopus, surrounding one’s research), engagement on social media channels. In addition to SSRN, etc.). Plum Analytics methods (who is accessing research outputs, and where reporting on usage through embedded match on these artifacts and aggregate they’re located), openness (the availability system widgets, Kudos pulls in the article activity as dashboards, documenting of the research to a worldwide ), Attention Score from Altmetric and trends and reach at the artifact, researcher, and fun (whimsical measures, such as one’s the Times Cited number from Web of departmental, or organization level. The popularity in Japan). Users may register for Science. Dubbed by one wag “Hootsuite company sells its PlumX Dashboards product free by linking a Twitter account and then for academia,”18 Kudos is proof that the to institutions where staff can curate import publication information from an altmetrics phenomenon has reached its next researcher and departmental profiles and ORCID profile. Impactstory is committed evolutionary stage: marketing. manage publications associated with those to open data and open source, and rewards individuals and groups. Its PlumX metrics authors with high rankings for publishing What’s Next allow institutions to embed reports directly in open venues. Reporting depends on into institutional repositories or web pages. publication identifiers like DOIs, and Altmetrics clearly have not supplanted It also offers a reporting object not unlike without them is somewhat inaccurate. but instead have supplemented the the Altmetric donut—a sunburst-shaped Journal Impact Factor. However, merely graphical representation of the categories Several browser plugins allow scholars to replacing one contested number with a and venues in which the research makes quickly look up their article metrics on the fly. suite of metrics lacking context or nuance the most impact. Other tools in the Plum Altmetric’s free Bookmarklet for Researchers lives would only give campus administrators Analytics suite report on grant activities, in one’s bookmarks bar, and when viewing and research funders a new yardstick to present benchmark comparisons, or a research article of interest, a user can click misapply and misunderstand in the same offer funding opportunities matching an an “Altmetric it!” button to view the article’s old ways. The challenge for proponents and organization’s research profile. Attention Score donut. The service currently works on pages containing a DOI, publisher adopters of altmetrics is to work toward Not content to simply purchase off-the- sites where the embedded article metadata is shared understanding of (and standards shelf solutions, leading research universities, sufficient to achieve a match, and in a range of for) what to measure and precisely what originally in the UK and now worldwide, databases such as PubMed or the open-access the measurements mean. In addition to are working with Elsevier to develop a arXiv, just one example of a specialized eprint the NISO and Snowball Metrics initiatives framework of research metrics that can be repository for the sciences where meaningful mentioned above, it is encouraging shared and compared across institutions. metrics are of obvious value. The handy Lazy to see organizations building a new The pilot Snowball Metrics project has Scholar browser extension developed by Colby open framework for metrics. The DOI produced two Snowball Metrics Recipe Vorland for Chrome (but recently disabled for registration agency Crossref, for example, Books, the latest with two dozen scholarly Firefox) is something of a Swiss Army knife expects to launch its Crossref Event Data communication measures such as research for academics, presenting the Altmetric score, service in 2017. This service will track DOIs output and citations, public engagement, the citation count from Google Scholar, and as mentioned, shared, bookmarked, or and altmetrics based on Web of Science, the Journal Impact Factor for a journal article; discussed outside of the formal literature, Scopus, Google Scholar, and other sources. in addition the Lazy Scholar toolbar allows beginning with Wikipedia, , one to locate the full text, save to Mendeley, F1000Prime, Twitter, and other sites. Such share on Twitter, and annotate documents. “event” statistics (which cover linking, For Individuals Finally, there is Scholarmeter, from the School bookmarking, commenting, social sharing— of Informatics and Computing at Indiana estimated to be more than 100,000 Unlike the high-stakes commercial University, a browser plugin in beta that offers occurrences daily) will be freely available to enterprises Altmetric and Plum Analytics, analytics on Google Scholar citation patterns. individuals to process and interpret. This the nonprofit Impactstory is funded by the level of coordination is an important step National Science Foundation and the Alfred Want to boost your scholarly visibility? in moving toward the interoperability and P. Sloan Foundation. Led by altmetrics Kudos is a service (free to individuals) that transparency that Euan Adie has called for, leading lights Heather Piwowar and Jason provides a set of tools to raise the profile assuring that source data is auditable, open, Priem, with an advisory board that includes of academics and their publications. and meaningful.19

June 2017 CHOICE 7 BIBLIOGRAPHIC ESSAY

Altmetric. Digital Science. https://www.altmetric. Carpenter, Todd. “Plum Goes Orange— Endnotes com/. ($$) Elsevier Acquires Plum Analytics.” The Scholarly Kitchen (February 2, 2017). https:// Altmetric Explorer for Institutions. https:// 1. Johan Bollen (Indiana Univ.), quoted in scholarlykitchen.sspnet.org/2017/02/02/plum- www.altmetric.com/products/explorer-for- Van Noorden, “Metrics,” 864. goes-orange-elsevier-acquires-plum-analytics/. institutions/ ($$) 2. Monastersky, “The Number That’s CiteScore: Journal Metrics. Elsevier. https:// Altmetric Explorer for Publishers. https://www. Devouring Science.” journalmetrics.scopus.com/. (O) altmetric.com/products/explorer-for- 3. Garfield, “The Impact Factor.” publishers/ ($$) CiteULike: Everyone’s Library. http://www. .org/. (O) 4. Garfield, “The History and Meaning of Altmetric Support. https://help.altmetric.com/ the Journal Impact Factor.” support/home/. Crossref. https://www.crossref.org/. (O) 5. Beyond Bibliometrics, 115–121. Altmetrics: A Practical Guide for Librarians, Crossref Event Data. http://eventdata.crossref. 6. For a comprehensive history, see De Researchers and Academics, ed. by Andy org/. (O) Tattersall. Facet Publishing, 2016. Bellis, Bibliometrics and Citation DataCite. https://www.datacite.org/. (O) The Altmetrics Conference: An Annual Conference Analysis. De Bellis, Nicola. Bibliometrics and Citation for People Who Work with Alternative Metrics. 7. Hirsch, “An Index to Quantify an Analysis: From the to http://altmetricsconference.com/. Individual’s Scientific Research Cybermetrics. Scarecrow, 2009. “Altmetrics: What, Why and Where?” [Special Output,” 16569. DORA News Archives. American Society for Cell Section]. Bulletin of the American Society for Biology. http://www.ascb.org/dora/dora- 8. Borgman, Scholarship in the Digital Age, Information Science and Technology 39, no. archives/. 239. 4 (April–May, 2013). http://www.asis.org/ 9. Björneborn and Ingwersen, “Toward a Bulletin/Apr-13/. Eigenfactor: Revealing the Structure of Science. http://www.eigenfactor.org/. Basic Framework for Webometrics.” Article-Influence Score. Thomson Reuters. 10. Garfield, “Citation Indexes for Science,” http://ipscience-help.thomsonreuters.com/ EndNote. Clarivate Analytics. http://endnote. 108, 110. incitesLiveJCR/glossaryAZgroup/g4/7790- com/ (CH, Jul’15, 52-5637). TRS.html. 11. Priem, et al., “Altmetrics: A Manifesto.” Evidence Based Library and Information Practice. Article-Level Metrics. SPARC. https://sparcopen. University of Alberta Learning Services. 12. Mounce, “Open Access and Altmetrics.” org/our-work/article-level-metrics/. https://ejournals.library.ualberta.ca/index.php/ EBLIP. 13. Barbic, et al., “An Analysis of Altmetrics Article-Level Metrics (ALMs). Public Library of in Emergency Medicine.” Science. https://www.plos.org/article-level- F1000: . http://f1000.com/ (CH, 14. “Science Metrics.” metrics. (O) Apr’07, 44-4201). ($$) 15. Krausman, Bishop, and Ottogalli, arXiv. Cornell University Library. https://arxiv. F1000Prime. http://f1000.com/prime/. (&) “Promoting Research beyond org/ (CH, Jun’15, 52-5098) . F1000 Research: F1000 Faculty Reviews. https:// Publication.” Barbic, David, et al. “An Analysis of Altmetrics f1000research.com/browse/f1000-faculty- 16. The Formation of CrossRef: A Short in Emergency Medicine.” Academic Emergency reviews. (O) History. Medicine 23, no. 3 (March 1, 2016): 251–68. figshare: Credit for All Your Research. Digital doi:10.1111/acem.12898. Science. https://figshare.com/. (&) 17. Goethe on , 7. Beyond Bibliometrics: Harnessing Multidimensional The Formation of CrossRef: A Short History. 18. Rapple, “Hootsuite for academia?” Indicators of Scholarly Impact, ed. by Blaise Crossref, 2009. https://www.crossref.org/pdfs/ 19. Adie, “The Rise of Altmetrics,” 80. Cronin and Cassidy R. Sugimoto. MIT, 2014 CrossRef10Years.pdf. (CH, Dec’14, 52-1729). Funding Data—Crossref Metadata Search. https:// Björneborn, Lennart, and Peter Ingwersen. search.crossref.org/funding. (O) Works Cited “Toward a Basic Framework for Webometrics.” Journal of the American Society for Information Garfield, Eugene. “Citation Indexes for Science: Key to pricing information for altmetrics tools/ Science and Technology 55, no. 14 (December A New Dimension in Documentation through websites: the symbol ($$) indicates a subscription 1, 2004): 1216–1227. doi:10.1002/asi.20077. Association of Ideas.” Science 122, no. 3159 resource; contact publisher for pricing. The symbol (July 15, 1955): 108–111. doi:10.1126/ Book Citation Index. Clarivate Analytics. http:// science.122.3159.108. (&) indicates some free content, but added wokinfo.com/products_tools/products/. ($$) functionality requires a subscription or membership. ______. “The History and Meaning of Sites accompanied by a price symbol (O) are open- Bookmarklet for Researchers. Altmetric. https:// the Journal Impact Factor.” JAMA 295, no. access resources, freely available to individuals. URLs www.altmetric.com/products/free-tools/ 1 (January 4, 2006): 90–93. doi:10.1001/ were last visited March 2017. bookmarklet/. (O) jama.295.1.90. Academia. https://www.academia.edu/. (O) Borgman, Christine L. Scholarship in the Digital ______. “The Impact Factor.” Current Age: Information, Infrastructure, and the Contents no. 25 (June 20, 1994): 3–7; Adie, Euan. “The Rise of Altmetrics.” In Internet. MIT, 2007 (CH, Apr’08, 45-4424). Altmetrics: A Practical Guide for Librarians, reprinted as The Thomson Reuters Impact Researchers and Academics, ed. by Andy Bulletin of the American Society for Information Factor. Clarivate Analytics. http://wokinfo. Tattersall, 67–82. Facet Publishing, 2016. Science and Technology. . http://www.asis. com/essays/impact-factor/. org/Bulletin/. Goethe on Art, selected, ed., and trans. by John

8 CHOICE June 2017 BIBLIOGRAPHIC ESSAY

Gage. University of California, 1980. Distinct but Complementary.” Bulletin of the 49-0313). (&) American Society for Information Science and Google Scholar. https://scholar.google.com/. (O) PubMed Commons: A Forum for Scientific Technology 39, no. 4 (April–May, 2013): 14– Discourse. National Center for Biotechnology Google Scholar Citations. https://scholar.google. 17. doi:10.1002/bult.2013.1720390406. Information. https://www.ncbi.nlm.nih.gov/ com/intl/en/scholar/citations.html. NISO Alternative Assessment Metrics (Altmetrics) pubmedcommons/. (&) Hirsch, J. E. “An Index to Quantify an Initiative. National Information Standards PubPeer: The Online Journal Club. https:// Individual’s Scientific Research Output.” Organization. http://www.niso.org/topics/tl/ pubpeer.com/. Proceedings of the National Academy of altmetrics_initiative/. Sciences of the of America 102, Rapple, Charlie. “Hootsuite for academia? How to Open Review. Research Gate. https:// no. 46 (November 15, 2005): 16569–72. Increase the Visibility, Downloads and Impact of www.researchgate.net/publicliterature. doi:10.1073/pnas.0507655102. Publications Using Kudos.” The London School OpenReviewInfo.html of Economics and Political Science Impact Blog. Holmberg, Kim. Altmetrics for Information Open Science Framework: A Scholarly Commons to http://blogs.lse.ac.uk/impactofsocialsciences/ Professionals: Past, Present and Future. Chandos Connect the Entire Research Cycle. Center for 2016/06/24/hootsuite-for-academia-how-to- Publishing, 2016. Open Science. https://osf.io/. increase-the-visibility-downloads-and-impact-of- iCite. Office of Portfolio Analysis, National Insti- publications-using-kudos/ Outputs of the NISO Alternative Assessment Project. tutes of Health. https://icite.od.nih.gov/. (O) NISO RP-25-2016. National Information ReadCube. Digital Science. https://www.readcube. Impactstory: Discover the Online Impact of Your Standards Organization, 2016. http://www. com/. Research. https://impactstory.org/. (O) niso.org/apps/group_public/download. Reddit. https://www.reddit.com/. International DOI Foundation. http://www.doi. php/17091/NISO%20RP-25-2016%20 Reference Manager. Clarivate Analytics. http:// org/. Outputs%20of%20the%20NISO%20 Alternative%20Assessment%20Project.pdf. wokinfo.com/products_tools/bibliographic/ Journal Citation Reports. Clarivate Analytics. refman/ (CH, Jul’09, 46-5932). http://wokinfo.com/products_tools/analytical/ ORCID: Connecting Research and Researchers. RefWorks. ProQuest. https://refworks.proquest.com/. jcr/. ($$) http://orcid.org/. (&) Research Evaluation. Oxford. http://rev. Journal of the Association for Information Science Papers: Your Personal Library of Research. http:// oxfordjournals.org/. and Technology. Wiley. http://onlinelibrary. papersapp.com/. ($$) wiley.com/journal/10.1002/(ISSN)2330-1643. Peerage of Science: Better Peer Review. https:// Researcher ID. Clarivate Analytics. http://wokinfo. com/researcherid/. (&) Konkiel, Stacey, Natalia Madjarevic, and Amy www.peerageofscience.org/. (&) Rees. Altmetrics for Librarians: 100+ Tips, PLOS. Public Library of Science. https://www. ResearchGate. https://www.researchgate.net/. (O) Tricks, and Examples. Altmetric, 2016. http:// .org/ (CH, Nov’08, 46-1247). Responsible Metrics. https://responsiblemetrics.org/. dx.doi.org/10.6084/m9.figshare.3749838. PLOS Collections: Altmetrics. Public Library of Roemer, Robin Chin, and Rachel Borchardt. Krausman, Paul R., Kristopher Bishop, and Science. http://collections.plos.org/altmetrics. “Altmetrics.” Library Technology Reports 51, no. Steven Ottogalli. “Promoting Research PLOS ONE. Public Library of Science. http:// 5 (July 2015). https://journals.ala.org/ltr/issue/ beyond Publication.” The Journal of Wildlife journals.plos.org/plosone/static/publish. view/515. Management 80, no. 3 (April 1, 2016): 385– 386. doi:10.1002/jwmg.1045 Plum Analytics. http://plumanalytics.com/. ($$) ______. “From Bibliometrics to Altmetrics: A Changing Scholarly Landscape.” College & Kudos. https://www.growkudos.com/. (&) PlumX Dashboards. http://plumanalytics.com/ Research Libraries News 73, no. 10 (November products/plumx-dashboards/. ($$) Lazy Scholar. http://www.lazyscholar.org/. (O) 1, 2012): 596–600. http://crln.acrl.org/ Priem, Jason, et al. “Altmetrics: A Manifesto” content/73/10/596.full.pdf+html. Lewis, Ruth, Cathy C. Sarli, and Amy M. Suiter. (October 26, 2010). http://altmetrics.org/ Scholarly Output Assessment Activities. SPEC ______. “Keeping Up with ... Altmetrics.” manifesto/. Kit 346. Association of Research Libraries, Keeping Up With (January 2014). http://www. 2015. http://publications.arl.org/Scholarly- Priem, Jason, with Stewart Wills. “Altmetrics, ala.org/acrl/publications/keeping_up_with/ Output-Assessment-SPEC-Kit-346/. Today and Tomorrow.” The Scholarly Kitchen altmetrics. (July 1, 2013). https://scholarlykitchen. Mendeley. Elsevier. https://www.mendeley.com/ ______. Meaningful Metrics: A 21st Century sspnet.org/2013/07/01/scholarly-kitchen- (CH, May’10, 47-4752). (&) Librarian’s Guide to Bibliometrics, Altmetrics, podcast-jason-priem-on-altmetrics-today-and- and Research Impact. Association of College Michaels, Ann. “Ask the Chefs: What Is The tomorrow/. and Research Libraries, 2015. http://www. Role of Social Media in Scholarly Publishing?” ProCite. Informer Technologies. http://procite. ala.org/acrl/sites/ala.org.acrl/files/content/ The Scholarly Kitchen (July 28, 2016). https:// software.informer.com/ (CH, Feb’08, 45-2929). publications/booksanddigitalresources/ scholarlykitchen.sspnet.org/2016/07/28/ask- digital/9780838987568_metrics_OA.pdf. the-chefs-what-is-the-role-of-social-media-in- Publons. http://home.publons.com/. (&) scholarly-publishing/. San Francisco Declaration on Research Assessment PubMed. U.S. National Institutes of Health, (DORA). American Society for Cell Biology. Monastersky, Richard. “The Number That’s National Library of Medicine. https://www. http://www.ascb.org/dora/. Devouring Science.” The Chronicle of Higher ncbi.nlm.nih.gov/pubmed/ (CH, Feb’06, 43- Education 52, no. 8 (October 14, 2005): 3422). (&) Scholarly Communication Toolkit. Association of A12–A17. http://www.chronicle.com/article/ College and Research Libraries. http://acrl. PubMed Central. U.S. National Institutes of the-number-thats-devouring/26481/. libguides.com/scholcomm/toolkit/home. Health, National Library of Medicine. https:// Mounce, Ross. “Open Access and Altmetrics: www.ncbi.nlm.nih.gov/pmc/ (CH, Sep’11, The Scholarly Kitchen: What’s Hot & What’s

June 2017 CHOICE 9 BIBLIOGRAPHIC ESSAY

Cooking in Scholarly Publishing. Society for by the Sector for the Sector. https://www. S00733ED1V01Y201609ICR052. Scholarly Publishing. https://scholarlykitchen. snowballmetrics.com/. (O) University of Pittsburg. Altmetrics: What sspnet.org/. SocArXiv Preprints: Open Library of the Social are Altmetrics? http://pitt.libguides.com/ Scholarly Metrics under the Microscope: From Sciences. https://osf.io/preprints/socarxiv. (O) Altmetrics. Citation Analysis to Academic Auditing, ed. SSRN [Social Science Research Network]. Utrecht University Library. Research Impact & by Blaise Cronin and Cassidy R. Sugimoto. Elsevier. https://www.ssrn.com/en/ (CH, Visibility: Traditional and Altmetrics. http:// ASIS&T Series. Information May’15, 52-4526). (&) Today, 2014 (CH, Dec’14, 52-1729). libguides.library.uu.nl/researchimpact/. Stuart, David. Web Metrics for Library and Van Noorden, Richard. “Metrics: A Profusion of Scholarmeter. School of Informatics and Information Professionals. Facet, 2014. Computing, Indiana University. http:// Measures,” Nature 465, no. 7300 (June 16, scholarometer.indiana.edu/. (O) Tananbaum, Greg. Article-Level Metrics: A SPARC 2010): 864–866. doi:10.1038/465864a. Primer. SPARC, 2013. https://sparcopen.org/ ScienceDirect. Elsevier. http://www.sciencedirect. wp-content/uploads/2016/01/SPARC-ALM- Web of Science. Clarivate Analytics. http:// com/ (CH, Nov’12, 50-1430). Primer.pdf. wokinfo.com/products_tools/products/. ($$) “Science Metrics” [Specials]. Nature (June 16, Tattersall, Andy. “Introduction” in Altmetrics: A Wilsdon, James, et al. The Metric Tide: Report of 2010). http://www.nature.com/news/specials/ Practical Guide for Librarians, Researchers and the Independent Review of the Role of Metrics metrics/index.html. Academics, ed. by Andy Tattersall, 1–10. Facet in Research Assessment and Management. Higher Education Funding Council for Scientometrics: An International Journal for All Publishing, 2016. England, 2015. https://doi.org/10.13140/ Quantitative Aspects of the Science of Science, Thelwall, Michael. Introduction to Communication in Science and Science Webometrics: Quantitative Web Research RG.2.1.4929.1363. Policy. Springer. http://link.springer.com/ for the Social Sciences. Synthesis Lectures WorldCat. https://www.worldcat.org/ (CH, journal/11192. on Information Concepts, Retrieval, Sep’11, 49-0004). and Services, vol. 1, no. 1. Morgan Scopus. Elsevier. http://www.elsevier.com/online- and Claypool, 2009. doi:10.2200/ Wouters, Paul, and Rodrigo Costas. Users, tools/scopus/ (CH, Jan’15, 52-2504). ($$) S00176ED1V01Y200903ICR004. Narcissism and Control: Tracking the Impact of Snowball Metrics Recipe Book, 2nd ed., ed. by Lisa Scholarly Publications. SURFfoundation, 2012. ______. Web Indicators for Research Colledge. Snowball Metrics, 2014. https:// https://www.surf.nl/en/knowledge-base/2011/ Evaluation: A Practical Guide. Synthesis www.snowballmetrics.com/wp-content/ report-users-narcissism-and-control.html. Lectures on Information Concepts, uploads/snowball-recipe-book_HR.pdf. Retrieval, and Services, vol. 8, no. 4. Zotero. https://www.zotero.org/ (CH, Jun’08, Snowball Metrics: Standardized Research Metrics— Morgan and Claypool, 2016. doi:10.2200/ 45-5309).

10 CHOICE June 2017