Altmetrics Make Their Mark Alternative Measures Can Yield Useful Data on Achievement — but Must Be Used Cautiously
Total Page：16
File Type：pdf, Size：1020Kb
CAREERS TURNING POINT US cancer researcher feels NATUREJOBS FACEBOOK Science-careers NATUREJOBS For the latest career squeezed by budget cuts p.493 advice and information go.nature.com/4lzxra listings and advice www.naturejobs.com IMAGEZOO/CORBIS RESEARCH IMPACT Altmetrics make their mark Alternative measures can yield useful data on achievement — but must be used cautiously. BY ROBERTA KWOK paper on his CV accompanied by the journal’s number of times a paper has been shared on impact factor and the article’s number of cita- social-media websites or saved using online teve Pettifer and his colleagues did not tions — in this case, about 80. But when he research tools. heavily promote their 2008 paper on digi- came up for promotion this year, he realized When Pettifer submitted his annotated tal library tools. So it came as a surprise that tracking citations was not going to tell CV for the first round of promotion review, Swhen, in August 2012, Pettifer got an e-mail the whole story about the paper’s influence. his mentor expressed confusion. He took a from the Public Library of Science (PLOS), Impact factor is a crude measure that applies look and said, “What the hell are these badges based in San Francisco, California. A PLOS only to the journal, not to specific articles, he doing in your CV?” recalls Pettifer. “But once representative told him that people had viewed says; citations take a long time to accumulate, I explained them, he said, ‘Well, give it a go.’” or downloaded the article (D. Hull et al. PLoS and people may not cite a paper even if it influ- Pettifer submitted his CV for the second Comput. Biol. 4, e1000204; 2008) more than ences their thinking. So he added the number round — and got his promotion. He does not 53,000 times. It was the most-accessed review of views to the CV entry. And he did not stop know for sure whether the metrics helped, but ever to be published in any of the seven PLOS there. he plans to use them on future grant applica- journals. The paper had come out just as biolo- Next to many of the papers listed, Pettifer tions. “I’m definitely a convert,” he says. gists’ interest in digital publishing was building added labels indicating scholarly and public and the number of tools was exploding, says engagement. The labels were generated by OUTSIDE THE BOX Pettifer, a computer scientist at the University ImpactStory in Carrboro, North Carolina, one ‘Altmetrics’, a term coined in 2010 by Impact- of Manchester, UK. “It hit the right note at the of several services that gauges research impact Story co-founder Jason Priem, refers to a range right time,” he says. using a combination of metrics — in this case, of measures of research impact that go beyond At one time, Pettifer would have listed the a wide range of data sources, including the citations. Several altmetrics services have 22 AUGUST 2013 | VOL 500 | NATURE | 491 © 2013 Macmillan Publishers Limited. All rights reserved CAREERS emerged in the past few years (see ‘Four a researcher’s work. ImpactStory then creates link to a particular paper; each article will have ways to score’). They produce reports that a profile showing how frequently each product multiple URLs, so conducting such a search gauge impact by taking into account not just has been viewed, saved, discussed, cited or rec- manually would be very time-consuming. academic citations, but also digital use and ommended online. The reports can even suggest potential sharing of data — which can include the num- Other services take a more article-centric collaborators or journals. For example, if an ber of times a paper has been tweeted, ‘liked’ approach. Altmetric in London allows users to informatics paper is mentioned a lot by biolo- on Facebook, covered by the media or blogs, access data on individual papers using a book- gists, the author might consider publishing downloaded, cited on Wikipedia or book- marklet — a browser bookmark that executes his or her next article in a biology journal to marked online. Some services also evaluate JavaScript commands. (Altmetric is funded increase exposure, says Heather Piwowar, research products such as software, data sets partly by Digital Science, a sister company to co-founder of ImpactStory. and slideshows by tracking the number of Nature Publishing Group.) Users install the people who have used or viewed the product bookmarklet in their Internet browsers; then, MEASURES OF CAUTION online (see Nature 500, 243–245; 2013). when they come across a paper that they are Despite the benefits, researchers and evalua- Altmetrics offer researchers a way to show- interested in, they click the bookmarklet button. tors must interpret altmetrics data cautiously. case the impact of A report pops up in the corner of the browser, Data sets might not be comprehensive: not all papers that have not providing altmetrics that include a score indi- services detect news stories that do not give yet gathered many cating how much online attention the paper has URLs for the study, for example. The popular- citations, and to received. The score takes into account the num- ity of social-media sites changes over time, so demonstrate engage- ber of people who have read or mentioned the it is unrealistic to expect a paper published in LATHAM JOHN T. ment with the public. article, as well as the relative importance of the 2008 to generate as many tweets as one pub- They can be accessed medium and the mentioner. Newspaper cover- lished in 2013. And some disciplines, such through journals or age is weighted more heavily than tweets, and as computational biology, are more active independent web- tweets by individuals more heavily than those than others on social media, so comparisons sites, and can track by journals promoting their content. between disciplines may be unfair. the impact of par- Many journals display some altmetrics on To get the most meaningful information, ticular data sets or their sites automatically; these might be gen- users should dig into the underlying data. papers, or evaluate “It hit the right erated in-house or provided by an external Although a paper’s Altmetric score can sug- the combined influ- note at the right service. Every article published by PLOS, for gest whether it is worth clicking through to the ence of publications time.” example, includes an online metrics tab show- more detailed report, “qualitative assessment and products pro- Steve Pettifer ing data such as views, downloads and social- is far more important than the number”, says duced by multiple media mentions. A feature called Article-Level Euan Adie, founder of Altmetric. researchers in a department. Metrics Reports lets users search for PLOS To help users to interpret the data, most But these services must be used wisely. They papers by criteria such as author or keyword, services put numbers in context. Impact- are not meant for strict quantitative compari- and generates a summary metrics report for the Story normalizes data by publication year and sons; nor do they always distinguish between set of results, including article usage by paper includes percentiles — it might, for example, positive and negative attention. And although age and maps of authors’ locations. Several note that a given paper has more readers on scientists can include altmetrics in job and grant journal publishers, including Nature Publishing the online reference manager Mendeley than applications and annual reports, they must Group in London and Cell Press in Cambridge, 97% of papers indexed that year. Altmetric select relevant data and clearly explain the con- Massachusetts, display data from Altmetric on shows results normalized by journal, which text to avoid provoking mistrust or confusion. their sites, and John Wiley & Sons in Hoboken, allows fairer comparison of papers in disci- Some altmetrics services generate profiles New Jersey, began a trial with the metrics firm in pline-specific publications. And in May, PLOS that summarize the impact of a researcher’s May. HighWire Press, an electronic-publishing began offering Relative Metrics, a service that products. ImpactStory allows scientists to platform at Stanford University in Palo Alto, lets users see how a paper compares to other import lists of items such as papers and soft- California, is collaborating with ImpactStory PLOS articles in the same subject area, using ware from existing user profiles at websites to add altmetrics to its journal websites. tools such as graphs of article views. such as Google Scholar, which automatically Altmetrics enable scientists to see ripples Including altmetrics in decisions on grants, tracks a researcher’s papers, or the online soft- generated by their research that might other hiring and tenure requires careful considera- ware-code repository GitHub. Scientists can wise go unnoticed. Individual researchers tion. Gerald Rubin, executive director of the also manually enter the digital object identifi- can try to track buzz on their own, but data- Howard Hughes Medical Institute’s Janelia ers (DOIs) of their papers, or input their Open aggregation and updating services make it Farm Research Campus in Ashburn, Virginia, Researcher and Contributor ID (ORCID), a much easier. These services also automate is sceptical of altmetrics that do not explicitly unique identifier that can be used to tag all of difficult tasks, such as finding all tweets that indicate quality, such as number of tweets. He FOUR WAYS TO SCORE A quartet of services offers free metrics reports that go beyond citations. ImpactStory Altmetric PLOS Article-Level Metrics Plum Analytics Products Papers, software, data sets and more Papers, data sets, some books Papers published by the Public Papers, books, patents and tracked Library of Science (PLOS) more What you get Profile page, metrics badges, application Bookmarklet, metrics badges, API Summary reports, WordPress Profile page (currently in programming interface (API; a means widget, API testing), API for software to access the altmetrics) Publishers Various, including eLife, Pensoft Include Nature Publishing Group, Cell PLOS Medwave (forthcoming Publishers, PeerJ Press, BioMed Central this year) Major Alfred P.