Bibliometrics: Tracking Research Impact by Selecting the Appropriate Metrics
Total Page：16
File Type：pdf, Size：1020Kb
Asian Journal of Andrology (2016) 18, 296–309 © 2016 AJA, SIMM & SJTU. All rights reserved 1008-682X www.asiaandro.com; www.ajandrology.com Open Access INVITED REVIEW Bibliometrics: tracking research impact by selecting Others the appropriate metrics Ashok Agarwal1, Damayanthi Durairajanayagam1,2, Sindhuja Tatagari1,3, Sandro C Esteves4, Avi Harlev5, Ralf Henkel6, Shubhadeep Roychoudhury1,7, Sheryl Homa8, Nicolás Garrido Puchalt9, Ranjith Ramasamy10, Ahmad Majzoub1, Kim Dao Ly11, Eva Tvrda12, Mourad Assidi13, Kavindra Kesari14, Reecha Sharma15, Saleem Bani Hani16, Edmund Ko17, Muhammad Abu‑Elmagd13, Jaime Gosalvez18, Asher Bashiri19 Traditionally, the success of a researcher is assessed by the number of publications he or she publishes in peer‑reviewed, indexed, high impact journals. This essential yardstick, often referred to as the impact of a specific researcher, is assessed through the use of various metrics. While researchers may be acquainted with such matrices, many do not know how to use them to enhance their careers. In addition to these metrics, a number of other factors should be taken into consideration to objectively evaluate a scientist’s profile as a researcher and academician. Moreover, each metric has its own limitations that need to be considered when selecting an appropriate metric for evaluation. This paper provides a broad overview of the wide array of metrics currently in use in academia and research. Popular metrics are discussed and defined, including traditional metrics and article‑level metrics, some of which are applied to researchers for a greater understanding of a particular concept, including varicocele that is the thematic area of this Special Issue of Asian Journal of Andrology. We recommend the combined use of quantitative and qualitative evaluation using judiciously selected metrics for a more objective assessment of scholarly output and research impact. Asian Journal of Andrology (2016) 18, 296–309; doi: 10.4103/1008-682X.171582; published online: ??? Keywords: article‑level metrics; bibliometrics; citation counts; h‑index; impact factor; research databases; research impact; research productivity; traditional metrics INTRODUCTION productivity of the researcher and their institution, it does not provide A peer-reviewed research paper serves as a platform to disseminate the an indication of the quality and significance of a research publication, results of a scientific investigation, creating an opportunity to expose nor does it indicate the impact the research or the researcher has. the work publicly and for other scholars to assimilate the published Moreover, the reputation and standing of a researcher heavily depend knowledge.1 Using the study outcomes, other researchers can further on the impact the individual has had within his or her own research validate, refute, or modify the hypotheses in developing their own community or field of study. Thus, along with the growing number of publications, the question arises as to how a researcher’s influence, research or clinical practice.2 In the academic and industry settings, standing and reputation, as well as research impact can be measured. researchers are highly encouraged to engage in scientific research A quantitative measure indicates the value of an individual’s and publish study results in a timely manner using the best possible research to their institution. As a measurable index of research impact, platform. This “publish or perish” attitude drives research productivity, it can then be used as a form of assessment in granting research albeit with its own positive and negative consequences. funds, awarding academic rank and tenure, in determining salaries Essentially, the number of publications and the relevant impact and projecting annual research targets, as well as staff assessment and thereof are widely regarded as measures of the qualification of a given hiring new staff, for appropriate selection of examiners for doctoral researcher, and thus of his or her reputation. The same is valid and students, or as selection of plenary or keynote speakers in scientific practiced for the reputation and standing of scientific journals. While conferences. It is also a useful method for demonstrating return on the total number of publications alone may be used to derive the investment to funding bodies, as well as to the host institution, industry, 1American Center for Reproductive Medicine, Cleveland Clinic, Cleveland, OH, USA; 2Faculty of Medicine, Universiti Teknologi MARA, Selangor, Malaysia; 3Case Western Reserve University, Cleveland, OH, USA; 4Androfert, Andrology and Human Reproduction Clinic, Campinas, Brazil; 5Department of Obstetrics and Gynecology, Soroka University Medical Center, Ben-Gurion University of the Negev, Beersheba, Israel; 6Department of Medical Bioscience, University of the Western Cape, Bellville, South Africa; 7Department of Life Science and Bioinformatics, Assam University, Silchar, Assam, India; 8Andrology Solutions, London, UK; 9IVF Laboratory Instituto Universitario, IVI Valencia, Valencia, Spain; 10Department of Urology, University of Miami Miller School of Medicine, Miami, Florida, USA; 11Texas A and M Health Science Center, College of Medicine, Bryan, Texas, USA; 12Department of Animal Physiology, Slovak University of Agriculture, Nitra, Slovakia; 13Center of Excellence in Genomic Medicine Research, King Abdulaziz University, Jeddah, KSA; 14School of Life Sciences, Jaipur National University, Jaipur, Rajasthan, India; 15Department of Health Services, Saint Joseph’s University, Philadelphia, PA, USA; 16Department of Medical Laboratory Sciences, Jordan University of Science and Technology, Irbid, Jordan; 17Department of Urology, Loma Linda University, School of Medicine, Loma Linda, California, USA; 18Department of Biology, Genetics Unit, Autonomous University of Madrid, Madrid, Spain; 19Maternal-Fetal Medicine and Ultrasound, Soroka University Medical Center, Ben-Gurion University of the Negev, Beersheba, Israel. Correspondence: Prof. A Agarwal ([email protected]) Bibliometrics A Agarwal et al 297 and the general public. A metric that offers the ability to identify key in a given field in peer-reviewed, Scopus or Institute of Scientific opinion-forming scientists in a particular area of research would also Information (ISI)-indexed, “high” impact journals. This straightforward offer valuable knowledge for the development of 1) programs within bibliometric indicator is commonly used as a metric to evaluate and/ academic and research institutions, and 2) policies in government or compare the research output and impact of individual researchers, agencies and relevant industries.3 research groups, or institutions. Conventionally, researchers with a Bibliometrics is the process of extracting measurable data through large number of publications would be considered highly productive, statistical analysis of published research studies and how the knowledge experienced, and successful. within a publication is used. The American psychologist and editor A researcher’s number of publications can easily be retrieved from of Science, James McKeen Cattell, was responsible for introducing any major research database, such as PubMed, Scopus, Web of Science, the concepts of scientometrics (the systematic measurement of Google Scholar, or ResearchGate. However, one has to keep in mind science) and scientific merit, as well as quantity (productivity) that the quantitative number generated for the same researcher may and quality (performance) to measure science. In the early 1950s, vary from one database to another. This is because certain databases bibliometrics was used by American psychologists as a method include only specific types of articles such as original papers, reviews, to systematically count their number of publications within their letters, and commentaries but exclude other contributions such as book discipline, which laid a background for future pioneering metric work.4 chapters, technical reports, and conference papers. Bibliometrics is one of the key methods to objectively measure We ran a web-based search on the lead author and utilized the the impact of scholarly publications while others include betweenness metrics obtained from popular databases to illustrate the possible centrality (i.e., how often users or citations pass through the particular diversity in search outcomes. As of November 18, 2015, Professor publication on their way to another publication from another one) Ashok Agarwal has the following number of published research and usage data (which includes Internet-based statistics such as articles in these respective databases: 319 in PubMed, 515 in Scopus™, number of views and downloads).5 The increase of data availability 313 in Web of Science®, more than 1500 in Google Scholar™, and and computational advances in the last two decades has led to an 853 in ResearchGate. Based on this quantity, we may readily assume overabundance of metrics, and indicators are being developed for that Professor Agarwal is a highly successful scientist and an expert different levels of research evaluation. The necessity to evaluate in his field. Given the diversity of these database search outcomes, individual researchers, research groups, or institutions by means of however, it is apparent that to make such a claim about a researcher, bibliometric indicators has also increased dramatically. This can be other metrics would need to be used in conjunction with the number attributed to the fact that the annual growth