2017 Metrics Tools - Current Strengths and Issues ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Author evaluation ​ ​ Strengths Known Issues ​ ​ h-index ● Combines measure of quantity ● Different for different fields of research – ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ (publications) and impact (citations) may be more useful for ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Provides high objectivity Science/Engineering, less suitable for ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Easily obtained Education and the Social Sciences ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Measures impact of researcher rather ● Depends on duration of researcher’s ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ than of any particular publication career ​ ​ ​ ​ ​ ​ ​ ​ ● Does not highlight highly cited papers ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Difficult to obtain for complete output ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ including all material types ​ ​ ​ ​ ​ ​ ● Different sources may give different ​ ​ ​ ​ ​ ​ ​ ​ values e.g. Google Scholar vs Web of ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Science ● Authors need to check that publications ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ counted belong to them ​ ​ ​ ​ ​ ​ i10-index ● Simple to calculate ● Used only in Google Scholar ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Authors need to check that publications ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ counted belong to them ​ ​ ​ ​ ​ ​ Publication count ​ ​ ● Easily obtained - most databases and ● Does not distinguish between different ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Google Scholar give publication counts document types and duration of ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ publication process ​ ​ ● Dependent on material types indexed by ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ particular databases ​ ​ ● Authors need to check that publications ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ counted belong to them ​ ​ ​ ​ ​ ​ Citation count ​ ​ ● Easily obtained – most databases and ● Will vary between databases ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Google Scholar give citation counts ● Depends on duration of researcher’s ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ career ● Depends on field of research ​ ​ ​ ​ ​ ​ ​ ​ ● Doesn’t benefit emerging and specialized ​ ​ ​ ​ ​ ​ ​ ​ fields of research ​ ​ ​ ​ ● Numbers could be skewed by ​ ​ ​ ​ ​ ​ ​ ​ self-citations ● Authors need to check that publications ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ counted belong to them ​ ​ ​ ​ ​ ​ Journal evaluation ​ ​ Strengths Known Issues ​ ​ Journal Impact factor (JIF)/CiteScore ​ ​ ​ ​ ​ ​ ● Standard, widely accepted measure ● Journal Impact Factors/CiteScore values ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Easy to understand ranking system should not be used to compare journals ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Journal Impact Factors/CiteScore are across disciplines ​ ​ ​ ​ ​ ​ ​ ​ becoming more important to the PBRF ● Discipline specific ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ process ● Open to manipulation ​ ​ ​ ​ Percentile/Quartile Rankings ​ ​ ● Simple to understand ● Used with Journal Impact Factor-ranked ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Easy to identify the top ranked journals in journals included in Journal Citation ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ a discipline Reports database ​ ​ ​ ​ ● Used mostly for Science ​ ​ ​ ​ ​ ​ SJR (SCIMago Journal & Country Rank) ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Provides an alternative to Journal Impact ● Citations from more established journals ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Factor/CiteScore. Similar to Eigenfactor are more valuable than those from ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ but based on the Scopus database lesser-known journals ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Giving different weight to citations from ● Conservative ranking system; favours ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ different journals reflects hierarchy of status quo ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ journals ● Discipline specific ​ ​ ● Only peer reviewed articles are counted ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ SNIP - Source Normalized Impact per Paper ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Elsevier’s impact factor is based on ● Dependent on the content of Scopus ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Scopus ● Doesn’t differentiate between the prestige ​ ​ ​ ​ ​ ​ ​ ​ ● Corrects for differences between of citing journals ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ disciplines ● Only peer reviewed articles are counted ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Quality Ranking ​ ​ ● Easy to understand and use ● Subject specific and not easy to compare ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Often adopted by organizations as an journals from different disciplines ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ official research evaluation tool ● Lists may reflect organizational, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ geographic, cultural or compiler bias ​ ​ ​ ​ ​ ​ ​ ​ Eigenfactor ● More robust than Journal Impact ● Not widely used ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Factor/CiteScore metrics ● Global rankings tending to move away ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Based on Web of Science database which from Web of Science ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ indexes the most academic journals ● Not easy to compare journals from ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Easy to find and use different disciplines ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Citations may include more than articles, ● Large journals will have high Eigenfactor ​ ​ ​ ​​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ e.g. editorials scores based on size alone ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Altmetrics General Strengths General Known Issues ​ ​ ​ ​ ​ ​ ● Provide a broader research impact ● Lack of consistent adoption across ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ measurement in both the social and disciplines and institutions ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ scholarly realm ● Lack of agreement about what metrics, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Can be used to augment traditional data sources or tools are most valuable ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ metrics ● Can be easily distorted or misinterpreted ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Gauge impact of research before it enters ● Measurement reflects all attention paid to ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ the citation cycle an output whether positive or negative - ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Accumulate faster compared to traditional may not reflect scholarly quality or impact ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ citation counts ​ ​ ● Measure the impact of different types of ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ scholarly or creative outputs (e.g. ​ ​ ​ ​ ​ ​ ​ ​ datasets or visual arts, software, ​ ​ ​ ​ ​ ​ ​ ​ presentations) Strengths Known Issues ​ ​ UoW Research Commons ​ ​ ​ ​ ● Items are publically available and ● Doesn’t contain all institutional output ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ accessible by internet search engines ● May not be as widely searched/used by ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Number of file downloads for items, researchers as other databases and ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ authors and collections, and top views by services ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ city and country are readily accessible ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ImpactStory ● Tracks wide range of research output ● Interpreting reports can be challenging ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ types owing to the high number of metrics ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Aggregates impact data from many displayed, and lack of context and ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ different sources comparison ​ ​ ● Calculates the impact of research outputs ● Experience with social media is helpful for ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ using DOI, PMID or arXivID as tracking interpreting the report ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ IDs ● Relies on resource identification ​ ​ ​ ​ ​ ​ standards for compiling the reports - a ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ lack of standards is a core problem ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Figshare ● Collaborates with Symplectic and all of ● Uploading a dataset on figshare can result ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ the altmetric solutions, also on Facebook, in numerous DOIs, since both the dataset ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Google+, Twitter, and Vimeo and the individual files are assigned DOIs ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Partners with publishers such as Nature, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ PLOS and F1000 (life sciences, clinical ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ research) ● Uploaded research is citable and ​ ​ ​ ​ ​ ​ ​ ​ trackable via a DOI ​ ​ ​ ​ ​ ​ ● Detailed reporting metrics are available ​ ​ ​ ​ ​ ​ ​ ​ for the institution ​ ​ ​ ​ ResearchGate ● Can be used in conjunction with other ● Care needs to be taken with copyright, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ sites such as Academia.edu, Mendeley, particularly with relation to published ​ ​​ ​ ​ ​ ​​ ​ ​ ​​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Google Scholar or figshare. versions of articles. Check first on ​ ​ ​ ​ ​ ​​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Uploaded full text publications are SHERPARoMEO ​ ​ ​ ​ ​ ​ ​ ​ indexed by Google ● Many of the publications available through ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ● Copies of papers (either pre- or ResearchGate are actually uploaded ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ post-review) and associated raw data can illegally in terms of publisher open access ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ be uploaded; all are searchable policy ​ ​ ​ ​ ​ ​ ​ ​ ● Collates useful information about journals, ● A high percentage of ResearchGate ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ such as impact factors, metrics and some members are postgraduate and other ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ details of open access policy students ​ ​ ​ ​ ​ ​ ​ ​ ● Provides an RG Score based on how ● There have been complaints about ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ other researchers interact with your unwanted email spamming ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ content, how often, and who they are ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Academia.edu ● Social networking platform which provides ● Enhanced service to researcher’s ​ ​ ​ ​ ​ ​