A comprehensive analysis of the journal evaluation system in China Ying HUANG1,2, Ruinan LI1, Lin ZHANG1,2,*, Gunnar SIVERTSEN3 1School of Information Management, Wuhan University, China 2Centre for R&D Monitoring (ECOOM) and Department of MSI, KU Leuven, Belgium 3Nordic Institute for Studies in Innovation, Research and Education, Tøyen, Oslo, Norway Abstract Journal evaluation systems are important in science because they focus on the quality of how new results are critically reviewed and published. They are also important because the prestige and scientific impact of journals is given attention in research assessment and in career and funding systems. Journal evaluation has become increasingly important in China with the expansion of the country’s research and innovation system and its rise as a major contributor to global science. In this paper, we first describe the history and background for journal evaluation in China. Then, we systematically introduce and compare the most influential journal lists and indexing services in China. These are: the Chinese Science Citation Database (CSCD); the journal partition table (JPT); the AMI Comprehensive Evaluation Report (AMI); the Chinese STM Citation Report (CJCR); the “A Guide to the Core Journals of China” (GCJC); the Chinese Social Sciences Citation Index (CSSCI); and the World Academic Journal Clout Index (WAJCI). Some other lists published by government agencies, professional associations, and universities are briefly introduced as well. Thereby, the tradition and landscape of the journal evaluation system in China is comprehensively covered. Methods and practices are closely described and sometimes compared to how other countries assess and rank journals. Keywords: scientific journal; journal evaluation; peer review; bibliometrics 1 1. Introduction China is among the many countries where the career prospects of researchers, in part, depend on the journals in which they publish. Knowledge of which journals are considered prestigious and which are of dubious quality is critical to the scientific community for assessing the standing of a research institution, for tenure decisions, grant funding, performance evaluations, etc. The process of journal evaluation dates back to the 1930s when the British mathematician, librarian and documentalist Samuel C. Bradford published his study on publications in geophysics and lubrication. The paper presented an empirical law now known as Bradford’s law of scattering along with the concept of core area journals (Bradford, 1934). Bradford influenced Eugene Garfield the USA, who subsequently published a groundbreaking paper on citation indexing, “Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas.” According to Garfield (1955), “the citation index ... may help a historian to measure the influence of an article — that is, its ‘impact factor’”. In the 1960s, Garfield conducted a large-scale statistical analysis of citations in the literature, reaching the conclusion that many citations were concentrated in just a few journals and the many remaining journals only accounted for a few citations (Garfield, 1963; 1964). Garfield went on to create the Institute for Scientific Information (ISI), then successively published the Science Citation Index (SCI), Social Science Citation Index (SSCI), and the Art and Humanities Citation Index databases. Assessing the quality of previously published research output is important in all contexts where research assessment takes place, for example, when evaluating the success of research projects and when distributing research funding (Pan Su et al., 2017). As part of the assessment, evaluating and ranking the quality of the journals where the output was published has become increasingly important (Mingers and Yang, 2017). Journal evaluation and rankings are used by governments, organizations, universities, schools, and departments to evaluate the quality and quantity of faculty research productivity, ranging from promotion and tenure to monetary rewards (Black et al., 2017). Even though the merit of using such systems is not universally agreed upon (Dobson, 2014) and even contested (Lin Zhang et al., 2017),it is, however, widely believed that the rank or citation impact of a journal is supposed to represent its prestige, influence, and probably the difficulty of having a paper accepted for publication in it (Pan Su et al., 2017). Over the past few years, the number of papers published in international journals by Chinese researchers has seen a dramatic increase to the point that, today, China is 2 one of the world’s leading science and technology producers. In tandem, government policies and guidance, especially the call to “publish your best work in your motherland to benefit the local society”, proposed by President Xi in 20161, are leading more papers to be published in China’s domestic journals. With these increases in the number of papers and journals, it is an important task to explore the strengths and weaknesses of various methods for evaluating journals and to discuss what journal evaluation systems that might be suitable for China’s ambitions to contribute both internationally and locally. The journal evaluation system in China was established gradually, beginning with the introduction of foreign journal evaluation theories around 60 years ago. Over the last 30 years, these foreign theories have been adopted, adapted, researched, and vigorously redeveloped (Lan Ma, 2016). In the past, journal evaluation and selection results were mainly used to help librarians develop their collections and to help readers better identify core journals. However, in recent years, the journal evaluation and ranking have increasingly been applied research evaluation and management, i.e., in tenure decisions, grant funding, and performance evaluations, etc. (Shu et al., 2020). Many institutions are increasingly relying on journal impact factors to evaluate papers and researchers. This is commonly referred to in China as “evaluating a paper based on the journal and ranking list”. The higher the journal’s rank and impact factor, the higher is the expected outcome of evaluations. In the ever-changing environment of scientific research evaluation, the research and practice of journal evaluation in China is also evolving to meet different needs. Many influential journal evaluations and indexing systems have been established since the 1990s, with their evaluation methods and standards becoming increasingly mature. These activities have played a positive role in promoting the development of scientific research and have also been helpful for improving the quality of academic journals. The aim of this study is to review the progress of journal evaluation in China and present a comprehensive analysis of the current state-of-art. Hence, the main body of this article is a comparative analysis of the most influential journal lists in China. The results not only offer a deeper understanding of journal evaluation methods and practices in China but also reveal some insights into the journal evaluation activities of other countries. Overall, the aim is to make a valuable contribution to improving the theory and practice of journal evaluation and promoting the sustainable and healthy development of journal management and evaluation systems in China. 1 http://www.xinhuanet.com//politics/2016-05/31/c_1118965169.htm 3 2. Journal evaluation in China 2.1 A brief history Journal evaluation in China dates back to the 1960s with some fairly distinct stages over its development. Zhang Qiyu and Wang Enguang first introduced the Science Citation Index to Chinese readers in 1964 (Yaoming Zhang, 2015). In 1973, Wu Erzhong introduced a core journal list for chemistry. This was the first mention of the concept of a “core journal” (Erzhong Wu, 1973); In 1982, Liansheng Meng (1982) finished his Master’s thesis entitled “Chinese science citation analysis”, and then, in 1995, he published the Chinese Science Citation Index (CSCI) with the support of National Natural Science Foundation of China (NSFC). The main feature of this stage of development was translating international practice in journal evaluation and apply it to the Chinese context wholesale. At the same time, exploring the potential application of laws related to journal evaluation became an important topic for researchers in library and information science. In 1988, Jing Qinshu and Xian Jiaxiu used the “citation method” to identify a list of “Chinese Natural Science Core Journals”, which included 104 core Chinese journals in the natural sciences and was recognized as the first Chinese journal list (Jing and Xian, 1988). Around that same time, a few institutions began to undertake journal evaluation activities. As some examples, the Institute of Scientific and Technical Information of China (ISTIC) (commissioned by the Ministry of Science and Technology - formerly the National Scientific and Technological Commission) firstly analyzed the international publications indexed by Science Citation Index (SCI)，Index to Scientific Reviews (ISR) and Index to Scientific＆Technical Proceeding (ISTP) since 1987 and started to select domestic scientific journals to analyze the publications and citations of scientific journals in China since 1989. During this process, 1189 journals were selected from 3025 scientific journals nationwide and were selected as the journals of statistical sources that were adjusted annually thereafter (Qian, 2006). Hence, this second stage