How Do Statisticians Perceive Statistics Journals?
Total Page:16
File Type:pdf, Size:1020Kb
General How DoStatisticians Perceive Statistics Journals? Vasilis THEOHARAKIS and Mary SKORDIA ceptionsof statisticians with different research interests may vary.In fact, in the UK wherefunds to universitiesare disbursed Sinceresearchers and academic institutions are increasingly basedon theResearch Assessment Exercise (RAE), impactfac- evaluatedbased on their publication record in peer reviewed torsor citationindices are not used to assessresearch output in journals,it is crucial to assess how the statistics community journals.Instead, the assessment of the RAE panelfor statis- perceivesstatistics journals. This study presents four subjective ticsis basedon the “ perceivededitorial standards of journals” qualitymetrics of statistics journals as expressed by different (http://www.hero.ac.uk/rae/criteria/crit24.htm). segmentsof statisticians. Based on aworldwidesample of 2,190 Despitethe fact that the assessment of journalsis acrucialis- statisticians,our ndingsindicate that the research interest and suefor theresearch community, it is asurprisethat the statistics geographicorigin of theresearcher have a signicant impact on community’s perceptionshave not been systematically exam- journalperceptions, which are highly correlated with a journal’s ined.W ethereforepose the following questions: What are the totalnumber of citations. mostpopular journals in the eldof statistics?Since promotion decisionsfrequently depend on the number of publications in KEY WORDS: Journalrankings; Statistics research. toptier journals, how do statisticians classify journals in tiers? Besidesone’ s perceptionabout a journal’s standing,how useful doresearchers nda particularjournal? Do statisticians from differentresearch or geographicareas or witha differenttype 1.INTRODUCTION ofemploymentvalue journals differently? How dothesubjec- Therecognition and development of anacademicinstitution tiveperceptions of journal quality relate to the more objective dependsheavily on its faculty’ s publicationrecord in presti- journalcitation measures? By addressing these questions, this giousjournals (Lane, Ray, and Glennon 1990). As aresult,an studyseeks to assist: (1) authorsin their search for aresearch increasedemphasis is placed on publishing in refereed jour- outlet,(2) departmentsin promotionand tenure decisions, and nalsand promotion criteria rest heavily on the faculty’ s pub- (3) journaleditors, by providingthem a viewof their journal’ s licationrecord (Gibbons 1990). In fact, not only is the pub- standing.W eshouldnote that while we examinethe percep- licationrecord one of the criteria for selectingFellows at the tualjournal rankings, there is a substantialoverlap in the quality AmericanStatistical Association (Bailar 1988), but it is also ofindividualarticles that appear in journalsof vastlydifferent usedto measure the productivity of countries and institutions reputation. for theircontributions to statistics (Genest 1997). Genest mea- suredinstitution and country research productivity based on the 2.SUR VEY INSTRUMENT AND METHODOLOGY numberof articles,number of authors, and page counts in 16 Sincewe soughtto examine journal perceptions over a broad internationaljournals publishing in statistical theory. Since he sample,we locatedfour publicly available membership direc- believedthe selection of these journals to be “ subjectiveand toriesof statisticians (American Statistical Association, Insti- far from comprehensive,”astudythat systematically identi es tuteof Mathematical Statistics, International Statistical Institute, therelevant journals would facilitate such studies. The need for andan online listing of UK-basedacademic statisticians, found identifyingrelevant journals was alsodemonstrated by Baltagi athttp:/ /www.swan.ac.uk/statistics/das/).Dueto thepervasive (1999)in his article on the ranking of individuals and institu- useof theInternet among statisticians, we developedan online tionsin applied econometrics. T odemonstrateimpact, Baltagi survey.Our questionnairerequested from participantsto place usedpage counts and citations of relevant articles from 15jour- statisticsjournals in rank order and at the same time provide nals,but could not control for journalquality since no jour- demographicinformation. The demographic variables were se- nalquality measure was available.Although citation reports do lectedin orderto beused as segmentation variables that could providean aggregate measure of a journal’s impact,the per- provideanswers tothequestions raised earlier .Therefore,par- ticipantswere askedto rank up totenstatistics journals that they Vasilis Theoharakisis Assistant Professor(E-mail: [email protected]), consideredas top tier (most rigorous, prestigious, and impor- andMary Skordia is Research Associate, AthensLaboratory of Business Admin- tant)and up to ten additional journals that they considered as istration(ALBA), Athinas & Areos 2A,V ouliagmeni166 71, Athens, Greece. secondtier. In addition, respondents were askedto list up toten Thisarticle wouldnot have been in place withoutthe support of colleagues who journalsthat they considered to bemostuseful in their work. A spentmany hours in providing direct input and feedback. The authors thank the editorand associate editorfor their very constructive comments thatsigni cantly listof 110 statistics journals was availableon pull-downmenus improvedthis article. (Appendix),but respondents could also llin anyother journal c 2003American Statistical Association DOI: 10.1198/0003130031414 TheAmerican Statistician, May 2003, V ol.57, No. 2 115 ® Table1. Respondents’ Pro le follows: 20 Highestacademic degree Rij j ARP = j= 1 ¤ (1 ARP 20); (1) Doctorate 1734 i 20 µ i µ Masters 355 P j= 1 Rij Bachelor’s 40 Other 24 where i denotesthe journal P and Rij isthe number of timesjour- No answer 37 nal i hasbeen ranked in the jthposition. Thus, a lowerARP TOTAL 2190 denotesa higherperceived journal importance. In addition, we reportthe percentage of respondentswho included the journal Typeof employment Facultymember 1234 intheir top ten with respect to thetotal number of respondents Governmentemployee 185 (%Top10)andjournal Usefulness thatcorresponds to the per- Researcher/clinicianat ahealth/medicalfacility 152 centageof respondents who listed the journal among the ones Manufacturingindustry employee 121 mostuseful in their work. But one hasto becarefulwhen ranking Privateconsultant 101 Serviceindustry employee 43 journalson anysingle measure of perceived quality. For exam- Retired 32 ple,if journal A isranked by 100 respondents who all place it in Actuary 3 the1st rst positionand journal B isranked by 101 respondents Other 179 whoall place it in the 20th position, then journal A wouldbe No answer 140 TOTAL 2190 rankedlower if journals were rankedbased on Familiarity. In orderto minimizesuch problems, we consideredmultiple qual- Geographicallocation itymeasures when performing the ranking of journals,by using NorthAmerica 1495 Index Europe 412 a weighted offamiliarityand rank (Theoharakis and Hirst Asia 149 2002)that is de ned as follows: LatinAmerica 57 20 Australia/New Zealand 37 Rij (21 j) Index = 100 j= 1 ¤ ¡ Africa 23 i ¤ 20 n No answer 17 P 21 ARP¤ TOTAL 2190 = 100 ¡ i %Top20 ¤ 20 ¤ i (0 Indexi 100); (2) titlethey wished. From thedirectories identi ed, we collected µ µ theE-mail addresses of 12,053 statisticians and proceeded by where i denotesthe journal and Rij isthe number of times sendingan E-mail invitation to themfor completingour online the journal i hasbeen ranked in the jthposition and n is the questionnaire(the questionnaire and full set of tablesare avail- numberof respondentsin the sample. Thus, the Index assigns to the jthposition a decreasingweight of (21 j)=20, with the ableat www.alba.edu.gr/ survey).The survey was pretestedon ¡ asampleof 30 statisticiansand minor alternations were made. rst rankposition carrying a weightof 20/20andthe last (20th) Twoweeks after the initialE-mail invitation, an E-mail reminder positiona weightof 1/ 20.W ealsoextend the original Index was sentto individuals who had not responded. byTheoharakisand Hirst (2002)to indicateits connectionwith Intotal, we received2,190 usable responses (521 from thesec- ARP and%T op20.As we listthe journalsbased on this Index, we ondwave) with a usableresponse rate of 18.2%.No signi cant presenteach individual measure and suggest that readers should differencesin the ranking of journalswere foundbetween rst examineeach journal individually across the metrics presented. andsecond wave respondents, that is, those who responded to thereminder E-mail, which may indicate that our sample does 3. RESULTS notsuffer from nonresponsebias. However, statisticians that donot believe in ranking journals, may have not participated. Wepresentjournals based on the weighted Index of famil- Nearlytwo-thirds of ourrespondents are from NorthAmerica, iarityand rank for ourworldwide sample and the two largest morethan half are faculty members, and nearly 80% of ourre- regionalsamples (T able2). Although the correlations between spondentshold a doctorate(T able1). Sixty seven percent of our ourperceptual metrics (%T op10,%T op20,ARP, andUseful- respondentsreplied that their institution uses the number and/ or ness)are high (T able3), the correlations of eachone of these characterof journalpublications for personneldecisions. metricswith our Index are even higher; the only exception is thecorrelation of ARP with%T op10that is about the same with thecorrelation of ARP withIndex (this is nota surprisesince %Top10depends on rankposition).