Are You an Invited Speaker? A Bibliometric Analysis of Elite Groups for Scholarly Events in Bioinformatics Senator Jeong, Sungin Lee, and Hong-Gee Kim Biomedical Knowledge Engineering Laboratory, Seoul National University, 28–22 YeonGeon Dong, Jongno Gu, Seoul 110–749, Korea. E-mail: {senator, sunginlee, hgkim}@snu.ac.kr Participating in scholarly events (e.g., conferences, work- evaluation, but it would be hard to claim that they have pro- shops, etc.) as an elite-group member such as an orga- vided comprehensive lists of evaluation measurements. This nizing committee chair or member, program committee article aims not to provide such lists but to add to the current chair or member, session chair, invited speaker, or award winner is beneficial to a researcher’s career develop- practices an alternative metric that complements existing per- ment.The objective of this study is to investigate whether formance measures to give a more comprehensive picture of elite-group membership for scholarly events is represen- scholars’ performance. tative of scholars’ prominence, and which elite group is By one definition (Jeong, 2008), a scholarly event is the most prestigious. We collected data about 15 global “a sequentially and spatially organized collection of schol- (excluding regional) bioinformatics scholarly events held in 2007. We sampled (via stratified random sampling) ars’ interactions with the intention of delivering and shar- participants from elite groups in each event. Then, bib- ing knowledge, exchanging research ideas, and performing liometric indicators (total citations and h index) of seven related activities.” As such, scholarly events are communica- elite groups and a non-elite group, consisting of authors tion channels from which our new evaluation tool can draw who submitted at least one paper to an event but were its supporting evidence. They consist of conferences, con- not included in any elite group, were observed using the Scopus Citation Tracker. The Kruskal–Wallis test gresses, symposia, workshops, seminars, and the like. The was performed to examine the differences among the events set the stage for publication of cutting-edge scien- eight groups. Multiple comparison tests (Dwass, Steel, tific research results, feedback from scientific communities, Critchlow–Fligner) were conducted as follow-up proce- exchanges of research interests, and sharing of ideas, best dures. The experimental results reveal that scholars in practices, and techniques. an elite group have better performance in bibliometric indicators than do others. Among the elite groups, the Why do researchers participate in scholarly events, and invited speaker group has statistically significantly what in those events can be identified as contributing factors the best performance while the other elite-group types for scholar evaluation? Rapid knowledge dissemination and are not significantly distinguishable. From this analy- subsequent peer acknowledgment may be the most impor- sis, we confirm that elite-group membership in scholarly tant reasons for event participation. Compared with scientific events, at least in the field of bioinformatics, can be utilized as an alternative marker for a scholar’s promi- journals, the knowledge-sharing process is much faster in nence, with invited speaker being the most important scholarly events. This may help explain the kinds of work pre- prominence indicator among the elite groups. sented in the events (e.g., more innovation oriented and more recent) whereas journals tend to serve as clearinghouses for Introduction settled knowledge bases (Michela Montesi, 2008). In terms of recency, the median time lag between conference pub- In considering performance of a scholar, which of his or lication and journal publication is 2 to 4 years in software her scholarly activities should be measured as representative engineering (Michela Montesi, 2008). In nanotechnology, of performance? Numerous studies (covered in the Back- the journal article time lag from research funding is approx- ground section) have drawn upon different factors such as imately 2 to 3 years while conference presentations happen citation ratios and editorial board membership. These studies right after the funding (Daim, Monalisa, Dash, & Brown, have presented factors that may be considered in scholar 2007). The longer the time to publication, the increased risk of knowledge obsolescence. Medical disciplines tend Received September 10, 2008; revised January 16, 2009; accepted January to cite more recent research than is the case in natural sci- 16, 2009 ences and engineering (Lariviere, Archambault, & Gingras, © 2009 ASIS&T • Published online 12 March 2009 in Wiley InterScience 2008); similarly, the medical and chemistry journals become (www.interscience.wiley.com). DOI: 10.1002/asi.21056 obsolete faster than do social science journals (Glänzel & JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 60(6):1118–1131, 2009 Schoepflin, 1995). The journals’ “aggregated cited half-life” recognition by promotions committees, contributions to sci- in bioinformatics tends to be shorter (biotechnology & ence, and being able to read articles before they are published applied microbiology = 5.6, medical informatics = 6.7) than (Quencer, Bruns, Perrin, & Thompson, 1998). When a jour- that of social sciences (education = 8.2, business > 10.0) nal editor-in-chief selects editorial board members, he or she (Thomson Reuters, 2008), suggesting that the field is in nature has to consider a candidate’s performance as a manuscript an innovation-oriented discipline. The need to communi- reviewer, expertise in his or her field, and quality of arti- cate data that may quickly become obsolete is an important cles submitted in the past (Castillo, 2008). Several studies factor in explaining why conferences stand at the forefront have exploited editorial board membership of journals to of scientific research; however, note that the role, speed, gauge scholars and academic programs (Chan, Fung, & Lai, and importance of conferences vary between disciplines. 2005; Gibbons, 1990; Gibbons & Fish, 1991; Kam C. Chan, For example, conferences are the main publication outlet in 2003). Among them, Urbancic (2004) showed that institu- computer science, but are irrelevant publications in sociology. tions ranked within the top 25 in editorial board membership Researchers’ participation in scholarly events as elite- also are highly ranked for publishing productivity in the group members is beneficial to their career development. research field of real estate. He concluded that editorial board Here, “elite group” refers to the academic leadership in a membership offers an alternative method for evaluating the scholarly event, which includes organizing committee chairs academic standing of academic programs. Bakker and Rigter and members, program committee chairs and members, ses- (1985) also stated that scientific productivity and eminence sion chairs, invited speakers, and award winners. One may may be important factors in being invited as an editor in med- expect that the elite-group members are examples of aca- ical science; this implies that editorial board members are demic excellence in their domains and have the ability to qualified specialists in a given domain. Zsindely, Schubert, facilitate scholarly communication. Many scientists, hence, and Braun (1982) found that productive editorial board mem- list elite-group memberships in their curriculum vitae. These bers have influence to recruit promising research articles, elite group are the participants of this study. and the academic status (total citations) of editorial board The importance of elite groups for scholarly events sug- members are positively correlated with the scientific quality gests that elite-group membership for a scholarly event could (impact factor) of journals in chemistry. Chan et al. (2005) indeed be used as a “prominence marker” for scientists. Thus, found editorial membership-based measures to be objective the objectives of this study are (a) to find whether scientists’ and potentially useful in that board members are typically bibliometric indicators are high when they are invited to be productive scholars. All of these findings have indicated that elite-group members for scholarly events and (b) to deter- a scientist’s journal editorial board membership is closely mine whether scholars’ bibliometric indicators vary among related with the scholar’s academic performance. Given that elite-group types. To these ends, we observed the bibliomet- editorial board membership for journals correlates with aca- ric indicators (total citations and h index) of the elite-group demic stature in their fields, it may be possible that similar members and the non-elite-group individuals for scholarly kinds of membership in conferences also would be indicators events in bioinformatics (catering to global audiences only). of academic stature. We then performed statistical analyses for the identified bib- liometric indicators. This article reports detailed methods and Dissertation Committee Membership results in the following sections. We use “conference,” “aca- demic conference,” and “event” interchangeably to refer to a In a preliminary study on dissertation advising impact met- scholarly event. rics (MPACT), Marchionini, Solomon, Davis, and Russell (2006) proposed a series of metrics to assess the mentor- ing impact of dissertation advisors and committee members, Background including the number of dissertations advised (A), the