ACRL News Issue (B) of College & Research Libraries

A n n C. SchafSCOMMUNIATIOCCHOLNARLY fner, editor On scholarly evaluation and scholarly communication Increasing the availability of quality work by David E. Shulenburger s the scholarly communication crisis In 1997, the Pew Higher Education I largely a creature o f the faculty evalua Roundtable published a treatise entitled “To tion system? Do academic department Publish and Perish,” which urged univer heads, deans, and members o f promotion sities to “place greater emphasis on qual and tenure committees simply count the ity rather than quantity in the promotion faculty members’ publications and award and tenure process.” salary increases, promotion, and tenure by In March 2000, a gathering o f academ the numbers? If w e reformed the faculty ics, administrators, and librarians drew up evaluation system, would the scholarly the “Tempe Principles for Emerging Sys communication crisis disappear? tems o f Scholarly Publishing,” which have One commonly encounters anecdotes since been endorsed by both the AAU and that appear to support affirmative answers NASULGC membership. One o f the prin to these questions. Faculty sometimes boast ciples states: “To assure quality and reduce o f publishing the “least publishable unit,” proliferation o f publications, the evalua a reference to dividing significant work into tion o f faculty should place a greater em several smaller pieces to derive the maxi phasis on quality o f publications and a mum number o f articles from it. Others reduced emphasis on quantity.” describe mechanical systems they have Thus both anecdote and study point to established that, upon rejection o f a manu the faculty evaluation system’s role in gen script by one journal, will automatically erating published scholarship that adds submit that manuscript to the journal next little to the fund o f knowledge. H ow im in the status pecking order, continuing portant is this problem? through as many journals as needed until I have served on and chaired faculty one finally agrees to publish the manu evaluation committees at the school and script. university level for more than 20 years. Dur At least two significant efforts aimed at ing those years, I have review ed many gaining control o f the scholarly communi resumes that list publications that are at cation crisis have identified the faculty best marginal when evaluated against the evaluation system as part o f the problem. criterion of generation of new knowledge. About the author David E. Shulenburger is provost at the University of Kansas in Lawrence, e-mail: [email protected] 808 / C&RL News ■ September 2001 Editor's note A common theme in the debate about librarians, publishers, faculty, and admin scholarly communication has been the istrators for many years. need for faculty to publish in a large num For this month’s column, we have in ber of publications to receive tenure. vited David Schulenburger, provost of the “Publish or perish” is an accepted con University of Kansas, to share his views cept in higher education everywhere. on this subject. Schulenburger is well Are tenure committees truly blind to known for his efforts to help us under the issues of quality? How do they deter stand the economics of scholarly commu mine quality? Do we have too many low- nication and to reform the scholarly com quality publications, or do they serve a munication process. We hope that this purpose in the scholarly communication column will help to spur discussion of process? The answers to these questions these issues on your campus.—A nn C. are complex and have been debated by Schaffner, [email protected] Why did the faculty member write them? almost always weeds out bad science. Why were they published? However, I do not believe that the pro Mark Twain said that one should not cess admits only research that makes a sig criticize others on the grounds that one nificant addition to knowledge. Peer re cannot stand perpendicular to himself. It viewers are simply too close to the process is very difficult for an author to determine to be expected to know what will be the ultimate worth of his or her research. judged by future generations to represent No one sets out to do inconsequential significant additions to the discipline. Thus work, and having invested weeks, months, the refereeing process tends to weed out or years in a project, it is expecting too the bad but does not eliminate the insig much of human beings to judge their work nificant. to be inconsequential. Thus the norm is to But back to those résumés. Based upon write up the work and submit it for peer my many discussions with provosts across review so that others make the judgment. the nation about the evaluation process, I But peer reviewers have similar diffi believe that evaluation committees at the culties. Referees are themselves research University of Kansas are similar to those ers. As researchers they are entangled in at most research-intensive institutions. In the web of knowledge and become easily our process, volume of publication alone fascinated by a new detail or by the carries no weight. Evaluation committees resubstantiation of an old one. They look examine the perceived significance of the to see whether the data used should be faculty member’s work and if, and only if, relied upon, whether the work followed it is perceived to be of significance do they the methods required to produce valid sci begin to measure the quantity of the work. ence, whether it appropriately built upon Quantity takes on importance once qual the literature, etc., and then make a judg ity is established. Doing very small ment from the middle of the same thicket amounts of quality work simply is not suf as to whether it should be published. ficient justification for the standard expec tation that 40 percent of a faculty member’s Refereeing weeds out the bad time should be devoted to research. I have great respect for the refereeing pro The committee’s judgment of the ulti cess. While I am aware of the growing criti mate significance of a faculty member’s cism of this process, I have faith that it work is suspect for the same reason that About the editor A n n C . Schaffner is an M BA Candidate at Simmons Graduate School of Managem ent e-mail: [email protected] C&RL News ■ September 2001 / 809 nals and ultimately—through the weight of The real damage done by the published findings in low-level, peer-reviewed faculty evaluation process ... is not journals—finds its way over time into the top journals in the field. If faculty evaluation com by rewarding faculty for quantity of mittees or peer reviewers were true judges of publication; it is ... by basing ultimate significance, such articles would command great respect at first reading rather quality judgments on the rigor of than suffer automatic dismissal because of the the peer review process in journals low esteem for the publications in which they where their work appears, a process originally appeared. The real damage done by the faculty which is perceived to be strongest evaluation process then is not by reward in the top-ranked journals. ing faculty for quantity of publication; it is by rewarding faculty for quality of publi cation and by basing quality judgments on peer reviewers’ evaluations are suspect: the rigor of the peer review process in jour committee members simply don’t have the nals where their work appears, a process right perspective to make an infallible judg which is perceived to be strongest in the ment. The evidence used by evaluation top-ranked journals. Evidence that this is committees comes from their own reading true is the lack of uproar when a library of the work, their judgment of the rigor of cancels a subscription to a journal per review given the work by the journal of ceived to be of low quality. The lack of publication, and, especially in promotion turmoil over such decisions confirms that and tenure cases, the opinion of outside the problem is the reinforcement of de reviewers who evaluate the entire body of mand for top-quality journals, not the pro the faculty member’s work. liferation of journals of low quality. The latter group is particularly impor tant as outside reviewers are chosen be What can be done? cause they are experts in the faculty What we must do is restore the public member’s field. Given the narrowness of goods nature of journals by reducing the some fields, only by including external ability of journals to use the market power reviewers can real expertise be brought to they possess to raise prices. There are many the evaluation process. By reviewing the efforts now underway to accomplish this entire body of work from the viewpoint of aim, and SPARC (Scholarly Publishing and the discipline, outside reviewers are in a Academic Resources Coalition) represents position to judge the cumulative impact of one such strategy. By sponsoring modestly the faculty member’s work. priced new journals edited and refereed This evaluation process places essen by top scholars, SPARC endeavors to ac tially zero weight on publication in so- celerate the supply of prestigious journals called “backwater” journals. Evaluation and thereby reduce the possibility of fur committees generally take for granted that ther price increases by existing top tier jour work appearing in such outlets got there nals. By creating products like BIOONE, either because the author judged it to be SPARC keeps in the public domain a large of little worth and sent it directly to the group of journals in the biological sciences journal or because it failed to gain accep for which prices will not be raised.

Load more