Evaluating Review Content for Book Selection: an Analysis of American History Reviews in Choice, American Historical Review, and Journal of American History
Total Page:16
File Type:pdf, Size:1020Kb
322 College & Research Libraries July 1997 Evaluating Review Content for Book Selection: An Analysis of American History Reviews in Choice, American Historical Review, and Journal of American History Allen Natowitz and Paula Wheeler Carlo This study applies content analysis to compare assessments of 153 titles in American history, each reviewed in Choice, American Historical Review (AHR), and Journal of American History (JAH) between 1988 and 1995. A numerical rating system is used to quantify reviewers’ over- all opinions and their use of specific evaluative criteria. Study findings include: most books were praised in all three journals; the mean overall rating was highest in Choice and lowest in JAH; little agreement existed in identifying either outstanding titles or inferior titles; reviews in Choice contained the fewest evaluative criteria, but appeared 8.5 to 10.6 months prior to reviews of the same titles in JAH and AHR. ook reviews have repeatedly (AHR), are two of the most important been identified as major deter- journals for specialists in U. S. history. The minants in the library acquisi- authors set out to determine the degree tions process.1 However, despite to which book reviews in AHR, JAH, and the widely acknowledged importance of Choice provide collection development book reviewing, researchers have indicated specialists with qualitative assessments the need for more critical evaluations of re- on which to base their acquisition choices. views.2 The authors examined this subject by To achieve this, the contents of 459 book conducting a comparative analysis of Ameri- reviews (of 153 American history books can history book reviews in three highly reviewed in each of the three journals) regarded journals. Among them, Choice were analyzed. In addition to the overall is identified as the primary review me- opinion expressed in each review, the dium for academic libraries. The other authors’ analysis considered the ways in publications, Journal of American History which each of seven evaluative criteria (JAH) and American Historical Review was employed in these reviews. Allen Natowitz is an Assistant Professor in the Library Department at the College of Staten Island; e- mail: [email protected]. Paula Wheeler Carlo is an Instructor in the History Department at Nassau Community College. The authors wish to thank Professor Ivan Smodlaka of the College of Staten Island for his generous assistance and expert guidance in the statistical component of this study. 322 Evaluating Review Content for Book Selection 323 Review of Literature they review.”10 The typical Choice review Some studies of the general characteris- of 175 to 200 words is briefer than a re- tics of American history book reviews al- view in either of the history journals. In ready have been conducted. In particu- this limited space, reviewers are expected lar, Bilhartz analyzed book reviews to consider a book’s value to an under- published over a thirty-year period in graduate collection, analyze qualifica- JAH to determine changing perceptions tions of the author(s), compare a book to of what constitutes “good history.”3 Casey similar titles, and assess its strengths and examined multiple reviews of one hun- weaknesses.11 Choice publishes reviews of dred titles in American history and found more than 6,000 books annually, or 28 to that 82 percent of the reviews were favor- 29 percent of all books received.12 Ap- able or enthusiastic, and 18 percent were proximately 350 of these reviews appear unfavorable or harshly negative.4 in Choice’s categories of North American history, geography, and area studies. AHR, the official journal of the American Choice publishes reviews of more Historical Association, has been in existence than 6,000 books annually, since 1895. It is published five times a year and has a circulation of 23,000.13 Although To the authors’ knowledge, however, AHR contains scholarly articles in all fields the issue of reviewer consensus regard- of history, “more than half of this important ing the same titles in history has not been bibliographic source is taken up by reviews.”14 sufficiently explored. Although some AHR publishes more reviews than any other studies have been conducted in other dis- scholarly journal, more than 1,000 a year from ciplines, they have led to divergent con- about 4,000 books received by its editors.15 clusions.5 In pursuit of this issue, the According to William Bishel, AHR’s Assistant present study explores the degree to Editor for Book Reviews, “most historians which book selectors can expect to find would admit that the AHR is the ‘journal of reviewer consensus for the same books record’ in our discipline.”16 AHR reviewers in three important academic publications. are required (in most cases) to have published As Fisher has remarked: “Readers have at least one original book-length monograph. the right to know what to expect from the Reviews range from 500 to 1,000 words in review section in different journals.”6 length (though typically fall between 600 and 650 words).17 The journal provides no spe- Characteristics of the Three Journals cific guidelines for review content other than With a circulation of 4,000, Choice is pub- that the reviewer “should give a clear assess- lished eleven times a year under the aus- ment of the book’s content and a critical as- pices of ACRL.7 Since its inception in 1964, sessment of its contribution to knowledge in its primary purpose has been “to fill an the field.”18 informational need for college libraries. JAH appears four times a year and is . .”8 It has been identified as “the most published under the auspices of the Orga- influential review organ for academic nization of American Historians, which is publications” and “is a basic selection tool the primary scholarly organization for for academic librarians.”9 Unlike subject- American history specialists.19 From its specific journals (such as AHR and JAH), inception in 1914 until 1964, it was known Choice publishes reviews of titles in the as the Mississippi Valley Historical Review. humanities, science and technology, so- Unlike AHR, which covers all areas of his- cial and behavioral sciences, as well as tory, JAH concentrates on American his- reference works. Its reviewers include tory. Like AHR, its primary readers are aca- “college and university faculty who are demicians. With a circulation of about actively teaching in the subject areas that 12,000, it has been described as a “substan- 324 College & Research Libraries July 1997 tive bibliographic source and a basic jour- five-month intervals, between September nal for any library collecting in the field.”20 1988 and September 1993. Of these, 153 Similar to AHR, reviewers are expected to titles also were reviewed in AHR (be- have major publications in the area(s) they tween June 1989 and December 1994) and are reviewing. JAH publishes fewer re- JAH (between December 1988 and March views per year than AHR—600 as opposed 1995). to 1,000—but because all of them are in At this point in the process of execut- American history, its coverage in this area ing a content analysis, researchers define is more thorough than AHR’s. The typical their categories. Most of the categories review is 500 words in length, although adopted for use in this study were se- some are 400 words long and others are lected from those employed in compa- longer.21 Like AHR, JAH offers reviewers rable projects. The authors drew upon the no specific guidelines. following categories from Bilhartz: com- pleteness of research, objectivity of author, Methodology quality of analysis, and unity of thesis. According to Krippendorff, “Content analy- Bilhartz’s categories of enjoyability of the sis is a research technique for making repli- narrative, literary style, and literary clar- cable and valid inferences from data in their ity are bundled as the single criterion of context.”22 Allen and Reser point out that con- readability in this study. His separate cri- tent analysis has been applied to a variety of teria of overall rating and value to the library and information science questions, field are combined here as overall opinion. and conclude that this methodology has con- In addition, the authors have adopted siderable potential for research.23 It was se- quality of editing, which first appeared in lected by the authors as an appropriate tool Macleod’s study, and created a new cat- for quantifying data in order to determine egory, placement of events or subject in how frequently certain characteristics or cat- historical context.28 egories appear in book reviews. However, de- Coding the content of the data is the spite its successful application in a wide vari- next stage in the process of content analy- ety of research topics, Busha and Harter have sis. All the books in this study received advised that in order for content analysis to an overall opinion, or rating, according function effectively as a system of measure- to the following scale:29 ment, categories must be defined clearly and 1 = highly unfavorable accurately, and the classification and measure- 2 = moderately unfavorable ment of data must be undertaken with objec- 3 = inconclusive (reviewer equivocated tivity, exactness, and rigor.24,25 or merely summarized content) A number of book-length monographs 4 = favorable provide detailed descriptions of content 5 = outstanding or significant contri- analysis methodology.26 The following bution to the field discussion of the process derives from the work of Budd, Thorp, and Donohew.27 Determination of the overall opinion Effective content analysis begins by was influenced by the general tone of a accurately formulating the research ques- review as well a reviewer’s opening or tion. This study asks: What is the com- concluding comments (e.g., not recom- parative value of the content of reviews mended = 1; superlative or major work = published in Choice, AHR, and JAH for 5).