<<

Review of General Copyright 1997 by the Educational Foundation 1997, Vol. 1, No. 3, 311-320 1089-2680/97153.00 Writing Narrative Reviews

Roy F. Baumeister Mark R. Leary Case Western Reserve University Wake Forest University

Narrative literature reviews serve a vital scientific function, but few resources help people learn to write them. As compared with empirical reports, literature reviews can tackle broader and more questions, can engage in more post hoc theorizing without the danger of capitalizing on chance, can make a stronger case for a null-hypothesis conclusion, and can appreciate and use methodological diversity better. Also, literature reviews can draw any of 4 conclusions: The hypothesis is correct, it has not been conclusively established but is the currently best guess, it is false, or the evidence permits no conclusion. Common mistakes of authors of manuscripts are described.

Narrative literature reviews form a vital part dents to apprentice with someone who has of most empirical articles, theses, and grant expertise in empirical methods. proposals, and of course many articles and Most are trained in chapters are devoted specifically to reviewing methods of data collection, and that training the literature on a particular topic. Literature generally includes how to write empirical reviews serve a scientific field by providing a manuscripts. Not surprisingly, researchers use much-needed bridge between the vast and their knowledge regarding empirical papers scattered assortment of articles on a topic and when they write literature reviews. Undoubtedly the reader who does not have time or resources there are important similarities between writing to track them down. Reviews also present literature reviews and writing empirical reports, conclusions of a scope and theoretical level that and indeed Bern (1987, 1995) has emphasized individual empirical reports cannot normally such parallels and similarities in giving valuable address. tips about writing. Yet, important differences For individual researchers, writing a major exist between writing empirical reports and literature review article is a very infrequent but writing literature reviews. In this article, we try often a very important career contribution. Yet, to offer an overview of the special problems, despite the importance of narrative literature advantages, opportunities, and pitfalls that reviews, no easy and available way to learn to pertain to narrative literature reviews, as com- write them is known. Research methods text- pared with writing empirical reports. do not usually explain how to do them, Our own collaboration began, perhaps fit- even though reviewing literature is an important tingly, with a literature review project. We had research method. Most graduate seminars in each by that point published a number of prior research methods likewise devote little or no literature review articles and chapters. What time to them. Apprenticeship with an accom- struck us, however, as we began our work plished literature reviewer seems to be one together was not how much we knew about the possible strategy to learn this technique, but process, but how ignorant we still were. With such specialists are rare, and moreover it is each new review paper, reviewers and editors generally considered more important for stu- raised new and important points that improved the style and impact of our reviews. In the Roy E Baumeister, Department of Psychology, Case absence of works that explained how to write a Western Reserve University; Mark R. Leary, Department of literature review, we continued to learn on a Psychology,Wake Forest University. trial-and-error, individual basis, guided by Correspondence concerning this article should be ad- editors and their consultants. Such repetitive dressed to Roy F. Baumeister, Department of Psychology, instruction is, of course, highly inefficient for Case Western Reserve University, 10900 Euclid Avenue, Cleveland, Ohio 44106-7123. Electronic mail may be sent the field, not to mention a large drain on the time via Internet to [email protected]. and energies of the editorial board of major 311 312 BAUMEISTER AND LEARY literature review journals, who find themselves more competing theories). In essence, the giving the same feedback and advice over and published literature provides a database from over. We hope this article can help future writers which the author draws conclusions about the benefit from the lessons we have learned. merits of existing conceptualizations. We should note that this article does not deal The leading review journals most commonly with meta-analysis papers, which constitute an publish articles that are aimed at theory important and valuable form of literature construction or theory evaluation. Authors review. Descriptions of how to do meta-analytic aspiring to write such reviews must therefore reviews are available elsewhere, however (Ea- recognize that their task is not simply assem- gly, 1987; Rosenthal, 1995). We do not see bling and describing past work but rather is one meta-analysis and narrative literature reviewing of building or testing theory. In important as in direct competition. Where meta-analysis is respects, such an article resembles a report of a usable (that is, when there are many studies laboratory : It describes empirical available testing the same hypothesis), it is evidence that evaluates a theoretical hypothesis. generally the preferred method. A narrative Each piece of evidence covered in the manu- literature review is valuable, however, when one script draws its value from how it helps build or is attempting to link together many studies on evaluate the overarching theory. different topics, either for purposes of reinterpre- A third type of literature review surveys the tation or interconnection. As such, narrative state of knowledge on a particular topic. Such literature reviewing is a valuable theory- reviews may provide useful overviews and building technique, and it may also serve integrations of an area, but they are not intended hypothesis-generating functions. Meta-analysis to offer novel ideas, new interpretations, or is, in contrast, a hypothesis-testing technique. sweeping conclusions. These reviews can be Narrative literature reviews also may be useful valuable as a means of pulling together what is for testing hypotheses when meta-analysis will known about a particular phenomenon, such as not work, such as when the studies are so for a grant proposal, or as a resource to teachers. methodologically diverse as to make meta- Because the theoretical contribution is minimal, analytic aggregation impractical. however, the leading journals are generally reluctant to publish manuscripts of this kind. Goals of Literature Reviews A fourth category of literature review has problem identification as its goal. The purpose is There are several different goals that literature to reveal problems, weaknesses, contradictions, reviewers may try to accomplish, and it is or controversies in a particular area of investiga- helpful to have one's goal clearly in mind while tion. The author may venture some tentative writing the . Five main goals can be solutions to the problems he or she identifies but distinguished, and these have implications for is more concerned with simply informing the the structuring of the article and its place--or field that some difficulty exists. Thus, such lack thereof in the literature. articles typically raise more questions than they The most ambitious goal of literature review answer, leaving it to future researchers to papers involves theory development. In such a straighten out the mess. These would appear in paper, the author's primary objective is to journals probably more as brief articles or propose a novel conceptualization or theory critiques than as full-length articles. Still, regarding some psychological phenomenon. identifying problems in the empirical literature The manuscript reviews the literature to provide can serve a valuable scientific function. a context for describing, elaborating, and A final, less common goal of a review article evaluating the new theory, or indeed the theory is to provide a historical account of the may be found in the integration of the material development of theory and research on a reviewed. A slightly less ambitious but more particular topic. Such papers are typically common type of literature focuses on theory organized chronologically and, although their evaluation. In this type of review, the author goal is primarily to trace the history of an idea, does not offer a new theoretical perspective but they typically provide an ongoing commentary rather reviews the literature relevant to the regarding the impact and shortcomings of validity of an existing theory (or often two or various contributions to the field. WRITING LITERATUREREVIEWS 313

Distinctive Aspects of Literature Reviews is to say that our empirical journals are filled with underinterpreted results. The editors are Most research psychologists have received correct in insisting that a single study (or even a some training in how to write empirical reports. set of studies) does not usually permit sweeping Not surprisingly, they use that knowledge when conclusions about the human condition, espe- they write literature reviews. Undoubtedly, there cially because the scientific rules of inference are important similarities, but we focus on the require caution and parsimony. Yet the re- differences between writing empirical reports searcher in our example is sometimes correct in and writing literature reviews because these the belief that the empirical finding reflects an offer the best opportunity to appreciate the important general principle---even if the editor special nature of the latter. In this section, is correct in objecting that the finding fails to therefore, we seek to cover what sets the rule out other general principles or alternative literature review apart from empirical reports-- possibilities at that level. The likely result is in terms of both problems and opportunities. therefore that despite many empirical studies on the topic, none of them can argue the broader Scope of Question and Level of Abstraction principle. That, of course, is where the literature review A first point about the usefulness of literature becomes useful. Literature reviews are vital to reviews is that they allow the researcher to the scientific field for bridging the gap in address much broader questions than a single interpretation. Certain broad conclusions may empirical study can. By focusing on patterns and indeed lie forever beyond the reach of any single connections among many empirical findings, a investigation, but a literature review that exam- literature review can address theoretical ques- ines and integrates the results of dozens of tions that are beyond the scope of any one study. studies can address them. Without literature At most, an empirical report can raise such reviews, the field might remain permanently implications in a brief and speculative way, unable to answer some of its most fascinating whereas a literature review can permit conclu- questions. sions about them. Most writers of empirical articles have Post Hoc Theorizing probably been pressured by reviewers and editors to rein in theoretical claims that were Graduate students in psychology are routinely seen as excessively speculative. The editorial taught the importance of delineating one's consultants point out, usually rightly, that the hypotheses in advance (i.e., prior to collecting zealous author's sweeping conclusions are not data). Established researchers continue to regard warranted by his or her data set. We have two it as questionable and possibly unethical to points to make about this. First, the incapacity of theorize after one's empirical results are known. the data set to justify such theoretical claims There are good reasons for such insistence, even probably does not reflect a flaw in that particular though many experts suspect that researchers do study, but rather it reflects a limitation in the continue to refine and develop their ideas after very nature of single data sets. Any single study the data have been analyzed. will rarely yield enough data that can justify Prominent among these reasons is the danger broad conclusions about human nature, human of capitalizing on chance. Random variation behavior, or the human condition. will yield a fair number of spurious significant Second, and perhaps more importantly, those findings. If researchers were permitted to for- sweeping claims that the author desires to make mulate their theories after completing the study, might well be true. (After all, there are they would invent reasons for these chance presumably some interesting and valid general- findings, which would then become incorpo- izations to be made.) A serious problem is then rated into the field's body of knowledge, thereby apparent. If such general patterns and principles polluting it with false conclusions. Requiring exist, but no single empirical study is adequate researchers to formulate hypotheses in advance to justify drawing such broad conclusions, how helps protect the field against these errors. are they to be found or contributed to the field? Such concerns do not apply to literature Another way of looking at the same problem reviews, however. As a result, it would therefore 314 BAUMEISTER AND LEARY be unfortunate and misguided for literature know all too well that inconclusive or null findings reviewers to limit themselves to a priori typically doom their chances of publication. A hypothesizing in the way that empiricists single study that fails to find significant results is should. The crucial difference is that the danger by nature ambiguous, so editors rarely publish of capitalizing on chance, which can be quite such papers. Knowing this, authors of empirical serious in a single empirical study with several reports often feel pressured to make the strongest dozen variables and possibly hundreds of possible case that their findings contribute a analyses, is negligible in a literature review. For clear, positive, unambiguous conclusion. a literature review to make such an error would That pressure should, however, be greatly require that dozens of studies somehow all point diminished for authors of literature reviews. Un- toward a wrong conclusion by statistical fluke, like most empirical reports, a literature review which would be highly improbable. (Other can make a useful contribution to the field by studies would almost certainly find the correct concluding that the existing data are inadequate answer.) Hence it should be regarded as entirely to answer some question. Such assessments of permissible for a literature reviewer to formulate the state of the literature help other empirical hypotheses after reading the literature. In this investigators know where to direct their efforts respect, assembling evidence by reading the and help editors judge the novelty and impor- literature is not the same as assembling evidence tance of future empirical findings. by collecting original data. In extreme cases, literature reviews can make Our view carries this point a step further: It is a positive contribution to the field even by not only permissible but positively desirable that concluding that no definitive answers can be a literature reviewer theorize after assembling drawn from the existing data, or that the null the evidence. A literature review allows one to hypothesis cannot be rejected. In particular, take a big step up in the level of abstraction (as some literature reviews tackle a set of related compared with the level of an empirical study). questions and may conclude that one or more of When one takes that step, armed with a newly them remains unanswered. An empirical investi- assembled set of facts and findings, one may gator who tested four hypotheses and learned often discover that one's original ideas formu- nothing conclusive about one of them would lated the issue wrongly or failed to anticipate probably be pressured to drop that one from his various questions and answers. It would be or her report. A literature review could conceiv- foolish (and costly to the scientific field) to insist ably be most useful for its identification of on sticking with one's original ideas. To put this which questions remain undecided. another way, literature reviewers can and should The value of a literature review that con- remain open to new ideas far longer than cludes that the evidence is inconclusive is empirical investigators. probably most apparent when the article is in the Ultimately, literature reviewers have far less problem identification category mentioned ear- control over their evidence than empirical lier. That is, one possible goal of a literature researchers because they are constrained by review is to indicate that a persistent problem or what other researchers have already done. To ambiguity renders some body of evidence less impose one's own a priori categories on that conclusive than is widely believed or perceived. body of evidence may often be an act of In such cases, noting that the field knows less procrustean rigidity that will yield misleading than it might have surmised is important. As we conclusions. Literature reviewers should allow noted, such a goal is more compatible with themselves to be led by their evidence far more briefer comments or critique articles than with than empirical researchers dare. Failure to do so full-size literature reviews aimed at developing can cripple the capacity of a literature review to and evaluating theory, but it nonetheless can fulfill its scientific function. serve a valuable corrective function.

The Value of Null Number of Possible Conclusions

The pressure for positive results is another As the previous section implies, the rules of difference between writing literature reviews vs. inference regarding permissible conclusions empirical results. Authors of empirical reports may be quite different for empirical investiga- WRITING LITERATURE REVIEWS 315 tions as opposed to literature reviews. We affairs is quite distinct from asserting either that suggest that whereas a successful empirical the hypothesis has been unequivocally sup- investigation permits only one type of conclu- ported or that the data are inconclusive. For sion, a successful literature review should researchers, this second conclusion entails that permit four. the burden of proof should at least be shifted Nearly every aspiring researcher is taught that onto other side of the , allowing the empirical studies test hypotheses that yield two field to assume for the time being that the possible outcomes, only one of which is hypothesis is true (which it probably is). For meaningful. Specifically, a study either rejects practitioners and applied psychologists, such a its null hypothesis (while supporting its non-null conclusion is particularly valuable because they hypothesis) or it fails to do so, and failure to need the best currently available answer when a reject the null hypothesis is deemed inherently client or practical problem cannot wait many inconclusive. The positive conclusion that ac- years until definitive proof may be forthcoming. companies the rejection of the null hypothesis is The third possible conclusion is that one does therefore the only useful conclusion an empiri- not know whether the given hypothesis is true or cal study can draw. In contrast, a literature false. This may arise because evidence is review can effectively and usefully draw any of lacking, is internally consistent and contradic- four types of conclusion. It seems desirable that tory, or suffers from one or two pervasive flaws literature reviewers keep these different possible (as opposed to widely varied flaws) that render it conclusions in mind and distinguish among ambiguous. As already noted, such a conclusion them, instead of simply using the yes-or-no is usually unpublishable in an empirical report, dichotomous logic that guides empirical work. but in a literature review it can make a valuable The first type of conclusion is the same as for contribution. an empirical study: The hypothesis is correct, at The fourth possible conclusion is that the least based on the present evidence. A literature hypothesis is false. A single study with a null reviewer who finds that multiple studies provide result is inconclusive, but if several dozen converging evidence for the same conclusion is studies all fail to support a given hypothesis, justified in saying that, at least given the current then probably it is wrong. Contrary evidence can state of knowledge, the hypothesis is supported. provide two distinct types of information about Indeed, given that the literature reviewer has the wrongness of the hypothesis: The hypothesis multiple studies and findings to work with, he or may be wrong (as in a null-hypothesis conclu- she can draw that conclusion with much more sion) or the opposite of the hypothesis is correct. confidence than can the author of any single This fourth type of conclusion is especially study. important when one is dealing with a belief or The second possible conclusion is that the hypothesis that has become widely accepted. It hypothesis, although not proven, is currently the is after all possible that a wrong theory will best guess and should be assumed to be true become generally accepted in any scientific until contrary evidence emerges. Such a conclu- field, perhaps especially a field such as psychol- sion might be reached if the evidence is subject ogy in which early writers put forward impor- to various flaws and biases but points consis- tant conclusions with little evidence and these tently to the same conclusion. If all the evidence have been passed down without question. A is flawed, but the flaws are different, then the single empirical study providing contrary evi- most parsimonious conclusion is that the dence will often encounter resistance to accep- hypothesis is correct. Such a conclusion must tance for publication if it goes against prevailing remain tentative, however, because it is possible views (and probably rightly so). A literature that a convergence of artifacts produced it. review of multiple findings may often be the This second type of conclusion has no only effective mechanism for the field to free analogue in empirical investigations, but it is itself of entrenched errors. important in literature reviews. Often one will finish reading a mass of literature fairly Methodological Convergence convinced that a hypothesis is correct but will be unwilling to assert that the findings provide The social and behavioral sciences address a definitive support for it. Logically this state of daunting assortment of theoretical issues, and 316 BAUMEISTER AND LEARY they have developed a mind-boggling assort- Inadequate Introduction ment of methods. In our view, this methodologi- cal diversity is a highly desirable response to a One common error is inadequate develop- serious problem: By and large, none of the ment of the introduction of the manuscript. methods is perfect. The endless quest to devise Authors may skimp on presenting their concep- new research methods reflects the perennial and tual and theoretical ideas early in a manuscript valid that existing methods are not for many reasons. Some authors seem to think sure guides to the truth. that the purpose of a literature review is simply If all methods are flawed, then the conclu- to describe a collection of relevant findings, so sions are limited by the flaws. Indeed, the nature no integrative theorizing is needed, and the of methodological flaws is that they permit introduction does not have any task except to alternative interpretations of the results. The convince the reader how important and interest- ing the topic is. (As we said, though, such judgment about whether to publish a given reviews have little chance of being published in empirical study often boils down to how leading journals.) Other authors postpone the seriously the conclusions are impaired by these integrative theory until the discussion section methodological weaknesses and flaws. after all the studies have been described, The rule of parsimony gives the literature because that sequence corresponds to the reviewer an important advantage over the author's own thought process as he or she empirical researcher, however, if the evidence reviewed the area--reading the evidence with being reviewed is methodologically diverse. an open mind and then drawing conclusions. Convergence of evidence across multiple re- Also, some authors may postpone their integra- search methods normally entails that the given tive "take-home message" until late in the hypothesis is the most parsimonious conclusion. manuscript because such a delay increases its Different flaws entail different alternative inter- dramatic impact. That is, they want to show pretations. Whereas one alternative interpreta- what a hopeless mess the literature on their topic tion may be enough to undermine an empirical seemed before demonstrating how their recon- study, it may not be able to account for all the ceptualization can bring order to the chaos. findings if the methods are diverse. Furthermore, as we said, empirical authors feel Hence, literature reviewers are obligated to some obligation not to mention post hoc attend to the methodological diversity, and not theorizing in the introduction. just the quantity, of evidence. A hypothesis For a literature review, however, it is usually supported by 50 studies may in fact be shakier, necessary to present a full and vigorously more dubious, and simply more wrong than a integrative theoretical framework early in a hypothesis supported by 5 studies, if the 50 used manuscript. Few readers can manage to wade the same method whereas the 5 used all different through 50 pages of text and dozens of facts and ones. Methodological diversity should be espe- findings before learning what the point is. And cially prized by narrative literature reviewers the simple description of findings without a because it is something they are uniquely novel theoretical contribution is typically not positioned to recognize and use in drawing sufficient to warrant publication. conclusions about the literature. We see two ways for authors to incorporate theoretical points early in a review manuscript. One is to present one's full theoretical conceptu- Common Mistakes alization up front, using the remainder of the manuscript to review the literature relevant to In this section, we discuss several common the theory. Alternatively, an author might mistakes that authors commit when undertaking provide a brief "bottom-line" preview of the major reviews of the literature. These are theory early, postponing its full elaboration until mistakes that have been pointed out to one or after the literature has been reviewed. Either both of us have at various times or that we have approach provides readers with a sufficient observed when we have read literature review context to make sense of the specific studies and manuscripts written by others. findings cited in the review. WRITING LITERATUREREVIEWS 317

Inadequate Coverage of Evidence evidence has to be presented at the operational level, not just at the abstract level of theoretical A surprisingly common flaw in literature conclusions. review manuscripts, especially those that are rejected for publication, is inadequate coverage Lack of Integration of the cited literature. This inadequacy stems from a basic uncertainty about how much detail The previous section cited the problem of to give. For example, some authors cover the failing to describe the operational aspects of the literature in an unbalanced fashion, devoting reviewed studies. The opposite problem is multiple pages to a thorough description of a equally destructive to the value of a literature few favored studies (often their own) but review. This problem occurs when authors cursory description of the rest. As journal describe the procedures and observations of reviewers, we have occasionally wanted to send various studies but fail to relate them to the the manuscript back and tell the author to fill in theoretical issues. the missing information before we could make All reviews, regardless of their primary goal, even a tentative evaluation. should provide an overarching conceptualiza- One common form of inadequacy involves tion, perspective, or point-of-view--what Stern- citing a study's conclusion without describing berg (1991) called a take-home message--and the method and specific results. That is, after all, not be content to merely recount previous ideas what many empirical articles do when citing and research. The broader imperative is that literature in their introductions. Because most authors of literature reviews must explain how authors of literature reviews are empirical the various studies fit together. A literature investigators, they may use the same style, even review that simply describes a series of studies unwittingly, when they try their hand at writing on some topic has not accomplished enough to a literature review. warrant publication. (Nor is it sufficient to A literature review loses considerable value, postpone the integration until the general however, if it fails to tell the reader the nature of discussion, because most readers cannot keep all the evidence it presents. "X causes Y (Refer- that information straight in their minds for that ence)" does not convey enough information, long.) especially for readers who may be skeptical of A literature review is primarily an integrative the author's conclusions or who want to think endeavor, and integration is best accomplished for themselves. In contrast, "in a sample of A, if the reader is frequently told how the method B produced result C (Reference), individual studies fit the broad theories and thereby supporting the view that X causes Y" is patterns. To the literature reviewer, covering all much more useful. It allows the reader to the relevant studies may seem like the most evaluate whether the conclusion fits the evi- important thing, but to the reader the important dence and to understand something about the thing is how they fit together. To be sure, the generality and methodological strength of that literature reviewer should first ensure that he or evidence. By neglecting to describe the nature of she has covered the research accurately and the evidence, the author of a literature review thoroughly. But literature reviewers should also forces the reader to rely simply on the author's ask themselves whether they have presented interpretation--"take my word for it." But the each study in a way that makes its relation to the very purpose of a literature review is to provide integrative themes clear and explicit. a basis for accepting a conclusion without taking someone's word for it. Lack of Critical Appraisal To put this another way: If a particular study contributes something of importance to a In the social and behavioral sciences, all literature review, the review should summarize conclusions are limited by the weaknesses and the gist of the method and results sections of that flaws of the evidence, and so it is essential for article. This does not have to be lengthy, and in the literature reviewer to point out and assess fact a skilled literature reviewer can often those flaws and weaknesses. Often, however, present the relevant aspects of a study's method authors of literature reviews neglect to do this. and results in a sentence or two. But the There are several reasons for such neglect, but 318 BAUMEISTER AND LEARY

none is good. Providing a critique of the quantity and especially the methodological evidence is an integral, even a central part of the diversity of the evidence, keeping in mind that job of reviewing literature. consistency across large quantities of method- One reason authors neglect critique is that ologically diverse evidence is the best available they are trying to build a case for a particular substitute for having the proverbial direct argument, and so they searched the literature pipeline to the truth. Finally, the author should and presented the results so as to make that point provide a summary as to how strong the most convincingly. They should realize, how- evidence is. ever, that overstating the case is a serious, unnecessary, and self-defeating error in ap- Failure to Adjust Conclusions proach. It would enhance rather than diminish the value of their manuscript to indicate the We have said that conclusions should be weaknesses in the evidence. Moreover, helping tempered by the flaws and weaknesses in the empirical investigators see what remains to be evidence. Sometimes authors of literature re- done will probably increase the usefulness of views will dutifully provide a critique of the (and future to) the article. evidence but then present strong, sweeping Another reason that authors of literature conclusions that seem to have ignored the reviews fail to provide critiques is stylistic. critique entirely. As in empirical papers, authors Criticizing every study could possibly double of literature reviews sometimes go beyond the the length of a manuscript, in addition to making data. This pattern may be particularly common it very tedious to read. One solution to this when an editor or advisor insists, after problem is to provide critiques of groups of reading a first draft, that critique should be studies rather than commenting on each study added, and so the author inserts the requested individually. Group or section critiques are often critique without revising the conclusions accord- useful because many studies on the same topic ingly. may be subject to similar flaws and criticisms. Earlier, we proposed four different possible Ultimately, the reader of the literature review conclusions that a literature review can draw does not need to know every flaw in every study. with respect to a hypothesis. It is the strength of Indeed, if one study has a flaw that is corrected the evidence that mostly decides among these in another study, the first study's flaw is rather four. The strength of the evidence encompasses irrelevant in the big picture. Rather, the reader its methodological rigor or lack thereof, the needs to know how strong or how weak the amount of evidence, its consistency, and its overall evidence for each main point is. Group methodological diversity. In particular, these or section critiques accomplish this better than may be needed to distinguish between the two criticizing each individual study. kinds of favorable judgments a literature review Another advantage of group critiques is that can make, namely whether the hypothesis is they permit evaluation of convergence across well established as correct vs. whether it is diverse methods, which, as we noted, is one of merely the best guess based on currently the advantages of literature reviews. To list a available evidence. flaw in every study may give the impression that all the evidence is so weak that no conclusion Blurring Assertion and Proof can be drawn. As we noted, however, if each study has a different flaw, parsimony may A simpler error that is related to the failure to dictate that the hypothesis be tentatively ac- describe the nature of evidence is the failure to cepted. distinguish between assertion and evidence. At Hence the most useful form of critique is issue are statements of the sort, "women are normally the following: After describing the smarter than men (Brown & Green, 1966)." methods and results of a group of studies Such statements leave unclear the crucial relevant to some point, the author should question of whether Brown and Green merely indicate briefly the major flaws in the methods asserted that women are smarter or actually and what alternative explanations they raise. provided supporting evidence. In psychology, Next, the consistency of the findings should be with its rich legacy of theoretical speculation considered. Then the author should assess the based on informal observation, it is quite easy to WRITING LITERATUREREVIEWS 319 blur the distinction between citing someone who Selectivity in a literature review can take stated an idea and citing someone who provided several forms. At worst, the author may be evidence for it. operating as an "intuitive lawyer" rather than in For writers of empirical reports, this distinc- a scientist mode, in the sense of trying to make a tion is not always important. An empirical report case for one particular position or conclusion usually cites past literature merely to provide a (Baumeister & Newman, 1994). This could lead context for its own evidence, which will be him or her to cover only material that fits that presented in detail in the method and results view and ignore the rest. The reader is therefore sections. In a literature review, however, the left unaware of material that would weaken or evidence is precisely in the presentation of past contradict the argument. Such an approach is at research, and so the nature of this evidence must best unfortunate and sloppy, at worst intellectu- be explained carefully and fully. Thus, for ally dishonest. literature reviews, the distinction between asser- A less bad but still unfortunate pattern is that tion and evidence is very important. of selective critique. In this pattern, the author There is undoubtedly some scholarly obliga- covers all the relevant evidence, both supportive tion to acknowledge important theoretical asser- and contrary to his or her view, but then applies tions by previous generations. Indeed, the bulk more rigorous methodological standards to the of the many citations to , contrary evidence than to the supportive evi- , and dozens of others refer to dence. Such thought patterns are common thoughtful theoretical proposals rather than sources of bias in everyday thinking (Kunda, empirically demonstrated facts, at least by 1990; Lord, Ross, & Lepper, 1979), and modern standards of empirical proof. It seems literature reviewers are undoubtedly subject to impractical to propose thaJ~eitations should only them. be made for empirical demonstrations. Ideally, a literature reviewer should spend However, we recommend that literature re- some time searching for counterexamples or viewers make a persistent effort to inform the domains of evidence that would seem to reader whether a cited source proved or merely contradict the main conclusions and patterns. asserted something. "Brown and Green pro- These can be included in the article as separate posed that women are smarter than men" would sections and given the same critical appraisal as clearly not be confused with "in a sample of the supporting evidence. If there are important middle-aged , women scored higher on a exceptions to the general patterns and conclu- test of logical reasoning than men (Brown & sions, the literature review is strengthened by Green, 1966)." Because most sources cited in a acknowledging them, and theory can be built literature review will presumably be empirical further by recognizing moderators and boundary reports, it seems most important to be explicit in conditions. If the exceptions are merely appar- pointing out whenever some source is being ent and do not on close inspection contradict the cited merely for making a speculative or main pattern, the manuscript is strengthened by theoretical assertion. pointing them out. Thus, either way, a literature review can be improved by a deliberate search Selective Review of Evidence for contrary evidence.

Although literature reviews are less subject Focusing on the Researchers Rather Than than empirical investigations to capitalizing on the Research chance, they are probably more susceptible to the danger of confirmation bias. Many good In his advice to writers of literature reviews, literature reviews involve seeing a theoretical Bem (1995) proposed that names of researchers pattern or principle in multiple spheres of should always be relegated to parentheses rather behavior and evidence, and putting together than occupying grammatically prominent posi- such a paper undoubtedly involves an aggres- tions in the sentences. We find this rule sive search for evidence that fits the hypoth- excessive and can think of instances in which esized pattern. Areas that do not pan out are easy the sentences do need to feature the names of to ignore or skip, but the result can be a researchers (e.g., "Jones found one thing, but misleading impression of universality. Smith found the opposite"). Nonetheless, our 320 BAUMEISTER AND LEARY disagreement with Bern is merely a matter of that goes far beyond what almost anyone else degree and exception, and we concur wholeheart- (even researchers who continue to collect data edly with the main thrust of his argument. Good on the topic) can achieve. Pointing out what writing of literature reviews requires a concerted remains to be studied is relatively easy from that effort to feature the findings and ideas. Down- privileged position, but quite difficult for almost playing the names of researchers (such as by anyone else. Moreover, indicating the directions putting citations in parentheses) is a valuable for future research can help an article influence stylistic device for ensuring that the article the field, both by telling researchers what they focuses on ideas and research rather than on should study and by enabling future researchers theorists and researchers. It also helps the writer to convince editors that their work is valuable to avoid the appearance of making ad hominem and important. . As a rule of thumb, starting para- graphs in a literature review with the name of a Concluding Remarks researcher is particularly problematic, often a sign that the writer is simply describing one Our position is that the narrative literature study after another without making a sufficient review occupies a special and privileged place effort to integrate them. in the scientific enterprise. Its opportunities, Another exception to the guidelines of epistemological constraints, and stylistic needs confining names to parentheses arises when a differ in important ways from empirical reports. previous author is being cited for asserting a By appreciating these differences, individual point on theoretical grounds rather than provid- researchers may become more effective consum- ing evidence. We have already insisted on the ers and more successful authors of literature importance of keeping a sharp distinction reviews. between sources who said something and sources who provided evidence to support it. The most convenient way to highlight the References former cases is to use precise verbs, such as Baumeister, R. E, & Newman, L. S. (1994). "James asserted..." or "Johnson theo- Self-regulation of cognitive inference and decision rized..." or "Watson speculated..." Again, processes. Personality and though, these are merely exceptions to Bem's Bulletin, 20, 3-19. rule. Bem, D. J. (1987). Writing the empirical journal article. In M. P. Zanna & J. M. Darley (Eds.), The Stopping at the Present compleat academic: A practical guide for the beginning social scientist (pp. 171-201). New York: Random House. A final mistake of authors of literature Bern, D. J. (1995). Writing a review article for reviews is to neglect to say, explicitly, what the Psychological Bulletin. Psychological Bulletin, implications for future research are. Usually this 118, 172-177. will be a subsection in the general discussion or Eagly, A. H. (1987) Sex differences in social other concluding section. Editors normally are behavior." A social-role interpretation. Hillsdale, not satisfied to publish a literature review that NJ: Erlbaum. summarizes a large number of studies that are Kunda, Z. (1990). The case for motivated reasoning. already published anyway, even if there are Psychological Bulletin, 108, 480--498. good theoretical conclusions and integrative Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased patterns. They want a literature review to point assimilation and attitude polarization: The effects of prior theories on subsequently considered out remaining unresolved issues and questions, evidence. Journal of Personality and Social if not specifically what remains to be done, as an Psychology, 37, 2098-2109. aid and perhaps a to further research. Rosenthal, R. (1995). Writing meta-analytic reviews. Such recommendations may seem minor, Psychological Bulletin, 118, 183-191. trivial, or obvious to the literature reviewer, but Sternberg, R. J. (1991). Editorial. Psychological they are not. They only seem that way because Bulletin, 109, 3-4. the literature reviewer is so well immersed in the topic. Reviewing a body of literature puts one in Received April 11, 1997 the privileged position of having a broad grasp Accepted April 17, 1997 •