
HANDBOOK OF RESEARCH METHODS IN SOCIAL AND PERSONALITY PSYCHOLOGY Editedby HARRY T. REIS Universityof Rochester CHARLES M. JUDD University of Colorado at Boulder a CAMBRIDGE .UNIVERSITY PRESS CHAPTER THREE Causal Inference and Generalization in Field Settings Experimental and Quasi-Experimental Designs STEPHENG. WEST, JEREMY C. BIESANZ, AND STEVENC. PITTS The purpose of this chapter is to introduce researchers of each design. Methods of strengthening each design in social psychology to designs that pemlit relatively type with respect to causal inference and generalization strong causal inferences in the field. We begin the of causal effects are also considered. chapter by considering some basic issues in inferring The emphasis on designs for field research in the causality, drawing on work by Rubin and his associates present chapter contrasts sharply with recent practice (Holland, 1986, 1988; Rubin, 1974, 1978) in statis- in basic social psychology in which the modal research tics and by Campbell and his associates (Campbell, design consists of a randomized experiment, conducted 1957; Campbell & Stanley, 1966; Cook, 1993; Cook in the laboratory, lasting no more than 1 hour, and us- & Campbell, 1979; Shadish, Cook, & Campbell, in ing undergraduate students as the participants (West, press) in psychology. Rubin's approach emphasized Newsom, & Fenaughty, 1992). This practice has clear formal statistical criteria for inference; Campbell's strengths that are articulated in other chapters in this approach emphasized concepts from philosophy of handbook. At the same time, the recent extraordinary science and the practical issues confronting social level of dominance of this practice potentially limits researchers. These approaches are then applied to pro- the generalization of social psychological findings in vide insights on a variety of difficult issues that arise important ways. At earlier points in the history of the in randomized experiments when they are conducted field, it was far easier to find examples of basic social in the field. In addition, Cook's (1993) new perspec- psychological hypotheses that were tested in both labo- tive on the generalization of causal effects is also dis- ratory and field settings (see Bickman & Henchy, 1972; cussed. We then turn to consideration of three classes Swingle, 1973, for collections of examples); currently, of quasi-experimental designs -the regression disconti- the majority of research conducted by social psychol- nuity design, the interrupted time series design, and the ogists in field settings is applied in nature. The earlier no~equivalent control group design -that can some- interplay between laboratory and field research often times provide a relatively strong basis for causal infer- provided a more convincing basis for generalization of ence. The application of the Rubin and the Campbell causal effects found in basic social psychological inves- frameworks helps identify strengths and weaknesses tigations. We suggest that perhaps at least some of the current widespread perception that many articles in our leading journals Ujust aren't that interesting any- Stephen G. West was partially supported by NIMH Grant P50- more" (Reis & Stiller, 1992, p. 470; see also Funder, MN39246 during the writing of this chapter. We thank Tom 1992) may be occasioned by the limited contexts in Belin, Tom Cook, Bill Graziano, Chick Judd, Chip Reichardt, which social psychological phenomena are studied and Harry Reis, Will Shadish, and the graduate students of the the distance of present research paradigms from the Fall 1997 Psychology 555 class (experimental and quasi- real world. experimental designs in research) at Arizona State University The current modal laboratory research paradigm for their comments and suggestionson an earlier version of this chapter. Correspondenceshould be directed to StephenG. West, is particularly good at establishing internal validity, Department of Psychology,Arizona State University, Tempe, AZ that is, the treatment caused the observed response. 85287-1104 (e-mail: [email protected]). However, as complications are introduced into the 40 ~ CAUSAL INFERENCE AND GENERALIZAnON IN FIELD SETTINGS 41 laboratory setting, establishing internal validity be- the causal effect is defined as: comes more tenuous. Experiments conducted over re- peated sessions involve attrition; requests for partici- Yi(u) -~(u). pants to bring peers or parents to the laboratory are not Here, t refers to the treatment condition, c refers to the always fulfilled. Basic and applied social psychological comparison condition (often a control group), Y is research conducted in field settings may further mag- the observed response (dependent measure), and u is nify these potential issues. Consequently, researchers the unit (in psychology, typically a specific participant) must increase their focus on articulating those threats on which we observe the effects of the two treatments. to internal validity, which Campbell (1957) termed Rubin's definition leads to a clear theoretical state- "plausible rival hypotheses," that may be problematic ment of what a causal effect is, but it also impli~s a in their specific research context. A central theme of fundamental problem. "It is impossible to observe the this chapter is the identification of major classes of value of Yi(u) and ~(u) on the same unit and, there- plausible rival hypotheses associated with each of the fore, it is impossible to observethe effect of t on u" design types and the use of specific design and analy- (Holland, 1986, p. 947, italics in original). We cannot sis strategies that can help rule them out. And, as we expose a preteenage boy to the 10-week smoking pre- will indicate, many of these strategies also turn out to vention program, measure his attitudes toward smok- have potential implications for traditional laboratory ing, then return the child to the beginning of the school experiments as well. year, expose him to the comparison (control) program and remeasure his attitudes toward smoking. Because of this fundamental problem of causal inference, we A FRAMEWORK FOR CAUSAL INFERENCE IN will not be able to observe causality directly. However, EXPERIMENTS AND QUASI-EXPERIMENTS by making specific assumptions, we can develop re- Over the past two decades, Donald Rubin and his search designs that permit us to infer causality. The colleagues in statistics (Angrist, Imbens, & Rubin, certainty of the causal inference will depend strongly 1996; Holland, 1986, 1988; Imbens & Rubin, 1997; on the viability of the assumptions that we make. Three Rosenbaum & Rubin, 1983, 1984; Rubin, 1974, 1978, design approaches that address the fundamental prob- 1986) have developed a very useful framework for lem of causal inference are available, which may be understanding the causal effects of treatments. This employed alone or in combination. After briefly pre- framework has come to be known as the Rubin Causal senting the first two design approaches, we focus on Model (RCM). The use of the RCMis particularly help- the third approach, randomization, because it is most ful in identifying strengths and limitations of many of often used in basic and applied social psychology. the experimental and quasi-experimental designs in which the independent variable is manipulated and Design Approaches to the Fundamental posttest measures are collected at only one point in Problem of Causal Inference time following the treatment. Consider the simple case of two treatments whose WITHIN-SUBJECTDESIGNS. In within-subject de- effects the researcher wishes to compare. For exam- signs, participants are exposed to treatment (i) and ple, the researcher may wish to compare a severe frus- their responses are measured, following which they tration (treatment condition) and a mild frustration are exposed to treatment (ii) and their responses are (comparison condition) in the level of aggressive re- measured. Within-subject designs make two strong as- sponses they produce (see Berkowitz, 1993). Or, the sumptions. First is temporal stability, which means that researcher may wish to compare a 10-week smoking the response does not depend on the time the treat- prevention program (treatment condition) and a no ment is delivered. Many factors such as normal hwnan program group (comparison condition) on attitudes to- development, historical events, fatigue, or even daily or ward smoking among teenagers (see Flay, 1986). monthly cycles (see Cook & Campbell, 1979, chapter Rubin begins with a consideration of the hypothet- 2) can affect participants' responses. Second is causal ical ideal conditions under which a causal effect could transience,which means that the effects of each treat- be observed. He defines the causal effect as the differ- ment and each measurement of the response do not ence between what would have happened to the par- persist over time. For example, in a study of the effects ticipant under the treatment condition and what would of a high-stress and a low-stress film clip on increases have happened to the same participant under the con- in blood pressure, the researcher would need to make trol condition under identical circumstances. That is, the strong assumptions that exposure to the initial film 'it"it 42 STEPHEN G. WEST, JEREMY C. BIESANZ, AND STEVEN C. PITTS clip did not affect the perception of the second clip, chanceof being assignedto the treatment and control the participant had fully returned to her baseline level conditions.2Common randomization methods include
Details
-
File Typepdf
-
Upload Time-
-
Content LanguagesEnglish
-
Upload UserAnonymous/Not logged-in
-
File Pages46 Page
-
File Size-