CHAPTER NINE Survey Research PBNNY S. VISSBR, JON A. KROSNICK, AND PAUL J. LAVRAWS Social psychologists have long recognized that every Frey, 1989; Lavrakas, 1993; Weisberg, Krosnick, 6 method of scientific inquiry is subject to limitations Bowen, 1996).We begin the chapter by suggestingwhy and that choosing among research methods inherently survey research may be valuable to sodal psychologists involves trade-offs. With the control of a laboratory and then outline the utility of various study designs. experiment, for example, comes an artificiality that Next, we review the basics of survey sampling and raises questions about the generalizability of results. questionnaire design. Fiy,we describe procedures And yet the 'naturalness" of a field study or an observa- for pretesting questionnairesand for data collection. tional study can jeopardize the validity of causal infer- ences. The inevitability of such limitations has led many REASONS FOR SOCIAL PSYCHOLOGISTS TO methodologists to advocate the use of multiple meth- CONDUCT SURVEY RBSEARCH ods and to insist that substantive conclusions can be most confidently derived by triangulating across mea- Social psychologists are interested in understanding sures and methods that have nonoverlapping strengths how people influence, and are influenced by, their so- and weaknesses (see, e.g., Brewer, this volume, Ch. 1; cial environment. And to the extent that social psy- Cpmpbell 6 Piske, 1959; Campbell 6 Stanley, 1963; chological phenomena are universal across different Cook 6 Campbell, 1969; Crano 6 Brewer, 1986; B. types of people, it makes little difference precisely with Smith, this volume, Ch. 2). whom social psychological research is conducted - This chapter describes a research methodology that even data collected from samples that are decidedly un- we believe has much to offer social psychologists in- representative of the general population can be used to terested in a multimethod approach: survey research. draw inferences about that population. Survey research is a specific type of field study that in- In recent years, however, psychologists have be- volves the collection of data from a sample of ele- come increasingly sensitive to the impact of disposi- ments (e.g., adult women) drawn from a well-defined tional and contextual factors on human thought and population (e.g., all adult women living in the United social behavior. Instead of broad statements about uni- States) through the use of a questionnaire (for more versal processes, social psychologists today are far more lengthy discussions, see Babbie, 1990; Fowler, 1988; likely to offer qualified accounts of which people, un- der which conditions, are likely to exhibit a partic- ular psychological phenomenon or process. And ac- This chapter was completed while the second author was a cordingly, social psychologists have increasingly turned Fellow at the Center for Advanced Study in the Behavioral their attention to interactions between various social Sciences, supported by National Science Foundation Grant psychological processes and characteristics of the indi- SBR-9022192. Correspondence should be directed to Penny S. vidual, such as personality traits, identification with a Visxr, Department of Psychology, Princeton University, Prince- ton, New Jersey 08544 (e-mail:pvisseSPrinceton.edu). or Jon social group or category, or membership in a distinct A. Krosnick. Department of Psychology, Ohio State University, culture. In many cases, the nature of basic social psy- 1885 Neil Avenue, Columbus, Ohio 43210 (e-mail: Krosnick chological processes has been shown to depend to a @osu.edu). large degree on characteristics of the individual. 2U PENNY S. VISSER, JON A. KROSNICK, AND PAUL J. LAVRAKAS The process by which attitude change occurs, for 6 Arenberg, 1983; Nesselroade 6 Baltes, 1974), have example, has been shown to differ for people who are more weakly established self-images (Mortimer, Pinch, low and high in what Petty and Cacioppo ( 1986) have 6 Kumka, 1982), and have less well-developed social termed 'need for cognition," a general enjoyment of identities (Alwin et al., 1991) than-do older adults. and preference for effordul thinking. Attitude change Because of these kinds of differences, Sears (1986) among people high in need for cognition tends to be argued, the field's reliance on participant pools of mediated by careful scrutiny of the arguments in a per- college-aged adults raises questions about the gener- suasive appeal, whereas attitude change among people ahability of some findings from social psychological low in need for cognition tends to be based on cues in laboratory research and may have contributed to a dis- the persuasive message or context, such as the attrac- torted portrait of 'human nature." However, the evi- tiveness of the source. dence Sears (1986) cited largely reveals the prevalence Similarly, attributions have been shown to differ de- of certain characteristics (e.g., the frequency of atti- pending on social group membership (see, e.g., Hew- tude change or the firmness of social identities), rather stone, Bond, 6 Wan, 1983). People tend to attribute than differences in the processes by which these char- positive behaviors by members of their own social acteristics or others emerge in different age groups. We group or category to stable, internal causes. Those same currently know so little about the operation of social positive behaviors performed by a member of a differ- psychological processes in other subsets of the popula- ent social group, however, are more likely to be at- tion that it is impossible to assess the extent of bias in tributed to transitory or external factors. this regard. According to much recent research, culture may Doing so will require studies of samples that are also moderate many social psychological phenomena. representative of the general population, and induc- Markus and Kitayama (1991), for example, have ar- ing most members of such samples to visit a labora- gued that members of different cultures have differ- tory seems practically impossible. Studying a repre- ent construals of the self and that these differences sentative sample through field research, however, is can have a profound impact on tlye nature of cogni- relatively easy and surprisingly practical. Using the ba- tive, emotional, and mo@vationalpmceses. Similarly, sic tenets of probability theory, survey researchers have Nisbett and his colleagues'(Coheh, Nisbett, Bowdle, 6 developed a number of effident strategies for drawing Schwan, 1996; Nisbra, 1993; Nisbett 6 Cohen, 1996) representative samples that are easy to contact. And have explored what they tcnned the 'culture of honorw when samples have been selected in such a manner, of the American South and have demonstrated marked social psychologists can confidently generalize findings differences in the cognitive, eimotiod, behavioral, and to the entire population. Furthermore, survey research even physiological reactions af southem men (relative provides ideal conditions for the exploration of Pro- to their northern counterparts) when confronted with cess x Individual Difference interactions because care- insult. fully selected samples reflect the full heterogeneity of These kinds of process-by-indi--difference in- the general population. teractions suggest that precisely who participates in so- There are two primary limitations of survey research cial psychological research can have a profound im- for social psychologists. First, surveys are more expen- pact on what results are obtained. And of course, for sive and time-consuming than most laboratory experi- the vast majority of social psychological research, that ments using captive participant pools. However, many 'who" has been the infamous college sophomore. Sears cost-saving approaches can be implemented. Second is (1986) has argued that the field's overwhelming re- the impracticality of executing elaborate scripted sce- liance on this narrow base of research -ts narios for social interaction, especially ones involving may represent a serious problem for sodal psychol- deception. Whereas these sorts of events can be ae- ogy. Pointing to various attributes that are charac- ated in labs with undergraduate participants, they are teristic of young adults, Sears (1986) suggested that tougher to do in the field. But as .we discuss shortly, the 'college sophomore" partidpant pool is unrep- many experimental procedures and manipulations can resentative of the general population in a num- be incorporated in surveys. ber of important ways. Among other things, young Put simply, social psychology can happily proceed adults are more susceptible to attitude change (Alwin, doing most of our research with college sophomores, Cohen, 6 Newcomb, 1991; Glenn, 1980; Krosnick 6 assuming that our findings generalize. And we can live Alwin, 1989; Sears, 1983), exhibit less stable person- with the skepticism of scholars from other disciplines ality traits (Caspi, Bem, 6 Elder, 1989; Costa, McCrae, who question that generalizabiity, having documented the profound impact that context and history have on that can be brought about by respondents' own be- social processes. Or we can accept the challenge and havior (e.g., misreporting true attitudes, failing to pay explore the replicabiity of our findings in the general close attention to a question), interviewer behavior population. Either we will confirm our assumptions of (e.g., misrecording responses, providing cues that lead generalizability or we will reline