Psychonomic Bulletin & Review 1998,5(2), 197-220

Selective hypothesis testing

DAVID M. SANBONMATSU and STEVEN S. POSAVAC University ofUtah, Salt Lake City, Utah FRANK R. KARDES University ofCincinnati, Cincinnati, Ohio and SUSAN p. MANTEL University ofToledo, Toledo, Ohio

A diverse set ofbiases that have been found to characterize judgment may be similarly mediated by a process ofselective hypothesis testing. Ourpaper begins with a definition ofselective hypothesis test­ ing and an explanation ofhow andwhen this process leads to error. We then review a diverse and often disconnected set offmdings in the person perception, judgment, cognition, attitudes, attribution, and rule discovery literatures thatcanbe explained by this process, Finally, we examine the question ofwhy the selective testing of hypotheses occurs. Although the psychological literature suggests that selec­ tive hypothesis testing contributes to a variety of errors, in many contexts it may be a useful and effi­ cient strategy that leads to satisfactory judgment.

The proliferation ofresearch on human judgment dur­ Although a vast body of research is reviewed that indi­ ing the last three decades has led to the identification of cates that a multitude oferrors may result from selective a multiplicity of errors that may characterize the obser­ hypothesis testing, in general this judgmental strategy is vations, interpretations, and evaluations ofeveryday life not necessarily a bad one. As we shall suggest, in many (see, e,g" Kahneman & Tversky, 1982; Nisbett & Ross, contexts the selective consideration ofhypotheses is a use­ 1980; Sherman & Corty, 1984). Every important form of ful and efficient strategy that leads to adaptive judgment. judgment-whetherit be prediction, categorization, causal attribution, covariation estimation, orattitude formation­ What Is Selective Hypothesis Testing? has been found to be susceptible to a variety ofbiases, Al­ In judgmentunder conditions ofuncertainty, an appro­ though the number and range ofjudgmental errors expli­ priate judgmental response is not readily apparent, given cated by researchers has been impressive, the development the available evidence. For our purposes, a hypothesis is of general frameworks and models for understanding a perceived possible estimation, interpretation, evaluation, these varied phenomena has not. Thus, in recent years, re­ explanation, rule, or solution. More specifically, hypothe­ searchers (e,g., Arkes, 1991; Fiedler, 1996; Klayman & ses are beliefs about the possible relation between vari­ Ha, 1987; Koehler, 1991; Kruglanski & Webster, 1996; ables, values, objects, or events. The hypothesized relation, Payne, Bettman, & Johnson, 1992) have begun making an ofcourse, may take a variety offorms. For example, a hy­ effort to develop frameworks that integrate the various pothesis may concern the correlation between variables, errors and biases that have been identified in the litera­ the causal relation between events, the favorableness or ture and that cut across different judgmental categories. characteristics ofan object, or the likelihood ofan event. What is becoming increasingly apparent to researchers The data or evidence available for judgment are often is that similar processes often underlie seemingly differ­ scattered and complex. The generation and testing ofhy­ ent categories ofjudgmental errors, In the present review, potheses facilitates judgment by guiding the gathering we suggest that many of the important biases that have and integration of information and by increasing the been identified in the cognition, person perception, attri­ speed with which a conclusion or inference can be drawn. bution, judgment, and attitudes literatures are related, in Hypothesis testing is basically a top-down approach en­ that they share a process of selective hypothesis testing, tailing selective processing that contrasts with a more data­ driven strategy in which inferences emerge from the avail­ able evidence. The authors thank Reid Hastie, Bill von Hippel, and two anonymous reviewers for making a number of excellent suggestions and asking a In a classic paper on scientific reasoning, Platt (1964) lot of hard questions. F.R.K. is in the Department of Marketing at argued that a comparison ofalternative hypotheses should Cincinnati. S.P.M. is in the Department of Marketing at Toledo. S.S.P. guide inductive inference. Strong inference begins with is now in the Department of Marketing in the Simon Graduate School the generation of plausible alternatives, followed by a ofBusiness Administration at the University of Rochester. Correspon­ dence concerning this paper should be addressed to D. M. Sanbon­ systematic gathering ofdiagnostic data that distinguishes matsu, Department of Psychology, University of Utah, Salt Lake City, the merits ofeach. According to Platt, the direct compar­ Utah 84112 (e-mail: [email protected]). ison of competing hypotheses allows for more rapid

197 Copyright 1998 Psychonomic Society, Inc. 198 SANBONMATSU, POSAVAC, KARDES, AND MANTEL development of knowledge-in particular, scientific is distinct from selective hypothesis testing in that the knowledge-than do other methods of inference. Of accepted hypothesis is the one that is perceived to be the course, Platt is not the only researcher to advocate the most plausible or preferred ofthe alternatives compared. consideration ofalternatives in human reasoning. Social Note that, in both processes, a set ofcriteria or assessment observers have long admonished people to consider the rules are applied. However, in comparative processing, the opposite or consider the alternatives before arriving at a criteria are used to compare the competing alternatives. conclusion (see, e.g, C. A. Anderson & Sechler, 1986; All judgments are explicitly or implicitly comparative. Arkes, 1991; Bacon, 1960; Hand, 1951; Hirt & Markman, However, in selective hypothesis testing, alternatives are 1995; Lord, Lepper, & Preston, 1984). not directly compared in order to arrive at a comparative These admonishments appear to be ineffectual in guid­ conclusion. Take, for example, the task ofpredicting the ing human judgment, for there exists ample evidence sug­ winner ofthe NBA championship. Comparative judgment gesting that comparative testing ofthis sort is not always would begin with the generation ofa list ofviable teams, performed. Instead ofthe identification and evaluation of followed by a systematic comparison ofthe strengths and plausible alternatives, selective hypothesis testing often oc­ weaknesses ofeach on the most relevant attribute dimen­ curs; individual hypotheses are tested serially, until one is sions. In contrast, in selective hypothesis testing, a partic­ found that meets the judgmental requirements. In selective ular team, such as the Chicago Bulls, might be nominated hypothesis testing, possibilities are assessed one at a time. and assessed. If the Bulls appear to have championship A focal hypothesis is generated on the basis ofsome crite­ qualities, the judgment that they will win is rendered. A ria or set ofstandards. The focal hypothesis, in turn, serves direct comparison of competing teams is not necessary, to guide the gathering and assimilation ofevidence. When ifa representation ofa championship team (i.e., knowledge the evidence for the focal hypothesis is sufficient or satis­ ofthe qualities ofa champion) is available to serve as the ficing (Simon, 1956, 1979), confirmation occurs, and the basis for confirmation. search for further information is truncated. Conversely, if Selective hypothesis testing sometimes is characterized the gathered evidence fails to support the hypothesis, a by the serial assessment of a multiplicity of different new alternative is generated. The process oftesting and re­ possibilities. An iterative process takes place, in which generating continues until a suitable response is found or focal hypotheses are repeatedly discarded and progres­ until the perceived benefits of gathering additional evi­ sively refined until a response is generated that satisfac­ dence are outweighed by the costs (Gettys & Fisher, 1979). torily accounts for the gathered evidence. On occasion, this In selective hypothesis testing, the generation ofalter­ process circles around to a reconsideration of the origi­ natives, the gathering of evidence, and inferences about nal hypothesis. In other instances, the initial hypothesis the meaning ofthe evidence are guided by a set ofcriteria, is recognized to be a mere starting point, from which the conditional rules, or judgmental standards. A necessary generation, testing, and revision process begins. This sort component of the criteria is a confirmation threshold, ofiterative process is especially likely to operate when a which amounts to the evidential requirements for con­ quantitative estimation is sought. Take the example ofa firmation. In selective hypothesis testing, the rejection or task that calls for the approximate age of a junior male acceptance ofa hypothesis is determined by whether the colleague. Selective hypothesis testing may begin with gathered evidence meets the confirmation criteria. the hypothesis that the colleague is around 30 but be ad­ In contrast, in comparativejudgment, a multiplicity of justed significantly upward to 45 with knowledge ofhis hypotheses are generated and then compared with one three children, then revised slightly downward with con­ another. A hypothesis must fulfill two requirements to sideration of his full head of hair, and so forth, until a be accepted: It must meet some minimum standard ofac­ satisfactory response is generated. Often the test hypoth­ ceptability, and it must be perceived to be superior to the esis serves as an anchor from which adjustments are generated alternatives. The decision-making literature sug­ made and new hypotheses are generated. gests that the comparison process may take a variety of Selective hypothesis testing is not necessarily charac­ forms (see, e.g., Abelson & Levi, 1985; Beach & Mitchell, terized by a complete neglect ofthe judgmental alterna­ 1978; Bettman, 1979; Einhorn & Hogarth, 1981; Payne tives. When the criteria are apt and evidence pertaining to et aI., 1992; Russo & Dosher, 1983; Siovic, Lichtenstein, the judgment is amply available, several alternatives may & Fischhoff, 1985). In some instances, holistic process­ be considered and rejected before one is accepted or ver­ ing may take place, in which an overall evaluation or as­ ified. Conversely, comparative judgment is not necessarily sessment ofeach alternative is first formed, followed by characterized by the consideration of all of the relevant a comparison of the overall assessments. Alternatively, alternatives. On occasion, the set of alternatives gener­ dimensional processing may occur, in which the alterna­ ated for comparison fails to include the most optimal tives are compared on a feature by feature basis. The spe­ judgmental response. Nevertheless, selective hypothesis cific comparison process that occurs is determined by testing is much more apt to be characterized by a neglect the judgmental context, the type of information that is of the relevant alternatives than is comparative judg­ available, processing motivations, and the specific nature ment. In the latter form ofprocessing, an attempt is made ofthejudgmental task (e.g., attribution vs. evaluation). Re­ to generate the most plausible alternatives. In contrast, in gardless of the specific process, comparative judgment selective hypothesis testing, alternatives are tested one at SELECTIVE HYPOTHESIS TESTING 199 a time. Because ofa confirmation bias, often little or no ing one stage (e.g., inference), whereas comparisons with consideration is given to any possibility except the initially alternatives may occur during another. generated focal hypothesis. Thus, the neglect ofalterna­ tives and the failure to examine evidence for alternatives Is Selective Hypothesis Testing New? generally reflects selective as opposed to comparative pro­ The obvious answer to this is no. Selective hypothesis cessing. testing is similar to the satisfiCing process delineated by The explicit goal of many judgments is to test a spe­ Simon (1956) in his seminal analysis ofdecision making. cific hypothesis. That is, interest is in whether a particular Moreover, a multitude ofresearchers have suggested that alternative is optimal or correct, as opposed to which al­ hypotheses are selectively tested with little consideration ternative is optimal or correct. For example, people are of the alternatives in judgment (see, e.g., Greenwald, often concerned with whether a specific product is the Pratkanis, Leippe, & Baumgardner, 1986; Klayman & Ha, best (e.g., Does Kodak manufacture the best film?), as 1987; Snyder & Swann, 1978). For example, Kahneman opposed to which product is best (e.g., what company and Tversky (1973) theorized that hypotheses may be as­ manufactures the best film?); whether a particular event sessed singularly on the basis oftheir representativeness. will occur (e.g., Will the 4gers win the Super Bowl?), as Van Wallendael and Hastie (1990) provided evidence opposed to what event will occur (e.g., Which team will that hypotheses may be represented, treated, and tested as win the Super Bowl?); and whether a specific factor independent of alternative hypotheses. More recently, caused an outcome (e.g., Is the manager responsible for Trope and Liberman (1996) postulated that hypotheses are the store's dwindling profits?), as opposed to what caused often tested in a pseudodiagnostic manner, without con­ the outcome (e.g., What is responsible for the store's di­ sideration ofthe alternatives. minished profits?). The testing ofa singular hypothesis Nevertheless, our framework is unique in emphasizing may be either selective or comparative. Although the the noncomparative nature of many comparative judg­ task is implicitly comparative, the evidence for a focal ments and in distinguishing selective hypothesis testing hypothesis may be selectively assessed on the basis ofa from comparative assessment and the specific compari­ set of standards. For example, judgment of whether a son processes that have been delineated by decision re­ manager is responsible for a store's losses may be based search. We identify the alternative to selective hypothe­ entirely on an evaluation of the competency that the sis testing as comparison rules, such as the lexicographic manager's decisions and actions demonstrate. Alterna­ and weighted average-based strategies (for a review, see tively, the test hypothesis may be assessed through a com­ Bettman, Johnson, & Payne, 1991). Our approach also parison with relevant options. In judging the culpability emphasizes the important role that the judgmental criteria of the manager, for example, managerial performance or standards play in the selective testing of hypotheses. may be assessed relative to other possible contributors, Hypotheses may be assessed on the basis ofa set ofstan­ such as a diminished demand for the store's products or dards, rather than through a comparison with specific al­ a general economic decline. ternatives. Often clear and well-calibrated judgmental Our selective versus comparative hypothesis testing standards are developed that facilitate the reliable and framework is algorithmic or representational, as opposed commensurate assessment of select hypotheses. The to computational or implementational (Marr, 1982). It is very reason that these standards are developed is to avoid representative ofthe processes through which evidential the necessity of time-consuming and effortful compar­ inputs are transformed into judgmental outputs and is isons with specific alternative hypotheses. Finally, we thought to underlie a variety ofspecific computational tasks conceptualize selective hypothesis testing as a serial pro­ orproblems, ranging from causal attribution to evaluation. cess that may be characterized by the sequential testing The framework is not implementational, in that it does of a multiplicity of focal hypotheses. The emphasis of not model the neural architecture underlying cognition. selective hypothesis testing as a serial process differs The distinction between comparative processing and from that ofmost hypothesis testing studies, which have selective hypothesis testing is idealized. Obviously, the tended to focus on a single slice of the process-a slice testing of hypotheses is not always strictly selective or in which a single focal hypothesis is tested and in which comparative. As in other dual-process models (e.g., the the shortcomings ofselective as opposed to comparative Elaboration Likelihood Model [Petty & Cacioppo, 1986]; testing are most apparent. This broader conception ofse­ the Heuristic-Systematic Model [Chaiken, Liberman, & lective hypothesis testing has given us a unique appreci­ Eagly, 1989]; the MODE Model [Fazio, 1990]), process­ ation for the usefulness ofthis strategy. ing is usually mixed, involving both testing ofa focal hy­ In general, the scope of most discussions of the un­ pothesis and limited comparison with an alternative or derlying processes and effects ofhypothesis testing have alternatives, particularly the complement that the focal been narrow, and there has been little in the way ofinte­ hypothesis is wrong. In a selective test of a hypothesis, grative frameworks (for important exceptions, see Fis­ however, the primary consideration is whether the evi­ chhoff & Beyth-Marom, 1983; Klayman & Ha, 1987; dence for the focal hypothesis fulfills a set of standards Kruglanski & Webster, 1996; Pyszczynski & Greenberg, or criteria. Trope and Liberman (1996) point out that, on 1987; Trope & Liberman, 1996). Researchers have tended occasion, the testing ofa hypothesis may be selective dur- to limit their discussions ofthe normality or rationality of 200 SANBONMATSU, POSAVAC, KARDES, AND MANTEL hypothesis testing to specific judgmental domains and to e.g., Hastie & Kumar, 1979; Rothbart, Evans, & Fulero, specific stages ofjudgment (e.g., search). The present 1979; Srull, 1981; for a review, see Stangor & McMillan, article attempts to organize a diverse and disconnected 1992). However, this selective search leads to error when set ofjudgmental findings into a single framework. Se­ a hypothesis is insufficiently general or overly specific lective hypothesis testing is conceptualized as a general (Klayman & Ha, 1987), because disconfirming instances strategy or process that affects many different categories that fall outside of the assumed category or rule are not ofjudgment, including prediction, causal attribution, co­ examined. This was the error observed in Wason's (1960) variation estimation, concept identification, trait infer­ well-known rule discovery experiments. In one experi­ ence, and evaluation. Finally, the paper attempts to provide ment, the subjects attempted to identify the rule (increas­ some general insight into the questions of when selec­ ing numbers) underlying a set ofthree numbers, or triples. tive hypothesis testing leads to good judgment and why The subjects were provided a triple that was an instance selective hypothesis testing occurs. Admittedly, much of of the rule (2,4,6) and were allowed to generate addi­ this discussion is speculative, as these issues have received tional triples to test their hypothesis. Wason observed scant attention in the experimental literature. However, that many ofthe subjects quickly generated overly narrow our analysis ofwhen and why is useful, in that it helps to hypotheses (e.g., consecutive even numbers), which they illuminate the functional nature of selective hypothesis attempted to test with a positive search. These subjects testing and may serve to guide future research. often became unjustifiably confident oftheir hypotheses, because they looked predominantly at triples (e.g., When Does Selective 8,10,12) that fit their hypothesized rule. They failed to Hypothesis Testing Lead to Error? recognize that the actual hypothesis was more general, as Selective hypothesis testing sometimes leads to error they rarely generated test cases (e.g., 1,2,3) that would because the best response is never considered. Instead, disconfirm their narrow view (see Klayman & Ha, 1987, an incorrect or suboptimal hypothesis is prematurely ac­ for an extensive discussion ofwhy the default strategy is cepted because ofbiases in the evaluation process. In the often a positive search). Basically, error occurs when there selective testing ofa hypothesis, alternatives are ignored. is supportive evidence for alternative hypotheses, as well Evidence for the focal hypothesis is gathered and inter­ as for the focal hypothesis. preted and then assessed on the basis of a set of criteria Selective testing sometimes contributes to biased con­ or standards. Unfortunately, shortcomings may charac­ ceptions of the focal hypothesis. According to Gilbert terize each of these stages of processing (for earlier re­ (1991), the very act ofunderstanding or comprehending views, see Fischhoff& Beyth-Marom, 1983; Pyszczynski a proposition entails believing it. Thus, simply compre­ & Greenberg, 1987; Trope & Liberman, 1996). In general, hending a focal hypothesis may increase its perceived selective testing is biased toward confirmation of the plausibility. In some instances, people assume the validity focal hypothesis. Although people do not necessarily in­ of hypotheses that they are prompted to test (Evett, terpret or search for evidence with the intention to confirm, Devine, Hirt, & Price, 1994; see also Gettys et aI., 1986). information often is processed in a manner that con­ They infer that there is a good reason for testing a hy­ tributes to an overestimation ofthe plausibility ofthe hy­ pothesis, even if it is selected arbitrarily. This inference pothesis being tested. may be drawn, in part, because ofthe cooperative rules Bias may enter first during the search for evidence that are assumed to govern social discourse (see, e.g., De­ (Klayman & Ha, 1987; Pyszczynski & Greenberg, 1987; lany & Hilton, 1991; Grice, 1975; Strack, Schwarz, & Snyder, 1981). In selective hypothesis testing, evidence Wanke, 1991). Additional research suggests that, during is sought that is expected to be true given the hypothesis, selective testing, the possibilities or components associ­ particularly instances in which the hypothesized outcome ated with the focal hypothesis are explicated or unpacked, or property is expected to be present (see, e.g., Borgida thus heightening recognition ofthe scope ofthe support & DeBono, 1989; Devine, Hirt, & Gehrke, 1990; Klayman for the focal hypothesis. In contrast, the scope ofthe al­ & Ha, 1989; Skov & Sherman, 1986; Trope & Mackie, ternative hypotheses and the associated evidence are not 1987, Experiment 3; Wason, 1960, 1968). Neglected dur­ fully realized or considered (Tversky & Koehler, 1994). ing the gathering ofinformation is evidence for alterna­ Finally, a related process contributing to confirmation is tive hypotheses (see, e.g., Gettys, Mehle, & Fisher, 1986; the expansion ofthe boundaries ofthe hypothesis. Often, Robinson & Hastie, 1985; Mehle, Gettys, Manning, Baca, the boundaries or scope of the hypothesis are redefined & Fisher, 1981; Van Wallendael & Hastie, 1990). This so as to be more inclusive. Consequently, evidence or data search strategy does not preclude the falsification of a are more likely to be perceived as consistent with the hy­ hypothesis, as instances are often discovered that do not pothesis. This process is suggested by studies ofthe usage hold true in the expected conditions (Fischhoff& Beyth­ of fault trees in problem solving, which have found that Marom, 1983; Klayman & Ha, 1987; Mynatt, Doherty, judges are insensitive to the omission of branches, or & Tweney, 1977). In fact, numerous studies indicate that fault categories (Fischhoff, Siovic, & Lichtenstein, 1978; information that is inconsistent with an expectation or hy­ Hirt & Castellan, 1988; Russo & Kolzow, 1994). When pothesis often is more heavily attended to and better re­ judges are presented with fewer alternatives in a fault membered than consistent or irrelevant information (see, tree, they attribute greater probability to the available or SELECTIVE HYPOTHESIS TESTING 201

remaining branches, rather than assigning greater respon­ Trope and Liberman (1996; see also Klar, 1990) point sibility to the nonpresent or all other problems branch. out that an implicit assumption underlying selective hy­ Judges are often insensitive to omissions, because they re­ pothesis testing (what they call pseudodiagnostic testing) define the available causal categories or branches more is that evidence consistent with the focal hypothesis is broadly (Hirt & Castellan, 1988). That is, the remaining inconsistent with alternative hypotheses. Often, this is a categories are expanded to be inclusive ofa greater num­ warranted assumption. An inability to solve simple arith­ ber of specific problems. The expansion of the bound­ metic problems is evidence consistent with the hypothe­ aries ofa focal hypothesis, ofcourse, is less likely to occur sis that a person has poor quantitative skills and inconsis­ ifthere is delineation ofthe alternatives (Dube-Rioux & tent with the alternative that the person is a mathematical Russo, 1988). genius. However, in other instances, the standards or cri­ In selective hypothesis testing, the test hypothesis teria that are used in testing a hypothesis are poorly devel­ serves to guide interpretations ofthe gathered evidence, oped or inappropriate for the context. As a consequence, as well as assessments of the validity of the evidence the evidence that is used in assessing the validity ofthe (see, e.g., Griffin & Ross, 1991). Unfortunately, the focal hypothesis is nondiagnostic. hypothesis sometimes biases the assimilation of infor­ Although selective hypothesis testing is characterized mation. Often, ambiguous evidence is perceived in a man­ by various processing biases, these biases do not always ner that is consistent with the focal hypothesis (see, e.g., contribute to confirmation. Obviously, much depends on Darley & Gross, 1983). Moreover, evidence that is in­ the nature ofthe evidence that is available in memory or consistent with the hypothesis is selectively challenged, the judgmental context. Confirmation occurs only ifthe discounted, or ignored (see, e.g., Ditto & Lopez, 1992; salient or accessible evidence can be gathered and inter­ Edwards & Smith, 1996; Gilovich, 1983; Lord, Ross, & preted in a way that supports the test hypothesis (see Lepper, 1979; Tweney & Doherty, 1983). Bias may also Lehman, Krosnick, West, & Li, 1992). When the evidence operate in the inferences that are drawn about the validity for all of the alternatives is weak, selective hypothesis ofthe hypothesis from the evidence. In selective testing, testing may actually diminish the likelihood of confir­ inferences are based primarily on the extent to which the mation. In this instance, perceivers learn ofthe marginal gathered information supports the focal hypothesis. Little support for the focal hypothesis. Unaware ofthe similar consideration is given to the extent to which alternative lack ofsupport for the alternatives, they sometimes pre­ hypotheses are also supported by the same evidence. In maturely dismiss the focal hypothesis (Sanbonmatsu, some instances, evidence is mistakenly believed to verify Posavac, & Stasney, 1997). The unwarranted rejection of the test proposition when, in fact, it is nondiagnostic or a select hypothesis may also occur because of a confir­ pseudodiagnostic information that also implicates alter­ mation threshold that is set too high. natives (Beyth-Marom & Fischhoff, 1983; Doherty, My­ Hypothesis testing does not occur in a motivational natt, Tweney, & Schiavo, 1979; Snyder & Swann, 1978). vacuum; wants and desires undoubtedly playa crucial The validity of selective hypothesis testing is heavily role in the assessment process (see Ditto & Lopez, 1992; dependent on the criteria that are used to assess the evi­ Kunda, 1990; Pyszczynski & Greenberg, 1987; Trope & dence. Sometimes error results from a reliance on fuzzy Liberman, 1996). Confirmation may be particularly likely test criteria; spurious inferences are drawn about the hy­ to occur when the hypothesis being assessed is one that pothesis, because ofthe lack ofa well developed, appro­ is preferred or one that the perceiver hopes is true. Here priate, and accessible set ofstandards for establishing cer­ selective, as opposed to comparative, processing is more tainty. On occasion, a suboptimal hypothesis is confirmed, likely to operate. Moreover, the selective testing of the because the standards are adjusted to fit the focal hy­ hypothesis is especially apt to be biased: The boundaries pothesis. For example, a professor might be asked to test of the hypothesis are more likely to be expanded, the the hypothesis that a new PhD is an excellent job candi­ confirmation threshold is more apt to be lowered, and in­ date. Ifthe professor lacks a clear sense ofwhat attributes formation is more likely to be assimilated in a way that are crucial to the judgment, the salient attributes ofthe contributes to confirmation of the desired conclusion. candidate may be perceived to be the most important ones, When extremely strong motivational pressures are oper­ leading to a premature endorsement. Consistent with this ating, poor judgment sometimes results, not because of notion, recent research indicates that the evidence for a the absence ofa sound strategy with which to test one's focal hypothesis or focal target is, indeed, sometimes over­ hypothesis, but from an unwillingness to put one's hy­ weighed (Sanbonmatsu, Kardes, Houghton, Ho, & Posa­ pothesis to the test. In these instances, judgment may be vac, 1998). In yet other instances, the unwarranted veri­ more a process ofjustifying motivated beliefs than of fication ofa hypothesis occurs because the confirmation testing hypotheses (see Baumeister & Newman, 1993; criteria are set too low. For example, a job candidate may Kunda, 1990). be erroneously perceived to be superior and deserving of an immediate job offer because ofthe usage ofoutdated Which Hypotheses Are Selectively Tested? standards that were valid during a period when the com­ Hypothesis generation may be guided by a variety of petition was less stiff. factors, including task demands and motivation, contex- 202 SANBONMATSU, POSAVAC, KARDES, AND MANTEL tual cues, and prior knowledge. Prior beliefs or experi­ The increased plausibility from merely understanding a ences often contribute to the generation of a particular hypothesis may affect how the available evidence is pro­ hypothesis in a heavily top-down manner. Because ofre­ cessed and which criteria are used in assessment. cent or chronic activation, a hypothesis may be highly Motivation undoubtedly plays a crucial role in deter­ accessible in memory (see, e.g., Bargh, Bond, Lombardi, mining which hypothesis is tested. Epistemic needs and & Tota, 1986; Higgins, Rholes, & Jones, 1977; Srull & the potential informativeness of hypotheses guide the Wyer, 1979) and thus be the first retrieved during a search generation and testing ofpropositions (see Klayman & Ha, for a possible response. A hypothesis may also be inferred 1987; Kruglanski, 1989; Kruglanski & Webster, 1996; on the basis of prior beliefs or expectancies. For exam­ Kunda, 1990; Trope & Liberman, 1996). Hypotheses ple, in social perception, a particular explanation ofbe­ that are relevant to selfor significant others are particularly havior may be considered because ofa stereotype that is likely to be examined. Job applicants, for example, strive held about a group ofwhich the actor is a member. Focal to calculate their chances (as opposed to others' chances) hypotheses that are inferred from prior beliefs are espe­ of being hired. Here, again, the primary concern is not cially likely to be confirmed during the test phase, because which hypothesis is true, but the likelihood that a spe­ they are often initially perceived to be highly credible or cific hypothesis is true. Gamblers, similarly, are interested viable. As a consequence, the likelihood that alternatives in whether they won the lottery, not who won the lottery. are considered and the standards for confirmation may Because hypotheses that people want to be true are es­ be low. Moreover, cognitions related to the prior beliefs pecially likely to be generated and tested, they are more or expectations are frequently available in memory to apt to be confirmed. Conversely, hypotheses that people support the test hypothesis. hope are not true are less apt to be selectively tested and In some instances, a hypothesis is suggested by the ev­ subsequently confirmed. For example, parents attempting idence that is initially encountered. Consequently, a hy­ to explain their teenagers' poor school performance may pothesis may form on-line during the initial processing be reluctant to consider the possibility that their child is of information (Asch, 1946; Hastie & Park, 1986). The intellectually handicapped or preoccupied by drugs and, newly formed hypothesis serves to guide the search for, thus, may be among the last to recognize these problems. and construal of, additional evidence. As we discussed earlier, often, selective hypothesis test­ Hypotheses, however, are not always self-generated. ing is a highly iterative process. Frequently, focal hypothe­ Often specific interpretations or choice options are sug­ ses are derivations ofpreviously disconfirmed hypothe­ gested and actively promoted by others. In group discus­ ses. Rather than starting all over, the available evidence sions and advertisements, for example, communicators is used to refine or adjust old hypotheses and to generate often vigorously argue the merits ofa particular position a revision. Unfortunately, the adjustments or corrections or behavior and implore us to consider it in ourjudgments that are made are not always sufficient (Einhorn & Hog­ and decisions. Hypotheses promoted by others can be arth, 1985; Gilbert, 1989; Tversky & Kahneman, 1974). highly susceptible to confirmation, because supportive ev­ Revisions that are suboptimal are sometimes generated and idence is frequently made salient during the communi­ confirmed because ofbiased processing and because of cation. Moreover, in instances where the communicator is the select evidence that was accessed previously in the test­ highly credible, little evidence may be needed for accep­ ing ofthe original focal hypothesis (Strack & Mussweiler, tance ofthe hypothesis (see, e.g., Chaiken & Maheswaran, 1997). Thus, even if a particular hypothesis is not ac­ 1992; Norman, 1976; Petty & Cacioppo, 1979). cepted, it may serve as an anchor that biases the genera­ In many settings, strong cues are present that prompt tion and testing ofnew hypotheses. the consideration ofa hypothesis. Sometimes a specific Finally, people are inclined to investigate hypotheses alternative is highly salient in the judgmental context. For that are testable, as opposed to those that are not (see, e.g., example, a particular restaurant may have a prominent Klayman & Ha, 1987). This certainly is true not only of place in the perceptual field ofa prospective diner. Simi­ laypersons but ofscientists, whose questions and hypothe­ larly, the application ofa viable job candidate may happen ses are commonly constrained by the limitations oftheir to be at the front of a stack of files. In other instances, laboratories and equipment. hypotheses are suggested subtly through framing. In fram­ In sum, some hypotheses are more likely than others to ing, a response option is described or presented in a way be selectively generated and tested. Hypotheses that are that cues a particular assessment or interpretation (see, accessible, cued or framed, suggested by others, motiva­ e.g., Levin & Gaeth, 1988; Tversky & Kahneman, 1981). tionally relevant, testable, generated on the basis ofprior Interestingly, even hypotheses that are not based on experience or knowledge, and derived from discarded prior expectations are susceptible to the various biases in hypotheses are especially apt to be considered. These hy­ processing that were reviewed earlier. Thus, hypotheses potheses, of course, are the ones most likely to be con­ that are suggested by framing, that are salient in the con­ firmed. As such, they are the most likely to be falsely text, and that people are prompted or instructed to test also confirmed. Again, selective hypothesis testing sometimes tend to be confirmed. This tendency to accept the given amounts to a first-come, first-confirmed process; the first hypotheses may be exacerbated because ofour propensity viable possibility generated has an enormous compara­ to believe that which we comprehend (Gilbert, 1991). tive advantage. Although the evidence for competing hy- SELECTIVE HYPOTHESIS TESTING 203 potheses may be equally or even more compelling, the generally is more expedient than comparative testing, on initially tested hypothesis may be prematurely accepted occasion, focusing on an erroneous hypothesis delays because ofbiased search and assessment processes or in­ the consideration ofa more optimal response. appropriate criteria. Researchers have implicitly or explicitly suggested It should be apparent from our discussion that hy­ that selective hypothesis testing underlies many of the potheses are not all the same. Hypotheses vary markedly reviewed phenomena. A variety ofdifferent experimental in the processes through which they are identified or strategies have been used to demonstrate this process. In generated. Perhaps more significantly, hypotheses differ general, a process ofselective hypothesis testing is sug­ greatly in terms of their hedonic value or motivational gested by three different patterns of experimental out­ significance. Finally, hypotheses vary considerably in the comes in the literature. degree ofcertainty with which they are held. All ofthese 1. In many studies, there is a presumed correct or accu­ factors shape the manner in which relevant evidence is rate judgmental response. A process ofselective hypoth­ processed, the criteria that are used, the degree of con­ esis testing is suggested by the finding that responses are sideration given to alternatives, and, thus, the likelihood more accurate when subjects are prompted to consider ofthe verification of the focal hypothesis (see Trope & viable alternatives. Liberman, 1996). When an initial hypothesis is tentatively 2. In other studies, subjects are prompted or influenced held, competing hypotheses are more apt to be considered, to test one of a number of possible hypotheses or judg­ and greater evidence is needed for confirmation (see mental responses. Selective hypothesis testing is suggested Kruglanski, 1989; Kruglanski & Webster, 1996; Trope, by acceptance ofthe test hypothesis or an increased like­ Bassok, & Alon, 1984). lihood ofthe focal response. 3. Finally, studies use thought listing, selective atten­ The Consequences of tion, or recall measures to provide more direct evidence Selective Hypothesis Testing for selective hypothesis testing. Typically, researchers at­ Selective hypothesis testing is a highly pervasive strat­ tempt to demonstrate that subjects focus on evidence for egy that is used in a wide range ofjudgments. We suggest the test hypothesis and neglect evidence for alternatives that a diverse set ofeffects that have been demonstrated or that subjects simply fail to consider alternatives. in the cognition, attribution, person perception, rule dis­ As we suggested earlier, selective hypothesis testing covery, and judgment literatures result from the selective tends to lead to error in an interrelated set ofconditions: assessment ofhypotheses. In the following sections, we when there are many viable alternatives, the evidence is identify a variety ofjudgmental phenomena and explain subject to selection and interpretation, the hypothesis is how each may be an instance ofselective hypothesis test­ ambiguous, or the confirmation criteria are inappropriate ing. We also review evidence suggesting that the testing of or fuzzy. Many ofthe errors or shortcomings that are re­ select hypotheses mediates the effect. Note that this is not viewed occur because the judgmental task given to sub­ intended to be a comprehensive review ofthe literature jects is characterized by the exact conditions in which on any particular judgmental phenomenon; it is only in­ selective hypothesis testing is fallible. That is, the judg­ tended to identify important tendencies that are mediated mental conditions are those in which selective hypothesis by a process of selective hypothesis testing. It should testing (and, admittedly, many other processes) leads to also be recognized that selective hypothesis testing is not error. First, the typical judgment in the reviewed studies the sole mediator ofmost ofthese judgmental effects; typ­ is characterized by a high degree ofuncertainty. The task ically, for many ofthe more robust phenomena, such as is generally a difficult one in which multiple hypotheses the hindsight bias and the overestimation ofprobability, are plausible and suboptimal hypotheses are supported by there are additional contributing processes. a subset of the available or presented evidence. In the In general, there are a number ofnegative judgmental classic rule discovery task used by Wason (1960), for ex­ consequences that may ensue from selective hypothesis ample, a multiplicity ofviable suboptimal rules could be testing. In some instances, selective hypothesis testing generated from the given case. Similarly, in many demon­ leads to judgment that is suboptimal or just plain wrong. strations of the overestimation of causality (e.g., San­ Individuals prematurely accept a selectively examined bonmatsu, Akimoto, & Biggs, 1993) and probability (e.g., possibility, without making use ofall ofthe relevant ev­ Sanbonmatsu et aI., 1997), judgmental tasks were delib­ idence and without considering superior alternatives. Se­ erately chosen in which a number of different explana­ lective hypothesis testing also contributes to enormous tions orpredictions were plausible. Second, in many tasks, variability in judgments both between- and within-persons, evidence is presented in a way that allows selectivity to even when the data base is fixed. Although the evidence occur. Individuals are required to retrieve evidence from present in a judgmental context may be the same (between­ memory (as in many causal inference, evaluation, and or within-persons), variedjudgments may result because probability estimation tasks), generate the evidence (as in different hypotheses are selectively tested and con­ rule discovery tasks), or wade through a morass ofstim­ firmed. Finally, the testing ofa select hypothesis may in­ uli (as in some covariation estimation tasks). Many judg­ crease the time and effort necessary to arrive at an satis­ mental errors would not occur if individuals were the factory conclusion or solution. Although selective testing passive recipients ofall ofthe relevant evidence or ifthe 204 SANBONMATSU, POSAVAC, KARDES, AND MANTEL information were structured for subjects in a coherent comes are treated as select hypotheses. In their study, ad­ manner (the exception here are some covariation esti­ vanced business students were told that a complex busi­ mation tasks, where errors occur even when all ofthe in­ ness decision resulted in a successful outcome, were not formation is presented or structured in some summary given outcome information, or were asked to assume that format). In other instances, the presented evidence is am­ the outcome was successful. The subjects then read a biguous or pseudodiagnostic and, thus, subject to different five-page brief on the decision and were told to analyze interpretations or inferences (as in many categorization factors that might have lead to success or failure. Later, tasks). Finally, most ofthe reviewedjudgmental tasks are the subjects were asked to ignore outcome information and ones in which people lack a well-learned algorithm or (I) estimate the probability of success for the decision, judgmental procedure. Errors result from a reliance on (2) indicate whether they would have approved the deci­ inappropriate standards. Probability overestimation, for sion, (3) judge how risky the decision was, and (4) indi­ example, sometimes occurs because people do not use a cate whether they would have invested in the company fol­ confirmation criterion that is high enough, given the lowing the decision. Control subjects, who were not strong alternative possibilities that exist. In other in­ provided with an outcome, did not exhibit the hindsight stances, "error" occurs because the subjects' confirmation bias. However, both subjects who were told that the out­ criterion is less stringent or different from that ofthe ex­ come was successful and subjects who were asked to as­ perimenter. In particular, the usage ofsufficiency, as op­ sume that the outcome was successful displayed bias on posed to necessity, criteria may contribute to the "errors" all measures. Thus, the outcome did not actually have to observed in many experiments (see, e.g., Lewicka, 1988). occur for the hindsight bias to operate. Instead, the sub­ jects merely had to hypothesize or assume that an out­ Probability Estimation and Prediction come occurred to overestimate its likelihood. Estimations of the probabilities of uncertain out­ Probability overestimation. One ofthe most pervasive comes, predictions offuture events, andjudgments about errors characterizing intuitive prediction is the numerical the foreseeability ofpast occurrences have been found to overestimation of the probability ofa focal hypothetical be susceptible to various forms oferror. This section re­ event. In an important but somewhat neglected series of views several shortcomings characterizing the estimation experiments, Teigen (1974a, 1974b, 1983) demonstrated ofprobability that may result from selective hypothesis that people commonly violate the fundamental conven­ testing. tion ofprobability theory-the convention that the prob­ Hindsight bias. People tend to overestimate the fore­ abilities assigned to an exhaustive set ofmutually exclusive seeability of past events when they know the outcome. events add up to one. For example, in one study (Teigen, Fischhoff(l975), for example, showed that subjects who 1974b; Experiment 2), the subjects were asked to esti­ received outcome information about obscure historical mate the numerical probability that a randomly selected events estimated that the outcome was more probable person would fall within one of six different height cat­ than did subjects who did not receive such information. egories. The mean perceived likelihood ofeach ofthe six Recent reviews (Christensen-Szalanski & Willham, 1991; heights was about 0.43, for a total probability of2.6. Thus, Hawkins & Hastie, 1990) conclude that hindsight effects the estimated probabilities of the hypothetical events result more from cognitive than from motivational factors were much higher than one. The overestimation ofprob­ (see, e.g., Campbell & Tesser, 1983). Hawkins and Hastie ability ofa hypothetical event has since been demonstrated (1990) suggest that hindsight effects occur, in part, be­ in a number ofstudies (see, e.g., Sherman, McMullen, & cause people rejudge evidence that is relevant to the judg­ Gavanski, 1992; Tversky & Koehler, 1994). ment once the outcome is known, and outcome knowl­ Studies by Sanbonmatsu et al. (1997, Experiments 1 edge guides the rejudgment process. Specifically, people and 2) suggest that the dramatic overestimations of the resample and reevaluate evidence from memory in a way numerical probability ofa focal event that have been ob­ that explains or justifies whatever occurred. Knowledge served in the literature result, in part, from a process of ofthe actual outcome appears to irrevocably alter the cog­ selective hypothesis testing. Subjects were asked to esti­ nitive representation ofthe event, thus contributing to an mate the likelihood that a randomly selected candidate overestimation ofthe perceived a priori likelihood ofthe from a pool offour was hired for a faculty position. Lists outcome (Fischhoff, 1977). of statements describing the favorable credentials and Cognitive explanations ofthe hindsight bias fit easily behavior of each of the four candidates were provided. into a selective hypothesis-testing framework. After an As expected, probability estimations ofthe focal candi­ outcome is learned or known, it serves as a select hypoth­ date's chances were significantly inflated. The subjects esis that guides the processing ofevidence relevant to the were subsequently given a surprise recall task in which judgment. Ignored in this procedure are outcomes that they were asked to list as many statements about all ofthe could have, but did not, occur. The evidence is rejudged in candidates as possible. They recalled more of the state­ a way that explains the outcome but not nonoccurrences, ments about the target candidate than about the others, thus contributing to an overestimation ofthe likelihood suggesting that they focused heavily on information ofthe actual outcome. Connolly and Bukszar (1990) re­ about the target candidate, even though the remaining in­ port data that are consistent with the idea that actual out- formation was equally relevant in arriving at a probability SELECTIVE HYPOTHESIS TESTING 205

estimate. Selective recall was strongly correlated with the they answered questions correctly, even when their an­ probability estimations; as the tendency to selectively swers were wrong (see also Lichtenstein, Fischhoff, & recall statements about, and presumably focus on, the test Phillips, 1982). Overconfidence is a robust phenomena candidate increased, the tendency to overestimate the that has been observed in clinical and medical diagnosis test candidate's chances increased. Thus, it appears that (Dawes, 1994; Eddy, 1982; Oskamp, 1982), answers to the overestimation of probability stemmed from the se­ general knowledge questions (Fischhoff, 1982; Fischhoff lective examination ofevidence for the focal candidate. et aI., 1977), and predictions about social and nonsocial Additional evidence for selective hypothesis testing in events (see, e.g., Dunning, Griffin, Milojkovic, & Ross, probability estimation is provided by demonstrations of 1990; Griffin & Tversky, 1992; Vallone, Griffin, Lin, & noncomplementary opinion revision. In a series ofmurder Ross, 1990). Moreover, overconfidence characterizes the mystery experiments, Van Wallendael and Hastie (Robin­ professionaljudgments ofphysicians, clinical psycholo­ son & Hastie, 1985; Van Wallendael & Hastie, 1990) gists, lawyers, and security analysts (Griffin & Tversky, found that clues affecting the perceived probability ofone 1992), as well as the everyday judgments oflaypersons. target suspect had little or no effect on the perceived prob­ Many researchers have suggested that overconfidence ability ofcompeting suspects. Thus, the subjective likeli­ stems from processes that are akin to selective hypothe­ hood ofthe suspects did not change in a complementary sis testing. Koehler (1991), for example, suggests that, fashion; instead, the assessment ofeach focal hypothesis when people assess the confidence with which they hold was relatively independent ofassessments ofcompeting a belief, they temporarily assume that the belief is true hypotheses. and search for evidence to support it. The selective gath­ Tversky and Koehler's (1994) support theory provides ering ofsupportive evidence and the neglect ofviable al­ an account ofprobability overestimation that is similar to ternatives (see, e.g., Hoch, 1985; Koriat, Lichtenstein, & the one presented here. According to support theory, prob­ Fischhoff, 1980) invariably leads people to overestimate ability overestimation, or subadditivity, results from the the likelihood that their hypothesis is true. In keeping with unpacking ofthe focal hypothesis and the failure to unpack this explanation, a number ofstudies have demonstrated the alternative hypothesis. Individuals recognize the scope that examining or explaining alternatives reduces confi­ ofand support for the focal hypothesis but not for the al­ dence in an initially held belief. Hoch, for example, ob­ ternative. This is similar to the current framework, which served that graduating college seniors generally were postulates that probability overestimations result from the overconfident in their prospects for future success on the selective consideration of the evidence for the focal hy­ job market. Seniors who were asked to generate reasons pothesis. Nevertheless, there are important differences be­ for why they might not be successful, however, were less tween the frameworks. Although the exact mechanisms overconfident than others who did not engage in such a underlying support theory's predictions are not entirely generation task. clear, the model appears to assume that comparisons are Overconfidence may be more pervasive than is gener­ made in all instances. The alternative that is compared to ally conveyed by the judgment and decision-making lit­ the focal hypothesis simply varies in the degree to which erature (for contrasting views, see Dawes & Mulford, it is unpacked. This contrasts with selective hypothesis 1996; Gigerenzer, 1991). Research on overconfidence testing, in which comparisons between the evidence for indicates that people tend to overestimate the likelihood the focal hypothesis and the evidence for the alternatives that their hypothesis or conclusion is true. However, work do not occur. The present framework also does not predict on probability estimation and causal inference (see, e.g., binary complementarity in all cases. We suggest that, Sanbonmatsu et aI., 1993; Teigen, 1974a, 1974b) suggests even in the two (dichotomous) alternatives case, testing that people are capable ofoverestimating the likelihood a hypothesis and testing the alternative can lead to very ofa multiplicity ofviable hypotheses or possibilities, not differentjudgments. Finally, in contrast to support theory, only the one that they might typically perceive to be the the present framework predicts that selective hypothesis most plausible. That is, individuals are capable of con­ testing may lead to an underestimation ofthe probability firming the likelihood of one of a number of different ofthe focal hypothesis when the supporting evidence is hypotheses as a function of what hypothesis they are weak or the confirmation criteria are too high (see Sanbon­ prompted to assess. This may occur because people often matsu et aI., 1997). have access to information suggesting multiple possibil­ Overconfidence. Frequently, people are overconfi­ ities but lack a well-formulated conclusion about what is dent ofthe accuracy oftheir decisions andjudgments (see, true. Note that, numerically, overconfidence in a partic­ e.g., Griffin & Tversky, 1992; Koehler, 1991). In a rep­ ular hypothesis entails underconfidence in the comple­ resentative study, Fischhoff, Slovic, and Lichtenstein mentary hypothesis. Subjectively, however, this is not the (1977) asked subjects to respond to a series of general case. Rather than explicitly underestimating the likelihood knowledge questions and to indicate their confidence in ofcomplementary or alternative hypotheses, people may each answer. The subjects reported confidence with ei­ simply fail to consider alternative hypotheses in assess­ ther probabilities or odds that their answers were correct. ing a particular possibility. When confidence was compared with performance, the Explanation and imagination. Explaining or imag­ results were clear: People were often highly confident that ining how an event might occur often leads people to 206 SANBONMATSU, POSAVAC, KARDES, AND MANTEL

overestimate the probability that the event will actually Chapman and 1. P. Chapman (1969) presented a series of occur (see, e.g., C. A. Anderson, 1983; Carroll, 1978; see cases in which the response or characteristic (e.g., large Koehler, 1991, for a review). Hirt and Sherman (1985), eyes) that was expected to be associated with a particular for example, found that explaining how a football team diagnosis (e.g., paranoia) was present in halfofthe cases might win an upcoming game increased subjects' per­ and absent in the remaining half. Although there was no ceptions of the likelihood of the team winning. Several actual relation between diagnosis and the presumed char­ researchers have proposed that some ofthe specific pro­ acteristics in the set ofcases presented, the subjects con­ cesses characterizing selective hypothesis testing mediate cluded that the expected or hypothesized relation was sup­ the effects of explanation and imagination instructions ported by the evidence. on likelihood estimations (Koehler, 1991; Sherman, Skov, Statistically, all four cells ofa 2 X 2 contingency table Hervitz, & Stock, 1981; see also Kahneman & Tversky's (Cell A, X presentlY present; Cell B, X presentlY absent; [1982] discussion ofsimulation). Essentially, subjects in Cell C, X absentlY present; Cell D, X absentlY absent) explanation and imagination experiments are instructed to are equally important. The information that is utilized in engage in selective hypothesis testing; they are prompted covariation estimation, however, varies considerably, often to generate evidence for the hypothetical outcome, to en­ as a function ofthe instructions, stimuli, and information vision scenarios through which the hypothetical outcome format (see Beyth-Marom, 1982). When people test the might occur and told not to consider the evidence for al­ relation between a target event or outcome and a set of ternative outcomes and scenarios in which alternative out­ predictor conditions, they are particularly likely to focus comes may occur. Not surprisingly, explanation and imag­ on cases in which the hypothesized or expected predictor ination instructions have been found to exacerbate many is present (Cells A and B), whereas cases in which the basic judgmental errors, including the tendency to overes­ outcome occurs or does not occur in the absence of the timate the effect ofa possible cause on an outcome (San­ predictor (Cells C and D) are especially likely to be ne­ bonmatsu et aI., 1993) and the overestimation ofthe prob­ glected (Arkes & Harkness, 1983; McKenzie, 1994; ability ofhypothetical events (Sanbonmatsu et aI., 1997). Shaklee & Mims, 1982; Ward & Jenkins, 1965; Wasser­ Some studies have required subjects to consider alter­ man, Dorner, & Kao, 1990). On occasion, observers focus native outcomes after explaining a hypothesis or decision. almost exclusively on instances in which the outcome Consistent with a selective hypothesis-testing view, the and the predictor co-occur (Cell A only; Smedslund, findings show that explaining how other alternatives might 1963). When the likelihood of the outcome in the pres­ occur or why other decisions may be correct eliminates ence of the hypothesized predictor is high, the estima­ the effects of explaining an initial hypothesis (see, e.g., tions of the covariation between the predictor and the C. A. Anderson & Sechler, 1986; Hirt & Markman, 1995; outcome tend to be inflated, even when the likelihood of Hoch, 1985). When people are prompted to consider al­ the outcome is equally high in the absence of the pre­ ternatives, they generate and test scenarios that would not sumed predictor variable. have been considered otherwise, thus leading to more Research by Shaklee and Mims (1982) suggests that moderate views ofthe plausibility ofthe initially explained the tendency to rely on information about the outcome in hypothesis. the presence ofthe predictor event (Cells A and B) is di­ minished when subjects are encouraged to consider the al­ Covariation Perception ternative. When the subjects were asked to assess whether Illusory correlation. Accurate assessment ofthe pat­ the outcome was more or less likely in the event's presence terns ofcovariation among events is a difficult task (for than in the event's absence, all of the relevant covaria­ reviews, see Alloy & Tabachnik, 1984; Arkes & Harkness, tion information tended to be used by the majority ofthe 1983; Beyth-Marom, 1982; Broniarczyk & Alba, 1994a, subjects. Thus, increasing the salience ofthe alternative 1994b; Crocker, 1981; Jennings, Amabile, & Ross, 1982; (i.e., the likelihood ofthe outcome in the absence ofthe McKenzie, 1994; Schustack & Sternberg, 1981; Ward & event) attenuated the tendency to selectively test the like­ Jenkins, 1965). People often fail to perceive the associ­ lihood ofthe outcome in the presence ofthe hypothesized ations that are present in an observed data set. In other predictor (for further discussion ofthe role ofhypothesis instances, they form illusory correlations in which the testing in covariation estimation, see McKenzie, 1994). degree ofassociation between two phenomena is overes­ Theory versus data-based covariation estimation. timated. Many cases ofillusory correlation formation may Related work by Jennings, Amabile, and Ross (1982) be mediated by a process ofselective hypothesis testing. compared data-based and theory-based correlationjudg­ Studies indicate that people are particularly apt to ments by presenting pairs ofstimuli that were either neu­ form illusory correlations that are consistent with pre­ tral (e.g., number pairs, sketches of people of various existing expectancies. Individuals commonly perceive the heights with walking sticks ofvarious heights) or mean­ pattern ofcovariation that is expected in a data set, even ingful (e.g., trait-behavior pairs) with respect to subjects' when there is no actual association between the variables prior beliefs or initial expectations. Data-based correla­ or events (see, e.g., Hamilton & Rose, 1980; Tversky & tionjudgments were zero or close to zero when stimulus Kahneman, 1973; see also Newcomb, 1929; Shweder, pairs were moderately but reliably correlated. By contrast, 1977; Shweder & D'Andrade, 1979). For example, L. 1. many of the theory-based correlation judgments were SELECTIVE HYPOTHESIS TESTING 207 substantially higher than was warranted by the data and Interestingly, additional research shows that merely tended to be consistent with the preexisting hypothesis assessing the role ofa putative cause may lead to an over­ or theory. Theory-based covariation estimates, ofcourse, estimation ofits effect on the outcome. For example, in a are instances in which a hypothesis about the relation study by Sanbonmatsu et al. (1993, Experiment 1) un­ guides processing. dergraduates were asked to estimate the role ofone ofthree The findings dramatically illustrate both the benefits possible, mutually exclusive causes of their university and the costs ofselective hypothesis testing. Hypotheses football team's poor record: poor coaching, lack ofsup­ serve to facilitate the recognition of associations be­ port from fans and the administration, or a lack oftalent tween events. In the absence ofa guiding hypothesis, peo­ and ability. Judgments ofcausality were made on a per­ ple often fail to learn ofan important relation that is pre­ centage scale;where 0% indicated no responsibility and sent in observed data. On occasion, however, selective 100% indicated complete responsibility. It was assumed hypothesis testing may contribute to an illusory correla­ that, ifthe mean percentage ofresponsibility assigned to tion characterized by an overestimation of the hypothe­ a focal cause exceeded 100% divided by the number of sized association between two events. causes (i.e., 33%), the responsibility ofthe focal cause was Blocking. Studies ofanimal and human learning have being overestimated. As expected, the percentage of re­ demonstrated that the prior conditioning ofa stimulus may sponsibility assigned to the focal cause was significantly block, or hinder, the subsequent conditioning ofother cues greater than 33% (M = 48.6%), even though the judged with which the stimulus is compounded or simultaneously cause was randomly assigned. Identical patterns ofover­ presented (see, e.g., Baker, Mercier, Vallee-Tourangeau, estimation were observed when the subjects estimated the Frank, & Pan, 1993; G. B. Chapman, 1991; G. B. Chapman frequency with which a putative cause led to an outcome. & Robbins, 1990; Dickinson, Shanks, & Evenden, 1984; Many instances ofextreme and unwarranted causal in­ Gluck & Bower, 1988; Kamin, 1968, 1969; Price & Yates, ference may stem from selective hypothesis testing; the 1993, 1995; Sanbonmatsu, Akimoto, & Gibson, 1994; asymmetric treatment ofevidence for a focal versus non­ Trabasso & Bower, 1968). For example, in the basic par­ focal causal hypothesis in information acquisition, in­ adigm used by Kamin (1968, 1969), two groups of rats terpretation, and integration may lead to the overestima­ were exposed to simultaneous light and noise, followed tion ofthe strength ofa hypothesized cause. A number of by a shock. Previously, one ofthe two groups ofrats had studies confirm that alternative explanations are rou­ been exposed to several pairings ofthe noise only and the tinely ignored in the estimation ofthe effects ofa partic­ shock. During the test phase, conditioning to the light ular cause and that the overestimation ofthe causal role of only was assessed. Rats previously conditioned to the noise a focal cause may be mediated by selective hypothesis showed substantially less fear responding to light than testing. Shaklee and Fischoff(1982) show that alternative rats in the comparison condition. Thus, the learned as­ causes are often given little consideration in the assess­ sociation between the noise and the shock blocked learn­ ment of a focal cause. In their study, the subjects were ing ofthe covariation between the light and the shock. given the opportunity to ask questions that would help to Many instances ofblocking may stem from selective explain why an event occurred (e.g., why was a driver hypothesis testing. Explanations are rarely characterized speeding?). Although several plausible causes were sug­ by an exploration ofall the possible causes ofan outcome gested, the subjects tended to seek evidence that clari­ (see, e.g., Hansen, 1980; Nisbett & Ross, 1980; Shaklee fied the role of an implicated focal cause. Other experi­ & Fischhoff, 1982). Instead, organisms may identify a ments show that estimations ofthe effect ofa focal cause likely cause and examine whether the outcome is predicted on an outcome are typically attenuated or discounted when by the putative cause in a satisficing manner. If the evi­ alternative causes are considered (see, e.g., Kruglanski, dence is largely consistent with the hypothesized expla­ Schwartz, Maides, & Hamel, 1978). For example, Van der nation, the search for additional causes may be truncated. Plight, Eiser, and Spears (1987) found that the perceived Consequently, the learning ofthe covariation between al­ contribution of any particular technology (e.g., nuclear ternative causes and the outcome may be blocked or at­ power) to the overall supply of energy in the United tenuated. Kingdom was attenuated when a multiplicity oftechnolo­ gies, as opposed to a single technology, were rated. It ap­ Causal Explanation pears that the subjects engaged in selective hypothesis Overestimation of causality. Explanations of the testing and neglected viable alternatives in assessing the causes ofeveryday outcomes or events are often flawed. contribution ofa single technology. When the consider­ People commonly overestimate the role ofa particular pos­ ation ofalternatives was prompted in the multiple tech­ sible cause ofan outcome, especially ifthe cause is one nologies condition, the causality attributed to any par­ that is expected, on the basis ofprior beliefs, to influence ticular technology was attenuated. the outcome. Thus, explanations ofevents have often been Direct evidence that the overestimation ofcausality is found to be consistent with prior beliefs or expectations mediated by selective hypothesis testing was provided by (see, e.g., Deaux & Emswiller, 1974; Feather & Simon, Sanbonmatsu et al. (1993, Experiment 2). The subjects 1975). judged the extent to which one of three factors-boring 208 SANBONMATSU, POSAVAC, KARDES, AND MANTEL lectures, laziness, or other priorities---eontributed to class groups are sometimes categorized in ways that are unjus­ absenteeism. The mean causality attributed to the focal tified (for reviews, see Fiske & Taylor, 1991; von Hippel, cause was badly inflated. While making the causality esti­ Sekaquaptewa, & Vargas, 1995). mation, the subjects listed the judgment-relevant thoughts Much of the variability and error in categorization is that went through their minds. An examination of the due to the priming ofrelevant constructs or the operation verbal protocols revealed that all but I ofthe 27 subjects ofexpectancies or prior beliefs. A large body ofresearch considered evidence for the target cause contributing to has demonstrated that people are particularly apt to cat­ the outcome. However, 10 ofthe 27 failed to consider any egorize a stimulus in terms ofprimed constructs; recent alternative causes, and only 2 ofthe 27 considered at least or frequent activation ofa category increases the likelihood two causes, even though other analyses indicated that that a person, object, or event will be perceived in terms of nearly all of the subjects perceived that all three factors that category (see, e.g., Bargh et aI., 1986; Herr, Sherman, contributed to the outcome. Moreover, the number ofal­ & Fazio, 1983; Higgins et aI., 1977; Srull & Wyer, 1979). ternative causes considered was negatively correlated with An even larger literature has documented the powerful the estimates of the responsibility of the focal cause, effects of preexisting beliefs, expectancies, or schemas suggesting that the selective consideration of evidence on categorization (for a review, see Fiske & Taylor, 1991). for the focal cause mediated the causal overestimations. A plethora ofresearch, for example, has shown that indi­ The fundamental attribution error. Social perceiv­ viduals often draw trait inferences that are consistent with ers often overestimate the influence ofdispositional fac­ specific expectancies, such as social stereotypes (see, e.g., tors and underestimate the effects of situational factors von Hippel et aI., 1995). Many ofthe effects ofexpectan­ on behavior (Gilbert & Malone, 1995; Jones, 1979; Ross, cies and priming on categorization are undoubtedly me­ 1977). Thisfundamental attribution error, or correspon­ diated by a process that is akin to selective hypothesis dence bias, may be mediated, in part, by selective hypoth­ testing. Studies indicate that expectations tend to bias the esis testing; social perceivers may hypothesize that the search for and the storage of evidence (see, e.g., Hastie behavior ofanother is dispositionally driven and engage in & Kumar, 1979; Rothbart et aI., 1979; Srull, 1981; von information processing that neglects the role ofsituational Hippel, Jonides, Hilton, & Narayan, 1993; for a review, causes and confirms the influence ofthe person. see Stangor & McMillan, 1992), the interpretation or Interestingly, a number ofstudies have shown that so­ construal ofevidence (see, e.g., Duncan, 1976; Griffin & cial perceivers are also susceptible to overestimating the Ross, 1991; Trope, 1986), and the assimilation ofevidence role ofsituational causes ofan outcome and neglecting the (see Lord et aI., 1979) in a selective manner that is con­ causal influence ofthe person (see, e.g., Krull, 1993; Quat­ sistent with the expected trait, characteristic, or pattern. trone, 1982; Sanbonmatsu et aI., 1993). When perceivers Limited consideration may be given to the extent to which are prompted to assess the causal role ofthe situation (but alternative categorizations or interpretations are sup­ not the person), they typically generate inflated estimates ported by the available data. In keeping with this selective ofcausality. This suggests that the fundamental attribu­ hypothesis interpretation, research has shown that the ef­ tion error is part ofthe much broader tendency to over­ fects of expectations on categorization are diminished estimate the effect ofa focal cause on an outcome. People when people are instructed to consider alternative hy­ appear to be capable ofconfirming the influence ofeither potheses (Lord et aI., 1984) or when conditions foster or situational or person causes, and possibly both, in explain­ promote the consideration ofalternative categorizations. ing behavior; the explanation that ensues depends on what Not all of the effects of expectancies or prior beliefs hypothetical cause or causes a perceiver is prompted or on categorization are mediated by a process ofselective motivated to assess. hypothesis testing. Rather than being treated as hy­ potheses to be tested, expectancies are often assumed Trait Inference prima facie to be true. Here, evidence is gathered to justify Trait judgments are characterized by considerable an expectancy rather than to evaluate it (Yzerbyt, Schan­ variability between-subjects, as well as within-subjects. dron, Leyens, & Rocher, 1994). Nevertheless, many ex­ Different people often see the same object, person, or event pectancies are assessed on the basis ofthe available data. in different ways (e.g., a stimulus person may be viewed This was nicely illustrated by Darley and Gross (1983), as smart by one perceiver, stupid by another), even when who found that social class stereotypes did not affect im­ the evidence or datum remains constant. Moreover, the pressions ofa lower class youth in the absence ofevidence. same person may see the same stimulus differently over Only when ambiguous information about the target per­ time. This sort ofvariability is particularly likely to occur son was presented were differential impressions oflower when the evidence is ambiguous and thus subject to dif­ versus middle class youths observed. Presumably, the ferent sorts ofinterpretations. Given the enormous vari­ evidence was selectively attended to and construed in a ability, a significant proportion of these categorizations way that permitted the confirmation of the stereotype. must be erroneous. Indeed, findings indicate that trait in­ Because expectancies are often treated as hypotheses, they ferences are often drawn that are marginally supported by are not always verified by the evidence. Thus, when con­ the available evidence. Social perception studies, in par­ crete, individuating information is present or available to ticular, have repeatedly demonstrated that members of contradict expectations, the specific information, rather SELECTIVE HYPOTHESIS TESTING 209 than expectations, tends to serve as the basis for catego­ For example, if a gambler focuses on the risk that is as­ rization (see, e.g., Fiske, Neuberg, Beattie, & Milberg, sociated with taking an additional card in blackjack, he or 1987; Locksley, Ortiz, & Hepburn, 1980). she may overestimate the probability ofbusting and make a decision to stand pat. When the likelihood of the pos­ Rule Discovery sible outcomes is uncertain, selectively focusing on one In some rule discovery tasks, people prematurely settle possibility and neglecting others may bias the probability for a response that is suboptimal or simply wrong. For ex­ estimations that shape preference (Gibson, Sanbonmatsu, ample, in the classic study by Wason (1960) mentioned ear­ & Posavac, 1997). lier, the subjects commonly generated and accepted solu­ Selective hypothesis-testing processes appear to sim­ tions (e.g., consecutive even numbers) that were narrower ilarly affect valuations ofspecific attributes or outcomes than the actual rule (increasing numbers). Many ofthe dif­ and generally bias the assessment ofchoice options. Find­ ficulties encountered in rule discovery tasks appear to stem ings in both the psychological and economics literatures from selective hypothesis testing. Studies ofconcept iden­ indicate that the preference for a valued option may be in­ tification show that, rather than starting with a set ofalter­ flated when the option is considered singularly or in iso­ native hypotheses in mind that are gradually eliminated lation. trial by trial, subjects typically adopt a win-stay/lose-shift Contingent valuation. Studies of contingent valua­ strategy in which they continue to use a reinforced hy­ tion have consistently shown that focusing on the worth pothesis until it is no longer reinforced (Bruner, Goodnow, ofa particular public good leads to a valuation that is dis­ & Austin, 1956; Levine, 1966, 1970; Millward & Spoehr, proportionately high, relative to the valuations ofcompa­ 1973; Restle, 1962; Trabasso & Bower, 1968). Often, little rable public goods and the sum of the collective public consideration is given to alternative hypotheses in the test­ goods (see Hausman, 1993). When people state their will­ ing ofa focal hypothesis. This was demonstrated by Taplin ingness to pay for a particular desired public good, they (1975), who presented subjects with geometric designs often generate unrealistically high values that would con­ that varied on four dimensions according to several differ­ sume a large part oftheir income (Diamond & Hausman, ent two-dimensional rules (e.g., conjunction, disjunction, 1993). Moreover, a particular public good tends to be val­ conditional, biconditional). The subjects were given an ini­ ued much higher when it appears first in a questionnaire tial hypothesis and were instructed to take as many trials as rather than later, after related goods have been considered were necessary to determine the validity of the initial hy­ (Samples & Hollier, 1990; Tolley & Randall, 1989, dis­ pothesis. Taplin observed that, when the initial hypothesis cussed in Mitchell & Carson, 1989). Finally, studies of was true, the number oftrials to criterion did not differ as the embedding effect demonstrate that the value assigned a function ofthe rule tested, even though the different rules to a good is much higher when the good is evaluated by or hypotheses differed in the minimum number of trials itselfthan when it is inferred from the willingness to pay needed to rule out alternatives (i.e., two trials for conjunc­ for a more inclusive good. Kahneman and Knetsch (1992), tion, three for disjunction and conditional, and four for bi­ for example, found that individuals reported that im­ conditional rules). Hence, tests of a focal rule were provements in rescue personnel and equipment were worth markedly insensitive to alternatives. $122 when directly assessed, but only $14 when the In line with a selective hypothesis-testing framework, value ofimprovements was inferred from the willingness studies have demonstrated that rule discovery improves to pay for the good environmental services, which in­ when individuals are prompted to consider alternative cluded rescue improvements. hypotheses (Tweney et aI., 1980). Similarly, subjects The apparent tendency to overestimate the relative have been found to perform better on the classic Wason worth ofa focal public good may stem from selective hy­ task when they engage in counterfactual thinking and pothesis testing. The judgmental task essentially prompts generate an alternative to the given hypothesis (Farris & respondents to test the hypothesis that the public good is Revlin, 1989). valuable. Moreover, many respondents begin with an ini­ tialliking for the target public good. The subsequent search Evaluation and Preference and interpretive processes may highlight the benefits or According to subjective utility theory (von Neumann & positive attributes of the entity. When similar and equal Morgenstern, 1947), information integration theory consideration is not given to the value ofalternative or com­ (N. H. Anderson, 1981, 1982), prospect theory (Tversky parable goods, the relative worth of the focal good may & Kahneman, 1981, 1992), and related variations (see, be exaggerated. Thus, goods considered first, in isolation e.g., Ajzen & Fishbein, 1980), the evaluation ofa choice from comparable options or without regard to more in­ option or prospect is an integration ofthe valuations and clusive categories, may be particularly apt to be overval­ perceived probabilities, or weights, ofthe associated out­ ued. Consistent with a selective hypothesis interpretation, comes or attributes. We have already reviewed the role Schkade and Payne (1993) found that, when competing al­ that selective hypothesis testing may play in probability ternatives are made salient, people tend to moderate their estimations and predictions. Interestingly, this research previously stated valuations ofthe target good. suggests that, ifa possible outcome is selectively focused Attitude polarization. Attitude polarization may be on, its relative weight or expectancy may be overestimated. similarly mediated by a process of selective hypothesis 210 SANBONMATSU, POSAVAC, KARDES, AND MANTEL testing. Prior work indicates that merely thinking about an proposition (Schuman & Presser, 1981). Similarly, vary­ object, person, or issue often leads to more extreme eval­ ing the focus of a question by asking about one of two uations. For example, Sadler and Tesser (1973) found groups leads to more extreme judgments concerning that that prompting subjects to think about an experimental group (Lehman et aI., 1992; Vallone, Ross, & Lepper, partner increased liking for a liked partner and increased 1985). For example, a study ofthe hostile media effect by disliking for a disliked partner. The initial attitude may Vallone et aI. (1985) demonstrated that asking about the be regarded as a focal hypothesis that guides the pro­ extent to which a news story was biased against Israel led cessing of additional information. Along these lines, respondents to conclude that the story was biased against Tesser (1978) suggests that, during thinking about an at­ Israel, whereas asking about the extent to which the story titude object, the initial attitudinal schema biases the as­ was biased against Arabs led to the conclusion ofan anti­ similation ofevidence and target-related inferences in a Arab bias. In each ofthese cases, the wording ofa ques­ manner that contributes to greater evaluative consistency tion may lead respondents to focus on one hypothesis. and attitude polarization. Moreover, commitment to the This hypothesis may be selectively tested and other pos­ initial attitude often leads to attempts to justify or con­ sibilities, most notably the evaluative converse, may be firm the attitude (Millar & Tesser, 1986). Ofcourse, ifthe largely ignored. The acquiescence bias may result from evidence about a target is evaluatively moderate or incon­ generating reasons to agree with a proposition; reasons sistent, attitude polarization may not occur (see Judd & why one might disagree are more likely to be considered Lusk, 1984; Linville, 1982). when subjects are asked whether they agree or disagree. Preference. Recent studies indicate that the most fa­ The hostile media effect may similarly result from focus­ vorably evaluated options are not always considered in ing on evidence concerning one party while neglecting preference judgment. For example, subjects in a series of evidence concerning the other party. studies by Posavac, Sanbonmatsu, and Fazio (1997) were asked to indicate the charity that they preferred most for Related Phenomena a donation. When the relevant charities were neither salient Selective hypotheses testing. Some judgmental tasks nor highly accessible in memory, the subjects commonly require the assessment ofthe plausibility ofa specific set indicated a preference for a moderately favored option. of hypotheses or alternatives. For example, a prediction Increasing the salience or accessibility ofalternative char­ may require the estimation of the conjoint likelihood of ities heightened considerably the likelihood ofpreference a set ofdistinct outcomes (e.g., what is the likelihood of judgments that were consistent with prior attitudes. The a, b, or c occurring?). Relatedjudgmental tasks require the findings suggest that, on occasion, preference may be char­ assessment ofthe plausibility ofeach ofa number ofal­ acterized by a process in which possibilities are serially ternatives in a given set. For example, a prediction task tested and a suboptimal option is prematurely accepted may require the estimation ofthe individual likelihoods (see also Laroche & Brisoux, 1989). ofa multiplicity of distinct outcomes (e.g., What is the Evaluative reversals. Several studies suggest, or at likelihood ofa? b? c?). In these instances, a process that least hint, that very different evaluations ofthe same ob­ is akin to selective hypothesis testing may operate, al­ ject may ensue as a consequence ofselective hypothesis though it might be better referred to as selective hypothe­ testing. For example, the subjects in a choice study by ses testing, in order to denote the multiplicity ofhypothe­ Shafir (1993) were given information about parents who ses assessed. Like selective hypothesis testing, selective were competing for custody of a child. One parent was hypotheses testing is characterized by the selective gath­ described by a greater number ofnegative and a greater ering and interpretation of evidence for the focal hy­ number ofpositive attributes. Both the subjects who were potheses and by a failure to adequately consider alterna­ asked to assess which parent should be awarded custody tives. The hypotheses are assessed on the basis of some and those who were asked to assess which parent should criteria or inference rules. As in selective hypothesis test­ not be awarded custody selected the same parent-the ing, this selective processing tends to increase the likeli­ parent with greater numbers of positive and negative hood ofthe confirmation ofthe set offocal hypotheses. qualities. Thus, the initial hypothesis that was tested (i.e., The biases characterizing the selective testing ofmul­ the hypothesis that a parent should be awarded custody tiple hypotheses have been examined in investigations of vs. the hypothesis that a parent should not be awarded cus­ fault trees, which generally have been used in a variety tody) appears to have led to very different characteriza­ ofcontexts to delineate potential causes ofan occurrence, tions ofthe same target. by organizing the causes into different branches. Fault trees Other work on survey responding indicates that slight have been used in experimental investigations ofjudg­ changes in the framing of questions often contribute to ment and in applied settings as decision aids (e.g., Joshua dramatic variations in evaluation (Krosnick & Alwin, & Garber, 1992). The ability ofsuch models to improve 1987; Lehman et aI., 1992; Schuman & Presser, 1981). the quality ofdecisions may be limited, however, because Studies ofthe acquiescence bias have shown that asking people tend to be insensitive to omissions of major po­ respondents whether they agree with a proposition typi­ tential causes (branches, in fault tree terminology) from cally results in more favorable responses than asking re­ fault trees (Dube-Rioux & Russo, 1988; Fischhoff et aI., spondents whether they either agree or disagree with a 1978; Hirt & Castellan, 1988; Russo & Kolzow, 1994). SELECTIVE HYPOTHESIS TESTING 211

This effect is quite robust, as both novices and experts bility of missing branches by having them generate and tend to be insensitive to omissions (Dube-Rioux & list causes not mentioned in the fault tree prior to estimat­ Russo, 1988; Fischhoff et aI., 1978). Basically, people ing the all other causes branch. When subjects generated tend to overestimate the relative frequency or plausibil­ causes in this manner, they assigned three times greater ity of the given branches, at the expense of causes or probability to the all other problems category than when branches that are not listed. they did not. Because the subjects were specifically re­ Fischhoffet ai. (1978) demonstrated this effect in a se­ quired to generate examples of the all other problems ries of studies in which subjects made estimates of the category, they gathered evidence that was sufficient to relative frequency with which each of several problems increase their subjective perceptions of the category's (e.g., battery charge insufficient, fuel system defective) probability. would cause an automobile to fail to start in 100 starting In related research on probability overestimation, failures. Accordingly, subjects' estimates reflected the im­ Koehler (1994) demonstrated that subjects were over­ portance they ascribed to each problem. Some subjects confident that the correct hypothesis was one ofa set of were given a list ofsix problems and a seventh category, alternatives that was either generated or presented. For ex­ all other problems. Other subjects were given a pruned ample, in one study (Experiment 1), subjects assessed fault tree-that is, a tree with at least one major cause the chances ofeither three self-generated or three other­ missing. The pruned trees consisted of three problems generated candidates for academy awards. The subjects and the all otherproblems category. Ifthe subjects in the generally overestimated the likelihood that the winner was pruned tree conditions were sensitive to the omission of one of the three possibilities, particularly when the hy­ the three branches, they should have allocated the prob­ potheses were generated by another, as opposed to them­ ability associated with each omitted branch to the all other selves. Thus, overconfidence tends to characterize sets of problems category. The subjects given a pruned fault tree hypotheses, just as overconfidence characterizes singu­ demonstrated insensitivity to omissions, because they al­ lar hypotheses. located insufficient probability to the all otherproblems Selective assessment and processing. A multitude of category andjudgedthe remaining branches to have higher studies has demonstrated that the selective assessment of probabilities (see also Hirt & Castellan, 1988). a particular target or the selective processing ofinforma­ Insensitivity to missing branches may be a result ofse­ tion about a target or event can dramatically affect com­ lective hypotheses testing. When the plausibility of parative judgment. For example, person perception stud­ branches specified in a fault tree is tested, confirmatory ies have shown that the selective processing ofevidence processing of evidence from memory tends to result in about a salient individual contributes to the formation of the perception that each specified branch is more plau­ illusory correlations that are characterized by an overes­ sible than is justified. In contrast, individuals fail to timation of the relative degree of association between a search for evidence of the influence of causes that are salient individual and an outcome or characteristic (for not specified in the fault tree. Consequently, the proba­ reviews, see McArthur, 1981; Taylor & Fiske, 1978). In bility ofthe all otherproblems branch is underestimated. a study by Taylor, Fiske, Close, Anderson, and Ruderman The mere presence of a branch appears to be sufficient (reported by Taylor & Fiske, 1978), the person who was to induce individuals to test the hypothesis that the branch most salient in a group was perceived to have contributed is a plausible cause of an event. When Fischhoff et al. more to the group discussion than had less salient persons, (1978, Experiment 5) presented a potential cause in two even though there was no difference in the actual contri­ branches, the cause was seen as more likely than when it bution. Thus, simply processing the available informa­ was presented as one branch. That is, the sum ofproba­ tion about a focal target may bias comparative judgment. bilities assigned to the two branches was larger than the Other studies indicate that selective assessment ofa focal probability assigned to the cause when it was presented target may lead to differential judgments ofthe focal tar­ as one branch. This result may have occurred because the get relative to others. For example, Klar, Medding, and subjects assessed the likelihood of each branch sepa­ Sarel (1996) demonstrated that a randomly assigned mem­ rately when the cause was presented in two branches. If ber ofa group was typically judged to be less vulnerable the subjects tested both branches in this manner, confir­ to a number ofnegative life events (e.g., an automobile ac­ matory processing ofevidence supportive ofeach branch cident) than the average group member. would result in the perception that each is likely (and, to­ Selective assessment and selective processing are gether, more likely than when the cause was presented in often similar to selective hypothesis testing in that com­ one branch). Further support for the select hypotheses parisons between the focal target and alternative targets testing interpretation ofinsensitivity to missing branches do not occur. Instead, the evidence for the focal target is is evidenced by a debiasing procedure described by Dube­ assessed singularly on the basis ofsome set ofcriteria or Rioux and Russo (1988). These authors investigated in­ standards. Selective hypothesis testing differs from se­ sensitivity to missing branches in a business context and lective processing and selective assessment in that a par­ found, as in other research, that people were insensitive ticular hypothesis about the target (a proposition about a to information missing from a fault tree. Dube-Rioux and specific relation between the target and an outcome or Russo attempted to sensitize their subjects to the possi- event) guides the judgmental process. Nevertheless, the 212 SANBONMATSU, POSAVAC, KARDES, AND MANTEL

processes are often intertwined, in that selective process­ activated response option or hypothesis is subsequently ing and assessment may develop into selective hypothe­ tested through various search and interpretive processes, sis testing. Frequently, a hypothesis or impression forms until the evidence for a hypothesis meets the threshold cri­ on-line during the initial encoding ofinformation (Asch, teria. Hypothesis testing does not necessarily involve a 1946; Hastie & Park, 1986). For example, the initial pro­ little person inside that makes a decision as to whether to cessing ofinformation about a salient person may lead to reject oraccept a hypothesis. Although the language ofin­ a particular impression about the relation between a salient ference may suggest the operation ofa homunculus, it is target and an outcome variable. This hypothesis, in turn, not a necessary presumption ofthe framework (see Read, may affect subsequent information search, construal, in­ Vanman, & Miller, 1997, for a discussion ofthe represen­ tegration, and retrieval processes in a way that biases later tation ofhypotheses in a connectionist model). judgment. Thus, although processing may not be initially guided by a hypothesis, one may form that influences judg­ When Does Selective Hypothesis ment. Moreover, selective assessments are often guided Testing Lead to Good Judgment? by specific expectancies or sets. Although the task may Researchers have delineated an impressive catalog of be to assess a focal target, hypotheses may be generated judgmental errors that may result from selective hypoth­ that guide the gathering and assimilation ofevidence. esis testing. This is a testimony both to the difficulty of This analysis suggests that some importantjudgmental many judgmental tasks and to the cleverness and insight phenomena that were not reviewed by our paper, such as of psychological investigators. In general, however, re­ salience effects, order effects (see, e.g., Asch, 1946; N. H. searchers have tended to focus on the contexts and tasks Anderson & Hubert, 1963), and the better-than-average in which biases operate and errors ensue, and much less and worse-than-average effects may be partially medi­ attention has been given to the broader set ofconditions ated by a process of selective hypothesis testing. It also under which people make good judgments (Einhorn & implies that some ofthe judgmental effects that were re­ Hogarth, 1981; Gigerenzer, 1991; Lopes, 1991). viewed may result from selective processing or selective Our review may lead some readers to conclude that se­ assessment, as well as from selective hypothesis testing. lective hypothesis testing is an especially specious judg­ mental strategy. However, investigations of decision Theoretical Caveats making and perceived similarity indicate that comparative Selective hypothesis testing is conceptualized as a assessment is similarly characterized by a variety ofbiases top-down process in which a subjective possibility or hy­ (for a review, see Medin, Goldstone, & Markman, 1995). pothesis guides the search for and construal ofevidence. For example, studies demonstrate that preference and We suggest that this process underlies many judgmental perceived similiarity often vary markedly as function of biases and errors that have been demonstrated in the lit­ which alternative serves as the starting point ofcompar­ erature. Nevertheless, as we indicated earlier, many ofthe ison (see, e.g., Houston, Sherman, & Baker, 1989; Kardes biases that were reviewed are mediated by a multiplicity & Sanbonmatsu, 1993; Tversky, 1977). Bias and error ofprocesses. This accounts for the robust nature ofmany also commonly characterize more data-drivenjudgmental of the phenomena. Interestingly, many of the reviewed approaches that rely on the integration ofa large amount judgmental biases, such as illusory correlation, blocking of evidence or on some straightforward algorithm (cf. effects, and embedding effects, also result from more basic Fiedler, 1996). Thus, selective hypothesis testing is hardly biases in the bottom-up integration ofthe data that is avail­ unique in being fallible. able in the judgmental environment (see Fiedler, 1996). Although selective hypothesis occasionally leads to Note that we have deliberately avoided the thorny mat­ error, it can be a highly effective approach to making a ter ofthe role ofconsciousness in hypothesis testing, be­ judgment. We speculate that selective hypothesis testing cause it is not one ofthe central issues ofthe paper. Nev­ is generally a useful strategy for making many of the ertheless, a few remarks on this matter should be made, judgments that are required in everyday life. Our analysis so as to avoid misunderstanding. Certainly, the informa­ suggests that the errors that have been documented so tion search processes characterizing hypothesis testing are extensively in the literature tend to occur in a limited set often controlled. Nevertheless, the generation and testing of interrelated conditions. Obviously, the testing of se­ of a focal hypothesis may often be relatively automatic. lect hypotheses is most likely to lead to error when the We would deny that judgmental options are always ver­ judgment is highly uncertain and there are many strong balized as hypotheses in the manner of"Gee, this is a good alternatives. In these instances, there is evidence to sup­ possibility." In fact, people are not always aware of the port multiple possibilities. Hence, the selective gathering hypotheses that guide processing. We would also deny that and interpretation ofevidence may lead to the premature the selective hypothesis-testing process described here and unwarranted acceptance ofa particular focal hypoth­ requires the postulation ofa homunculus or cognitive ex­ esis. Error also tends to occur when the evidence is am­ ecutive. Selective hypothesis testing operates because biguous and limited and when the information is available the available evidence does not meet the threshold for ac­ or presented in a manner that allows a biased search to tivating a particularjudgmental response. The most highly occur. Selective hypothesis testing also leads to poorjudg- SELECTIVE HYPOTHESIS TESTING 213 ment when the test hypothesis itself is ambiguous and are comparatively tested. However, a comparison ofa mul­ subject to redefinition or when the test criteria are inap­ tiplicity ofjudgmental options oran examination ofa large propriate or fuzzy. sample ofthe available data can be time-consuming and Fortunately, selective hypothesis testing does not al­ demanding processes that are impractical in many con­ ways operate under these difficult circumstances. Often, texts. Although selective hypothesis testing occasionally the social context and judgmental task permit the effective leads to error, there are a number ofgood reasons why this testing of select hypotheses. We suggest that the major­ process frequently operates. ity ofjudgments in everyday life, unlike those demanded Minimizing time and effort. Identifying viable re­ in the laboratory, are straightforward and mundane. sponse options often is a difficult and time-consuming Rather than attempting to explain, for example, the an­ task. In many instances, the relevant alternatives must be tecedents ofthe Boer War, people more commonly seek retrieved from memory or generated on the basis ofprior to make simpler judgments about the cause of the high knowledge and experience. This may be particularly de­ temperature ofa room, the approximate age ofa colleague, manding if a person is highly aroused or distracted by and the location of their keys. Most of these everyday some other ongoing activity. The difficulty ofretrieving judgments are not characterized by a high level ofuncer­ a relatively complete set ofviable alternatives may be fur­ tainty. Instead, the number of viable competitors is lim­ ther compounded by the part-list cuing effect-the ten­ ited, which restricts the potential for error. Moreover, the dency for the recognition or retrieval ofa subset ofthe al­ focal hypotheses often are unambiguous. For example, ternatives to increase the difficulty ofaccessing remaining there is little redefinition ofthe boundaries ofa hypothesis instances ofa category (see, e.g., Alba & Chattophadhyay, such as the keys are in thejacket. In addition, many every­ 1985, 1986; Rundus, 1973). Thus, in order to minimize day tasks afford clear evidence that allows the valid test­ time and limit effort, hypotheses often are serially gener­ ing of a focal hypothesis. For example, in testing a hy­ ated and tested. pothesis about the location ofone's keys, there is concrete Selective hypothesis testing is also performed in order evidence: whether the keys are present or not. As a con­ to minimize cognitive work during the assessment phase. sequence, evidence that is inconsistent with the focal hy­ The comparison ofa large number ofalternatives simul­ pothesis is detected and aptly applied. Finally, in many taneously is an unwieldy task; people are simply not cog­ judgments, the criteria are well calibrated and appropriate. nitively well equipped to assess the merits ofthree or four That is, individuals have a reliable set of standards for alternatives simultaneously (Tweney & Doherty, 1983). determining whether a hypothesis is correct. For example, Often, comparative processing necessitates aids, such as most laypersons have a well developed prototype that paper and pencil, to list and record the strength and weak­ may be used to determine whether a primate is a gorilla nesses ofthe alternatives and to keep the details straight. (as opposed to a chimpanzee or an orangutan). Similarly, Because only one alternative is assessed at a time in selec­ a well-learned algorithm or procedure may be available tive hypothesis testing, the cognitive demands are substan­ for testing hypotheses. For example, to test the proposition tially lower. the attendance at thefaculty meeting is unusually low, one Selective hypothesis testing is particularly likely to may simply count the individuals present and compare occur when the situation demands a quick response, when the total to the known average. As a consequence, hy­ the judgment or decision is not ofsufficient import to war­ potheses that are invalid are correctly rejected, and new rant a significant investment oftime and effort, or when possibilities are generated. Testing continues until an ap­ the judgmental context obstructs comparative process­ propriate judgmental response is verified. ing. When concerns about making an invalid response are Often, the first hypothesis that is generated and tested high, hypothesis testing may become less selective and is satisfactory or correct, because contextual cues, task de­ more comparative. Greater time and effort may be devoted mands, and preexisting knowledge aptly converge to lead to the task, and a more exhaustive set ofpossibilities may to the generation ofan acceptable focal hypothesis. Even be considered (Kruglanski, 1989). Alternatively, process­ ifthe conditions are not optimal and the biased assimila­ ing may be less hypothesis driven and based more heav­ tion ofevidence occurs, correctjudgment ensues, because ily on the available data. Interestingly, ifa preliminary hy­ the focal hypothesis happens to be the correct one. In sum, pothesis is confidently held, selective testing may occur, errors are made in a limited set ofcircumstances. In sim­ even when strong incentives to be accurate are present (see plejudgmental tasks, where the viable alternatives are few, Schwartz, 1982). the criteria are well established, and the evidence is ample The selective testing ofa hypothesis usually does not and clear, selective hypothesis testing leads to goodjudg­ occur when viable alternatives readily come to mind or are ment. These, however, are not the circumstances that have highly salient in the judgmental context (see, e.g., Mehle been studied in demonstrations ofjudgmental bias. et aI., 1981). When alternatives are specified or accessible, diagnostic information is typically sought, in order to Why Hypotheses Are Selectively Tested discriminate between the hypotheses (Bassok & Trope, Judgment is not always mediated by the selective test­ 1984; Trope & Bassok, 1983; Trope & Mackie, 1987). ing of focal hypotheses. In some instances, judgments Selective hypothesis testing is less likely to operate in di­ are primarily data driven, whereas, in others, hypotheses chotomous or binaryjudgments, because the examination 214 SANBONMATSU, POSAVAC, KARDES, AND MANTEL ofone ofthe possibilities often prompts some considera­ Selective hypothesis testing often works. As we sug­ tion ofthe alternative (Tversky & Koehler, 1994). gested earlier, selective hypothesis testing is a useful strat­ Too many plausible alternatives. In some tasks re­ egy that frequently leads to good judgment. Task require­ quiring the assessment of a particular hypothesis, a ments, prior knowledge and experience, contextual cues, plethora of alternatives are available. In many of these and the available evidence often aptly converge to lead to judgments, it is logistically impossible to consider the al­ the generation of a strong candidate or possibility. Fre­ ternatives and engage in comparative processing. Take, for quently, the first hypothesis that is selectively tested is the example, the task ofassessing whether a particular hur­ correct hypothesis. Moreover, often a well-established, dler is the world's best. A comparative analysis might en­ reliable set of standards or criteria for assessing the va­ tail an arduous sampling ofthe performances ofthe world's lidity ofhypotheses is available. In these instances, selec­ competitive hurdlers. A related problem is that it is often tive hypothesis testing tends to lead to good judgment, difficult to identify the relevant alternatives. For example, because only the correct response meets the threshold cri­ it is not easy to think ofall ofthe alternative occupations teria. Focal hypotheses that fail to pass the standards are to the test possibility that a target person is an accountant. discarded, and new hypotheses are generated and selec­ In judgments where the number ofalternatives is practi­ tively tested. cally limitless, a much more viable strategy is selective Often, it is not important to be exact or to make the best hypothesis testing. Again, in selective hypothesis testing, judgment. For example, it may not be essential to identify knowledge of the alternatives is not needed; confirma­ the absolutely best lunch spot or to provide an exact ex­ tion is based on whether the evidence for the focal hypoth­ planation for a colleague's aberrant behavior. Rather, one esis fulfills some criteria. Thus, for a selective test ofthe may simply want to identify a good lunch spot or provide hypothesis that a particular hurdler is the world's best, a reasonable explanation for the misbehavior. Conse­ all one needs is to know the hurdler's typical time and fin­ quently, criteria may be used that lead to a satisficing ish and to have a sense ofthe performances ofchampion though suboptimaljudgment. Although this might be clas­ hurdlers. Similarly, in the assessment of whether a per­ sified as an error by observers, from the individual's per­ son is an accountant, one needs only to know the typical spective, it is not an error, because the operative judgmen­ qualities ofaccountants. The development and application tal requirements were fulfilled. ofstandards allows one to avoid the multitude ofcompar­ Misunderstandingofthejudgmental requirements. isons that might otherwise be required. Although there are many good reasons why selective hy­ Somejudgmental tasks afford a multiplicity ofplausible pothesis testing occurs, there are also some fundamentally responses. For example, in an assessment ofthe mileage bad reasons. Sometimes people are lazy and unwilling ofa drive between two cities, there maybe hundreds ofvi­ to consider alternatives. Perhaps even worse, sometimes able alternatives (i.e., different mileage estimates), with no a hypothesis is selectively tested because ofa misunder­ one alternative appearing to be particularly compelling. standing of the task. On occasion, people fail to grasp Rather than attempting to generate and compare all ofthe the judgmental requirements and assume that they are relevant alternatives, a more efficient strategy is to engage being asked to assess the sufficiency ofa hypothesis, as in a selective process ofsuccessive approximation, where opposed to the necessity ofa hypothesis. For example, in a series offocal hypotheses are generated, tested, and re­ causal judgment experiments, subjects sometimes en­ fined. Here, the initial focal hypothesis is recognized to be gage in selective testing because they fail to recognize a reference point into which the evidence is integrated and that the given task is to assess whether an outcome was through which a satisfactory response is gradually approx­ caused by a factor, as opposed to whether a factor could imated. cause the outcome. In other cases, people are uninformed Some hypotheses are more important than others. about the task and simply do not understand the compar­ Hypotheses are not equally important. A hypothesis often ative nature ofjudgment. For example, in the probability is selected for testing because ofits motivational signif­ estimation research by Robinson and Hastie described icance, informational value, and/or relevance to self. In earlier (1985; Van Wallendael & Hastie, 1990), the like­ many situations, it is important to test some hypotheses lihood ofeach possibility was often calculated indepen­ before others. For example, ifthere is an unidentified an­ dently of the probabilities of competing possibilities. It imal rumbling about the tent, it is probably more prudent appears that selective testing of each possibility oc­ to check the possibility that it is a bear before entertaining curred, in part, because the subjects simply did not un­ the notion that it is a raccoon. Similarly, in trying to ac­ derstand the relative nature of probability estimations. count for the expression ofirritation on the boss's face, it Thus, on occasion, selective hypothesis testing is per­ is more important to examine one's own behavior before formed because ofa lack ofcognizance ofthe comparative considering which ofseveral colleagues might be acting implications ofa hypothesis. in an obnoxious or annoying manner. In these instances, a person might be eaten or fired ifhe or she generated and Concluding Remarks compared the various alternatives, instead ofengaging in Decades ofjudgment research have led to the identi­ a process ofselective hypothesis testing. fication ofa multiplicity ofbiases and errors that charac- SELECTIVE HYPOTHESIS TESTING 215 terize everyday observations and decisions. Some ob­ ofspecialization that has beset our field. Psychology has servers might derisively characterize this knowledge base become a fragmented set ofdomains or fiefs, out ofwhich as a disorganized catalog ofjudgmental shortcomings. researchers rarely stray. Identification ofcommon repre­ However, in this review, we have attempted to convey that sentational frameworks is important, because it may in­ many of the judgmental effects that have been estab­ crease interfacing between researchers in different do­ lished by different literatures are interrelated, in that they mains and generally facilitate the advancement ofmore share similar mechanisms. Findings as diverse as the em­ sophisticated general frameworks for understanding infor­ bedding effect, blocking, the fundamental attribution error, mation processing. and the hindsight bias are similar, in that they are often mediated by a process ofselective hypothesis testing. REFERENCES The recognition of the parallels in underlying pro­ cesses is important, because it suggests that the reviewed ABELSON. R. P., & LEVI, A. (1985). Decision making and decision theory. judgmental phenomena have similar boundary condi­ In G. Lindzey & A. Aronson (Eds.), Handbook ofsocialpsychology (Vol. 2, pp. 231-310). New York: Random House. tions. And, indeed, the literature suggests that many ofthe AJZEN, I., & FISHBEIN, M. (1980). Understanding attitudes and pre­ biases and errors are further interrelated, in that they are dicting social behavior. Englewood Cliffs, NJ: Prentice-Hall. moderated similarly by factors such as the motivation or ALBA, J. w., & CHATTOPADHYAY, A. (1985). Effects ofcontext and part­ opportunity to form a soundjudgment. These moderators, category cues on recall of competing brands. Journal ofMarketing Research, 22, 340-349. of course, vary in how they affect selective hypothesis ALBA, J. W., & CHATTOPADHYAY, A. (1986). Salience effects in brand testing. Some factors affect the generation ofhypotheses. recall. Journal ofMarketing Research, 23, 363-369. Priming, expectancies or prior beliefs, framing, and ALLOY, L. B., & TABACHNIK, N. (1984). Assessment ofcovariation by salience manipulations, for example, affect diverse cat­ humans and animals: The joint influence of prior expectations and egories ofjudgment (e.g., preference, explanation, pre­ current situational information. Psychological Review, 91,112-149. ANDERSON, C. A. (1983). Abstract and concrete data in the perseverance diction) similarly by influencing the hypothesis that is ofsocial theories: When weak data lead to unshakable beliefs. Journal selectively tested. In other instances, moderators affect the ofExperimental Social Psychology, 19, 93-108. manner in which e~idence is gathered or assimilated or ANDERSON, C. A., & SECHLER, E. S. (1986). Effects ofexplanation and the inferences that are drawn. Finally, moderator variables counterexplanation on the development and use of social theories. Journal ofPersonality & Social Psychology, 50, 24-34. influence the general processing strategy that is used. A ANDERSON, N. H. (198 I). Foundations ofiriformation integration theory. ubiquitous debiasing technique that has been identified New York: Academic Press. by many judgmental researchers is the consideration of ANDERSON, N. H. (1982). Methods ofiriformation integration theory. alternatives. We suggest that instructions to consider the New York: Academic Press. alternatives eliminate some judgmental errors by prompt­ ANDERSON, N. H., & HUBERT, S. (1963). Effects ofconcomitant verbal recall on order effects in personality impression formation. Journal ing comparative, rather than selective, hypothesis testing. ofVerbal Learning & Verbal Behavior, 2, 379-391. Interestingly, in doing so, the likelihood ofthe biases as­ ANDERSON, R. C., & PICHERT, J. W. (1978). Recall ofpreviously unre­ sociated with comparative processing are, ofcourse, in­ callable information following a shift in perspective. Journal ofVer­ creased. Some moderators, such as arousal or the need to bal Learning & Verbal Behavior, 17, 1-12. ARKES, H. R. (199 I). The costs and benefits ofjudgmental errors: Im­ avoid closure, have multiple effects on hypothesis testing. plications for debiasing. Psychological Bulletin, 110,486-498. For example, a high level ofarousal may reduce the thresh­ ARKES, H. R., & HARKNESS, A. R. (1983). Estimates ofcontingency be­ old criterion that is used in testing a select hypothesis and, tween two dichotomous variables. Journal ofExperimental Psychology: more generally, diminish the likelihood of comparative General, 112, 117-135. processing. AscH, S. E. (1946). Forming impressions ofpersonality. Journal ofAb­ normal & Social Psychology, 41, 258-290. The focus ofthe present review was the role ofselective BACON, E (1960). The new organon and related writings. New York: hypothesis testing in judgment. Judgment, ofcourse, is not Liberal Arts Press. (Original work published 1620) the only cognitive phenomenon that may be mediated by BAKER, A. G., MERCIER, P., VALLEE-TOURANGEAU, E, FRANK, R., & the selective testing ofhypotheses. Some ofthe strongest PAN, M. (1993). Selective associations and causality judgments: Presence ofa strong causal factor may reduce judgments ofa weaker demonstrations ofselective hypothesis testing have been one. Journal ofExperimental Psychology: Learning, Memory, & presented in studies ofproblem solving (see, e.g., Domi­ Cognition, 19, 414-432. nowski, 1972; Higgins & Chaires, 1980; Luchins, 1942). BARGH, J. A., BOND, R. N., LOMBARDI, W. L., & TOTA, M. E. (1986). The Hypothesis-testing processes have also been postulated additive nature ofchronic and temporary sources ofconstruct acces­ in perception (see, e.g., Neisser, 1967), decision making sibility. Journal ofPersonality & Social Psychology, 50, 869-878. BASSOK, M., & TROPE, Y. (1984). People's strategies for testing hypothe­ (see, e.g., Simon, 1956), and reading comprehension and ses about another's personality: Confirmatory or diagnostic? Social memory (see, e.g., R. C. Anderson & Pichert, 1978; Just & Cognition, 2, 199-216. Carpenter, 1975; Kintsch & Young, 1984). Thus, the basic BAUMEISTER, R. E, & NEWMAN, L. S. (1993). Self-regulation ofcognitive hypothesis-testing processes that were the focus ofthis inference and decision processes. Personality & Social Psychology Bulletin, 20, 3-19. review are certainly not limited to judgment. BEACH, L. R., & MITCHELL, T. R. (1978). A contingency model for the Recognition ofthe theoretical parallels that exist across selection ofdecision strategies. Academy ofManagement Review, 3, different domains is important in light ofthe high degree 439-449. 216 SANBONMATSU, POSAVAC, KARDES, AND MANTEL

BETTMAN, 1. R. (1979). An information processing theory ofconsumer act-availability heuristic. EINHORN, H.1., & HOGARTH, R. M. (1981). Behavioral decision theory: Journal ofExperimental Social Psychology, 14, 88-96. Processes ofjudgment and choice. AnnualReview ofPsychology, 32, CHAIKEN, S., LIBERMAN, A., & EAGLY, A. H. (1989). Heuristic and sys­ 53-88. tematic processing within and beyond the persuasion context. In 1. S. EINHORN, H.1., & HOGARTH, R. M. (1985). Ambiguity and uncertainty Uleman & 1. A. Bargh (Eds.), Unintended thought (pp. 212-252). in probabilistic inference. Psychological Review, 92, 433-461. New York: Guilford. EVETT, S. R., DEVINE, P. G., HIRT, E. R., & PRICE, 1. (1994). The role of CHAIKEN, S., & MAHESWARAN, D. (1992). Heuristic processing can bias the hypothesis and the evidence in the trait hypothesis testing process. systematic processing: Effects ofsource credibility, argument ambigu­ Journal ofExperimental Social Psychology, 30, 456-481. ity, and task importance on attitudejudgment. Journal ofPersonality & FARRIS, H. H., & REVLIN, R. (1989). Sensible reasoning in two tasks: Social Psychology, 66, 460-473. Rule discovery and hypothesis evaluation. Memory & Cognition, 17, CHAPMAN, G. B. (1991). Trial order affects cue interaction in contingency 221-232. judgment. Journal ofExperimental Psychology: Learning, Memory, & FAZIO, R. H. (1990). Multiple processes by which attitudes guide be­ Cognition, 17, 837-854. havior: The MODE model as an integrative framework. In M. P. CHAPMAN, G. B., & ROBBINS, S. 1. (1990). Cue interaction in human Zanna (Ed.), Advances in experimental social psychology (Vol. 23, contingency judgment. Memory & Cognition, 18, 537-545. pp. 75-109). San Diego: Academic Press. CHAPMAN, L.1., & CHAPMAN, 1. P. (1969). Illusory correlation as an ob­ FEATHER, N. T, & SIMON, 1. G. (1975). Reactions to male and female stacle to the use ofvalid psychodiagnostic signs. Journal ofAbnormal success and failure in sex-linked occupations: Impressions ofperson­ Psychology, 74, 271-280. ality, causal attributions, and perceived likelihood ofdifferent conse­ CHRISTENSEN-SZALANSKI, 1.1.1., & WILLHAM, C. E (1991). The hind­ quences. Journal ofPersonality & Social Psychology, 50, 229-238. sight bias: A meta-analysis. Organizational Behavior & Human de­ FIEDLER, K. (1996). Explaining and simulating judgment biases as an cision Processes, 48,147-168. aggregation phenomenon in probabilistic, multiple-cue environ­ CONNOLLY, T, & BUKSZAR, E. W (1990). Hindsight bias: Self-flattery or ments. Psychological Review, 103, 193-214. cognitive error? Journal ofBehavioral Decision Making, 3, 205-211. FISCHHOFF, B. (1975). Hindsight = foresight: The effect of outcome CROCKER, 1. (1981). 1udgment ofcovariation by social perceivers. Psy­ knowledge on judgment under uncertainty. Journal ofExperimental chological Bulletin, 90, 272-292. Psychology: Human Perception & Peiformance, 1,288-299. DARLEY, 1. M., & GROSS, P. H. (1983). A hypothesis-confirming bias in la­ FISCHHOFF, B. (1977). Perceived informativeness offacts. Journal ofEx­ beling effects. Journal ofPersonality & Social Psychology, 44, 20-33. perimental Psychology: Human Perception & Peiformance, 3, 349-358. DAWES, R. M. (1994). House ofcards: Psychology and psychotherapy FISCHHOFF, B. (1982). Debiasing. In D. Kahneman, P. Siovic, & A. Tver­ built on myth. New York: Free Press. sky (Eds.), Judgment under uncertainty: Heuristics and biases DAWES, R. M., & MULFORD, M. (1996). The false consensus effect and (pp. 422-444). New York: Cambridge University Press. overconfidence: Flaws in judgment or flaws in how we study judg­ FISCHHOFF, B., & BEYTH-MAROM, R. (1983). Hypothesis evaluation ment? Organizational Behavior & Human Decision Processes, 65, from a Bayesian perspective. Psychological Review, 90, 239-260. 201-211. FISCHHOFF, B., SWVIC, P., & LICHTENSTEIN, S. (1977). Knowing with cer­ DEAUX, K., & EMSWILLER, T (1974). Explanations of successful per­ tainty: The appropriateness ofextreme confidence. Journal ofExperi­ formance on sex-linked tasks: What is skill for the male is luck for mental Psychology: Human Perception & Peiformance, 3, 552-564. the female. Journal ofPersonality & Social Psychology, 29, 80-85. FISCHHOFF, B., SLOVIC, P., & LICHTENSTEIN, S. (1978). Fault trees: Sen­ DELANY, D. E., & HILTON, D.1. (1991). Conversational implicature, con­ sitivity of estimated probability failures to problem representation. scious representation, and the conjunction fallacy. Social Cognition, Journal ofExperimental Psychology: Human Perception & Perfor­ 9,85-110. mance, 4, 330-344. DEVINE, P. G., HIRT, E. R., & GEHRKE, E. M. (1990). Diagnostic and FISKE, S. T, NEUBERG, S. L., BEATTIE, A. E., & MILBERG. S. J. (1987). confirmation strategies in trait hypothesis testing. Journal ofPerson­ Category-based and attribute-based reactions to others: Some infor­ ality & Social Psychology, 58, 952-963. mational conditions of stereotyping and individuating processes. DIAMOND, P. A., & HAUSMAN,1. A. (1993). On contingent valuation: Journal ofExperimental Social Psychology, 23, 399-427. Measurement ofnonuse values. In 1. A. Hausman (Ed.), Contingent FISKE, S. T, & TAYLOR, S. E. (1991). Social cognition (2nd ed.). New valuation: A critical assessment. New York: Elsevier, North-Holland. York: McGraw-Hill. DICKINSON, A., SHANKS, D. R., & EVENDEN, 1. L. (1984).1udgements of GETTYS, C. E, & FISHER, S. D. (1979). Hypothesis plausibility and hy- SELECTIVE HYPOTHESIS TESTING 217

pothesis generation. Organizational Behavior & Human Decision Pro­ explaining hypothetical events. Journal ofExperimental Social Psy­ cesses, 24, 93-110. chology, 21, 519-543. GETTYS, C. F., MEHLE, T., & FISHER, S. (1986). Plausibility assessments HOCH, S. J. (1985). Counterfactual reasoning and accuracy in predict­ in hypothesis generation. Organizational Behavior & Human Deci­ ing personal events. Journal ofExperimental Psychology: Learning, sion Processes, 37, 14-33. Memory, & Cognition, 11, 719-731. GIBSON, 8., SANBONMATSU, D. M., & POSAVAC, S. S. (1997). Focal bias, HOUSTON, D. A., SHERMAN, S. 1., & BAKER, S. M. (1989). The influence probability overestimation, and gambling choice: The influence ofbi­ ofunique features and direction ofcomparison on preferences. Jour­ ased hypothesis testing on subsequent judgment. Journal ofExperi­ nal ofExperimental Social Psychology, 25, 121-141. mental Psychology: Applied, 3,126-142. JENNINGS, D., AMABILE, T. M., & Ross, L. (1982). Informal covariation GIGERENZER, G. (1991). How to make cognitive illusions disappear: Be­ assessment: Data-based vs. theory-based judgments. In A. Tversky, yond "heuristics and biases." European Review ofSocial Psychology, D. Kahneman, & P. Siovic (Eds.), Judgment under uncertainty: Heuris­ 2,83-115. tics and biases (pp. 21 I-230). New York: Cambridge University Press. GILBERT, D. T. (1989). Thinking lightly about others: Automatic com­ JONES, E. E. (1979). The rocky road from acts to dispositions. Ameri­ ponents ofthe social inference process. In 1. S. Uleman & 1. A. Bargh can Psychologist, 34, 107-1 17. (Eds.), Unintended thought (pp. 189-211). New York: Guilford. JOSHUA, S. c., & GARBER, N. J. (1992). A causal analysis oflarge vehi­ GILBERT, D. T. (1991). How mental systems believe. American Psychol­ cle accidents through fault-tree analysis. RiskAnalysis, 12, 173- I87. ogist,46,107-119. JUDD, C. M., & LUSK, C. M. (1984). Knowledge structures and evalua­ GILBERT, D. T., & MALONE, P. S. (1995). The correspondence bias. Psy­ tive judgments: Effects of structural variables on judgmental ex­ chological Bulletin, 117, 21-38. tremity. Journal ofPersonality & Social Psychology, 46, I 193-1207. GILOVICH, T. K. (1983). Biased evaluation and persistence in gambling. JUST, M. A., & CARPENTER, P. A. (1975). Comparative studies ofcom­ Journal ofPersonality & Social Psychology, 40, 797-808. prehension: An investigation of Chinese, Norwegian, and English. GLUCK, M. A., & BOWER, G. H. (1988). From conditioning to category Memory & Cognition, 3, 465-473. learning: An adaptive network model. Journal ofExperimental Psy­ KAHNEMAN, D., & KNETSCH, J. L. (1992). Valuing public goods: The chology: General, 117, 227-247. purchase ofmoral satisfaction. Journal ofEnvironmental Economics GREENWALD, A. G., PRATKANIS, A. R., LEIPPE, M. R., & BAUMGARD­ & Management, 22, 57-70. NER, M. H. (1986). Under what conditions does theory obstruct re­ KAHNEMAN, D., & TVERSKY, A. (1973). On the psychology ofpredic­ search progress? Psychological Review, 93, 216-229. tion. Psychological Review, 80, 237-251. GRICE, H. P. (1975). Logic and conversation. In P. Cole & 1. L. Morgan KAHNEMAN, D., & TVERSKY, A. (1982). Variants ofuncertainty. Cogni­ (Eds.), Syntax and semantics: 3. Speech acts (pp. 41-58). New York: tion, 11,143-157. Academic Press. KAMIN, L. J. (1968). "Attention-like" processes in classical conditioning. GRIFFIN, D. W., & Ross, L. (1991). Subjective construal, social infer­ In M. R. Jones (Ed.), Miami Symposium on the Prediction ofBehav­ ence, and human misunderstanding. In M. P. Zanna (Ed.), Advances ior, 1967: Aversive behavior(pp. 9-31). Coral Gables, FL: University in experimental socialpsychology (Vol. 24, pp. 319-359). New York: ofMiami Press. Academic Press. KAMIN, L. J. (1969). Predictability, surprise, attention and conditioning. GRIFFIN, D., & TVERSKY, A. (1992). The weighing of evidence and the In 8. A. Campbell & R. M. Church (Eds.), Punishment and aversive determinants ofconfidence. Cognitive Psychology, 24, 411-435. behavior (pp. 279-296). New York: Appleton-Century-Crofts. HAMILTON, D. L., & ROSE, T. L. (1980). Illusory correlation and the KARDES, F. R., & SANBONMATSU, D. M. (1993). Direction ofcompari­ maintenance of stereotypic beliefs. Journal ofPersonality & Social son, expected feature correlation, and the set-size effect in preference Psychology, 39, 832-845. judgment. Journal ofConsumer Psychology, 2, 39-54. HAND, L. (1951). Morals in public life. In The spirit ofliberty: Papers KINTSCH, w., & YOUNG, S. R. (1984). Selective recall of decision­ and addresses ofLearned Hand (pp. 225-252). New York: Knopf. relevant information from texts. Memory & Cognition, 12, I 12- I 17. HANSEN, R. D. (1980). Commonsense attribution. Journal ofPersonal­ KLAR, Y. (1990). Linking structures and sensitivity to judgment relevant ity & Social Psychology, 39, 996-1009. information in statistical and logical tasks. Journal ofPersonality & HASTIE, R., & KUMAR, P. A. (1979). Person memory: Personality traits Social Psychology, 59, 841-858. as organizing principles in memory for behaviors. Journal ofPer­ KLAR, Y., MEDDING, A., & SAREL, D. (1996). Nonunique vulnerability: sonality & Social Psychology, 37, 25-38. Singular versus distributional conceptions ofprobability and unreal­ HASTIE, R., & PARK, 8. (1986). The relationship between memory and istic optimism in personal risks judgments. Organizational Behavior judgment depends on whether the judgment task is memory-based or & Human Decision Processes, 67, 229-245. on-line. Psychological Review, 93, 258-268. KLAYMAN, J., & HA, Y. (1987). Confirmation, disconfirrnation, and in­ HAUSMAN, J. A. (1993). Contingent valuation: A critical assessment. formation in hypothesis testing. Psychological Review, 94, 211-228. New York: Elsevier, North-Holland. KLAYMAN, J., & HA, Y. (1989). Hypothesis testing in rule discovery: HAWKINS, S. A., & HASTIE, R. (1990). Hindsight: Biased judgments of Strategy, structure, and content. Journal ofExperimental Psychology: past events after outcomes are known. Psychological Bulletin, 107, Learning. Memory, & Cognition, 15,596-604. 311-327. KOEHLER, D. J. (1991). Explanation, imagination, and confidence in HERR, P. M., SHERMAN, S. J., & FAZIO, R. H. (1983). On the conse­ judgment. Psychological Bulletin, 110,499-519. quences ofpriming: Assimilation and contrast effects. Journal ofEx­ KOEHLER, D. J. (1994). Hypothesis generation and confidence in judg­ perimental Social Psychology, 19,323-340. ment. Journal ofExperimental Psychology: Learning, Memory, & HIGGINS, E. T., & CHAIRES, W. M. (1980). Accessibility of interrela­ Cognition, 20, 461-469. tional constructs: Implications for stimulus encoding and creativity. KORIAT, A., LICHTENSTEIN, S., & FISCHHOFF, B. (1980). Reasons for Journal ofExperimental Social Psychology, 16,348-361. confidence. Journal ofExperimental Psychology: Human Learning HIGGINS, E. T., RHOLES, W. S., & JONES, C. R. (1977). Category acces­ & Memory, 6,107-118. sibility and impression formation. Journal of Experimental Social KROSNICK, J., & ALWIN, D. F. (1987). An evaluation ofa cognitive the­ P~vchology, 13, 141-154. ory ofresponse-order effects in survey measurement. Public Opinion HIRT, E. R., & CASTELLAN, N. J., JR. (1988). Probability and category Quarterly, 51, 201-219. redefinition in the fault tree paradigm. Journal ofExperimental Psy­ KRUGLANSKI, A. W. (1989). Lay epistemics and human knowledge. chology: Human Perception & Performance, 14,122-131. New York: Plenum. HIRT, E. R., & MARKMAN, K. D. (1995). Multiple explanation: A con­ KRUGLANSKI, A. W, SCHWARTZ, J. M., MAIDES, S., & HAMEL, I. Z. sider-an-alternative strategy for debiasingjudgments. Journal ofPer­ (1978). Covariation, discounting, and augmentation: Towards a clari­ sonality & Social Psychology, 69,1069-1086. fication ofattributional principles. Journal ofPersonalitv, 46, 176-189. HIRT, E. R., & SHERMAN, S. J. (1985). The role of prior knowledge in KRUGLANSKI, A. w., & WEBSTER, D. M. (1996). Motivated closing of 218 SANBONMATSU, POSAVAC, KARDES, AND MANTEL

the mind: "Seizing" and "unfreezing." Psychological Review, 103, tion bias is a simulated research environment: An experimental study 263-283. ofscientific inference. Quarterly Journal ofExperimenal Psychology, KRULL, D. S. (1993). Does the grist change the mill? The effect ofthe 29,85-95. perceiver's inferential goal on the process of social inference. Per­ NEISSER, U. (1967). Cognitive psychology. Englewood Cliffs, NJ: sonality & Social Psychology Bulletin, 19, 340-348. Prentice-Hall. . KUNDA, Z. (1990). The case for motivated reasoning. Psychological NEWCOMB, T. M. (1929). The consistency ofcertain extrovert-introvert Bulletin, 108,480-498. behaviorpatterns in 51 problem boys. New York: Columbia Univer­ LAROCHE, M., & BRISOUX, J. E. (1989). Incorporating competition into sity, Teachers College, Bureau ofPublications. consumer behavior models: The case ofthe attitude-intention rela­ NISBETT, R, & Ross, L. (1980). Human inference: Strategies andshort­ tionship. Journal ofEconomic Psychology, 10,343-362. comings ofsocial judgment. Englewood Cliffs, NJ: Prentice-Hall. LEHMAN, D. R., KROSNICK, J. A., WEST, R L., & LI, F. (1992). The focus NORMAN, R (1976). When what is said is important: A comparison ofex­ ofjudgment effect: A question wording effect due to hypothesis con­ pert and attractive sources. Journal ofExperimental Social Psychology, firmation bias. Personality & Social Psychology Bulletin, 18,690-699. 12, 294-300. LEVIN, I. P., & GAETH, G. J. (1988). How consumers are affected by the OSKAMP, S. (1982). Overconfidence in case-studyjudgments. In D. Kah­ framing of attribute information before and after consuming the neman, P. Siovic, & A. Tversky (Eds.), Judgment under uncertainty: product. Journal ofConsumer Research, 15,374-378. Heuristics and biases (pp. 287-293). New York: Cambridge Univer­ LEVINE, M. (1966). Hypothesis behavior by humans during discrimi­ sity Press. nation learning. Journal ofExperimental Psychology, 71, 331-338. PAYNE, J. W., BETTMAN, 1. R, & JOHNSON, E. 1. (1992). Behavioral de­ LEVINE, M. (1970). Human discrimination learning: The subset­ cision research: A constructive processing perspective. Annual Re­ sampling assumption. Psychological Bulletin, 74, 397-404. view ofPsychology, 43, 87-131. LEWICKA, M. (1988). On objective and subjective anchoring ofcognitive PETTY, R E., & CACIOPPO, J. T. (1979). Issue involvement can increase acts: How behavioural valence modifies reasoning schemata. In W. 1. or decrease persuasion by enhancing message-relevant cognitive re­ Baker, L. P. Mos, H. V Rappard, & H. 1. Stam (Eds.), Recent trends in sponses. Journal ofPersonality & Social Psychology, 37, 1915-1926. theoretical psychology. New York: Springer-Verlag. PETTY, R E., & CACIOPPO, J. T. (1986). The elaboration likelihood model LICHTENSTEIN, S., FISCH HOFF, 8., & PHILLIPS, L. D. (1982). Calibration ofpersuasion. In L. Berkowitz (Ed.), Advances in experimental social ofprobabilities: The state ofthe art to 1980. In D. Kahneman, P. Slovic, psychology (Vol. 19, pp. 123-205). San Diego: Academic Press. & A Tversky (Eds.), Judgment under uncertainty: Heuristics and bi­ PLATT, J. R (1964). Strong inference. Science, 146, 347-353. ases (pp. 306-334). New York: Cambridge University Press. POSAVAC, S. S., SANBONMATSU, D. M., & FAZIO, R H. (1997). Consider­ LINVILLE, P. W. (1982). The complexity-extremity effect and age-based ing the best choice: Effects ofthe salience and accessibility ofalter­ stereotyping. Journal ofPersonality & Social Psychology, 42, 193-211. natives on attitude-decision consistency. Journal ofPersonality & So­ LocKSLEY, A., ORTIZ, v., & HEPBURN, C. (1980). Social categorization cial Psychology, 72, 253-261. and discriminatory behavior: Extinguishing the minimal intergroup PRICE, P. C., & YATES, J. F. (1993). Judgmental overshadowing: Further discrimination effect. Journal ofPersonality & Social Psychology, 39, evidence of cue interaction in contingency judgment. Memory & 773-783. Cognition, 21, 561-572. loPES, L. A. (1991). The rhetoric ofirrationality. Theory & Psychology, PRICE, P. c., & YATES, J. F. (1995). Associative and rule-based accounts 1,65-82. ofcue interaction in contingency judgment. Journal ofExperimental LORD, C. G., LEPPER, M. R, & PRESTON, E. (1984). Considering the op­ Psychology: Learning, Memory, & Cognition, 21,1639-1655. posite: A corrective strategy for social judgment. Journal ofPerson­ PvSZCZYNSKI, T. A, & GREENBERG, J. (1987). Toward an integration of ality & Social Psychology, 47,1231-1243. cognitive and motivational perspectives on social inference: A biased loRD, C. G., Ross, L., & LEPPER, M. R (1979). Biased assimilation and hypothesis testing model. In M. P. Zanna (Ed.), Advances in experi­ attitude polarization: The effects of prior theories on subsequently mental socialpsychology (Vol. 20, pp. 297-340). New York: Academic considered evidence. Journal ofPersonality & Social Psychology, 37, Press. 2098-2109. QUATTRONE, G. A (1982). Overattribution and unit formation: When LUCHINS, A S. (1942). Mechanization in problem solving. Psychological behavior engulfs the person. Journal ofPersonality & Social Psy­ Monographs, 54, No. 248. chology, 42, 593-607. MARR, D. (1982). Vision: A computational investigation into the human READ, S. J., VANMAN, E. J., & MILLER, L. C. (1997). Connectionism, representation andprocessing ofvisual information. San Francisco: parallel constraint satisfaction processes, and Gestalt principles: Freeman. (Re)introducing cognitive dynamics to social psychology. Personality McARTHUR, L. Z. (1981). What grabs you? The role ofattention in im­ & Social Psychology Review, 1,26-53. pression formation and causal attribution. In E. T. Higgins, C. P. Her­ RESTLE, F. (1962). The selection ofstrategies in cue learning. Psycho­ man, & M. P. Zanna (Eds.), Social cognition: The Ontario Symposium logical Review, 69, 329-343. (Vol. I, pp. 201-246). Hillsdale, NJ: Erlbaum. ROBINSON, L. B., & HASTIE, R. (1985). Revision ofbeliefs when a hy­ McKENZIE, C. R. M. ( 1994). The accuracy ofintuitive judgment strate­ pothesis is eliminated from consideration. Journal ofExperimental gies: Covariation assessment and Bayesian inference. Cognitive Psy­ Psychology, 11, 443-456. chology, 26, 209-239. Ross, L. (1977). The intuitive psychologist and his shortcomings: Dis­ MEDIN, D. L., GOLDSTONE, R. L., & MARKMAN, A. 8. (1995). Compar­ tortions in the attribution process. In L. Berkowitz (Ed.), Advances in ison and choice: Relations between similarity processes and decision experimental social psychology (Vol. 10, pp. 174-221). New York: processes. Psychonomic Bulletin & Review, 2, 1-19. Academic Press. MEHLE, T., GETTYS, C. F., MANNING, c., BACA, S., & FISHER, S. (1981). ROTHBART, M., EVANS, M., & FULERO, S. (1979). Recall for confirm­ The availability explanation of excessive plausibility assessments. ing events: Memory processes and the maintenance ofsocial stereo­ Acta Psychologica, 49,127-140. typing. Journal ofExperimental Social Psychology, 15, 343-355. MILLAR, M. G., & TESSER, A (1986). Thought-induced attitude change: RUNDUS, D. (1973). Negative effects of using list items as recall cues. The effects of schema structure and committment. Journal ofPer­ Journal ofVerbal Learning & Verbal Behavior, 12,43-50. sonality & Social Psychology, 51, 259-269. Russo, J. E., & DOSHER, 8. A. (1983). Strategies for multiattribute binary MILLWARD, R. 8., & SPOEHR, K. T. (1973). The direst measurement of choice. Journal ofExperimental P~ychology: Learning. Memory, & hypothesis-testing strategies. Cognitive Psychology, 4, 1-38. Cognition, 9, 676-696. MITCHELL, R. c., & CARSON, R. T. (EDS.) (1989). Using surveys to Russo, J. E., & KOLZOW, K. J. (1994). Where is the fault in fault trees? value public goods. Washington, DC: Resources for the Future. Journal ofExperimental Psychology: Human Perception & Perfor­ MYNATT, C. R, DOHERTY, M. E., & "IWENEY, R. D. (1977). Confirma- mance, 20, 17-32. SELECTIVE HYPOTHESIS TESTING 219

SADLER, 0., & TESSER, A (1973). Some effects of salience and time other people. In E. T. Higgins, C. P. Herman, & M. P. Zanna (Eds.), upon interpersonal hostility and attraction during social isolation. Social cognition: The Ontario Symposium (Vol. I, Pl'. 277-303). Sociometry, 36, 99-112. Hillsdale, NJ: Erlbaum. SAMPLES, K. c., & HOLLIER, J. R. (1990). Contingent valuation of SNYDER, M., & SWANN, W. B. (1978). Behavioral confirmation in so­ wildlife resources in the presence ofsubstitutes and complements. In cial interaction: From social perception to social reality. Journal of R. L. Johnson & G. V. Johnson (Eds.), Economic valuation ofnatural Experimental Social Psychology, 14,148-162. resources: Issues, theory and applications (Pl'. 177-192). Boulder, SRULL, T. K. (1981). Person memory: Some tests ofassociative storage CO: Westview. and retrieval models. Journal ofExperimental Psychology: Human SANBONMATSU, D. M., AKIMOTO, S. A., & BIGGs, E. (1993). Overesti­ Learning & Memory, 7, 440-462. mating causality: Attributional effects of confirmatory processing. SRULL, T. K., & WYER, R. S. (1979). The role ofcategory accessibility Journal ofPersonality & Social Psychology, 65, 892-903. in the interpretation of information about persons: Some determi­ SANBONMATSU, D. M., AKIMOTO, S. A, & GIBSON, B. D. (1994). Stereo­ nants and implications. Journal ofPersonality & Social Psychology, type-based blocking in social explanation. Personality & Social Psy­ 37,1660-1672. chology Bulletin, 20, 71-81. STANGOR, C., & McMILLAN, D. (1992). Memory for expectancy con­ SANBONMATSU, D. M., KARDES, E R., HOUGHTON, D. C., Ho, E. A, & gruent and expectancy incongruent information: A review ofthe social POSAVAC, S. S. (1998). Overweighing the given information in multi­ and social developmental issues. Psychological Bulletin, 111,42-61. attribute judgment. Manuscript submitted for publication. STRACK, E, & MUSSWEILER, T. (1997). Explaining the enigmatic an­ SANBONMATSU, D. M., POSAVAC, S. S., & STASNEY, R. (1997). The sub­ choring effect: Mechanisms ofselective accessibility. Journal ofPer­ jective beliefs underlying probability overestimation. Journal ofEx­ sonality & Social Psychology, 73, 437-446. perimental Social Psychology, 33, 276-295. STRACK, E, SCHWARZ, N., & WANKE, M. (1991). Semantic and prag­ SCHKADE, D. A., & PAYNE, J. W. (1993). Where do the numbers come matic aspects ofcontext effect in social and psychological research. from? How people respond to CV questions. In 1. A. Hausman (Ed.), Social Cognition, 9, 111-125. Contingent valuation: A critical assessment. New York: Elsevier, TAPLIN, J. E. (1975). Evaluation ofhypotheses in concept identification. North-Holland. Memory & Cognition, 3, 85-96. SCHUMAN, H., & PRESSER, S. (1981). Questions andanswers in attitude TAYLOR, S. E., & FISKE, S. T. (1978). Salience, attention, and attribu­ surveys. New York: Academic Press. tion: Top ofthe head phenomena. In L. Berkowitz (Ed.), Advances in SCHUSTACK, M. W., & STERNBERG, R. J. (1981). Evaluation ofevidence experimental social psychology (Vol. II, Pl'. 249-288). New York: in causal inference. Journal ofExperimental Psychology: General, Academic Press. 110,101-120. TEIGEN, K. H. (1974a). Overestimation of subjective probabilities. SCHWARTZ, B. (1982). Reinforcement-induced behavioral stereotypy: Scandinavian Journal ofPsychology, 15, 56-62. How not to teach people to discover rules. Journal ofExperimental TEIGEN, K. H. (I974b). Subjective sampling distributions and the addi­ Psychology: General, 111,23-59. tivity of estimates. Scandinavian Journal ofPsychology, 15, 50-55. SHAFIR, E. (1993). Choosing versus rejecting: Why some options are TEIGEN, K. H. (1983). Studies in subjective probability 111: The unim­ both better and worse than others. Memory & Cognition, 21, 546-556. portance of alternatives. Scandinavian Journal ofPsychology, 24, SHAKLEE, H., & FISCH HOFF, B. (1982). Strategies ofinformation search 97-195. in causal analysis. Memory & Cognition, 10, 520-530. l'ESSER, A. (1978). Self-generated attitude change. In L. Berkowitz (Ed.), SHAKLEE, H., & MIMS, M. (1982). Sources oferror in judging event co­ Advances in experimental socialpsychology (Vol. II, Pl'. I-57). New variations: Effects of memory demands. Journal ofExperimental York: Academic Press. Psychology: Learning, Memory, & Cognition, 8, 208-224. TRABASSO, T., & BOWER, G. H. (1968). Attention in learning: Theory SHERMAN, S. J., & CORTY, E. (1984). Cognitive heuristics. In R. S. and research. New York: Wiley. Wyer, Jr., & T. K. Srull (Eds.), Handbook ofsocial cognition (Vol. I, TROPE, Y. (1986). Identification and inferential processes in disposi­ 1'1'.189-286). Hillsdale, NJ: Erlbaum. tional attribution. Psychological Review, 93, 239-257. SHERMAN, S. J., McMuLLllN, M. N., & GAVANSKI, 1. (1992). Natural TROPE, Y, & BASSOK, M. (1983). Information gathering strategies in sample spaces and the inversion ofconditional judgments. Journal of hypothesis testing. Journal ofExperimental Social Psychology, 19, Experimental Social Psychology, 28, 401-421. 560-576. SHERMAN, S. J., SKOV, R. B., HERVITZ, E. E, & STOCK, C. B. (1981). TROPE, Y, BASSOK, M., & AWN, E. (1984). The questions lay inter­ The effects of explaining hypothetical future events: From possibil­ viewers ask. Journal ofPersonality, 52, 90-106. ity to probability to actuality and beyond. Journal ofExperimental TROPE, Y, & LIBERMAN, A (1996). Social hypothesis testing: Cognitive Social Psychology, 17, 142-158. and motivational mechanisms. In E. T. Higgins & A W. Kruglanski SHWEDER, R. A (1977). Likeness and likelihood in everyday thought: (Eds.), Social psychology,' Handbook ofbasic principles (Pl'. 239-270). Magical thinking in judgments about personality. Current Anthro­ New York: Guilford. pology, 18,637-658. TROPE, Y, & MACKIE, D. M. (1987). Sensitivity to alternatives in social SHWEDER, R. A, & D'ANDRADE, R. G. (1979). Accurate reflection or hypothesis testing. Journal ofExperimental Social Psychology, 23, systematic distortion? A reply to Block, Weiss, and Thorne. Journal 445-459. ofPersonality & Social Psychology, 37, 1075-1084. TVERSKY, A (1977). Features of similarity. Psychological Review, 84, SIMON, H. A. (1956). Rational choice and the structure ofthe environ­ 327-353. ment. Psychological Review, 63, 129-138. TVERSKY, A, & KAHNEMAN, D. (1973). Availability: A heuristic for SIMON, H. A. (1979). Rational decision making in business organizations. judging frequency and probability. Cognitive Psychology,S, 207-232. American Economic Review, 69, 493-513. TVERSKY, A., & KAHNEMAN, D. (1974). Judgment under uncertainty: SKOV, R. B., & SHERMAN, S. J. (1986). Information-gathering processes: Heuristics and biases. Science, 221, 453-458. Diagnosticity, hypothesis-confirmatory strategies, and perceived hy­ TVERSKY, A., & KAHNEMAN, D. (1981). The framing of decisions and pothesis confirmation. Journal ofExperimental Social Psychology, the psychology ofchoice. Science, 211, 453-458. 22,93-121. TVERSKY, A., & KAHNEMAN, D. (1992). Advances in prospect theory: SLOVIC, P., LICHTENSTEIN, S., & FISCHHOFF, B. (1985). Decision mak­ Cumulative representation of uncertainty. Journal ofRisk & Uncer­ ing. In R. C. Atkinson, R. 1. Hernstein, G. D. Lindzey, & R. D. Luce tainty, 5, 297-323. (Eds.), Handbook ofexperimental psychology (Vol. 2, Pl'· 673-738). TVERSKY, A, & KOEHLER, D. J. (1994). Support theory: A nonexten­ New York: Wiley. sional representation ofsubjective probability. Psychological Review, SMEDSLUND, J. (1963). The concept ofcorrelation in adults. Scandina­ 101,547-567. vian Journal ofPsychology, 4, 165-173. TwENEY, R. D., & DOHERTY, M. E. (1983). Rationality and the psy­ SNYDER, M. (1981). Seek, and ye shall find: Testing hypotheses about chology of inference. Synthese, 57, 139-161. 220 SANBONMATSU, POSAVAC, KARDES, AND MANTEL

TWENEY, R D., DoHERTY, M. E., WORNER, W. J., PLISKE, D. B., MYNATT, Advances in experimental social psychology (Vol. 27, pp. 177-254). C. R., GROSS, K. A., & ARKKELIN, D. L. (1980). Strategies ofrule dis­ San Diego: Academic Press. covery in an inference task. Quarterly Journal ofExperimental Psy­ VON NEUMANN, 1., & MORGENSTERN, O. (1947). Theory ofgames and chology,32,109-123. economic behavior. Princeton, NJ: Princeton University Press. VALLONE, R. P., GRIFFIN, D. W, LIN, S., & Ross, L. (1990). Overconfi­ WARD, W D., & JENKINS, H. M. (1965). The display ofinformation and dent prediction of future actions and outcomes by self and others. the judgment of contingency. Canadian Journal ofPsychology, 19, Journal ofPersonality & Social Psychology, 58, 582-592. 231-241. VALLONE, R P., Ross, L., & LEPPER, M. R. (1985). The hostile media WASON, P. C. (1960). On the failure to eliminate hypotheses in a concep­ phenomenon: Biased perception and perceptions of media bias in tual task. Quarterly Journal ofExperimental Psychology, 12, 129-140. coverage of the Beirut massacre. Journal ofPersonality & Social WASON, P. C. (1968). Reasoning about a rule. Quarterly Journal ofEx­ Psychology, 49, 577-585. perimental Psychology, 23, 273-281. VAN DER PLIGHT, J., ElSER, J. R, & SPEARS, R (1987). Comparative WASSERMAN, E. A., DORNER, W W, & KAO, S. F. (1990). Contributions judgements and preferences: The influence ofthe number ofresponse of specific cell information to judgments of interevent contingency. alternatives. British Journal ofSocial Psychology, 26, 269-280. Journal ofExperimental Psychology: Learning, Memory, & Cognition, VAN WALLENDAEL, L. R, & HASTIE, R. (1990). Tracing the footsteps 16, 509-521. ofSherlock Holmes: Cognitive representations ofhypothesis testing. YZERBYT, V. Y, SCHANDRON, G., LEYENS, J. P., & ROCHER, S. (1994). Memory & Cognition, 18,240-250. Social judgeability: The impact of meta-informational cues on the VON HIPPEL, W., 10NIDES, J., HILTON, 1. L., & NARAYAN, S. (1993). In­ use ofstereotypes. Journal ofPersonality & Social Psychology, 66, hibitory effect ofschematic processing on perceptual encoding. Jour­ 48-55. nal ofPersonality & Social Psychology, 64, 921-935. VON HIPPEL, W., SEKAQUAPTEWA, D., & VARGAS, P. (1995). On the role (Manuscript received May 19, 1996; ofencoding processes in stereotype maintenance. In M. P. Zanna (Ed.), revision accepted for publication October 21, 1997.)