Selective Hypothesis Testing

Psychonomic Bulletin & Review 1998,5(2), 197-220 Selective hypothesis testing DAVID M. SANBONMATSU and STEVEN S. POSAVAC University ofUtah, Salt Lake City, Utah FRANK R. KARDES University ofCincinnati, Cincinnati, Ohio and SUSAN p. MANTEL University ofToledo, Toledo, Ohio A diverse set ofbiases that have been found to characterize judgment may be similarly mediated by a process ofselective hypothesis testing. Ourpaper begins with a definition ofselective hypothesis test ing and an explanation ofhow andwhen this process leads to error. We then review a diverse and often disconnected set offmdings in the person perception, judgment, cognition, attitudes, attribution, and rule discovery literatures thatcanbe explained by this process, Finally, we examine the question ofwhy the selective testing of hypotheses occurs. Although the psychological literature suggests that selec tive hypothesis testing contributes to a variety of errors, in many contexts it may be a useful and effi cient strategy that leads to satisfactory judgment. The proliferation ofresearch on human judgment dur Although a vast body of research is reviewed that indi ing the last three decades has led to the identification of cates that a multitude oferrors may result from selective a multiplicity of errors that may characterize the obser hypothesis testing, in general this judgmental strategy is vations, interpretations, and evaluations ofeveryday life not necessarily a bad one. As we shall suggest, in many (see, e,g" Kahneman & Tversky, 1982; Nisbett & Ross, contexts the selective consideration ofhypotheses is a use 1980; Sherman & Corty, 1984). Every important form of ful and efficient strategy that leads to adaptive judgment. judgment-whetherit be prediction, categorization, causal attribution, covariation estimation, orattitude formation What Is Selective Hypothesis Testing? has been found to be susceptible to a variety ofbiases, Al In judgmentunder conditions ofuncertainty, an appro though the number and range ofjudgmental errors expli priate judgmental response is not readily apparent, given cated by researchers has been impressive, the development the available evidence. For our purposes, a hypothesis is of general frameworks and models for understanding a perceived possible estimation, interpretation, evaluation, these varied phenomena has not. Thus, in recent years, re explanation, rule, or solution. More specifically, hypothe searchers (e,g., Arkes, 1991; Fiedler, 1996; Klayman & ses are beliefs about the possible relation between vari Ha, 1987; Koehler, 1991; Kruglanski & Webster, 1996; ables, values, objects, or events. The hypothesized relation, Payne, Bettman, & Johnson, 1992) have begun making an ofcourse, may take a variety offorms. For example, a hy effort to develop frameworks that integrate the various pothesis may concern the correlation between variables, errors and biases that have been identified in the litera the causal relation between events, the favorableness or ture and that cut across different judgmental categories. characteristics ofan object, or the likelihood ofan event. What is becoming increasingly apparent to researchers The data or evidence available for judgment are often is that similar processes often underlie seemingly differ scattered and complex. The generation and testing ofhy ent categories ofjudgmental errors, In the present review, potheses facilitates judgment by guiding the gathering we suggest that many of the important biases that have and integration of information and by increasing the been identified in the cognition, person perception, attri speed with which a conclusion or inference can be drawn. bution, judgment, and attitudes literatures are related, in Hypothesis testing is basically a top-down approach en that they share a process of selective hypothesis testing, tailing selective processing that contrasts with a more data driven strategy in which inferences emerge from the avail able evidence. The authors thank Reid Hastie, Bill von Hippel, and two anonymous reviewers for making a number of excellent suggestions and asking a In a classic paper on scientific reasoning, Platt (1964) lot of hard questions. F.R.K. is in the Department of Marketing at argued that a comparison ofalternative hypotheses should Cincinnati. S.P.M. is in the Department of Marketing at Toledo. S.S.P. guide inductive inference. Strong inference begins with is now in the Department of Marketing in the Simon Graduate School the generation of plausible alternatives, followed by a ofBusiness Administration at the University of Rochester. Correspon dence concerning this paper should be addressed to D. M. Sanbon systematic gathering ofdiagnostic data that distinguishes matsu, Department of Psychology, University of Utah, Salt Lake City, the merits ofeach. According to Platt, the direct compar Utah 84112 (e-mail: [email protected]). ison of competing hypotheses allows for more rapid 197 Copyright 1998 Psychonomic Society, Inc. 198 SANBONMATSU, POSAVAC, KARDES, AND MANTEL development of knowledge-in particular, scientific is distinct from selective hypothesis testing in that the knowledge-than do other methods of inference. Of accepted hypothesis is the one that is perceived to be the course, Platt is not the only researcher to advocate the most plausible or preferred ofthe alternatives compared. consideration ofalternatives in human reasoning. Social Note that, in both processes, a set ofcriteria or assessment observers have long admonished people to consider the rules are applied. However, in comparative processing, the opposite or consider the alternatives before arriving at a criteria are used to compare the competing alternatives. conclusion (see, e.g, C. A. Anderson & Sechler, 1986; All judgments are explicitly or implicitly comparative. Arkes, 1991; Bacon, 1960; Hand, 1951; Hirt & Markman, However, in selective hypothesis testing, alternatives are 1995; Lord, Lepper, & Preston, 1984). not directly compared in order to arrive at a comparative These admonishments appear to be ineffectual in guid conclusion. Take, for example, the task ofpredicting the ing human judgment, for there exists ample evidence sug winner ofthe NBA championship. Comparative judgment gesting that comparative testing ofthis sort is not always would begin with the generation ofa list ofviable teams, performed. Instead ofthe identification and evaluation of followed by a systematic comparison ofthe strengths and plausible alternatives, selective hypothesis testing often oc weaknesses ofeach on the most relevant attribute dimen curs; individual hypotheses are tested serially, until one is sions. In contrast, in selective hypothesis testing, a partic found that meets the judgmental requirements. In selective ular team, such as the Chicago Bulls, might be nominated hypothesis testing, possibilities are assessed one at a time. and assessed. If the Bulls appear to have championship A focal hypothesis is generated on the basis ofsome crite qualities, the judgment that they will win is rendered. A ria or set ofstandards. The focal hypothesis, in turn, serves direct comparison of competing teams is not necessary, to guide the gathering and assimilation ofevidence. When ifa representation ofa championship team (i.e., knowledge the evidence for the focal hypothesis is sufficient or satis ofthe qualities ofa champion) is available to serve as the ficing (Simon, 1956, 1979), confirmation occurs, and the basis for confirmation. search for further information is truncated. Conversely, if Selective hypothesis testing sometimes is characterized the gathered evidence fails to support the hypothesis, a by the serial assessment of a multiplicity of different new alternative is generated. The process oftesting and re possibilities. An iterative process takes place, in which generating continues until a suitable response is found or focal hypotheses are repeatedly discarded and progres until the perceived benefits of gathering additional evi sively refined until a response is generated that satisfac dence are outweighed by the costs (Gettys & Fisher, 1979). torily accounts for the gathered evidence. On occasion, this In selective hypothesis testing, the generation ofalter process circles around to a reconsideration of the origi natives, the gathering of evidence, and inferences about nal hypothesis. In other instances, the initial hypothesis the meaning ofthe evidence are guided by a set ofcriteria, is recognized to be a mere starting point, from which the conditional rules, or judgmental standards. A necessary generation, testing, and revision process begins. This sort component of the criteria is a confirmation threshold, ofiterative process is especially likely to operate when a which amounts to the evidential requirements for con quantitative estimation is sought. Take the example ofa firmation. In selective hypothesis testing, the rejection or task that calls for the approximate age of a junior male acceptance ofa hypothesis is determined by whether the colleague. Selective hypothesis testing may begin with gathered evidence meets the confirmation criteria. the hypothesis that the colleague is around 30 but be ad In contrast, in comparativejudgment, a multiplicity of justed significantly upward to 45 with knowledge ofhis hypotheses are generated and then compared with one three children, then revised slightly downward with con another. A hypothesis must fulfill two requirements to sideration of his full head of hair, and so

Load more