Download Preprint
Total Page:16
File Type:pdf, Size:1020Kb
Demand characteristics - 1 Sixty years after Orne’s American Psychologist article: A conceptual analysis of “Demand Characteristics” Olivier Corneille UCLouvain & Peter Lush University of Sussex [Preprint] Words count: 14188 (excluding references) Running title: Demand characteristics. Keywords: Demand characteristics, Social Influence, Methods, Attitudes, Evaluative Conditioning, Rubber Hand Illusion. Contact author: Olivier Corneille, UCLouvain, PSP_IPSY, 10 Place du Cardinal Mercier. B1348 LLN. Belgium. Email: [email protected] Funding: The preparation of this article was supported by an FRS-FNRS grant (T.0061.18) awarded to Olivier Corneille. Author Note: We are grateful for the comments we received from the members of the LIPLab (University of Ghent) and Zoltan Dienes (University of Sussex). Demand characteristics - 2 ABstract The present article proposes a critical discussion of “demand characteristics”. This notion, despite its longstanding and widespread use in behavioral research, suffers from significant conceptual ambiguities. After briefly discussing the historical roots of “demand characteristics” and stressing that this notion remains highly influential in psychological research (Section 1), we proceed to its conceptual analysis (Section 2). Here, we discuss the complex nature of “demand characteristics” regarding effects, mechanisms, and measures. Then, we propose a Multifaceted Demand Characteristics Model (Section 3), and we apply it to two research domains of high contemporary interest: Evaluative Conditioning (Section 4.1) and Rubber Hand Illusion (Section 4.2). In the General Discussion, we summarize the central insights of the current analysis and discuss when and how theorization around “demand characteristics” may prevent and help advance behavioral research and interventions. Demand characteristics - 3 Sixty years after Orne’s American Psychologist article: A Conceptual Analysis of “Demand Characteristics” Supply always comes on the heels of demand Robert Collier Introduction: Because one of our main points is that “demand characteristics” bears several meanings, it is uneasy to start with its definition. We may, however, first approximate it by defining “demand characteristics” as “the totality of task-orienting cues that govern subjects’ hypotheses about role expectations.” (Rosenthal & Rosnow, 2009, p. 781). “Demand characteristics” are commonly seen as experimental artifacts that weaken the validity of a study, as a “methodological issue” (Nichols & Edlund, 2015). In all likelihood, the notion is discussed in most introductory courses on behavioral methods. However, we will see that this notion has become much more complicated and contradictory over the years. It involves cues situated inside but also outside the lab. It applies to experimental but also to non-experimental settings. And it relates to both undesirable and desirable effects. Despite the widespread and longstanding use of this notion, surprisingly little empirical, theoretical, and conceptual work has been delivered on it. As Nichols and Maner (2008) rightly pointed out in one of the few empirical examinations of the phenomenon: “few empiricists have directly examined effects of demand on participant behavior or sought to identify factors that may increase or decrease the likelihood of demand-induced responding. » (p. 159). Likewise, no consensual definition of “demand characteristics” exists. This contrasts with the use of other psychological constructs whose meaning has been recently questioned, such as “social priming” (Sherman & Rivers, 2021), “implicit attitudes” (Corneille & Hütter, 2020), “impulsivity” (Stahl et al., 2014; Strickland & Johnson, 2020), “depression” (Santor, Demand characteristics - 4 Gregus, & Welch, 2006), or “interoceptive sensibility” (Desmedt et al., 2021) that are associated with a wealth of empirical studies and theoretical efforts. To understand this peculiarity, it is important first to consider the historical roots of “demand characteristics”. We do so in the first section of this article (see Section I: “Historical roots of demand characteristics”). Because a thorough historical analysis of this notion has been proposed by Sharpe and Whelton (2016), we keep this first section relatively short. We highlight, however, the high contemporary relevance of this old notion. In particular, we discuss how the current replication crisis essentially reproduces (no pun intended) heated debates that shook experimental social psychology research early on, following the publication of Orne’s (1962) article on “demand characteristics”. In the next section, we proceed to a conceptual analysis of “demand characteristics” (see Section II: A Conceptual analysis of “demand characteristics”). Here, we point out that this notion’s polysemous and underspecified nature may generate unnecessary conflicts in psychological research. Mechanisms and measurement issues are also discussed in this second section. Next (Section 3: “A Conceptual model of “demand characteristics”), we present and discuss a conceptual model of this notion: The Multifaceted Demand Characteristics Model (see Figure 1). For illustrative purpose, we apply this model to contemporary research on Evaluative Conditioning and Rubber Hand Illusion (see Section IV: “An application of the model to Evaluative Conditioning and Rubber Hand Illusion research”). In the General Discussion, we summarize the central insights of the current analysis and discuss when and how theorization around “demand characteristics” may prevent but also help advance behavioral research and interventions. I. Historical roots of “demand characteristics” Although the concept of demand characteristics was initially developed by Orne (1959) to address specific issues in hypnosis research, wider interest in demand characteristics Demand characteristics - 5 stems from Orne's later observation that research participants are often willing to comply with experimental instructions to complete boring and noxious tasks. Orne (1962) reasoned that study participants generally want to be helpful to the experimenter. They want to be "good subjects" and infer that, if they are instructed to complete the tasks, this means the tasks are worthwhile. The fact that participants form sensible expectations about the experimenter (i.e., "the experimenter is competent and considerate, so the task must be valuable") is reminiscent of the research on conversation rules (Grice, 1975) and, more generally, of the pragmatic approach to communication (Sperber & Wilson, 1986). To illustrate, when asked: "Can you hand me the pepper?", sensible respondents will go beyond the immediate meaning of the question and interpret it as an implicit request to hand on the pepper. The pragmatic approach to communication has proved influential in social psychological research. It is known to influence measurement (e.g., Schwartz, 2010). The fact that “demand characteristics” have much to do with communication pragmatics, and bear major implications for measurement, is evident in this quotation borrowed from Orne (1962): “(…) if a test is given twice with some intervening treatment, even the dullest college student is aware that some change is expected, particularly if the test is in some obvious way related to the treatment.” (Orne, 1962, p. 779). At the time, however, Orne’s theorization was inspired by the work of Gestalt psychologists. According to Gestalt theory, individuals inform their judgments and decisions by integrating cues present in their environment (see in particular Brunswik’s lens model). This integration is thought to be holistic (i.e., the total is more than the sum of the parts). When applied to an experimental situation, a Gestalt theorization suggests that participants integrate more cues than the experimenter might think when interpreting their role in the experiment: “The response to any specific set of stimuli, then, is a function of both the stimulus and the subject’s recognition of the total context.” (Orne, 2009, p. 111). According Demand characteristics - 6 to this analysis, any experimental situation is a problem (Rosenzweig, 1933) that participants attempt to solve by making inferences from a large set of cues, with the generally benevolent intention to help the experimenter. This analysis upset Orne’s contemporaries. First, the reference to participants’ phenomenology was considered heresy by the then-dominant behaviorist approach. The idea that “it is essential for the investigator to understand how a particular experimental situation is perceived by the subject to draw sensible inference from the subject's responses” (Orne, 1973, p. 157) seemed inconsistent with behavioral investigations taking place at the time, for instance on classical and operant conditioning (see Section 4.1). Second, and more dramatically, Orne had opened a box of worms that questioned the validity of influential social psychological studies completed at the time, including Berkowitz’ studies on the “weapons effect” (e.g., Page & Scheidt, 1971), Milgram’s studies on obedience (e.g., Orne & Holland, 1968), Zimbardo’s prison experiment (e.g., Banuazizi & Movahedi, 1975), and Staats and Staats’s studies on the classical conditioning of attitudes (Page, 1969). In particular, Orne questioned the fact that participants in Milgram’s obedience studies were deceived. Instead, he suggested that these participants may have played a role - occasionally, one that involved looking torn and anxious. This interpretation infuriated