University of Groningen Do Our Psychological Laws Apply Only To
Total Page:16
File Type:pdf, Size:1020Kb
University of Groningen Do Our Psychological Laws Apply Only to College Students? Stroebe, Wolfgang; Gadenne, Volker; Nijstad, Bernard A. Published in: Basic and Applied Social Psychology DOI: 10.1080/01973533.2018.1513362 IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below. Document Version Publisher's PDF, also known as Version of record Publication date: 2018 Link to publication in University of Groningen/UMCG research database Citation for published version (APA): Stroebe, W., Gadenne, V., & Nijstad, B. A. (2018). Do Our Psychological Laws Apply Only to College Students? External Validity Revisited. Basic and Applied Social Psychology, 40(6), 384-395. https://doi.org/10.1080/01973533.2018.1513362 Copyright Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons). The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license. More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne- amendment. Take-down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim. Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum. Download date: 03-10-2021 Basic and Applied Social Psychology ISSN: 0197-3533 (Print) 1532-4834 (Online) Journal homepage: https://www.tandfonline.com/loi/hbas20 Do Our Psychological Laws Apply Only to College Students?: External Validity Revisited Wolfgang Stroebe, Volker Gadenne & Bernard A. Nijstad To cite this article: Wolfgang Stroebe, Volker Gadenne & Bernard A. Nijstad (2018) Do Our Psychological Laws Apply Only to College Students?: External Validity Revisited, Basic and Applied Social Psychology, 40:6, 384-395, DOI: 10.1080/01973533.2018.1513362 To link to this article: https://doi.org/10.1080/01973533.2018.1513362 © 2019 The Author(s). Published with license by Taylor and Francis Group, LLC. Published online: 15 Oct 2018. Submit your article to this journal Article views: 141 View Crossmark data Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=hbas20 BASIC AND APPLIED SOCIAL PSYCHOLOGY 2018, VOL. 40, NO. 6, 384–395 https://doi.org/10.1080/01973533.2018.1513362 Do Our Psychological Laws Apply Only to College Students?: External Validity Revisited Wolfgang Stroebea, Volker Gadenneb, and Bernard A. Nijstada aUniversity of Groningen; bUniversity of Linz ABSTRACT That most psychological research is conducted with students led to concerns that psycho- logical laws apply only to this population. These fears are based on Campbell and Stanley’s concept of external validity that specifies the extent to which research findings can be gen- eralized. This concept is based on an inductivist philosophy. As philosophers of science have argued since Hume, one cannot derive general laws from singular observations. Instead, one develops theories and uses empirical studies to test these theories. This solves the problem of generalization because the domain of applicability is specified by the theory. Reports that studies result in different findings when conducted in different cultures are unproblematic as long as these differences can be explained with psychological theories. There seems to be a growing concern in psychology This renewed interest for the (lack of) diversity in about the lack of diversity in the subject populations subject populations stems partly from cross-cultural on which we test our theories. For example, Henrich, research showing that psychological findings often do Heine, and Norenzayan (2010) criticized in a widely not generalize across cultures (e.g., Henrich et al., cited article that “behavioral scientists routinely pub- 2010). It also partly stems from a sincere desire to lish broad claims about human psychology and behav- improve scientific practices within psychology, along ior. based on samples drawn entirely from Western, with more attention for statistical power, replication, Educated, Industrialized, Rich, and Democratic and transparency (e.g., Funder et al., 2014). For (WEIRD) societies” (p. 61). Arnett (2008) likewise example, referring to the replication crisis, Kitayama noted that most conclusions from psychological (2017) noted, “Ultimately, I believe that this inten- — research are based on American samples and that tional expansion of the subject base not only in size — conclusions that “apply to all human beings” have to but also in diversity in our science is the best step ” be “based on studies of diverse sectors of the human toward addressing the challenges we face today (p. population” (p. 602). It has even been suggested that 359). Despite these good intentions, in this article we this lack of diversity may be at the root of our so- argue that the assumption that the populations on called replication crisis.1 Thus, Simons, Yuichi, and which we test our theories need to be representative Lindsay (2017) argued that replication failures could of the population to which these theories are applied “ ” often be due to researchers failing to sample from the is wrong. As a consequence, blindly (i.e., without same subject population that had been employed in theory) replicating findings among different popula- the original study. Along similar lines, Kitayama tions is likely a waste of resources. (2017), as the incoming editor of Journal of Personality and Social Psychology (Attitudes and Social The notion of external validity Cognition section), stated in his editorial that “if the The erroneous belief that, in scientific research, sub- same effects occur in a sample that is very different ject populations need to be “representative” has a long from the original one, this will constitute a ‘big plus’ tradition in scientific discourse but received methodo- that could bring the paper above the threshold for logical justification through the concept of external publication in JPSP:ASC” (p. 359). validity. This concept was developed originally by CONTACT Wolfgang Stroebe [email protected] Department of Social and Organizational Psychology, University of Groningen, Grote Kruisstraat 2/1, 9712 TS Groningen, The Netherlands. ß 2019 The Author(s). Published with license by Taylor and Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by- nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. BASIC AND APPLIED SOCIAL PSYCHOLOGY 385 Campbell in an article in Psychological Bulletin in the general statement. Under these conditions, the 1957 but became popular through a little booklet enti- inference from the observation statement to the general tled “Experimental and Quasi-Experimental Designs statement is regarded as valid or justified. for Research” (Campbell & Stanley, 1966). This little The first inductive principle is unproblematic, booklet—the reprint of a chapter that Campbell and because it is concerned only with the question of how Stanley wrote for the Handbook of Research on to generate theories, not with their truth or justifica- Teaching (Gage, 1963)—was one of the most influen- tion. Induction may be used as a heuristic for theory tial methodological contributions to psychological building. The usefulness of induction for that purpose research ever published. It constituted compulsory is, however, controversial: Some consider it as very reading for generations of graduate students. Partly useful, but others point out that explanatory theories, due to its accessible writing, it probably contributed which refer to unobservable structures and processes, more to the improvement of research in the social sci- can hardly be created by generalizing observable states ences than any other methodological publication. of affairs. But we need not discuss this here, as the Campbell and Stanley (1966) popularized the dis- problem of induction is not mainly associated with tinction between two types of validity, namely, the formation of theories. internal validity and external validity. They stated that It is the second principle that was famously internal validity is the basic minimum without which criticized by Hume (1748; see also Popper, 1959). An any experiment is uninterpretable: Did in fact the inference from singular statements to a general experimental treatments make a difference in this statement cannot be founded on deductive logic. specific experimental instance? External validity asks Moreover, the attempt to justify induction by arguing the question of generalizability: To what populations, settings, treatment variables, and measurement that induction has proved successful in the past is cir- variables can this effect be generalized? (Campbell & cular. An inference from past success to future success