Practical Advice for Conducting Ethical Online Experiments and Questionnaires for United States Psychologists
Total Page:16
File Type:pdf, Size:1020Kb
Behavior Research Methods 2008, 40 (4), 1111-1128 doi: 10.3758/BRM.40.4.1111 Practical advice for conducting ethical online experiments and questionnaires for United States psychologists KIMBERLY A. BARCHARD University of Nevada, Las Vegas, Nevada AND JOHN WILLIAMS University of Northern Iowa, Cedar Falls, Iowa It is increasingly easy and, therefore, increasingly common to conduct experiments and questionnaire stud- ies in online environments. However, the online environment is not a data collection medium that is familiar to many researchers or to many research methods instructors. Because of this, researchers have received little information about how to address ethical issues when conducting online research. Researchers need practical suggestions on how to translate federal and professional ethics codes into this new data collection medium. This article assists United States psychologists in designing online studies that meet accepted standards for informed consent, deception, debriefing, the right to withdraw, security of test materials, copyright of participants’ materi- als, confidentiality and anonymity, and avoiding harm. Online psychological research is inexpensive, can result sent electronically from the participant’s computer to the in large sample sizes and high statistical power, and allows researcher’s computer, whereas in many in-person stud- researchers to include participants from distant geographical ies, the participant completes measures on paper. Without areas (Birnbaum, 2004a; Reips, 2000). Across a wide range careful consideration of the implications of such method- of variables, online studies lead to the same conclusions as ological differences, researchers who are new to online laboratory studies (Krantz & Dalal, 2000), but they can also research (and the number of such researchers increases have greater external validity and generalizability than does each year) are likely to design studies that fail to meet ac- laboratory research (Reips, 2000). Finally, the widespread cepted standards (Mathy, Kerr, & Haydin, 2003). availability of easy-to-use Web development software and Fortunately, the ethical issues facing online psycho- programs designed specifically to create online surveys (see logical research are the same as the issues faced by in- Table 1) makes creating and running online studies rela- person psychological research, and therefore, the same tively easy, requiring little technical knowledge (Barchard ethics guidelines can assist us. This article will focus on & Pace, in press). For all these reasons, online psychological the guidelines that apply to most psychological research in research is becoming relatively common. The purpose of the United States: Chapter 45 of the Code of Federal Reg- this article is to provide psychologists with practical advice ulations (CFR), Part 46: Protection of Human Subjects for conducting online studies in an ethical manner. (CFR 46, 2001); Title 17, Copyright Laws (Copyright The number of online studies is large and increasing Law, 2003); and the American Psychological Association’s dramatically. In 2004, over 278 Internet studies were ad- (APA’s) Ethical Principles of Psychologists and Code of vertised on laboratory Web sites (Birnbaum, 2004a). On Conduct (American Psychological Association, 2002). one site, Birnbaum (2004a) found a 425% increase from Not all laws and codes are relevant to ethical issues, and 1998 to 2003. In addition, there are Web page studies that no one ethical system (e.g., utilitarianism, etc.) is adopted are advertised in other ways, e-mail studies, and observa- by all United States researchers; we have selected these tional studies in which researchers join already existing laws and codes because they apply to all psychologists groups, and these types of studies have also been increas- doing research in the United States and are particularly ing. Given the cost and time savings of collecting data elec- relevant to ethical issues in online research. This article tronically, online research will likely continue to increase goes beyond previous discussions on creating ethical on- in popularity. line studies (e.g., Birnbaum, 2004b; Ess, 2002, 2007; Ess Online studies differ in many ways from in-person stud- & the Association of Internet Researchers, 2002; Fran- ies conducted in research labs. For example, data are often kel & Siang, 1999; Kraut et al., 2004; Nosek, Banaji, & K. A. Barchard, [email protected] 1111 Copyright 2008 Psychonomic Society, Inc. 1112 BARCHARD AND WILLIAMS Table 1 Computer Programs for Creating Online Studies Program Developer Available From Easy-to-Use Web Development Software Amaya (free) W3C www.w3.org/Amaya/ Authorware 7 Adobe www.adobe.com/products/authorware/ Dreamweaver CS4 Adobe www.adobe.com/products/dreamweaver/ First Page 2006 EvrSoft www.evrsoft.com/ Flash CS4 Adobe www.adobe.com/products/flash/ SharePoint Web Designer 2007 Microsoft office.microsoft.com/en-us/sharepointdesigner/default.aspx Expression Web 2 Microsoft www.microsoft.com/expression/ Programs Specifically Designed to Create Online Surveys and/or Web Experiments EventHandler 4.0c UbiDog Productions www.event-handler.com/ LiveCycle Designer ES Adobe www.adobe.com/products/livecycle/designer/ mrInterview SPSS www.spss.com/mrinterview/ SurveySolutions/EFM Perseus Development Corporation www.perseus.com/ SurveyMonkey SurveyMonkey.com www.surveymonkey.com/ The Survey System Creative Research Systems www.surveysystem.com/ SurveyWiz (free) Birnbaum psych.fullerton.edu/mbirnbaum/programs/surveyWiz.htm WEXTOR (free for educational and noncommercial uses) Reips & Neuhaus psych-wextor.unizh.ch/wextor/en/ Greenwald, 2002; Peden & Flashinski, 2004; Wood, Grif- fit this profile: Identifying information might be col- fiths, & Eatough, 2004) by including the APA Ethics Code lected, sensitive questions might be asked, or the study and U.S. Copyright Laws and by providing specific steps advertisement might be seen by children or adults who for addressing each ethical issue. Psychologists should are not legally able to provide consent. This article pro- keep in mind that their local institutional review board vides practical advice about how to design and run those (IRB) or administrative unit may require that additional types of studies and how to convince your local IRB that steps be taken to meet the standards presented here or may you have adequately addressed the ethical issues that are require adherence to additional standards. raised by your study design. We first will discuss the dif- A single article cannot adequately address the ethical ferences between such studies when they are conducted issues facing every possible type of online study. Online online and when they are conducted in person, and then research includes questionnaires and tests, interviews, and we will discuss how to ensure that an online study is naturalistic observation and can be conducted through Web ethical. Wherever possible, we will include several sug- pages, e-mails, chat rooms, bulletin boards, and graphics- gestions that address a particular ethical issue, so that based simulations (Kraut et al., 2004). It includes topics researchers can select the technique that is most effective as wide-ranging as online shopping (Byrom & Medway, for their particular research project, while minimizing the 2004), psychometric equivalence (Buchanan, Johnson, & influence on data quality. These discussions may provide Goldberg, 2005; Meyerson & Tryon, 2003), drug dealing a model for how to think about ethical issues as technol- (Coomber, 1997), hate crimes (Glaser, Dixit, & Green, ogy changes over time. 2002), sexsomnia (Mangan & Reips, 2007), and eating disorder support groups (Walstrom, 2004). This article DIFFERENCES BETWEEN ONLINE focuses on one common type of online psychological RESEARCH AND IN-PERSON RESEARCH research, in which legally competent adults are explic- itly invited to participate in research. Examples of such Online psychological research differs from in-person studies can be found on the University of Mississippi’s psychological research in many ways (Mathy et al., 2003), (2000) Psych Experiments Web site, Hanover College’s and many of these differences present challenges to de- (2008) Psychological Research on the Net Web site, the signing an ethical study. The first obvious difference is Social Psychology Network’s (1996–2008) social psy- that researchers usually have no direct contact with par- chology research Web site, and two Web sites at the Uni- ticipants. Ethically, there are two benefits of lack of direct versity of Zurich: the Web Experimental Psychology Lab contact. It is easier to ensure complete anonymity, which (Reips, 1995–2008) and the Web Experiment List (Reips, may explain why online studies obtain higher response 2001–2008). rates on sensitive questions (McCabe, 2004). And partici- One relatively common type of online study poses few pants do not feel as much social pressure to stay, which ethical challenges. This is a study in which anonymous increases their ability to discontinue participation if they participants complete rating scales that do not address become uncomfortable (Birnbaum, 2004b; Fricker & any sensitive issues and that is advertised in such a way Schonlau, 2002; Kaplowitz, Hadlock, & Levine, 2004). that the only people who will see the study advertisement Lack of direct contact does, however, present three chal- are legally competent adults. This type of minimal risk lenges to designing an ethical study. First, researchers