?f ~P~f ST~~YI~~ ?f ~f~f Artifacts and Ethics in Behavioral Research Ralph L. Rosnow Temple University Robert Rosenthal Harvard University II w. H. FREEMAN AND COMPANY NEW YORK ACQUISITIONS EDITOR: Susan Finnemore Brennan We are as sailors who are forced to rebuild their ship on the open sea, PROJECT EDITOR: Christine Hastings without ever being able to start fresh from the bottom up. Wherever COVER AND TEXT DESIGNER: Victoria Tomaselli a beam is taken away, immediately a new one must take its place, ILLUSTRATION COORDINATOR: Bill Page PRODUCTION COORDINATOR: Maura Studley and while this is done, the rest of the ship is used as support. In this COMPOSITION: W. H. Freeman Electronic Publishing Center/Susan Cory way, the ship may be completely rebuilt like new with the help of the MANUFACTURING: RR Donnelley & Sons Company old beams and driftwood-but only through gradual rebuilding. 1 Library of Congress Cataloging-in-Publication Data Rosnow, Ralph L. People studying people: artifacts and ethics in behavioral research/ Ralph L. Rosnow, Robert Rosenthal. p. cm. Includes bibliographical references and indexes. ISBN 0- 7167-3070-7. -ISBN 0-7167- 3071-5 (pbk.) 1. Psychology-Research. 2. Social sciences-Research. I. Rosenthal, Robert, 1933- II. Title BF76.5.R645 1997 150'.7'2-dc21 96-48978 CIP © 1997 by W. H. Freeman and Company No part of this book may be reproduced by any mechanical, photographic, or electronic process, or in the form of a phonographic recording, nor may it be stored in a retrieval system, transmitted, or otherwise copied for public or private use, without written permission from the publisher. Printed in the United States of America First printing, 1997 Contents Preface ix Rrtif acts in Behavioral Research l The Nature of Artifacts 1 Clever Hans 3 The Hawthorne Studies S Rosenzweig's Critique 7 Resistance to the Artifact Idea 9 Random Versus Systematic Error 11 Biasing Effects of Investigators 13 Interactional and Noninteractional Bias 13 The Investigator as Observer 16 Interpretation of Data 18 Intentional Error 21 Biosocial Attributes of Investigators 24 Psychosocial Attributes of Investigators 2 7 Situational Factors 28 Modeling Effects 32 Coping with Investigator Biases 36 Expectations as Self-Fulnlling Prophecies 41 Expectancy Bias 41 A Sample of Studies 43 The First 34S Studies 46 Moderation of Interpersonal Expectancies S1 Mediation of Interpersonal Expectancies S2 Teacher Expectancy Effects S4 Controls for Expectancy Effects SS Expectancy Control Design S9 The Person Behind the Look 63 Demand Characteristics 63 The "Good Subject" Effect 65 Evaluation Apprehension 68 The Obedient Subject 69 The Dayyan's Decree 72 Mediation of Participation-Related Artifacts 77 Minimizing Experimental Artifacts 82 Detecting the Occurrence of Artifacts 85 The Volunteer Subject 89 io Why Study the Volunteer Subject? 89 SU Quantifying Volunteer Bias 91 sp Reliability of Volunteering 94 ke Comparing Volunteers and Nonvolunteers 96 iz Characteristics of the Volunteer Subject 97 Situational Determinants of Volunteering 102 al~ Volunteer Status and Research Outcomes 105 ing Power and the Pseudovolunteer 107 mu Predicting the Direction of Bias 108 ab cor. Ethical and Methodological Challenges 113 ing Tides of Change 113 thy Ethics and Artifacts in Collision 115 alo The "Ten Commandments" of the APA 118 IS Opportunity Knocks More Than Once 121 Not a Cannon, but a Popgun 124 Beneficence, Respect, and Justice 125 Where Are We Heading? 126 Risks and Benefits 127 artij Waste Not, Want Not 130 sear Pfu1 Notes 133 nee1 Name Index 163 Edv Subject Index 169 Cro1 Kau Viii t~~pter l nrtif acts in Behavioral Research All scientific inquiry is subject to error, and it is far better to be aware of this, to study the sources in an attempt to reduce it, and to estimate the magnitude of such errors in our findings, than to be ignorant of the errors concealed in the data. One must not equate ignorance of error with the lack of error. The lack of demonstration of error in certain fields of inquiry often derives from the nonexistence of methodological research into the problem and merely denotes a less advanced stage of that profession. The Nature of Artifacts f he effort to understand human behavior must of the oldest of human behaviors. ~ itself be one But for all the centuries of effort, there is much we do not yet know. The unsolved behavioral prob­ lems of mental illness, racism, sexism, and vio­ lence, both the idiosyncratic and institutionalized, bear witness to how much we do not yet know. Because of the urgency of the questions waiting to I . I Chapter Rrtifacts in Behavioral Research J , be answered, it should not be surprising that behavioral scien- We can conceptualize this portion of the complexity of be­ 21 tists, and the publics that support them, suffer from a certain im- havior, which can be attributed to the intrinsic human aspects of n patience. That impatience is understandable, but perhaps, from behavioral and social research, as a set of artifacts (or systematic time to time, we need to remind ourselves that Neurath's "ship of errors) to be isolated, measured, considered, and, sometimes, science" has not been afloat very long in the behavioral sciences. eliminated. That is, artifacts are not simply inconsequential effects The application of that reasoning and of those procedures in a research design; they may actually jeopardize the validity that we call the scientific method to the understanding of human of the researcher's inferences from his or her results. Another behavior is of relatively recent origin. Yet what we have learned way of saying this is that artifacts are unintended or uncon­ about human behavior in the short period, say, from Wilhelm trolled human aspects of the research situation that confound Wundt's founding of experimental psychology in Leipzig in 1879 the investigator's conclusions about what went on in the study. until now is out of all proportion to what was learned in the pre­ As we shall see later, artifacts can also teach us about topics ceding centuries. Although most of what we want to know is still of substantive interest. Indeed, as William McGuire observed, unknown, this success of the application of "scientific method" today's artifact may be tomorrow's independent variable;2 that to the study of human behavior has given us great hope of an ac­ is, the same conditions discounted as "nuisance variables" at celerating return of Jpiowledge on our investment of time and one time may later be exploited as variables of interest in their effort. To be sure, this application of what we think of as scien­ own right. tific method has not simplified human behavior; instead, it has shown us more clearly just how complex human behavior really is. In modern behavioral science, the research participant (or Clever Hans "research s~bject") serves as our model of people in general, or The systematic investigation of artifacts began in the late 1950s, at least of a certain kind of person. We know that behavior is the bulk of the research following in the 1960s and 1970s. Well be­ complex because we sometimes change this person's world ever fore this work began in earnest, there had been early indications so slightly and detect substantial changes in his or her behavior, that artifacts might be lurking in the investigative procedures used while at other times we change that world greatly and detect by behavioral scientists. A classic case showed that artifacts could hardly any changes in behavior. It is as if the research partici­ result from the observer's expectations or hypotheses. In this 1904 pants' minds were somehow in a different world from ours. We case, a German psychologist, Oscar Pfungst, carried out a six-week also know that behavior is complex because a careful experiment investigation of a remarkable horse owned by a mathematics conducted in one place at one time often yields results very dif­ teacher named Wilhelm von Osten.3 Since the beginning of ferent from the "same" experiment conducted in another place recorded civilization, there had been reports of "learned animals;' at another time. We have learned that much of the complexity of but no animal had so captured the imagination of the general human behavior is inherent, but we have also learned that some public and of European psychology as von Osten's horse, "Clever of this complexity may result from uncontrolled aspects of the Hans." Pfungst's elegant series of studies solved the riddle of research situation, especially from the interaction between the Hans's amazing performance and implied how easily observers researcher and the participant. can be deceived by the self-fulfilling nature of their expectations. Artifacts in Behavioral Research I Chapter I I Before Pfungst had entered the scene, rumors had circulated unwitting movements and mannerisms. Someone would ask 4J widely of the astounding "intellectual" feats performed by Hans. Hans a question requiring a long tapping response, and the per- JS By tapping his hoof according to a code taught to him by von son might then lean forward as though settling down for a long Osten, Hans was said to answer questions put to him in German. wait. Hans responded to the questioner's forward movement, Asked to spell a word, Hans would tap out the letters, aided not to the actual question, and kept tapping away with his hoof ostensibly by a code table set in front of him. To respond yes to until the questioner communicated his expectancy that Hans a question, he would nod his head up and down; to respond would stop tapping. This the questioner might do by suddenly no, he would execute a deliberate sideways motion.