Experimenter Expectancy Effects Data Collection: Validity & Ethics A kind of “self-fulfilling prophesy” during which researchers unintentionally “produce the results they want”. Two kinds… Data Integrity Modifying Participants’ Behavior – Expectancy effects & their control • Experimenter expectancy effects – Subtle differences in treatment of participants in different • Participant Expectancy Effects conditions can change their behavior… • Single- and Double-blind designs – Inadvertently conveying response expectancies/research – Researcher and Participant Bias & their control hypotheses • Reactivity & Response Bias – Problems with participants – Difference in performance due to differential quality of • Observer Bias & Interviewer Bias – Problems w/ researchers instruction or friendliness of the interaction – Effects of attrition on initial equivalence Data Collection Bias (much like observer bias) • Ethical Considerations – Many types of observational and self-report data need to be – Informed Consent “coded” or “interpreted” before they can be analyzed – Researcher Honesty – Subjectivity and error can creep into these interpretations – usually leading to data that are biased toward expectations – Levels of Disclosure

Participant Expectancy Effects A kind of “demand characteristic” during which participants modify their behavior to respond/conform to “how they should act”. Two kinds… Social Desirability – When participants intentionally or unintentionally modify their behavior to match “how they are expected to behave” – Well-known social psychological phenomenon that usually happens between individual’s and their “peer group” – Can also happen between researcher and participants Acquiescence/Rejection Response – If participant thinks they know the research hypothesis or know the behavior that is expected of them they can “try to play along” (acquiescence) or “try to mess things up” (rejection response) – Particularly important during within-groups designs – if participants think study is “trying to change their behavior” Single & Double-blind Procedures Reactivity & Response Bias One way to limit or minimize the various biasing effects we’ve • Both of these refer to getting “less than accurate” data from the participants discussed is to limit the information everybody involved has Reactivity is the term commonly used when talking about observational data In Single Blind Procedures the participant doesn’t know the collection – the participant may behave “not naturally” if they know they are being hypotheses, the other conditions in the study, and ideally, the observed or are part of a study particular condition they are in (i.e., we don’t tell how the task – Naturalistic & disguised participant observation methods are intended to or manipulation is designed to change their behavior) avoid this – Habituation and desensitization help when using undisguised participant In Double-blind Procedures neither the participant nor the observation data collector/data coder knows the hypotheses or other • Response Bias is the term commonly used when talking about self-report data collection and describes a situation in which the participant responds information that could bias the interaction/reporting/coding of how they think they “should” the researcher or the responses of the participants – The response might be a reaction to cues the researcher provides – Social Desirability is when participants describe their character, opinions Sometimes this simply can’t be done (especially the researcher- or behavior as they think they “should” or to present a certain impression blind part) because of the nature of the variables or the of themselves hypotheses involved (e.g., hard to hide the gender of a – Protecting participants’ anonymity and participant-researcher rapport are participant from the researcher who is coding the video tape) intended to increase the honesty of participant responses

Observer Bias & Interviewer Bias Both of these are versions of “seeing what you want to see” Observer Bias is the term commonly used when talking about observational data collection – Both observational data collection and data coding need to be done objectively and accurately – Automation & instrumentation help – so does using multiple observers/coders and looking for consistency • Interviewer Bias is the term commonly used when talking about self-report data collection – How questions are asked by interviewers or the interviewers’ reactions to answers can drive response bias – More of a challenge with face-to-face interviews – Computerized and paper-based procedures help limit this

Effects of participant-research gender, race, age, personality, etc. match/mismatch have been shown to influence the behavior of both !!! Data collection biases & inaccuracies -- summary Attrition – also known as drop-out, data loss, response refusal, & experimental mortality Type of Data Collection Attrition endangers initial equivalence of subject variables Observational Self-report • random assignment is intended to produce initial equivalence of Observer Bias Interviewer Bias subject variables – so that the groups (IV conditions) have equivalent means on all subject variables (e.g., age, gender, “inaccurate data “coaching” or motivation, prior experience, intelligence, topical knowledge, etc.) recording/coding” “inaccurate recording/coding” • attrition can disrupt the initial equivalence – producing inequalities • “differential attrition” – related to IV condition differences – is particularly likely to produce inequalities Reactivity Response Bias • e.g., If one condition is “harder” and so more participants “reacting” to “dishonest” drop out of that condition, there is likely to be a being observed responding “motivation” difference between the participants Participant Researcher Participant remaining in the two conditions (i.e., those remaining in the harder condition are more motivated).

So, “attrition” works much like “self assignment” to trash initial equivalence Both involve a non-random determination of who provides data for what condition of the study!

Imagine a study that involves a “standard treatment” and an “experimental treatment”… • random assignment would be used to ensure that the participants in the two groups are equivalent • self-assignment is likely to produce non-equivalence (different “kinds” of folks likely to elect the different treatments) • attrition (i.e., rejecting the randomly assigned condition) is similarly likely to produce non-equivalence (different “kinds” of folks likely to remain in the different treatments) How to combat attrition… Ethical Considerations -- participation • educate participants about the important role of random Research ethics are summarized in the “risk-benefit” trade-off assignment to the validity of the research model. • if there is differential “value” of the different treatments or What do participants risk when participating ? conditions (especially in a “treatment vs no-treatment” – social (embarrassment), psychological (learning uncomfortable comparison) – offer folks an opportunity to participate in the things about themselves), or even physical risk preferred condition after data collection – risk might be from manipulation, task, data collection or being “associated” with the research • replacement of participants who drop out of the study trades off with … •If there is a more aversive condition, then ask the participants What are the benefits of the research ? before assignment if they would still participate even if they were in the aversive condition. Then, only allow the people who say yes – to society (knowledge gained) or the participant (remuneration – pay or research credit or direct benefit of the treatment) to be in the study (note implications for external validity) Each university has an Institutional Review Board (IRB) that must approve the • collect data about possible confounding variables for statistical manipulations, procedures, data collection and data storage of all research involving comparison later human participants, under the review of the federal government. Individuals or universities that violate the relevant guidelines can/have been denied research support • replication & convergence of the study (grants), research/data collection privileges and are legally responsible to participants!

Voluntary Informed Consent without Deception

• Before participating each participant must read and sign a document that describes his/her participation (including random assignment) and all related activities, as well as the possible social, psychological or physical risks involved in that participation. • No information may be withheld from the participant the possession of which might alter her/his decision to give informed consent • “Deception” is withholding information from the participant that might possibly alter their decision whether or not to participate – Sometimes, the IRB approves research with deceptive elements as long as the risk/benefit ratio is ultimately positive • The participant is free to withdraw informed consent and stop participating in the research at any time with no consequences This guarantee is the cornerstone of Ethical Research !!! Ethical Considerations – reporting research Levels of disclosure Assuring participants that their responses are “safe” is important • When proposing and reporting research, researchers must be completely forthcoming concerning the procedures and when requesting their participation. resulting data. •Privacy • APA format & style is designed to ensure that all the features of – no one knows the info; Participant does not disclose it (not the study are adequately represented as part of the collected as data) research report • Anonymity • Remember, the central idea behind “scientific empiricism” is that – no direct connection between info and identity science is a “public enterprise” • Confidentiality – researcher has connection between info and identity, but doesn’t • not only the results but how those results are obtained is disclose connection made public • Group Disclosure • any properly trained individual should be able to understand – info about “group” is released procedures and results and (if properly outfitted) to – must avoid “indirect disclosure” for small groups replicate the findings • “Masked” Individual Disclosure • this level of procedural disclosure helps replication and – identity is hidden by pseudonym convergence efforts • Individual Disclosure – requires explicit informed consent