Developmental Psychology
Total Page:16
File Type:pdf, Size:1020Kb
Developmental Psychology Does Mentioning “Some People” and “Other People” in a Survey Question Increase the Accuracy of Adolescents' Self-Reports? David Scott Yeager and Jon A. Krosnick Online First Publication, September 12, 2011. doi: 10.1037/a0025440 CITATION Yeager, D. S., & Krosnick, J. A. (2011, September 12). Does Mentioning “Some People” and “Other People” in a Survey Question Increase the Accuracy of Adolescents' Self-Reports?. Developmental Psychology. Advance online publication. doi: 10.1037/a0025440 Developmental Psychology © 2011 American Psychological Association 2011, Vol. ●●, No. ●, 000–000 0012-1649/11/$12.00 DOI: 10.1037/a0025440 BRIEF REPORT Does Mentioning “Some People” and “Other People” in a Survey Question Increase the Accuracy of Adolescents’ Self-Reports? David Scott Yeager and Jon A. Krosnick Stanford University A great deal of developmental research has relied on self-reports solicited using the “some/other” question format (“Some students think that . but other students think that. .”). This article reports tests of the assumptions underlying its use: that it conveys to adolescents that socially undesirable attitudes, beliefs, or behaviors are not uncommon and legitimizes reporting them, yielding more valid self-reports than would be obtained by “direct” questions, which do not mention what other people think or do. A meta-analysis of 11 experiments embedded in four surveys of diverse samples of adolescents did not support the assumption that the some/other form increases validity. Although the some/other form led adolescents to think that undesirable attitudes, beliefs, or behaviors were more common and resulted in more reports of those attitudes and behaviors, answers to some/other questions were lower in criterion validity than were answers to direct questions. Because some/other questions take longer to ask and answer and require greater cognitive effort from participants (because they involve more words), and because they decrease measurement accuracy, the some/other question format seems best avoided. Keywords: adolescence, surveys, questionnaires, social desirability, self-concept Supplemental materials: http://dx.doi.org/10.1037/a0025440.supp Developmental psychologists often measure adolescents’ atti- the some/other format have been asked in major national surveys tudes, beliefs, and behaviors by asking questions in what might be of adolescents, including the National Longitudinal Survey of called a “some/other” format. Such questions first state what Youth, 1997 Cohort (NLSY:97; Bureau of Labor Statistics, 2005), “some” people like, believe, or do, then state what “other” people the National Education Longitudinal Survey, 1988 Cohort (NELS: like, believe, or do, and then ask the participant to describe himself 88; Ingels, Abraham, Karr, Spencer, & Frankel, 1990), the Youth or herself. For example, the some/other format is employed in Studies Series (Jennings & Niemi, 1991), and the 4-H Study of Harter’s (1982, 1988) Perceived Competence Scale for children Positive Youth Development (Lerner, von Eye, Lerner, Lewin- and adolescents. These batteries offer items such as, “Some stu- Bizan, & Bowers, 2010). As a result, many important lessons dents have a lot of friends, while other students don’t have very learned about the development of self-concept across adolescence many friends” (Harter, 1982, p. 91) and then ask participants to have relied on the some/other question format (see, e.g., Harter, indicate whether they are more like the first type of student or the 2006). second type of student. However, all such questions could have been asked more effi- Harter’s (1982, 1988) batteries have been used extensively in ciently in what could be called the “direct form,” by asking, for research on self-concept during the last 30 years, and variants of example, “Do you have many friends?” This approach requires participants to process fewer words and might therefore reduce the total cognitive burden of a questionnaire. Furthermore, reading fewer words might mean that participants could answer more David Scott Yeager and Jon A. Krosnick, Department of Psychology, questions during a fixed amount of time, thus allowing researchers Stanford University. to collect more useful data. Preparation of this article was supported by a Spencer Foundation Harter (1982) argued that any added administration time and Dissertation Writing Fellowship to the first author. The data were collected under a grant from the Thrive Foundation for Youth. The authors thank cognitive burden associated with the some/other format is worth- Jesse Corburn, Bill Hessert, Jason Singer, and Matt Williams for their help while because this format is likely to elicit more accurate self- collecting the data and Kinesis Survey Technologies for the use of their descriptions on matters where social desirability pressures might Internet questionnaire administration platform. Jon Krosnick is University be present: “The effectiveness of this [some/other] question format Fellow at Resources for the Future. lies in the implication that half of the children in the world (or in Correspondence concerning this article should be addressed to David Scott Yeager, Stanford University, Stanford, CA 94305, or to Jon A. one’s reference group) view themselves in one way, whereas the Krosnick, Stanford University, 432 McClatchy Hall, Stanford, CA 94028. other half view themselves in the opposite manner. That is, this E-mail: [email protected] or [email protected] type of question legitimizes either choice” (Harter, 1982, p. 89). 1 2 YEAGER AND KROSNICK Hence, if an adolescent is reluctant to offer an answer because Method doing so would be embarrassing, the some/other question form may allow him or her to feel more comfortable answering hon- Participants and Procedure estly. As appealing as the some/other format might seem from this Eleven experiments were conducted with four samples of high point of view, it might also cause unintended distortions in self- school students. The first sample was composed of 9th and 10th reports. By explicitly referring to the distribution of others’ atti- grade students in a low-income, urban public high school in tudes, beliefs, or behaviors, such questions might induce a cogni- Oakland, California. The second was composed of 10th grade tive focus on that distribution that might not have occurred students in a similar school in New York City. The third was otherwise. Inducing this might increase respondent burden further, composed of 9th grade students at a middle- to upper income by causing people to think about something they would not oth- suburban public high school near San Jose, California. The fourth erwise have thought about, rather than simply focusing on their included 9th and 10th grade students at a low-income, urban public own attitudes, beliefs, or behaviors when generating a self-report. high school in San Jose, California. Consent and assent were Mentioning the attributes of others might be especially prob- obtained from 85% of the invited participants (Sample 1: N ϭ 160; lematic in another way as well. By suggesting that the population Sample 2: N ϭ 152; Sample 3: N ϭ 72; Sample 4: N ϭ 171). They of adolescents is about equally split in terms of an attribute, the ranged in age from 13 to 16 years old. some/other question format might change adolescents’ perceptions Across the three urban schools (Samples 1, 2, and 4), 15.8% of of social norms. According to social comparison theory, people participants were African American, 21% were Asian American, compare themselves to their peers in order to determine the ap- 51.5% were Hispanic, 15% were White, and the rest named a propriateness of their own attitudes, beliefs, and behaviors (Go- different race/ethnicity. Forty-two percent were boys. More than ethals & Darley, 1977; see also Cohen & Prinstein, 2006). Chang- 75% received free or reduced price lunch, and only 13% of their ing young people’s perceptions of others might lead some to see mothers had a college degree or more education. In the suburban their own responses to a question as less normative and hence less school (Sample 3), 1% were African American, 43% were Asian appropriate, which might prevent them from reporting them accu- American, 10% were Hispanic, 43% were White, and the rest rately. Furthermore, by leading some participants to feel more out named a different race/ethnicity. Fifty-four percent were boys. of step with their peers, the some/other format might motivate Almost none of the students in Sample 3 received free or reduced them to change their answers to match the perceived norm due to price lunch, and 72% of their mothers had a college degree or more processes of informational social influence or conformity (Cialdini education. & Goldstein, 2004; Deutsch & Gerard, 1955). This process may The participants took about 30 min to complete questionnaires be particularly strong for adolescents because they are espe- on computers during school hours. After completing the question- cially reliant on peers as they develop their self-concepts and naires, students were thanked and debriefed. They were not com- may therefore be more susceptible to social influence (Cohen & pensated. Prinstein, 2006; Hergovich, Sirsch, & Felinger, 2002; Steinberg & Silverberg, 1986). Consequently, the some/other format Measures might lead some adolescents to give answers that they might not otherwise have given and that are less accurate reflections of Target items. The target items used in the experiments