The Development of Gender Stereotypes About STEM Abilities
Total Page:16
File Type:pdf, Size:1020Kb
The Development of Gender Stereotypes About STEM and Verbal Abilities: A Preregistered Meta-Analysis Protocol David I. Miller American Institutes for Research Jillian E. Lauer New York University Ryan T. Williams Courtney Tanenbaum American Institutes for Research Please direct correspondence to David I. Miller ([email protected]). The following document accompanies our preregistration on the Open Science Framework website made on February 19, 2020 (https://osf.io/29egh/registrations). This project is funded by the National Science Foundation (NSF Award No. DUE-1920401). Any opinions, findings, and conclusions or recommendations expressed in this project’s publications, including in this document, are those of the research team and do not necessarily reflect the views of the NSF. Abstract and Rationale This project will study the origins of beliefs and motivational factors that could potentially limit girls’ and women’s full participation in science, technology, engineering, and mathematics (STEM) fields, as well as contribute to boys’ underachievement in verbal domains (e.g., reading and writing). Specifically, this synthesis project aims to bring clarity to the mixed findings on (a) how gender stereotypes about STEM and verbal abilities first develop and (b) how they relate to gender gaps in STEM. Several studies of children have found the expected stereotype of superior male ability in mathematics and science, but others have found only in-group bias or even stereotypes of female superiority. Less research has focused on verbal ability stereotypes, but they are also critical because pro-female verbal stereotypes could potentially draw girls away from quantitative fields, contributing to gender gaps in STEM outcomes. Verbal stereotypes might also limit boys’ academic success in reading, writing, and language domains, which generally show moderate to large gaps favoring girls in test performance. This project consists of two meta-analyses that will analyze variation in (a) mean levels of children’s gender stereotypes about STEM and verbal abilities and (b) these stereotypes’ correlations with motivational STEM outcomes such as confidence and interests. In both meta-analyses, focal moderators will include child demographics, cultural contexts, and measurement characteristics. Knowledge from this project will help bring clarity to the miXed developmental literature on STEM ability stereotypes, in addition to synthesizing insights from the emerging literature on verbal ability stereotypes. Understanding why one study finds stereotypes strongly favoring males, whereas another study finds the opposite, will be critical to foster cumulative, replicable science and build integrative theories of stereotype development. Furthermore, synthesizing how ability stereotypes relate to outcomes such as confidence and interests can build fundamental knowledge on how these beliefs might relate to gender gaps in STEM participation. Though both meta-analyses will examine STEM and verbal ability stereotypes, the second meta-analysis will focus on just STEM outcomes (i.e., relating STEM and verbal stereotypes to STEM outcomes, but not verbal stereotypes to verbal outcomes, for which there is likely limited research). We plan to present the two meta-analyses as two separate journal article manuscripts. American Institutes for Research STEM Ability Stereotypes Meta-Analysis: Preregistration—1 PRISMA-P 2015 Checklist This checklist has been adapted for use with protocol submissions to Systematic Reviews from Table 3 in Moher D et al: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews 2015 4:1 Information Page Section/topic # Checklist item reported number(s) Yes No ADMINISTRATIVE INFORMATION Title Identify the report as a protocol of a Identification 1a 1 systematic review If the protocol is for an update of a previous Update 1b N/A systematic review, identify as such If registered, provide the name of the registry Registration 2 (e.g., PROSPERO) and registration number 1 in the Abstract Authors Provide name, institutional affiliation, and e- mail address of all protocol authors; provide Contact 3a 1 physical mailing address of corresponding author Describe contributions of protocol authors Contributions 3b 5 and identify the guarantor of the review If the protocol represents an amendment of a previously completed or published protocol, Amendments 4 identify as such and list changes; otherwise, 5 state plan for documenting important protocol amendments Support Indicate sources of financial or other support Sources 5a 1, 5 for the review Provide name for the review funder and/or Sponsor 5b 1, 5 sponsor Describe roles of funder(s), sponsor(s), Role of 5c and/or institution(s), if any, in developing the 5 sponsor/funder protocol INTRODUCTION Describe the rationale for the review in the Rationale 6 1, 8, 9 context of what is already known Provide an eXplicit statement of the question(s) the review will address with reference to participants, interventions, Objectives 7 comparators, and outcomes (PICO) 1, 6 American Institutes for Research STEM Ability Stereotypes Meta-Analysis: Preregistration—2 Information Page Section/topic # Checklist item reported number(s) Yes No METHODS Specify the study characteristics (e.g., PICO, study design, setting, time frame) and report Eligibility criteria 8 characteristics (e.g., years considered, 12-13 language, publication status) to be used as criteria for eligibility for the review Describe all intended information sources Information (e.g., electronic databases, contact with study 9 9-11 sources authors, trial registers, or other grey literature sources) with planned dates of coverage Present draft of search strategy to be used Search strategy 10 for at least one electronic database, including AppendiX A planned limits, such that it could be repeated STUDY RECORDS Describe the mechanism(s) that will be used Data 11a to manage records and data throughout the 15 management review State the process that will be used for selecting studies (e.g., two independent Selection 14, 11b reviewers) through each phase of the review process AppendiX B (i.e., screening, eligibility, and inclusion in meta-analysis) Describe planned method of eXtracting data from reports (e.g., piloting forms, done Data 15-16, 11c independently, in duplicate), any processes collection process AppendiX C for obtaining and confirming data from investigators List and define all variables for which data will be sought (e.g., PICO items, funding 15-16, Data items 12 sources), any pre-planned data assumptions AppendiX C and simplifications List and define all outcomes for which data Outcomes and 13 will be sought, including prioritization of main 8, 16-18 prioritization and additional outcomes, with rationale Describe anticipated methods for assessing risk of bias of individual studies, including Risk of bias in 14 whether this will be done at the outcome or 15 individual studies study level, or both; state how this information will be used in data synthesis DATA Describe criteria under which study data will 15a 13 be quantitatively synthesized Synthesis If data are appropriate for quantitative synthesis, describe planned summary 15b 16-24 measures, methods of handling data, and methods of combining data from studies, American Institutes for Research STEM Ability Stereotypes Meta-Analysis: Preregistration—3 Information Page Section/topic # Checklist item reported number(s) Yes No including any planned eXploration of consistency (e.g., I 2, Kendall’s tau) Describe any proposed additional analyses 15c (e.g., sensitivity or subgroup analyses, meta- 16-24 regression) If quantitative synthesis is not appropriate, 15d N/A describe the type of summary planned Specify any planned assessment of meta- 21, Meta-bias(es) 16 bias(es) (e.g., publication bias across studies, AppendiX D selective reporting within studies) Confidence in Describe how the strength of the body of cumulative 17 23 evidence will be assessed (e.g., GRADE) evidence American Institutes for Research STEM Ability Stereotypes Meta-Analysis: Preregistration—4 Amendment Timeline Project Status at Time of Initial Preregistration We submitted an initial draft of the following protocol for an NSF grant proposal. We responded to NSF panel reviewers’ comments, after which NSF notified us of winning the grant on July 9, 2019 (Award No. DUE-1920401). We then reviewed the proposed plan in more detail with an eXternal advisory board of five eXperts (Andrei Cimpian, Beth Kurtz-Costes, Catherine Riegle-Crumb, Jo Boaler, Larry Hedges) during a 2-hour meeting on November 4, 2019. Lastly, our project’s internal quality assurance review, Martyna Citkowicz, reviewed this document before we finalized and preregistered it on the Open Science Framework website on February 19, 2020 (https://osf.io/29egh/registrations). Hence, this protocol collectively represents input from project team members, anonymous NSF grant panel reviewers, eXternal advisory board members, and our project’s designated internal quality assurance reviewer. As noted in the following section on “Explanation of Existing Data,” the PI had analyzed a small subset of studies on children’s ability stereotypes (14 studies) six years prior to this preregistration, but he did not reanalyze those data to form this current protocol, which greatly differs in scope and approach from that initial investigation. We used the following standardized table from PROSPERO, an international prospective