Registry of Efficacy and Effectiveness Studies
Total Page:16
File Type:pdf, Size:1020Kb
Registry of Efficacy and Effectiveness Studies Study Title: An Impact Evaluation of a Math Peer Coaching Intervention on 8th Grade Pre-Algebra and High School Algebra I Achievement (A Proving Ground Project, SY 2020-2021) Registry ID: 4680.1v1 Version History The first version of this entry was published on March 17, 2021 3:40:45 PM EDT Currently viewing this version. Section I: General Study Information PI name: Thomas Kane PI affiliation: Harvard University Co-PI name: Douglas Staiger Co-PI affiliation: Dartmouth College Primary Funding Source(s): Bill & Melinda Gates Foundation Award Number(s): - IRB Name: Faster, Cheaper Evidence-Gathering for US Education IRB Approval Date: 2015-07-27 IRB Approval Number: - Other Registration Name: - Other Registration Date: - Other Registration Number: - Study Start Date: - PDF Exported September 26,2021 02:29 EDT. Page 1 Study End Date: - Intervention Start Date: 2020-11-02 Timing of entry: Prior to collection of outcome data Brief Abstract: The study district is piloting a classroom-based math intervention that pairs students together based on a skills assessment and provides them with materials to facilitate structured peer coaching. Students in 8th grade Pre-Algebra and high school Algebra I courses were randomly assigned to either participate in a set number of intervention sessions in addition to their usual math instruction or continue with business as usual. The study will measure the effect of the intervention on students' performance on formative math assessments. Keywords: K12 Proving Ground Algebra Pre-Algebra Math Mathematics High School Middle School Peer Coaching COVID Hybrid Remote Comments: This study is part of a larger project called Proving Ground, a partnership between the Center for Education Policy Research at Harvard University and a network of Local Education Agencies (LEAs), and is being conducted in collaboration with Impact Florida, a nonprofit organization that supports education agencies in Florida. Section II starts on the next page. PDF Exported September 26,2021 02:29 EDT. Page 2 Section II: Description of Study Type of Intervention: Curriculum/Product Topic Area of Intervention: Mathematics and Science Education Number of intervention arms: 1 Target school level: 10, 11, 12, 8, 9 Target school type: Suburban Location of Implementation: United States : South Further description of location: Small suburban public school district Brief Description of Intervention Condition: Students assigned to treatment will be paired together based on their scores on a benchmark math skills assessment. Students will receive targeted math skills worksheets that guide them through a structured peer coaching process under the supervision of a trained teacher. Students are then assessed again and re-paired for the next session. Based on scheduling needs, some treatment students will participate in two 30-minute sessions each week during their study hall period, some treatment students will participate in three 20-minute sessions each week during their supplementary math period, and some treatment students will participate in three 15-minute sessions each week during their lunch period. Brief Description of Comparison Condition: Students assigned to control will continue with business as usual. Based on scheduling needs, some control students will use study hall in an undirected fashion, and some control students will use their supplementary math period to receive other math support as usual. Comparison condition: Business-as-usual Comments: - Section III: Research Questions Confirmatory research questions: Question 1: Does a math peer coaching intervention improve the math achievement of 8th grade Pre-Algebra and high school PDF Exported September 26,2021 02:29 EDT. Page 3 Algebra I students compared to a control group of students who continue with business as usual? Exploratory research questions: No Questions added yet. Comments: - Section IV-A: Study Design (Selection) Study Design: Randomized Trial (RT) Comments: - Section IV-B: Study Design (Input) Study Design: Input Unit of random assignment of intervention: Student Assignment within sites or blocks: Yes Define the sites or blocks: Math Class Section Probability of assignment to treatment the same across sites or blocks: Yes Probability of assignment to treatment: 0.5 Unit outcome data measured: Student Intermediate clusters between unit of random assignment and unit of measurement: No Comments: - Design Classification Based on the responses above, this study has been classified as: RT: Multisite (Blocked) PDF Exported September 26,2021 02:29 EDT. Page 4 Section V: Sample Characteristics Approximate number of students in the comparision condition within each block (Math Class Section): 8 Approximate number of students in the intervention condition within each block (Math Class Section): 8 Number of blocks (Math Class Section): 31 Were there certain students that were targeted for the study? No Were there certain students that were excluded from the study? Yes - Students with study hall scheduling conflicts were excluded Were there certain blocks ( Math Class Section) that were targeted for the study? Yes - 8th grade Pre-Algebra and high school Algebra I class sections were targeted Were there certain blocks ( Math Class Section) that were excluded from the study? No Comments: - Section VI: Outcomes (Input) Confirmatory question 1: Outcome Measure 1 Outcome domain: Student Achievement - Mathematics Minimum detectable effect size: Outcome measure: Formative Math Assessment Scale of outcome measure: Continuous Normed or state test: Yes Same outcome measure in treatment and comparison groups: Yes Section VII: Outcomes (Input) Baseline data collected prior to start of intervention: Yes Description of baseline data: Formative math assessment administered before the launch of the intervention, student demographic characteristics Covariates you plan to include in the model: English Language Learner Status, Gender, Grade, Race, Special Education Status, Student Pretest Analytic model: See additional materials PDF Exported September 26,2021 02:29 EDT. Page 5 Plan to handle cases with missing outcome data: Delete cases with missing data for the outcome being analyzed Comments: A hierarchical Bayesian regression model may be used for supplementary analysis. Section VIII: Additional Information Links: No links have been added yet. Files: File Name: peer_coaching_analytic_model.pdf Description: Description of analytic model used to estimate the intent-to-treat effect. Comments: - PDF Exported September 26,2021 02:29 EDT. Page 6.