Registry of Efficacy and Effectiveness Studies

Study Title: An Impact Evaluation of a Math Peer Coaching Intervention on 8th Grade Pre-Algebra and High School Algebra I Achievement (A Proving Ground Project, SY 2020-2021)

Registry ID: 4680.1v1

Version History

The first version of this entry was published on March 17, 2021 3:40:45 PM EDT Currently viewing this version.

Section I: General Study Information

PI name: Thomas Kane PI affiliation:

Co-PI name: Co-PI affiliation: Dartmouth

Primary Funding Source(s): Bill & Melinda Gates Foundation

Award Number(s): -

IRB Name: Faster, Cheaper Evidence-Gathering for US Education

IRB Approval Date: 2015-07-27

IRB Approval Number: -

Other Registration Name: -

Other Registration Date: -

Other Registration Number: -

Study Start Date: -

PDF Exported September 26,2021 02:29 EDT. Page 1 Study End Date: -

Intervention Start Date: 2020-11-02

Timing of entry: Prior to collection of outcome data

Brief Abstract: The study district is piloting a classroom-based math intervention that pairs students together based on a skills assessment and provides them with materials to facilitate structured peer coaching. Students in 8th grade Pre-Algebra and high school Algebra I courses were randomly assigned to either participate in a set number of intervention sessions in addition to their usual math instruction or continue with business as usual. The study will measure the effect of the intervention on students' performance on formative math assessments.

Keywords: K12 Proving Ground Algebra Pre-Algebra Math Mathematics High School Middle School Peer Coaching COVID Hybrid Remote

Comments: This study is part of a larger project called Proving Ground, a partnership between the Center for Education Policy Research at Harvard University and a network of Local Education Agencies (LEAs), and is being conducted in collaboration with Impact Florida, a nonprofit organization that supports education agencies in Florida.

Section II starts on the next page.

PDF Exported September 26,2021 02:29 EDT. Page 2 Section II: Description of Study

Type of Intervention: Curriculum/Product

Topic Area of Intervention: Mathematics and Science Education

Number of intervention arms: 1

Target school level: 10, 11, 12, 8, 9

Target school type: Suburban

Location of Implementation: United States : South

Further description of location: Small suburban public school district

Brief Description of Intervention Condition: Students assigned to treatment will be paired together based on their scores on a benchmark math skills assessment. Students will receive targeted math skills worksheets that guide them through a structured peer coaching process under the supervision of a trained teacher. Students are then assessed again and re-paired for the next session. Based on scheduling needs, some treatment students will participate in two 30-minute sessions each week during their study hall period, some treatment students will participate in three 20-minute sessions each week during their supplementary math period, and some treatment students will participate in three 15-minute sessions each week during their lunch period.

Brief Description of Comparison Condition: Students assigned to control will continue with business as usual. Based on scheduling needs, some control students will use study hall in an undirected fashion, and some control students will use their supplementary math period to receive other math support as usual.

Comparison condition: Business-as-usual

Comments: -

Section III: Research Questions

Confirmatory research questions: Question 1: Does a math peer coaching intervention improve the math achievement of 8th grade Pre-Algebra and high school

PDF Exported September 26,2021 02:29 EDT. Page 3 Algebra I students compared to a control group of students who continue with business as usual?

Exploratory research questions: No Questions added yet.

Comments: -

Section IV-A: Study Design (Selection)

Study Design: Randomized Trial (RT)

Comments: -

Section IV-B: Study Design (Input)

Study Design: Input

Unit of random assignment of intervention: Student

Assignment within sites or blocks: Yes

Define the sites or blocks: Math Class Section

Probability of assignment to treatment the same across sites or blocks: Yes

Probability of assignment to treatment: 0.5

Unit outcome data measured: Student

Intermediate clusters between unit of random assignment and unit of measurement: No

Comments: -

Design Classification

Based on the responses above, this study has been classified as: RT: Multisite (Blocked)

PDF Exported September 26,2021 02:29 EDT. Page 4 Section V: Sample Characteristics

Approximate number of students in the comparision condition within each block (Math Class Section): 8

Approximate number of students in the intervention condition within each block (Math Class Section): 8

Number of blocks (Math Class Section): 31

Were there certain students that were targeted for the study? No

Were there certain students that were excluded from the study? Yes - Students with study hall scheduling conflicts were excluded

Were there certain blocks ( Math Class Section) that were targeted for the study? Yes - 8th grade Pre-Algebra and high school Algebra I class sections were targeted

Were there certain blocks ( Math Class Section) that were excluded from the study? No

Comments: -

Section VI: Outcomes (Input)

Confirmatory question 1: Outcome Measure 1 Outcome domain: Student Achievement - Mathematics Minimum detectable effect size: Outcome measure: Formative Math Assessment Scale of outcome measure: Continuous Normed or state test: Yes Same outcome measure in treatment and comparison groups: Yes

Section VII: Outcomes (Input)

Baseline data collected prior to start of intervention: Yes

Description of baseline data: Formative math assessment administered before the launch of the intervention, student demographic characteristics

Covariates you plan to include in the model: English Language Learner Status, Gender, Grade, Race, Special Education Status, Student Pretest

Analytic model:

See additional materials

PDF Exported September 26,2021 02:29 EDT. Page 5 Plan to handle cases with missing outcome data: Delete cases with missing data for the outcome being analyzed

Comments: A hierarchical Bayesian regression model may be used for supplementary analysis.

Section VIII: Additional Information

Links: No links have been added yet.

Files: File Name: peer_coaching_analytic_model.pdf Description: Description of analytic model used to estimate the intent-to-treat effect.

Comments: -

PDF Exported September 26,2021 02:29 EDT. Page 6