Assisting Students Struggling with Mathematics: Response to Intervention (Rti) for Elementary and Middle Schools

Home , 5 to 9, Operation (mathematics), Quantity, Support (mathematics), Support (measure theory)

IES PRACTICE GUIDE WHAT WORKS CLEARINGHOUSE

AssistingAssisting StudentsStudents StrugglingStruggling withwith Mathematics:Mathematics: ResponseResponse toto InterventionIntervention (RtI)(RtI) forfor ElementaryElementary andand MiddleMiddle SchoolsSchools

NCEE 2009-4060 U.S. DEPARTMENT OF EDUCATION The Institute of Education Sciences (IES) publishes practice guides in education to bring the best available evidence and expertise to bear on the types of systemic challenges that cannot currently be addressed by single interventions or programs. Authors of practice guides seldom conduct the types of systematic literature searches that are the backbone of a meta-analysis, although they take advantage of such work when it is already published. Instead, authors use their expertise to identify the most important research with respect to their recommendations, augmented by a search of recent publications to ensure that research citations are up-to-date.

Unique to IES-sponsored practice guides is that they are subjected to rigorous external peer review through the same office that is responsible for independent review of other IES publications. A critical task for peer reviewers of a practice guide is to determine whether the evidence cited in support of particular recommendations is up-to-date and that studies of similar or better quality that point in a different direction have not been ignored. Because practice guides depend on the expertise of their authors and their group decisionmaking, the content of a practice guide is not and should not be viewed as a set of recommendations that in every case depends on and flows inevitably from scientific research.

The goal of this practice guide is to formulate specific and coherent evidence-based recommendations for use by educators addressing the challenge of reducing the number of children who struggle with mathematics by using “response to intervention” (RtI) as a means of both identifying students who need more help and providing these students with high-quality interventions. The guide provides practical, clear information on critical topics related to RtI and is based on the best available evidence as judged by the panel. Recommendations in this guide should not be construed to imply that no further research is warranted on the effectiveness of particular strategies used in RtI for students struggling with mathematics. IES PRACTICE GUIDE

Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools

April 2009

Panel Russell Gersten (Chair) Instructional Research Group

Sybilla Beckmann University of Georgia

Benjamin Clarke Instructional Research Group

Anne Foegen Iowa State University

Laurel Marsh Howard County Public School System

Jon R. Star Harvard University

Bradley Witzel Winthrop University

Staff Joseph Dimino Madhavi Jayanthi Rebecca Newman-Gonchar Instructional Research Group

Shannon Monahan Libby Scott Mathematica Policy Research

NCEE 2009-4060 U.S. DEPARTMENT OF EDUCATION This report was prepared for the National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences under Contract ED-07-CO-0062 by the What Works Clearinghouse, which is operated by Mathematica Policy Research, Inc.

Disclaimer

The opinions and positions expressed in this practice guide are the authors’ and do not necessarily represent the opinions and positions of the Institute of Education Sci- ences or the U.S. Department of Education. This practice guide should be reviewed and applied according to the specific needs of the educators and education agency using it, and with full realization that it represents the judgments of the review panel regarding what constitutes sensible practice, based on the research available at the time of publication. This practice guide should be used as a tool to assist in decisionmaking rather than as a “cookbook.” Any references within the document to specific educational products are illustrative and do not imply endorsement of these products to the exclusion of other products that are not referenced.

U.S. Department of Education Arne Duncan Secretary

Institute of Education Sciences Sue Betka Acting Director

National Center for Education Evaluation and Regional Assistance Phoebe Cottingham Commissioner

April 2009

This report is in the public domain. Although permission to reprint this publication is not necessary, the citation should be:

Gersten, R., Beckmann, S., Clarke, B., Foegen, A., Marsh, L., Star, J. R., & Witzel, B. (2009). Assisting students struggling with mathematics: Response to Interven- tion (RtI) for elementary and middle schools (NCEE 2009-4060). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies. ed.gov/ncee/wwc/publications/practiceguides/.

This report is available on the IES website at http://ies.ed.gov/ncee and http://ies. ed.gov/ncee/wwc/publications/practiceguides/.

Alternative formats On request, this publication can be made available in alternative formats, such as Braille, large print, audiotape, or computer diskette. For more information, call the Alternative Format Center at 202–205–8113. Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools Contents

Introduction 1 The What Works Clearinghouse standards and their relevance to this guide 3 Overview 4 Summary of the Recommendations 5 Scope of the practice guide 9 Checklist for carrying out the recommendations 11 Recommendation 1. Screen all students to identify those at risk for potential mathematics difficulties and provide interventions to students identified as at risk. 13 Recommendation 2. Instructional materials for students receiving interventions should focus intensely on in-depth treatment of whole numbers in kindergarten through grade 5 and on rational numbers in grades 4 through 8. These materials should be selected by committee. 18 Recommendation 3. Instruction during the intervention should be explicit and systematic. This includes providing models of proficient problem solving, verbalization of thought processes, guided practice, corrective feedback, and frequent cumulative review. 21 Recommendation 4. Interventions should include instruction on solving word problems that is based on common underlying structures. 26 Recommendation 5. Intervention materials should include opportunities for students to work with visual representations of mathematical ideas and interventionists should be proficient in the use of visual representations of mathematical ideas. 30 Recommendation 6. Interventions at all grade levels should devote about 10 minutes in each session to building fluent retrieval of basic arithmetic facts. 37 Recommendation 7. Monitor the progress of students receiving supplemental instruction and other students who are at risk. 41 Recommendation 8. Include motivational strategies in tier 2 and tier 3 interventions. 44 Glossary of terms as used in this report 48 Appendix A. Postscript from the Institute of Education Sciences 52 Appendix B. About the authors 55 Appendix C. Disclosure of potential conflicts of interest 59 Appendix D. Technical information on the studies 61 References 91

( iii ) ASSISTING STUDENTS STRUGGLING WITH MATHEMATICS: RESPONSE TO INTERVENTION (RTI) FOR ELEMENTARY AND MIDDLE SCHOOLS

List of tables

Table 1. Institute of Education Sciences levels of evidence for practice guides 2

Table 2. Recommendations and corresponding levels of evidence 6

Table 3. Sensitivity and specificity 16

Table D1. Studies of interventions that included explicit instruction and met WWC Standards (with and without reservations) 69

Table D2. Studies of interventions that taught students to discriminate problem types that met WWC standards (with or without reservations) 73

Table D3. Studies of interventions that used visual representations that met WWC standards (with and without reservations) 77–78

Table D4. Studies of interventions that included fact fluency practices that met WWC standards (with and without reservations) 83

List of examples

Example 1. Change problems 27

Example 2. Compare problems 28

Example 3. Solving different problems with the same strategy 29

Example 4. Representation of the counting on strategy using a number line 33

Example 5. Using visual representations for multidigit addition 34

Example 6. Strip diagrams can help students make sense of fractions 34

Example 7. Manipulatives can help students understand that four multiplied by six means four groups of six, which means 24 total objects 35

Example 8. A set of matched concrete, visual, and abstract representations to teach solving single-variable equations 35

Example 9: Commutative property of multiplication 48

Example 10: Make-a-10 strategy 49

Example 11: Distributive property 50

Example 12: Number decomposition 51

( iv ) Introduction In some cases, recommendations reflect evidence-based practices that have been Students struggling with mathematics may demonstrated as effective through rigor- benefit from early interventions aimed at ous research. In other cases, when such improving their mathematics ability and evidence is not available, the recommen- ultimately preventing subsequent failure. dations reflect what this panel believes are This guide provides eight specific recom- best practices. Throughout the guide, we mendations intended to help teachers, clearly indicate the quality of the evidence principals, and school administrators use that supports each recommendation. Response to Intervention (RtI) to identify students who need assistance in mathe- Each recommendation receives a rating matics and to address the needs of these based on the strength of the research evi- students through focused interventions. dence that has shown the effectiveness of a The guide provides suggestions on how recommendation (table 1). These ratings— to carry out each recommendation and strong, moderate, or low—have been de- explains how educators can overcome fined as follows: potential roadblocks to implementing the recommendations. Strong refers to consistent and generalizable evidence that an intervention pro- The recommendations were developed by gram causes better outcomes.1 a panel of researchers and practitioners with expertise in various dimensions of Moderate refers either to evidence from this topic. The panel includes a research studies that allow strong causal conclu- mathematician active in issues related sions but cannot be generalized with as- to K–8 mathematics education, two pro- surance to the population on which a fessors of mathematics education, sev- recommendation is focused (perhaps be- eral special educators, and a mathematics cause the findings have not been widely coach currently providing professional de- replicated)—or to evidence from stud- velopment in mathematics in schools. The ies that are generalizable but have more panel members worked collaboratively to causal ambiguity than offered by experi- develop recommendations based on the mental designs (such as statistical models best available research evidence and our of correlational data or group comparison expertise in mathematics, special educa- designs for which the equivalence of the tion, research, and practice. groups at pretest is uncertain).

The body of evidence we considered in de- Low refers to expert opinion based on rea- veloping these recommendations included sonable extrapolations from research and evaluations of mathematics interventions theory on other topics and evidence from for low-performing students and students studies that do not meet the standards for with learning disabilities. The panel con- moderate or strong evidence. sidered high-quality experimental and quasi-experimental studies, such as those meeting the criteria of the What Works Clearinghouse (http://www.whatworks. ed.gov), to provide the strongest evidence of effectiveness. We also examined studies of the technical adequacy of batteries 1. Following WWC guidelines, we consider a posi- of screening and progress monitoring tive, statistically significant effect or large effect measures for recommendations relating size (i.e., greater than 0.25) as an indicator of to assessment. positive effects.

( 1 ) Introduction

Table 1. Institute of Education Sciences levels of evidence for practice guides

In general, characterization of the evidence for a recommendation as strong requires both studies with high internal validity (i.e., studies whose designs can support causal conclusions) and studies with high external validity (i.e., studies that in total include enough of the range of participants and settings on which the recommendation is focused to support the conclu- sion that the results can be generalized to those participants and settings). Strong evidence for this practice guide is operationalized as: • A systematic review of research that generally meets the standards of the What Works Clearinghouse (WWC) (see http://ies.ed.gov/ncee/wwc/) and supports the effectiveness of Strong a program, practice, or approach with no contradictory evidence of similar quality; OR • Several well-designed, randomized controlled trials or well-designed quasi-experiments that generally meet the standards of WWC and support the effectiveness of a program, practice, or approach, with no contradictory evidence of similar quality; OR • One large, well-designed, randomized controlled, multisite trial that meets WWC standards and supports the effectiveness of a program, practice, or approach, with no contradictory evidence of similar quality; OR • For assessments, evidence of reliability and validity that meets the Standards for Educa- tional and Psychological Testing.a

In general, characterization of the evidence for a recommendation as moderate requires studies with high internal validity but moderate external validity, or studies with high external validity but moderate internal validity. In other words, moderate evidence is derived from studies that support strong causal conclusions but when generalization is uncertain, or studies that support the generality of a relationship but when the causality is uncertain. Moderate evidence for this practice guide is operationalized as: • Experiments or quasi-experiments generally meeting the standards of WWC and supporting the effectiveness of a program, practice, or approach with small sample sizes and/or other conditions of implementation or analysis that limit generalizability and no contrary evidence; OR Moderate • Comparison group studies that do not demonstrate equivalence of groups at pretest and therefore do not meet the standards of WWC but that (a) consistently show enhanced outcomes for participants experiencing a particular program, practice, or approach and (b) have no major flaws related to internal validity other than lack of demonstrated equivalence at pretest (e.g., only one teacher or one class per condition, unequal amounts of instructional time, highly biased outcome measures); OR • Correlational research with strong statistical controls for selection bias and for discerning influence of endogenous factors and no contrary evidence; OR • For assessments, evidence of reliability that meets the Standards for Educational and Psychological Testingb but with evidence of validity from samples not adequately representative of the population on which the recommendation is focused.

In general, characterization of the evidence for a recommendation as low means that the recommendation is based on expert opinion derived from strong findings or theories in Low related areas and/or expert opinion buttressed by direct evidence that does not rise to the moderate or strong levels. Low evidence is operationalized as evidence not meeting the standards for the moderate or high levels. a. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999). b. Ibid.

( 2 ) Introduction

The What Works Clearinghouse Following the recommendations and sug- standards and their relevance to gestions for carrying out the recommen- this guide dations, Appendix D presents information on the research evidence to support the The panel relied on WWC evidence stan- recommendations. dards to assess the quality of evidence supporting mathematics intervention pro- The panel would like to thank Kelly Hay- grams and practices. The WWC addresses mond for her contributions to the analysis, evidence for the causal validity of instruc- the WWC reviewers for their contribution tional programs and practices according to to the project, and Jo Ellen Kerr and Jamila WWC standards. Information about these Henderson for their support of the intricate standards is available at http://ies.ed.gov/ logistics of the project. We also would like ncee/wwc/references/standards/. The to thank Scott Cody for his oversight of the technical quality of each study is rated and overall progress of the practice guide. placed into one of three categories: Dr. Russell Gersten • Meets Evidence Standards—for random- Dr. Sybilla Beckmann ized controlled trials and regression Dr. Benjamin Clarke discontinuity studies that provide the Dr. Anne Foegen strongest evidence of causal validity. Ms. Laurel Marsh Dr. Jon R. Star • Meets Evidence Standards with Reser- Dr. Bradley Witzel vations—for all quasi-experimental studies with no design flaws and randomized controlled trials that have problems with randomization, attrition, or disruption.

• Does Not Meet Evidence Screens—for studies that do not provide strong evidence of causal validity.

( 3 ) Assisting Students concluded that all students should receive preparation from an early age to ensure Struggling with their later success in algebra. In particular, Mathematics: Response the report emphasized the need for math- to Intervention (RtI) ematics interventions that mitigate and prevent mathematics difficulties. for Elementary and Middle Schools This panel believes that schools can use an RtI framework to help struggling students prepare for later success in mathemat- Overview ics. To date, little research has been conducted to identify the most effective ways Response to Intervention (RtI) is an early de- to initiate and implement RtI frameworks tection, prevention, and support system that for mathematics. However, there is a rich identifies struggling students and assists body of research on effective mathematics them before they fall behind. In the 2004 interventions implemented outside an RtI reauthorization of the Individuals with Dis- framework. Our goal in this practice guide abilities Education Act (PL 108-446), states is to provide suggestions for assessing were encouraged to use RtI to accurately students’ mathematics abilities and imple- identify students with learning disabilities menting mathematics interventions within and encouraged to provide additional sup- an RtI framework, in a way that reflects ports for students with academic difficul- the best evidence on effective practices in ties regardless of disability classification. mathematics interventions. Although many states have already begun to implement RtI in the area of reading, RtI ini- RtI begins with high-quality instruction tiatives for mathematics are relatively new. and universal screening for all students. Whereas high-quality instruction seeks to Students’ low achievement in mathemat- prevent mathematics difficulties, screen- ics is a matter of national concern. The re- ing allows for early detection of difficul- cent National Mathematics Advisory Panel ties if they emerge. Intensive interventions Report released in 2008 summarized the are then provided to support students poor showing of students in the United in need of assistance with mathematics States on international comparisons of learning.4 Student responses to interven- mathematics performance such as the tion are measured to determine whether Trends in International Mathematics and they have made adequate progress and (1) Science Study (TIMSS) and the Program for no longer need intervention, (2) continue International Student Assessment (PISA).2 to need some intervention, or (3) need A recent survey of algebra teachers as- more intensive intervention. The levels of sociated with the report identified key intervention are conventionally referred deficiencies of students entering algebra, to as “tiers.” RtI is typically thought of as including aspects of whole number arith- having three tiers.5 Within a three-tiered metic, fractions, ratios, and proportions.3 RtI model, each tier is defined by specific The National Mathematics Advisory Panel characteristics.

2. See, for example, National Mathematics Ad- 4. Fuchs, Fuchs, Craddock et al. (2008). visory Panel (2008) and Schmidt and Houang 5. Fuchs, Fuchs, and Vaughn (2008) make the (2007). For more information on the TIMSS, see case for a three-tier RtI model. Note, however, http://nces.ed.gov/timss/. For more information that some states and school districts have imple- on PISA, see http://www.oecd.org. mented multitier intervention systems with more 3. National Mathematics Advisory Panel (2008). than three tiers.

( 4 ) Overview

• Tier 1 is the mathematics instruction student performance data is critical in that all students in a classroom receive. this tier. Typically, specialized person- It entails universal screening of all stu- nel, such as special education teachers dents, regardless of mathematics profi- and school psychologists, are involved ciency, using valid measures to identify in tier 3 and special education services.14 students at risk for future academic However, students often receive rele- failure—so that they can receive early vant mathematics interventions from a intervention.6 There is no clear consen- wide array of school personnel, includ- sus on the characteristics of instruction ing their classroom teacher. other than that it is “high quality.”7 Summary of the Recommendations • In tier 2 interventions, schools provide additional assistance to students who This practice guide offers eight recom- demonstrate difficulties on screening mendations for identifying and supporting measures or who demonstrate weak students struggling in mathematics (table progress.8 Tier 2 students receive sup- 2). The recommendations are intended to plemental small group mathematics be implemented within an RtI framework instruction aimed at building targeted (typically three-tiered). The panel chose to mathematics proficiencies.9 These in- limit its discussion of tier 1 to universal terventions are typically provided for screening practices (i.e., the guide does 20 to 40 minutes, four to five times each not make recommendations for general week.10 Student progress is monitored classroom mathematics instruction). Rec- throughout the intervention.11 ommendation 1 provides specific suggestions for conducting universal screening • Tier 3 interventions are provided to effectively. For RtI tiers 2 and 3, recom- students who are not benefiting from mendations 2 though 8 focus on the most tier 2 and require more intensive as- effective content and pedagogical prac- sistance.12 Tier 3 usually entails one- tices that can be included in mathematics on-one tutoring along with an appropri- interventions. ate mix of instructional interventions. In some cases, special education ser- Throughout this guide, we use the term vices are included in tier 3, and in oth- “interventionist” to refer to those teachers special education is considered an ing the intervention. At a given school, the additional tier.13 Ongoing analysis of interventionist may be the general classroom teacher, a mathematics coach, a spe- 6. For reviews see Jiban and Deno (2007); Fuchs, cial education instructor, other certified Fuchs, Compton et al. (2007); Gersten, Jordan, school personnel, or an instructional as- and Flojo (2005). sistant. The panel recognizes that schools 7. National Mathematics Advisory Panel (2008); rely on different personnel to fill these National Research Council (2001). roles depending on state policy, school 8. Fuchs, Fuchs, Craddock et al. (2008); Na- resources, and preferences. tional Joint Committee on Learning Disabilities (2005). 9. Fuchs, Fuchs, Craddock et al. (2008). Recommendation 1 addresses the type of screening measures that should be used in 10. For example, see Jitendra et al. (1998) and Fuchs, Fuchs, Craddock et al. (2008). tier 1. We note that there is more research 11. National Joint Committee on Learning Dis- on valid screening measures for students in abilities (2005). 12. Fuchs, Fuchs, Craddock et al. (2008). 13. Fuchs, Fuchs, Craddock et al. ��(2008); National 14. National Joint Committee on Learning Dis- Joint Committee on Learning Disabilities (2005). abilities (2005).

( 5 ) Overview

Table 2. Recommendations and corresponding levels of evidence

Recommendation Level of evidence

Tier 1

1. Screen all students to identify those at risk for potential mathematics Moderate difficulties and provide interventions to students identified as at risk.

Tiers 2 and 3

2. Instructional materials for students receiving interventions should focus intensely on in-depth treatment of whole numbers in kindergar- Low ten through grade 5 and on rational numbers in grades 4 through 8. These materials should be selected by committee.

3. Instruction during the intervention should be explicit and systematic. This includes providing models of proficient problem solving, verbal- Strong ization of thought processes, guided practice, corrective feedback, and frequent cumulative review.

4. Interventions should include instruction on solving word problems Strong that is based on common underlying structures.

5. Intervention materials should include opportunities for students to work with visual representations of mathematical ideas and interven- Moderate tionists should be proficient in the use of visual representations of mathematical ideas.

6. Interventions at all grade levels should devote about 10 minutes in each Moderate session to building fluent retrieval of basic arithmetic facts.

7. Monitor the progress of students receiving supplemental instruction Low and other students who are at risk.

8. Include motivational strategies in tier 2 and tier 3 interventions. Low

Source: Authors’ compilation based on analysis described in text.

( 6 ) Overview

kindergarten through grade 2,15 but there providing professional development for are also reasonable strategies to use for stu- interventionists. dents in more advanced grades.16 We stress that no one screening measure is perfect Next, we highlight several areas of re- and that schools need to monitor the prog- search that have produced promising find- ress of students who score slightly above or ings in mathematics interventions. These slightly below any screening cutoff score. include systematically teaching students about the problem types associated with Recommendations 2 though 6 address the a given operation and its inverse (such as content of tier 2 and tier 3 interventions problem types that indicate addition and and the types of instructional strategies subtraction) (recommendation 4).18 We also that should be used. In recommendation 2, recommend practices to help students we translate the guidance by the National translate abstract symbols and numbers Mathematics Advisory Panel (2008) and into meaningful visual representations the National Council of Teachers of Math- (recommendation 5).19 Another feature ematics Curriculum Focal Points (2006) that we identify as crucial for long-term into suggestions for the content of inter- success is systematic instruction to build vention curricula. We argue that the math- quick retrieval of basic arithmetic facts ematical focus and the in-depth coverage (recommendation 6). Some evidence exists advocated for proficient students are also supporting the allocation of time in the in- necessary for students with mathematics tervention to practice fact retrieval using difficulties. For most students, the content flash cards or computer software.20 There of interventions will include foundational is also evidence that systematic work with concepts and skills introduced earlier in properties of operations and counting the student’s career but not fully under- strategies (for younger students) is likely stood and mastered. Whenever possible, to promote growth in other areas of math- links should be made between founda- ematics beyond fact retrieval.21 tional mathematical concepts in the intervention and grade-level material. The final two recommendations address other considerations in implementing tier At the center of the intervention recom- 2 and tier 3 interventions. Recommenda- mendations is that instruction should be tion 7 addresses the importance of moni- systematic and explicit (recommendation toring the progress of students receiving 3). This is a recurrent theme in the body of valid scientific research.17 We explore 18. Jitendra et al. (1998); Xin, Jitendra, and Deat- the multiple meanings of explicit instruc- line-Buchman (2005); Darch, Carnine, and Gersten tion and indicate which components of (1984); Fuchs et al. (2003a); Fuchs et al. (2003b); explicit instruction appear to be most re- Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, lated to improved student outcomes. We and Finelli (2004); Fuchs, Fuchs, Craddock et al. (2008) Fuchs, Seethaler et al. (2008). believe this information is important for districts and state departments to have 19. Artus and Dyrek (1989); Butler et al. (2003); Darch, Carnine, and Gersten (1984); Fuchs et as they consider selecting materials and al. (2005); Fuchs, Seethaler et al. (2008); Fuchs, Powell et al. (2008); Fuchs, Fuchs, Craddock et 15. Gersten, Jordan, and Flojo (2005); Gersten, al. (2008); Jitendra et al. (1998); Walker and Po- Clarke, and Jordan (2007). teet (1989); Wilson and Sindelar (1991); Witzel 16. Jiban and Deno (2007); Foegen, Jiban, and (2005); Witzel, Mercer, and Miller (2003); Wood- Deno (2007). ward (2006). 17. Darch, Carnine, and Gersten (1984); Fuchs 20. Beirne-Smith (1991); Fuchs, Seethaler et al. et al. (2003a); Jitendra et al. (1998); Schunk and (2008); Fuchs et al. (2005); Fuchs, Fuchs, Hamlett Cox (1986); Tournaki (2003); Wilson and Sindelar et al. (2006); Fuchs, Powell et al. (2008). (1991). 21. Tournaki (2003); Woodward (2006).

( 7 ) Overview

interventions. Specific types of formative and those with scores slightly above or assessment approaches and measures are below the cutoff score on screening mea- described. We argue for two types of ongo- sures with broader measures of mathemat- ing assessment. One is the use of curricu- ics proficiency. This information provides lum-embedded assessments that gauge how the school with a sense of how the overall well students have learned the material in mathematics program (including tier 1, tier that day’s or week’s lesson(s). The panel 2, and tier 3) is affecting a given student. believes this information is critical for interventionists to determine whether they Recommendation 8 addresses the impor- need to spend additional time on a topic. It tant issue of motivation. Because many of also provides the interventionist and other the students struggling with mathematics school personnel with information that have experienced failure and frustration can be used to place students in groups by the time they receive an intervention, within tiers. In addition, we recommend we suggest tools that can encourage active that schools regularly monitor the prog- engagement of students and acknowledge ress of students receiving interventions student accomplishments.

( 8 ) Scope of the The scope of this guide does not include recommendations for special education practice guide referrals. Although enhancing the validity of special education referrals remains Our goal is to provide evidence-based sug- important and an issue of ongoing discus- gestions for screening students for mathe- sion23 and research,24 we do not address matics difficulties, providing interventions it in this practice guide, in part because to students who are struggling, and moni- empirical evidence is lacking. toring student responses to the interventions. RtI intentionally cuts across the bor- The discussion of tier 1 in this guide re- ders of special and general education and volves only around effective screening, be- involves school-wide collaboration. There- cause recommendations for general class- fore, our target audience for this guide in- room mathematics instruction were beyond cludes teachers, special educators, school the scope of this guide. For this reason, psychologists and counselors, as well as studies of effective general mathematics administrators. Descriptions of the ma- instruction practices were not included in terials and instructional content in tier 2 the evidence base for this guide.25 and tier 3 interventions may be especially useful to school administrators selecting The studies reviewed for this guide in- interventions, while recommendations cluded two types of comparisons among that relate to content and pedagogy will groups. First, several studies of tier 2 in- be most useful to interventionists.22 terventions compare students receiving multicomponent tier 2 interventions with The focus of this guide is on providing students receiving only routine classroom RtI interventions in mathematics for stu- instruction.26 This type of study provides dents in kindergarten through grade 8. This evidence of the effectiveness of providing broad grade range is in part a response tier 2 interventions but does not permit to the recent report of the National Math- conclusions about which component is ematics Advisory Panel (2008), which em- most effective. The reason is that it is not phasized a unified progressive approach possible to identify whether one particular to promoting mathematics proficiency for component or a combination of compo- elementary and middle schools. Moreover, nents within a multicomponent interven- given the growing number of initiatives tion produced an effect. Second, several aimed at supporting students to succeed in algebra, the panel believes it essential 23. Kavale and Spaulding (2008); Fuchs, Fuchs, to provide tier 2 and tier 3 interventions to and Vaughn (2008); VanDerHeyden, Witt, and struggling students in grades 4 through 8. Gilbertson (2007). Because the bulk of research on mathemat- 24. Fuchs, Fuchs, Compton et al. (2006). ics interventions has focused on students 25. There were a few exceptions in which general in kindergarten through grade 4, some rec- mathematics instruction studies were included in ommendations for students in older grades the evidence base. When the effects of a general are extrapolated from this research. mathematics instruction program were specified for low-achieving or disabled students and the intervention itself appeared applicable to teaching tier 2 or tier 3 (e.g., teaching a specific operational strategy), we included them in this study. Note that disabled students were predominantly 22. Interventionists may be any number of school learning disabled. personnel, including classroom teachers, special 26. For example, Fuchs, Seethaler et al. (2008) educators, school psychologists, paraprofession- examined the effects of providing supplemen- als, and mathematics coaches and specialists. tal tutoring (i.e., a tier 2 intervention) relative to The panel does not specify the interventionist. regular classroom instruction (i.e., tier 1).

( 9 ) Scope of the practice guide

other studies examined the effects of two or estimation), concepts (knowledge of methods of tier 2 or tier 3 instruction.27 properties of operations, concepts involv- This type of study offers evidence for the ing rational numbers, prealgebra con- effectiveness of one approach to teaching cepts), problem solving (word problems), within a tier relative to another approach and measures of general mathematics and assists with identifying the most ben- achievement. Measures of fact fluency eficial approaches for this population. were also included because quick retrieval of basic arithmetic facts is essential for The panel reviewed only studies for prac- success in mathematics and a persistent tices that sought to improve student math- problem for students with difficulties in ematics outcomes. The panel did not con- mathematics.28 sider interventions that improved other academic or behavioral outcomes. Instead, Technical terms related to mathematics the panel focused on practices that ad- and technical aspects of assessments (psy- dressed the following areas of mathematics chometrics) are defined in a glossary at the proficiency:operations (either computation end of the recommendations.

27. For example, Tournaki (2003) examined the effects of providing supplemental tutoring in an operations strategy (a tier 2 intervention) relative to supplemental tutoring with a drill and practice 28. Geary (2004); Jordan, Hanich, and Kaplan approach (also a tier 2 intervention). (2003).

( 10 ) Checklist for carrying out the in-depth coverage of rational numbers as recommendations well as advanced topics in whole number arithmetic (such as long division). Recommendation 1. Screen all students to identify those at risk for  Districts should appoint committees, potential mathematics difficulties and including experts in mathematics instruc- provide interventions to students tion and mathematicians with knowledge identified as at risk. of elementary and middle school mathematics curricula, to ensure that specific  As a district or school sets up a screen- criteria are covered in-depth in the cur- ing system, have a team evaluate potential riculum they adopt. screening measures. The team should select measures that are efficient and reason- Recommendation 3. Instruction during ably reliable and that demonstrate predic- the intervention should be explicit and tive validity. Screening should occur in the systematic. This includes providing beginning and middle of the year. models of proficient problem solving, verbalization of thought processes,  Select screening measures based on guided practice, corrective feedback, the content they cover, with an emphasis and frequent cumulative review. on critical instructional objectives for each grade.  Ensure that instructional materials are systematic and explicit. In particular, they  In grades 4 through 8, use screen- should include numerous clear models of ing data in combination with state testing easy and difficult problems, with accom- results. panying teacher think-alouds.

 Use the same screening tool across a  Provide students with opportunities district to enable analyzing results across to solve problems in a group and commu- schools. nicate problem-solving strategies.

Recommendation 2. Instructional  Ensure that instructional materials in- materials for students receiving clude cumulative review in each session. interventions should focus intensely on in-depth treatment of whole Recommendation 4. Interventions numbers in kindergarten through should include instruction on solving grade 5 and on rational numbers in word problems that is based on grades 4 through 8. These materials common underlying structures. should be selected by committee.  Teach students about the structure of  For students in kindergarten through various problem types, how to categorize grade 5, tier 2 and tier 3 interventions problems based on structure, and how to should focus almost exclusively on prop- determine appropriate solutions for each erties of whole numbers and operations. problem type. Some older students struggling with whole numbers and operations would  Teach students to recognize the com- also benefit from in-depth coverage of mon underlying structure between famil- these topics. iar and unfamiliar problems and to transfer known solution methods from familiar to  For tier 2 and tier 3 students in grades unfamiliar problems. 4 through 8, interventions should focus on

( 11 ) Checklist for carrying out the recommendations

Recommendation 5. Intervention  Teach students in grades 2 through materials should include opportunities 8 how to use their knowledge of proper- for students to work with visual ties, such as commutative, associative, representations of mathematical and distributive law, to derive facts in ideas and interventionists should their heads. be proficient in the use of visual representations of mathematical ideas. Recommendation 7. Monitor the progress of students receiving  Use visual representations such as supplemental instruction and other number lines, arrays, and strip diagrams. students who are at risk.

 If visuals are not sufficient for develop-  Monitor the progress of tier 2, tier 3, ing accurate abstract thought and answers, and borderline tier 1 students at least once use concrete manipulatives first. Although a month using grade-appropriate general this can also be done with students in upper outcome measures. elementary and middle school grades, use of manipulatives with older students should  Use curriculum-embedded assess- be expeditious because the goal is to move ments in interventions to determine toward understanding of—and facility whether students are learning from the with—visual representations, and finally, to intervention. These measures can be used the abstract. as often as every day or as infrequently as once every other week. Recommendation 6. Interventions at all grade levels should devote about  Use progress monitoring data to re- 10 minutes in each session to building group students when necessary. fluent retrieval of basic arithmetic facts. Recommendation 8. Include  Provide about 10 minutes per ses- motivational strategies in tier 2 and sion of instruction to build quick retrieval tier 3 interventions. of basic arithmetic facts. Consider using technology, flash cards, and other materi-  Reinforce or praise students for their als for extensive practice to facilitate au- effort and for attending to and being en- tomatic retrieval. gaged in the lesson.

 For students in kindergarten through  Consider rewarding student accom grade 2, explicitly teach strategies for ef- plishments. ficient counting to improve the retrieval of mathematics facts.  Allow students to chart their progress and to set goals for improvement.

( 12 ) Recommendation 1. aspects of what is often referred to as number sense.30 They assess various as- Screen all students to pects of knowledge of whole numbers— identify those at risk for properties, basic arithmetic operations, potential mathematics understanding of magnitude, and applying mathematical knowledge to word prob- difficulties and provide lems. Some measures contain only one interventions to aspect of number sense (such as magnitude comparison) and others assess four students identified to eight aspects of number sense. The sin- as at risk. gle-component approaches with the best ability to predict students’ subsequent mathematics performance include screen- The panel recommends that schools ing measures of students’ knowledge of and districts systematically use magnitude comparison and/or strategic universal screening to screen all counting.31 The broader, multicomponent students to determine which students measures seem to predict with slightly have mathematics difficulties and greater accuracy than single-component require research-based interventions. measures.32 Schools should evaluate and select screening measures based on their Effective approaches to screening vary in reliability and predictive validity, with efficiency, with some taking as little as 5 particular emphasis on the measures’ minutes to administer and others as long specificity and sensitivity. Schools as 20 minutes. Multicomponent measures, should also consider the efficiency of which by their nature take longer to ad- the measure to enable screening many minister, tend to be time-consuming for students in a short time. administering to an entire school population. Timed screening measures33 and Level of evidence: Moderate untimed screening measures34 have been shown to be valid and reliable. The panel judged the level of evidence supporting this recommendation to be mod- For the upper elementary grades and mid- erate. This recommendation is based on a dle school, we were able to locate fewer series of high-quality correlational studies studies. They suggest that brief early with replicated findings that show the abil- screening measures that take about 10 ity of measures to predict performance in minutes and cover a proportional sam- mathematics one year after administration pling of grade-level objectives are reason- (and in some cases two years).29 able and provide sufficient evidence of reliability.35 At the current time, this research Brief summary of evidence to area is underdeveloped. support the recommendation

A growing body of evidence suggests that 30. Berch (2005); Dehaene (1999); Okamoto and there are several valid and reliable ap- Case (1996); Gersten and Chard (1999). proaches for screening students in the pri- 31. Gersten, Jordan, and Flojo (2005). mary grades. All these approaches target 32. Fuchs, Fuchs, Compton et al. (2007). 33. For example, Clarke and Shinn (2004). 29. For reviews see Jiban and Deno (2007); Fuchs, 34. For example, Okamoto and Case (1996). Fuchs, Compton et al. (2007); Gersten, Jordan, 35. Jiban and Deno (2007); Foegen, Jiban, and and Flojo (2005). Deno (2007).

( 13 ) Recommendation 1. Screen all students to identify those at risk

How to carry out this more than 20 minutes to administer, recommendation which enables collecting a substantial amount of information in a reasonable 1. As a district or school sets up a screen- time frame. Note that many screening ing system, have a team evaluate potential measures take five minutes or less.38 We screening measures. The team should select recommend that schools select screen- measures that are efficient and reasonably ing measures that have greater effi- reliable and that demonstrate predictive va- ciency if their technical adequacy (pre- lidity. Screening should occur in the begin- dictive validity, reliability, sensitivity, ning and middle of the year. and specificity) is roughly equivalent to less efficient measures. Remember The team that selects the measures should that screening measures are intended include individuals with expertise in mea- for administration to all students in a surement (such as a school psychologist or school, and it may be better to invest a member of the district research and eval- more time in diagnostic assessment of uation division) and those with expertise in students who perform poorly on the mathematics instruction. In the opinion of universal screening measure. the panel, districts should evaluate screening measures on three dimensions. Keep in mind that screening is just a means of determining which students are likely to • Predictive validity is an index of how need help. If a student scores poorly on a well a score on a screening measure screening measure or screening battery— earlier in the year predicts a student’s especially if the score is at or near a cut later mathematics achievement. Greater point, the panel recommends monitoring predictive validity means that schools her or his progress carefully to discern can be more confident that decisions whether extra instruction is necessary. based on screening data are accurate. In general, we recommend that schools Developers of screening systems recom- and districts employ measures with mend that screening occur at least twice predictive validity coefficients of at a year (e.g., fall, winter, and/or spring).39 least .60 within a school year.36 This panel recommends that schools alle- viate concern about students just above or • Reliability is an index of the consistency below the cut score by screening students and precision of a measure. We recom- twice during the year. The second screen- mend measures with reliability coeffi- ing in the middle of the year allows another cients of .80 or higher.37 check on these students and also serves to identify any students who may have been at • Efficiency is how quickly the universal risk and grown substantially in their mathe- screening measure can be adminis- matics achievement—or those who were on- tered, scored, and analyzed for all the track at the beginning of the year but have students. As a general rule, we suggest not shown sufficient growth. The panel that a screening measure require no considers these two universal screenings to determine student proficiency as distinct 36. A coefficient of .0 indicates that there is no from progress monitoring (Recommenda- relation between the early and later scores, and tion 7), which occurs on a more frequent a coefficient of 1.0 indicates a perfect positive relation between the scores. 37. A coefficient of .0 indicates that there is no 38. Foegen, Jiban, and Deno (2007); Fuchs, Fuchs, relation between the two scores, and a coeffi- Compton et al. (2007); Gersten, Clarke, and Jordan cient of 1.0 indicates a perfect positive relation (2007). between the scores. 39. Kaminski et al. (2008); Shinn (1989).

( 14 ) Recommendation 1. Screen all students to identify those at risk

basis (e.g., weekly or monthly) with a select these grade levels, districts, county offices, group of intervention students in order to or state departments may need to develop monitor response to intervention. additional screening and diagnostic measures or rely on placement tests provided 2. Select screening measures based on the by developers of intervention curricula. content they cover, with an emphasis on critical instructional objectives for each grade. 4. Use the same screening tool across a district to enable analyzing results across schools. The panel believes that content covered in a screening measure should reflect the The panel recommends that all schools instructional objectives for a student’s within a district use the same screening grade level, with an emphasis on the most measure and procedures to ensure ob- critical content for the grade level. The Na- jective comparisons across schools and tional Council of Teachers of Mathematics within a district. Districts can use results (2006) released a set of focal points for from screening to inform instructional de- each grade level designed to focus instruc- cisions at the district level. For example, tion on critical concepts for students to one school in a district may consistently master within a specific grade. Similarly, have more students identified as at risk, the National Mathematics Advisory Panel and the district could provide extra re- (2008) detailed a route to preparing all sources or professional development to students to be successful in algebra. In the that school. The panel recommends that lower elementary grades, the core focus of districts use their research and evaluation instruction is on building student under- staff to reevaluate screening measures an- standing of whole numbers. As students nually or biannually. This entails exam- establish an understanding of whole num- ining how screening scores predict state bers, rational numbers become the focus testing results and considering resetting of instruction in the upper elementary cut scores or other data points linked to grades. Accordingly, screening measures instructional decisionmaking. used in the lower and upper elementary grades should have items designed to as- Potential roadblocks and solutions sess student’s understanding of whole and rational number concepts—as well as com- Roadblock 1.1. Districts and school person- putational proficiency. nel may face resistance in allocating time resources to the collection of screening data. 3. In grades 4 through 8, use screening data in combination with state testing results. Suggested Approach. The issue of time and personnel is likely to be the most sig- In the panel’s opinion, one viable option nificant obstacle that districts and schools that schools and districts can pursue is to must overcome to collect screening data. use results from the previous year’s state Collecting data on all students will require testing as a first stage of screening. Students structuring the data collection process to who score below or only slightly above a be efficient and streamlined. benchmark would be considered for subsequent screening and/or diagnostic or The panel notes that a common pitfall is placement testing. The use of state testing a long, drawn-out data collection process, results would allow districts and schools with teachers collecting data in their class- to combine a broader measure that covers rooms “when time permits.” If schools are more content with a screening measure that allocating resources (such as providing an is narrower but more focused. Because of intervention to students with the 20 low- the lack of available screening measures at est scores in grade 1), they must wait until

( 15 ) Recommendation 1. Screen all students to identify those at risk

all the data have been collected across high on the previous spring’s state as- classrooms, thus delaying the delivery sessment, additional screening typically of needed services to students. Further- is not required. more, because many screening measures are sensitive to instruction, a wide gap Roadblock 1.3. Screening measures may between when one class is assessed and identify students who do not need services another is assessed means that many stu- and not identify students who do need dents in the second class will have higher services. scores than those in the first because they were assessed later. Suggested Approach. All screening measures will misidentify some students as One way to avoid these pitfalls is to use data either needing assistance when they do collection teams to screen students in a not (false positive) or not needing assis- short period of time. The teams can consist tance when they do (false negative). When of teachers, special education staff includ- screening students, educators will want to ing such specialists as school psychologists, maximize both the number of students Title I staff, principals, trained instructional correctly identified as at risk—a measure’s assistants, trained older students, and/or sensitivity—and the number of students local college students studying child devel- correctly identified as not at risk—a mea- opment or school psychology. sure’s specificity. As illustrated in table 3, screening students to determine risk can Roadblock 1.2. Implementing universal result in four possible categories indicated screening is likely to raise questions such by the letters A, B, C, and D. Using these as, “Why are we testing students who are categories, sensitivity is equal to A/(A + C) doing fine?” and specificity is equal to D/(B + D).

Suggested Approach. Collecting data Table 3. Sensitivity and specificity on all students is new for many districts STUDENTS and schools (this may not be the case for ACTUALLY AT RISK elementary schools, many of which use Yes No screening assessments in reading).40 But screening allows schools to ensure that all STUDENTS Yes A (true B (false IDENTIFIED positives) positives) students who are on track stay on track AS BEING and collective screening allows schools to No C (false D (true AT RISK evaluate the impact of their instruction negatives) negatives) on groups of students (such as all grade 2 students). When schools screen all students, a distribution of achievement from The sensitivity and specificity of a mea- high to low is created. If students consid- sure depend on the cut score to classify ered not at risk were not screened, the children at risk.41 If a cut score is high distribution of screened students would (where all students below the cut score are consist only of at-risk students. This could considered at risk), the measure will have create a situation where some students at a high degree of sensitivity because most the “top” of the distribution are in real- students who truly need assistance will be ity at risk but not identified as such. For upper-grade students whose scores were 41. Sensitivity and specificity are also influenced by the discriminant validity of the measure and 40. U.S. Department of Education, Office of Plan- its individual items. Measures with strong item ning, Evaluation and Policy Development, Policy discrimination are more likely to correctly iden- and Program Studies Service (2006). tify students’ risk status.

( 16 ) Recommendation 1. Screen all students to identify those at risk

identified as at risk. But the measure will those resources when using screening have low specificity since many students data to make instructional decisions. Dis- who do not need assistance will also be tricts may find that on a nationally normed identified as at risk. Similarly, if a cut score screening measure, a large percentage of is low, the sensitivity will be lower (some their students (such as 60 percent) will be students in need of assistance may not be classified as at risk. Districts will have to identified as at risk), whereas the specific- determine the resources they have to pro- ity will be higher (most students who do vide interventions and the number of stu- not need assistance will not be identified dents they can serve with their resources. as at risk). This may mean not providing interventions at certain grade levels or providing Schools need to be aware of this tradeoff interventions only to students with the between sensitivity and specificity, and lowest scores, at least in the first year of the team selecting measures should be implementation. aware that decisions on cut scores can be somewhat arbitrary. Schools that set a cut There may also be cases when schools score too high run the risk of spending re- identify large numbers of students at risk sources on students who do not need help, in a particular area and decide to pro- and schools that set a cut score too low run vide instruction to all students. One par- the risk of not providing interventions to ticularly salient example is in the area of students who are at risk and need extra in- fractions. Multiple national assessments struction. If a school or district consistently show many students lack proficiency in finds that students receiving intervention fractions,42 so a school may decide that, do not need it, the measurement team rather than deliver interventions at the should consider lowering the cut score. individual child level, they will provide a school-wide intervention to all students. A Roadblock 1.4. Screening data may iden- school-wide intervention can range from a tify large numbers of students who are at supplemental fractions program to profes- risk and schools may not immediately have sional development involving fractions. the resources to support all at-risk students. This will be a particularly severe problem in low-performing Title I schools.

Suggested Approach. Districts and schools need to consider the amount of 42. National Mathematics Advisory Panel resources available and the allocation of (2008); Lee, Grigg, and Dion (2007).

( 17 ) Recommendation 2. Level of evidence: Low

Instructional materials The panel judged the level of evidence for students receiving supporting this recommendation to be low. interventions should This recommendation is based on the professional opinion of the panel and several focus intensely on recent consensus documents that reflect in-depth treatment input from mathematics educators and research mathematicians involved in issues of whole numbers in related to kindergarten through grade 12 kindergarten through mathematics education.44 grade 5 and on rational Brief summary of evidence to numbers in grades support the recommendation 4 through 8. These The documents reviewed demonstrate a materials should be growing professional consensus that coverage of fewer mathematics topics in more selected by committee. depth and with coherence is important for all students.45 Milgram and Wu (2005) suggested that an intervention curriculum The panel recommends that individuals for at-risk students should not be over- knowledgeable in instruction and simplified and that in-depth coverage of mathematics look for interventions that key topics and concepts involving whole focus on whole numbers extensively numbers and then rational numbers is in kindergarten through grade 5 and critical for future success in mathematics. on rational numbers extensively in The National Council of Teachers of Math- grades 4 through 8. In all cases, the ematics (NCTM) Curriculum Focal Points specific content of the interventions will (2006) called for the end of brief ventures be centered on building the student’s into many topics in the course of a school foundational proficiencies. In making year and also suggested heavy emphasis on this recommendation, the panel is instruction in whole numbers and rational drawing on consensus documents numbers. This position was reinforced by developed by experts from mathematics the 2008 report of the National Mathematics education and research mathematicians Advisory Panel (NMAP), which provided de- that emphasized the importance of tailed benchmarks and again emphasized these topics for students in general.43 in-depth coverage of key topics involving We conclude that the coverage of whole numbers and rational numbers as fewer topics in more depth, and crucial for all students. Although the latter with coherence, is as important, and two documents addressed the needs of all probably more important, for students students, the panel concludes that the in- who struggle with mathematics. depth coverage of key topics is especially

44. National Council of Teachers of Mathemat- ics (2006); National Mathematics Advisory Panel (2008); Milgram and Wu (2005). 45. National Mathematics Advisory Panel (2008); 43. National Council of Teachers of Mathemat- Schmidt and Houang (2007); Milgram and Wu ics (2006); National Mathematics Advisory Panel (2005); National Council of Teachers of Math- (2008). ematics (2006).

( 18 ) Recommendation 2. Instructional materials for students receiving interventions

important for students who struggle with the reasoning that underlies algorithms for mathematics. addition and subtraction of whole numbers, as well as solving problems involv- How to carry out this ing whole numbers. This focus should in- recommendation clude understanding of the base-10 system (place value). 1. For students in kindergarten through grade 5, tier 2 and tier 3 interventions should Interventions should also include materi- focus almost exclusively on properties of als to build fluent retrieval of basic arith- whole numbers46 and operations. Some metic facts (see recommendation 6). Ma- older students struggling with whole num- terials should extensively use—and ask bers and operations would also benefit from students to use—visual representations of in-depth coverage of these topics. whole numbers, including both concrete and visual base-10 representations, as well In the panel’s opinion, districts should as number paths and number lines (more review the interventions they are con- information on visual representations is sidering to ensure that they cover whole in recommendation 5). numbers in depth. The goal is proficiency and mastery, so in-depth coverage with 2. For tier 2 and tier 3 students in grades 4 extensive review is essential and has through 8, interventions should focus on in- been articulated in the NCTM Curriculum depth coverage of rational numbers as well Focal Points (2006) and the benchmarks as advanced topics in whole number arith- determined by the National Mathematics metic (such as long division). Advisory Panel (2008). Readers are recommended to review these documents.47 The panel believes that districts should review the interventions they are consid- Specific choices for the content of interven- ering to ensure that they cover concepts tions will depend on the grade level and involving rational numbers in depth. The proficiency of the student, but the focus focus on rational numbers should include for struggling students should be on whole understanding the meaning of fractions, numbers. For example, in kindergarten decimals, ratios, and percents, using visual through grade 2, intervention materials representations (including placing fractions would typically include significant atten- and decimals on number lines,48 see recom- tion to counting (e.g., counting up), num- mendation 5), and solving problems with ber composition, and number decomposi- fractions, decimals, ratios, and percents. tion (to understand place-value multidigit operations). Interventions should cover the In the view of the panel, students in meaning of addition and subtraction and grades 4 through 8 will also require additional work to build fluent retrieval of 46. Properties of numbers, including the associa- basic arithmetic facts (see recommenda- tive, commutative, and distributive properties. tion 6), and some will require additional 47. More information on the National Mathemat- work involving basic whole number topics Advisory Panel (2008) report is available at ics, especially for students in tier 3. In the www.ed.gov/about/bdscomm/list/mathpanel/ index.html. More information on the National opinion of the panel, accurate and fluent Council of Teachers of Mathematics Curricu- lum Focal Points is available at www.nctm.org/ focalpoints. Documents elaborating the National 48. When using number lines to teach rational Council of Teachers of Mathematics Curriculum numbers for students who have difficulties, it is Focal Points are also available (see Beckmann et important to emphasize that the focus is on the al., 2009). For a discussion of why this content is length of the segments between the whole num- most relevant, see Milgram and Wu (2005). ber marks (rather than counting the marks).

( 19 ) Recommendation 2. Instructional materials for students receiving interventions

arithmetic with whole numbers is neces- panel’s view, the intervention program sary before understanding fractions. The should include an assessment to assist in panel acknowledges that there will be placing students appropriately in the in- periods when both whole numbers and tervention curriculum. rational numbers should be addressed in interventions. In these cases, the balance Potential roadblocks and solutions of concepts should be determined by the student’s need for support. Roadblock 2.1. Some interventionists may worry if the intervention program 3. Districts should appoint committees, in- is not aligned with the core classroom cluding experts in mathematics instruction instruction. and mathematicians with knowledge of elementary and middle school mathematics Suggested Approach. The panel believes curriculum, to ensure that specific criteria that alignment with the core curriculum is (described below) are covered in depth in not as critical as ensuring that instruction the curricula they adopt. builds students’ foundational proficiencies. Tier 2 and tier 3 instruction focuses In the panel’s view, intervention materials on foundational and often prerequisite should be reviewed by individuals with skills that are determined by the students’ knowledge of mathematics instruction and rate of progress. So, in the opinion of the by mathematicians knowledgeable in el- panel, acquiring these skills will be neces- ementary and middle school mathematics. sary for future achievement. Additionally, They can often be experts within the district, because tier 2 and tier 3 are supplemental, such as mathematics coaches, mathematics students will still be receiving core class- teachers, or department heads. Some dis- room instruction aligned to a school or tricts may also be able to draw on the exper- district curriculum (tier 1). tise of local university mathematicians. Roadblock 2.2. Intervention materials Reviewers should assess how well interven- may cover topics that are not essential to tion materials meet four criteria. First, the building basic competencies, such as data materials integrate computation with solv- analysis, measurement, and time. ing problems and pictorial representations rather than teaching computation apart Suggested Approach. In the panel’s opin- from problem-solving. Second, the mate- ion, it is not necessary to cover every topic rials stress the reasoning underlying cal- in the intervention materials. Students will culation methods and focus student atten- gain exposure to many supplemental top- tion on making sense of the mathematics. ics (such as data analysis, measurement, Third, the materials ensure that students and time) in general classroom instruc- build algorithmic proficiency. Fourth, the tion (tier 1). Depending on the student’s materials include frequent review for both age and proficiency, it is most important consolidating and understanding the links to focus on whole and rational numbers in of the mathematical principles. Also in the the interventions.

( 20 ) Recommendation 3. mathematics.49 Our panel supports this recommendation and believes Instruction during the that districts and schools should select intervention should be materials for interventions that reflect this orientation. In addition, professional explicit and systematic. development for interventionists should This includes providing contain guidance on these components models of proficient of explicit instruction. problem solving, Level of evidence: Strong verbalization of Our panel judged the level of evidence thought processes, supporting this recommendation to be guided practice, strong. This recommendation is based on six randomized controlled trials that met corrective feedback, WWC standards or met standards with and frequent reservations and that examined the effectiveness of explicit and systematic in- cumulative review. struction in mathematics interventions.50 These studies have shown that explicit and systematic instruction can significantly The National Mathematics Advisory improve proficiency in word problem solv- Panel defines explicit instruction as ing51 and operations52 across grade levels follows (2008, p. 23): and diverse student populations.

• “Teachers provide clear models for Brief summary of evidence to support solving a problem type using an the recommendation array of examples.” The results of six randomized controlled • “Students receive extensive practice trials of mathematics interventions show in use of newly learned strategies extensive support for various combina- and skills.” tions of the following components of explicit and systematic instruction: teacher • “Students are provided with demonstration,53 student verbalization,54 opportunities to think aloud (i.e., talk through the decisions they make and the steps they take).” 49. National Mathematics Advisory Panel (2008). 50. Darch, Carnine, and Gersten (1984); Fuchs • “Students are provided with et al. (2003a); Jitendra et al. (1998); Schunk and extensive feedback.” Cox (1986); Tournaki (2003); Wilson and Sindelar (1991). The NMAP notes that this does not mean 51. Darch, Carnine, and Gersten (1984); Jitendra that all mathematics instruction should et al. (1998); Fuchs et al. (2003a); Wilson and Sin- be explicit. But it does recommend that delar (1991). struggling students receive some explicit 52. Schunk and Cox (1986); Tournaki (2003). instruction regularly and that some 53. Darch, Carnine, and Gersten (1984); Jitendra et al. (1998); Fuchs et al. (2003a); Schunk and of the explicit instruction ensure that Cox (1986); Tournaki (2003); Wilson and Sindelar students possess the foundational skills (1991). and conceptual knowledge necessary 54. Jitendra et al. (1998); Fuchs et al. (2003a); for understanding their grade-level Schunk and Cox (1986); Tournaki (2003).

( 21 ) Recommendation 3. Instruction during the intervention should be explicit and systematic

guided practice,55 and corrective feed- Similarly, four of the six studies included back.56 All six studies examined interven- immediate corrective feedback,61 and the tions that included teacher demonstra- effects of these interventions were positions early in the lessons.57 For example, tive and significant on word problems and three studies included instruction that measures of operations skills, but the ef- began with the teacher verbalizing aloud fects of the corrective feedback compo- the steps to solve sample mathematics nent cannot be isolated from the effects of problems.58 The effects of this component other components in three cases.62 of explicit instruction cannot be evaluated from these studies because the demonstra- With only one study in the pool of six in- tion procedure was used in instruction for cluding cumulative review as part of the students in both treatment and compari- intervention,63 the support for this compo- son groups. nent of explicit instruction is not as strong as it is for the other components. But this Scaffolded practice, a transfer of control study did have statistically significant pos- of problem solving from the teacher to the itive effects in favor of the instructional student, was a component in four of the six group that received explicit instruction studies.59 Although it is not possible to parse in strategies for solving word problems, the effects of scaffolded instruction from the including cumulative review. other components of instruction, the intervention groups in each study demonstrated How to carry out this significant positive gains on word problem recommendation proficiencies or accuracy measures. 1. Ensure that instructional materials are Three of the six studies included opportu- systematic and explicit. In particular, they nities for students to verbalize the steps should include numerous clear models of to solve a problem.60 Again, although ef- easy and difficult problems, with accompa- fects of the interventions were statistically nying teacher think-alouds. significant and positive on measures of word problems, operations, or accuracy, To be considered systematic, mathematics the effects cannot be attributed to a sin- instruction should gradually build profi- gle component of these multicomponent ciency by introducing concepts in a logical interventions. order and by providing students with numerous applications of each concept. For example, a systematic curriculum builds 55. Darch, Carnine, and Gersten (1984); Jiten- student understanding of place value in dra et al. (1998); Fuchs et al. (2003a); Tournaki an array of contexts before teaching pro- (2003). cedures for adding and subtracting two- 56. Darch, Carnine, and Gersten (1984); Jitendra et al. (1998); Schunk and Cox (1986); Tournaki digit numbers with regrouping. (2003). 57. Darch, Carnine, and Gersten (1984); Fuchs Explicit instruction typically begins with et al. (2003a); Jitendra et al. (1998); Schunk and a clear unambiguous exposition of con- Cox (1986); Tournaki (2003); Wilson and Sindelar cepts and step-by-step models of how (1991). 58. Schunk and Cox (1986); Jitendra et al. (1998);�� Darch, Carnine, and Gersten (1984). 61. Darch, Carnine, and Gersten (1984); Jiten- 59. Darch, Carnine, and Gersten (1984); Fuchs dra et al. (1998); Tournaki (2003); Schunk and et al. (2003a); Jitendra et al. (1998); Tournaki Cox (1986). (2003). 62. Darch, Carnine, and Gersten (1984); Jitendra 60. Schunk and Cox (1986); Jitendra et al. (1998); et al. (1998); Tournaki (2003). Tournaki (2003). 63. Fuchs et al. (2003a).

( 22 ) Recommendation 3. Instruction during the intervention should be explicit and systematic

to perform operations and reasons for the students.67 This phase of explicit in- the procedures.64 Interventionists should struction begins with the teacher and the think aloud (make their thinking pro- students solving problems together. As cesses public) as they model each step of this phase of instruction continues, stu- the process.65,66 They should not only tell dents should gradually complete more students about the steps and procedures steps of the problem with decreasing guid- they are performing, but also allude to the ance from the teacher. Students should reasoning behind them (link to the under- proceed to independent practice when lying mathematics). they can solve the problem with little or no support from the teacher. The panel suggests that districts select instructional materials that provide inter- During guided practice, the teacher should ventionists with sample think-alouds or ask students to communicate the strate- possible scenarios for explaining concepts gies they are using to complete each step and working through operations. A crite- of the process and provide reasons for rion for selecting intervention curricula their decisions.68 In addition, the panel materials should be whether or not they recommends that teachers ask students to provide materials that help intervention- explain their solutions.69 Note that not only ists model or think through difficult and interventionists—but fellow students—can easy examples. and should communicate how they think through solving problems to the inter- In the panel’s view, a major flaw in many ventionist and the rest of the group. This instructional materials is that teachers are can facilitate the development of a shared asked to provide only one or two models language for talking about mathematical of how to approach a problem and that problem solving.70 most of these models are for easy-to-solve problems. Ideally, the materials will also Teachers should give specific feedback assist teachers in explaining the reason- that clarifies what students did correctly ing behind the procedures and problem- and what they need to improve.71 They solving methods. should provide opportunities for students to correct their errors. For example, if a 2. Provide students with opportunities to student has difficulty solving a word prob- solve problems in a group and communicate lem or solving an equation, the teacher problem-solving strategies. should ask simple questions that guide the student to solving the problem correctly. For students to become proficient in per- Corrective feedback can also include re- forming mathematical processes, explicit teaching or clarifying instructions when instruction should include scaffolded prac- students are not able to respond to ques- tice, where the teacher plays an active tions or their responses are incorrect. role and gradually transfers the work to

67. Tournaki (2003); Jitendra et al. (1998); Darch, 64. For example, Jitendra et al. (1998); Darch, Carnine, and Gersten (1984). Carnine, and Gersten (1984); Woodward (2006). 68. For example, Schunk and Cox (1986). 65. See an example in the summary of Tournaki (2003) in appendix D. 69. Schunk and Cox (1986); Tournaki (2003). 66. Darch, Carnine, and Gersten (1984); Jiten- 70. �� dra et al. (1998); Fuchs et al. (2003a); Schunk Carnine, and Gersten (1984). and Cox (1986); Tournaki (2003); Wilson and Sin- 71. Tournaki (2003); Jitendra et al. (1998); Darch, delar (1991). Carnine, and Gersten (1984).

( 23 ) Recommendation 3. Instruction during the intervention should be explicit and systematic

3. Ensure that instructional materials include As a part of professional development, be cumulative review in each session. sure to convey the benefits that extended practice (not only worksheets) and cumu- Cumulative reviews provide students with lative review can have for student per- an opportunity to practice topics previ- formance. If professional development ously covered in depth. For example, when is not an option, teachers can also work students are working with fractions, a with mathematics coaches to learn how to cumulative review activity could provide implement the intervention. them with an opportunity to solve some problems involving multiplication and di- Roadblock 3.2. Interventionists may not vision of whole numbers. In the panel’s be expert with the underlying mathemat- opinion, this review can ensure that the ics content. knowledge is maintained over time and helps students see connections between Suggested Approach. For intervention- various mathematical ideas. ists to explain a mathematical process accurately and develop a logical think-aloud, Potential roadblocks and solutions it is important for them to understand the underlying mathematics concept and the Roadblock 3.1. Interventionists may be un- mathematical reasoning for the process. familiar with how to implement an interven- Professional development should provide tion that uses explicit instruction, and some participants with in-depth knowledge of may underestimate the amount of practice the mathematics content in the interven- necessary for students in tiers 2 and 3 to tion, including the mathematical reason- master the material being taught. ing underlying procedures, formulas, and problem-solving methods.72 The panel be- Suggested Approach. Districts and lieves that when interventionists convey schools should set up professional devel- their knowledge of the content, student opment sessions for interventionists to understanding will increase, misconcep- observe and discuss sample lessons. The tions will decrease, and the chances that panel believes that it is important for pro- students solve problems by rote memory fessional development participants to ob- will be reduced. serve the intervention first hand. Watching a DVD or video of the intervention being Roadblock 3.3. The intervention materials used with students can give the partici- may not incorporate enough models, think- pants a model of how the program should alouds, practice, and cumulative review. be implemented. Suggested Approach. Intervention pro- Interventionists should also have hands- grams might not incorporate enough mod- on experience, teaching the lessons to els, think-alouds, practice, or cumulative each other and practicing with students. review to improve students’ mathematics Role-playing can give interventionists performance.73 practice with modeling and think-alouds, since it is important for them to stop and Consider using a mathematics coach or reflect before formulating an explanation specialist to develop a template listing for their thinking processes. The train- the essential parts of an effective lesson, ers can observe these activities, provide feedback on what participants did well, 72. National Mathematics Advisory Panel (2008); and offer explicit suggestions for improv- Wu (2005) http://math.berkeley.edu/~wu/ ing instruction. Northridge2004a2.pdf. 73. Jitendra et al. (1996); Carnine et al. (1997).

( 24 ) Recommendation 3. Instruction during the intervention should be explicit and systematic

including the number of models, accom- A team of teachers, guided by the math- panying think-alouds, and practice and ematics coach/specialist, can determine cumulative review items students need to the components that should be added to understand, learn, and master the content. the program.

( 25 ) Recommendation 4. Level of evidence: Strong

Interventions should The panel judged the level of evidence include instruction supporting this recommendation to be on solving word strong. This recommendation is based on nine randomized controlled trials that problems that is met WWC standards or met standards based on common with reservations and that examined the effectiveness of word problem-solving underlying structures. strategies.78 Interventions that teach students the structure of problem types79— and how to discriminate superficial from Students who have difficulties in substantive information to know when mathematics typically experience severe to apply the solution methods they have difficulties in solving word problems learned80—positively and marginally or related to the mathematics concepts and significantly affect proficiency in solving operations they are learning.74 This is a word problems. major impediment for future success in any math-related discipline.75 Brief summary of evidence to support the recommendation Based on the importance of building proficiency and the convergent findings Research demonstrates that instruction on from a body of high-quality research, solving word problems based on under- the panel recommends that interventions lying problem structure leads to statisti- include systematic explicit instruction cally significant positive effects on mea- on solving word problems, using sures of word problem solving.81 Three the problems’ underlying structure. randomized controlled trials isolated this Simple word problems give meaning practice. In these studies, intervention- to mathematical operations such as ists taught students to identify problems subtraction or multiplication. When of a given type by focusing on the prob- students are taught the underlying lem structure and then to design and structure of a word problem, they not execute appropriate solution strategies only have greater success in problem for each problem. These techniques typi- solving but can also gain insight into cally led to significant and positive effects the deeper mathematical ideas in word on word-problem outcomes for students problems.76 The panel also recommends systematic instruction on the structural 78. Jitendra et al. (1998); Xin, Jitendra, and Deat- connections between known, familiar line-Buchman (2005); Darch, Carnine, and Gersten word problems and unfamiliar, new (1984); Fuchs et al. (2003a); Fuchs et al. (2003b); Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, problems. By making explicit the Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. underlying structural connections (2008); Fuchs, Seethaler et al. (2008). between familiar and unfamiliar problems, 79. Jitendra et al. (1998); Xin, Jitendra, and Deat- students will know when to apply the line-Buchman (2005); Darch, Carnine, and Ger- solution methods they have learned.77 sten (1984). 80. ��Fuchs et al. (2003a); Fuchs et al. (2003b); Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, 74. Geary (2003); Hanich et al. (2001). Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. 75. National Mathematics Advisory Panel (2008); (2008); Fuchs, Seethaler et al. (2008). McCloskey (2007). 81. Jitendra et al. (1998); Xin, Jitendra, and Deat- 76. Peterson, Fennema, and Carpenter (1989). line-Buchman (2005); Darch, Carnine, and Ger- 77. Fuchs, Fuchs, Finelli et al. (2004). sten (1984).

( 26 ) Recommendation 4. Interventions should include instruction on solving word problems experiencing difficulties in mathematics How to carry out this across grade levels.82 recommendation

Six other randomized controlled trials 1. Teach students about the structure of vari- took the instructional intervention on ous problem types, how to categorize prob- problem structure a step further. They lems based on structure, and how to determine demonstrated that teaching students to appropriate solutions for each problem type. distinguish superficial from substantive information in problems also leads to Students should be explicitly taught about marginally or statistically significant posi- the salient underlying structural features tive effects on measures of word problem of each problem type.85 Problem types are solving.83 After students were explicitly groups of problems with similar math- taught the pertinent structural features ematical structures. For example, change and problem-solution methods for differ- problems describe situations in which a ent problem types, they were taught su- quantity (such as children or pencils) is perficial problem features that can change either increased or decreased (example 1). a problem without altering its underlying Change problems always include a time structure. They were taught to distinguish element. For these problems, students substantive information from superficial determine whether to add or subtract by information in order to solve problems determining whether the change in the that appear new but really fit into one of quantity is more or less. the categories of problems they already know how to solve. They were also taught Example 1. Change problems that the same underlying problem structures can be applied to problems that The two problems here are addition and are presented in graphic form (for exam- subtraction problems that students may ple, with tables or maps). These are pre- be tempted to solve using an incorrect op- cisely the issues that often confuse and eration. In each case, students can draw a derail students with difficulties in math- simple diagram like the one shown below, ematics. These six studies consistently record the known quantities (two of three demonstrated marginally or statistically of A, B, and C) and then use the diagram to significant positive effects on an array decide whether addition or subtraction is of word problem-solving proficiencies the correct operation to use to determine for students experiencing difficulties in the unknown quantity. mathematics.84

Problem 1. Brad has a bottlecap collection. After Madhavi gave Brad 28 more bottle- 82. Jitendra et al. (1998); Xin, Jitendra, and Deat- caps, Brad had 111 bottlecaps. How many line-Buchman (2005); Darch, Carnine, and Ger- bottlecaps did Brad have before Madhavi sten (1984). gave him more? 83. �� Problem 2. Brad has a bottlecap collection. Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, After Brad gave 28 of his bottlecaps to Mad- Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. havi, he had 83 bottlecaps left. How many (2008); Fuchs, Seethaler et al. (2008). bottlecaps did Brad have before he gave 84. �� Madhavi some? Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. (2008); Fuchs, Seethaler et al. (2008). 85. Xin, Jitendra, and Deatline-Buchman (2005).

( 27 ) Recommendation 4. Interventions should include instruction on solving word problems

In contrast, compare problems have no 2. Teach students to recognize the common time element (example 2). They focus on underlying structure between familiar and comparisons between two different types unfamiliar problems and to transfer known of items in two different sets (pears and solution methods from familiar to unfamil- apples, boys and girls, hot and cold items). iar problems. Students add or subtract by determining whether they need to calculate the A known familiar problem often appears unknown difference (subtract), unknown as a new and unfamiliar problem to a stu- compared amount (add), or unknown ref- dent because of such superficial changes erent amount (subtract). as format changes (whether it is written in traditional paragraph form or as an adver- tisement for a brochure), key vocabulary Example 2. Compare problems changes (half, one-half, ½), or the inclusion of irrelevant information (additional story elements such as the number of buttons on a child’s shirt or the size of a storage container for a compare problem).88 These superficial changes are irrelevant to understanding the mathematical demands of a problem. But while focusing on these irrelevant superficial changes, students can find it difficult to discern the critical Although these problem types seem simple common underlying structure between the and intuitive to adults and mathematically new and the old problems and to apply the precocious students, they are not neces- solution that is part of their repertoire to sarily obvious for students requiring math- the new unfamiliar problem. ematics interventions. To build understanding of each problem type, we recommend To facilitate the transfer of the known so- initially teaching solution rules (or guiding lution from the familiar to the unfamiliar questions that lead to a solution equation) problem, students should first be shown for each problem type through fully and explicitly that not all pieces of information partially worked examples, followed by in the problem are relevant to discerning student practice in pairs.86 the underlying problem structure.89 Teach- ers should explain these irrelevant superfi- Visual representations such as those in ex- cial features explicitly and systematically, ample 2 can be effective for teaching stu- as described in recommendation 3.90 This dents how to categorize problems based instruction may be facilitated by the use on their structure and determine a solu- of a poster displayed in the classroom that tion method appropriate for the underlying lists the ways familiar problems can be- structure (see recommendation 5 for more come unfamiliar because of new wording or information on visual representations).87 situations (such as information displayed in Teachers can present stories with unknown information and work with students in 88. Fuchs et al. (2003a); Fuchs et al. (2003b); using diagrams to identify the problem Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, type and transform the information in the Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. (2008); Fuchs, Seethaler et al. (2008). diagram into a mathematics equation to 89. Fuchs et al. (2003a); Fuchs et al. (2003b); solve for the unknown quantity. Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. 86. Fuchs, Fuchs, Finelli et al. (2004). (2008); Fuchs, Seethaler et al. (2008). 87. Xin, Jitendra, and Deatline-Buchman (2005). 90. Fuchs, Fuchs, Finelli et al. (2004).

( 28 ) Recommendation 4. Interventions should include instruction on solving word problems chart versus paragraph form) or the ways Potential roadblocks and solutions relevant to problem type. The students must also be provided with opportunities Roadblock 4.1. In the opinion of the panel, to explain why a piece of information is the curricular material may not classify relevant or irrelevant.91 problems into problem types.

We suggest that students practice sets of Suggested Approach. The interventionist problems with varied superficial features may need the help of a mathematics coach, and cover stories. Students who know how a mathematics specialist, or a district or to recognize and solve a “change” problem state curriculum guide in determining type with whole numbers should know the problem types and an instructional that they can apply the same strategy sequence for teaching them to students. to a structurally similar word problem The key issue is that students are taught that looks different because of changes in to understand a set of problem structures wording and the presence of additional related to the mathematics they are learn- story elements (example 3).92 ing in their intervention.

Roadblock 4.2. As problems get complex, Example 3. Solving different so will the problem types and the task of problems with the same strategy discriminating among them.

• Mike wants to buy 1 pencil for each Suggested Approach. As problems get of his friends. Each packet of pencils more intricate (such as multistep problems), contains 12 pencils. How many pack- it becomes more difficult for students to ets does Mike have to buy to give 1 determine the problem type, a critical step pencil to each of his 13 friends? that leads to solving the problem correctly. It is important to explicitly and systemati- • Mike wants to buy 1 pencil for each of cally teach students how to differentiate his friends. Sally wants to buy 10 pen- one problem type from another. cils. Each box of pencils contains 12 pencils. How many boxes does Mike Interventionists will need high-quality have to buy to give 1 pencil to each professional development to ensure that of his 13 friends? they convey the information clearly and accurately. The professional development program should include opportunities for participants to determine problem types, justify their responses, and practice explaining and modeling problem types to peers and children. Trainers should pro- 91. Fuchs, Fuchs, Craddock et al. (2008); Fuchs, vide constructive feedback during the Fuchs, Finelli et al. (2004); Fuchs, Fuchs, Prentice practice sessions by telling participants et al. (2004). both what they did well and what aspects 92. Fuchs, Fuchs, Finelli et al. (2004). of their instruction need improvement.

( 29 ) Recommendation 5. used in problem solving. Occasional and unsystematic exposure (the norm Intervention materials in many classrooms) is insufficient and should include does not facilitate understanding of the relationship between the abstract opportunities for symbols of mathematics and various students to work with visual representations. visual representations Level of evidence: Moderate of mathematical ideas and interventionists The panel judged the level of evidence supporting this recommendation to be mod- should be proficient erate. This recommendation is based on in the use of visual 13 randomized controlled trials that met WWC standards or met standards with representations of reservations.94 These studies provide sup- mathematical ideas. port for the systematic use of visual representations or manipulatives to improve achievement in general mathematics,95 A major problem for students who prealgebra concepts,96 word problems,97 struggle with mathematics is weak and operations.98 But these representations understanding of the relationships were part of a complex multicomponent between the abstract symbols intervention in each of the studies. So, it is of mathematics and the various difficult to judge the impact of the repre- visual representations.93 Student sentation component alone, and the panel understanding of these relationships believes that a moderate designation is ap- can be strengthened through the propriate for the level of evidence for this use of visual representations of recommendation. mathematical concepts such as solving equations, fraction equivalence, and Brief summary of evidence to the commutative property of addition support the recommendation and multiplication (see the glossary). Such representations may include Research shows that the systematic use of number lines, graphs, simple drawings visual representations and manipulatives of concrete objects such as blocks or may lead to statistically significant or sub- cups, or simplified drawings such as stantively important positive gains in math ovals to represent birds.

94. Artus and Dyrek (1989); Butler et al. (2003); In the view of the panel, the ability Darch, Carnine, and Gertsen (1984); Fuchs et al. to express mathematical ideas using (2005); Fuchs, Seethaler et al. (2008); Fuchs, Powell visual representations and to convert et al. (2008); Fuchs, Fuchs, Craddock et al. (2008); visual representations into symbols is Jitendra et al. (1998); Walker and Poteet (1989); Wilson and Sindelar (1991); Witzel (2005); Witzel, critical for success in mathematics. A Mercer, and Miller (2003); Woodward (2006). major goal of interventions should be 95. Artus and Dyrek (1989); Fuchs et al. (2005). to systematically teach students how 96. Witzel, Mercer, and Miller (2003). to develop visual representations and 97. Darch, Carnine, and Gersten (1984); Fuchs et how to transition these representations al. (2005); Fuchs, Seethaler et al. (2008); Fuchs, to standard symbolic representations Fuchs, Craddock et al. (2008); Jitendra et al. (1998); Wilson and Sindelar (1991). 93. Hecht, Vagi, and Torgesen (2007). 98. Woodward (2006).

( 30 ) Recommendation 5. Intervention materials should include opportunities for the student

achievement.99 Four studies used visual manipulatives and visual representations representations to help pave the way for to promote understanding of math at a students to understand the abstract version more abstract level.108 One of these inter- of the representation.100 For example, one ventions positively affected general math of the studies taught students to use visual achievement,109 but the other had no effect representations such as number lines to on outcome measures tested.110 In the other understand mathematics facts.101 The four four studies, manipulatives and visual rep- studies demonstrated gains in mathematics resentations were presented to the students facts and operations102 and word problem sequentially to promote understanding proficiencies,103 and may provide evidence at a more abstract level.111 One interven- that using visual representations in inter- tion that used this method for teaching ventions is an effective technique. fractions did not show much promise,112 but the other three did result in positive Three of the studies used manipulatives gains.113 One of them taught 1st graders in the early stages of instruction to rein- basic math concepts and operations,114 and force understanding of basic concepts and the other two taught prealgebra concepts operations.104 One used concrete models to low-achieving students.115 such as groups of boxes to teach rules for multiplication problems.105 The three stud- How to carry out this ies largely showed significant and positive recommendation effects and provide evidence that using manipulatives may be helpful in the initial 1. Use visual representations such as number stages of an intervention to improve profi- lines, arrays, and strip diagrams. ciency in word problem solving.106 In the panel’s view, visual representations In six of the studies, both concrete and vi- such as number lines, number paths, strip sual representations were used, and over- diagrams, drawings, and other forms of pic- all these studies show that using some torial representations help scaffold learn- combination of manipulatives and visual ing and pave the way for understanding the representations may promote mathemati- abstract version of the representation. We cal understanding.107 In two of the six, recommend that interventionists use such instruction did not include fading of the abstract visual representations extensively and consistently. We also recommend that interventionists explicitly link visual rep- 99. Following WWC guidelines, an effect size greater than 0.25 is considered substantively resentations with the standard symbolic important. representations used in mathematics. 100. Jitendra et al. (1998); Walker and Poteet (1989); Wilson and Sindelar (1991); Woodward (2006). 101. Woodward (2006). 108. Artus and Dyrek (1989); Fuchs, Powell et al. (2008). 102. Woodward (2006). 109. Artus and Dyrek (1989). 103. Jitendra et al. (1998); Walker and Poteet (1989); Wilson and Sindelar (1991). 110. Fuchs, Powell et al. (2008). 104. Darch et al. (1984); Fuchs, Seethaler et al. 111. Fuchs et al. (2005); Butler et al. (2003); Witzel (2008); Fuchs, Fuchs, Craddock et al. (2008). et al. (2003); Witzel (2005). 105. Darch et al. (1984). 112. Butler et al. (2003). 106. Darch at al. (1984); Fuchs, Sethaler et al. 113. Fuchs et al. (2005); Witzel, Mercer, and Miller (2008); Fuchs, Fuchs, Craddock et al. (2008). (2003); Witzel (2005). 107. Artus and Dyrek (1989); Butler et al. (2003); 114. Fuchs et al. (2005)�. Fuchs, Powell et al. (2008); Fuchs et al. (2005); Wit- 115. Witzel, Mercer, and Miller (2003); Witzel zel (2005); Witzel, Mercer, and Miller (2003). (2005).

( 31 ) Recommendation 5. Intervention materials should include opportunities for the student

In early grades, number lines, number and subtraction. Example 5 (p. 34) shows paths, and other pictorial representations how a student can draw a picture to solve are often used to teach students founda- a multidigit addition problem. In the fig- tional concepts and procedural operations ure, circles represent one unit and lines of addition and subtraction. Although represent units of 10. number lines or number paths may not be a suitable initial representation in some In upper grades, diagrams and pictorial situations (as when working with multipli- representations used to teach fractions cation and division), they can help concep- also help students make sense of the basic tually and procedurally with other types of structure underlying word problems. Strip problems. Conceptually, number lines and diagrams (also called model diagrams and number paths show magnitude and allow bar diagrams) are one type of diagram that for explicit instruction on magnitude com- can be used. Strip diagrams are drawings of parisons. Procedurally, they help teach narrow rectangles that show relationships principles of addition and subtraction op- among quantities. Students can use strip erations such as “counting down,” “count- diagrams to help them reason about and ing up,” and “counting down from.” solve a wide variety of word problems about related quantities. In example 6 (p. 34), the The figure in example 4 shows how a num- full rectangle (consisting of all three equal ber line may be used to assist with counting parts joined together) represents Shauntay’s strategies. The top arrows show how a child money before she bought the book. Since 2 learns to count on. He adds 2 + 5 = . To she spent ⁄3 of her money on the book, two start, he places his finger on 2. Then, he of the three equal parts represent the $26 jumps five times to the right and lands on she spent on the book. Students can then 7. The arrows under the number line show reason that if two parts stand for $26, then how a child subtracts using a counting each part stands for $13, so three parts down strategy. For 10 – 3 = , she starts stand for $39. So, Shauntay had $39 before with her finger on the 10. Then, she jumps she bought the book. three times to the left on the number line, where she finishes on 7. 2. If visuals are not sufficient for developing accurate abstract thought and answers, use The goal of using a number line should concrete manipulatives first. Although this be for students to create a mental num- can also be done with students in upper el- ber line and establish rules for movement ementary and middle school grades, use of along the line according to the more or less manipulatives with older students should be marking arrows placed along the line. Such expeditious because the goal is to move to- rules and procedures should be directly ward understanding of—and facility with— tied to the explicit instruction that guided visual representations, and finally, to the the students through the use of the visual abstract. representation.116 Manipulatives are usually used in lower Pictorial representations of objects such grades in the initial stages of learning as as birds and cups are also often used to teachers introduce basic concepts with teach basic addition and subtraction, and whole numbers. This exposure to concrete simple drawings can help students under- objects is often fleeting and transitory. The stand place value and multidigit addition use of manipulatives in upper elementary school grades is virtually nonexistent.117

116. Manalo, Bunnell, and Stillman (2000). Note that this study was not eligible for review because 117. Howard, Perry, and Lindsay (1996); Howard, it was conducted outside the United States. Perry, and Conroy (1995).

( 32 ) Recommendation 5. Intervention materials should include opportunities for the student

The panel suggests that the interventionist Second, in the upper grades, use concrete use concrete objects in two ways. objects when visual representations do not seem sufficient in helping students First, in lower elementary grades, use understand mathematics at the more ab- concrete objects more extensively in the stract level. initial stages of learning to reinforce the understanding of basic concepts and Use manipulatives expeditiously, and focus operations.118 on fading them away systematically to reach the abstract level.120 In other words, Concrete models are routinely used to explicitly teach students the concepts and teach basic foundational concepts such operations when students are at the con- as place value.119 They are also useful in crete level and consistently repeat the in- teaching other aspects of mathematics structional procedures at the visual and such as multiplication facts. When a mul- abstract levels. Using consistent language tiplication fact is memorized by question across representational systems (manip- and answer alone, a student may believe ulatives, visual representations, and ab- that numbers are to be memorized rather stract symbols) has been an important than understood. For example, 4 × 6 equals component in several research studies.121 24. When shown using manipulatives (as Example 8 (p. 35) shows a set of matched in example 7, p. 35), 4 × 6 means 4 groups concrete, visual, and abstract representa- of 6, which total as 24 objects. tions of a concept involving solving single- variable equations.

118. Darch at al. (1984); Fuchs, Seethaler et al. 120. ; Fuchs et al. (2005); Witzel (2005); Witzel, (2008) Fuchs, Fuchs, Craddock et al. (2008). Mercer, and Miller (2003). 119. Fuchs et al. (2005); Fuchs, Seethaler et al. 121. Fuchs et al. (2005); Butler et al. (2003); Witzel (2008); Fuchs, Powell et al. (2008). (2005); Witzel, Mercer, and Miller (2003).

Example 4. Representation of the counting on strategy using a number line

0 1 2 3 4 5 6 7 8 9 10

( 33 ) Recommendation 5. Intervention materials should include opportunities for the student

Example 5. Using visual representations for multidigit addition

A group of ten can be drawn with a long line to indicate that ten ones are joined to form one ten:

Simple drawings help make sense of two-digit addition with regrouping: 36

+27 63

Example 6. Strip diagrams can help students make sense of fractions

Shauntay’s money at first

2 Shauntay spent /3 of the money she had on a book that cost $26. How much money did Shauntay have before she bought the book? $26 book

2 parts $26 1 part $26 ÷ 2 = $13 3 parts 3 × $13 = $39

Shauntay’s had $39

( 34 ) RECOMMENDATION 5. INTERVENTION MATERIALS SHOULD INCLUDE OPPORTUNITIES FOR THE STUDENT

Example 7. Manipulatives can help students understand that four multiplied by six means four groups of six, which means 24 total objects

Example 8. A set of matched concrete, visual, and abstract representations to teach solving single-variable equations

3 + X = 7

Solving the Equation with Solving the Equation Solving the Equation Concrete Manipulatives with Visual with Abstract Symbols (Cups and Sticks) Representations of Cups and Sticks

A +X = +X = 3 + 1X = 7

B − − − − −3 −3

C X X = = 1X = 4 1 1 D

E X = X = X = 4

Concrete Steps A. 3 sticks plus one group of X equals 7 sticks B. Subtract 3 sticks from each side of the equation C. The equation now reads as one group of X equals 4 sticks D. Divide each side of the equation by one group E. One group of X is equal to four sticks (i.e., 1X/group = 4 sticks/group; 1X = 4 sticks)

( 35 ) Recommendation 5. Intervention materials should include opportunities for the student

Potential roadblocks and solutions use manipulatives in the initial stages stra- tegically and then scaffold instruction to Roadblock 5.1. In the opinion of the the abstract level. So, although it takes time panel, many intervention materials pro- to use manipulatives, this is not a major vide very few examples of the use of visual concern since concrete instruction will representations. happen only rarely and expeditiously.

Suggested Approach. Because many cur- Roadblock 5.3. Some interventionists ricular materials do not include sufficient may not fully understand the mathemati- examples of visual representations, the cal ideas that underlie some of the repre- interventionist may need the help of the sentations. This is likely to be particularly mathematics coach or other teachers in true for topics involving negative numbers, developing the visuals. District staff can proportional reasoning, and interpretations also arrange for the development of these of fractions. materials for use throughout the district. Suggested Approach. If interventionists Roadblock 5.2. Some teachers or interven- do not fully understand the mathematical tionists believe that instruction in concrete ideas behind the material, they are un- manipulatives requires too much time. likely to be able to teach it to struggling students.123 It is perfectly reasonable for Suggested Approach. Expeditious use of districts to work with a local university manipulatives cannot be overemphasized. faculty member, high school mathemat- Since tiered interventions often rely on ics instructor, or mathematics special- foundational concepts and procedures, ist to provide relevant mathematics in- the use of instruction at the concrete level struction to interventionists so that they allows for reinforcing and making explicit feel comfortable with the concepts. This the foundational concepts and operations. can be coupled with professional devel- Note that overemphasis on manipulatives opment that addresses ways to explain can be counterproductive, because stu- these concepts in terms their students will dents manipulating only concrete objects understand. may not be learning to do math at an abstract level.122 The interventionist should

122. Witzel, Mercer, and Miller (2003). 123. Hill, Rowan, and Ball (2005); Stigler and Hiebert (1999).

( 36 ) Recommendation 6. instruction in the intervention.127 These studies reveal a series of small but positive Interventions at all effects on measures of fact fluency128 and grade levels should procedural knowledge for diverse student devote about 10 minutes populations in the elementary grades.129 In some cases, fact fluency instruction was in each session to one of several components in the interven- building fluent retrieval tion, and it is difficult to judge the impact of the fact fluency component alone.130 of basic arithmetic facts. However, because numerous research teams independently produced similar findings, we consider this practice worthy Quick retrieval of basic arithmetic of serious consideration. Although the re- facts is critical for success in search is limited to the elementary school mathematics.124 Yet research has found grades, in the panel’s view, building fact that many students with difficulties fluency is also important for middle school in mathematics are not fluent in students when used appropriately. such facts.125 Weak ability to retrieve arithmetic facts is likely to impede Brief summary of evidence to understanding of concepts students support the recommendation encounter with rational numbers since teachers and texts often assume The evidence demonstrates small positive automatic retrieval of facts such as effects on fact fluency and operations for 3 × 9 = and 11 – 7 = as they the elementary grades and thus provides explain concepts such as equivalence support for including fact fluency activi- and the commutative property.126 For ties as either stand-alone interventions that reason, we recommend that about or components of larger tier 2 interven- 10 minutes be devoted to building this tions.131 These positive effects did not, proficiency during each intervention however, consistently reach statistical sig- session. Acknowledging that time may nificance, and the findings cannot be -ex be short, we recommend a minimum trapolated to areas of mathematics outside of 5 minutes a session. of fact fluency and operations.

Level of evidence: Moderate

The panel judged the level of evidence 127. Beirne-Smith (1991); Fuchs, Seethaler et al. supporting this recommendation to be (2008); Fuchs, Fuchs, Hamlett et al. (2006); Fuchs moderate. This recommendation is based et al. (2005); Fuchs, Powell et al. (2008); Tournaki (2003); Woodward (2006). on seven randomized controlled trials that 128. Fuchs, Seethaler et al. (2008); Fuchs, Fuchs, met WWC standards or met standards with Hamlet et al. (2006); Fuchs et al. (2005); Fuchs, reservations and that included fact fluency Powell et al. (2008); Tournaki (2003); Woodward (2006). 129. Beirne-Smith (1991); Fuchs, Seethaler et al. (2008); Fuchs et al. (2005); Tournaki (2003); Woodward (2006). 124. National Mathematics Advisory Panel 130. Fuchs, Seethaler et al. (2008); Fuchs et al. (2008). (2005). 125. Geary (2004); Jordan, Hanich, and Kaplan 131. Beirne-Smith (1991); Fuchs et al. (2005); (2003); Goldman, Pellegrino, and Mertz (1988). Fuchs, Fuchs, Hamlet et al. (2006); Fuchs, 126. Gersten and Chard (1999); Woodward (2006); Seethaler et al. (2008); Fuchs, Powell et al. (2008); Jitendra et al. (1996). Tournaki (2003); Woodward (2006).

( 37 ) Recommendation 6. Interventions at all grade levels should devote about 10 minutes

Two studies examined the effects of being per session.137 Since these components taught mathematics facts relative to the were typically not independent variables effects of being taught spelling or word in the studies, it is difficult to attribute any identification using similar methods.132 In positive effects to the component itself. both studies, the mathematics facts group There is evidence, however, that strategy- demonstrated positive gains in fact flu- based instruction for fact fluency (such ency relative to the comparison group, but as teaching the counting-on procedure) is the effects were significant in only one of superior to rote memorization.138 the studies.133 How to carry out this Another two interventions included a facts recommendation fluency component in combination with a larger tier 2 intervention.134 For example, 1. Provide about 10 minutes per session of in the Fuchs et al. (2005) study, the final 10 instruction to build quick retrieval of basic minutes of a 40 minute intervention ses- arithmetic facts. Consider using technology, sion were dedicated to practice with addi- flash cards, and other materials for extensive tion and subtraction facts. In both stud- practice to facilitate automatic retrieval. ies, tier 2 interventions were compared against typical tier 1 classroom instruc- The panel recommends providing about 10 tion. In each study, the effects on mathe- minutes each session for practice to help matics facts were small and not significant, students become automatic in retrieving though the effects were generally positive basic arithmetic facts, beginning in grade in favor of groups that received the inter- 2. The goal is quick retrieval of facts using vention. Significant positive effects were the digits 0 to 9 without any access to pen- detected in both studies in the domain of cil and paper or manipulatives. operations, and the fact fluency component may have been a factor in improving Presenting facts in number families (such students’ operational abilities. as 7 × 8 = 56, 8 × 7 = 56, 56/7 = 8, and 56/8 = 7) shows promise for improving Many of the studies in the evidence base student fluency.139 In the panel’s view, included one or more of a variety of com- one advantage of this approach is that ponents such as teaching the relationships students simultaneously learn about the among facts,135 making use of a variety nature of inverse operations. of materials such as flash cards and computer-assisted instruction,136 and teaching In the opinion of the panel, cumulative re- math facts for a minimum of 10 minutes view is critical if students are to maintain fluency and proficiency with mathematics facts. An efficient way to achieve this is 132. Fuchs, Fuchs, Hamlett et al. (2006); Fuchs, Powell et al. (2008). to integrate previously learned facts into practice activities. To reduce frustration 133. In Fuchs, Fuchs, Hamlett et al. (2006), the effects on addition fluency were statistically sig- and provide enough extended practice so nificant and positive while there was no effect on that retrieval becomes automatic (even for subtraction fluency. 134. Fuchs, Seethaler et al. (2008); Fuchs et al. 137. Beirne-Smith (1991); Fuchs et al. (2005); (2005). Fuchs, Fuchs, Hamlett et al. (2006); Fuchs, 135. Beirne-Smith (1991); Fuchs et al. (2005); Seethaler et al. (2008); Fuchs, Powell et al. (2008); Fuchs, Fuchs, Hamlett et al. (2006); Fuchs, Tournaki (2003); Woodward (2006). Seethaler et al. (2008); Woodward (2006). 138. Beirne-Smith (1991); Tournaki (2003); Wood- 136. Beirne-Smith (1991); Fuchs et al. (2005); ward (2006). Fuchs, Fuchs, Hamlett et al. (2006); Fuchs, 139. Fuchs et al. (2005); Fuchs, Fuchs, Hamlett et Seethaler et al. (2008); Fuchs, Powell et al. (2008). al. (2006); Fuchs, Seethaler et al. (2008).

( 38 ) Recommendation 6. Interventions at all grade levels should devote about 10 minutes

those who tend to have limited capacity to Note that learning the counting-up strategy remember and retrieve abstract material), not only improves students’ fact fluency145 interventionists can individualize practice but also immerses students in the commu- sets so students learn one or two new facts, tative property of addition. For example, practice several recently acquired facts, students learn that when the larger number and review previously learned facts.140 If is presented second (3 + 5 = ), they can students are proficient in grade-level math- rearrange the order and start counting up ematics facts, then the panel acknowledges from 5. In the view of the panel, this linkage that students might not need to practice is an important part of intervention. After each session, although periodic cumulative this type of instruction, follow-up practice review is encouraged. with flash cards might help students make the new learning automatic. 2. For students in kindergarten through grade 2, explicitly teach strategies for effi- 3. Teach students in grades 2 through 8 how cient counting to improve the retrieval of to use their knowledge of properties, such mathematics facts. as commutative, associative, and distributive law, to derive facts in their heads. It is important to provide students in kindergarten through grade 2 with strategies Some researchers have argued that rather for efficiently solving mathematics facts as than solely relying on rote memorization a step toward automatic, fluent retrieval. and drill and practice, students should The counting-up strategy has been used use properties of arithmetic to solve com- to increase students’ fluency in addition plex facts involving multiplication and di- facts.141 This is a simple, effective strategy vision.146 These researchers believe that that the majority of students teach them- by teaching the use of composition and selves, sometimes as early as age 4.142 But decomposition, and applying the distribu- students with difficulties in mathematics tive property to situations involving mul- tend not to develop this strategy on their tiplication, students can increasingly learn own, even by grade 2.143 There is evidence how to quickly (if not automatically) re- that systematic and explicit instruction in trieve facts. For example, to understand this strategy is effective.144 and quickly produce the seemingly difficult multiplication fact 13 × 7 = , students Students can be explicitly taught to find are reminded that 13 = 10 + 3, something the smaller number in the mathematics they should have been taught consistently fact, put up the corresponding number of during their elementary career. Then, since fingers, and count up that number of fin- 13 × 7 = (10 + 3) × 7 = 10 × 7 + 3 × 7, the gers from the larger number. For example, fact is parsed into easier, known problems to solve 3 + 5 = , the teacher identifies 10 × 7 = and 3 × 7 = by applying the smaller number (3) and puts up three of the distributive property. Students can fingers. The teacher simultaneously says then rely on the two simpler multiplication and points to the larger number before facts (which they had already acquired) to counting three fingers, 6, 7, 8. quickly produce an answer mentally.

140. Hasselbring, Bransford, and Goin (1988). The panel recommends serious consid- Note that there was not sufficient information to eration of this approach as an option for do a WWC review. students who struggle with acquisition of 141. Beirne-Smith (1991); Tournaki (2003). 142. Siegler and Jenkins (1989). 145. Tournaki (2003). 143. Tournaki (2003). 146. Robinson, Menchetti, and Torgesen (2002); 144. Tournaki (2003). Woodward (2006).

( 39 ) Recommendation 6. Interventions at all grade levels should devote about 10 minutes

facts in grades 2 through 8. When choos- practice be less tedious.147 Players may be ing an intervention curriculum, consider motivated when their scores rise and the one that teaches this approach to students challenge increases. Further recommenda- in this age range. Note, however, that the tions for motivating students are in recom- panel believes students should also spend mendation 8. time after instruction with extensive practice on quick retrieval of facts through Roadblock 6.2. Curricula may not include the use of materials such as flash cards enough fact practice or may not have ma- or technology. terials that lend themselves to teaching strategies. Roadblocks and solutions Suggested Approach. Some contempo- Roadblock 6.1. Students may find fluency rary curricula deemphasize fact practice, practice tedious and boring. so this is a real concern. In this case, we recommend using a supplemental program, Suggested Approach. Games that pro- either flash card or technology based. vide students with the opportunity to practice new facts and review previously learned facts by encouraging them to beat their previous high score can help the 147. Fuchs, Seethaler et al. (2008).

( 40 ) Recommendation 7. sideration of the standards for measurement established by a joint committee of Monitor the progress national organizations.149 of students receiving supplemental Brief summary of evidence to support the recommendation instruction and other students Although we found no studies that addressed the use of valid measures for strug- who are at risk. gling students within an RtI framework, nonexperimental studies demonstrate the technical adequacy of various progress Assess the progress of tier 2 and tier monitoring measures.150 Measures for the 3 students regularly with general primary grades typically reflect aspects of outcome measures and curriculum number sense, including strategic counting, embedded measures. Also monitor numeral identification, and magnitude com- regularly the progress of tier 1 students parisons.151 Studies investigating measures who perform just above the cutoff for the elementary grades focus mostly on score for general outcome measures the characteristics of general outcome mea- so they can be moved to tier 2 if they sures that represent grade-level mathemat- begin to fall behind. ics curricula in computation and in mathematics concepts and applications.152 Widely In addition, use progress monitoring used, these measures are recommended by data to determine when instructional the National Center for Student Progress changes are needed. This includes Monitoring.153 Less evidence is available regrouping students who need to support progress monitoring in middle continuing instructional support within school.154 But research teams have devel- tier 2 or tier 3, or moving students oped measures focusing on math concepts who have met benchmarks out of typically taught in middle school,155 basic intervention groups and back to tier 1. facts,156 and estimation.157

Information about specific progress monitoring measures is available in Appendix D. A list of online resources 149. The American Psychological Association, is in the text below. the American Educational Research Associa- tion, and the National Council on Measurement Level of evidence: Low in Education (1999). 150. For example, Clarke et al. (2008); Foegen The panel judged the level of evidence and Deno (2001); Fuchs et al. (1993); Fuchs, Fuchs, Hamlett, Thompson et al. (1994); Leh et al. (2007); supporting this recommendation to be Lembke et al. (2008). low. No studies that met WWC standards 151. For example, Clarke et al. (2008); Lembke et 148 supported this recommendation. In- al. (2008); Bryant, Bryant, Gersten, Scammacca, stead, the recommendation is based on and Chavez (2008). the panel’s expert opinion as well as con- 152. Fuchs and Fuchs (1998); Fuchs et al. (1999). 148. The technical adequacy studies of mathe- 153. www.studentprogress.org. matics progress monitoring measures were not 154. Foegen (2008). experimental; the researchers typically used cor- 155. Helwig, Anderson, and Tindal (2002). relation techniques to evaluate the reliability and criterion validity of the measures and regression 156. Espin et al. (1989). methods to examine sensitivity to growth. 157. Foegen and Deno (2001); Foegen (2000).

( 41 ) Recommendation 7. Monitor the progress of students receiving supplemental instruction

How to carry out this Choose progress monitoring measures recommendation with evidence supporting their reliability, validity, and ability to identify growth. 1. Monitor the progress of tier 2, tier 3, and This will require input from individu- borderline tier 1 students at least once a als with expertise in these areas, typi- month using grade-appropriate general out- cally school psychologists or members of come measures. district research departments. Consider whether the measure produces consistent General outcome measures typically take results (reliability) and provides informa- 5 to 10 minutes to administer and should tion that correlates with other measures be used at least monthly to monitor tier of mathematics achievement (criterion 2 and tier 3 students. General outcome validity). Ability to identify growth helps measures use a sample of items from the interventionists ensure that students are array of concepts covered over one year to learning and making progress toward an assess student progress. They provide a annual goal through the full array of ser- broad perspective on student proficiency vices they are receiving. in mathematics. They target concepts such as magnitude comparison, counting ability, In some cases, general outcome measures and knowledge of place value for students may also be used for screening, as de- in kindergarten and grade 1, and increas- scribed in recommendation 1. Resources ingly complex aspects of place value and that teachers can turn to for identifying proficiency with operations for students appropriate measures include the Na- in grades 2 through 6. Examining student tional Center on Student Progress Moni- performance on these measures allows toring’s review of available tools (http:// teachers to determine whether students are www.studentprogress.org/) and the Re- integrating and generalizing the concepts, search Institute on Progress Monitoring skills, and strategies they are learning in the (http://www.progressmonitoring.org/). core curriculum and the intervention.158 2. Use curriculum-embedded assessments In addition to monitoring the progress of in interventions to determine whether stu- tier 1 and tier 2 students, the panel recom- dents are learning from the intervention. mends monitoring the progress of border- These measures can be used as often as line tier 1 students with general outcome every day159 or as infrequently as once every measures on a monthly basis. Since these other week.160 students scored just above the cut score, they were not selected for supplemental Many tier 2 and tier 3 intervention pro- instruction. The panel suggests using one grams (commercially developed, re- standard error of measurement (a statistic searcher developed, or district developed) available in the technical information for include curriculum-embedded assess- the measures) above the cut score to de- ments (sometimes called unit tests, mas- fine the range of scores for borderline stu- tery tests, or daily probes). The results of dents. Using this approach, teachers can these assessments can be used to deter- continue to monitor the progress of stu- mine which concepts need to be reviewed, dents whose scores fell just above the cut which need to be re-taught, and which have score and determine whether they should been mastered. Curriculum-embedded as- receive supplemental instruction. sessments are often administered daily

159. Bryant, Bryant, Gersten, Scammacca, and Chavez (2008). 158. Fuchs, Fuchs, and Zumeta (2008). 160. Jitendra (2007).

( 42 ) Recommendation 7. Monitor the progress of students receiving supplemental instruction

for students in kindergarten161 and grade recommends using progress monitoring 1 and biweekly for students in grades 2 data to regroup students within tiers so that through 6.162 These assessments usually the small groups used in tier 2 interventions do not possess the same high technical are as homogeneous as possible. If a stu- characteristics of the general outcome dent does not fit into any of the intervention measures. Curriculum-embedded assess- groups from his or her class, consider put- ments often result in very useful informa- ting the child in an intervention group from tion for interventionists because they can another class if the schedule permits. detect changes in student performance in the concepts and skills being taught at the Roadblocks and solutions time. Interventionists need to be cautious about assuming that mastery of individ- Roadblock 7.1. Students within classes are ual skills and concepts will translate into at very different levels. This can make it dif- improvements in overall proficiency. As a ficult to group students into appropriate tier result, the panel recommends using both 2 and tier 3 intervention groups. general outcome measures and curriculum-embedded assessments for students Suggested Approach. If students within receiving interventions. a class are at such diverse levels that appropriate tier 2 and tier 3 intervention If the intervention program does not in- groups cannot be made, consider grouping clude curriculum-embedded assessments, students across classes. This will facilitate use efficient, reliable, and valid screen- clustering students with similar needs. For ing measures, which can also be used as example, teachers of upper elementary progress monitoring measures (see recom- students may find that students who have mendation 1). not yet mastered basic concepts in a particular area (fractions) are spread across 3. Use progress monitoring data to regroup several classrooms. Putting these students students when necessary. in a single tier 2 intervention group would be the most efficient means of meeting Since student skill levels change over their needs, rather than trying to provide time and in varying degrees, the panel one or two students in each class with services duplicated across classrooms. In 161. For example, one tier 2 intervention pro- such a case, a math specialist, parapro- gram for 1st and 2nd grade students reported by fessional, or other school personnel who Bryant, Bryant, Gersten, Scammacca, and Chavez have received training can conduct the (2008) included daily activity-level progress intervention. monitoring that consisted of four oral or written problems drawn from the content focus for that day. Teachers were instructed that a majority of Roadblock 7.2. There is insufficient the students in the group had to complete at least time for teachers to implement progress three of the four problems correctly to consider monitoring. the daily lesson successful. 162. A parallel example in grades 3 and beyond can be found in Jitendra’s Solving math word Suggested Approach. If teachers are too problems instructional materials on teaching busy to assess student progress with moni- word problems (2007). There are many other ex- toring measures, consider training parapro- amples in available commercial programs. fessionals or other school staff to do so.

( 43 ) Recommendation 8. Level of evidence: Low

Include motivational The panel judged the level of evidence strategies in tier 2 and supporting this recommendation to be tier 3 interventions. low. This recommendation is based on the professional opinion of the panel, and on nine studies that met WWC standards Adults can sometimes forget how or met standards with reservations that challenging so-called “basic” arithmetic included motivational strategies in the is for students in tier 2 and tier 3 intervention.166 Although one of these interventions. Many of these students studies demonstrated that praising strug- have had experiences of failure and gling students for their effort significantly frustration with mathematics by the improved their ability to solve subtraction time they receive an intervention. They problems with regrouping,167 other stud- may also have a particularly difficult ies included a motivational component as time storing and easily retrieving one of several components of the inter- information in their memories.163 vention. In the opinion of the panel, these Therefore, it seems particularly studies did not show that a motivational important to provide additional component is essential but suggest that it motivation for these students.164 may be useful for improving mathematics achievement.168 Praising students for their effort and for being engaged as they work Brief summary of evidence to through mathematics problems is a support the recommendation powerful motivational tool that can be effective in increasing students’ One study that met WWC standards exam- academic achievement.165 Tier 2 and ined the effects of a motivational component tier 3 interventions should include by comparing the performance of students components that promote student who received praise for their effort dur- effort (engagement-contingent ing subtraction instruction with those who rewards), persistence (completion- did not receive praise.169 This study found contingent rewards), and achievement significant positive effects on student sub- (performance-contingent rewards). traction scores in favor of providing effort These components can include praise feedback.170 Although this study provides and rewards. Even a well-designed some evidence of the effectiveness of a mo- intervention curriculum may falter tivational strategy, it is the only study that without such behavioral supports.

166. Fuchs et al. (2005); Fuchs, Fuchs, Cradock et al. (2008); Schunk and Cox (1986); Fuchs, Seethaler et al. (2008); Heller and Fantuzzo (1993); Artus and Dyrek (1989); Fuchs, Fuchs et al. (2003b); Fuchs, Fuchs, Hamlett, Phillips et al. (1994); Fuchs, Fuchs, Finelli et al. (2006). 163. Geary (2003). 167. Schunk and Cox (1986). 164. The scope of this practice guide limited the motivational strategies reviewed to strategies 168. There is an extensive literature on motiva- used in studies of students struggling with math- tional strategies outside the scope of this prac- ematics. For a wider review of effective motiva- tice guide. For more information on motivational tional strategies used in classrooms, see Epstein strategies see Epstein et al. (2008) and Halpern et al. (2008) and Halpern et al. (2007). et al. (2007). 165. Schunk and Cox (1986); Fuchs et al. (2005). 169. Schunk and Cox (1986). Fuchs, Fuchs, Craddock et al. (2008). 170. Schunk and Cox (1986).

( 44 ) Recommendation 8. Include motivational strategies

explicitly tested the effects of motivational of students who graphed and set goals.176 strategies on mathematics outcomes. The other four studies did not isolate the effects of graphing progress.177 Because In two studies, students received points this recommendation is based primarily for engagement and attentiveness,171 and on the opinion of the panel, the level of in three studies, students were provided evidence is identified as low. with prizes as tangible reinforcers for accurate mathematics problem-solving.172 How to carry out this However, in each of these studies, it was recommendation not possible to isolate the effects of reinforcing attentiveness and accuracy. For 1. Reinforce or praise students for their effort example, in two of the studies, students and for attending to and being engaged in in tier 2 tutoring earned prizes for accura- the lesson. cy.173 Although in both studies, the tier 2 intervention group demonstrated substan- Verbally praise students for their effort178 tively important positive and sometimes and for listening carefully and following significant gains on a variety of mathemat- the lesson in a systematic fashion (engage- ics measures relative to the students who ment-contingent rewards).179 The panel remained in tier 1, it is not possible to iso- believes that praise should be immediate late the effects of the reinforcers from the and specific to highlight student effort provision of tier 2 tutoring. Another study and engagement. But we also believe that examined the impact of parental involve- it is ineffective to offer generic and empty ment on students’ mathematics achieve- praise (“good job!” or “keep up the good ment and found statistically significant work!”) that is not related to actual effort. positive effects on operations and general Instead, praise is most effective when it math achievement.174 However, because points to specific progress that students the parental involvement component was are making and recognizes students’ ac- multifaceted, it is not possible to attribute tual effort.180 Systematically praising stu- the positive effects to rewards alone. dents for their effort and engagement may encourage them to remain focused on the Five studies in the evidence base included completion of their work. interventions in which students graphed their progress and in some cases set goals 2. Consider rewarding student accom for improvement on future assessments.175 plishments. One experimental study examined the effects of student graphing and goal set- Consider using rewards to acknowledge ting as an independent variable and found completion of math tasks (completion- substantively important positive effects contingent rewards) and accurate work on measures of word problems in favor (performance-contingent rewards). This can be done by applauding or verbally 171. Fuchs et al. (2005); Fuchs, Fuchs, Craddock et al. (2008). 172. Fuchs et al. (2005); Fuchs, Seethaler et al. 176. Fuchs et al. (2003b). (2008); Fuchs, Fuchs, Craddock et al. (2008). 177. Artus and Dyrek (1989); Fuchs, Seethaler et 173. Fuchs et al. (2005); Fuchs, Seethaler et al. al. (2008); Fuchs, Fuchs, Hamlett, Phillips et al. (2008). (1994); Fuchs, Fuchs, Finelli et al. (2006). 174. Heller and Fantuzzo (1993). 178. Schunk and Cox (1986). 175. Artus and Dyrek (1989); Fuchs, Seethaler et 179. For example, Fuchs et al. (2005); Fuchs, al. (2008); Fuchs et al. (2003b); Fuchs, Fuchs, Ham- Fuchs, Craddock et al. (2008). lett, Phillips et al. (1994); Fuchs, Fuchs, Finelli 180. See Bangert-Drowns et al. (1991) and Halpern et al. (2006). et al. (2007) for a review.

( 45 ) Recommendation 8. Include motivational strategies praising students for actual accomplish- Roadblocks and suggested ments, such as finishing assignments, im- approaches proving their score from 70 percent to 80 percent correct, or giving students points Roadblock 8.1. Rewards can reduce genu- or tokens each time they answer a problem ine interest in mathematics by directing stu- correctly, which they can use to “buy” tangi- dent attention to gathering rewards rather ble rewards at a later time.181 Again, praise than learning math. should be specific rather than generic.182 Consider notifying the student’s parents Suggested Approach. It is important to to inform them of their child’s successes in inform interventionists that research in mathematics by phone or email or in a note other content areas has demonstrated that sent home with the student.183 Remember rewards and praise increase the likelihood that parents of these students are likely to of students’ academic success without receive notification of problems rather than diminishing their interest in learning.189 successes, and some evidence suggests that Given the frequent history of failure for this specific positive attention might sup- many of these students, at least in the ele- port achievement growth.184 mentary grades, we suggest using rewards and praise to encourage effort, engage- 3. Allow students to chart their progress and ment, and achievement. As students learn to set goals for improvement. and succeed more often in mathematics, interventionists can gradually fade the Several of the interventions in the evi- use of rewards because student success dence base for this practice guide had stu- will become an intrinsic reward. The WWC dents graph their progress on charts185 and Reducing Behavior Problems in the Elemen- set goals for improving their assessment tary School Classroom Practice Guide190 is scores.186 For example, students might a good reference for more information on graph their scores on a chart showing a se- the use of rewards and praise. ries of thermometers, one for each session of the intervention.187 At the beginning of Roadblock 8.2. It is difficult to deter- each session, students can examine their mine appropriate rewards for individual charts and set a goal to beat their previ- students. ous score or to receive the maximum score. This type of goal setting is believed to help Suggested Approach. Consider each stu- students develop self-regulated learning dent’s interests before choosing an appro- because students take independent respon- priate reward. Also consider using oppor- sibility for setting and achieving goals.188 tunities to engage in activities students are interested in as rewards to reinforce effort, engagement, and accurate work. Parents may also have ideas for rewards that will 181. For example, Fuchs et al. (2005); Fuchs, Seethaler et al. (2008); Fuchs, Fuchs, Craddock help motivate their children. Schools can et al. (2008). engage parents in rewarding students and 182. Halpern et al. (2007). coordinate efforts to reward children at 191 183. For example, Heller and Fantuzzo (1993). home as well. 184. For example, Heller and Fantuzzo (1993). 185. Artus and Dyrek (1989); Fuchs, Fuchs, Ham- lett, Phillips et al. (1994); Fuchs, Fuchs, Finelli et al. (2006); Fuchs, Seethaler et al. (2008). 186. Fuchs et al. (2003b). 189. Epstein et al. (2008). 187. See the procedure in Fuchs et al. (2003b). 190. Epstein et al. (2008). 188. Fuchs et al. (1997). 191. For example, Heller and Fantuzzo (1993).

( 46 ) Recommendation 8. Include motivational strategies

Roadblock 8.3. Providing feedback and can be done when the teacher grades the rewarding achievement detracts from class- student’s work. To reduce the amount of room instructional time. It is difficult to fit time it takes for students to “buy” prizes it into the classroom schedule. with accumulated points or tokens, ask students to choose which prize they want Suggested Approach. Verbally prais- before they “buy” it. The prizes can then be ing students for their effort individually distributed quickly at the end of the day, and their engagement in small group les- so that students are not distracted by the sons requires very little time. Awarding items throughout the school day. points or tokens for correct responses

( 47 ) Glossary of terms as “minimum addend strategy” or the strategy used in this report of “counting on from larger”). The commutative property of multipli- The associative property of addition cation states that A × B = B × A for all num- states that (A + B) + C = A + (B + C) for all bers A, B. This property allows for flex- numbers A, B, C. This property allows for ibility in calculating products and helps flexibility in calculating sums. For exam- lighten the load of learning the basic mul- ple, to calculate 85 + 97 + 3, we do not have tiplication facts. As shown in example 9, to add 85 and 97 first but may instead cal- once a child has learned the multiplication culate the easier sum 85 + (97 + 3), which fact 3 × 7 = 21, the child will also know the is 85 + 100, which equals 185. The associa- fact 7 × 3 = 21 if the child understands the tive property is also used in deriving basic commutative property. addition facts from other basic facts and therefore helps with learning these basic Example 9: Commutative property facts. For example, to add 8 + 5, a child can of multiplication think of breaking the 5 into 2 + 3, combining the 2 with the 8 to make a 10, and then adding on the 3 to make 13. In an equation, this can be recorded as: 8 + 5 = 8 + (2 + 3) = (8 + 2) + 3 = 10 + 3 = 13.

The associative property of multiplica- 3 groups of 7 3 × 7 dots tion states that (A × B) × C = A × (B × C) for all numbers A, B, C. This property allows for flexibility in calculating products. For example, to calculate 87 × 25 × 4, we do not have to multiply 87 and 25 first but may instead calculate the easier product 87 × (25 × 4), which is 87 × 100, which equals 8,700. The 7 groups of 3 7 × 3 dots associative property is also used in deriv- Both use the same array, so the total ing basic multiplication facts from other is the same: 3 × 7 = 7 × 3 basic facts and therefore helps with learning these facts. For example, to calculate 7 × 6, a child who already knows 7 × 3 = 21 Concurrent validity refers to the correla- can double the 21 to calculate that 7 × 6 = 42. tion between the assessment that is being In an equation, this can be recorded as: investigated and a similar assessment 7 × 6 = 7 × (3 × 2) = (7 × 3) × 2 = 21 × 2 = 42. when the assessments are completed at the same point in time. Correlation coef- The commutative property of addition ficients range from -1 to 1. A correlation states that A + B = B + A for all numbers coefficient close to 1 indicates a strong A, B. This property allows for flexibility in overlap between the assessments. calculating sums and helps lighten the load of learning the basic addition facts. For ex- Counting up/on is a strategy that young ample, to add 2 + 9, a child might want to children can use to solve addition (and count on 9 from 2 by counting “3, 4, 5, 6, . . . subtraction) problems. To calculate 8 + 3 11,” which is cumbersome. By applying the by counting on, the child starts with 8 and commutative property, a child can instead then “counts on” 3 more, saying “9, 10, 11.” add 9 + 2 by counting on 2 from 9, by say- The child may use fingers in order to de- ing “10, 11,” which is much easier (this is the termine when to stop counting.

( 48 ) Glossary of terms

Counting up/on from a larger addend strategy, children must know the “10 part- is a strategy in which children apply count- ner” (number that can be added to make ing on to solve an addition problem, but 10) for each number from 1 to 9 and must first apply the commutative property, if also know how to break each number into necessary, in order to count on from the a sum of two (positive whole) numbers in larger number. For example, to calculate all possible ways. Furthermore, the child 3 + 9, a child first changes the problem to must understand all the “teen” numbers 9 + 3 and then counts on 3 from 9. (from 11 to 19) as a 10 and some ones (for example, 15 is 10 and 5 ones). Counting up/on to solve unknown addend and subtraction problems. Count- Example 10: Make-a-10 strategy ing on can be used to solve unknown addend problems such as 11 + ? = 15. To solve this problem, the child can count on from 8 + 5 11 to 15, counting 12, 13, 14, 15 and raising one finger for each number. Since 4 fingers 2 3 were raised, 11 + 4 = 15. To solve a subtraction problem such as 15 minus 11 = ? using counting on, the child must first understand that the problem can be reformu- 8 + 5 lated as 11 + ? = 15. Then the child can use counting on as described previously. Note too that solving this subtraction problem using the counting on strategy is much = 10 + 3 = 13 easier than counting down 11. Note too that the reformulation of a subtraction problem as an unknown addend problem Derived fact strategies in multiplica- is important in its own right because it tion and division are strategies in which connects subtraction with addition. children use multiplication facts they already know to find related facts. For exam- In mathematics assessment, criterion- ple 5 × 8 is half of 10 × 8, and similarly for related validity means that student scores all the “5 times” facts (these are examples on an assessment should correspond to of applying the associative property). Also, their scores or performance on other in- 9 × 8 is 8 less than 10 × 8 , and similarly for dicators of mathematics competence, such all the “9 times” facts (these are examples as teacher ratings, course grades, or stan- of applying the distributive property). To dardized test scores. calculate 4 × 7, we can double the double of 7, that is, the double of 7 is 14 and the Derived fact strategies in addition and double of 14 is 28, which is 4 times 7. All subtraction are strategies in which chil- the “4 times” facts can be derived by dou- dren use addition facts they already know bling the double (these are examples of to find related facts. Especially important applying the associative property). among derived fact strategies are the make- a-10 methods because they emphasize base The distributive property relates addi- 10 structure. As shown in example 10, to tion and multiplication. It states that A × add 8 + 5, a child can think of breaking (B + C) = (A × B) + (A × C) for all numbers the 5 into 2 + 3, combining the 2 with the A, B, C. This property allows for flexibil- 8 to make a 10, and then adding on the 3 ity in calculating products. For example, to make 13 (see the associative property of to calculate 7 × 13, we can break 13 apart addition). Note that to use this make-a-10 by place value as 10 + 3 and calculate

( 49 ) Glossary of terms

7 × 10 = 70 and 7 × 3 = 21 and add these needing help in mathematics) divided by two results to find 7 × 13 = 91. In an equa- the sum of this value and the number of tion, this can be recorded as: 7 × 13 = 7 × false negatives, while specificity is equal (10 + 3) = (7 × 10) + (7 × 3) = 70 + 21 = 91. to the number of true negatives divided This strategy of breaking numbers apart by the sum of this value and the number by place value and applying the distribu- of false positives (students misidentified tive property is the basis for the common during screening). method of longhand multiplication. The distributive property is also used in deriv- A general outcome measure refers to a ing basic multiplication facts from other measure of specific proficiencies within basic facts and therefore helps in learning a broader academic domain. These profi- these facts. As shown in example 11, to ciencies are related to broader outcomes. calculate 6 × 7, a child who already knows For example, a measure of oral reading flu- 6 × 5 = 30 and 6 × 2 = 12 can add the 30 ency serves as a general outcome measure and 12 to calculate that 6 × 7 = 30 + 12 = 42. of performance in the area of reading. The In an equation, this can be recorded as: measures can be used to monitor student 6 × 7 = 6 × (5 + 2) = (6 × 5) + (6 × 2) = 30 + 12 = 42. progress over time.

Example 11: Distributive property Interventionist refers to the person teaching the intervention. The interventionist might be a classroom teacher, instructional assistant, or other certified We can break school personnel. 6 rows of 7 into 6 rows of 5 and The magnitude of a quantity or number is 6 rows of 2. its size, so a magnitude comparison is a comparison of size. The term magnitude is generally used when considering size in an approximate sense. In this case, we often describe the size of a quantity or number 6 x 7 = 6 x 5 + 6 x 2 very roughly by its order of magnitude, = 30 + 12 = 42 which is the power of ten (namely 1, 10, 100, 1000, . . . . or 0.1, 0.01, 0.001, . . . .) that the quantity or number is closest to. Efficiency is how quickly the universal screening measure can be administered, Number composition and number de- scored, and analyzed for all the students composition are not formal mathematical tested. terms but are used to describe putting numbers together, as in putting 2 and 3 together False positives and false negatives are to make 5, and breaking numbers apart, as in technical terms used to describe the mis- breaking 5 into 2 and 3. For young children, identification of students. The numbers a visual representation like the one shown of false positives and false negatives are on the next page is often used before intro- related to sensitivity and specificity. As ducing the traditional mathematical notation depicted in table 3 (p. 16) of this guide, 2 + 3 = 5 and 5 = 2 + 3 for number composi- sensitivity is equal to the number of true tion and decomposition. positives (students properly identified as

( 50 ) Glossary of terms

Example 12: Number decomposition do in mathematics a year or even two or three years later.

Response to Intervention (RtI) is an early detection, prevention, and support 5 system in education that identifies struggling students and assists them before they fall behind.

2 3 Sensitivity indicates how accurately a screening measure predicts which students are at risk. Sensitivity is calculated by determining the number of students who end up having difficulty in mathemat- A number line is a line on which locations ics and then examining the percentage of for 0 and 1 have been chosen (1 is to the those students predicted to be at risk on right of 0, traditionally). Using the distance the screening measure. A screening mea- between 0 and 1 as a unit, a positive real sure with high sensitivity would have a number, N, is located N units to the right of high degree of accuracy. In general, sen- 0 and a negative real number, -N, is located sitivity and specificity are related (as one N units to the left of 0. In this way, every increases the other usually decreases). real number has a location on the number line and every location on the number line Specificity indicates how accurately a corresponds to a real number. screening measure predicts which students are not at risk. Specificity is cal- A number path is an informal precursor culated by determining the number of to a number line. It is a path of consecu- students who do not have a deficit in math- tively numbered “steps,” such as the paths ematics and then examining the percent- found on many children’s board games age of those students predicted to not be at along which game pieces are moved. De- risk on the screening measure. A screening termining locations on number paths only measure with high specificity would have requires counting, whereas determining a high degree of accuracy. In general, sen- locations on number lines requires the no- sitivity and specificity are related (as one tion of distance. increases the other usually decreases).

Reliability refers to the degree to which Strip diagrams (also called model dia- an assessment yields consistency over grams and bar diagrams) are drawings of time (how likely are scores to be similar if narrow rectangles that show relationships students take the test a week or so later?) among quantities. and across testers (do scores change when different individuals administer the test?). A validity coefficient serves as an index Alternate form reliability tells us the ex- of the relation between two measures and tent to which an educator can expect sim- can range from -1.0 to 1.0, with a coeffi- ilar results across comparable forms or cient of .0 meaning there is no relation versions of an assessment. between the two scores and increasing positive scores indicating a stronger posi- Predictive validity is the extent to which tive relation. a test can predict how well students will

( 51 ) Appendix A. particular types of studies for drawing causal conclusions about what works. Postscript from Thus, one typically finds that a strong the Institute of level of evidence is drawn from a body of Education Sciences randomized controlled trials, the moderate level from well-designed studies that do not involve randomization, and the What is a practice guide? low level from the opinions of respected authorities (see table 1, p. 2). Levels of evi- The health care professions have em- dence also can be constructed around the braced a mechanism for assembling and value of particular types of studies for communicating evidence-based advice to other goals, such as the reliability and va- practitioners about care for specific clini- lidity of assessments. cal conditions. Variously called practice guidelines, treatment protocols, critical Practice guides also can be distinguished pathways, best practice guides, or simply from systematic reviews or meta-analyses practice guides, these documents are sys- such as What Works Clearinghouse (WWC) tematically developed recommendations intervention reviews or statistical meta- about the course of care for frequently en- analyses, which employ statistical meth- countered problems, ranging from physi- ods to summarize the results of studies cal conditions, such as foot ulcers, to psy- obtained from a rule-based search of the chosocial conditions, such as adolescent literature. Authors of practice guides sel- development.192 dom conduct the types of systematic literature searches that are the backbone of Practice guides are similar to the prod- a meta-analysis, although they take ad- ucts of typical expert consensus panels vantage of such work when it is already in reflecting the views of those serving published. Instead, authors use their ex- on the panel and the social decisions that pertise to identify the most important come into play as the positions of individ- research with respect to their recommen- ual panel members are forged into state- dations, augmented by a search of recent ments that all panel members are willing publications to ensure that the research to endorse. Practice guides, however, are citations are up-to-date. Furthermore, the generated under three constraints that do characterization of the quality and direc- not typically apply to consensus panels. tion of the evidence underlying a recom- The first is that a practice guide consists mendation in a practice guide relies less of a list of discrete recommendations that on a tight set of rules and statistical algo- are actionable. The second is that those rithms and more on the judgment of the recommendations taken together are in- authors than would be the case in a high- tended to be a coherent approach to a quality meta-analysis. Another distinction multifaceted problem. The third, which is is that a practice guide, because it aims for most important, is that each recommen- a comprehensive and coherent approach, dation is explicitly connected to the level operates with more numerous and more of evidence supporting it, with the level contextualized statements of what works represented by a grade (strong, moder- than does a typical meta-analysis. ate, or low). Thus, practice guides sit somewhere be- The levels of evidence, or grades, are tween consensus reports and meta-analyses usually constructed around the value of in the degree to which systematic processes are used for locating relevant research and 192. Field and Lohr (1990). characterizing its meaning. Practice guides

( 52 ) Appendix A. Postscript from the Institute of Education Sciences

are more like consensus panel reports than expertise to be a convincing source of rec- meta-analyses in the breadth and complex- ommendations. IES recommends that at ity of the topic that is addressed. Practice least one of the panelists be a practitioner guides are different from both consensus with experience relevant to the topic being reports and meta-analyses in providing addressed. The chair and the panelists are advice at the level of specific action steps provided a general template for a practice along a pathway that represents a more-or- guide along the lines of the information less coherent and comprehensive approach provided in this appendix. They are also to a multifaceted problem. provided with examples of practice guides. The practice guide panel works under a Practice guides in education at the short deadline of six to nine months to pro- Institute of Education Sciences duce a draft document. The expert panel members interact with and receive feed- IES publishes practice guides in educa- back from staff at IES during the develop- tion to bring the best available evidence ment of the practice guide, but they under- and expertise to bear on the types of sys- stand that they are the authors and, thus, temic challenges that cannot currently be responsible for the final product. addressed by single interventions or programs. Although IES has taken advantage One unique feature of IES-sponsored prac- of the history of practice guides in health tice guides is that they are subjected to care to provide models of how to proceed rigorous external peer review through the in education, education is different from same office that is responsible for inde- health care in ways that may require that pendent review of other IES publications. practice guides in education have some- A critical task of the peer reviewers of a what different designs. Even within health practice guide is to determine whether care, where practice guides now number the evidence cited in support of particular in the thousands, there is no single tem- recommendations is up-to-date and that plate in use. Rather, one finds descriptions studies of similar or better quality that of general design features that permit point in a different direction have not been substantial variation in the realization ignored. Peer reviewers also are asked to of practice guides across subspecialties evaluate whether the evidence grade as- and panels of experts.193 Accordingly, the signed to particular recommendations by templates for IES practice guides may vary the practice guide authors is appropriate. across practice guides and change over A practice guide is revised as necessary to time and with experience. meet the concerns of external peer reviews and gain the approval of the standards and The steps involved in producing an IES- review staff at IES. The process of external sponsored practice guide are first to select peer review is carried out independent of a topic, which is informed by formal sur- the office and staff within IES that insti- veys of practitioners and requests. Next, a gated the practice guide. panel chair is recruited who has a national reputation and up-to-date expertise in the Because practice guides depend on the topic. Third, the chair, working in collabo- expertise of their authors and their group ration with IES, selects a small number of decisionmaking, the content of a practice panelists to co-author the practice guide. guide is not and should not be viewed as a These are people the chair believes can set of recommendations that in every case work well together and have the requisite depends on and flows inevitably from scientific research. It is not only possible but 193. American Psychological Association also likely that two teams of recognized (2002). experts working independently to produce

( 53 ) Appendix A. Postscript from the Institute of Education Sciences

a practice guide on the same topic would individual school district might obtain on generate products that differ in important its own because the authors are national respects. Thus, consumers of practice authorities who have to reach agreement guides need to understand that they are, among themselves, justify their recom- in effect, getting the advice of consultants. mendations in terms of supporting evi- These consultants should, on average, pro- dence, and undergo rigorous independent vide substantially better advice than an peer review of their product.

Institute of Education Sciences

( 54 ) Appendix B. elementary school teachers at the Uni- versity of Georgia and wrote a book for About the authors such courses, Mathematics for Elementary Teachers, published by Addison-Wesley, Panel now in a second edition. She is especially interested in helping college faculty learn Russell Gersten, Ph.D., is the director of to teach mathematics content courses for the Instructional Research Group in Los elementary and middle grade teachers, Alamitos, California, as well as professor and she works with graduate students emeritus in the College for Education at and postdoctoral fellows toward that end. the University of Oregon. Dr. Gersten re- As part of this effort, Dr. Beckmann di- cently served on the National Mathematics rects the Mathematicians Educating Future Advisory Panel, where he co-chaired the Teachers component of the University of Work Group on Instructional Practices. He Georgia Mathematics Department’s VIGRE has conducted meta-analyses and research II grant. Dr. Beckmann was a member of syntheses on instructional approaches for the writing team of the National Council teaching students with difficulties in math- of Teachers of Mathematics Curriculum ematics, early screening in mathematics, Focal Points for Prekindergarten through RtI in mathematics, and research on num- Grade 8 Mathematics, is a member of the ber sense. Dr. Gersten has conducted nu- Committee on Early Childhood Mathemat- merous randomized trials, many of them ics of the National Research Council, and published in major education journals. He has worked on the development of several has either directed or co-directed 42 ap- state mathematics standards. Recently, plied research grants addressing a wide Dr. Beckmann taught an average grade array of issues in education and has been a 6 mathematics class every day at a local recipient of many federal and non-federal public school in order to better understand grants (more than $17.5 million). He has school mathematics teaching. She has won more than 150 publications and serves several teaching awards, including the on the editorial boards of 10 prestigious General Sandy Beaver Teaching Professor- journals in the field. He is the director of ship, awarded by the College of Arts and the Math Strand for the Center on Instruc- Sciences at the University of Georgia. tion (which provides technical assistance to the states in terms of implementation of Benjamin Clarke, Ph.D., is a research No Child Left Behind) and the director of associate at the Instructional Research research for the Southwest Regional Edu- Group and Pacific Institutes for Research. cational Laboratory. He serves as a co-principal investigator on three federally funded research grants in Sybilla Beckmann, Ph.D., is a profes- mathematics instructions and assessment. sor of mathematics at the University of His current research includes testing the Georgia. Prior to arriving at the Univer- efficacy of a kindergarten mathematics sity of Georgia, Dr. Beckmann taught at curriculum, evaluating the effectiveness Yale University as a J. W. Gibbs Instructor of a grade 1 mathematics intervention of Mathematics. Dr. Beckmann has done program for at-risk students and exam- research in arithmetic geometry, but her ining the effects of a computer software current main interests are the mathemati- program to build student understanding cal education of teachers and mathematics of and fluency with computational proce- content for students at all levels, but espe- dures. He also serves as the deputy direc- cially for Pre-K through the middle grades. tor of the Center on Instruction Mathemat- Dr. Beckmann developed three mathe- ics. Dr. Clarke was a 2002 graduate of the matics content courses for prospective University of Oregon School Psychology

( 55 ) Appendix B. About the authors

program. He was the recipient of AERA professional development to teachers of Special Education Dissertation Award for kindergarten through grade 5 for multiple his work in early mathematics assess- schools in Howard County. She works with ment. He has continued to investigate and both general educators and special educa- publish articles and materials in this area tors through demonstration lessons, co- and has presented his work at national teaching situations, and school-based pro- conferences. fessional development. She also oversees and coordinates interventions for students Anne Foegen, Ph.D., is an associate pro- struggling in mathematics. fessor in the Department of Curriculum and Instruction at Iowa State University. Jon R. Star, Ph.D., is an assistant profes- Her research focuses on the mathematics sor of education at Harvard University’s development of students with disabilities, Graduate School of Education. Dr. Star is including efforts to develop measures that an educational psychologist who stud- will allow teachers to monitor the prog- ies children’s learning of mathematics in ress of secondary students in mathemat- middle and high school, particularly al- ics. Dr. Foegen has also been involved in gebra. His current research explores the examining the mathematics performance development of flexibility in mathemati- of students with disabilities on large-scale cal problem solving, with flexibility de- assessments, such as the National Assess- fined as knowledge of multiple strategies ment of Educational Progress. Her current for solving mathematics problems and the work in progress monitoring extends re- ability to adaptively choose among known search in curriculum-based measurement strategies on a particular problem. He also (CBM) in mathematics from kindergarten investigates instructional and curricular through grade 12. Her particular focus interventions that may promote the devel- is on studies at the middle school and opment of mathematical understanding. high school levels. In a related project, Dr. Star’s most recent work is supported Dr. Foegen developed CBM measures and by grants from the Institute of Education conducted technical adequacy research Sciences at the U.S. Department of Educa- on the use of the measures to track section and the National Science Foundation. ondary students’ progress in algebra and In addition, he is interested in the preser- prealgebra. vice preparation of middle and secondary mathematics teachers. Before his graduate Laurel Marsh, M.S.E. and M.A.T., is a studies, he spent six years teaching middle professional education instructor who and high school mathematics. serves as a math coach at Swansfield El- ementary School, Howard County School Bradley Witzel, Ph.D., is an associate pro- District, in Columbia, Maryland. Also an fessor and coordinator of special education instructor for the University of Maryland at Winthrop University in Rock Hill, South Baltimore County and Johns Hopkins Uni- Carolina. He has experience in the class- versity, she has served as an elementary room as an inclusive and self-contained teacher at multiple levels. Ms. Marsh re- teacher of students with higher incidence ceived a Master of Arts in Teaching from disabilities as well as a classroom assistant Johns Hopkins University, with a concen- and classroom teacher of students with low tration in both Early Childhood Education incidence disabilities. Dr. Witzel has taught and Elementary Education, as well as a undergraduate and graduate courses in Master of Science in Education from Johns educational methods for students with Hopkins University, with a concentration disabilities and secondary students with in School Administration and Supervision. disabilities coupled with the needs of Eng- As a math support teacher, she provides lish language learning. He has supervised

( 56 ) Appendix B. About the authors

interns in elementary, secondary, and spe- Educational Research Association, Society cial education certification tracks as well for the Scientific Study of Reading, Coun- as inclusion practices. He has given numer- cil for Exceptional Children, International ous professional presentations including Reading Association, and Association for strategic math, algebra instruction, word Supervision and Curriculum Development. problem solving, parent involvement, and He consults nationally in early literacy and motivational classroom management. He reading comprehension instruction. has published research and practitioner articles in algebra and math education as Madhavi Jayanthi, Ph.D., is a research well as positive behavior interventions for associate at Instructional Research Group, students with and without learning disabili- Los Alamitos, California. Dr. Jayanthi has ties. He has also written several books and more than 10 years experience in working book chapters on mathematics education on grants by the Office of Special Educa- and interventions. Overall, he is concerned tion Programs and Institute of Education with the development of special education Sciences. She has published extensively teachers and works to provide researched- in peer-reviewed journals such as Journal validated practices and interventions to of Learning Disabilities, Remedial and Spe- professionals and preprofessionals. cial Education, and Exceptional Children. Her research interests include effective Staff instructional techniques for students with disabilities and at-risk learners, both in the Joseph A. Dimino, Ph.D., is a senior areas of reading and mathematics. research associate at the Instructional Research Group in Los Alamitos, Cali- Shannon Monahan, Ph.D., is a survey re- fornia, where he is the coordinator of a searcher at Mathematica Policy Research. national research project investigating She has served as a reviewer for the What the impact of teacher study groups as a Works Clearinghouse for the Reducing Be- means to enhance the quality of reading havior Problems in the Elementary School instruction for 1st graders in high-poverty Classroom practice guide and the early schools and co-principal investigator for a childhood interventions topic area, and study assessing the impact of collaborative she coordinated the reviews for this prac- strategic reading on the comprehension tice guide. Dr. Monahan has worked exten- and vocabulary skills of English language sively on the development and evaluation learners and English-speaking 5th graders. of mathematics curricula for low-income Dr. Dimino has 36 years of experience as children. Currently, she contributes to sev- a general education teacher, special edu- eral projects that evaluate programs in- cation teacher, administrator, behavior tended to influence child development. Her specialist, and researcher. He has exten- current interests include early childhood sive experience working with teachers, program evaluation, emergent numeracy, parents, administrators, and instructional culturally competent pedagogy, measures assistants in instruction, early literacy, development, and research design. reading comprehension strategies, and classroom and behavior management in Rebecca A. Newman-Gonchar, Ph.D., urban, suburban, and rural communities. is a research associate with the Instruc- He has published in numerous scholarly tional Research Group. She has experi- journals and co-authored books in reading ence in project management, study design comprehension and early reading inter- and implementation, and quantitative and vention. Dr. Dimino has delivered papers qualitative analysis. Dr. Newman-Gon- at various state, national, and interna- char has worked extensively on the de- tional conferences, including the American velopment of observational measures for

( 57 ) Appendix B. About the authors

beginning and expository reading instruc- Libby Scott, M.P.P., is a research analyst tion for two major IES-funded studies of at Mathematica Policy Research and a for- reading interventions for Title I students. mer classroom educator. She has experi- She currently serves as a reviewer for the ence providing research support and con- What Works Clearinghouse for reading and ducting data analysis for various projects mathematics interventions and Response on topics related to out-of-school time pro- to Intervention. Her scholarly contribu- grams, disconnected youth, home school- tions include conceptual, descriptive, and ing households, and item development for quantitative publications on a range of top- a teacher survey. She also has experience ics. Her current interests include Response evaluating an out-of-school time program to Intervention, observation measure de- for middle school students. Ms. Scott used velopment for reading and mathematics her background in classroom teaching and instruction, and teacher study groups. education-related research to support the She has served as the technical editor for panel in translating research findings into several publications and is a reviewer for practitioner friendly text. Learning Disability Quarterly.

( 58 ) Appendix C. this series. Dr. Gersten provided guidance on the product as it relates to struggling Disclosure of potential and English language learner students. conflicts of interest This topic is not covered in this practice guide. The panel never discussed the Practice guide panels are composed of in- Houghton Mifflin series. Russell Gersten dividuals who are nationally recognized has no financial stake in any program or experts on the topics about which they practice mentioned in the practice guide. are rendering recommendations. The In- stitute of Education Sciences (IES) expects Sybilla Beckmann receives royalties on that such experts will be involved profes- her textbook, Mathematics for Elementary sionally in a variety of matters that relate Teachers, published by Addison-Wesley, a to their work as a panel. Panel members division of Pearson Education. This text- are asked to disclose their professional book, used in college mathematics courses involvements and to institute deliberative for prospective elementary teachers, is processes that encourage critical exami- not within the scope of the review of this nation of the views of panel members as practice guide. they relate to the content of the practice guide. The potential influence of panel Ben Clarke has developed and authored members’ professional engagements is journal articles about early numeracy further muted by the requirement that measures that are referenced in the RtI- they ground their recommendations in evi- Mathematics practice guide. Dr. Clarke dence documented in the practice guide. does not have a current financial stake In addition, the practice guide undergoes in any company or publishing of the independent external peer review prior measures. to publication, with particular focus on whether the evidence related to the rec- Anne Foegen conducts research and has ommendations in the practice guide has developed progress monitoring assessments been appropriately presented. that are referred to in the RtI-Mathematics practice guide. Currently, she has no finan- The professional engagements reported cial interests in these products, which are by each panel member that appear most not available commercially. The algebra closely associated with the panel recom- measures are disseminated through Iowa mendations are noted below. State University on a fee-based schedule to cover the costs of personnel for training Russell Gersten has written articles on and materials (not for profit). Dr. Foegen issues related to assessment and screen- also has published papers that describe ing of young children with potential dif- the measures and received a grant that is ficulties in learning mathematics and is supporting a portion of the research. She currently revising a manuscript on this occasionally does private consulting related topic for the scholarly journal, Exceptional to research on use of the mathematics prog- Children. However, there is no fiscal re- ress monitoring measures. ward for this or other publications on the topic. He is a royalty author for what may Jon R. Star consults for a company owned become the Texas or national edition of by Scholastic, which produces mathemat- the forthcoming (2010/11) Houghton Mif- ics educational software. Scholastic may flin reading series, Journeys. At the time produce other curricula related to mathe- of publication, Houghton Mifflin has not matics, but the panel makes no recommen- determined whether they will ever release dations for selecting specific curricula.

( 59 ) Appendix C. Disclosure of potential conflicts of interest

Bradley Witzel wrote the book, Compu- delivered workshop presentations on the tation of Fractions, and is currently writ- structure of RtI (not associated with the ing Computation of Integers and Solving RtI-Mathematics practice guide). The work Equations, through Pearson with Dr. Paul on his books is separate from that of the Riccomini. He is also writing the book, RtI RtI-Mathematics practice guide panel, and Mathematics, through Corwin Press with he does not share his work from the panel Dr. Riccomini. Additionally, Dr. Witzel has with the books’ co-authors.

( 60 ) Appendix D. stitute on Progress Monitoring197 and the National Center on Progress Monitoring Technical information were also used.198 on the studies The studies of screening measures all Recommendation 1. used appropriate correlational designs.199 Screen all students to identify In many cases, the criterion variable was those at risk for potential some type of standardized assessment, mathematics difficulties and often a nationally normed test (such as provide interventions to students the Stanford Achievement Test) or a state identified as at risk. assessment. In a few cases, however, the criterion measure was also tightly aligned Level of evidence: Moderate with the screening measure.200 The latter set is considered much weaker evidence The panel examined reviews of the tech- of validity. nical adequacy of screening measures for students identified as at risk when making Studies also addressed inter-tester this recommendation. The panel rated the reliability,201 internal consistency,202 test- level of evidence for recommendation 1 as retest reliability,203 and alternate form reli- moderate because several reviews were ability.204 Many researchers discussed the available for evidence on screening mea- content validity of the measure.205 A few sures for younger students. However, even discussed the consequential valid- there was less evidence available on these ity206—the consequences of using screen- measures for older students. The panel ing data as a tool for determining what relied on the standards of the American requires intervention.207 However, these Psychological Association, the American studies all used standardized achievement Educational Research Association, and the measures as the screening measure. National Council on Measurement in Edu- cation194 for valid screening instruments In recent years, a number of studies of along with expert judgment to evaluate screening measures have also begun to the quality of the individual studies and to determine the overall level of evidence for this recommendation. 197. http://www.progressmonitoring.net/. 198. www.studentprogress.org. Relevant studies were drawn from recent 199. Correlational studies are not eligible for comprehensive literature reviews and re- WWC review. ports195 as well as literature searches of 200. For example, Bryant, Bryant, Gersten, Scam- databases using key terms (such as “for- macca, and Chavez (2008). mative assessment”). Journal articles sum- 201. For example, Fuchs et al. (2003a). marizing research studies on screening 202. For example, Jitendra et al. (2005). in mathematics,196 along with summary 203. For example, VanDerHeyden, Witt, and Gil- information provided by the Research In- bertson (2003). 204. For example, Thurber, Shinn, and Smol 194. American Educational Research Associa- kowski (2002). tion, American Psychological Association, and 205. For example, Clarke and Shinn (2004); Ger- National Council on Measurement in Education sten and Chard (1999); Foegen, Jiban, and Deno (1999). (2007). 195. For example, the National Mathematics Advi- 206. Messick (1988); Gersten, Keating, and Irvin sory Panel (2008). (1995). 196. Gersten et al. (2005); Fuchs, Fuchs, Compton 207. For example, Compton, Fuchs, and Fuchs et al. (2007); Foegen et al. (2007). (2007).

( 61 ) Appendix D. Technical information on the studies

report sensitivity and specificity data.208 proficiency measures.212 Still others de- Because sensitivity and specificity pro- veloped a broader measure that assessed vide information on the false positive and multiple proficiencies in their screening.213 false negative rates respectively, they are An example of a single proficiency embed- critical in determining the utility of a mea- ded in a broader measure is having stu- sure used in screening decisions linked dents compare magnitudes of numbers. to resource allocation. Note that work on As an individual measure, magnitude com- sensitivity and specificity in educational parison has predictive validity in the .50 screening is in its infancy and no clear to .60 range,214 but having students make standards have been developed. magnitude comparisons is also included in broader measures. For example, the Num- The remainder of this section presents ber Knowledge Test (NKT)215 requires stu- evidence in support of the recommenda- dents to name the greater of two verbally tion. We discuss the evidence for measures presented numbers and includes problems used in both the early elementary and assessing strategic counting, simple addi- upper elementary grades and conclude tion and subtraction, and word problems. with a more in-depth example of a screen- The broader content in the NKT provided ing study to illustrate critical variables to stronger evidence of predictive validity216 consider when evaluating a measure. than did single proficiency measures.

Summary of evidence Further information on the characteristics and technical adequacy of curriculum- In the early elementary grades, mea- based measures (CBM) for use in screen- sures examined included general out- ing in the elementary grades was summa- come measures reflecting a sampling of rized by Foegen, Jiban, and Deno (2007). objectives for a grade level that focused They explained that measures primarily on whole numbers and number sense. assessed the objectives of operations or These included areas of operations and the concepts and applications standards procedures, number combinations or basic for a specific grade level. A smaller num- facts, concepts, and applications.209 Mea- ber of measures assessed fluency in basic sures to assess different facets of number facts, problem solving, or word problems. sense—including measures of rote and Measures were timed and administration strategic counting, identification of numer- time varied between 2 and 6 minutes for als, writing numerals, recognizing quanti- operations probes and 2 to 8 minutes for ties, and magnitude comparisons—were concepts and applications. Reliability evi- also prevalent.210 Some research teams dence included test-retest, alternate form, developed measures focused on a single internal consistency, and inter-scorer, aspect of number sense (such as strategic with most reliabilities falling between .80 counting),211 and others developed batter- and .90, meeting acceptable standards ies to create a composite score from single for educational decisionmaking. Similar evidence was found for validity with most

212. Bryant, Bryant, Gersten, Scammacca, and 208. Locuniak and Jordan (2008); VanDerHey- Chavez (2008). den et al. (2001); Fuchs, Fuchs, Compton et al. 213. Okamoto and Case (1996). (2007). 214. Lembke et al. (2008); Clarke and Shinn 209. Fuchs, Fuchs, Compton et al. (2007). (2004); Bryant, Bryant, Gersten, Scammacca, and 210. Gersten, Clarke, and Jordan (2007); Fuchs, Chavez (2008). Fuchs, and Compton et al. (2007). 215. Okamoto and Case (1996). 211. Clarke and Shinn (2004). 216. Chard et al. (2005).

( 62 ) Appendix D. Technical information on the studies

concurrent validity coefficients in the .50 measures to assess. Included were number to .70 range. Lower coefficients were found sense measures that assessed knowledge for basic fact measures ranging from .30 of counting, number combinations, non- to .60. Researchers have also begun to de- verbal calculation, story problems, num- velop measures that validly assess magni- ber knowledge, and short and working tude comparison, estimation, and prealge- memory. The authors used block regres- bra proficiencies.217 sion to examine the added value of the math measures in predicting achievement A study of evaluating a mathematics above and beyond measures of cognition, screening instrument—Locuniak and age, and reading ability (block 1), which Jordan (2008) accounted for 26 percent of the variance on 2nd grade calculation fluency. Adding A recent study by Locuniak and Jordan the number sense measures (block 2) in- (2008) illustrates factors that districts creased the variance explained to 42 per- should consider when evaluating and se- cent. Although the research team found lecting measures for use in screening. The strong evidence for the measures assess- researchers examined early mathematics ing working memory (digit span), number screening measures from the middle of knowledge, and number combinations, kindergarten to the end of second grade. the array of measures investigated is in- The two-year period differs from many dicative that the field is still attempting to of the other screening studies in the area understand which critical variables (math- by extending the interval from within a ematical concepts) best predict future dif- school year (fall to spring) to across sev- ficulty in mathematics. A similar process eral school years. This is critical because has occurred in screening for reading dif- the panel believes the longer the interval ficulties where a number of variables (such between when a screening measure and a as alphabetic principle) are consistently criterion measure are administered, the used to screen students for reading dif- more schools can have confidence that stu- ficulty. Using the kindergarten measures dents identified have a significant deficit with the strongest correlations to grade 2 in mathematics that requires intervention. mathematics achievement (number knowl- The Locuniak and Jordan (2008) study also edge and number combinations), the re- went beyond examining traditional indices searchers found rates of .52 for sensitivity of validity to examine specificity and sen- and .84 for specificity. sitivity. Greater sensitivity and specificity of a measure ensures that schools provide Another feature that schools will need resources to those students truly at risk to consider when evaluating and select- and not to students misidentified ing measures is whether the measure is timed. The measures studied by Locuniak The various measures studied by Lo- and Jordan (2008) did not include a timing cuniak and Jordan (2008) also reflected component. In contrast, general outcome mathematics content that researchers measures include a timing component.218 consider critical in the development of No studies were found by the panel that a child’s mathematical thinking and that examined a timed and untimed version of many researchers have devised screening the same measure.

217. Foegen et al. (2007). 218. Deno (1985).

( 63 ) Appendix D. Technical information on the studies

Recommendation 2. foundational concepts and procedures. Instructional materials for students However, we concluded that the focus on receiving interventions should coverage of fewer topics in more depth, focus intensely on in-depth and with coherence, advocated for general treatment of whole numbers in education students is as important, and kindergarten through grade 5 and probably more important, for students on rational numbers in grades 4 who struggle with mathematics. We could through 8. These materials should not locate any experimental research that be selected by committee. supported our belief, however. Therefore, we indicate clearly that we are reflecting Level of evidence: Low a growing consensus of professional opinion, not a convergent body of scientific The panel based this recommendation evidence—and conclude that the level of on professional opinion; therefore, the evidence is low. evidence rating is low. The professional opinion included not only the views of Summary of evidence the panel members, but also several recent consensus documents that reflect Three seminal publications were con- input from mathematics educators and sulted in forming our opinion.221 Milgram research mathematicians involved in is- and Wu (2005) were among the first to sues related to K–12 mathematics educa- suggest that an intervention curriculum tion.219 Each of these documents was infor at-risk students should not be over- fluenced to some extent by comparisons simplified and that in-depth coverage of of curricula standards developed by the key topics and concepts involving whole 50 states in the United States with nations numbers and then rational numbers was with high scores on international tests of critical for future success in mathematics. mathematics performance, such as the They stressed that mastery of this mate- Trends in International Mathematics and rial was critical, regardless of how long it Science Study (TIMSS) and the Program for takes. Many before had argued about the International Student Assessment (PISA) importance of mastery of units before (including the Czech Republic, Flemish proceeding forward.222 Milgram and Wu Belgium, Korea, and Singapore).220 We note, argued that stress on precise definitions however, that these international compari- and abstract reasoning was “even more sons are merely descriptive and thus do critical for at-risk students” (p. 2). They not allow for causal inferences. In other acknowledged this would entail extensive words, we do not know whether their more practice with feedback and considerable focused curricula or other factors contrib- instructional time. ute to higher performance. The National Council of Teachers of Math- We note that some of the other reports ematics Curriculum Focal Points (2006) we describe here do not directly address made a powerful statement about reform the needs of students who receive in- of mathematics curriculum for all students terventions to boost their knowledge of by calling for the end of brief ventures into many topics in the course of a school year.

219. Milgram and Wu (2005); National Council of Teachers of Mathematics (2006); National Math- 221. Milgram and Wu (2005); National Council of ematics Advisory Panel (2008). Teachers of Mathematics (2006); National Math- 220. For more information on the TIMSS, see ematics Advisory Panel (2008). http://nces.ed.gov/timss/. For more information 222. For example, Bloom (1980); Guskey (1984); on PISA, see www.oecd.org. Silbert, Carnine, and Stein (1989).

( 64 ) Appendix D. Technical information on the studies

The topics it suggests emphasize whole The panel found six studies223 conducted numbers (properties, operations, prob- with low achieving or learning disabled lem solving) and especially fractions and students224 between 2nd and 8th grades related topics involving rational numbers that met WWC standards or met standards (proportion, ratio, decimals). The report is with reservations and included compo- equally clear that algorithmic proficiency nents of explicit and systematic instruc- is critical for understanding properties tion.225 Appendix table D1 (p. 69) provides of operations and related concepts and an overview of the components of explicit that algorithmic proficiency, quick re- instruction in each intervention, including trieval of mathematics facts, and in-depth the use of teacher demonstration (such knowledge of such concepts as place value as verbalization during demonstration and properties of whole numbers are all and the use of multiple examples), stu- equally important instructional goals. This dent verbalization (either as a procedural position was reinforced by the report of requirement or as a response to teacher the National Mathematics Advisory Panel questions), scaffolded practice, cumula- (2008) two years later, which provided de- tive review, and corrective feedback. The tailed benchmarks and again emphasized relevant treatment and comparison groups in-depth coverage of key topics involving compared in each study and the outcomes whole numbers and rational numbers as found for each domain are included in the crucial for all students. table, as are grade-level, typical session length, and duration of the intervention. In the view of the panel, students in intervention programs need to master material Because of the number of high-quality on whole numbers and rational numbers, randomized and quasi-experimental de- and they must ultimately work with these sign studies using explicit and systematic concepts and principles at an abstract mathematics instruction across grade lev- level. We feel that it is less important for els and diverse student populations, the 4th graders in an intervention program frequency of significant positive effects, to cover the entire scope and sequence and the fact that numerous research teams of topics from the year before. Instead, independently produced similar findings, the aim is to cover the key benchmarks the panel concluded that there is strong articulated in the National Mathematics evidence to support the recommendation Advisory Panel report, involving whole to provide explicit and systematic instruc- numbers and rational numbers that stu- tion in tier 2 mathematics interventions. dents do not fully grasp, and build proficiencies they lack.

Recommendation 3. Instruction during the intervention should be explicit and systematic. This 223. Darch, Carnine, and Gersten (1984); Fuchs includes providing models of et al. (2003a); Jitendra et al. (1998); Tournaki proficient problem solving, (2003); Schunk and Cox (1986); Wilson and Sin- verbalization of thought processes, delar (1991). guided practice, corrective feedback, 224. These students specifically had difficulties and frequent cumulative review. with mathematics. 225. For this practice guide, the components of Level of evidence: Strong explicit and systematic mathematics instruction are identified as providing models of proficient problem solving, verbalizing teacher and student The panel judged the level of evidence sup- thought processes, scaffolded practice, cumula- porting the recommendation to be strong. tive review, and corrective feedback.

( 65 ) Appendix D. Technical information on the studies

Summary of evidence numerous models of solving easy and hard problems proficiently. Demonstration with Teacher demonstrations and think-alouds. easy and hard problems and the use of The panel suggests that teachers verbal- numerous examples were not assessed ize the solution to problems as they model as independent variables in the studies problem solving for students. Tournaki reviewed. However, Wilson and Sindelar (2003) assessed this approach by compar- (1991) did use numerous examples in ining a group of students whose teachers struction for both groups evaluated. The had demonstrated and verbalized an ad- key difference between the groups was that dition strategy (the Strategy group) against students in the treatment group were ex- a group of students whose teacher did not plicitly taught problem-solving strategies verbalize a strategy (the Drill and Practice through verbal and visual demonstrations group). As depicted in appendix table D1, while students in the comparison group the effects on an assessment of single-digit were not taught these strategies. This study addition were significant, positive, and demonstrated substantively important pos- substantial in favor of the students whose itive effects with marginal significance in teacher had verbalized a strategy.226 favor of the treatment group.228

All six studies examined interventions that Scaffolded practice. Scaffolded practice, included teacher demonstrations early in a transfer of control of problem solving the mathematics lessons.227 For example, from the teacher to the student, was a Schunk and Cox (1986), Jitendra et al. component of mathematics interventions (1998), and Darch, Carnine, and Gersten in four of the six studies.229 In each study, (1984) all conducted studies in which in- the intervention groups that included scaf- struction began with the teacher verbaliz- folded practice demonstrated significant ing the steps to solve sample mathemat- positive effects; however, it is not possible ics problems. Because this demonstration to parse the effects of scaffolded instruc- procedure was used to instruct students tion from the other components of ex- in both treatment and comparison groups, plicit instruction in these multicomponent the effects of this component of explicit in- interventions. struction cannot be evaluated from these studies. However, the widespread use of Student verbalization. Three of the six teacher demonstration in interventions studies230 included student verbalization that include other components of explicit of problem-solution steps in the interven- instruction supports the panel’s conten- tions. For example, Schunk and Cox (1986) tion that this is a critical component of assessed the effect of having students ver- explicit instructional practice. balize their subtraction problem-solving steps versus solving problems silently. For teacher demonstration, the panel spe- There were significant and substantial cifically recommends that teachers provide positive effects in favor of the group that

226. Note that during the intervention, students 228. For this guide, the panel defined margin- in the Strategy condition were also encouraged to ally significant as a p-value in the range of .05 verbalize the problem-solving steps and that this to .10. Following WWC guidelines, an effect size may also be a factor in the success of the inter- greater than 0.25 is considered substantively vention. The Tournaki (2003) study is described important. in more detail below. 229. Darch, Carnine, and Gersten (1984); Fuchs 227. Darch, Carnine, and Gersten (1984); Fuchs et al. (2003a); Jitendra et al. (1998); Tournaki et al. (2003a); Jitendra et al. (1998); Tournaki (2003). (2003); Schunk and Cox (1986); Wilson and Sin- 230. Schunk and Cox (1986); Jitendra et al. (1998); delar (1991). Tournaki (2003).

( 66 ) Appendix D. Technical information on the studies

verbalized steps. The Tournaki (2003) In summary, the components of explicit intervention also included student ver- and systematic instruction are consistently balization among other components and associated with significant positive effects had significant positive effects. Among on mathematics competency, most often other intervention components, Jitendra when these components are delivered in et al. (1998) included student verbalization combination. An example of a study that through student responses to a teacher’s examines the effects of a combination of facilitative questions. Again, the effects these components is described here. were substantively important or statistically significant and positive, but they can- A study of explicit and systematic not be attributed to a single component in instruction—Tournaki (2003) this multi component intervention. Explicit and systematic instruction is a Corrective feedback. Four of the six stud- multicomponent approach, and an in- ies included immediate corrective feed- tervention examined in Tournaki (2003) back in the mathematics interventions.231 exemplifies several components in com- For example, in the Darch, Carnine, and bination. This study was conducted with Gersten (1984) study, when a student 42 students in grade 2 special education made an error, teachers in the treatment classrooms.234 The students, between 8 group would first model the appropriate and 10 years old, were classified as learn- response, then prompt the students with ing disabled with weaknesses in both read- questions to correct the response, then ing and mathematics. Twenty-nine were reinforce the problem-solving strategy boys, and 13 were girls. steps again. In three of the studies,232 the effects of the corrective feedback compo- Prior to the intervention, the students nent cannot be isolated from the effects completed a pretest assessment consist- of the other instructional components; ing of 20 single-digit addition problems however, the effects of the interventions (such as 6 + 3 = ). Internal consistency including corrective feedback were posi- of the assessment was high (Cronbach’s tive and significant. alpha of .91). Student performance on the assessment was scored for accuracy and Cumulative review. The panel’s assertion latency (the time it took each student to that cumulative review is an important complete the entire assessment). The ac- component of explicit instruction is based curacy score is a measure of student abil- primarily on expert opinion because only ity to perform mathematical operations, one study in the evidence base included and the latency score is an indication of cumulative review as a component of the student fluency with single-digit addition intervention.233 This study had positive sig- facts. After the intervention, students nificant effects in favor of the instructional completed a posttest assessment that was group that received explicit instruction in identical to the pretest. strategies for solving word problems. Students were randomly assigned to one of three groups (two instruction groups and

231. Darch, Carnine, and Gersten (1984); Jiten- dra et al. (1998); Tournaki (2003); Schunk and Cox (1986). 234. The sample also included 42 grade 2 stu- 232. Darch, Carnine, and Gersten (1984); Jitendra dents from general education classrooms, but et al. (1998); Tournaki (2003). only the results for the special education students 233. Fuchs et al. (2003a). are presented as relevant to this practice guide.

( 67 ) Appendix D. Technical information on the studies

a comparison group).235 Students in the as she says, “5, 6, 7, 8. How many did I two instruction groups met individually end up with? Eight. I’ll write 8 to the right with a graduate assistant for a maximum of the equal sign.” After writing the num- of eight 15-minute supplemental math- ber, the teacher finishes modeling by say- ematics sessions on consecutive school ing, “I’ll read the whole problem: 5 plus 3 days. Instructional materials for both equals 8.” groups consisted of individual worksheets for each lesson with a group of 20 sin- The teacher and student solved two prob- gle-digit addition problems covering the lems together through demonstration and range from 2 + 2 = to 9 + 9 = . structured probing. The student was then asked to solve a third problem indepen- The Strategy instruction group received dently while verbalizing the strategy steps explicit and systematic instruction to im- aloud. When a student made an error, the prove fact fluency. The instruction began teacher gave corrective feedback. The stu- with the teacher modeling the minimum dent was asked to solve the remaining prob- addend strategy for the students and lems without verbalization and to work thinking aloud. This strategy is an ef- as fast as possible, but when an error oc- ficient approach for solving single-digit curred, the teacher interrupted the lesson addition problems (such as 5 + 3 = ). and reviewed the steps in the strategy. The teacher began by saying, “When I get a problem, what do I do? I read the prob- Students in the Drill and Practice group lem: 5 plus 3 equals how many? Then I find were asked to solve the problems as the smaller number.” Pointing to the num- quickly as possible. At the completion of ber, the teacher says, “Three. Now I count each lesson, the teacher marked the stu- fingers. How many fingers am I going to dent’s errors and asked the student to re- count? Three.” The teacher counts three compute. If the error persisted, the teacher fingers, points to the larger number and told the student the correct answer. Re- says, “Now, starting from the larger num- sults indicate significant and substantial ber, I will count the fingers.” The teacher positive effects in favor of the Strategy points to the 5, then touches each finger group, which received explicit and systematic instruction, relative to Drill and 235. Students in the comparison group received Practice group, which received a more tra- only the pretest and posttest without any supple- ditional approach. In this study, the com- mental mathematics instruction outside their bination of teacher demonstration, student classroom. Because the scope of the practice verbalization, and corrective feedback was guide is examining the effects of methods of successful in teaching students with math- teaching mathematics for low-achieving students, the comparison group findings are not ematics difficulties to accurately complete included here. single-digit addition problems.

( 68 ) Appendix D. Technical information on the studies a Outcomes 1.01* .56 (n.s.) 1.01* 2.09* 1.79* 2.21* 1.10* .82~ Domain Operations Word problems Transfer Word problems Word problems Operations Transfer Word problems

sessions Duration 45 minutes/ session; 6 sessions 40–45 minutes/ session; 17–20 sessions 25–40 minutes/ session; 36 11 sessions 11 30 minutes/ session; 15 minutes/ session; up to 8 sessions 30 minutes/ session; sessions 14 3 4 2 2–5 2–4 6–8 level Grade



review Cumulative     feedback Corrective     Guided practice - in the intervention     tions Student verbaliza - Components of explicit of Components instruction included       demon stration Teacher Teacher

onal basal basal onal ti Comparison Counting-on Counting-on strategy versus instruction drill and practice Explicit strategy versus instruction basal traditional instruction Continuous verbalizations by students versus no student verbalizations instruction Explicit visual strategy versus instruction basal traditional instruction Strategy Strategy versus instruction sequencing of practice problems Instruction on on Instruction solving word problems that is based on common underlying structures versus tradi

: Authors’ analysis based on studies in table. ect size is not significant(n.s.). Outcomes are reported as effect sizes.For a p-value the.05, < effect significant is size .10, (*); for the a effect< p-value size is marginally significant≥ .10, (~); for the ap-value eff

Tournaki (2003) Darch, Carnine, and Gersten (1984) Study Schunk and Cox (1986) Jitendra et al. et Jitendra (1998) Wilson and Sindelar (1991) Fuchs et al. et Fuchs (2003a) a. Source (with and(with without reservations) Table D1. Studies of interventions that included explicit instruction and met WWC Standards WWC Standards met and instruction explicit included that interventions of Studies D1. Table

( 69 ) Appendix D. Technical information on the studies

Recommendation 4. Interventions we conclude that there is strong evidence should include instruction on to support this recommendation. solving word problems that is based on common underlying structures. Summary of evidence

Level of evidence: Strong Teach the structure of problem types. In three of the studies, students were taught The panel rated the level of evidence as to identify problems of a given type and strong. We located nine studies that met the then to design and execute appropriate standards of the WWC or met the standards solution strategies for each type.239 In one with reservations and demonstrated sup- of these interventions, students learned to port for the practice of teaching students to represent the problem using a schematic solve word problems based on their under- diagram.240 Once students learned to iden- lying structures.236 Appendix table D2 (p. 73) tify the key problem features and map provides an overview of each of the inter- the information onto the diagram, they ventions examined in these nine studies. learned to solve for unknown quantities in word problems while still representing the In all nine interventions, students were problem using a schematic diagram. This taught to recognize the structure of a word intervention had significant and positive problem in order to solve it, and they were effects on a word problem outcome based taught how to solve each problem type.237 on a test of problems similar to those Six of the studies took the instruction on taught during the intervention. problem structure a step further. Students were taught to distinguish superficial from In another intervention that also led to a substantive information in word problems significant and positive effect on a word in order to transfer solution methods from problem outcome, students were taught to familiar problems they already knew how discriminate between multiplication and to solve to problems that appeared unfamil- addition problems, and between multiplica- iar.238 Because of the large number of high- tion and division problems.241 To discrimi- quality randomized studies conducted that nate multiplication from addition problems, examined this practice and because most students were taught that if a problem asks of the interventions examined led to sig- them to use the same number multiple times nificant and positive effects on word prob- (sometimes signaled by the words “each” lem outcomes for children designated as and "every”) to obtain the total number, low achieving and/or learning disabled, the problem requires multiplication. If the problem does not ask the student to use the 236. Jitendra et al. (1998); Xin, Jitendra, and Deat- same number multiple times to obtain the line-Buchman (2005); Darch, Carnine, and Gersten total number, the problem requires addition. (1984); Fuchs et al. (2003a); Fuchs et al. (2003b); Next, after students learned the relationship Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, between multiplication and division through Finelli et al. (2004); Fuchs, Seethaler et al. (2008); the concept of number families, they learned Fuchs, Fuchs, Craddock et al. (2008). to multiply when the two smaller numbers 237. Jitendra et al. (1998); Xin, Jitendra, and Deat- line-Buchman (2005); Darch, Carnine, and Gersten are given without the big number and to (1984); Fuchs et al. (2003a); Fuchs et al. (2003b); divide when the big number is given. Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, Finelli et al. (2004); Fuchs, Seethaler et al. (2008); Fuchs, Fuchs, Craddock et al. (2008). 239. Jitendra et al. (1998); Xin, Jitendra, and Deatline-Buchman (2005); Darch, Carnine, and 238. Fuchs et al. (2003a); Fuchs et al. (2003b); Gersten (1984). Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. 240. Xin, Jitendra, and Deatline-Buchman (2005). (2008); Fuchs, Seethaler et al. (2008). 241. Darch, Carnine, and Gersten (1984).

( 70 ) Appendix D. Technical information on the studies

Transfer solution methods from familiar A study of teaching students problem types to problems that appear un- to transfer solution methods—Fuchs, familiar. In addition to teaching students Fuchs, Finelli, Courey, and Hamlett to recognize and solve different problem (2004) types, six of these studies taught students how to transfer solution methods to prob- Fuchs and colleagues (2004) conducted lems that appear different but really re- a study that investigated the effects of quire the same solution methods as those teaching students how to transfer known they already know how to solve.242 In each solution methods to problems that are of these interventions, students were first only superficially different from those taught the pertinent structural features they already know how to solve.245 The and problem-solution methods for differ- authors randomly assigned 24 teachers to ent problem types. Next, they were taught three groups: 1) transfer instruction, 2) ex- about superficial problem features that can panded transfer instruction, and 3) regular change a problem without altering its struc- basal instruction (comparison group).246 ture or solution (for example, different for- The 351 students in these 24 classes that mat, different key vocabulary, additional or were present for each of the pretests and different question, irrelevant information) posttests were participants in the study. and how to solve problems with varied cover stories and superficial features. The intervention included 25- to 40-minute lessons, approximately twice per week for In all six studies, word problem outcome 17 weeks.247 Students in the expanded trans- measures ranged from those where the fer condition learned basic math problem- only source of novelty was the cover story solving strategies in the first unit of instruc- (immediate transfer), to those that varied tion (six sessions over three weeks). They one or more superficial features (near or were taught to verify that their answers far transfer). In five cases243 the average make sense; line up numbers from text to impact of the intervention on these out- perform math operations; check operations; come measures was positive and signifi- and label their work with words, monetary cant for the samples designated as low signs, and mathematical symbols. achieving and/or learning disabled, and in one case,244 the impact was marginally The remaining units each focused on one of significant. These studies show that in- four problem types: 1) shopping list prob- struction on problem structure and trans- lems (buying multiple quantities of items, ferring known solution methods to unfa- each at a different price); 2) buying bag prob- miliar problems is consistently associated lems (determining how many bags contain- with marginally or statistically significant ing a specified number of objects are needed positive effects on word problem solving to come up with a desired total number of proficiencies for students experiencing mathematics difficulties. 245. Fuchs, Fuchs, Finelli et al. (2004). 246. Since the comparison between the expanded transfer condition and the control condition (regular basal instruction) is most relevant to this 242. Fuchs et al. (2003a); Fuchs et al. (2003b); practice guide, we do not discuss the transfer Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, instruction condition here. Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. 247. Although this intervention was taught in a (2008); Fuchs, Seethaler et al. (2008). whole-class format, the authors reported sepa- 243. Fuchs et al. (2003a); Fuchs et al. (2003b); Fuchs, rate effects for students classified as low achiev- Fuchs, Prentice et al. (2004); Fuchs, Fuchs, Finelli et ing and for students classified as learning dis- al. (2004); Fuchs, Fuchs, Craddock et al. (2008). abled; therefore, the results are relevant to this 244. Fuchs, Seethaler et al. (2008). practice guide.

( 71 ) Appendix D. Technical information on the studies

objects); 3) half problems (determining what In the seventh session, teachers instructed half of some total amount is); and 4) picto- students on three additional superficial graph problems (summing two addends, problem features including irrelevant infor- one derived from a pictograph). There were mation, combining problem types, and mix- seven sessions within each unit. ing superficial problem features. Teachers taught this lesson by discussing how prob- In sessions one through four, with the lems encountered in “real life” incorporate help of a poster listing the steps, students more information than most problems that learned problem-solution rules for solving the students know how to solve. They used the type of problem being taught in that a poster called Real-Life Situations to illus- particular unit. In the first session, teachers trate each of these superficial problem fea- discussed the underlying concepts related tures with a worked example. Next, students to the problem type, presented a worked worked in pairs to solve problems that var- example, and explained how each step of ied real-life superficial problem features and the solution method was applied in the then completed a problem independently. example. After presenting several worked examples, the teachers presented partially The authors used four measures to deter- worked examples while the students ap- mine the results of their intervention on plied the steps of the solution method. Stu- word problem-solving proficiencies. The dents then completed one to four problems first measure used novel problems struc- in pairs. Sessions two through four were tured the same way as problems used in similar, but more time was spent on par- the intervention. The second incorporated tially worked examples and practice, and novel problems that varied from those used at the end of each session, students com- in instruction in terms of the look or the pleted a problem independently. vocabulary or question asked. The third incorporated novel problems that varied In sessions five and six, teachers taught by the three additional transfer features students how to transfer the solution meth- taught in session seven. The fourth was a ods using problems that varied cover sto- measure designed to approximate real-life ries, quantities, and one transfer feature problem solving. Although this intervention per problem. In session five, the teachers was taught in a whole-class format, the au- began by explaining that transfer means thors separated results for students classi- to move and presented examples of how fied as low performing248 and for students students transfer skills. Then, teachers classified as learning disabled. The average taught three transfer features that change impacts on these four outcome measures a problem without changing its type or were positive and significant for both the solution, including formatting, unfamiliar sample designated as low performing and vocabulary, and posing a different ques- the sample designated as learning disabled. tion. These lessons were facilitated by a It is notable that the intervention had a posi- poster displayed in the classroom about tive and significant impact on the far trans- the three ways problems change. Again, fer measure (the measure that approximated teachers presented the information and real-life problem solving). This study dem- worked examples, and moved gradually onstrates a successful approach for instruct- to partially worked examples and prac- ing students with mathematics difficulties tice in pairs. Session six was similar to on solving word problems and transferring session five, but the students spent more solution methods to novel problems. time working in pairs, and they completed a transfer problem independently. 248. Using pretest scores on the first transfer problem-solving measure, the authors designated each student as low performing, average performing, or high performing. ( 72 ) Appendix D. Technical information on the studies

.73* .86* .66~ 1.01* 1.10* 4.75* 1.23* 1.79* 2.05* 1.87* 3.08* 2.09* .56 (n.s.) .60 (n.s.) .60

Outcomes Domain Transfer Concepts Word problems Word problems Word problems Word problems Word problems Word problems Word problems Word problems Word problems Word problems Word problems Word problems

f c e b d combined Low achieving Low achieving Low achieving Low achieving Low achieving Low achieving Low achieving Low achieving and low achieving Learning disabled Learning disabled Learning disabled Learning disabled Learning disabled Learning disabled/ Learning s on ssi sessions sessions sessions sessions sessions se sessions sessions sessions Duration Number of session; 11 session; 11 session; 12 session; 32 session; 36 session; 38 session; 36 session; 34 minutes not 30 minutes/ 60 minutes/ reported; 32 session; 17-20 session; 17-20 20–30 minutes/ 20–30 minutes/ 25–40 minutes/ 25–40 minutes/ 25–40 minutes/ 40–45 minutes/ 3 3 3 3 4 3 3 2–5 6–8 level Grade Comparison Instruction solving on word problems that basedis common on underlying structures versus traditional basal instruction Instruction solving on word problems based common on underlying structures versus general strategy instruction Instruction solving on word problems that basedis common on underlying structures versus nonrelevant instruction Instruction solving on word problems that basedis common on underlying structures versus nonrelevant instruction Instruction solving on word problems that basedis common on underlying structures versus traditional basal instruction Instruction solving on word problems that basedis common on underlying structures versus traditional basal instruction Instruction solving on word problems that basedis common on underlying structures versus traditional basal instruction Instruction solving on word problems that basedis common on underlying structures versus traditional basal instruction Instruction solving on word problems that basedis common on underlying structures versus traditional basal instruction

Authors’ analysis based on studies in table. ing speech delay. hav Outcomes are reported as effect sizes.For a p-value the.05, < effect significant is size .10, (*); for the a effect< p-value size is marginally significant≥ .10, (~); for the ap-value Fifteen students in this sample were classified as having a learning disability and five were classified as having an “other” disability. Seventeen students in this sample were classified as having a learning disability, five as beingeducable mentallyretarded, and three as being seriously emotionallyEighteen disturbed. students in this sample were classified as having a learning disability, one as being seriously emotionally disturbed, and three were not labeled. Twenty-two students in this sample were classified as having a learning disability, one as being mildly mentally retarded, one as having a behavior disorder, and three as deficit/hyperactivity disorder. effect size is significantnot (n.s.). Thirteen students in this sample were classified as having a learning disability, one as having mental retardation, eight as havingspeech a disorder, and two as havingattention-

Fuchs, Fuchs, Prentice al.et (2004) Xin, Jitendra, and Deatline-Buchman (2005) Fuchs, SeethalerFuchs, al. et (2008) Fuchs, Fuchs, Craddock Fuchs, Fuchs, al.et (2008) al. et (1998) Jitendra Fuchs et al. et (2003a) Fuchs Study and Carnine, Darch, (1984) Gersten Fuchs, Fuchs, Finelli Finelli Fuchs, Fuchs, et al.et (2004) Fuchs et al. et (2003b) Fuchs Source: Table D2. Studies of interventions that taught students to discriminate problem types that met WWC standards WWC standards met that types problem discriminate to students taught that interventions of Studies D2. Table a. b. c. d. e. f. (with or(with without reservations)

( 73 ) Appendix D. Technical information on the studies

Recommendation 5. Intervention In one study, visual representations were materials should include used to teach mathematics facts.253 In all opportunities for students to 13 studies, representations were used to work with visual representations understand the information presented in of mathematical ideas and the problem. Specifically, the representa- interventionists should be proficient tions helped answer such questions as in the use of visual representations what type of problem it is and what oper- of mathematical ideas. ation is required. In all 13 studies, visual representations were part of a complex Level of evidence: Moderate multicomponent instructional intervention. Therefore, it is not possible to ascer- The panel judged the level of evidence tain the role and impact of the representa- for this recommendation to be moderate. tion component. We found 13 studies conducted with students classified as learning disabled or low Of the 13 studies, 4 used visual represen- achieving that met WWC standards or met tations, such as drawings or other forms standards with reservations.249 Four in of pictorial representations, to scaffold particular examined the impact of tier 2 in- learning and pave the way for the under- terventions against regular tier 1 instruc- standing of the abstract version of the tion.250 Appendix table D3 (p. 77) provides representation.254 Jitendra et al. (1998) an overview of these 13 studies. Note that examined the differential effects of two in an attempt to acknowledge meaning- instructional strategies, an explicit strat- ful effects regardless of sample size, the egy using visual representations and a panel followed WWC guidelines and con- traditional basal strategy. Students were sidered a positive statistically significant taught explicitly to identify and differenti- effect, or an effect size greater than 0.25, ate among word problems types and map as an indicator of positive effects.251 the features of the problem onto the given diagrams specific to each problem type. Summary of evidence The intervention demonstrated a nonsignificant substantively important positive The representations in 11 of the 13 studies effect. Wilson and Sindelar (1991) used a were used mainly to teach word problems diagram to teach students the “big num- and concepts (fractions and prealgebra).252 ber” rule (e.g., when a big number is given, subtract) (ES = .82~). Woodward (2006) ex- plored the use of visuals such as a number 249. Artus and Dyrek (1989); Darch, Carnine, and Gersten (1984); Fuchs, Powell et al. (2008); Fuchs, line to help students understand what an Fuchs, Craddock et al. (2008); Fuchs, Seethaler et abstract fact such as 6 × 7 = meant. al. (2008); Fuchs et al. (2005); Jitendra et al. (1998); The study yielded a substantively impor- Butler et al. (2003); Walker and Poteet (1989); Wil- tant positive effect on mathematics facts, son and Sindelar (1991); Witzel, Mercer, and Miller and a positive and marginally significant (2003); Witzel (2005); Woodward (2006). average effect on operations. 250. Fuchs, Powell et al. (2008); Fuchs, Fuchs, Craddock et al. (2008); Fuchs, Seethaler et al. (2008); Fuchs et al. (2005). 251. For more details on WWC guidelines for substantively important effects, see the What Works Clearinghouse Procedures and Standards Fuchs et al. (2005); Walker and Poteet (1989); Handbook (WWC, 2008). Wilson and Sindelar (1991); Witzel, Mercer, and Miller (2003). 252. Jitendra et al. (1998); Butler et al. (2003); Wit- zel (2005); Darch, Carnine, and Gersten (1984); 253. Woodward (2006). Fuchs, Powell et al. (2008); Fuchs, Fuchs, Crad- 254. Jitendra et al. (1998); Walker and Poteet (1989); dock et al. (2008); Fuchs, Seethaler et al. (2008); Wilson and Sindelar (1991); Woodward (2006).

( 74 ) Appendix D. Technical information on the studies

Three studies used manipulatives in the presented to the students sequentially to early stages of instruction to reinforce un- promote scaffolded instruction.257 This derstanding of basic concepts and opera- model of instruction, with its underpin- tions.255 For example, Darch et al. (1984) ning in Bruner’s (1966) work, is referred used concrete models such as groups of to as a concrete to representation to ab- boxes to teach rules for multiplication stract (CRA) method of instruction. The problems. Similarly, Fuchs, Fuchs, Crad- CRA method is a process by which stu- dock et al. (2008) used manipulatives in dents learn through the manipulation of their tutoring sessions to target and teach concrete objects, then through visual rep- the most difficult concepts observed in resentations of the concrete objects, and the classroom. In another study, Fuchs, then by solving problems using abstract Seethaler et al. (2008) used concrete ma- notation.258 Fuchs et al. (2005) taught 1st terials and role playing to help students grade students basic math concepts (e.g., understand the underlying mathemati- place value) and operations initially using cal structure of each problem type. In all concrete objects, followed by pictorial rep- these studies, manipulatives were one as- resentations of blocks, and finally at the pect of a complex instructional package. abstract level (e.g., 2 + 3 = ) without the The studies resulted in mostly significant use of manipulatives or representations. positive domain average effect sizes in the Butler et al. (2003) examined the differen- range of .60 to 1.79. tial impact of using two types of scaffolded instruction for teaching fractions, one In six studies, both concrete and visual that initiated scaffolding at the concrete representations were used to promote level (concrete-representation-abstract) mathematical understanding.256 For exam- and the other that started at the repre- ple, Artus and Dyrek (1989) used concrete sentation level (representation-abstract). objects (toy cars, empty food wrappers) Neither variation resulted in significant and visuals (drawings) to help students differences. understand the story content, action, and operation in the word problems (ES = .87~). Witzel (2005) and Witzel, Mercer, and Miller Likewise, Fuchs, Powell et al. (2008) used (2003) investigated the effectiveness of manipulatives in the initial stages and later the scaffolded instruction using the CRA pictorial representations of ones and tens method to teach prealgebra (e.g., X – 4 = in their software program (ES = .55, n.s.). 6) to low-achieving students and students However, in both studies, concrete objects with disabilities. Using an explicit instruc- and visual representations were not part tional format, Witzel taught students ini- of an instructional format that promoted tially using manipulatives such as cups and systematic scaffolded learning. In other sticks. These were replaced with drawings of words, instruction did not include fading the same objects and finally faded to typical the manipulatives and visual representa- abstract problems using Arabic symbols (as tions to promote understanding of math seen in most textbooks and standardized at the more typical abstract level. exams). Both studies resulted in statistically significant or substantively important In the remaining four studies, manipu- positive gains (Witzel, Mercer, and Miller, latives and visual representations were 2003 and ES = .83*; Witzel, 2005 and ES = .54, n.s.). One of these studies is described in more detail here. 255. Darch et al. (1984); Fuchs, Seethaler et al. (2008); Fuchs, Fuchs, Craddock et al. (2008). 256. Artus and Dyrek (1989); Butler et al. �� 257. Butler et al. �� Fuchs, Powell et al. (2008); Fuchs et al. ��- (2005); Witzel, Mercer, and Miller (2003). zel (2005); Witzel, Mercer, and Miller (2003). 258. Witzel (2005).

( 75 ) Appendix D. Technical information on the studies

A study of CRA instruction—Witzel, The students in both groups were taught Mercer, and Miller (2003) to transform equations with single variables using a five-step 19-lesson sequence In 2003, Witzel, Mercer, and Miller published of algebra equations. In each session, a study that investigated the effects of using the teacher introduced the lesson, mod- the CRA method to teach prealgebra.259 The eled the new procedure, guided students participants in the study were teachers and through procedures, and began to have students in 12 grade 6 and 7 classrooms in students working independently. For the a southeastern urban county. Each teacher treatment group, these four steps were taught one of two math classes using CRA used for instruction at the concrete, rep- instruction (treatment group) and the other resentational, and abstract stages of each using abstract-only traditional methods concept. Teachers taught the concrete les- (traditional instruction group). Of those sons using manipulative objects such as participating, 34 students with disabili- cups and sticks, the representational les- ties260 or at risk for algebra difficulty261 in sons using drawings of the same objects, the treatment group were matched with 34 and the abstract lessons using Arabic sym- students with similar characteristics across bols. For the traditional instruction group, the same teacher’s classes in the traditional the teachers covered the same content instruction group. for the same length of time (50 minutes), but the teachers used repeated abstract lessons rather than concrete objects and pictorial representations.

259. Witzel, Mercer, and Miller (2003). A 27-item test to measure knowledge on 260. These students were identified through school single-variable equations and solving for services as those who needed additional support, had a 1.5 standard deviation discrepancy between a single variable in multiple-variable equa- ability and achievement, and had math goals listed tions was administered to the students one in their individualized education plans. week before treatment (pretest), after the 261. These students met three criteria: per- last day of the treatment (posttest), and formed below average in the classroom according three weeks after treatment ended (follow- to the teacher, scored below the 50th percentile in mathematics on the most recent statewide up). The CRA intervention had a positive achievement test, and had socioeconomically and significant effect on knowledge of the disadvantaged backgrounds. prealgebra concepts assessed.

( 76 ) Appendix D. Technical information on the studies b (continued) .95* .56* .30* .34~ .66~ . 87~ 1.01* 1.79* .55 (n.s) -14 (n.s.) .12 (n.s.) .07 (n.s.) .35 (n.s.) .56 (n.s.) .60 (n.s.) .60 -.07 (n.s.) Outcomes

Domain concepts Concepts Concepts problems problems problems Operations achievement achievement Math generalMath Math generalMath Word problems Word problems Word problems Word problems Word problems Word problems Word problems Transfer—word Transfer—word Transfer—math Transfer—math Transfer—story

ons i inutes/ ession; session; session; s session; session; session; session; session; session; Duration 6 sessions 17 sess 17 11 sessions 11 10 sessions 10 45 sessions 45 36 sessions 36 38 sessions 38 48 sessions48 45 minutes/ 30 minutes/ 30 m 40 minutes/ 90 minutes/ 17–20 sessions 17–20 15–18 minutes/ 20–30 minutes/ 20–30 minutes/ 40–45 minutes/ 3 4 1 3 3 2–5 6–8 4–6 6–8 level Grade a Comparison ut reservations) Instructional and objects intervention concrete using lecture traditional versus drawings representational format Instructional and objects intervention concrete using representational versus drawings representational drawings only Instructional intervention using concrete materials and role playing versus no instruction condition Instructional objects intervention concrete using versus traditional basal instruction Instructional objects intervention concrete using and representational drawings versus no instruction condition Instructional objects intervention concrete using and pictorial representations versus no instruction condition Instructional objects intervention concrete using versus no instruction condition Instructional intervention using diagrammatic diagrammatic using intervention Instructional representations versus traditional basal instruction diagrammatic using intervention Instructional representations versus traditional instruction

Study Artus and Dyrek (1989) Butler et al.Butler et (2003) Fuchs, Seethaler Seethaler Fuchs, al.et (2008) Darch, Carnine, and Gersten (1984) al. et Fuchs (2005) Fuchs, Powell Powell Fuchs, al.et (2008) Fuchs, Fuchs, Craddock al. et (2008) Jitendra et al. et Jitendra (1998) and Walker Poteet (1989) Table D3. Studies of interventions that used visual representations that met WWC standards WWC standards met that representations visual used that interventions of Studies D3. Table and(with witho

( 77 ) Appendix D. Technical information on the studies

b .83* .82~ .11~ .55 (n.s.) .54 (n.s.) Outcomes

Domain Concepts Concepts Math facts Operations (prealgebra) (prealgebra) Word problems

sion; ses session; session; session; Duration 14 sessions 14 19 sessions 19 19 sessions 19 20 sessions 20 25 minutes/ 50 minutes/ 50 minutes/ 30 minutes/ 4 6,7 6,7 2–4 level Grade a (continued) Comparison Instructional intervention using diagrammatic diagrammatic using intervention Instructional representations versus instruction without diagrams Instructional objects intervention concrete using and pictorial representations versus traditional instruction Instructional intervention using pictorial pictorial using intervention Instructional representations versus an intervention using not representations Instructional objects intervention concrete using and pictorial representations versus traditional instruction

value ≥ .10, the effectsignificant is not size (n.s.). value ≥ .10, -

Authors’ analysis based on studies in table. Instructional interventions in all the studies listed were multicomponent in nature, with visuals being those one of components. areOutcomes reported.10, the size aseffect is effect For a sizes. significantthe for effectp-value is < (*), size .05, a < marginallyp-value (~); significant a p for

Study Wilson and Sindelar (1991) Witzel, and Mercer, (2003) Miller Woodward (2006) Witzel (2005) Table D3. Studies of interventions that used visual representations that met WWC standards WWC standards met that representations visual used that interventions of Studies D3. Table and(with without reservations) b. a. Source:

( 78 ) Appendix D. Technical information on the studies

Recommendation 6. acknowledges that the broader implication Interventions at all grade levels that general mathematics proficiency will should devote about 10 minutes improve when fact fluency improves is the in each session to building fluent opinion of the panel. retrieval of basic arithmetic facts. Summary of evidence Level of evidence: Moderate The panel recognizes the importance of The panel judged the level of evidence knowledge of basic facts (addition, sub- supporting the recommendation to be traction, multiplication, and division) for moderate. We found seven studies con- students in kindergarten through grade 4 ducted with low-achieving or learning and beyond. Two studies examined the ef- disabled students between grades 1 and fects of teaching mathematics facts relative 4 that met WWC standards or met stan- to the effects of teaching spelling or word dards with reservations and included fact identification using similar methods.263 In fluency instruction in an intervention.262 both studies, the mathematics facts group Appendix table D4 (p. 83) provides an over- demonstrated substantively important or view of the studies and indicates whether statistically significant positive gains in fact fluency was the core content of the facts fluency relative to the comparison intervention or a component of a larger group, although the effects were significant intervention. The relevant treatment and in only one of these two studies.264 comparison groups in each study and the outcomes for each domain are included Another two interventions included a facts in the table. Grade level, typical session fluency component in combination with a length, and duration of the intervention larger tier 2 intervention.265 For example, are also in the table. in the Fuchs et al. (2005) study, the final 10 minutes of a 40-minute intervention ses- Given the number of high-quality ran- sion were dedicated to practice with addi- domized and quasi-experimental design tion and subtraction facts. In both studies, studies conducted across grade levels tier 2 interventions were compared against and diverse student populations that in- typical tier 1 classroom instruction. In each clude instruction in fact fluency as either study, the effects on mathematics facts an intervention or a component of an in- were not significant. Significant positive -ef tervention, the frequency of small but fects were detected in both studies in the substantively important or significant domain of operations, and the fact fluency positive effects on measures of fact flu- component may have been a factor in im- ency and mathematical operations (effect proving students’ operational abilities. sizes ranged from .11 to 2.21), and the fact that numerous research teams inde- Relationships among facts. The panel sug- pendently produced similar findings, the gests emphasizing relationships among panel concluded that there is moderate basic facts, and five of the studies examined evidence to support the recommendation to provide instruction in fact fluency for both tier 2 and tier 3 mathematics interventions across grade levels. The panel 263. Fuchs, Fuchs, Hamlett et al. (2006); Fuchs, Powell et al. (2008). 264. In Fuchs, Fuchs, Hamlett et al. (2006), the 262. Fuchs, Fuchs, Hamlett et al. (2006); Fuchs, effects on addition fluency were positive while Seethaler et al. (2008); Fuchs et al. (2005). Beirne- there was no effect on subtraction fluency. Smith (1991); Fuchs, Powell et al. (2008); Tournaki 265. Fuchs, Seethaler et al. (2008); Fuchs et al. (2003); Woodward (2006). (2005).

( 79 ) Appendix D. Technical information on the studies

exemplify this practice.266 In Woodward appeared, the student typed the fact. If the (2006), the Integrated Strategies group was student made an error, the correct fact was specifically taught the connection between displayed with accompanying audio, and single-digit facts (e.g., 4 × 2 = ) and ex- the student had the opportunity to type tended facts (40 × 2 = ). In Fuchs et al. the fact again. For two of the studies, the (2005, 2006c, 2008e), mathematics facts duration of the presentation on the screen are presented in number families (e.g., was tailored to the student’s performance 1 + 2 = 3 and 3 – 2 = 1). Beirne-Smith (1991) (with less time as the student gained profi- examined the effects of a counting up/on ciency) and the difficulty of facts increased procedure that highlighted the relationship as competency increased.269 between facts versus a rote memorization method that did not highlight this relation- Time. The panel advocates dedicating ship. There was a substantively important about 10 minutes a session to building fact nonsignificant positive effect in favor of fluency in addition to the time dedicated to the group that was taught the relationship tier 2 and tier 3 interventions. The seven between facts. Note that fact relationships studies supporting this recommendation were not isolated as independent variables dedicated a minimum of 10 minutes a ses- in this study. sion to fact fluency activities.

Materials to teach math facts. The stud- Explicit teaching strategies for building ies used a variety of materials to teach fact fluency. Another three studies in the mathematics facts. Woodward (2006) used evidence base address methods for teach- worksheets, number lines, and arrays of ing basic facts to students by comparing blocks projected on overheads to help stu- instructional approaches.270 Both Beirne- dents visualize fact strategies. Tournaki Smith (1991) and Tournaki (2003) investi- (2003) also used worksheets. Three studies gated the effects of being taught a counting included flash cards.267 For example, the up/on strategy relative to a rote memoriza- Fuchs, Seethaler et al. (2008) intervention tion procedure for promoting fact fluency. included flash cards with individual ad- In the Beirne-Smith (1991) study, perhaps dition and subtraction problems on each not surprisingly as both interventions card. Students had up to two minutes to re- were taught to enhance fact fluency, the spond to as many cards as they could, and addition facts competency of students in they were provided with corrective feed- both groups improved. However, there back on up to five errors each session. was a substantively important nonsignificant positive effect in favor of the count- Three studies included computer assisted ing-on group when the two groups were instruction to teach mathematics facts.268 compared. In the Tournaki (2003) study, In all three interventions, students used a the latency of responses on a fact fluency computer program designed to teach addi- posttest decreased271 while the accuracy of tion and subtraction facts. In this program, posttest responses significantly increased a mathematics fact was presented briefly on the computer screen. When the fact dis- 269. Fuchs et al. (2005); Fuchs, Fuchs, Hamlett et al. (2006). 266. Beirne-Smith (1991); Fuchs et al. (2005); 270. Beirne-Smith (1991); Tournaki (2003); Wood- Fuchs, Fuchs, Hamlett et al. (2006), Fuchs, ward (2006). Seethaler et al. (2008); Woodward (2006). 271. The latency decrease was marginally sig- 267. Beirne-Smith (1991); Fuchs, Seethaler et al. nificant. A decrease in latency indicates that stu- (2008); Fuchs, Powell et al. (2008). dents in the counting-on group were answering 268. Fuchs et al. (2005); Fuchs, Fuchs, Hamlett et fact problems more quickly than students in the al. (2006); Fuchs, Powell et al. (2008). rote memorization group.

( 80 ) Appendix D. Technical information on the studies

for the counting-on group relative to the session. Although these components of the rote memorization group. interventions were most often not independent variables in the studies, they are Similarly, Woodward (2006) examined all advocated by the panel. An example of an integrated approach that combined a study investigating the effects of a fact instruction in strategies, visual repre- fluency intervention is detailed here. sentations, and timed practice drills (the Integrated Strategies group) versus a tra- A study of a fact fluency intervention— ditional timed drill and practice approach Fuchs, Powell, Hamlett, and Fuchs (2008). for building multiplication fact fluency (the Timed-Practice Only group). In the Inte- This study was conducted with 127 stu- grated Strategies group, difficult facts were dents in grade 3 classrooms in Tennessee taught through derived fact strategies or and Texas.273 The students were all iden- doubling and doubling again strategies. tified as having either math difficulties or When WWC multiple comparison adjust- math and reading difficulties. ments were applied to outcomes, none of the multiplication fact outcomes were Before the intervention, the students com- significant, though effects were substan- pleted several pretest assessments. The tively important and positive in favor of assessment that related to fact retrieval the integrated approach.272 The operations consisted of one subtest with three sets domain showed mixed effects with approx- of 25 addition fact problems and a second imation scores in favor of the integrated subtest with three sets of 25 subtraction approach and operations scores in favor fact problems. Students had one minute to of the Timed-Practice Only group. write answers for each set within a subtest. Internal consistency for the sets ranged be- In summary, the evidence demonstrates tween .88 and .93. Scores on sets of items substantively important or statistically were combined into a single fact retrieval significant positive effects for including score. After the intervention, students fact fluency activities as either stand-alone completed this same fact retrieval assess- interventions or components of larger tier ment among a battery of posttests. 2 interventions. However, because these effects did not consistently reach statisti- Students were randomly assigned to one cal significance, the panel is cautious and of four groups (three mathematics instruc- acknowledges that the level of evidence tion groups and a reading instruction com- for this recommendation is moderate. parison group). For this recommendation, There is also evidence that strategy-based we report only on the comparison between instruction for fact fluency (e.g., teaching the Fact Retrieval group (n = 32) and the the counting-on procedure) is a superior Word Identification comparison group (n approach over rote memorization. Further, = 35).274 Students in both groups met in- many of the studies included here taught the relationships among facts, used a variety of materials such as flash cards and 273. This study met standards with reservations because of high attrition. The sample initially computer assisted instruction, and taught included 165 students randomized to the condi- math facts for a minimum of 10 minutes a tions and 127 in the postattrition sample. The authors did demonstrate baseline equivalence 272. When a study examines many outcomes or of the postattrition sample. findings simultaneously, the statistical signifi- 274. The third group was Procedural/Estimation cance of findings may be overstated. The WWC Tutoring, which targeted computation of two- makes a multiple comparison adjustment to pre- digit numbers. The fourth group was a combi- vent drawing false conclusions about the number nation of Procedural/Estimation Tutoring and of statistically significant effects (WWC, 2008). Fact Retrieval.

( 81 ) Appendix D. Technical information on the studies

dividually with a tutor275 for 15 to 18 min- as possible in two minutes. After three utes during three sessions each week for consecutive sessions with a minimum of 15 weeks. 35 correct responses, the student was presented with a second set of flash cards that Sessions for the Fact Retrieval instruction contained a number line similar to the CAI group consisted of three activities. First, number line. The student was asked to re- the students received computer assisted spond with the appropriate mathematics instruction (CAI). In the computer pro- facts to accompany the number line for gram, an addition or subtraction math- as many cards as possible within the time ematics fact appeared on the screen for frame. Corrective feedback was provided 1.3 seconds. When the fact disappeared, for a maximum of five errors per flash card the student typed the fact using short- activity. The third activity during Fact Re- term memory. A number line illustrated trieval instruction focused on cumulative the mathematics fact on the screen with review. Students were allotted two minutes colored boxes as the student typed. If the to complete 15 mathematics fact problems student typed the fact correctly, applause using paper and pencil. was heard, and the student was awarded points. Each time the student accumulated Students in the Word Identification com- five points, animated prizes (e.g., a pic- parison group received computer assisted ture of a puppy) appeared in the student’s instruction and participated in repeated animated “treasure chest.” If the student reading with corrective feedback during typed the mathematics fact incorrectly, their sessions. The content was tailored the fact reappeared and the student was to the student’s reading competency level prompted to type it again. as determined by a pretest.

The second instructional activity, flash Results indicated significant positive ef- card practice, began after 7.5 minutes of fects on fact fluency in favor of the group CAI. Flash card practice with corrective that received fact retrieval instruction feedback included two types of flash cards. relative to the comparison group that re- The first set of flash cards depicted writ- ceived instruction in word identification. ten facts without answers. Students were These results suggest that it is possible encouraged to answer as many problems to teach struggling students mathematics facts in as small an amount of time as 45 275. There were 22 tutors. Some were masters minutes of instruction a week when using or doctoral students. Most had teaching or tutor- flash cards and CAI. ing experience.

( 82 ) Appendix D. Technical information on the studies

.11~ .68* .40* .60* .71~ 1.10* 2.21* .19 (n.s.) .55 (n.s.) .33 (n.s.) .26 (n.s.) .26 .34 (n.s.) Effect size Outcomes Domain Transfer Operations Operations Operations Operations Operations Fact fluency Fact fluency Fact fluency Fact fluency Fact fluency Fact fluency

16 weeks 15 weeks 18 weeks 12 weeks Duration 8 sessions over 3 sessions/week; 3 sessions/week; (2 minutes on math on minutes (2 40 minutes/session 40 minutes/session 20–minutes/session 20–minutes/session 10 minutes/session; minutes/session; 10 15 minutes/session, minutes/session, 15 25 minutes/session, minutes/session, 25 30 minutes/session; minutes/session; 30 (10 minutes on math on minutes (10 facts); 3 times/week; facts); facts); 3 times/week; facts); 5 times/week; 4 weeks 15–18 minutes/session; minutes/session; 15–18 consecutive school days 5 sessions/week; 4 weeks 4 2 3 1 1 3 1–5 level Grade a Component Component Intervention Intervention Intervention Intervention Intervention component Intervention or - - a Comparison Strategies for learning for Strategies facts timed plus math fact drills versus timed math fact drills only Counting-on strategy learning for math facts versus drill and practice math of facts Intervention in math facts versus nonrelevant instruction Intervention in math facts versus relevant non instruction Multicomponent intervention (with one com ponent on math facts) versus no instruction math on ponent facts) condition Counting on methodCounting on learning for facts versus memorizationrote facts of Multicomponent intervention (with one com ponent on math facts) versus no instruction math on ponent facts) condition

: Authors’ analysis based on studies in table. 003) Intervention means that the entire intervention was on math facts. Component means that math facts were one part just the of intervention. for a p-value ≥ .10, thesignificant effect is not size a p-value(n.s.). for ≥ .10, Outcomes areOutcomes reported.10, the as size effect effectis For a sizes. significant the p-value for effect is < (*); .05, size a < marginallyp-value significant (~);

Tournaki Woodward (2006) (2 Fuchs, Powell Powell Fuchs, al.et (2008) Fuchs, Fuchs, Hamlett al. et (2006) al. et Fuchs (2005) Study Beirne-Smith (1991) Fuchs, Seethaler Seethaler Fuchs, al.et (2008) b. a. Source Table D4. Studies of interventions that included fact fluency practices that met WWC standards standards met WWC that practices fluency fact included that interventions of Studies D4. Table and(with without reservations)

( 83 ) Appendix D. Technical information on the studies

Recommendation 7. Monitor the this second body of research has been con- progress of students receiving ducted primarily in special education set- supplemental instruction and other tings and therefore is less relevant to the students who are at risk. RtI focus of this practice guide. As a result, we focus on the technical adequacy studies Level of evidence: Low in this appendix. Note that because similar and often identical measures are used for The panel rated the level of evidence for screening and progress monitoring, many this recommendation as low. The panel of the studies reviewed here overlap with relied on the standards of the American those discussed for recommendation 1 Psychological Association, the American on screening. The same measure may be Educational Research Association, and the used as both a screening measure and a National Council on Measurement Educa- progress monitoring measure; however, tion276 for valid assessment instruments, the psychometric properties of these mea- along with expert judgment, to evaluate sures are more firmly established when the quality of the individual studies and used as screening measures with fewer to determine the overall level of evidence researchers investigating the function of for this recommendation. the measures for modeling growth when used for progress monitoring. This dispar- Evidence for the recommendation included ity in the research base leads to the panel research studies on mathematics progress assigning a moderate level of evidence to monitoring,277 summary reviews of math- Recommendation 1 and a low level of evi- ematics progress monitoring research,278 dence to Recommendation 7. and summary information provided by the Research Institute on Progress Monitor- The technical adequacy studies of math- ing279 and the National Center on Progress ematics progress monitoring measures Monitoring.280 Very little research evidence were not experimental; the researchers specifically addresses the use of math- typically used correlational techniques to ematics progress monitoring data within evaluate the reliability and criterion valid- the context of RtI. ity of the measures and regression methods to examine sensitivity to growth. If Most research on mathematics progress progress monitoring measures are to be monitoring measures falls into two cat- deemed trustworthy, relevant empiri- egories. One group of studies examines cal evidence includes data on reliability, the technical adequacy of the measures, concurrent criterion validity, and sensi- including their reliability, validity, and tivity to growth. Evidence of reliability sensitivity to growth. The second investi- generally includes data on inter-scorer gates teacher use of the measures to mod- agreement,281 internal consistency,282 test- ify instruction for individual students in retest reliability,283 and alternate form order to enhance achievement; the bulk of reliability.284 Evidence of concurrent criterion validity is gathered by examining relations between scores on the progress 276. American Educational Research Association, monitoring measures and other indica- American Psychological Association, and National Council on Measurement in Education (1999). 277. Clarke et al. (2008); Foegen and Deno (2001); 281. For example, Fuchs, Fuchs, Hamlett, Thomp- Fuchs et al. (1993); Fuchs, Fuchs, Hamlett, Thompson son et al. (1994). et al. (1994); Leh et al. (2007); Lembke et al. (2008). 282. For example, Jitendra, Sczesniak, and Deat- 278. Foegen, Jiban, and Deno (2007). line-Buchman (2005). 279. www.progressmonitoring.net/. 283. For example, Clarke and Shinn (2004). 280. www.studentprogress.org/. 284. VanDerHeyden et al. (2001).

( 84 ) Appendix D. Technical information on the studies tors of proficiency in mathematics. Com- Summary of evidence mon criterion measures include scores on group and individual standardized Progress monitoring in the primary grades. tests of mathematics, course grades, and Measures for the primary grades typically teacher ratings.285 reflect aspects of number sense, including strategic counting, numeral identification, Although issues of reliability and criterion and magnitude comparison. Among the validity are common to both screening and studies examining sensitivity to growth progress monitoring measures, a critical in the early grades, researchers have re- feature specific to progress monitoring is lied on separate measures for each of the that the measures be sensitive to growth. different aspects of numeracy.286 Other If teachers are to use progress monitor- researchers have combined individual ing measures to evaluate the effects of in- measures to create composite scores287 struction on student learning, researchers or used more comprehensive multiskill must provide evidence that student scores measures.288 But so far, the focus of these on the measures change over time, thus studies has been on screening rather than providing an indication of their learning. on progress monitoring. Reliability coef- Most research studies have examined sen- ficients for these measures generally ex- sitivity to growth by administering paral- ceed .85. Concurrent criterion validity co- lel forms of a measure across a period of efficients with standardized achievement several weeks or months. In some studies, tests are generally in the .5 to .7 range.289 students receive typical instruction, and Mean rates of weekly growth reported in in others, teachers adapt and refine the the literature vary widely, ranging from .1 instruction in response to the progress to .3290 problems a week to .2 to more than monitoring data (often in special educa- 1.0 problems.291 tion contexts). In either case, evidence of sensitivity to growth typically involves Progress monitoring in the elementary computing regression equations to deter- grades. Two types of measures have been mine slopes of improvement and report- investigated for monitoring the mathemat- ing these as mean weekly growth rates ics learning of students in the elementary for a group of students. As an example, grades. The bulk of the research, con- if a progress monitoring measure has a ducted by a research group led by Dr. Lynn mean weekly growth rate of .5, teachers Fuchs, investigates the characteristics of could expect that, on average, a student’s general outcome measures that represent score would increase by 1 point every two grade-level mathematics curricula in com- weeks. Growth rates reported in the litera- putation and in mathematics concepts and ture vary considerably across measures applications.292 These measures were de- and grade levels; no established stan- veloped in the late 1980s and early 1990s, dards exist for acceptable rates of student growth under typical instruction. 286. For example, Clarke et al. (2008); Lembke et al. (2008). We discuss the evidence for measures 287. Bryant, Bryant, Gersten, Scammacca, and used across the elementary and middle Chavez (2008). school grades and conclude with a more 288. Fuchs, Fuchs, Compton et al. (2007). in-depth example of a technical adequacy 289. Chard et al. (2005); Clarke and Shinn (2004); study of mathematics progress monitor- Clarke et al. (2008); Lembke et al. (2008). ing measures. 290. Lembke et al. (2008). 291. Chard et al. (2005). 285. For example, Foegen and Deno (2001); Fuchs 292. Fuchs and Fuchs (1998); Fuchs, Hamlett, et al. (2003a); Chard et al. (2005). and Fuchs (1998).

( 85 ) Appendix D. Technical information on the studies

reflecting the Tennessee state elementary facts. In contrast, Fuchs, Hamlett, and Fuchs mathematics curriculum of that time. The (1998) found correlations between the Stan- measures continue to be widely used and ford Achievement Test Math Computation are recommended by the National Center subtest297 and general outcome measures for Student Progress Monitoring.293 Teach- of computation to range from .5 to .9 across ers should carefully examine the content grades 2 through 5. In general, concurrent of the measures to ensure that they are criterion validity coefficients for elementary representative of the existing mathematics mathematics progress monitoring measures curricula in their states and districts. are in the .5 to .6 range.

The second type of measure is not broadly Evidence of sensitivity to growth for el- representative of the instructional curricu- ementary measures exists for the com- lum as a whole, but instead serves as an putation and concepts/applications mea- indicator of general proficiency in math- sures developed by Fuchs and for the word ematics. Examples of such measures in- problem-solving measures developed by clude basic facts (number combinations)294 Jitendra. Mean growth rates for the Fuchs and word problem solving.295 Because the measures range from .25 to .70. A study by general outcome measures are representa- Shapiro and colleagues,298 using the same tive of the broader curriculum, they offer measures for students with disabilities, re- teachers more diagnostic information sulted in mean growth rates of .38 points about student performance in multiple per week for both types of measures. Mean aspects of mathematics competence; this weekly growth rates for the Jitendra mea- advantage is often gained by using measures were .24 points per week. sures that are longer and require more administration time. The indicator measures Progress monitoring in middle school. Less are more efficient to administer for regular evidence is available to support progress progress monitoring but may be as useful monitoring in middle school.299 Research for diagnostic purposes. teams have developed measures focusing on math concepts typically taught in Evidence of the reliability of the measures middle school,300 basic facts301 and esti- is generally strong, with correlation co- mation.302 Criterion validity across the efficients above .8, except for the word types of measures varies, but the majority problem-solving measures developed by of correlations coefficients fall in the .4 to Jitendra’s research team, which are slightly .5 range. Helwig and colleagues303 found lower.296 Concurrent criterion validity mea- higher correlation coefficients with high- sures have included group and individual stakes state tests in the range of .6 to .8. achievement tests. Validity correlation coef- Reliability estimates including alternate ficients range widely across measure types form, inter-rater, and test-retest were all of and grade levels. At the lower end, Espin et sufficient quality. Greater rates of growth al. (1989) found correlations between the were found for the non–concept-based Wide Range Achievement Test and basic measures with rates around .25 units per fact measures in the .3 to .5 range for basic 297. Gardner, Rudman, Karlsen, and Merwin (1982). 293. www.studentprogress.org. 298. Shapiro, Edwards, and Zigmond (2005). 294. Espin et al. (1989); VanDerHeyden, Witt, and 299. Foegen (2008). Naquin (2003). 300. Helwig, Anderson, and Tindal (2002). 295. Jitendra, Sczesniak, and Deatline-Buchman (2005); Leh et al. (2007). 301. Espin et al. (1989). 296. Jitendra, Sczesniak, and Deatline-Buchman 302. Foegen and Deno (2001); Foegen (2000). (2005); Leh et al. (2007). 303. Helwig, Anderson, and Tindal (2002).

( 86 ) Appendix D. Technical information on the studies

week. A recent study304 compared these teachers administered the measures using measure types with two grade 6 mea- standardized procedures, including admin- sures305 similar to the measures described istration time limits of 6 to 8 minutes, de- above assessing student understanding of pending on grade level. operations and concepts for their grade level. In this case, middle school students The results of the study illustrate the types in grades 6, 7, and 8 were assessed using of data educators should consider to deter- multiple measures. Evidence was found mine if a mathematics progress monitor- that even into grades 7 and 8, using grade ing measure is trustworthy. The authors 6 measures focusing on operations and report evidence of the reliability of the mathematical concepts still shows reliabil- concepts and application measures by de- ity, validity, and sensitivity to growth. scribing internal consistency coefficients for students at each grade level (which An example of a study of the technical ranged from .94 to .98). Concurrent crite- adequacy of mathematics progress rion validity was examined by computing monitoring measures—Fuchs, Fuchs, correlations between student scores on Hamlett, Thompson, Roberts, Kupek, the concepts and applications general out- and Stecker (1994) come measures and their scores on three subscales of the Comprehensive Test of A study conducted by Fuchs, Fuchs, Ham- Basic Skills (Computation, Concepts and lett, Thompson, Roberts, Kupek, and Applications, and Total Math Battery). Stecker (1994) illustrates the type of tech- Results are reported for each subscale at nical adequacy evidence that education each of the three grade levels, with coef- professionals can use when evaluating and ficients ranging from .63 to .81. Consider- selecting mathematics progress monitoring these results in the general context of ing measures. The research team exam- mathematics progress monitoring mea- ined the technical features of grade-level sures summarized above, teachers could general outcome measures of mathematics feel confident that the concepts and ap- concepts and applications. The measures plications measures demonstrated strong were developed by analyzing the Tennes- levels of reliability and criterion validity see mathematics curriculum at grades 2 in this study. through 6 to identify critical objectives essential for mathematics proficiency at A final consideration is the degree to which each grade level. The researchers created the measures are sensitive to student 30 alternate forms at each grade level and growth. To explore this feature, the re- conducted numerous pilot tests to refine searchers completed a least-squares re- the items and determine appropriate time gression analysis between calendar days limits for administration. and scores on the progress monitoring measures; the scores were then converted A total of 140 students in grades 2 through 4 to represent weekly rates of improvement. participated in the study, completing weekly The results ranged from an average in- versions of the measures for 20 weeks. All crease of .40 problems per week in grade students were enrolled in general education 2 to .69 in grade 4. Together with the evi- classrooms; about 8 percent of the students dence of reliability and criterion validity, had been identified as having learning dis- the mean growth rate data suggest that abilities. The students’ general education teachers can have confidence that students will show improvements in their scores on 304. Foegen (2008). the measures as their mathematics learn- 305. Fuchs, Hamlett, and Fuchs (1998); Fuchs ing progresses. et al. (1999).

( 87 ) Appendix D. Technical information on the studies

One factor not evident in the technical ad- and on the limited evidence base.307 The equacy data in this study, but critical for evidence base is described below. teachers to consider, is the alignment between the general outcome measure, the Summary of evidence instructional curriculum, and expected learning outcomes. This study produced Reinforce effort. The panel advocates re- strong technical adequacy data when it inforcing or praising students for their ef- was conducted in the early 1990s. Teach- fort. Schunk and Cox (1986) examined the ers considering alternative mathematics effects of providing effort-attributional progress monitoring measures to rep- feedback (e.g., “You’ve been working hard”) resent the instructional curriculum are during subtraction instruction versus no advised to review the content of these effort feedback and found significant posi- measures in light of current learning ex- tive effects on subtraction posttests in pectations for students at each grade level. favor of providing effort feedback. This Given changes in mathematics curricula study, described in greater detail below, over the past 10 to 15 years, it is impor- was one of two studies in the evidence tant to evaluate the degree to which the base that examined a motivational strat- measures continue to represent important egy as an independent variable. mathematics outcomes. Reinforce engagement. The panel also rec- Recommendation 8. Include ommends reinforcing students for attending motivational strategies in tier 2 to and being engaged in lessons. In two of and tier 3 interventions. the studies, students received “points” for engagement and attentiveness as well as for Level of evidence: Low accuracy.308 Accumulated points could be applied toward “purchasing” tangible rein- The panel judged the level of evidence sup- forcers. It is not possible to isolate the effects porting this recommendation to be low. The of reinforcing attentiveness in the studies. panel found nine studies306 conducted with In Fuchs et al. (2005), both the treatment low-achieving or learning disabled students and comparison groups received reinforce- between grades 1 and 8 that met WWC stan- ment, and in Fuchs, Fuchs, Craddock et al. dards or met standards with reservations (2008), the contrast between reinforcement and included motivational strategies in an and no reinforcement was not reported. But intervention. However, because only two of the presence of reinforcers for attention and these studies investigated a motivational engagement in these two studies echoes the strategy in a tier 2 or tier 3 mathematics in- panel’s contention that providing reinforce- tervention as an independent variable, the ment for attention is particularly important panel concluded that there is low evidence for students who are struggling. to support the recommendation. The panel recommends this practice for students in tier 2 and tier 3 based both on our opinion 307. The scope of this practice guide limited the evidence base for this recommendation to studies that investigated mathematics interventions for students with mathematics difficulties and included motivational components. There is an extensive literature on motivational strategies 306. Fuchs et al. (2005); Fuchs, Fuchs, Craddock outside the scope of this practice guide, and the et al. (2008); Fuchs, Seethaler et al. (2008); Artus panel acknowledges that there is considerable and Dyrek (1989); Fuchs et al. (2003b); Fuchs, debate in that literature on the use of rewards Fuchs, Hamlett, Phillips et al. (1994); Fuchs, as reinforcers. Fuchs, Finelli et al. (2006); Heller and Fantuzzo 308. Fuchs et al. (2005); Fuchs, Fuchs, Craddock (1993); Schunk and Cox (1986). et al. (2008).

( 88 ) Appendix D. Technical information on the studies

Consider rewarding accomplishments. The graphed their progress and in some cases panel recommends that interventionists set goals for improvement on future as- consider using rewards to acknowledge sessments.310 One experimental study accurate work and possibly notifying par- examined the effects of student graphing ents when students demonstrate gains. In and goal setting as an independent vari- three of the studies, students were pro- able and found substantively important vided prizes as tangible reinforcers for ac- positive nonsignificant effects in favor of curate mathematics problem solving.309 In students who graphed and set goals.311 In both Fuchs et al. (2005) and Fuchs, Seetha- two studies, the interventions included ler et al. (2008), students in tier 2 tutoring graphing in both groups being compared; earned prizes for accuracy. In both stud- therefore, it was not possible to isolate the ies, the tier 2 intervention group demon- effects of this practice.312 In another two strated substantively important positive studies, students in the treatment groups and sometimes significant gains relative to graphed their progress as one component the students who remained in tier 1. But it of multicomponent interventions.313 Al- is not possible to isolate the effects of the though it is not possible to discern the ef- reinforcers from the provision of tier 2 fect of graphing alone, in Artus and Dyrek tutoring. In Fuchs, Fuchs, Craddock et al. (1989), the treatment group made margin- (2008), the authors note that the provision ally significant gains over the comparison of “dollars” that could be exchanged for group on a general mathematics assess- prizes was more effective than rewarding ment, and in Fuchs, Seethaler et al. (2008), students with stickers alone. Because this there were substantively important posi- was not the primary purpose of the study, tive non-significant effects on fact retrieval the reporting of the evidence for that find- in favor of the treatment group. ing was not complete; therefore, a WWC review was not conducted for that finding. In summary, the evidence base for motivational components in studies of students In a fourth study, Heller and Fantuzzo struggling with mathematics is limited. (1993) examined the impacts of a parental One study that met evidence standards involvement supplement to a mathematics demonstrated benefits for praising strug- intervention. The parental involvement gling students for their effort. Other stud- component included parents providing ies included reinforcement for attention, rewards for student success as well as pa- engagement, and accuracy. Because the ef- rental involvement in the classroom. The fects of these practices were not examined performance of students who received as independent variables, no inferences the parental involvement component in can be drawn about effectiveness based on addition to the school-based intervention these studies. Because this recommenda- significantly exceeded the performance of tion is based primarily on the opinion of students in only the school-based inter- the panel, the level of evidence is identi- vention. Because the parental involvement fied as low. component was multifaceted, it is not possible to attribute the statistically significant positive effects to rewards alone. 310. �� et al. (2008); Fuchs et al. (2003b); Fuchs, Fuchs, Hamlett, Phillips et al. (1994); Fuchs, Fuchs, Fi- Allow students to chart their progress and nelli et al. (2006). to set goals for improvement. Five studies 311. Fuchs et al. (2003b). included interventions in which students 312. Fuchs, Fuchs, Hamlett, Phillips et al. (1994); Fuchs, Fuchs, Finelli et al. (2006). 309. Fuchs et al. (2005); Fuchs, Seethaler et al. 313. Artus and Dyrek (1989); Fuchs, Seethaler (2008); Fuchs, Fuchs, Craddock et al. (2008). et al. (2008).

( 89 ) Appendix D. Technical information on the studies

An example of a study that investigated we report only on the comparison be- a motivational component—Schunk tween three groups. One group (n = 30) and Cox (1986). received effort feedback in addition to performance feedback during the first three This study was conducted with 90 stu- sessions. Another group (n = 30) received dents in grades 6 through 8 classrooms effort feedback in addition to performance in six schools in Texas. The mean age of feedback during the last three sessions. the students was 13 years and 7 months, A third group (n = 30) did not receive ef- and all students were identified as having fort feedback (received only performance learning disabilities in mathematics. feedback).314 Effort feedback consisted of the proctor commenting to the student, Before the intervention, the students com- “You’ve been working hard.” Students in pleted a pretest assessment that consisted both effort feedback groups received 15 of 25 subtraction problems that required statements of effort feedback across the regrouping operations. After the interven- entire intervention. tion, a similar assessment of 25 subtraction problems was completed as a post- Results indicated significant positive ef- test. A separate reliability assessment fects for effort feedback relative to the demonstrated that the two forms of the comparison group regardless of when the subtraction assessment were highly cor- student received the effort feedback. These related (r = .82). results suggest that effort feedback is ben- eficial for learning disabled students who Students were stratified by gender and may not otherwise recognize the causal school and then randomly assigned to one link between effort and outcomes. of nine experimental groups. In all groups, the students received instruction for solv- 314. Other group distinctions were related ing subtraction problems in 45-minute to student verbalization and are described in sessions conducted over six consecutive the discussion of recommendation 3 (explicit school days. For this recommendation, instruction).

( 90 ) References Beirne-Smith, M. (1991). Peer tutoring in arithmetic for children with learning American Educational Research Associa- disabilities. Exceptional Children, 57 tion, American Psychological Associa- (4), 330–337. tion, & National Council on Measurement Bloom, B. S. (1980). All our children learn- in Education. (1999). The standards for ing. New York: McGraw-Hill. educational and psychological testing. Bruner, J. (1966). Toward a theory of in- Washington, DC: American Educational struction. Cambridge, MA: Belknap Research Association Publications. Press of Harvard University Press. American Psychological Association. Bryant, B. R., Bryant, D. P., Kethley, C., Kim, (2001). Publication manual of the Ameri- S. A., Pool, C., & Seo, Y. (2008). Prevent- can Psychological Association (5th ed.). ing mathematics difficulties in the pri- Washington, DC: Author. mary grades: The critical features of American Psychological Association. instruction in textbooks as part of the (2002). Criteria for practice guideline equation. Learning Disability Quarterly, development and evaluation. Amer- 31(1), 21. ican Psychologist, 57(December), Bryant, D. P., & Bryant, B. R. (2005). Early 1058–1061. mathematics identification and interven- Artus, L. M., & Dyrek, M. (1989). The ef- tion: Working with students at-risk for fects of multiple strategy intervention mathematics difficulties. [Power Point on achievement in mathematics. Un- presentation]. University of Texas Sys- published master’s thesis, Saint Xavier tem/TEA. Retrieved January 2007 from College, Chicago. www.k8accesscenter.org/documents/ Baker, S. K., Gersten, R., Dimino, J., & Grif- SERP-Math.DCAIRppt.ppt fiths, R. (2004). The sustained use of Bryant, D. P., Bryant, B. R., Gersten, R., research-based instructional practice: Scammacca, N., & Chavez, M. M. (2008). A case study of peer-assisted learning Mathematics intervention for first- and strategies in mathematics. Remedial second-grade students with mathemat- and Special Education, 25(1), 5–24. ics difficulties: The effects of tier 2 in- Baker, S., Gersten, R., Flojo, J., Katz, R., tervention delivered as booster lessons. Chard, D., & Clarke, B. (2002). Prevent- Remedial and Special Education, 29(1), ing mathematics difficulties in young 20–32. children: Focus on effective screening of Bryant, D. P., Bryant, B. R., Gersten, R. M., early number sense delays (Technical Scammacca, N. N., Funk, C., Winter, Report No. 0305). Eugene, OR: Pacific A., et al. (2008). The effects of tier 2 Institutes for Research. intervention on the mathematics per- Bangert-Drowns, R. L., Kulik, C. C., Kulik, formance of first-grade students who J. A., & Morgan, M. (1991). The instruc- are at risk for mathematics difficulties. tional effects of feedback in test-like Learning Disability Quarterly, 31(2), events. Review of Educational Research, 47–63. 61(2), 213–238. Bryant, D. P., Smith, D. D., & Bryant, B. R. Beckmann, S., Fuson, K., SanGiovanni, J., (2008). Teaching students with special & Lott Adams, T. (Eds.), (2009). Focus in needs in inclusive classrooms. Boston, grade 5: Teaching with curriculum focal MA: Allyn & Bacon. points. Reston, VA: National Council of Butler, F. M., Miller, S. P., Crehan, K., Bab- Teachers of Mathematics. bitt, B., & Pierce, T. (2003). Fraction in- Berch, D. B. (2005). Making sense of number struction for students with mathemat- sense. Journal of Learning Disabilities, ics disabilities: Comparing two teaching 38(4), 333–339. sequences. Learning Disabilities Re- search & Practice, 18(20), 99 –111.

( 91 ) References

Carnine, D., Jitendra, A., & Silbert, J. (1997). from http://ies.ed.gov/ncee/wwc/ A descriptive analysis of mathematics publications/practiceguides. curricular materials from a pedagogi- Espin, C. A., Deno, S. L., Maruyama, G., & cal perspective. Remedial and Special Cohen, C. (1989, April). The basic aca- Education, 18(2), 66–81. demic skills sample (BASS): An instrument Chard, D. J., Clarke, B., Baker, S., Otter- for the screening and identification of stedt, J., Braun, D., & Katz, R. (2005). children at risk for failure in regular edu- Using measures of number sense to cation classrooms. Paper presented at an- screen for difficulties in mathematics: nual meeting of American Educational Re- Preliminary findings. Assessment for Ef- search Association, San Francisco, CA. fective Intervention, 30(2), 3–14. Field, M., & Lohr, K. (Eds.). (1990). Clini- Clarke, B., Baker, S., Smolkowski, K., & cal practice guidelines: Directions for a Chard, D. J. (2008). An analysis of early new program. Washington, DC: National numeracy curriculum-based measure- Academies Press. ment: Examining the role of growth in Foegen, A. (2000). Technical adequacy of student outcomes. Remedial and Special general outcome measures for mid- Education, 29(1), 46 –57. dle school mathematics. Diagnostique, Clarke, B., & Shinn, M. R. (2004). A prelimi- 25(3), 175–203. nary investigation into the identification Foegen, A. (2008). Progress monitoring in and development of early mathematics middle school mathematics: options and curriculum-based measurement. School issues. Remedial and Special Education, Psychology Review, 33(2), 234–248. 29(4), 195 –207. Compton, D., Fuchs, L., & Fuchs, D. (2007). Foegen, A., & Deno, S. L. (2001). Identify- The course of reading disability in first ing growth indicators for low-achieving grade: Latent class trajectories and early students in middle school mathematics. predictors. Unpublished manuscript. Journal of Special Education, 35(1), 4–16. Darch, C. (1989). Comprehension instruc- Foegen, A., Jiban, C., & Deno, S. (2007). Prog- tion for high school learning disabled ress monitoring measures in mathemat- students. Research in Rural Education, ics: A review of the literature. The Jour- 5(3), 43–49. nal of Special Education, 41(2), 121–139. Darch, C., Carnine, D., & Gersten, R. (1984). Fuchs, D., Compton, D. L., & Fuchs, L. Explicit instruction in mathematics (2007). Innovations in identifying learn- problem solving. Journal of Educational ing disabilities using responsiveness-to- Research, 77(6), 351–359. instruction. Unpublished manuscript. Dehaene, S. (1999). The number sense: How Fuchs, L. S., Compton, D. L., Fuchs, D., the mind creates mathematics. New Paulsen, K., Bryant, J. D., & Hamlett, C. L. York: Oxford University Press. (2005). The prevention, identification, Deno, S. (1985). Curriculum-based mea- and cognitive determinants of math dif- surement: The emerging alternative. ficulty. Journal of Educational Psychol- Exceptional Children, 52(3), 219–232. ogy, 97(3), 493–513. Epstein, M., Atkins, M., Cullinan, D., Fuchs, L. S., & Fuchs, D. (1998). Treatment Kutash, K., and Weaver, R. (2008). Re- validity: A unifying concept for recon- ducing Behavior Problems in the Elemen- ceptualizing the identification of learn- tary School Classroom: A Practice Guide ing disabilities. Learning Disabilities Re- (NCEE #2008-012). Washington, DC: Na- search & Practice, 13(4), 204–219. tional Center for Education Evaluation Fuchs, L. S., & Fuchs, D. (2008). Best prac- and Regional Assistance, Institute of tices in progress monitoring reading Education Sciences, U.S. Department and mathematics at the elementary of Education. Retrieved January 2009 grades. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology (5th

( 92 ) References

ed.). Bethesda, MD: National Association Fuchs, L. S., Fuchs, D., Hamlett, C. L., Pow- of School Psychologists. ell, S. R., Capizzi, A. M., & Seethaler, P. M. Fuchs, L. S., Fuchs, D. L., Compton, D. L., (2006). The effects of computer-assisted Bryant, J. D., Hamlett, C. L., & Seetha- instruction on number combination skill ler, P. M. (2007). Mathematics screening in at-risk first graders. Journal of Learn- and progress monitoring at first grade: ing Disabilities, 39(5), 467–475. Implications for responsiveness to in- Fuchs, L. S., Fuchs, D., Hamlett, C. L., Thomp- tervention. Exceptional Children, 73(3), son, A., Roberts, P. H., & Kupek, P., et al. 311–330. (1994). Formative evaluation of academic Fuchs, L. S., Fuchs, D., Compton, D. L., Pow- progress: How much growth can we ex- ell, S. R., Seethaler, P. M., Capizzi, A. M., pect? Diagnostique, 19(4), 23–49. et al. (2006). The cognitive correlates Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, of third-grade skill in arithmetic, algo- L., & Germann, G. (1993). Formative rithmic computation, and arithmetic evaluation of academic progress: How word problems. Journal of Educational much growth can we expect? School Psychology, 98, 29–43. Psychology Review, 22(1), 27–48. Fuchs, L. S., Fuchs, D., Craddock, C., Hol- Fuchs, L. S., Fuchs, D., & Hollenbeck, K. N. lenbeck, K. N., & Hamlett, C. L. (2008). (2007). Extending responsiveness to in- Effects of small-group tutoring with and tervention to mathematics at first and without validated classroom instruc- third grades. Learning Disabilities Re- tion on at-risk students’ math problem search & Practice, 22(1), 13–24. solving: Are two tiers of prevention Fuchs, L. S., Fuchs, D., & Karns, K. (2001). better than one? Journal of Educational Enhancing kindergartners’ mathemati- Psychology, 100(3), 491–509. cal development: Effects of peer-assisted Fuchs, L. S., Fuchs, D., Finelli, R., Courey, learning strategies. Elementary School S. J., & Hamlett, C. L. (2004). Expanding Journal, 101(5), 495–510. schema-based transfer instruction to Fuchs, L. S., Fuchs, D., Karns, K., Hamlett, help third graders solve real-life mathe- C. L., Katzaroff, M., & Dutka, S. (1997). matical problems. American Educational Effects of task-focused goals on low- Research Journal, 41(2), 419–445. achieving students with and without Fuchs, L. S., Fuchs, D., Finelli, R., Courey, learning disabilities. American Educa- S. J., Hamlett, C. L., Sones, E. M., et al. tional Research Journal, 34(3), 513–543. (2006). Teaching third graders about Fuchs, L. S., Fuchs, D., Karns, K., Hamlett, real-life mathematical problem solving: C., Katzaroff, M., & Dutka, S. (1998). A randomized controlled study. Elemen- Comparisons among individual and tary School Journal, 106(4), 293–312. cooperative performance assessments Fuchs, L. S., Fuchs, D., Hamlett, C. L., & and other measures of mathematics Karns, K. (1998). High achieving stu- competence. Elementary School Journal, dents’ interactions and performance 99(1), 23–51. on complex mathematical tasks as a Fuchs, L. S., Fuchs, D., Phillips, N. B., Ham- function of homogeneous and hetero- lett, C. L., & Karns, K. (1995). Acquisition geneous pairings. American Educational and transfer effects of classwide peer- Research Journal, 35(2), 227–267. assisted learning strategies in mathe- Fuchs, L. S., Fuchs, D., Hamlett, C. L., Phil- matics for students with varying learn- lips, N. B., & Bentz, J. L. (1994). Class- ing histories. School Psychology Review, wide curriculum-based measurement: 24(4), 604–620. Helping general educators meet the Fuchs, L. S., Fuchs, D., Prentice, K., Burch, challenge of student diversity. Excep- M., Hamlett, C. L., Owen, R., et al. (2003a). tional Children, 60(6), 518 –537. Explicitly teaching for transfer: Effects on third-grade students’ mathematical

( 93 ) References

problem solving. Journal of Educational Gardner, E. F., Rudman, H. C., Karlsen, Psychology, 95(2), 293–305. G. A., & Merwin, J. C., (1982). Stanford Fuchs, L. S., Fuchs, D., Prentice, K., Burch, M., achievement test (17th edition). Cleve- Hamlett, C. L., Owen, R., et al. (2003b). En- land, OH: Psychological Corporation. hancing third-grade students’ mathemat- Geary, D. C. (2003). Learning disabilities in ical problem solving with self-regulated arithmetic: Problem-solving differences learning strategies. Journal of Educa- and cognitive deficits. In H. L. Swanson, tional Psychology, 95(2), 306–315. K. R. Harris, & S. Graham (Eds.), Hand- Fuchs, L. S., Fuchs, D., Prentice, K., Hamlett, book of learning disabilities (pp. 199– C. L., Finelli, R., & Courey, S. J. (2004). 212). New York: Guildford Press. Enhancing mathematical problem solv- Geary, D. C. (2004). Mathematics and learning among third-grade students with ing disabilities. Journal of Learning Dis- schema-based instruction. Journal of abilities, 37(1), 4–15. Educational Psychology, 96(4), 635 – 6 47. Gersten, R., & Chard, D. (1999). Number Fuchs, L. S., Fuchs, D., Stuebing, K., Fletcher, sense: Rethinking arithmetic instruc- J. M., Hamlett, C. L., & Lambert, W. (2008). tion for students with mathematical Problem solving and computational skill: disabilities. The Journal of Special Edu- Are they shared or distinct aspects of cation, 33(1), 18–28. mathematical cognition? Journal of Edu- Gersten, R., Clarke, B. S., & Jordan, N. C. cational Psychology, 100(1), 30 – 47. (2007). Screening for mathematics dif- Fuchs, D., Fuchs, L. S., & Vaughn, S. (Eds.). ficulties in K–3 students. Portsmouth, (2008). Response to intervention. Newark, NH: RMC Research Corporation, Center DE: International Reading Association. on Instruction. Fuchs, L. S., Fuchs, D., & Zumeta, R. O. Gersten, R., Jordan, N. C., & Flojo, J. R. (2008). A curricular-sampling approach (2005). Early identification and inter- to progress monitoring: Mathemat- ventions for students with mathematics ics concepts and applications. Assess- difficulties. Journal of Learning Disabili- ment for Effective Intervention, 33(4), ties, 38(4), 293–304. 225–233. Gersten, R., Keating, T., & Irvin, L. (1995). Fuchs, L. S., Hamlett, C. L., & Fuchs, D. The burden of proof: Validity as im- (1998). Monitoring basic skills progress: provement of instructional practice. Basic math computation (2nd ed.). Aus- Exceptional Children, 61, 510–519. tin, TX: PRO-ED. Goldman, S. R., Pellegrino, J. W., & Mertz, Fuchs, L. S., Hamlett, C. L., & Fuchs, D. D. L. (1988). Extended practice of basic (1998). Monitoring basic skills progress: addition facts: Strategy changes in Basic math concepts and applications. learning disabled students. Cognition Austin, TX: PRO-ED. and Instruction, 5(3), 223–265. Fuchs, L. S., Powell, S. R., Hamlett, C. L., & Guskey, T. R. (1984). The influence of Fuchs, D. (2008). Remediating computa- change in instructional effectiveness tional deficits at third grade: A random- upon the affective characteristics of ized field trial. Journal of Research on teachers. American Educational Re- Educational Effectiveness, (1),1 2–32. search Journal, 21(2), 245–259. Fuchs, L. S., Seethaler, P. M., Powell, S. R., Haager, D., Klinger, J., & Vaughn, S. (2007). Fuchs, D., Hamlett, C. L., & Fletcher, J. M. Evidence-based reading practices for re- (2008). Effects of preventative tutoring sponse to intervention. Baltimore, MD: on the mathematical problem solving Paul H. Brooks Publishing Company. of third- grade students with math and Halpern, D., Aronson, J., Reimer, N., Simp- reading difficulties. Exceptional Chil- kins, S., Star, J., & Wentzel, K. (2007). dren, 74(2), 155–173. Encouraging girls in math and science (NCER 2007-2003). Washington, DC:

( 94 ) References

National Center for Education Research, Howard, P., Perry, B. & Conroy, J. (1995, Institute of Education Sciences, U.S. De- November). Manipulatives in K-6 math- partment of Education. Retrieved Janu- ematics learning and teaching. Paper ary 2009, from http://ies.ed.gov/ncee/ presented at the annual conference of wwc/publications/practiceguides. the Australian Association for Research Hanich, L. B., Jordan, N. C., Kaplan, D., & in Education, Hobart. Dick, J. (2001). Performance across dif- Howard, P., Perry, B., & Lindsay, M. (1996, ferent areas of mathematical cognition November). Mathematics and manipula- in children with learning difficulties. tives: Views from the secondary schools. Journal of Educational Psychology, 93(3), Paper presented at the joint annual 615–626. meeting of the Educational Research Harry, B., & Klingner, J. K. (2006). Why are Association, Singapore and Australian so many minority students in special Association for Research in Education, education? Understanding race and dis- Singapore. ability in schools. New York: Teachers Jiban, C., & Deno, S. L. (2007). Using math College Press. and reading curriculum-based mea- Hasselbring, T. S., Bransford, J. D., & Goin, L. I. surements to predict state mathematics (1988). Developing math automaticity in test performance. Assessment for Effec- learning handicapped children: The role tive Intervention, 32(2), 78–89. of computerized drill and practice. Focus Jitendra, A. K. (2007). Solving math word on Exceptional Children, 20(6), 1–7. problems: Teaching students with learn- Hecht, S. A., Vagi, K. J., & Torgesen, J. K. ing disabilities using schema-based in- (2007). Fraction skills and proportional struction. Austin, TX: PRO-ED. reasoning. In D. B. Berch & M. M. Maz- Jitendra, A., Carnine, D., & Silbert, J. (1996). zocco (Eds.), Why is math so hard for Descriptive analysis of fifth grade di- some children? The nature and origins vision instruction in basal mathemat- of mathematical learning difficulties and ics programs: Violations of pedagogy. disabilities (pp. 121–132). Baltimore, MD: Journal of Behavioral Education, 6(4), Paul H. Brookes Publishing Co. 381–403. Heller, L. R., & Fantuzzo, J. W. (1993). Re- Jitendra, A. K., Griffin, C. C., McGoey, K., ciprocal peer tutoring and parent part- Gardill, M. C., Bhat, P., & Riley, T. (1998). nership: Does parent involvement make Effects of mathematical word problem a difference? School Psychology Review, solving by students at risk or with mild 22(3), 517–534. disabilities. The Journal of Educational Heller, K., Holtzman, W., & Messick, S. Research, 91(6), 345–355. (1982). Placing children in special educa- Jitendra, A. K., Sczesniak, E., & Deatline- tion: A strategy for equity. Washington, Buchman, A. (2005). An exploratory DC: National Academies Press. validation of curriculum-based math- Helwig, R., Anderson, L., & Tindal, G. (2002). ematical word-problem-solving tasks as Using a concept-grounded, curriculum- indicators of mathematics proficiency based measure in mathematics to pre- for third graders. School Psychology Re- dict statewide test scores for middle view, 34(3), 358–371. school students with LD. The Journal of Johnson, E. S., Jenkins, J. R., Petscher, Y., & Special Education, 36(2), 102–112. Catts, H. W. (2008). How can we improve Hill, H. C., Rowan, B., & Ball, D. L. (2005). Ef- the accuracy of screening instruments? fects of teachers’ mathematical knowl- Manuscript submitted for publication. edge for teaching on student achieve- Jordan, N. C., Hanich, L. B., & Kaplan, D. ment. American Educational Research (2003). A longitudinal study of mathe- Journal, 42(2), 371–406. matical competencies in children with specific mathematics difficulties versus

( 95 ) References

children with comorbid mathematics learning disabilities. Learning Disability and reading difficulties. Child Develop- Quarterly, 23(2), 137–156. ment, 74(3), 834–850. McCloskey, M. (2007). Quantitative literacy Kaminski, R., Cummings, K. D., Powell- and developmental dyscalculias. In Smith, K. A., & Good, R. H. I. (2008). Best D. B. Berch & M. M. Mazzocco (Eds.), practices in using Dynamic Indicators Why is math so hard for some children? of Basic Early Literacy Skills (DIBELS) for The nature and origins of mathematical formative assessment and evaluation. learning difficulties and disabilities. Bal- In A. Thomas & J. Grimes (Eds.), Best timore: Paul H. Brookes Publishing. practices in school psychology V: Vol. 4 Messick, S. (1988). The once and future is- (pp. 1181–1204). Bethesda, MD: National sues of validity: Assessing the meaning Association of School Psychologists. and consequences of measurement. In Kavale, K., & Spaulding, L. (2008). Is re- H. Wainer & H. I. Braun (Eds.), Test valid- sponse to intervention good policy for ity (pp. 33–44). Hillsdale, NJ: Lawrence specific learning disability? Learning Erlbaum Associates. Disabilities Research & Practice, 23(4), Milgram, R. J., & Wu, H. S. (2005). The key 169 –179. topics in a successful math curriculum. Lee, J., Grigg, W., & Dion, G. (2007). The Retrieved November 26, 2008, from nation’s report card: Mathematics 2007 http://math.berkeley.edu/~wu/ (NCES 2007–494). Washington, DC: Na- National Academy of Sciences. (2002). Mi- tional Center for Education Statistics, nority students in special and gifted Institute of Education Sciences, U.S. education. Washington, DC: National Department of Education. Academies Press. Leh, J. M., Jitendra, A. K., Caskie, G. I. L., National Council of Teachers of Mathemat- & Griffin, C. C. (2007). An evaluation ics. (2006). Curriculum focal points for of curriculum-based measurement of prekindergarten through grade 8 math- mathematics word problem-solving ematics: A quest for coherence. Reston, measures for monitoring third-grade VA: Author. students’ mathematics competence. National Joint Committee on Learning Assessment for Effective Intervention, Disabilities. (2005). Responsiveness to 32(2), 90–99. intervention and learning disabilities. Lembke, E. S., & Foegen, A. (2009). Iden- Learning Disability Quarterly, 28(4), tifying early numeracy indicators for 249–260. kindergarten and first-grade students. National Mathematics Advisory Panel. Learning Disabilities Research & Prac- (2008). Foundations for success: The tice, 24(1), 12–20. final report of the national mathemat- Lembke, E. S., Foegen, A., Whittaker, T. A., ics advisory panel. Washington, DC: U.S. & Hampton, D. (2008). Establishing tech- Department of Education. nically adequate measures of progress National Research Council. (2001). Add- in early numeracy. Assessment for Effec- ing it up: Helping children learn math- tive Intervention, 33(4), 206 –214. ematics. Washington, DC: National Locuniak, M. N., & Jordan, N. C. (2008). Academies Press. Using kindergarten number sense to Okamoto, Y., & Case, R. (1996). Exploring predict calculation fluency in second the microstructure of children’s central grade. Journal of Learning Disabilities, conceptual structures in the domain of 41(5), 451–459. number. Monographs of the Society for Manalo, E., Bunnell, J. K., & Stillman, J. A. Research in Child Development, 61(1–2), (2000). The use of process mnemonics 27–58. in teaching students with mathematics Organisation for Economic Co-operation and Development. (2008). Programme

( 96 ) References

for international student assessment Stigler, J. W., & Hiebert, J. (1999). The teach- (PISA).Retrieved December 5, 2008, from ing gap: Best ideas from the world’s http://www.pisa.oecd.org/ teachers for improving education in the Parmar, R. S. (1992). Protocol analysis of classroom. New York: Free Press. strategies used by students with mild Thurber, R. S., Shinn, M. R., & Smolkowski, K. disabilities when solving arithmetic (2002). What is measured in mathematics word problems. Diagnostique, 17(4), tests? Construct validity of curriculum- 227–243. based mathematics measures. School Peterson, P. L., Fennema, E., & Carpenter, Psychology Review, 31(4), 498. T. (1989). ��Using knowledge of how stu- Tournaki, N. (2003). The differential effects dents think about mathematics. Educa- of teaching addition through strategy tional Leadership, 46(4), 42–46. instruction versus drill and practice to President’s Commission of Excellence in students with and without learning dis- Special Education. (2002). A new era: Re- abilities. Journal of Learning Disabilities, vitalizing special education for children 36(5), 449–458. and their families. Washington, DC: U.S. University of Minnesota. (2008). Research Department of Education. Institute on Progress Monitoring. Re- Robert, R. S., & Jenkins, E. (1989). How chil- trieved December 5, 2008, from http:// dren discover new strategies. Hillsdale, www.progressmonitoring.net/ NJ: Lawrence Erlbaum Associates. U.S. Department of Education. (n.d.). Robinson, C. S., Menchetti, B. M., & Torge- Trends in international mathematics and sen, J. K. (2002). Toward a two-factor science study (TIMSS). [Overview]. Re- theory of one type of mathematics dis- trieved December 5, 2008, from http:// abilities. Learning Disabilities Research nces.ed.gov/timss/ & Practice, 17(2), 81–89. U.S. Department of Education, Office of Schmidt, W. H., & Houang, R. T. (2007). Lack Planning, Evaluation and Policy Devel- of focus in mathematics: Symptom or opment, Policy and Program Studies cause? Lessons learned. In T. Loveless Service. (2006). Reading first implemen- (Ed.), What international assessments tell tation evaluation: Interim report. Wash- us about math achievement. Washing- ington, DC: Author. ton, DC: Brookings Institution Press. U.S. Office of Special Education. (2008). Schunk, D. H., & Cox, P. D. (1986). Strategy National Center on Student Progress training and attributional feedback Monitoring. [Web site]. Retrieved De- with learning disabled students. Jour- cember 5, 2008, from http://www.stu- nal of Educational Psychology, 78(3), dentprogress.org/ 201–209. VanDerHeyden, A. M., Witt, J. C., & Gilbert- Shapiro, E. S., Edwards, L., & Zigmond, N. son, D. (2007). A multiyear evaluation of (2005). Progress monitoring of math- the effects of a response to intervention ematics among students with learning (RtI) model on identification of children disabilities. Assessment for Effective In- for special education. Journal of School tervention, 30(2), 15–32. Psychology, 45(2), 225–256. Shinn, M. R. (Ed.). (1989). Curriculum-based VanDerHeyden, A. M., Witt, J. C., & Naquin, measurement: Assessing special chil- G. (2003). Development and validation dren. New York: Guilford Press. of a process for screening referrals to Siegler, R. S., & Jenkins, E. (1989). How chil- special education. School Psychology dren discover new strategies. Hillsdale, Review, 32(2), 20 4 –227. NJ: Lawrence Erlbaum Associates. VanDerHeyden, A. M., Witt, J. C., Naquin, Silbert, J., Carnine, D., & Stein, M. (1989). G., & Noell, G. (2001). The reliability and Direct instruction mathematics. Colum- validity of curriculum-based measure- bus, OH: Merrill. ment readiness probes for kindergarten

( 97 ) References

students. School Psychology Review, Witzel, B. S. (2005). Using CRA to teach 30(3), 363. algebra to students with math diffi- Vaughn, S., & Fuchs, L. S. (2003). Redefining culties in inclusive settings. Learning learning disabilities as inadequate re- Disabilities—A Contemporary Journal, sponse to instruction: The promise and 3(2), 49–60. potential problems. Learning Disabili- Witzel, B. S., Mercer, C. D., & Miller, M. D. ties Research & Practice, 18, 137–146. (2003). Teaching algebra to students Vaughn, S., & Fuchs, L. S. (2006). A re- with learning difficulties: An investiga- sponse to Competing views: A dialogue tion of an explicit instruction model. on response to intervention: Why re- Learning Disabilities Research & Prac- sponse to intervention is necessary but tice, 18 (2), 121–131. not sufficient for identifying students Woodward, J. (2006). Developing automa- with learning disabilities. Assessment ticity in multiplication facts: Integrating for Effective Intervention, 32(1), 58–61. strategy instruction with timed practice Walker, D. W., & Poteet, J. A. (1989). A com- drills. Learning Disability Quarterly, parison of two methods of teaching 29(4), 269–289. mathematics story problem-solving Wu, H. (2005). Must content dictate peda- with learning disabled students. Na- gogy in mathematics education (No. tional Forum of Special Education Jour- 3840). University of California: Depart- nal, 1, 44–51. ment of Mathematics. What works clearinghouse: Procedures and Xin, Y. P., Jitendra, A. K., & Deatline-Buch- standards handbook (version 2.0). (2008). man, A. (2005). Effects of mathematical Retrieved March 2009, from http:// word-problem-solving instruction on ies.ed.gov/ncee/wwc/pdf/wwc_proce- middle school students with learning dures_v2_standards_handbook.pdf. problems. Journal of Special Education, Wilson, C. L., & Sindelar, P. T. (1991). Direct 39(3), 181–192. instruction in math word problems: Stu- dents with learning disabilities. Excep- tional Children, 57(6), 512–519.

( 98 )