University of Tennessee, Knoxville TRACE: Tennessee Research and Creative Exchange Doctoral Dissertations Graduate School 8-2011 Validation of the Monitoring Academic Progress: Reading (MAP: R): Development and Investigation of a Group-Administered Comprehension-Based Tool for RTI Angela Nicole Hilton-Prillhart [email protected] Follow this and additional works at: https://trace.tennessee.edu/utk_graddiss Part of the Educational Assessment, Evaluation, and Research Commons Recommended Citation Hilton-Prillhart, Angela Nicole, "Validation of the Monitoring Academic Progress: Reading (MAP: R): Development and Investigation of a Group-Administered Comprehension-Based Tool for RTI. " PhD diss., University of Tennessee, 2011. https://trace.tennessee.edu/utk_graddiss/1082 This Dissertation is brought to you for free and open access by the Graduate School at TRACE: Tennessee Research and Creative Exchange. It has been accepted for inclusion in Doctoral Dissertations by an authorized administrator of TRACE: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. To the Graduate Council: I am submitting herewith a dissertation written by Angela Nicole Hilton-Prillhart entitled "Validation of the Monitoring Academic Progress: Reading (MAP: R): Development and Investigation of a Group-Administered Comprehension-Based Tool for RTI." I have examined the final electronic copy of this dissertation for form and content and recommend that it be accepted in partial fulfillment of the equirr ements for the degree of Doctor of Philosophy, with a major in School Psychology. R. Steve McCallum, Sherry Mee Bell, Major Professor We have read this dissertation and recommend its acceptance: Christopher Skinner, Amy Broemmel Accepted for the Council: Carolyn R. Hodges Vice Provost and Dean of the Graduate School (Original signatures are on file with official studentecor r ds.) Validation of the Monitoring Academic Progress: Reading (MAP: R): Development and Investigation of a Group-Administered Comprehension-Based Tool for RTI A dissertation Presented for the Doctor of Philosophy Degree The University of Tennessee, Knoxville Angela Hilton-Prillhart August, 2011 Copyright 2011© by Angela Hilton-Prillhart All rights reserved. ii Dedication To my husband, Chris, for your unconditional love and support. My sons, Evan, Riley, and Joseph for your patience with this project and the amount of time devoted to its completion. To my father, Grady Hilton, for your positive example, love, and sacrifice. In memory of my mother, Emma Hilton, daughter, Jessica Mounger, sister, Tonya Edmonson, and grandparents, Ralph and Margaret Tyree, and Irene Hilton…you are always with me. “Be present in all things and thankful for all things." ~Maya Angelou iii Acknowledgements This project would not have been possible without the guidance and direction of Dr. Sherry Bell and Dr. Steve McCallum. I thank them for the opportunity to work on this project and for their direction during conception, development and implementation of MAP: R. I also appreciate the technical and edit suggestions offered during the writing of this dissertation. Their expertise with testing and psychometrics has been invaluable. I also wish to thank the members of Dr. McCallum‟s research group that participated in this project. Michael Hopkins‟ contributions were essential to the completion of this project. Without his time and data management expertise, this project could not have been completed. I also wish to thank Kelli Miller, Ashlee Cornett, Kristen Maurer, and Amelia Davis for their feedback and work on editing the probes. Additionally, I appreciate the willingness of Dr. Christopher Skinner and Dr. Amy Broemmel to serve on my dissertation committee and the feedback and suggestions they provided. Steve Dunkin and Robyn O‟Dell were instrumental in the implementation of the MAP: R probes as part of the Response to Intervention program in their school district. I appreciate their patience and willingness to advocate for this project. Their hard work will hopefully translate into more students receiving evidence-based intervention in their system. Finally, I thank my family for providing encouragement and support throughout this project. Their generosity and understanding of the time commitment required to complete this project made its completion possible. iv Abstract Monitoring Academic Progress: Reading (MAP: R), a silent, group-administered screener was piloted as part of a comprehensive Response to Intervention program. MAP: R along with AIMSweb© Maze and STAR reading were administered to 1,688 students in Grades 1-3. Overall alternate-form reliabilities for MAP: R resulted in moderately high stability (Grade 1 = .79, Grade 2 = .78, and Grade 3 = .75). Test-retest reliability was .90 for Grade1, .84 for Grade 2, and .89 for Grade 3. Concurrent validity, correlations for MAP: R and AIMSweb© Maze ranged from .43 to .69, with correlations for MAP: R and STAR ranging from .48 to .67. Predictive validity was assessed using end-of-the-year STAR reading scores as the criterion for MAP: R and AIMSweb© Maze. Results of a stepwise regression indicated that MAP: R scores predicted 37% of the variance in STAR scores and AIMSweb© Maze failed to add additional predictive variance. Data support the utility of MAP: R as a reading screener for progress monitoring within a Response to Intervention framework, though additional data are needed. v Table of Contents DEDICATION .............................................................................................................................. III LIST OF TABLES ..................................................................................................................... VIII CHAPTER 1 ................................................................................................................................... 1 LITERATURE REVIEW ................................................................................................... 1 Response to Intervention......................................................................................... 2 Measuring Fluency and Comprehension with Norm-Referenced Tests ................. 4 Standardized Measures of Reading Fluency ........................................................... 9 Curriculum-Based Measures of Reading Fluency ................................................ 12 Integrating Standardized and Curriculum-Based Measurement with RTI ........... 14 MAP: R Reading Assessment ............................................................................... 19 Statement of the Problem .................................................................................................. 20 CHAPTER II ................................................................................................................................. 21 METHOD ......................................................................................................................... 21 Participants ............................................................................................................ 21 Instrument Development ....................................................................................... 21 CHAPTER III ............................................................................................................................... 38 RESULTS ......................................................................................................................... 38 Reliability of the MAP: R ..................................................................................... 38 Validity of the MAP: R ......................................................................................... 40 CHAPTER IV ............................................................................................................................... 43 DISCUSSION ................................................................................................................... 43 vi Reliability .............................................................................................................. 45 Validity ................................................................................................................. 47 Limitations and Future Research .......................................................................... 49 LIST OF REFERENCES .............................................................................................................. 52 APPENDICES .............................................................................................................................. 76 Appendix A ....................................................................................................................... 77 Appendix B ....................................................................................................................... 97 Appendix C ..................................................................................................................... 107 Appendix D ..................................................................................................................... 133 Appendix E ..................................................................................................................... 155 Appendix F...................................................................................................................... 170 VITA ..........................................................................................................................................