Technical Guide Using a Valid and Reliable Assessment of College and Career Readiness Across Grades K–12 Technical Guide Excepting those parts intended for classroom use, no part of this publication may be reproduced in whole or in part, or stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without written permission of the publisher. For information regarding permission, write to Scholastic Inc., 557 Broadway, New York, NY 10012. Scholastic Inc. grants teachers who have purchased SRI College & Career permission to reproduce from this book those pages intended for use in their classrooms. Notice of copyright must appear on all copies of copyrighted materials. Portions previously published in: Scholastic Reading Inventory Target Success With the Lexile Framework for Reading, copyright © 2005, 2003, 1999; Scholastic Reading Inventory Using the Lexile Framework, Technical Manual Forms A and B, copyright © 1999; Scholastic Reading Inventory Technical Guide, copyright © 2007, 2001, 1999; Lexiles: A System for Measuring Reader Ability and Text Difficulty, A Guide for Educators, copyright © 2008; iRead Screener Technical Guide by Richard K. Wagner, copyright © 2014; Scholastic Inc. Copyright © 2014, 2008, 2007, 1999 by Scholastic Inc. All rights reserved. Published by Scholastic Inc. ISBN-13: 978-0-545-79638-5 ISBN-10: 0-545-79638-5 SCHOLASTIC, READ 180, SCHOLASTIC READING COUNTS!, and associated logos are trademarks and/or registered trademarks of Scholastic Inc. LEXILE and LEXILE FRAMEWORK are registered trademarks of MetaMetrics, Inc. Other company names, brand names, and product names are the property and/or trademarks of their respective owners. Table of Contents Table of Contents Introduction ................................................................................... 7 Features of SRI College & Career: Foundational Reading Assessment and Reading Comprehension Assessment Subtests ....................................................... 9 Purposes and Uses of SRI College & Career .................................................. 11 Limitations of SRI College & Career .......................................................... 13 Theoretical Framework of Reading Ability and the Lexile Framework in Support of the Reading Comprehension Assessment ........................................ 15 Readability Formulas and Reading Levels ....................................................16 The Lexile Framework for Reading........................................................... 18 Validity of the Lexile Framework for Reading.................................................. 21 College and Career Readiness and Text Complexity............................................ 26 Lexile Item Bank ........................................................................... 30 Description of the Foundational Reading Assessment and Reading Comprehension Assessment Subtests of SRI College & Career ..................................................... 33 Test Materials.............................................................................. 34 Test Administration and Scoring ............................................................. 37 Interpreting SRI College & Career Scores ..................................................... 42 Using SRI College & Career Results .......................................................... 54 Development of SRI College & Career ......................................................... 61 The Foundational Reading Assessment Development .......................................... 63 The Reading Comprehension Assessment Development ....................................... 66 Copyright © 2014 by Scholastic Inc. All rights reserved. Copyright © 2014 by Scholastic Inc. All rights reserved. Reliability .................................................................................... 87 Internal Consistency Reliability Coefficients for the Foundational Reading Assessment............ 88 Standard Error of Measurement ............................................................. 89 Sources of Measurement Error for the Reading Comprehension Assessment .................... 91 Forecasted Comprehension Error for the Reading Comprehension Assessment ................ 102 Validity..................................................................................... 106 Content Validity .......................................................................... 107 Criterion-Related Validity.................................................................. 108 Construct Validity......................................................................... 120 Appendices ................................................................................ 133 Appendix A: Lexile Framework Map ........................................................ 134 Appendix B: Fall and Spring Norm Tables ................................................... 136 Appendix C: References................................................................... 138 2 SRI College & Career Table of Contents List of Figures and Tables List of Tables Table 17 ........................................ 84 Descriptive statistics for each test administration group Table 1.......................................... 22 in the comparison study, April and May 2005. Results from linking studies conducted with the Lexile Framework for Reading. Table 18 ........................................ 88 Internal consistency reliability coefficients (coefficient alpha) Table 2.......................................... 24 for Foundational Reading Assessment scores overall and Correlations between theory-based calibrations produced by grade. by the Lexile equation and rank order of unit in basal readers. Table 19 ........................................ 89 Table 3.......................................... 25 Standard errors of measurement (SEM) for selected Correlations between theory-based calibrations produced Foundational Reading Assessment scores, by grade. by the Lexile equation and the empirical item difficulty. Table 20 ........................................ 90 Table 4.......................................... 29 Mean SEM on the Reading Comprehension Assessment Lexile ranges aligned to college- and career-readiness by extent of prior knowledge. expectations, by grade. Table 21 ........................................ 92 Table 5.......................................... 44 Standard errors for selected values of the length of Foundational Reading Assessment total fluency scores the text. and the corresponding DIBELS Next composite score percentiles. Table 22 ........................................ 94 Analysis of 30 item ensembles providing an estimate Table 6.......................................... 51 of the theory misspecification error. Comprehension rates for the same individual with aterials of varying comprehension difficulty. Table 23 ........................................ 96 Old method text readabilities, resampled SEMs, and Table 7.......................................... 51 new SEMs for selected books. Comprehension rates of different-ability readers with the same material. Table 24 ........................................ 97 Lexile measures and standard errors across item writers. Table 8.......................................... 53 Performance standard proficiency bands for the Reading Table 25 ....................................... 101 Comprehension Assessment in Lexile measures, by grade. Reading Comprehension Assessment marginal reliability estimates. Table 9.......................................... 64 Percentages of students falling in three DIBELS Next Table 26 ....................................... 102 composite score benchmark classifications. Reading Comprehension Assessment reader consistency Copyright © 2014 by Scholastic Inc. All rights reserved. Copyright © 2014 by Scholastic Inc. All rights reserved. estimates over a four-month period, by grade. Table 10 ........................................ 65 Combining accuracy and latency into fluency scores: Table 27 ....................................... 103 Four possible response patterns. Confidence intervals (90%) for various combinations of comprehension rates and standard error of differences Table 11 ........................................ 70 (SED) between reader and text measures. Distribution of items in SRI item bank, by Lexile zone. Table 28 ....................................... 109 Table 12 ........................................ 71 Kindergarten predictive validity coefficients for Foundational ETS DIF categories. Reading Assessment scores as predictors of DIBELS Next Table 13 ........................................ 72 criterion scores, by grade. Reading Comprehension Assessment differential item Table 29 ....................................... 110 functioning, by comparison groups. First-grade predictive validity coefficients for Foundational Table 14 ........................................ 81 Reading Assessment scores as predictors of DIBELS Next Student responses to Question 7: preferred test format. criterion scores, by grade. Table 15 ........................................ 83 Table 30 ....................................... 111 Relationship between the Reading Comprehension Assessment Second-grade predictive validity coefficients for Foundational (interactive) and Reading Comprehension Assessment (print). Reading Assessment scores as predictors of DIBELS Next Table 16 .......................................