TECHNICAL REPORT | JUNE 2014 Development of the ATOS® Readability Formula Written for Renaissance Learning® by Michael Milone, Ph.D. Introduction by Dr. Andrew Biemiller, Professor Emeritus, University of Toronto © 2014 by Renaissance Learning, Inc. All rights reserved. Printed in the United States of America. All logos, designs, and brand names for Renaissance’s products and services, including but not limited Accelerated Reader, AR, ATOS, Renaissance, Renaissance Learning, and Star Reading, are trademarks of Renaissance. This publication is protected by U.S. and international copyright laws. It is unlawful to duplicate or reproduce any copyrighted material without authorization from the copyright holder. For more information, contact: Renaissance P.O. Box 8036 Wisconsin Rapids, WI 54495-8036 (800) 338-4204 www.renaissance.com [email protected] 06/14 ©Copyright 2014 Renaissance Learning, Inc. All rights reserved. 2 Contents 4 Introduction by Dr. Andrew Biemiller, Professor Emeritus, University of Toronto 7 Overview of the ATOS® Readability Formula 7 Readability and its importance 9 Evolution of readability measures 10 Developing the ATOS® Readability Formula 15 Adjustments to ATOS® 16 Refinements to ATOS®—2013 17 The Graded Vocabulary List 22 Alternate scale for ATOS® 22 Conclusion Appendices 23 Appendix A: Studies of factors thought to affect readability 36 Appendix B: Accelerated Reader® Goal-Setting Chart 37 References Tables 13 Table 1. Correlation of semantic and syntactic variables to text difficulty 13 Table 2. Types of transformations 14 Table 3. Squared correlations of the best predictors with each of the selected six transformations 21 Table 4. Examples of grade assignments for derivative forms of base words 23 Table A1. Students and books at each grade level (Proper nouns study) 24 Table A2. Grade 3 middle quartiles (Fiction versus nonfiction study) 26 Table A3. Grade 6 middle quartiles (Fiction versus nonfiction study) 27 Table A4. Grade 4 lower quartile (High dialogue study) 29 Table A5. Grade 5 middle quartiles (High dialogue study) 30 Table A6. Students and books at each grade level (High interest/low readability study) 31 Table A7. Grade 7 middle quartiles (High interest/low readability study) 34 Table A8. Analysis of within-text readability of 10 books based on samples of varying length 36 Table B1. Accelerated Reader® Goal-Setting Chart Figures 8 Figure 1. How ATOS® and interest level relate to the Common Core Model of Text Complexity 18 Figure 2. New Graded Vocabulary List (GVL) improves word pool at all grade levels 32 Figure A1. Analysis of within-text readability of 10 books based on 1/8 book samples ©Copyright 2014 Renaissance Learning, Inc. All rights reserved. 3 Introduction By Dr. Andrew Biemiller Professor Emeritus, University of Toronto The importance of vocabulary development Vocabulary is the most important determinant of language comprehension. The vocabulary needed to comprehend a specific story or expository text amounts to knowledge of all or most of the words in that text. Some word meanings are usually learned earlier than other words. Thus, if most of the words in a text are known by the majority of children by age 5, that text would be considered appropriate for reading to kindergartners, or for reading by beginning readers. Conversely, a text with 5% or more words not known by most children in grade 2 is not appropriate for primary-grade children. By grade 4, texts with more than 15% of words unknown by grade 2 will prove Andrew Biemiller, Ph.D., is a graduate of Harvard College difficult to understand. For grade 6, texts with more than 20% of words and Cornell University. He unknown by grade 2 will be difficult (Chall & Dale, 1995). taught at the Institute of Child Study, University of Toronto for Both before starting kindergarten and at the end of grade 2, children with the 36 years. He was responsible largest vocabularies (highest 25%) know twice as many word meanings as for the teacher education program at the Institute for 15 children with the smallest vocabularies (lowest 25%). Without effective years. His recent research and intervention, it will take second-grade children with low vocabularies another publications have concerned 3 or 4 years to acquire as many words as children with high vocabularies what word meanings are have at the end of grade 2 (Biemiller & Slonim, 2001). Children with low acquired, the order of meanings vocabularies by the end of the primary grades are likely to have poor acquired, and effective academic outcomes as late as high school (Cunningham & Stanovich, 1997). methods for teaching word meanings. His main current These differences mean that, without vocabulary intervention, many or most research concerns identifying children in the low-vocabulary group will have much poorer academic word meanings that are outcomes than children in the highest vocabulary group. particularly useful for instruction at the primary and Effective reading is the key—Both reading to and by children One upper-elementary levels. He of the strongest tools teachers can use to build vocabulary during these published Words Worth Teaching (SRA/McGraw-Hill) in primary years is to use effective books with children. These are books that 2009, which summarizes this teachers read to children, after which they spend time discussing relevant work. He is active as an vocabulary. Books with many unfamiliar words are less suitable for children educational consultant to the to read at these young ages, while they are still working hard to consolidate U.S. Institute of Education their basic reading skills. Science (Dept. of Education), the U.S. National Institute of Child and Human Development, In the upper elementary grades, wide book reading is crucial to continuing to publishers (especially add to students’ vocabularies. Children need to be encouraged not only to Renaissance Learning and read a lot but also to find meanings for unknown words as they encounter SRA/McGraw-Hill), state them. They can do this by asking classmates and the teacher, by attending to departments of education, and in-text word explanations (appositions), and by working with classmates on researchers. lists of possible words from texts in use. Some digital texts include word meaning clarifications. Students can also use dictionaries, but these are often not as helpful as other methods (Stahl & Nagy, 2006). ©Copyright 2014 Renaissance Learning, Inc. All rights reserved. 4 The role of readability formulas in effective reading Teachers call on measures of text difficulty, such as Renaissance’s ATOS Readability Formula, both to assist in finding effective books for reading to students and to guide students’ selections of appropriately challenging books for independent reading. Books at an appropriate grade level for independent reading are likely to provide some challenge in the form of a few (2% to 6%) unfamiliar word meanings. At this level, students have a reasonable chance of inferring word meanings, or acquiring them “latently” (without conscious acquisition, Landauer, 2007). They can also ask about a few unknown words. If there are too many unfamiliar meanings, students tend to get bogged down and cease trying to understand passages. If there are too few unfamiliar meanings, the text will provide little opportunity to acquire new vocabulary. The ATOS® Graded Vocabulary List Part of what fuels the ATOS formula is a check of the words in a book against the ATOS Graded Vocabulary List. A recent update to ATOS—based on data gathered for over 2.5 billion words in more than 170,000 books for which Renaissance has Accelerated Reader Reading Practice Quizzes—included enlarging the Graded Vocabulary List fourfold so that it now includes more than 100,000 words. The Graded Vocabulary List is Renaissance’s listing of vocabulary used in books from kindergarten to college level. Words are identified by the grade level at which they are likely to be understood, based on a number of sources including Living Word Vocabulary (Dale & O’Rourke, 1981), The Educator’s Word Frequency Guide (Zeno, Ivens, Millard, & Duvvuri, 1995), Words Worth Teaching (Biemiller, 2009), words used in standard school achievement tests, and other sources. This recently-revised list is one of three criteria (sentence length, word length, and word level) used by the ATOS formula in determining probable grade levels at which books could be profitably used. (Note that for low-vocabulary students, the “probable grade level” for books may be at least one or two grades lower than the child’s current grade.) In other words, the “grade (readability) level” of a text refers to the “reading level” of a child, not necessarily the child’s age. Additional Accelerated Reader Quizzes are published each week, meaning the revised Graded Vocabulary List reflects current school reading and is best suited for determining readability for school students. The ATOS Readability Formula uses word grade levels and handles simple grammatical variations (plurals, tenses), treating such variations the same as the root word. Homographs or words with more than one meaning (e.g., lean on the wall vs. lean meat) are a challenge for all readability formulas. ATOS assigns readability levels on the basis of the most common meaning of a word. “Simple” homographs can have completely different meanings. Teachers need to point these out when they occur. (More figurative homographs may not be as much of a problem. “Leaning on me when you’re having problems” may be understood if you know leaning as a physical act.) Teachers will have to exercise some care in deciding which word meanings in books deserve attention, and which matter less. To a large extent, this depends on whether the word meaning is likely to be encountered again in other books. Conclusion Teachers require not only good and appropriate books for their students, but also assistance with greatly needed vocabulary supports. ATOS provides both through Renaissance’s Accelerated Reader program.