An Examination of the UK’s Key Stage Tests’ Reading Section Complexity

Eleanor E. Sanford-Moore, Ph.D., Senior Vice President of Research; Heather Koons, Ph.D., Director of Research Services; and Laura Bush, Research Specialist The study examined the text complexity of the English reading The units of analysis in this study were the English reading pas- sections from Key Stage 1 and 2 tests from 2003 to 2016. The sages as presented in the reading section of the tests for (1) 2016 official sample sections from the Department for Educa- Key Stage 1, years 2003, 2004, and 2016 and (2) Key Stage 2 tion were also included. Three primary research questions were from years 2003 to 2016. Due to changes in test administration investigated: 1) How has the reading section text complexity through the years, the Key Stage 1 test was not administered changed over the years for Key Stage 1 and 2? 2) How closely for years 2005 through 2015. The study sample also includes aligned were the sample sections and administered sections DfE’s Key Stage 1 and Key Stage 2 sample reading sections for 2016? 3) Are the text complexity measures from the Key for 2016. The Key Stage 2 Levels 3-5 and 6 tests were discon- Stage 1 and 2 reading sections comparable to the median text tinued for 2016 and beyond. The final sample was comprised complexity measures from textbooks written for Years 2 and 6? of 77 reading passages.

The results indicate that there has been little variation in the reading section complexity for Key Stage 1 through the three years of test administrations and that the text complexity of the 2016 reading section was closely aligned with that of the official Reading sections from each Key Stage 1 and 2 tests were sample section. However, the findings indicate that the text collected, converted to electronic text and prepared for analy- complexity of the Key Stage 2 reading section has been more sis. All non-prose text (titles, indices, glossaries, page num- varied through the years. In addition, a difference was seen in bering, and poetry) was removed along with pupil instructions the text complexity of the official reading sample and the 2016 and questions. The collection of reading passages was then administered reading section. Both Key Stage 1 and 2 2016 submitted to the Lexile Analyser and each passage was given reading sections had higher text complexity measures than the a Lexile text measure. In addition, the reading passages from median textbook measures for their respective years. each test were submitted to the Lexile Analyser and each reading section was given a Lexile text measure.

The United Kingdom’s Department for Education (DfE) tests the attainment of maintained school pupils in their final year of MetaMetrics® measured the reading sections’ complexity us- both Key Stage 1 and Key Stage 2—Years 2 and 6, respec- ing The Lexile® Framework for Reading and the Lexile Analyz- tively—through an assessment known colloquially as the SATs er®. The resulting Lexile® measures were summarised by Key (Standard Assessment Tests). These tests are developed Stage, administration year, and level where applicable. The through the Standards and Testing Agency (STA), a DfE exec- Lexile measure represents a pupil’s reading ability or a text’s utive agency, for pupil and school monitoring. Over the years, complexity (or difficulty) followed by an L (for Lexile measure). the levels and content of the tests have changed. Tests ad- The Lexile scale ranges from 0L and below for early readers ministered in 2016 were the first administration of Key Stage and texts to above 1600L for advanced readers and texts. The tests that reflect updates to the National Curriculum. Prior to Lexile Analyser is a software program specially designed to 2016, there was a Key Stage 2 test that assessed reading evaluate the reading demand of text based on its semantic levels 3 to 5 and a separate, more rigorous, Key Stage 2 test and syntactic characteristics and determine its Lexile measure. that assessed reading level 6 (BBC, 2016). With the new Na- Independent psychometric studies of the Lexile scale indicate tional Curriculum, there is now a single Key Stage 2 test which that it is a valid and reliable measure of reader ability and text includes content that assesses reading abilities formerly ad- complexity (Mesmer, 2007; White & Clement, 2001). Extensive dressed in the leveled tests. The former marking scheme has information about the development of The Lexile Framework for been replaced by a scaled score system so pupil achieve- Reading and the Lexile Analyzer can be found at ment across the ability continuum can be precisely identified. www.Lexile.com. Table 1. Key Stage Tests, reading section Lexile measure (overall) and passage range (2003-2016). The reading passages were grouped by administration year, Key Stage, and level where applicable. Individual passages received a text complexity measure (Lexile) to describe the Administration Passage Lexile Overall Lexile Year Test Level N Range Measure range of text complexity of passages within individual admin- istrations. An overall measure of complexity for each test’s 2003 KS1 5 540L-950L 700L reading section was determined by grouping the reading pas- 2004 KS1 6 640L-940L 800L sages from the section together and calculating one overall KS1 Lexile measure for the section. 2016 Sample 2 670L-820L 700L 2016 KS1 4 450L-780L 720L

2003 KS2 3-5 3 870L-1070L 930L

Table 1 presents the summary information for the English read- 2004 KS2 3-5 3 610L-1030L 730L ing section of the Key Stage tests. The first two columns pro- 2005 KS2 3-5 3 830L-930L 890L vide the year and Key Stage. The third column notes level 2006 KS2 3-5 3 950L-1140L 1090L specification when applicable. The fourth column shows the 2007 KS2 3-5 4 710L-1160L 1070L number of passages measured. The fifth column lists the mini- mum and maximum Lexile measures of the passages in the 2008 KS2 3-5 2 720L-1090L 850L reading section, and the sixth column shows the overall text 2009 KS2 3-5 2 680L-930L 760L complexity of the reading section of each test. 2010 KS2 3-5 4 480L-980L 810L 2011 KS2 3-5 4 690L-960L 870L The overall Lexile measure of each Key Stage 2 test’s reading 2012 KS2 3-5 4 930L-1080L 990L section is shown in Figure 1 to illustrate the change in Lexile measures over time. These text complexity measures focus 2013 KS2 3-5 5 780L-1060L 990L solely on the challenge presented by the passages and do not 2014 KS2 3-5 3 820L-1090L 1020L take into account the questions posed to pupils. 2015 KS2 3-5 3 870L-1120L 1040L 2011 KS2 6 2 790L-1200L 1050L 2012 KS2 6 3 1060L-1230L 1100L 2013 KS2 6 2 770L-990L 840L The three Key Stage 1 reading sections and the DfE sample section had overall text complexity measures within a 100L 2014 KS2 6 2 1040L-1080L 1060L range of each other. Of great interest currently is the difference 2015 KS2 6 2 1060L-1190L 1110L between the official DfE sample reading section and the ad- KS2 ministered reading section for 2016. The overall reading section 2016 Sample 3 860L-990L 910L measures differed by only 20L. However, the passage 2016 KS2 3 880L-1160L 1110L measures of the two reading sections varied with the sample passages being more complex (670L to 820L) than the admin- different levels of attainment were evaluated (i.e. Levels 3 to 5 istered passages (450L to 780L). for most pupils, Level 6 for more advanced pupils). As stated A previous study investigating the complexity of textbooks avail- earlier, the Level 6 tests have been discontinued in conjunction able for England’s primary and secondary pupils (Sanford- with the new National Curriculum but were included in this anal- Moore, Koons, & Bush, 2016) found that the median Lexile ysis for historical context. The overall Lexile measures for Key measure for Year 2 textbooks was 610L, approximately 100L Stage 2 reading sections have varied over the past 14 years. At lower than the overall reading section measures for the 2016 times, the Level 3 to 5 reading sections have had complexity test administrations. The interquartile range, indicating where measures that equaled or surpassed the complexity measures the middle 50% of texts fell around the median, was only 60L of the Level 6 reading sections. (i.e. 580L to 640L). This difference between textbook complexi- For Key Stage 2, the 2016 administered reading section had ty for Year 2 pupils and the Key Stage reading section com- an overall complexity measure of 1110L, with a passage range plexity indicates a disconnect between the two. However, this of 280L (880L to1160L). This indicates that the 2016 reading finding should be approached with caution as the sample size section was equal to or more challenging than the previous for Year 2 textbooks was small (n = 9). Key Stage 2 reading sections. One may assume this to be by The Key Stage 2 reading passages covered years 2003 to design, as the prior tests (Levels 3 to 5 and 6) were replaced 2016 including tests from the previous administrations when with a single test intended to capture all abilities. The 2016 Figure 1. Key Stage 2 Tests, reading section text complexity (overall Lexile measure) (2003-2016).

administered reading section succeeded in bridging the pas- tion. However, because in many cases, textbooks are not the sage complexity of both the former Level 3 to 5 and Level 6 primary source of classroom materials, it may be difficult to passages. The sample Key Stage 2 reading section released assess the relationship between textbook exposure, practice, by DfE prior to the 2016 administration had an overall measure and assessment. of 910L with a passage range of 130L (860L to 990L). This In conclusion, this study finds that in terms of text complexity overall measure fell in the middle range of complexity for the the 2016 Key Stage 1 reading section is generally well aligned previous Level 3 to 5 Key Stage 2 reading sections and below with the sample reading section and previous reading sec- the middle range of complexity for the previous Level 6 Key tions. The 2016 Key Stage 2 reading section is more complex Stage 2 reading sections. Consequently, the 2016 sample than the 2016 sample section when measured by the Lexile materials did not include examples to match the more difficult Framework. However, at the passage level, the Key Stage 2 passages placed in the administered test. test incorporated passages with a similar range of complexity Again, comparing the Key Stage 2 reading section results with as observed on previously administered tests. Further re- the results from the previously described textbook study, the search into the complexity of materials utilized in the class- Key Stage 2 reading section utilizes text that is more complex room (open source materials, et cetera) throughout the Key than the text typically found in Year 6 textbooks. The median Stages would deepen the understanding of the relationship textbook Lexile measure for Year 6 was 800L, with an interquar- between instructional and evaluation materials tile range of 105L (i.e. 750L to 855L) (n = 15) (Sanford-Moore,

Koons, & Bush, 2016). This indicates a +300L difference in complexity between the median complexity of textbooks availa- ble for use with Year 6 pupils and the Key Stage 2 reading sec-

References

BBC News (2016). Primary tests: What are the changes? Retrieved from: http://www.bbc.com/news/education-35907385

Sanford-Moore, E.; Koons, H; and Bush, L. (2016). “Primary and Secondary Textbook Complexity in England.” Retrieved from: https://metametrics.s3.amazonaws.com/public/dynamic/ international/pdfs/MM_Primary_Secondary_England_Research_Digital.pdf

For more information contact [email protected]