Achievement of 15-Year- Olds in England: PISA 2015 National Report December 2016 John Jerrim and Nikki Shure. UCL Institute of Education. Contents Acknowledgements 3 Executive summary 4 Chapter 1. Introduction 11 Chapter 2. Achievement in science 23 Chapter 3. Achievement in different aspects of scientific literacy 47 Chapter 4. Achievement in mathematics 64 Chapter 5. Achievement in reading 83 Chapter 6. Variation in PISA scores by pupil characteristics 101 Chapter 7. Differences in achievement between schools 117 Chapter 8. School management and resources 129 Chapter 9. Pupils’ experiences of their time in science classes at school 146 Chapter 11. PISA across the UK 181 Appendix A. Background to the PISA study 207 Appendix B. Sample design and response rates 218 Appendix C. Testing statistical significance in PISA across cycles 224 Appendix D. The conversion of PISA scores into years of schooling 227 Appendix E. PISA 2015 mean scores 228 Appendix F. Long-term trends in PISA scores 231 Appendix G. Mean scores in the science sub-domains 236 Appendix H. List of figures 239 Appendix I. List of tables 243 2 Acknowledgements This report represents a multi-team effort. We are grateful to the teams at Educational Testing Service (ETS), Westat, cApStAn Linguistic Control, Pearson and the German Institute for International Education Research (DIPF) for their support and guidance throughout the project. In England we are grateful to the team at the Department for Education that oversaw the work, in particular Adrian Higginbotham, Emily Knowles, Bethan Knight, Joe Delafield and David Charlton. The team at RM Education (RM) managed the research consortium and the process of collecting and checking the data as well as the production of reports for participating schools; we are grateful to Dawn Pollard, Daryl Brown and Sam Smith for overseeing that. Also to Martin Ripley and his team of assessors at World Class Arena Ltd (WCAL), who marked the responses. At the UCL Institute of Education we are grateful to our colleagues John Micklewright and Phil Rose for their input. Finally, we are especially grateful to the staff and students at the participating schools for their time and effort in administering and completing the assessments and questionnaires. 3 Executive summary Introduction The Programme for International Student Assessment (PISA), led by the Organisation for Economic Co-operation and Development (OECD), provides evidence on how the achievement and abilities of 15-year-olds varies across countries. PISA is conducted every three years, and pupils are tested in four subjects (science, mathematics, reading and collaborative problem solving), with one subject the particular focus each time. Together these data enable us to benchmark ourselves against the rest of the world, and to spot particular strengths and weaknesses in our education system. PISA is conducted every three years and is centred around a direct assessment of pupils’ science, mathematics and reading abilities. Each year one of these subjects is covered in more detail – science in 2015 – and pupils are also assessed in an innovative domain – collaborative problem solving1 in 2015. In 2015 PISA was administered in the majority of countries as a computer-based assessment (CBA) for the first time. Over 70 countries participated in PISA 2015, including all members of the OECD and all four countries within the United Kingdom. In England, PISA 2015 was conducted in November to December 2015, with a sample of 5,194 pupils in England from across 206 schools. The vast majority of England’s participating pupils were born between September 1999 and August 2000, meaning they came to the end of primary school during 2010, and were the last cohort to take the GCSE examinations before they are reformed. Highlights The average science, mathematics and reading scores of pupils in England have not changed since 2006. Our 15-year-olds continue to perform significantly above the OECD average in science whilst they remain at the OECD average for mathematics. For the first time in 2015, pupils in England perform significantly, but only just, above the OECD average in reading. Although there has been no significant change in England’s absolute score, our performance relative to other countries has changed since 2012 as they improve or decline around us. The OECD average has fallen (but only significantly in science) meaning that England’s reading performance is now above average despite having not changed since 2012, and our relative science position has increased compared to 2012 as other countries’ average scores have dropped. Whilst performance in England has not changed there have been changes in other parts of the United Kingdom, notably declines in average science performance in Scotland and Wales. East Asian countries continue to dominate the top positions in PISA. Singapore tops 1 The results of the collaborative problem solving assessment will be released by the OECD in 2017. 4 PISA 2015 in science, reading and mathematics. Shanghai, which came top across all three subjects in 2012, has been joined in PISA 2015 by three more Chinese provinces (Beijing, Jiangsu and Guangdong) and is no longer reported as a separate entity. China (B-S-J-G) performs similar to England in science and reading. It continues to outperform England in mathematics. Achievement in science The average science score in England has remained consistent since 2006 and is higher than the average score of 15-year-olds in 52 countries. There are just nine countries where the mean science score is at least 10 points (four months of schooling) ahead of England, including Singapore, Japan, Estonia and Taiwan – the top science performers in 2015. Although England’s average science score has not changed since 2006, other countries have moved around us. For example, Australia and New Zealand have undergone a sustained fall in their scores since 2006 and are now at a similar level to England, having been previously ahead. The average science score has also fallen in Finland, though it remains a high-performing country. Portugal and Macao, meanwhile, are two of the few countries where there has been a statistically significant and sustained improvement in science achievement since 2006. The comparatively high science performance of England’s high-achievers is a notable strength of the English educational system; this country has some of the best young scientists anywhere in the world. There are only three countries (Singapore, Taiwan and Japan) where the top 10 per cent of pupils are more than one school term (four months of schooling) ahead of their peers in England. However, the gap between the highest and lowest achieving pupils in science is also bigger in England than in many other OECD countries. Pupils in England achieve approximately the same scores in what PISA defines as the living scientific system (which roughly equates to biology), physics and earth and space sciences. This is similar to the situation in many of the top-performing countries. Although boys in England have achieved a higher average score in science than girls in the past, in 2015 there is no evidence of a gender gap in performance. Achievement in mathematics The average mathematics score for England has remained stable since 2006. There are 18 countries where the mean score is at least a third of a year of schooling ahead of England, and 36 countries where the mean mathematics score is at least a third of a year of schooling below. The top seven ranked jurisdictions in PISA mathematics are all within East Asia. It is of note that while China are among the top seven performers; mathematics is the only subject in which China significantly outperforms England in 2015. 5 Although England’s average mathematics score has remained stable, a number of countries have caught England up over the last decade, including Italy, Portugal and Russia. On the other hand, the Czech Republic, Australia, New Zealand and Iceland all had higher average mathematics scores than England in 2006, but their mean mathematics score is now similar to ours. In tandem with their declining science performance, Finland, New Zealand, Australia and the Netherlands experienced substantial declines in average mathematics scores since 2006. England’s top achievers in mathematics do not stand out in the same way as our top scientists. England has a similar proportion of high-achieving pupils as the average across members of the OECD, and our top maths performers are similarly placed internationally as we are in terms of our average performance. Meanwhile, the relatively poor mathematics skills of England’s low-achieving pupils stands out as a weakness of England’s education system. England’s lowest achievers have mathematics skills that are significantly below the mathematics skills of the lowest achievers in several other countries. It is also notable how the bottom 10 per cent of mathematics performers in England trail those in Northern Ireland and Scotland, despite both countries having very similar average mathematics scores to England. Indeed, England has a particularly unequal distribution of 15-year-olds’ mathematics achievement. The gap between the highest and lowest achieving pupils in mathematics in England is above the OECD average and is equivalent to over eight years of schooling. Boys continue to out-perform girls in mathematics in England (and most other countries). The mathematics skills of boys in England is, on average, around a third of a year of schooling ahead of girls. This compares to the results for reading, in which girls do better, and science where girls and boys are equal. Achievement in reading As is the case with science and mathematics, there is no evidence of a significant change in average reading scores in England since 2006. There are nine countries where the mean reading score is at least a third of a year of schooling ahead of England, with the top performing countries including Singapore, Hong Kong, Canada, Finland and Ireland.