Report Achievement of 15-Year Olds in Wales: PISA 2018 National Report National Foundation for Educational Research (NFER) Juliet Sizmur, Robert Ager, Jenny Bradshaw, Rachel Classick, Maria Galvis, Joanna Packer, David Thomas and Rebecca Wheater Achievement of 15-year-old pupils in Wales PISA 2018 National report Juliet Sizmur, Robert Ager, Jenny Bradshaw, Rachel Classick, Maria Galvis, Joanna Packer, David Thomas and Rebecca Wheater Published in December 2019 By the National Foundation for Educational Research, The Mere, Upton Park, Slough, Berkshire SL1 2DQ www.nfer.ac.uk © 2019 National Foundation for Educational Research Registered Charity No. 313392 ISBN: 978-1-9-11039-96-9 How to cite this publication: Sizmur, J., Ager, R., Bradshaw, J., Classick, R., Galvis, M., Packer, J., Thomas D. and Wheater R. (2019). Achievement of 15-year-olds in Wales: PISA 2018 National report. Slough: NFER 2 Contents List of figures 7 List of tables 10 Acknowledgements 14 Executive summary 15 1 Introduction 20 1.1 What is PISA? 20 1.1.1 Background to PISA 20 1.1.2 Participating countries 21 1.2 What does PISA measure? 22 1.2.1 The PISA 2018 assessment framework 22 1.2.2 The PISA questionnaires 22 1.3 How does PISA measure attainment? 23 1.3.1 How PISA samples are chosen 24 1.3.2 How PISA assesses pupils 25 1.4 Organisation of this report 26 2 Reading 30 2.1 Wales’ performance in reading 31 2.2 Reading in PISA 2018 33 2.2.1 Changes between 2009 and 2018 in the PISA assessment of reading 34 2.3 International results 35 2.4 Reading subscale scores 39 2.4.1 Locating information 39 2.4.2 Understanding 41 2.4.3 Evaluating and reflecting 42 2.4.4 Reading fluency 44 2.4.5 Text classification 45 2.4.6 Source 46 2.5 Differences between highest and lowest achievers 47 2.5.1 Distribution of scores 47 3 2.5.2 Performance across PISA proficiency levels 50 2.6 Differences between boys and girls 54 3 Pupils 59 3.1 Pupil background 60 3.1.1 Socio-economic background 60 3.1.2 Immigration background and language 64 3.2 Pupils’ attitudes to reading inside and outside school 66 3.2.1 Perceptions of competence in reading 66 3.2.2 How do pupils read books? 67 3.2.3 Reading engagement 68 3.2.4 Time spent reading for enjoyment 69 3.3 Pupils’ experience of reading inside and outside school 70 3.3.1 Pupils’ reading practices 70 3.3.2 Pupils’ digital reading practices 71 3.3.3 Metacognition 72 3.4 Pupil wellbeing 75 3.5 Future aspirations 79 4 Science 82 4.1 Wales’ performance in science 83 4.2 International results 84 4.3 Differences between highest and lowest achievers 89 4.3.1 Distribution of scores 89 4.3.2 Performance across PISA proficiency levels 92 4.4 Differences between boys and girls 94 5 Mathematics 96 5.1 Wales’ performance in mathematics 97 5.2 International results 98 5.3 Differences between highest and lowest achievers 102 5.3.1 Distribution of scores 102 5.3.2 Performance across PISA proficiency levels 105 5.4 Differences between boys and girls 106 4 6 Schools 109 6.1 Variation in scores within and between schools 110 6.2 School management and policies 111 6.2.1 School admissions 111 6.2.2 Grouping policies 112 6.2.3 Equity-oriented policies 114 6.2.4 Assessment and accountability 116 6.3 School climate 118 6.3.1 Teacher and pupil behaviour affecting school climate 118 6.3.2 Parental engagement 121 6.3.3 Extra-curricular activities 122 6.3.4 Disciplinary climate 124 6.3.5 Bullying 124 6.3.6 Competitiveness and cooperation 127 6.4 Resources 130 6.4.1 ICT 130 6.4.2 Problems due to resource shortages 137 6.5 Teachers 139 6.5.1 Teacher qualifications 139 7 PISA in the UK 141 7.1 Reading 143 7.1.1 Mean scores in reading 143 7.1.2 Distribution of performance in reading 147 7.1.3 Performance at each proficiency level in reading 149 7.1.4 Gender differences in reading 150 7.2 Science 152 7.2.1 Mean scores in science 152 7.2.2 Distribution of performance in science 154 7.2.3 Performance at each science proficiency level 155 7.2.4 Gender differences in science 156 7.3 Mathematics 158 5 7.3.1 Mean scores in mathematics 158 7.3.2 Distribution of performance in mathematics 159 Performance at each mathematics proficiency level 160 7.3.3 Gender differences in mathematics 162 7.4 Trends in performance 163 7.5 Schools and pupils 166 7.5.1 School differences 166 7.5.2 Differences in pupils’ socio-economic background 168 7.5.3 Differences in pupils’ attitudes and aspirations 170 References 174 Appendix A Background to the study 176 A1 The development of the study 176 A2 What PISA measures – sample questions 177 A3 What the proficiency levels and PISA scale scores mean 188 A4 Study administration 199 A5 The PISA sample in Wales 200 Appendix B Reading Tables 205 Appendix C Science Tables 239 Appendix D Mathematics Tables 246 Appendix E Notes on PISA International Scale Scores 256 Appendix F Effort Thermometer 258 6 List of figures Figure 2.1 Trends over time in reading scores in Wales and the OECD 32 Figure 2.2 Trends in reading scores for a selection of countries that performed similarly to Wales in 2009 38 Figure 2.3 Reading process subscale scores across countries: locating information 40 Figure 2.4 Reading process subscale across countries: understanding 42 Figure 2.5 Reading process subscale scores across countries: evaluating and reflecting 43 Figure 2.6 Reading source subscale scores across countries: single-source text vs. multiple-source texts 46 Figure 2.7 Attainment gap in reading scores in Wales and the OECD 48 Figure 2.8 Attainment gap in reading scores across PISA 2018 countries 49 Figure 2.9 Attainment gap in countries with similar performance to Wales at either the 10th or 90th percentiles 50 Figure 2.10 Reading proficiency levels in Wales and the OECD 51 Figure 2.11 Reading proficiency levels by cognitive process in Wales 52 Figure 2.12 Reading proficiency levels by reading source in Wales 53 Figure 2.13 Gender differences in reading scores in Wales and the OECD 54 Figure 2.14 Gender differences in reading scores across PISA 2018 countries 55 Figure 2.15 Gender differences in reading processes in Wales and the OECD 56 Figure 2.16 Gender differences in reading source in Wales and OECD 57 Figure 3.1 Reading performance by ESCS Index quartile 62 Figure 3.2 Average ratings of usefulness of strategies for understanding and remembering text 73 Figure 3.3 Average ratings of usefulness of strategies for summarising a difficult text74 Figure 3.4 Average ratings of responses to the receipt of an email telling pupils they have won a smartphone 75 7 Figure 3.5 Percentage of pupils agreeing and disagreeing with questions about the extent to which their life had meaning 76 Figure 3.6 Percentage of pupils who reported never, rarely, sometimes and always for each positive feeling 77 Figure 3.7 Percentage of pupils who reported never, rarely, sometimes and always for each negative feeling 78 Figure 4.1 Trends over time in science scores in Wales and the OECD 84 Figure 4.2 Trends in science scores for a selection of countries that performed similarly to Wales in 2009 88 Figure 4.3 Attainment gap in science scores in Wales and the OECD 90 Figure 4.4 Attainment gap in science scores across PISA 2018 countries 91 Figure 4.5 Attainment gap in countries with similar performance to Wales at either the 10th or 90th percentiles 92 Figure 4.6 Science proficiency levels in Wales and the OECD average 93 Figure 4.7 Gender gap for Wales compared to the OECD average 94 Figure 4.8 Gender differences in science scores across PISA 2018 countries 95 Figure 5.1 Trends over time in mathematics scores in Wales and the OECD 98 Figure 5.2 Trends in mathematics scores for countries that performed similarly to Wales in 2009 101 Figure 5.3 Attainment gap in mathematics scores in Wales and the OECD 103 Figure 5.4 Attainment gap in mathematics scores across PISA 2018 countries 104 Figure 5.5 Attainment gap in countries with similar performance to Wales at either the 10th or 90th percentiles 105 Figure 5.6 Mathematics proficiency levels in Wales and the OECD 106 Figure 5.7 Gender differences in maths scores in Wales and the OECD 107 Figure 5.8 Gender differences in mathematics scores across PISA 2018 countries 108 Figure 7.1 Mean reading scores across the UK 144 8 Figure 7.2 Attainment gap in reading scores across the UK 148 Figure 7.3 Percentage of pupils reaching each reading level in the UK 150 Figure 7.4 Gender differences in reading scores across the UK 151 Figure 7.5 Mean science scores across the UK 153 Figure 7.6 Attainment gap in science scores across the UK 154 Figure 7.7 Percentage of pupils reaching each science level in the UK 156 Figure 7.8 Gender differences in science scores across the UK 157 Figure 7.9 Mean mathematics scores across the UK 158 Figure 7.10 Attainment gap in mathematics scores across the UK 160 Figure 7.11 Percentage of pupils reaching each mathematics level in the UK 161 Figure 7.12 Gender differences in mathematics scores across the UK 162 Figure 7.13 Trends in reading scores across the UK 164 Figure 7.14 Trends in science scores across the UK 165 Figure 7.15 Trends in mathematics scores across the UK 165 Figure 7.16 Reading performance of UK countries and OECD by ESCS quartile 169 9 List of tables Table 1.1 List of 79 countries that took part in PISA 2018 21 Table 2.1 PISA International results for reading: participants with significantly HIGHER reading scores than Wales 36 Table 2.2 PISA International results for reading: participants with SIMILAR reading scores to Wales (not statistically significantly different) 37 Table 2.3 PISA International results for reading: