Achievement of 15-Year- Olds in Wales: PISA 2015 National Report December 2016
Total Page:16
File Type:pdf, Size:1020Kb
Achievement of 15-Year- Olds in Wales: PISA 2015 National Report December 2016 John Jerrim and Nikki Shure. UCL Institute of Education. 1 Contents List of figures 6 List of tables 10 Executive summary 14 Chapter 1. Introduction 20 1.1 What is the policy background to this report? 22 1.2 What data have been collected as part of PISA 2015? 23 1.3 How was the PISA 2015 sample recruited in Wales? And how representative is it of the population? 25 1.4 Have there been any important changes since the last PISA wave? 29 1.5 What can PISA tell us? (And what can it not tell us?) 31 1.6 How will the rest of the report be structured? 32 Chapter 2. Achievement in science 35 2.1 What is the mean PISA science score in Wales, and how does this compare to other countries? 37 2.2 How have average PISA science scores in Wales changed over time? How does this compare to other countries? 39 2.3 What proportion of pupils in Wales reach each science achievement level? 42 2.4 How do the science scores of the highest achieving pupils in Wales compare to other countries? 45 2.5 How do the science scores of the lowest achieving pupils in Wales compare to other countries? 47 2.6 How big is the gap between the pupils with the strongest and weakest science skills? How does Wales compare to other countries in this respect? 50 Chapter 3. Achievement in different aspects of scientific literacy 54 3.1 Do pupils have the same proficiency across the PISA physical, living and earth and space science systems? 56 3.2 How do average scores vary in Wales across the three core scientific ‘competencies’ measured by PISA? 58 3.3 How does pupils’ knowledge of scientific content compare to their knowledge of scientific processes and procedures? 61 3.4 Example question 1. Slope face investigation. 64 2 3.5 Example question 2. Bird migration. 67 Chapter 4. Achievement in mathematics 70 4.1 What is the average PISA mathematics score in Wales, and how does this compare to other countries? 71 4.2 How have average PISA mathematics scores in Wales changed over time? How does this compare to other countries? 73 4.3 What proportion of pupils in Wales reach each mathematics proficiency level? 76 4.4 How do the PISA mathematics scores of the highest achieving pupils in Wales compare to other countries? 78 4.5 How do the mathematics scores of the lowest achieving pupils in Wales compare to other countries? 80 4.6 How big is the gap between the pupils with the strongest and weakest mathematics skills? How does Wales compare to other countries in this respect? 83 Chapter 5. Achievement in reading 85 5.1 What is the average PISA reading score in Wales, and how does this compare to other countries? 86 5.2 How have average PISA reading scores in Wales changed over time? How does this compare to other countries? 88 5.3 What proportion of pupils in Wales reach each reading proficiency level? 90 5.4 How do the PISA reading scores of the highest achieving pupils in Wales compare to other countries? 92 5.5 How do the reading scores of the lowest achieving pupils in Wales compare to other countries? 95 5.6 How big is the gap between the pupils with the strongest and weakest reading skills? How does Wales compare to other countries in this respect? 98 Chapter 6. Variation in PISA scores by pupil characteristics 100 6.1 How big is the gender gap in PISA test scores? 102 6.2 How pronounced is the relationship between socio-economic status and pupils’ PISA test scores? 108 6.3 To what extent do socio-economically disadvantaged pupils succeed against the odds? 112 3 6.4 Do immigrants in Wales achieve lower PISA test scores than their peers who were born in the UK? 115 6.5 How do PISA scores differ between pupils who completed the test in English versus Welsh? 118 Chapter 7. Differences in achievement between schools 123 7.1 To what extent does variation in science achievement occur within schools versus between schools in Wales? How does this compare to other countries? 124 7.2 How do PISA test scores differ between English and Welsh medium schools? 127 7.3 How do PISA test scores in Wales vary by school support category? 130 Chapter 8. School management and resources 134 8.1 How do headteachers in Wales manage their staff and schools? 136 8.2 Do headteachers in Wales believe they have access to sufficient resources to support pupils’ learning? 139 8.3 Are schools in Wales well-equipped to support pupils’ learning in science? 142 8.4 How do headteachers view the conduct of their staff? 144 8.5 What quality assurance processes are used in schools? 147 Chapter 9. Pupils’ aspirations and future plans 151 9.1 Do pupils connect studying science with future careers? 152 9.2 What types of careers interest pupils? To what extent are 15-year-olds interested in a career in science? 156 9.3 What are the characteristics of pupils who plan to attend university? What factors are associated with their plans? 161 Chapter 10. Pupils’ experiences of learning science in school 165 10.1 How much time do pupils spend studying science per week? How does this compare to other subject areas? 167 10.2 What activities take place in science classrooms in Wales? Is this similar to other countries? 172 10.3 Is low-level disruption in science classrooms a more common occurrence in Wales than in other countries? 174 10.4 How do pupils in Wales perceive the feedback they receive from their science teachers? 177 4 10.5 Do pupils in Wales feel that they receive regular support from their teachers during their science classes? 179 Chapter 11. PISA in the UK 184 11.1 How do average PISA test scores compare across the UK? 186 11.2 What proportion of 15-year-olds across the UK do not have basic science, mathematics and reading skills? 188 11.3 How have average PISA scores changed across the UK since 2006? 191 11.4 How has the performance of the highest and lowest achieving pupils changed across the UK since 2006? 192 11.5 Are gender gaps in achievement bigger in some parts of the UK than others? 195 11.6 How does the relationship between socio-economic status and achievement vary across the UK? 197 11.7 How do headteachers’ views on the factors hindering instruction differ across the UK? 199 11.8 Are there differences across the UK in the amount of instruction 15- year-olds receive - both inside and outside of school? 201 Appendix A. Background to the PISA study 210 A.1. Development of the study 210 A.2. What does PISA measure? 211 A.3. What do the PISA proficiency levels mean? 215 A.4. The PISA test design 215 A.5. Administration 217 Appendix B. Sample design and response rates 222 Appendix C. Testing statistical significance in PISA across cycles 227 Appendix D. The conversion of PISA scores into years of schooling 230 Appendix E. The PISA proficiency levels 231 Appendix F. Revisions made to PISA 2012 scores 235 5 List of figures Chapter 2 Figure 2.1 Mean PISA science scores for Wales between 2006 and 2015 .............. 40 Figure 2.2 The percentage of pupils in Wales reaching each PISA science level .... 43 Figure 2.3 The percentage of top-performing science pupils compared to mean PISA science scores: a cross-country analysis ................................................................. 44 Figure 2.4 The 90th percentile of PISA science scores for Wales between 2006 and 2015 ......................................................................................................................... 47 Figure 2.5 The 10th percentile of PISA science scores for Wales between 2006 and 2015 ......................................................................................................................... 49 Figure 2.6 A comparison of the 90th to 50th percentile and 50th to 10th percentile science achievement gap across countries .............................................................. 52 Chapter 3 Figure 3.1 The ‘slope face investigation’ item .......................................................... 65 Figure 3.2 The percentage of pupils who answer the slope face investigation question correctly across countries .......................................................................... 67 Figure 3.3 Proportion of pupils answering the ‘bird migration’ question correctly versus average PISA science scores ....................................................................... 69 Chapter 4 Figure 4.1 Mean PISA mathematics scores for Wales between 2006 and 2015 ...... 74 Figure 4.2 The proportion of pupils reaching each mathematics proficiency level ... 76 Figure 4.3 The percent of top-performing pupils in mathematics compared to mean PISA mathematics scores: a cross-country analysis ................................................ 77 Figure 4.4 The 90th percentile of PISA mathematics scores for Wales between 2006 and 2015 .................................................................................................................. 80 Figure 4.5 The 10th percentile of PISA mathematics scores for Wales between 2006 and 2015 .................................................................................................................. 82 6 Chapter 5 Figure 5.1 Mean PISA reading scores for Wales between 2006 and 2015 .............. 88 Figure 5.2 The proportion of pupils in Wales reaching each PISA reading level ...... 91 Figure 5.3 The percentage of top-performing pupils in reading compared to mean PISA reading scores: a cross-country analysis ........................................................ 92 Figure 5.4 The 90th percentile of reading scores for Wales: 2006 to 2015 ............... 93 Figure 5.5 The 10th percentile of PISA reading scores for Wales between 2006 and 2015 ......................................................................................................................... 97 Chapter 6 Figure 6.1 The average PISA reading score for boys versus girls ......................... 106 Figure 6.2 Average PISA scores for boys and girls in Wales since 2006 ............... 107 Figure 6.3 The ‘impact’ and ‘strength’ of the relationship between socio-economic status and PISA science scores ............................................................................