NFER Education Briefings Key insights from PISA 2018 for the United Kingdom What is PISA? The Programme for International Student Assessment (PISA) is an important worldwide research study involving schools and pupils from 79 participating countries and economies. It is run by the Organisation for Economic Co-operation and Development (OECD) and takes place every three years. PISA is designed to examine how 15-year-olds can apply what they have learned in school to real life situations. Pupils are asked to use their skills of reasoning, interpretation and problem solving rather than simply remembering facts. Each cycle of PISA has a different focus. The main focus in PISA 2018 was reading, but it also included questions on mathematics and science1. PISA also gathers extensive background information about pupils’ home and school learning environments and experiences. This information can provide powerful insights into how well education systems are functioning, and how teaching and learning can be improved for all pupils in the UK and around the world. Who is NFER? What can PISA tell us about The lives of children and young people reading, mathematics and worldwide are inextricably linked to science in the UK? the amount and quality of education they receive. At NFER, our mission is to PISA is designed to assess pupils’ skills in improve outcomes for future generations reading, mathematics and science. everywhere and to support positive change across education systems. It allows us to draw comparisons with a large number of other countries from We do this by: around the world and, more importantly, to • creating and sharing research evidence monitor education in our own countries over and insights on education policy and time in a global context. practice PISA provides information to support • informing policymakers and other key improvements to education systems decision-makers worldwide and all four UK countries • providing insights, resources and support participate. to strengthen practice in the classroom. Making a positive impact on children’s and young people’s experience of education, their outcomes and on teaching and learning is at the heart of everything we do. In the majority of countries that participated in PISA, pupils who saw themselves as competitive scored higher in reading than pupils who perceived themselves as less competitive (OECD, 2019). Pupils in all UK countries expressed a high degree of disapproval of bullying behaviours (Sizmur et al, 2019). Internationally, the results from PISA 2018 showed that the more anti-bullying attitudes there were in a pupil’s school, the less likely they were to be bullied 1 (OECD, 2019). Some pupils in Scotland also answered questions on global competence. Interpreting the results It is important that the PISA results are interpreted appropriately, as a simplistic view can be misleading. The following points should, therefore, be borne in mind. Comparisons between countries Rather than focussing on the rankings of countries, it is more important to know whether countries are statistically similar or different. For example, a country may have an improved score in a particular subject, but may have fallen in the rankings due to the improved performance of other countries, or just the mix of countries in any given cycle. Changes over time An increase or decrease in score from one cycle to the next does not necessarily indicate that national achievement has improved or declined. It is crucial to consider whether a score is statistically different from previous scores and that changes have not arisen solely by chance. Moreover, any change in scores may not necessarily be attributed directly to national education policies, as it may take many years to understand the full picture. Further analysis is often needed to follow up initial headline findings. Variations in pupil achievement A simple average score does not provide a full picture of a particular country. For this reason, PISA also looks at the range of pupil achievement within countries. It examines the differences between the highest and lowest scores and the proportions of pupils working at high, medium and low international benchmarks, known as proficiency levels. For example, two countries could have the same overall mean score but one might have all its pupils demonstrating similar ‘average’ performance, while the other has high proportions of pupils achieving very high scores, but also high proportions with very low scores, perhaps not even reaching basic proficiency levels. This would suggest less equity across pupils. This kind of evidence can be used to identify particular groups of pupils where resources can be targeted most effectively. Comparisons with national exams PISA assessments are not curriculum based and use different methods from national examinations, as they have different goals and outputs. PISA is designed to look at the performance of education systems, not of individual pupils. Comparisons with national exams should be interpreted with some caution. Education policy PISA provides in-depth information about education systems, schools and pupils. It explores the relationship between contextual background factors and pupil achievement. This information can help countries to make evidence-based decisions, using PISA results alongside other sources of evidence to develop and adapt education policy. For more information visit www.nfer.ac.uk/international 3 • England • Northern Ireland How do the results compare across the UK? • Scotland • Wales Reading was the main Reading focus of the OECD 505 504 PISA study in 2018. scores 501 PISA conceives of reading as a broad set of competencies that allows readers to engage OECD average (487) with written information, 483 presented in one or Pupils in England, Scotland and Northern Ireland scored more texts, for a specific significantly higher than the average score across the OECD. The purpose. average score in Wales was similar to the OECD average but significantly lower than England, Scotland and Northern Ireland. Achievement gap in reading Proportions of low and PISA shows some variation in the size of the achievement high performers in reading gap in different countries. In reading, the gap between highest and lowest achievers was largest in England and smallest in Scotland. 17% England 12% Lowest (10th percentile) Highest (90th percentile) 372 262 634 Northern 368 255 623 18% 9% Ireland 383 244 627 15% 10% 359 250 608 Scotland 354 260 614 300 700 22% Wales 7% Reading trends over time Reading in Scotland has improved since 2015 23% OECD 9% while the trend is stable in other parts of the UK. average 506 505 504 Lowest Top performers: 500 500 500 499 performers: Pupils reaching Below Level 2 Levels 5 and 6 501 499 which is defined in reading are 496 498 497 by OECD as basic considered to be 495 495 proficiency. ‘top performers’ in 493 493* the global context. 491 490 487 486 England had the largest proportion 483 of high-scoring pupils (significantly 481 476 477 higher than the OECD average). 480 Scotland had the smallest proportion of pupils working at 2006 2009 2012 2015 2018 the lowest levels (below basic *The mean score of that year is was statistically proficiency). 4 different from the mean score in 2018. Pupils in England scored significantly higher Science Science in PISA 2018 than the other UK nations and the OECD Northern is defined by three scores average for science. Pupils’ scores in Ireland, Scotland and Wales were statistically competencies of similar to each other and to the OECD average. explaining phenomena 507 scientifically; evaluating and designing scientific enquiry; and 491 interpreting data and 490 evidence scientifically. OECD average 488 (489) Achievement gap in science Proportions of low and In science, the gap between highest and lowest achievers high performers in science was largest in England and smallest in Wales. 17% 10% Lowest (10th percentile) Highest (90th percentile) England 375 260 635 370 239 609 19% Northern 5% Ireland 366 251 617 371 232 603 21% Scotland 7% 365 244 609 300 700 20% Wales 5% Science trends over time OECD 22% 7% The trend in science was downwards in Northern Ireland, average Wales and Scotland while England has remained stable. 516 516 515 Lowest Top performers: 512 performers: Pupils reaching 515* 514* 513* Below Level 2 Levels 5 and 6 507 which is defined in science are 511* 508* by OECD as basic considered to be 507* proficiency. ‘top performers’ in 505* 500 the global context. 498* 498* 497 England had the largest proportion 495 496 491 of high-scoring pupils (significantly 490 higher than the OECD). Wales had 491 491 489 a significantly smaller proportion of 485 488 high-achieving pupils than across the OECD. England also had the 2006 2009 2012 2015 2018 smallest proportion of low-scoring pupils (significantly lower than the *The mean score of that year was statistically OECD) while other UK countries different from the mean score in 2018. were similar to the OECD average. 5 The highest attainment for mathematics was in The construct of Mathematics England, where scores were significantly higher mathematical literacy scores than other UK countries and higher than the used in PISA 2018 OECD average. Scotland, Wales and Northern describes the capacities Ireland were not significantly different from each other or from the OECD average. of individuals to reason 504 mathematically and use mathematical concepts, procedures, facts and tools to describe, 492 explain and predict OECD average 489 487 phenomena. (489) Achievement gap in mathematics Proportions of low and high In mathematics, the gap between highest and lowest performers in mathematics achievers was largest in Scotland and smallest in Wales. Lowest (10th percentile) Highest (90th percentile) 19% England 14% 383 240 623 377 223 600 20% Northern 8% Ireland 367 243 610 381 211 592 23% Scotland 11% 370 235 605 300 700 21% Wales 7% Mathematics trends over time England and Wales have improved their maths score OECD Scotland Northern Ireland 24% 11% over time, has declined and average has remained stable.