Supporting Schools With The Use Of Data

Explanation of Data Packs, Access and the Timetable

This document provides a brief description of the performance analyses produced on behalf of the Secondary School Data Group and how they can be accessed. All of the analyses and comparative reports are available on your secure Individual School Web Area (http://www.cornwall.gov.uk/csfdata). Usernames and passwords were issued to the Headteacher, please contact us with any problems. We hope that you find these web areas a user-friendly way of referencing the data packs.

Contents

1. Secondary School Data Group – background and membership

2. Description of analyses and availability on web areas

2.1 Explanation of Data Packs 2.2 Self-Evaluation reports 2.3 School Context 2.4 Key Stage 2 & 3 2.5 Key Stage 4 (GCSE and equivalent) 2.6 Key Stage 5 (A-level and equivalent)

3. Data definitions and calculations - differences at school, LA and nationally

4. National examples of good practice in using data (Extracts from “Releasing Potential, Raising Attainment: Managing Data in Secondary Schools”

4.1 Who should get what data and why? 4.2 What the LA should provide the school 4.3 Managing the process throughout the year

Ben Crowe Statistics Team October 2011 Explanation of Data Packs and Access

1. Secondary School Data Group – background and membership

The Secondary School Data Group was set-up initially to respond to issues raised by the government's proposals for publishing performance tables and to 'add value' to them. The Data Group has developed into a pro-active group dedicated to providing useful information (data and analysis) to schools to help in their self- evaluation. The work is informed by national developments, and the group is continually reviewing and improving a wide range of comparative data produced at both school and pupil level in areas such as finance, staffing, departmental performance, pupil progress and estimates (LA and Fischer Family Trust) and analysis using P Scales. Secondary schools also have access to national data and comparisons through RAISEonline and FFT Live data comparing the school with the national pupil performance information. There will also be a variety of data available within the school to help with pupil tracking and progress monitoring. From 2009, CASH have agreed to share the information with school names rather than the school codes used previously which were confidential to the Headteacher.

Feedback is welcomed on any analysis, either through the Data Group members or on an individual basis. The members of the Data Group are Heather Jenkins (Looe Community School), Sandra Critchley (Falmouth School), Bill Marshall (Humphry Davy School), John Morey (Chair), an FE College representative for post 16 discussions, Ben Crowe, Nicky Hill and Kevin Shailes.

2. Description of data analyses and availability on web areas

The main change to the secondary web area is a reduction in the number of headings on the home page, providing an explanation section, copies of self evaluation reports (such as RAISEonline reports), school context reports and analyses by Key Stage. Within each of the analyses by Key Stage there is a section on historical performance, value added and predictive analyses, thereby providing more consistency in the way the analyses are structured within each Key Stage.

2.1 Explanation of Data Packs and Timetables

The latest version of this explanatory document, as well as examples of good practice in using data, and details of the LA P-Scales exercise.

2.2 Self-Evaluation reports

Copies of the OFSTED RAISEonline Full and CVA Exceptions reports, the post 16 PANDA and LA Stats Team RAISE Online training materials. Also copies of the Fischer Family Trust Self-Evaluation reports.

2.3 School Context

Deprivation analysis, the latest School Forecast and Budget NOR figures and School Census data reports on staffing, numbers on roll and school contextual data.

Consistent Financial Reporting benchmarking data compiled by the CSF Accountancy team.

Behaviour and Attendance Partnerships Data Mapping plots whole school annual absence (%) against the number of fixed exclusions as a percentage of the January School Census number on roll (academic year). Whole school annual absence uses total number of sessions missed both authorised and unauthorised divided by the total number of sessions possible for the School up to May Half term.

2.4 Key Stage 2 & 3

Includes Historical Performance and Predictive Analyses. 2.4.1 Key Stage 2 & 3 Historical Performance

Key Stage 2 into secondary summary report presents the distribution of pupils entering the school according to their Key Stage 2 results. The data is calculated from a live pupil results database and reflects pupil migration.

Key Stage 3 ‘School’ summary report (Teacher Assessments from 2009) presents a breakdown of the individual school results one year at a time (core and non core subjects) alongside the Cornwall LA and national summary results, for the whole cohort and by gender. A further analysis (up to 2008) and Teacher Assessments (several years compared) provides charts of the percentage achieving levels 5+, 6+ and Average Point Scores in English, Reading, Writing, Maths and Science for all schools. Key Stage 3 monitoring sheets provide detailed historical breakdowns of the individual school results (core and non core subjects) for several years, including target information in comparison with Cornwall and national results.

2.4.2 Key Stage 3 Predictive Analysis

Fischer Family Trust Estimate Analyses at school and pupil level present estimates of the likely attainment in the key summary measures at KS3 based on prior attainment. Fischer Family Trust Type B predictions are based upon individual pupil’s prior attainment, gender and a variety of other contextual pupil and school data using contextualised statistical models. The DCSF no longer consider type B estimates as being sufficiently challenging and are recommending you consider type D estimates. This is because type B estimates are based upon assumptions of past performance rather than improvement. We will provide all the estimate types at school level and type D at pupil level electronically on RM SecureNet.

LA Target Setting provides copies of your school's Key Stage 4 target setting proforma and combined rationale questionnaire and Absence proforma previously used in the target setting process. This page also provides access to a National Strategies Integrated Dataset spreadsheet showing a wide range of historic performance measures and measures of progress, including detailed information on 3 and 4 levels progress and CVA by school and with Cornwall and national comparison.

DfE Statutory Target Setting and the Fischer Family Trust (FFT)

Recent changes to legislation have removed the requirement for schools to provide targets to the DfE for attainment, progress and absence (via the Local Authority), however target setting is still important for schools. In a recent email to schools that we sent we attached a briefing from FFT discuss the merits of continuing to use the estimates they provide alongside other information and knowledge in the school to enable realistic and challenging targets to be set for individuals or cohorts.

“…target setting itself is still at the heart of good planning to support continuous school improvement and is intended to challenge expectations for both pupils and schools”

We’ve made further background materials available highlighting the information that FFT provide at http://www.cornwall.gov.uk/fft to help with making the best use of the information. 2.5 Key Stage 4 (GCSE and equivalent)

Includes Historical Performance, Value Added and Predictive Analyses.

The provisional GCSE/GNVQ results arrive from the National Consortium for Examination Results (NCER) in October. Individual school results and the LA summaries are available to Headteachers on the Individual Secondary School Web Areas. The reports are based upon data supplied to the National Consortium for Examination results (NCER) by the examination boards.

2.5.1 Key Stage 4 Historical Performance

RAISE Online Relative Performance Indicators (RPI) - the analysis included is a graphical representation of the Relative Performance Indicators (RPI) from RAISEonline. These are being plotted in chart form to replace the NCER residuals, also included is the original RPI report from RAISE online in excel format. There are two types of plot included, the first which is the School by Subject RPI plot, which shows all the (full course non-Vocational) GCSE’s Relative Performance Indicators within a school. The school by subject analysis is a comparison of how well each pupil who took a particular subject achieved in that subject, compared to how these pupils achieved across all their subjects, with the difference between the school measure and the national measure being the relative performance indicator. The second plot is the Subject by School RPI plot, which shows one subject with all schools which had pupils sit this subject.

School comparisons (including Graphical Trends) includes Provisional Performance Summary report (GG1) and Performance Summary Trend Report (TR401) for various GCSE attainment measures for your school with LA and national comparisons as appropriate. Also includes Reporting by Groups (GCG1) of pupils by SEN Stage, Free School Meals (FSM) and ethnic group in GCSE/GNVQ measures and Floor Standards Reports (FLS401).

Subject comparisons includes summary of grades by subject (G3 and G2) for all schools in Cornwall and the LA and National grade distribution. English and Maths Performance Summary (GEM) including level 1 and level 2 qualification measures. The New Gold Standard report – English and Maths Combinations concentrates on different combinations of near misses of 5+ A*-C inc Eng & Maths. Several reports on the English Baccalaureate and it’s component parts are also included. 2.5.2 Key Stage 4 Value Added Performance

Progress Levels show the 3 levels and 4 levels progress measures for all 31 Secondary schools. This report consists of 3 sections, one for each of English, Maths and Science Subjects with an LA figure for comparison included at the bottom of each page. Levels to Grades Transition Tables are provided for English, maths and science, for all pupils, and by gender. Rather than looking at Average Point Score (across English, maths and science) they compare each pupil’s individual KS2 subject test level (on the vertical) with their KS4 grade in the same subject (on the horizontal). The tables are provided with numbers and percentages for your school, and with percentages for the LA and Nationally. From 2009 they include 3 levels of progress indicators. We also can now run the Scatter Plots which present the 2006 KS2 APS to KS4 Total Point score, 2006 KS2 APS to KS4 Average Points Score and 2006 KS2 APS to each most popular subject taken in 2011.

2.5.3 Key Stage 4 Predictive Analyses

Fischer Family Trust Estimate Analyses at school and pupil level present estimates of the likely attainment in the key summary measures at KS4 based on prior attainment at Key Stages 2 and 3. Fischer Family Trust Type B predictions are based upon individual pupil’s prior attainment, gender and a variety of other contextual pupil and school data using contextualised statistical models. Now that statutory school level target setting has been removed schools have more flexibility in considering the FFT data and how it fits with the context of their school in using the estimates to set pupil targets.

DfE Statutory Target Setting and the Fischer Family Trust (FFT)

Recent changes to legislation have removed the requirement for schools to provide targets to the DfE for attainment, progress and absence (via the Local Authority), however target setting is still important for schools. In a recent email to schools that we sent we attached a briefing from FFT discuss the merits of continuing to use the estimates they provide alongside other information and knowledge in the school to enable realistic and challenging targets to be set for individuals or cohorts.

“…target setting itself is still at the heart of good planning to support continuous school improvement and is intended to challenge expectations for both pupils and schools”

We’ve made further background materials available highlighting the information that FFT provide at http://www.cornwall.gov.uk/fft to help with making the best use of the information. 2.6 Key Stage 5 (A-level and equivalent)

Includes Historical Performance and Value Added Analyses.

The provisional GCE, VCE and GNVQ Advanced results arrive from the National Consortium for Examination Results (NCER) in October. Individual school results and the LA summaries are available to Headteachers on the Individual Secondary School Web Areas. The reports are based upon provisional data supplied to the National Consortium for Examination results (NCER) by the examination boards. From 2010, where possible Cornwall averages are presented for School 6th forms, FE Colleges and both combined.

The future of UCAS points on the EPAS reports is currently under debate, more focus seems to have been placed on the validation of the QCDA points by NCER.

2.6.1 Key Stage 5 Historical Performance

School comparisons includes a School Level Performance Summary (AA1) which contains percentage summary grades at A, A-B and A-E for candidates taking Any, 1 or more, 2 or more or 3 or more A-level equivalents and average QCA points per pupil and entry.

Subject comparisons includes summary of grades by subject (A3 and A2) for all schools in Cornwall and the LA and National grade distribution.

2.6.2 Key Stage 5 Value Added Performance (up to 2008 and available again from 2010)

Pupil level value added analysis as School and Cornwall Charts of the GCSE/GNVQ to A, AS or Vocational A level for secondary schools in Cornwall based on pupils who were in Year 13 in Summer who took A/AS matched to their GCSE/GNVQ examinations. We have now run Scatter Plots which present the KS4 APS to KS5 Total Point score, KS4 APS to KS5 Average Points Score and KS4 APS to each most popular KS5 subject taken in 2011. 3. Data definitions and calculations - differences at school, LA and nationally

The aim of the Secondary School Data Group has been to provide comparative analyses with other schools in Cornwall and with national averages. At times schools have queried the accuracy of the LA data, and the way that it is processed. We have investigated the differences between the data that the LA holds and the data that schools have, and concluded that the main differences often lie in the interpretation of the data, although the school will often have a more up to date source of the data due to the latest regrades etc.

Key Stage 3 Teacher Assessment results are collected direct from schools from 2009.

Since 2005 the GCSE results are published for those students at the end of KS4, i.e. being taught in Year 11, and not by age-group. Before 2005, the calculations used the number of pupils aged 15 as at 31st August in the previous year, who were recorded on the school’s DCSF School Census. A-Level results reported are calculated from the sum of achievements of candidates aged 17 or 18 who are at the end of two years of advanced study and who attempted at least one single A-level or equivalent. Where a subject taken at AS level is subsequently awarded at A-level, the AS result is discounted.

The analyses are always run on the provisional data and then updated with the final data when it is available. In order to help schools who query the LA’s data or calculations, the Statistics Team can produce, on request, the various lists of students and results that have been used in the analyses.

Key differences between EPAS and DfE Performance Tables/Statistical First Releases

There are often observed differences between a school’s results in EPAS, the Performance Tables and SFR data. NCER working with RM Data Solutions aim to provide a comprehensive and accurate data set to LAs, whilst balancing the need for timely data. NCER are pleased to continue to be able to provide LAs with the earliest national data set available as a result. Please see below for some key differences between these data sets:

 EPAS data comes from the Awarding Bodies which means that initially EPAS data will not contain adjustments to the cohort carried out as part of the DfE Schools Checking exercise. Performance Tables and SFR data comes from the DfE School Checking site, after schools have been able to check, appeal against and amend their results.  EPAS is pupil based, which means EPAS will only ever have one record for a pupil and his or her exam results. Whereas, the Performance Tables are centre based and can have multiple records for pupils involved in qualifications at different centres  DfE Performance Tables data does not include LAs such as the Channel Islands and Services Schools.  Prior Attainment bandings in EPAS use point scores derived from whole levels and therefore may differ slightly from Performance Tables Prior Attainment Groups which use Fine Grades. The EPAS Group is investigating the possibility of using fine grades in the future 4. National examples of good practice in using data

In the web area under Explanation of Data Packs and Timetables (see 2.1) various nationally available examples of good practice in using data are referred to, with a web link providing access to further examples. The following extracts are from the DfES booklet (October 2002), “Releasing Potential, Raising Attainment: Managing Data in Secondary Schools” (ref. DfES/0722/2002). This contains best practice, questions that schools should ask of the data and a timetable of who should do what and when. Extracts from this document are reproduced in the following pages and have been updated locally over time.

4.1 Who should get what data and why?

Once collected, data needs to be accessible to those who are going to make use of it. The following table summarises good practice:

Who Purpose Data Required

Senior Management Team  Pupils’ Key Stage 2 and 3 To agree cohort targets. levels, individual pupil targets, RAISEonline, FFT, other data To establish cohort baselines within the school.  Departments’ results, school To evaluate subject and teacher and national subject RPIs, effectiveness. subject attainment trends, class lists, previous analyses. To identify training priorities, specifically  Individual School Web Areas for enhancing teaching and learning  Pupils’ prior attainment scores. To identify underachieving individual and  Pupil progress analyses. groups of pupils

To identify underachieving groups of  RAISEonline, FFT. Head of Department / pupils.  NCER data and returns. Subject leaders / Heads  Individual School Web Areas of Year To identify pupils who are doing well and  School’s OFSTED report. those who are underachieving.  Pupil’s prior attainment scores. To agree department / cohort targets.  Pupil progress analyses. To evaluate subject and teacher effectiveness

To establish class / teaching group  FFT. Class / subject teachers baselines.  Pupils’ prior attainment scores. To set individual pupil targets.  Pupil progress analyses. To monitor pupil progress.

To identify underachieving groups of pupils.

To monitor own performance and appropriateness of T&L strategies used. 4.2 What the LA should provide the school

What a school should expect What a school should do Telephone on result’s day to seek Heads Share initial analyses with subject leaders to form reaction to the results and offer support if basis of initial report to SMT needed

By the beginning of the Autumn term Discuss analyses with SMT and Governors, and  Analyses of GCSE [including equivalent] share with subject leaders through: both within and between schools  Subject area meetings where pupil  Residual analyses of subject performance examination and test performance and both within and between schools progress is reviewed.  Trends of performance over time  A review of results and performance  Analyses showing how close schools came between: to meeting their targets.  Line managers and subject leaders  Subject leaders and subject teachers

By Autumn half term, schools should have Compare FFT data against their own initial received Indicative GCSE and Key Stage FFT analyses. data for its Year 10 and Year 8 cohorts  How closely do they match?  Can the school do better than this?  Which pupil’s will have to be successful in order for such targets to be met?  Which subject areas will have to improve?

In the second half of the Autumn term the LA By the time of this meeting the school should should discuss with each secondary school: have:  Its previous year’s examination and  Carried out a thorough review and self- test performance evaluation of its previous year’s examination  Subject strengths and areas of and test performance weakness.  Identified areas of strength and initiated  Progress against statutory targets for action to address areas of weakness the last academic year  Reviewed and monitored Year 9 and Year 11  The school’s own self evaluation of pupils’ progress towards this year’s statutory pupil performance based on targets RAISEonline and LA analyses.  Formed and initial view as to the statutory  The school’s own targets for that targets for pupils in Years 8 and 10 academic year a and its progress towards them  The school’s own targets for the next academic year.

Throughout the year the LA should be: Schools should:  Providing or facilitating CPD for secondary  Ensure that all teachers and managers attend school teachers and managers in the use relevant training so they may become and interpretation of pupil performance proficient in the use and interpretation of pupil data performance data  Supporting schools in challenging those  Analyse pupil performance data to identify subject areas where standards are not those subject areas where performance is not sufficiently high sufficiently high  Providing or facilitating appropriate ICT  … and initiate action to deal with such under support to schools achievement  Sharing good practice between schools.  encourage staff to attend network meetings 4.3 Managing the process throughout the year

The process of managing data is fundamentally linked to the cycle of the school year. The following table shows an example of how the process could be managed:

Data Generation Predicting and Target Setting Progress Review August GCSE and A Level results September Collate Key Stage 2 data Provisional analysis of Key Conduct initial analysis of GCSE and A from feeder primary Stage 2 and standardised test level results. schools data. Map outcomes onto teacher Test Year 7 pupils on predictions at individual pupil level. standardised tests [CATs Map GCSE results onto Key Stage 3 or reading tests]. results at individual pupil level

Construct historical review of attainment at GCSE October Analyse School and LA data, and RAISEonline and FFT when available Pupil and school Analyse KS 2 data and comparative information construct / compare school arrives in school via LA profile with national profile. from National Consortium for Use new progress measures Examination Results. to predict KS 4 results.

Analyse standardised test data to construct profile of pupil intake.

Use KS 2 and standardised test data to predict KS 3 and GCSE results.

Compare non-verbal and verbal reasoning scores November Internal assessment data Conduct formal review of progress in on Year 8 and 10. Key Stages 3 and 4 using up-to-date progress measures.

Construct historical review of progress.

Conduct formal review of relative performance of subjects at GCSE. December Finalise pupil and cohort targets within school.

January Trial/ mock GCSE Analyse trial examination results to examinations assess progress towards targets. February March April May June Teacher assessments for Key Stage 3. July Provisional analysis of Key Stage 3 results.