Strategy, Planning & Governance. Potential Teaching Excellence Framework Measures. Professor Wyn Morgan Dr Tony Strike February 2016 2 Potential Teaching Excellence Framework Measures Contents Introduction 3 1 Average Tariff 4 2 UG degree outcomes 6 3 Student/Staff ratio (SSRs) 8 4 Destination of leavers: Positive destinations 10 5 Salaries for new graduates 12 6 NSS: Overall Satisfaction 14 7 NSS: Teaching on my course 16 8 NSS: Learning resources 18 9 International Staff 20 10 Students no longer in HE after 1 year 22 11 Students from LPN (low participation neighbourhoods) 24 12 Applications per registration 26 13 Proportion from state sector 28 14 About the data 30 1 The University of Sheffield 2 Potential Teaching Excellence Framework Measures Introduction This is a consultation document for internal use only. The next step in the building of national proposals for a By providing a set of available measures with data, Strategy, Teaching Excellence Framework (TEF) is to be a technical Planning & Governance aims to stimulate an internal debate consultation. In anticipation of universities being invited to on which, if any, of these measures the University might seek submit comments later in 2016, the university can develop to support over others if metrics are still to be a part of and strengthen its own view. In the first year it is likely the TEF. By showing the data for the University of Sheffield, the TEF will be based on results from Quality Assurance. by comparing it to the rest of the Russell Group and to the Subsequently, it is proposed the TEF will include metrics sector, it may be possible to identify defining strengths and used as proxies to indicate teaching excellence. This necessary areas for improvement. suggestion has already produced a series of responses to the government’s Green Paper as to why this approach Working closely as we are with colleagues in the Department could be viewed as problematic. of Business, Innovation and Skills, who are based in Sheffield, on the development of the TEF, we recognise we do not The University of Sheffield Students’ Union has been have a determining influence on the national policy. We can clear that much of what students come to recognise as provide greater clarity on what the University of Sheffield educational excellence in higher education is in the ethos means by high quality teaching and educational excellence and values of the learning community, which is tangible but and how we intend to evidence that to others. resistant to objectification and measurement. In showing some possible proxy measures, we are preparing for the Results are shown here at an institutional level of analysis; TEF technical consultation without wanting to fall into aware that students often make decisions at the subject the epistemological trap of legitimising the object of our level and that there are subject level variances in the concerns. This pamphlet shows potential measures that may underlying data. be incorporated into the TEF, but does not propose that such measures are used. The commentary provided with Dr Tony Strike each measure offers a critical appraisal. 3 Potential Teaching Excellence Framework Measures 1. Average Tariff on Entry This is an input measure and might say something about the seen reputation of the University and its ability to attract academically able students. Being part of a student group which is academically able may of a measure of entry tariff might be to discourage access influence the quality of the educational experience for those in favour of elitism. If linked to learning gain; measured by students through the level and pace of the teaching and comparing degree classification to average entry tariff, through peer learning. However, if tariff is simply measuring the incentive could be for institutions to accept lower tariff reputation and the University is adding no value then teaching students and award more good honours degrees. excellence is not indicated by this measure. Universities which have a specific policy of accepting students with lower The Russell Group performs well on this measure. grades as part of an access policy will tend to have their The University of Sheffield does well on this measure in the average score depressed. The unintended consequence sector and sits in the middle of the Russell Group. 4 Potential Teaching Excellence Framework Measures Average Tariff Top 5 Russell Group 1 The University of Cambridge 601.0 1 The University of Cambridge 601.0 2 The University of Oxford 570.8 2 The University of Oxford 570.8 3 Imperial College of Science, Technology and Medicine 566.8 3 Imperial College of Science, Technology and Medicine 566.8 4 London School of Economics and Political Science 531.4 4 London School of Economics and Political Science 531.4 5 University of Durham 522.4 5 University of Durham 522.4 7 University College London 505.2 Sheffield +/- 3 8 The University of Bristol 484.6 21 The University of Manchester 430.6 9 The University of Edinburgh 482.2 22 The University of Leeds 429.8 10 The University of Warwick 479.7 23 Glasgow School of Art 426.6 12 The University of Glasgow 464.8 24 The University of Sheffield 425.6 13 The University of Exeter 460.2 25 University of Nottingham 425.4 15 King’s College London 452.3 26 The University of Birmingham 425.2 19 The University of York 433.9 27 University of Newcastle-upon-Tyne 423.6 21 The University of Manchester 430.6 22 The University of Leeds 429.8 Bottom 5 24 The University of Sheffield 425.6 149 Birkbeck College 221.2 25 University of Nottingham 425.4 150 Harper Adams University 220.0 26 The University of Birmingham 425.2 151 University College Birmingham 218.7 27 University of Newcastle-upon-Tyne 423.6 152 London Metropolitan University 217.7 29 Cardiff University 421.2 153 Institute of Education 203.6 32 Queen Mary University of London 408.8 33 The University of Southampton 407.5 35 The University of Liverpool 402.8 45 The Queen’s University of Belfast 378.8 Definition: Average tariff of all subjects taken; home students only Source: HESA sector data, 2013/14 5 The University of Sheffield 2. UG degree outcomes This is an output measure and might indicate the effectiveness of teaching at an institution in enabling students to achieve good degree results. The QAA ensures high academic standards across all Including undergraduate degree outcomes as a measure in higher education institutions, and so measuring degree the TEF could promote grade inflation, as institutions would classifications would demonstrate the number of students be able to succeed in the TEF by lowering the standards that achieved a certain standard of education. However, required for higher level degree classifications. It could also degree outcomes are influenced by the quality of students disadvantage those institutions that are successful in widening recruited by an institution, and does not indicate the quality participation, as there is evidence that some WP students are of the education that they receive or the value added to less likely to achieve higher degree classifications. a student’s education. While the QAA ensures standards, institutions still set and mark their own exams, so data The Russell Group performs well on this measure. on undergraduate degree outcomes is not comparable The University of Sheffield does well on this measure in the between institutions. sector and sits in the middle of the Russell Group. 6 Potential Teaching Excellence Framework Measures Good Honours Top 5 % Russell Group % 1 Courtauld Institute of Art 90.2 4 The University of Oxford 87.7 2 Central School of Speech and Drama 87.9 5 University of Durham 87.4 3 Royal Academy of Music 87.8 14 London School of Economics and Political Science 82.1 4 The University of Oxford 87.7 16 The University of Exeter 81.3 5 University of Durham 87.4 17 University College London 80.3 19 The University of York 79.8 Sheffield +/- 3 22 The University of Warwick 78.7 32 Royal Holloway and Bedford New College 75.9 25 The University of Leeds 77.8 33 Leeds College of Art 75.5 27 The University of Birmingham 76.6 34 The University of East Anglia 75.4 28 The University of Bristol 76.4 35 The University of Sheffield 75.4 31 The University of Southampton 76.0 36 Bournemouth University 75.2 35 The University of Sheffield 75.4 37 The University of Winchester 74.9 42 Imperial College of Science, Technology and Medicine 73.7 38 Bath Spa University 74.5 43 University of Nottingham 73.6 47 Cardiff University 71.9 Bottom 5 50 The University of Cambridge 71.1 148 Royal Conservatoire of Scotland 59.4 51 University of Newcastle-upon-Tyne 71.0 149 The Robert Gordon University 39.4 52 King’s College London 70.6 150 SRUC (Scotland’s Rural College) 30.4 55 The University of Manchester 70.1 151 University of the Highlands and Islands 19.8 58 The Queen’s University of Belfast 69.7 152 The University of the West of Scotland 19.3 68 The University of Edinburgh 67.8 76 The University of Liverpool 66.9 84 Queen Mary University of London 65.5 100 The University of Glasgow 62.9 Definition: Good honours defined as 1st and 2.1 Source: HESA sector data, 2013/14 7 The University of Sheffield 3. Student/Staff ratio (SSRs) There is evidence that students particularly value time spent learning in small groups. A greater proportion of staff to students might allow students Measuring SSRs could also reward inefficiency, as institutions to benefit from more direct contact with their teachers, could perform better in the TEF by assigning more staff to and allow staff a greater capacity to focus on the needs of teach than is optimal for student learning.