LTC14-P108 11 December 2014

Learning and Teaching Committee

Paper Title: Transforming the Experience of Students Through Assessment (TESTA)

Author: Dr Hardeep Basra (Quality Enhancement Officer, Centre for Academic Practice)

1. Specific Decision A paper to note Required by Committee ADTs are asked to consider the adoption of TESTA 2. Relevance to University Building Excellence Strategy Raising standards and aspirations: “our focus will be on……. a high quality student experience”. Growing capacity and influence: “we will listen to our students……to ensure that our academic provision develops in line with their requirements”.

3. Executive Summary TESTA is a methodology designed to address assessment and feedback issues at the programme-level. It is built on a robust, triangulated research methodology with qualitative and quantitative elements, and underpinned by educational principles and research literature.

The main aim of TESTA is to enhance the student learning experience from assessment by providing evidence to programme teams about assessment and feedback patterns and to help teams to identify ways of improving assessment design in the interests of better learning outcomes. The approach of TESTA has been to collect programme data, analyse and collate this into a readable case study, and engage in a conversation with the whole programme team about the findings.

TESTA has been implemented on over 100 programmes across 40 national and international universities, which reflects the level of success the approach has achieved.

The aim at is to pilot the approach and accordingly the paper provides information about TESTA and outlines the merits for adopting it. 4. Essential Background Jessop, T, McNab, N. And Gubby, L. (2012) Mind the gap: An analysis of how quality Information assurance procedures influence programme assessment patterns. Active Learning in Higher Education. 13(3) 143-154. Available at: http://alh.sagepub.com/content/13/2/143.full.pdf+html Jessop, T, El Hakim, Y. and Gibbs, G. (2011) The TESTA Project: Research Inspiring Change. Educational Developments 12(4), 12-16. Jessop, T, Lawrence, P. and Clarke, H. (2011) TESTA: Case Study of a Change Process. BA Primary, . Available at: http://escalate.ac.uk/studentfeedback 5. Risks, Risk Mitigation N/A and Governance/ Accountability 6. Implications for other Curriculum Review activities Programme Review/Validation

7. Resource and Cost N/A 8. Alternative Options N/A considered 9. Other Groups/Individuals Morag Bell, Nick Allsopp and Carol Robinson, TESTA consultants, University of consulted. Birmingham 10. Future Actions, ADTs to consider the adoption of TESTA. If interested then please contact Hardeep Timescales & Frequency Basra at the Centre for Academic Practice to discuss further. of Review by this Committee. 11. Success Criteria (KPIs) N/A 12. University Executive comment (required for Council papers only)

Transforming the Experience of Students Through Assessment (TESTA)

What is TESTA? It started out as a National Teaching Fellowship project on programme assessment, led by the University of Winchester (2009-2012). TESTA’s objective is to improve the quality of student learning through addressing programme level assessment, which was done by mapping out assessment environments, developing interventions and evaluating them. It was £200,000 project that was done in partnership with four universities: University of Winchester, , University of and the .

TESTA Premise: • Assessment drives what students pay attention to, and defines the actual curriculum (Ramsden 1992) • Feedback is significant (Hattie, 2009; Black and William, 1998) • Programme is central to influencing change

What is the problem TESTA is addressing?

Assessment innovations at the individual module level often fail to address assessment problems at the programme-level, some of which, such as too much summative assessment and not enough formative assessment, are a direct consequence of module-focused course design and innovation. QAA course specification requirements (of outcomes and criteria) at the module level have not succeeded in clarifying for students the goals and standards they should be orienting their overall effort towards. There needs to be more consistency between modules, across programmes, and a greater emphasis on progressively developing students’ internalisation of programme-level standards, over time, rather than relying on documentation to specify criteria at the level of assignments or modules. Co-ordinated programme-wide assessment policy and practice is required to address both these.

How does TESTA address this problem?

TESTA maps programme level data from degree programmes. The aim is to collate information that provides an overview of assessment- the quantity of assessment, balance of formative and summative, variety, distribution of assessment and its impact on student effort, feedback practices, the clarity of goals and standards, and the relationship between these factors and students’ overall perception of their degree. Using this baseline data, programme teams are devising targeted interventions to address specific programme-level assessment issues. Simultaneously, the project is deepening understanding of the relationship between quality assurance frameworks, and programme assessment changes through high- level strategic engagement with senior managers within universities.

TESTA Methodology: 1) Programme Assessment Audit; which maps the student experience over a three year degree programme • Analyse programme documentation and discuss with programme and module conveners to identify the characteristics of the programme e.g. • Analyse marks from coursework and exams • Identify the number of times student work is marked summatively (contributing to degree classification or pass-fail conditions) • Quantify the amount of formative only work • Quantify the amount of oral feedback (where possible) • Estimate the word count of written feedback (where possible) • Estimate the timeliness of return of assessed work • Map the sequence of assessment tasks across modules and levels and specify how assessment works across a programme i.e. formative assessments feeding forward into subsequent tasks and provision of practice for developing performance. • Report on the programme assessment characteristics • Establish a student’s perspective on assessment

2)Assessment Experience Questionnaire (AEQ); which measures variables like the clarity of goals and standards, the quality and quantity of feedback, and student approaches to learning, and their overall satisfaction. Focus Groups with students which provide a rich picture of student experience on the degree programme.

3) Programme Team Meeting Analyse assessment profile, AEQ and focus group data and use the summary to target appropriate interventions/enhancements.

4) Evaluate the impact of any changes made

Main TESTA findings prior to implementing the TESTA programme:

• Huge variations in practice • Emphasis on summative assessment • Lack of formative assessment • Huge variety of assessment, un-sequenced • Students describe lack of common standards and approaches • Students not clear about goals and standards

TESTA Success:

Over 100 programmes across 40 universities, national and international implementation and success; 35 universities in the UK that have used/using TESTA, some of which include:

University of Birmingham UCL Exeter University London Met University of West Scotland Liverpool John Moores Glasgow University Sheffield Hallam Queen Mary University of New South Wales University of Delhi Durban University of Technology

How are Universities using TESTA?

The Universities that have engaged with TESTA have not adopted it across their whole institution instead TESTA is being used at individual programme level dependent upon interest and need. It has applied to both undergraduate and postgraduate programmes. Deciding which programmes undergo the TESTA approach can be influenced by the following: • Programme leaders interest in learning and teaching • NSS scores, suggesting need for change in assessment and feedback • Institutional support • Wider University strategy • Part of other university processes (periodic review, validation process) • Resources

Many programmes that applied the TESTA methodology requested to undergo the process after learning about it at institutional level via workshops, presentations and success on other programmes. Many institutions have started off small and then cascaded on the back of its initial success (for example, , started on 3 programmes, now 28 programmes have completed TESTA and many more have been influenced and informed by its principles and findings). Some Universities have adopted faculty/departmental wide approaches (UCL, Exeter). The HEA Change Programme 2011/2012 has also been an influential factor in increasing the adoption of TESTA.

Who could be involved in a TESTA project: • Programme Leaders/ADTs • Centre for Academic Practice • TESTA Team (external consultants) • Students (as participants and as researchers) Note: Those who could be involved will vary according to the approach adopted.

TESTA and Student Engagement: TESTA gives a platform for the student voice as it allows them to have a meaningful mechanism to reflect on their experiences about assessment and feedback. The student voice is recorded through questionnaires and/or focus groups. There is also the possibility for students to participate in the research process. For example students can act as changes agents (Exeter), students as democratic partners (Bath), students as educational researchers (Winchester), students as collaborators.

Why should Loughborough engage with TESTA? • Improvements in NSS scores on assessment and feedback on all TESTA programmes • TESTA can help make evidence-informed and programmatic enhancements to assessment and feedback • Several programmes with 100% satisfaction ratings post TESTA • TESTA provides a platform for student engagement, and compliments QAA definition of student engagement and the university desire for students as partners in learning • TESTA has delivered improved NSS scores on student engagement, average 2% increase, highest increase 7%. • TESTA can help reduce the workload of both student and staff • TESTA aims to make assessment more effective in order to engage and motivate students in their learning efforts • TESTA engages with Quality Assurance processes • The data collection stage of the TESTA process can be undertaken within existing university structures, as long as the appropriate TESTA training is undertaken. Relative cost is low. • The TESTA experience has been described as ‘positive, painless and valuable’ • TESTA has the potential to raise the institutional awareness and be a trigger for change of pedagogical practice • TESTA squarely fits in with the University strategy for ‘Building Excellence’, in particular in recognising that in order to deliver excellence, programmes of study, curriculum design and content delivery will be essential to this strategy.

What next: 1. Need to identify a number of programmes that would pilot the TESTA approach. Could report to LTC or do a staff workshop on TESTA. (School of Business and Economics, The Design School and the School of Electronic, Electrical and Systems Engineering-conversations with ADTs at these schools have identified these as potential candidates) 2. Set up a university project board/steering group 3. Arrange a training workshop on TESTA methodology, cost £600 for one day or £2000 for two day training session for multiple staff and student researchers (includes working knowledge of all elements of data collection and ‘aftercare’. 4. Work with the TESTA team and in particular with Tansy Jessop, TESTA project leader, and arrange an initial meeting.

References: Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education. 1(1): 3-31. Gibbs, G. & Dunbar-Goddet, H. (2009). Characterising programme-level assessment environments that support learning. Assessment & Evaluation in Higher Education. 34,4: 481-489. Hattie, J. (2007) The Power of Feedback. Review of Educational Research. 77(1) 81-112. Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large- scale study of students’ learning in response to different assessment patterns. Assessment and Evaluation in Higher Education. 39(1) 73-88. Jessop, T, McNab, N & Gubby, L. (2012) Mind the gap: An analysis of how quality assurance processes influence programme assessment patterns. Active Learning in Higher Education. 13(3). 143-154.