Unlocking the Impact of Tertiary Teachers' Perceptions of Student

Research Report Unlocking the impact of tertiary teachers’ perceptions of student evaluations of teaching Sarah J. Stein, Dorothy Spiller, Stuart Terry, Trudy Harris, Lynley Deaker and Jo Kennedy Authors Sarah Stein, Dorothy Spiller, Stuart Terry, Trudy Harris, Lynley Deaker, Jo Kennedy Publishers Ako Aotearoa National Centre for Tertiary Teaching Excellence PO Box 756 Wellington 6140 This project was supported through the Ako Aotearoa National Project Fund 2009, in the Research and Implementation Projects funding stream. The Guide Using student evaluations to enhance teaching practice: Closing the loop (Stein et al., 2012) complements this Research Report. Published September 2012 Key Words student feedback; formal institutional evaluations/appraisals systems; teacher perceptions; worth of student evaluations; perceptions of limitations; engagement with and use of student feedback; professional development; feedback to students aligning policy, perceptions and practice. http://akoaotearoa.ac.nz/student-evaluations ISBN: 978-1-927202-15-9 This work is published under the Creative Commons 3.0 New Zealand Attribution Non- commercial Share Alike Licence (BY-NC-SA). Under this licence you are free to copy, distribute, display and perform the work as well as to remix, tweak, and build upon this work non-commercially, as long as you credit the author/s and license your new creations under the identical terms. Key Words student feedback; formal institutional evaluations/appraisals systems; teacher perceptions; worth of student evaluations; perceptions of limitations; engagement with and use of student feedback; professional development; feedback to students aligning policy, perceptions and practice. Executive Summary Introduction This study explored the extent to which teacher thinking and practices at all stages of the teaching and learning cycle are informed by learner feedback gathered through the institutional student evaluation process. This process includes the standard student evaluation/appraisal of teaching and course questionnaires and the associated policies, systems and procedures administered centrally by institutions. This investigation of the influence of student evaluations/appraisals on teacher professional development and student learning aimed to contribute directly to Ako Aotearoa's National Project Fund 2009 priority area of Strategic Professional Development Initiatives. A core goal of this project was to provide insights to assist tertiary institutions and individuals in order to strengthen the relationship between student feedback, teaching development and the quality of the student learning experience. Origins of the Research and Research Goals The researchers’ experience of working with staff in their own institutions prompted a number of questions around teacher views about, and use of, student evaluations/appraisals. These questions included the potential impact of teacher perceptions of the institutional use of evaluations/appraisals data for quality monitoring purposes and for promotion considerations. Other sentiments that the researchers wanted to probe were academic suspicion of students’ capacity to make judgements about their learning and scepticism about the quality of evaluation instruments. Additionally, the research team was motivated by a concern that there might be major shortcomings in teacher engagement with student feedback from formal evaluation processes and that student evaluation/appraisal data were not broadly used to inform teaching development and the quality of the student learning experience. iii Finally, the researchers were interested to find out to what extent students were included in ongoing dialogue around evaluation/appraisal. These initial questions and concerns provided the genesis for the study and evolved into the broad research question: How do current evaluation processes and practices influence teachers’ thinking and behaviours in relation to student learning at all stages of the teaching and learning cycle? and the contributing sub-questions: 1. What are the perceptions that tertiary teachers hold about student evaluation/appraisal? 2. What factors (causes, influences) affect these views? 3. How do tertiary teachers engage with student evaluations/appraisals? Design and Methods In order to investigate teacher perceptions and engagement and use of student evaluation feedback, the study explored tertiary teachers’ perceptions about evaluations/appraisals in three institutions, the University of Otago (OU), The University of Waikato (WU) and the Otago Polytechnic (OP). The researchers were academic developers and appraisals personnel from these three institutions. An interpretivist research approach framed the study that drew on a combination of quantitative and qualitative data, including a questionnaire and interview. The research questions, the questionnaire design and the interview questions were informed and sharpened by a literature review, as well as an environmental scan of evaluation policies and practices made public by New Zealand polytechnics and universities through their websites. The literature review focussed, in particular, on questions of teacher perceptions of, and engagement with, student evaluations and provided a benchmark against which the researchers evaluated the research findings that emerged as the study progressed. The broad trends in the literature indicated a reasonably positive disposition among academics towards undertaking formal evaluations/appraisals. They also reinforced the caveats that had been reported in the institutions participating in this study and suggested that engagement with, and use of, student evaluation data was very limited, lagging well behind the apparently positive perceptions. An online questionnaire was run with academic staff over a period of three weeks. In all, 2426 teaching staff from across the three institutions were invited to respond to the questionnaire and 1065 responses were received (44 per cent). Statistical analysis was carried out on the responses to the Likert-style questions of the questionnaire and thematic analysis was used to investigate responses to the open comment questions. Semi-structured interviews were subsequently conducted using questions that were developed on the basis of the themes that emerged from the questionnaire findings. Sixty interviewees were selected from volunteers – 20 from each institution – to provide as broad a sample as possible in terms of academic discipline, career stage and seniority. The interviews enabled the implications of the themes that became evident from the questionnaire data to be explored in more detail on a one-to-one basis. As with the questionnaire comment data, thematic analysis was used to draw out key themes from the interviews. Through an analysis of the findings from all across all data sets, in the light of the literature, conclusions could be made about perceptions held by staff between and across all participating institutions. Findings Sub-questions 1 and 2: Teacher perceptions of evaluations/appraisals and reasons for these views In line with the literature review, and somewhat counter to the researchers’ initial hunch, there was widespread recognition in the questionnaire responses that participants thought that it was worthwhile collecting evaluation/appraisals data (73 per cent). The most commonly identified beneficial reasons were to inform teacher and/or course development (19 per cent) and identify students’ learning needs (19 per cent). Of the limitations identified, the most common included: shortcomings in the current evaluations/appraisals system (14 per cent); quality of student responses (10 per cent); and the use of the same instrument for quality and development purposes (seven per cent). The relatively small numbers of respondents who expressed a concern about the dual purposes of evaluations/appraisals was a surprise to the v researchers, who had received a different subjective impression from their contact with academic staff. The 60 interviews provided further illumination about academics’ perceptions of evaluations/appraisals, thereby reinforcing the findings of the questionnaire and suggesting some other aspects that needed further exploration. A strong reservation voiced in the interviews at the two universities (nine at OU; 11 at WU) concerned the quality of student feedback, with a smaller number questioning the quality of student judgements at OP (six). At the universities there was also concern about limitations in the institutions’ evaluation/appraisal survey instruments (eight at OU; eight at WU), but this was not a noticeable concern in the OP interviews. Another interview finding that was raised by OU interviewees was the potential manipulation of the evaluations/appraisals process by academics (eight). Other problems mentioned were timing of evaluations/appraisals, and the associated unease about institutional use of the data (eight at OU; three at WU; and two at OP), including a concern about institutional reliance on one evaluation source to base assumptions and decisions. A contextual difference that emerged in interviews with OP interviewees was that small classes and close contact in skills-based teaching meant teachers had many opportunities to gather informal feedback on their students’ learning and consequently formal evaluation/appraisal data were seen as less significant. The interviews purposefully explored two additional areas, namely, teaching philosophy and the personal response that may

Load more