Final report 2011 Peer review of online learning and teaching Lead institution: University of South Australia Partner institution/s Edith Cowan University Griffith University Lancaster University (UK) Monash University Queensland University of Technology RMIT University University of Southern Queensland University of Tasmania Project leader: Denise Wood [email protected] Project team Martin Freney UniSA Marty Friedel Programmer Richard Lamb UniSA Sheila Scutter UniSA Dale Wache UniSA Kirstin Wache Research Assistant Report Authors Denise Wood Sheila Scutter Dale Wache <www.unisanet.unisa.edu.au/peerreview> Peer review of online learning and teaching 1 Support for this project has been provided by the Australian Learning and Teaching Council Limited, an initiative of the Australian Government. The views expressed in this report do not necessarily reflect the views of the Australian Learning and Teaching Council. This work is published under the terms of the Creative Commons Attribution- Noncommercial-ShareAlike 3.0 Australia Licence. Under this Licence you are free to copy, distribute, display and perform the work and to make derivative works. Attribution: You must attribute the work to the original authors and include the following statement: Support for the original work was provided by the Australian Learning and Teaching Council Ltd, an initiative of the Australian Government. Non-commercial: You may not use this work for commercial purposes. Share Alike: If you alter, transform, or build on this work, you may distribute the resulting work only under a licence identical to this one. For any reuse or distribution, you must make clear to others the licence terms of this work. Any of these conditions can be waived if you get permission from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-sa/3.0/au/ or send a letter to Creative Commons, 171 Second St, Suite 300, San Francisco, CA 94105, USA. Requests and inquiries concerning these rights should be addressed to the Australian Learning and Teaching Council, PO Box 2375, Strawberry Hills NSW 2012 or through the website <http://www.altc.edu.au> ISBN 978-1-921856-43-3 2011 The project team acknowledges the following members of the University of South Australia editorial team who assisted with the final revisions and formatting of this report: Philip Alvaro Alexia Bushby Jessica Coveney Elyse Ellgar Phillip Guglielmo Carly Smith Judith Timoney Peer review of online learning and teaching 2 Contents Project leader: Denise Wood ..................................................................................... 1 [email protected] ......................................................................................... 1 Project team ............................................................................................................... 1 Martin Freney UniSA .................................................................................................. 1 Marty Friedel Programmer .......................................................................................... 1 Richard Lamb UniSA .................................................................................................. 1 Sheila Scutter UniSA .................................................................................................. 1 Dale Wache UniSA ..................................................................................................... 1 Kirstin Wache Research Assistant .............................................................................. 1 Report Authors ......................................................................................................... 1 Denise Wood .............................................................................................................. 1 Sheila Scutter ............................................................................................................ 1 Dale Wache ................................................................................................................ 1 <www.unisanet.unisa.edu.au/peerreview> ................................................................... 1 1 Introduction............................................................................................................... 6 2 Objectives ................................................................................................................ 6 3 Approach and methodology ...................................................................................... 7 Project management ............................................................................... 7 Approach taken ...................................................................................... 12 Evaluation plan ...................................................................................... 15 4 Peer review system ................................................................................................ 15 Content: banks, criteria and additional information .................................. 16 Educative aspects .................................................................................. 19 Inclusivity ............................................................................................... 20 Areas of focus ........................................................................................ 20 Use of the peer review system ................................................................ 23 Summary ............................................................................................... 25 5 Case studies ........................................................................................................... 25 6 Evaluation findings ................................................................................................. 26 Formative evaluation .............................................................................. 26 Summative evaluation ............................................................................ 35 7 Advancing existing knowledge ................................................................................ 50 Relationship to research in the area ........................................................ 50 Scholarly engagement ............................................................................ 50 8 Analysis of critical factors ....................................................................................... 51 Factors critical to the success of the approach ........................................ 51 Peer review of online learning and teaching 3 Factors that impeded success ................................................................ 52 Other comments ..................................................................................... 52 9 Applicability to other institutions .............................................................................. 53 Applicability of outcomes ........................................................................ 53 Applicability of approach ......................................................................... 54 10 Dissemination ....................................................................................................... 54 The publications and presentations arising from the project are as follows54 Peer reviewed journal publications .......................................................... 54 Peer reviewed conference papers ........................................................... 55 Engaged dissemination strategies ........................................................... 55 11 Links to other ALTC projects ................................................................................. 57 12 Outcomes ............................................................................................................. 58 Project outputs ....................................................................................... 58 Project outcomes ................................................................................... 58 13 Further action ....................................................................................................... 58 14 Conclusions and recommendations ....................................................................... 60 15 References ........................................................................................................... 62 15.1 Additional sources ............................................................................................. 64 16.1 Peer reviewed publications ................................................................................ 69 Peer reviewed journal publications .......................................................... 69 Peer reviewed conference papers ........................................................... 69 16.2 Additional presentations, workshops and fora .................................................... 69 16.3 Online peer review instrument questionnaire ................................... 71 16.4 Recruitment materials ........................................................................................ 79 16.5 Permission for participation of academic staff in trials ........................................ 84 16.6 Custom course evaluation questions for evaluation of student experience using the peer ..................................................................................................................... 88 review system for self and peer