Pathfinder Programme Project Plan

University of Bradford

Version 2.0

1. Project Overview

1.1 Background

The University as a Pathfinder organisation

The University of Bradford Pathfinder bid suggested a number of ways in which the University demonstrated the characteristics of a Pathfinder organisation and these can be summarised with illustrative examples as follows:  Strong and sustained commitment to the use, integration and embedding of e- learning technology right across the range of educational provision to meet educational goals. The technology serves clear educational ends and is not an end in itself. For example, The University revised its Learning, Teaching and Assessment (LTA) Strategy in 2004/05 and made e-learning one of the eight major sections, with a clear emphasis on the need for the technology to meet educational goals.  Clear strategies and processes/procedures to support all staff and students in the effective use of current e-learning technologies and their further development. For example, at the strategic level, the emphasis in the LTA Strategy is reinforced in the University’s E-Strategy. At the process level, we have introduced a major programme of staff development (from training to project support and evaluation), and integrated the necessary technical and educational support through our Learning Technology Support Team.  Evidence of innovation in the application of learning technologies across subject areas on a significant scale and the active support of experimentation For example, current innovations and pilots across the University include all the major technologies which are prominent in contemporary discussion of e-learning trends, e.g. wikis, blogs, podcasting, social networking, student response systems, learning objects, and e-portfolios. Some of this work is externally funded (e.g. JISC, Subject Centres, CETLs)  Commitment to ongoing evaluation and systematic scrutiny of its educational practices and the dissemination of outcomes of such evaluation. For example, there are our previous internal evaluations (e.g. of the VLE) and our recent work on benchmarking.  Open communication, both across the institution and externally, which specifically encourages exchange and debate over the value of e-learning technologies in relation to educational goals. For example, as well as the opportunities for staff to engage in debates through workshops and seminars, committee structures and working groups have enabled broad consideration of these important issues. As well as demonstrating these features, we argued that the University of Bradford has some distinctive elements and combinations which will make our expansion of e-learning of particular interest and value to the sector, as listed below;  Coordinated Strategy development and implementation, supported by standard project methodology (the 'Bradford Method', a standardised methodology which has been specifically developed for application across the University and in all major

1 projects, based on Prince 2).  The cultural mix of our students and staff and our commitment to supporting diversity.  Our employability record. Alongside our strong record in widening participation, the University is currently rated as number I in the region and we intend to use e-learning to help us maintain this success.  The need to balance development with commercial provision. We are a small institution and therefore have to think very carefully about balancing our expenditure, e.g. between commercial and open source software.  New support structures. The ongoing reorganisation of our learner support services aims to provide models for effective practice and is discussed below under project rationale. The newly established Learner Development Unit is already piloting a range of technology-based solutions to enable students to access learning materials in a flexible and appropriate manner.  Work with DRC on the provision of support for disabled students re LTA.

The underpinning rationale for the Pathfinder project

Building on this context, our Pathfinder bid developed two major aims which are described in the next section. The relevant background to these aims is as follows:

Support for our model of the ‘extended student’

The University is currently re-organising its Learner Support Services to include a range of services for staff and students: learner and student development; careers and employability; counselling and disability services; library infrastructure and services; computing infrastructure and services; educational development and learning technology support; staff development; and the graduate school for postgraduate students. Many universities have put some of these services together but we believe that this precise mix is unique in UK HE. The main purpose of this reorganisation is to develop integrated services which meet learner needs and also provide the necessary staff support to adapt to new demands and expectations.

Alongside this reorganisation of service functions, current policy development includes work on the First Year Experience, student behaviour and engagement, retention, assessment, and pdp (with a particular emphasis on electronic progress files).

Building on these developments, and incorporating the lessons we have learned from our partnership in the LearnHigher CETL and the JISC-funded ELP project on progress files with local schools and colleges, we are working on the model of the ‘extended student’ and are starting to use e-learning to work with prospective students and to support new students from the point where they express a definite interest in our courses (incorporating insights from developments at other universities such as ‘Stepping Stones’ at Bournemouth and 'Getting Started' at Sunderland). The long-term aim is to develop a model of integrated support which covers the complete student life-cycle, which enables students to effectively manage important transitions, and which recognises that students are now operating in an increasingly sophisticated technological environment both inside and outside their university experience (e.g. as in Green and Hannon’s report on the digital generation, Demos, 2007, http://www.demos.co.uk/publications/theirspace ). But we must take care to develop a model which reflects the variety of student experience and backgrounds and is not just a recipe for a limited number of advanced ‘digital natives’. The aim therefore of this aspect of the bid is to develop systems and structures which use e-learning to support first year students (both

2 before they join the course and during that first year) and which provides the necessary co- ordination between the central and the specific course/programme provision.

Expanding and embedding support processes for e-Assessment

The National Student Survey has identified assessment methods and assessment feedback as important issues across the sector. At Bradford, these issues are now part of a debate which will lead to more comprehensive policy development regarding assessment. Based on a series of pilots, we believe that innovative e-assessment in general and computer-aided assessment (CAA) in particular can make an important contribution. However, while we have invested in the underpinning technology (both hardware and software) which should support the expanding demand for this service over the next few years, we do not yet have the physical facilities to make this easy to administer (e.g. large classes demand multiple sittings which are difficult to organise) and we have not yet developed the integrated administration which will enable this on a large scale. Projects in the E-Strategy are being developed to provide the support for physical facilities; the focus in this Pathfinder bid is developing the administrative and support systems, again with particular emphasis on support for the first year experience.

Relationship to national strategies

The University aims and aspirations are directly aligned with national initiatives such as the HEFCE Strategy for E-Learning and this Pathfinder project (alongside related activities in the University) maps directly on to many of the main success criteria from this strategy, as summarised in the table below;

Extracts from the HEFCE Criteria of Success Relationship to this Pathfinder bid and related University initiatives ‘ICT is commonly accepted into all aspects This Pathfinder project aims to of the student experience of higher education, integrate and further embed ICT with innovation for enhancement and flexible into student induction and into learning, connecting areas of HE with other assessment. aspects of life and work.’ ‘Due to more coherence and collaboration, This Pathfinder project directly technical issues have been addressed to give addresses important interfaces better value for money.’ which will enable technical systems to work effectively. ‘Students are able to access information, This is a prime motive which tutor support, expertise and guidance, and underpins our notion of the communicate with each other effectively ‘extended student’, e.g. where wherever they are.’ we aim to provide enhanced communication before the formal commencement of the course ‘Tutors have tools for course design to The embedding of CAA to enable better communication between them support both formative and and their students, giving feedback and summative assessment will targeted support.’ improve feedback and support. ‘Institutions are able to build appropriate The ‘extended student’ model infrastructure and resources support for demands high levels of system integrating registration and learning functions.’ integration.

3 ‘Lifelong learning networks support While regional networks are not connectivity between institutions to provide a specific Pathfinder focus, seamless access for students and staff. ‘ these will be developed as we investigate the most appropriate transition support. ‘Staff are supported at all stages to develop Appropriate staff development is appropriate skills in e-learning’ an important aspect of both our Pathfinder aims

1.2 Aims and Objectives

This project focuses on improving the way that the institution supports the teaching and learning experience of first year undergraduates through e-learning. It has two aims: a) To transform the learning experience of the ‘extended student’. This aim focuses on:  enabling students to manage the process of initial transition and engagement with the institution more effectively by using technology to help them identify their levels of confidence on entry to the University;  developing appropriate systems to ensure a bespoke programme of skills development can be identified, delivered and evaluated for each individual student.

To achieve this aim, a new website (www.brad.ac.uk/developme) will be launched which will make use of emerging Web 2.0 technologies to support this process. Materials will be broken down into 'chunks' of learning. Each of these re-useable learning objects will further incorporate the use LAMS to provide adaptive release of materials. Some objects will be made available via mobile devices for students to download and use on the go. The project will evaluate the effectiveness of these methods and build a model of effective learner support which demonstrates how central provision and course/programme provision can be effectively integrated. The use of an open source VLE (Moodle) will enable students to engage with materials prior to entry and also allow the evaluation of a comparable VLE to BlackBoard as part of our 5 year review into e-learning. b) To embed support processes for e-Assessment: The focus in this Pathfinder project is to develop the administrative and support systems to ensure reliable and secure large-scale implementation of CAA with first year students so that we can subsequently roll this out to all students. This will deliver a specification for the administrative and support functions to ensure resilient and manageable online formative and summative assessment routes as well as detailed case studies of the large-scale implementation of CAA and will have as its outcomes both the expanded use of both formative and summative computer-aided assessment in first year courses and a system to support the effective large-scale implementation of computer-aided assessment across the university.

1.3 Stakeholder Analysis

Key stakeholders are:

 Academic staff at the University of Bradford and partner institutions  Students – initially, first year undergraduates, but ultimately other students too  Future students / potential students (currently at college, secondary school)

4  Learner Support Services at University of Bradford (IT, Technical and Customer Services, TQEG, LDU, Learning Technology Team)  Corporate Services (Financial Services, Estates, Academic Administration/Academic Standards and Support Unit)  E-strategy, Pro-Vice-Chancellor (Strategic Systems Development/e-Strategy)  Electronic and Distance Learning Committee  Blackboard / e-learning systems project Board  Invigilators  QuestionMark Computing  Members of other Pathfinder Projects  Wider HE community

1.4 Outputs and Outcomes

Tangible deliverables (including reports):

Aim (a)  An improved method to assess the initial levels of confidence of students prior to and on entry to the University  The development of a cohort wide action plan of development for new students which explicitly identifies levels of confidence, current evidence of ability and areas of development that need to be addressed  Exemplar online learning activities which utilise LAMS to structure materials in a number of diverse courses (to include Computing, Combined Studies, Psychology, Management, Foundation Year, Health Studies, Pharmacy)  A cohort wide picture of the needs of new students  Enhanced access to online learning materials to include podcasts, interactive Powerpoint files and discussion forums  University wide dissemmination of ideas and objectives to showcase 'best practice' across the institution  Research evidence of students needs on entry to the University  A series of reuseable learning objects to enable students to engage with their personal development

Aim (b) A system to support the effective implementation of computer-aided assessment across the university, consisting specifically of  Official university guidelines for CAA (formative and summative), specifying administrative and support functions to ensure resilient and manageable online formative and summative assessment routes  Pedagogical guidelines (“best practice”) document for recommended use of software for formative and summative assessment  Training sessions for instructors (in Sept 07 and Jan 08) in use of QMP and other software  CAA induction and training sessions for existing and new invigilators  In-house training materials for use of QMP  Web-based resources for use of QMP  Research evidence (quantitative and qualitative) on use of CAA in undergraduate courses to inform the sector; detailed case studies of institutional implementation of CAA  Development of banks of online assessment materials in QMP

5  Baseline study and Needs analysis (SWOT) report on use of CAA at UoB

Covering both Aims, the following Reports:  This final project plan (by 30 May 2007)  Interim Report 1 (7 September 2007)  Interim Report 2 (7 December 2007)  Interim Report 3 (7 March 2008)  Final Report (30 May 2008 - Peer reviewed)

Knowledge and experience we hope to build and share:

 Identifying good practice in student support in transition and initial engagement  Increased understanding of the needs of new students on arrival at University  Extended use of online learning materials to support student induction  Increased understanding of the student experience, their expectations and aspirations on arrival  Capacity building and shared knowledge and skills across groups and individuals throughout the university and its partners  Sustainable development and capacity building of staff and student skills in e- assessment  Identifying good practice in e-assessment  Extended range of applications of CAA used across first year courses  Expanded use of both formative and summative computer-assisted assessment in first year courses (based on needs, not pre-set quotas)  More sophisticated understanding of the processes necessary to move e-learning applications from relatively-small scale pilots to large scale adoption  New cross-institutional collaborations based on the CAMEL model

6 1.5 Risk Analysis

Risk Probability Severity Score Action to prevent/manage risk (1-5) (1-5) (PxS)

Staffing 1 3 3 Internal secondments employed so we are not dependent on external advertising

Lack of Publicity campaign to support the project engagement by Support from Pro-Vice Chancellor for staff in providing Learning and Teaching content for the Dissemination of activities through First development of 2 5 10 Year Experience Forum learning Support for staff time from Project Manager materials and (Becka) reuseable Support from education development team learning objects

Training provided to support uploading of Failure to materials engage with new 1 5 5 Additional technical support provided from VLE LDU

Lack of 1 5 5 Support from Pro-Vice Chancellor for organisational Learning and Teaching support Support from education development team

Competing deadlines and Monitoring by Steering Group diversion of staff 3 3 9 Buy-out of Project Manager time to ensure onto other focus on Pathfinder project projects/activities

Technical 2 4 8 Use of tested systems difficulties with Contact with other users across the UK incorporating Strength of in-house technical support new sytems

Legal issues with 2 3 6 Access to range of materials through copyright on LearnHigher CETL as well as experience of materials self-generation

7 1.6 Project Management

Each of the two aims will have a project working group to monitor week-by week progress. In addition, there will be a Steering Group. The project will adhere to the Bradford Method of project management. Proposed constituency for the project steering group is as follows:

Member Role in Institution/external Role in Pathfinder Project

Professor Peter Hartley Head of Teaching Quality Project Director Enhancement Group

Becka Currant Senior Adviser, Learner Project Manager, work packages Development Unit for Aim (a)

John Dermo Learning Technologist, Project Manager, work packages Learner Support Services for Aim (b)

Professor Rae Earnshaw Pro-vice-chancellor e- Chair of Steeering Group[ strategy

Unit for Educational Professor Ivan Reid Independent Internal Evaluation research and Evaluation

Paul Bacsich External consultant "Critical friend"

Malcolm Ryan University of Greenwich External evaluation

Will Stewart e-learning advisor Adviser, Aims (a) and (b)

John Fairhall Mobile technology Adviser Adviser, Aim (a)

Gil Harel Instructor, School of Experienced user of CAA and Informatics Adviser, Aim (b)

This core team contains an appropriate blend of educational experience (including one National Teaching Fellow), project management (e.g. recent and current projects funded by JISC) and technical expertise. As well as the core staff team listed above, we will involve key staff from learner/student development, and key academic staff (including the team currently successful in the bid for JISC ‘community of practice’ status).

The project steering group will meet at least three times across the project. Institutional reports will be made to the Electronic and Distance Learning Working Group and Learning and Teaching Sub-Committee.

8 2. Project Planning

2.1 Workpackages

Aim 1: Develop Me! Online

Brief description Outputs/deliverables Start date End date

Establish website Completed website structure 01/05/2007 01/07/2007 (www.brad.ac.uk/developme)

Upload content Content accessible by internal and external audience 01/05/07 01/09/07

Series of exemplar reuseable learning objects available in pilot Develop reuseable learning objects 01/05/07 01/09/07 courses

Identify and contract external consultant to Completed contract 01/06/07 01/08/07 provide technical support

Timescales document Produce project documentation 01/05/07 01/06/07 Overview of ‘develop me’ online

Academic staff briefed and recruited for project with commitment to Achieve input from academic staff 01/05/07 14/06/07 providing content as necessary

Pilot develop me online Obtain feedback on usability of website prior to public access 01/08/07 12/08/07

1 Brief description Outputs/deliverables Start date End date

Make Develop Me available for new Students able to access website once received firm offer of place 13/08/07 31/05/08 students

Obtain feedback on Develop Me website Report into effectiveness of develop me online 01/10/07 31/01/08 and activities

Continue to develop website and add more Enhanced website with additional reuseable learning activities 01/10/07 31/05/08 activities

Develop expertise in using LAMS to design Series of LAMS based activities for students and staff 01/05/07 31/05/08 learning activities

Aim 2: E-Assessment

Brief description Outputs/deliverables Start date End date

Identify and hire project manager Seconded staff to free up Project Manager 01/05/07 31/05/07

Baseline Study Baseline report - to be included in interim report 1 04/06/07 07/09/07

Needs Analysis (SWOT analysis) Report - to be included in interim report 1 04/06/07 07/09/07

Develop pedagogical guidelines Best practice document for teaching staff 04/06/07 08/01/08

2 Brief description Outputs/deliverables Start date End date

Develop web-based resources for staff Web pages 04/06/07 08/01/08

Develop case studies on use of CAA Case study reports 04/06/07 30/04/08

Advertise/market CAA at UoB Posters, web announcements, presentations to academic schools 02/07/07 30/04/08

Official university guidelines for CAA (formative and summative), Develop systems and integration between specifying administrative and support functions to ensure resilient Technical Services, Customer Services, 09/07/07 11/01/08 and manageable online formative and summative assessment ASSU, and faculties. routes

Identify optimum use of learning spaces Report for UoB e-strategy board 04/06/07 30/04/08 (existing and new) for online assessment.

Training materials 09/07/07 14/09/09 Training for staff Training sessions 1 17/09/07 21/09/07 Training sessions 2 09/01/08 14/01/08

01/12/07 15/01/08 Training for invigilators (existing and new) Training sessions prior to exam weeks 01/04/08 13/05/08

Co-ordinate development of item banks Banks of assessment items/objects 01/10/07 30/04/08

3 2.2 Quality

Output/outcome Quality criteria Policy or procedure Technical standard Evidence of Responsibility compliance

For Aim (a)

Reports to the Project Manager Website and Accessibility of all Testing against given Accessibility standards project steering materials materials standards group

Reports to the Project Manager Ease of navigation and Survey of users; usage Website n/a project steering usability statistics group

Reports to the Project Manager Learning Meet learning Survey users and n/a project steering materials outcomes monitor performance group

Consistent and Reports to the Project Manager Learning Evaluation by internal appropriate pedagogic n/a project steering Materials and external experts design group

For Aim (b)

Baseline Study Validity, reliability and Existing data collection n/a Reports to the Project Manager and Needs usefulness for project - from QM perception project steering analysis will need approval databases; Surveys and group from steering group interviews in academic

4 schools

Output/outcome Quality criteria Policy or procedure Technical standard Evidence of Responsibility compliance

Guidelines, Pedagogical document and Feedback from TQEG, Pilot and incorporate guidelines and Accessibility standards Reports to the Project Manager instructors feedback resources project steering group

Validity and reliability - discussion with Case studies on Case studies to will need approval colleagues inside and n/a the project Project Manager use of CAA from steering group outside UoB steering group

Guidelines, Collaboration with Work collaboratively with document(s) Systems and ASSU, schools, IT stakeholders, to arrive at n/a and Reports to Project Manager integration services final draft of documents the project steering group

Identification of Collaboration with Reports to the optimum use of Discuss technical and technical services, project steering learning spaces pedagogical needs with estates, instructors, n/a group and e- Project Manager (existing and informed stakeholders learning technologists, strategy new) for online inside and outside UoB elearning advisers committee assessment.

Training sessions Reports to the Questionnaire and for staff and staff survey n/a project steering Project Manager analysis invigilators group

Development of Validity, reliability, Gather data and apply Interoperability standards Reports to the Project Manager

5 project steering item banks practicality sound testing principles (CETIS, IMS QTI) group

2.3 Evaluation

Aim (a)

Questions Indicators Baseline Source of information Collection method Schedule

What do you How will you What is the Where can this data be How will the data be When, where and who will want to know? prior situation? obtained? gathered? gather the data? know?

Does the Usage No current Monitoring of website and Web statistics Statistics compiled by project website website or activities team on weekly basis attract the materials for intended prospective users? students

Are the Usage No current Monitoring of website and Web statistics Statistics compiled by project students website or activities Survey team on weekly basis using the materials for Survey of users Survey conducted by LDU staff materials prospective with follow-up interviews supplied? students

Do students Usage and No current Survey of users Surveys Survey conducted by LDU staff gain short- satisfaction; website or Consultation with course Interviews with follow-up interviews term benefit? identification of materials staff Performance data

6 specific benefit

Do students Identification No current Survey of users Surveys Survey conducted by LDU staff gain long- of specific website or Consultation with course Interviews with follow-up interviews term benefit? benefits materials staff Performance data

Aim (b)

Evaluation Indicators Baseline Source of information Collection method Schedule Questions

What do you How will you What is the Where can this data be How will the data be When, where and who will gather the want to know? prior obtained? gathered? data? know? situation?

Has the use Comparison Statistics on Examinations office Routine collection of Examinations office responsible for of CAA of statistics current data on summative collection of data on summative expanded on usage summative examinations. assessment across the practice Need to develop system for collecting institution? available data on formative assessment

Are the Comparison Data Examinations office Routine collection of Examinations office responsible for procedures to of statistics available on data on summative collection of data on summative support CAA on issues and all summative examinations. assessment effective and problems assessments Need to develop system for collecting efficient? data on formative assessment

Is the use of Student Very limited Students Survey of users Need to develop system

7 CAA satisfaction data on a few Course and module Review of performance improving and modules performance and statistics on relevant feedback to performance feedback modules students?

8 2.4 Dissemination

Timing Dissemination Activity Audience Purpose Key Message ongoing Project Blog on Helga cluster; critical Progress report and feedback friends;project team ongoing Project Blog (on HEA; other Record of the project “as it happens” Wordpress) - external Pathfinders; HE community; project team

May 07 Pathfinder start-up, York HEA, JISC, Networking, Bonding Pathfinder institutions, critical friends

June 07 UoB LTA conference UoB academics Awareness raising, recruitment of Outline project aims and objectives and academic participants support staff

July 07 11th International CAA HE/FE Outline initial aims of project and share Outline aim 2 and elicit feedback Conference, Professionals experieces with colleagues in the sector from professionals Loughborough working in the field of online assessment

July 07 Amoeba meeting @ Other Update on progress, ongoing evaluation, as tbc Brighton Pathfinders in per CAMEL model Amoeba cluster, plus guests and

9 critical friend(s)

Sept 07 ALT conference HE Outline initial aims of project, give Outline project aims and objectives Professionals update and share experieces with colleagues working in e- in the sector learning

Oct 07 Amoeba @ Bradford Other Update on progress, ongoing evaluation, as tbc Pathfinders in per CAMEL model Amoeba cluster, plus guests and critical friend(s)

Nov 07 Pathfinder general tbc tbc tbc meeting

Jan 08 Amoeba @ Greenwich Other Update on progress, ongoing evaluation, as tbc Pathfinders in per CAMEL model Amoeba cluster, plus guests and critical friend(s)

March 08 Amoeba @ TVU Other Update on progress, ongoing evaluation, as tbc Pathfinders in per CAMEL model Amoeba cluster, plus guests and critical friend(s)

May 08 3rd European First Year HE Update sector on new developments in Disseminate outcomes of Experience Network Professionals retaining students and helping with initial workpackages which focus on initial Conference, working in the transition and induction transition and engagement. Wolverhampton field of student support and

10 retention

May 08 Final Pathfinder meeting tbc tbc tbc

June (or Pathfinder conference tbc tbc tbc Sep) 08

June 08 UoB LTA conference UoB academics Reflect on the project and explore how the Disseminate outcomes of the project and academic lessons learnt will be applied at UoB in the support staff future

July 08 QM Academic Online Update sector on new developments in Disseminate outcomes of Symposium assessment embedding e-assessment workpackages for aim (b) 2. practitioners from a range of sectors

July 08 12th International CAA HE/FE Update sector on new developments in Disseminate outcomes of Conference, Professionals embedding e-assessment workpackage 2. Loughborough working in the field of online assessment as held Other HEA events Other As per events As per events Pathfinders; wider HE community

Special dissemination HE community; Reflect on the project and explore how the Depends on the outcome of the April 08 event at Bradford (tbc?) academics and lessons learnt could be applied at other project (tbc) support staff at institutions University of Bradford, also

11 in West Yorks and North of England tbc JISC InfoNet Pathfinder tbc tbc tbc case studies tbc Scholarly output – HE community; Update and report on project Depends on the outcome of the journal articles academics and project support staff tbc HE community Reflect on the project and explore how the Depends on the outcome of the Other conferences – lessons learnt could be applied at other project F2F and online institutions

2.5 Sustainability and Further Developments

Project Outputs/Outcomes Action for Sustainability

Develop Me! website and online Embedded into institutional induction strategy with further embedding and roll out to additional schools learning activities during 2008-9.

Embedded e-assessment system Procedures developed during this project will become part of the code of practice for assessment