Course Experience Survey (CES)
Submission to: VCE
TITLE / Course Experience Survey (CES), Semester 2 2009: Participation and Outcomes.BACKGROUND / The Course Experience Survey (CES) is the University wide survey of student views on their learning experience within individual courses and has been administered with higher education and TAFE versions each semester since 2005. In addition to being an important tool in the collection and understanding of student feedback around teaching and learning issues data from the CES contributes to staff promotion applications.
Course level reports are provided to academic and teaching staff as well as University staff responsible for monitoring and improving the quality of the RMIT student learning environment. In addition a summary report is prepared for the Vice-Chancellor’s Executive each semester. This report is intended to summarize trends and issues emerging from the gathered student feedback and details participation rates in the survey by the RMIT organization. The report also provides proposed actions to improve efficiency of administration and use of the CES instrument.
SUMMARY / In 2009, 73% of all HE courses (excluding research) were surveyed with the CES and students from 1035 TAFE courses were surveyed. In total 4131 Higher Education course reports and 1394 TAFE course reports were prepared and distributed to academic and teaching staff.
Across all HE courses there was a three percent improvement between Semester 2 2008 and Semester 2, 2009 in the Good Teaching Scale (GTS) and 2.6% in the Overall Satisfaction Index (OSI). TAFE student feedback has remained largely unchanged at high levels on the GTS and the OSI.
Key areas for improvement remain, as in previous reports, the amount of comment on work students received from staff and the use of computer based and online teaching materials.
A review of the student feedback policy, the CES instruments, reporting of data and the surveys is presently being undertaken. Immediate recommended improvements include changes to online administration of the CES, and the method used to calculate survey participation rates. An options paper is being developed for presentation to the DVC(A).
LINKAGES TO STRATEGIC DOCUMENTS AND POLICIES / Academic Plan
Learning and Teaching Strategy
RESOURCE IMPLICATIONS / N/A
RISK
IMPLICATIONS / CES findings assist Schools to address risk related to student satisfaction.
IMPACT / N/A
RESPONSIBILITY AND COMMUNICATION / Deputy Vice-Chancellor (Academic)
ATTACHMENTS / N/A
RECOMMENDATION / That VCE note the report.
Submitted by:
Name: Professor Julianne Reid
Title: Deputy Vice-Chancellor (Academic)
Date: / Committee outcome:
[Completed by committee secretary]
Course Experience Survey (CES)
Semester 2, 2009
Participation and Outcomes
Survey Services Centre
Policy & Planning Group
February 2010
Survey Services Centre Feb 2010
1. Overview
Course Experience Survey (CES) outcomes for Semester 2 2009 show continuing high levels of satisfaction in students’ reported perceptions on the key indicators of good teaching and overall satisfaction in both the HE and TAFE sectors.
The CES instruments and administration, as well as the Student Feedback Policy underpinning the CES, are currently being reviewed. An options paper is under development for discussion.
2. Survey Findings
2.1 Higher Education
On the key statement of “Overall I am satisfied with the quality of this course” or Overall Satisfaction Index (OSI), RMIT HE students continue to report high and increasing levels of agreement. In Semester 2 2009 72.6% reported either agreeing or strongly agreeing with the statement on satisfaction compared with 70% for the corresponding Semester in 2008 and 68% in Semester 2 2007.
On the items comprising the Good Teaching Scale (GTS) Higher Education students’ views of their courses have shown a continual improvement with each previous comparable Semester since the commencement of the survey in 2006. The GTS outcome for Semester 2 2009 has risen 3% to 67.3 compared to Semester 2 2008 and increased 12% compared with 2006.
The items within the GTS that the University is rated the most highly by its HE students are-: “The teaching staff work hard to make this course interesting” and, “The teaching staff are extremely good at explaining things.” The GTS item that students are least likely to agree with is “The staff put a lot of time into commenting on my work”. The time spent commenting on work is also the lowest rated item of all 21 questions on the CES. This finding was the same for previous semesters and gives a clear direction for strategies focused on improving the overall GTS score.
Looking at all items on the survey the three most positive responses, in order, from HE students were on:-
· Assessment tasks in this course require me to demonstrate what I am learning.
· The learning objectives in this course are clear to me.
· The amount of work required in this course is about right.
Consistent with last Semester the items on the questionnaire that HE students rate their courses most poorly on are:-
· The staff put a lot of time into commenting on my work.
· There is effective use of other computer-based teaching materials in this course.
· The web-based (online) materials for this course are effective in assisting my learning.
2.2 TAFE
TAFE students reported overall satisfaction with the quality of their courses has remained at a high and stable level with 71% of respondents strongly agreeing or agreeing with this measure in both Semester 2 2008 and 2009. Since Semester 2, 2007 to 2009 the percentage agreement on this overarching question has improved seven percent.
The Good Teaching Scale outcome for TAFE has improved very marginally to 77% compared with Semester 2, 2008. The questions with the highest contribution to the TAFE GTS were: - “My instructors treat me with respect” and “My instructors provide opportunities to ask questions”.
Questions with the lowest contribution to the TAFE GTS outcome were:-
· My instructors make the course as interesting as possible
· My instructors understand my learning needs
In common with Semester 1, RMIT TAFE students rate the following aspects of their experience with teaching staff the highest of all questions on the survey:-
· My instructors treat me with respect.
· My instructors provide opportunities to ask questions.
· My instructors have a good knowledge of current workplace skills.
The three most poorly performed areas in the view of TAFE students are:-
· The web-based (online) materials for this course are effective in assisting my learning.
· There is effective use of other computer-based teaching materials in this course.
· I enjoy doing the work for this course.
2.3 College Comparisons
Comparing RMIT’s three Colleges, Design and Social Context (DSC) continues to receive higher ratings by students on the GTS and the OSI. Seventy five percent of the survey responses from students in this College report overall satisfaction with quality and 71% agree with the items on the GTS. Similarly in TAFE DSC students rate their courses with the highest GTS (78%) and 73% are satisfied with course quality.
Looking at each item of the CES by College shows that Business is rated more poorly on all items compared with total University responses. Notably Business students are eight percent less likely to agree with the statement: - “I enjoy doing the work for this course.”
The College of SEH continues a clear advantage compared to overall RMIT responses on facilities and online resources with the following items showing a 5% greater agreement than the total University responses:-
· The facilities (such as classrooms, lecture theatres, studios, labs) are adequate for this course.
· There is effective use of other computer-based teaching materials in this course.
· The web-based (online) materials for this course are effective in assisting my learning.
Alternatively SEH students rate their course teaching staff more poorly on the feedback provided with the following items six and five percent under the average:-
· The staff put a lot of time into commenting on my work.
· The teaching staff normally give me helpful feedback on how I am going in this course.
Survey data for the DSC shows the opposite pattern to SEH. That is to say, the items DSC outperforms most against the rest of the University – comment and feedback are the items SEH does least well. Conversely the SEH’s best areas – facilities and online and computer based materials resources - are the items the DSC does most poorly.
Comparing Schools in HE it was notable that Creative Media had a nine percent decrease in GTS ratings since Semester 2 2008. The School of Property Construction and Project Management on the other hand improved the GTS in that period by a massive 14%. The School of Art received the highest GTS student ratings.
TAFE students in DSC rate their courses equal or higher on all other items of the CES with the exception of:-
· The web-based (online) materials for this course are effective in assisting my learning.
· There is effective use of other computer-based teaching materials in this course.
· The facilities (such as classrooms, lecture theatres, studios, labs) are adequate for this course.
Conversely, Business TAFE students responded with less agreement to all items than the overall student body, apart from the same three items about online materials and computer based materials and the item: - “The course is well organized”.
SEH TAFE students rate their courses more poorly than all students on all items with the very marginal exceptions of enjoying the work for the course and that learning resources are useful.
2.4 Item Improvements
Comparing Semester 2 HE 2009 with the corresponding Semester in 2008 the following items have shown the greatest improvement:-
· The staff make a real effort to understand difficulties I might be having with my work.
· The teaching staff normally give me helpful feedback on how I am going in this course.
· The staff put a lot of time into commenting on my work.
In the TAFE sector the areas of greatest improvement, both by two percent, in the year were:-
· My instructors have a good knowledge of current workplace skills.
· The learning objectives in this course are clear to me.
2.5 Vietnam CES Outcomes
Reports were generated for 123 courses from Semester 2 in Vietnam from 12 Schools. An analysis of total Vietnam student responses compared with total HE responses for all students from all other campus locations shows that these students as a group were significantly less satisfied (9.6%) with the overall quality of their courses but reported the same levels of good teaching in their courses. Vietnam students were less satisfied with the amount of work required in the course, facilities and resources, class participation and assessment tasks.
3. CES Administration Issues
3.1 Review of the CES
The Policy and Planning Group have commenced a review of the student feedback policy, the CES instrument, reporting of data and the survey’s administration. A paper is currently being drafted to assist the Project Reference Group with options and recommendations. It is anticipated that the recommendations from this project will be implemented in the second semester of 2010 or the following Semester.
3.2 Provision of student qualitative data and return of hard copy forms
The current version of the CES contains two open ended items asking about what students regarded as the best aspects of their course and what could be improved. The SSC is not resourced to undertake the massive task of keying these student comments and include them as part of its standard reporting process. Up until the past Semester the practice has been that once the forms have been scanned and a quantitative report of outcomes sent the original forms have been returned to the teaching staff so they are able to read the comments.
In Semester 2 the SSC continued to return the original forms to staff but also provide a zipped file of the scanned images so that staff were able to read the forms in soft copy. These zipped files were then also made available to other staff under the provisions of the Student Feedback Policy, such as Heads of School. This improvement is allowing for a complete electronic archiving of all student feedback.
In Semester 1 of 2010 the SSC will continue to provide zipped image files together with the standard quantitative report but will retain the original paper forms in archive for a year. Academic and teaching staff will be able to request the hard copy forms if they wish by return email.
3.3 Online CES administration
During the latter stages of Semester 2 2009 the SSC was able to successfully trial with ITS the new online survey platform called Blue and integrate outcomes with existing reporting methods. The intention in the forthcoming Semester is to extend the use of Blue to all online surveying instead of the cumbersome RMIT publishing system. Early analysis of response rates suggests that, presumably because of the more personalized nature of presentation in Blue, the new platform may gain greater data representativeness than has previously been available.
3.4 Calculation of survey participation rates
Previously the SSC has reported HE participation rates by determining the number of non research courses offered in a given Semester and calculating a percentage of courses where surveys are returned for processing. However, the Student Feedback Policy only requires that courses are surveyed once a year and not each time they are offered. Consequently where a School has repeated a course in the second semester of the year, but only surveyed students in that course in first Semester, the participation rate for that School for the second semester would be underestimated. Commencing in this report participation rates are calculated for the calendar year and reflect what percentage of unique courses, which do not have a research career or are external outbound – both of which are not suitable for the CES – are surveyed at any time in the year.
3.5 Survey participation in TAFE
The number of courses and competencies surveyed has reduced very significantly in TAFE during 2009. However there have been no substantial changes to the administration of the TAFE CES during the year so it is uncertain what the cause may be. The SSC will meet individually will all School Student Feedback Coordinators to discuss this issue, determine the reasons and recommend whatever changes to procedure will assist to lift the TAFE participation rate in the CES.