National Survey of Student Engagement

NSSE national survey of student engagement

2011 Mount Royal Benchmark Comparisons

This Top Line Report compares the performance of Mount Royal with participating Canadian institutions, Canadian participants attending selected institutions with high student satisfaction and low research intensity, and all 2011 NSSE participants through NSSE’s five Benchmarks of Effective Educational Practice. The Benchmark comparisons illustrate how student engagement at Mount Royal differs from comparable NSSE student participants.

Office of Institutional Analysis & Planning Mount Royal University, September 2011

2011

COMPARISON GROUP DETAILS The peer groups that were selected for comparison with Mount Royal in 2011 were: • Canada participants (n=67) (see Table A); 1 • Canada (CA) High Satisfaction (n=11) participants ; and 2 • All NSSE 2011 participants (n=740) .

TABLE A: Canada Participants a

Institution Name b City Province

Alberta College of Art & Design Calgary AB Algoma University Sault Ste. Marie ON Ambrose University College Calgary AB Bishop's University Sherbrooke QC Brandon University Brandon MB Brescia University College London ON Brock University St. Catharines ON Cape Breton University Sydney NS Carleton University Ottawa ON Concordia University Montreal QC Dalhousie University Halifax NS École de technologie supérieure Montréal QC Huron University College London ON King's University College at the University of Western Ontario London ON King's University College, The Edmonton AB Lakehead University Thunder Bay ON Laurentian University/Université Laurentienne Sudbury ON McGill University Montreal QC McMaster University Hamilton ON Memorial University of Newfoundland, St. John's Campus St. John's NL Mount Allison University Sackville NB Mount St. Vincent University Halifax NS Nipissing University North Bay ON Nova Scotia Agricultural College Truro NS OCAD University Toronto ON Queen's University Kingston ON Quest University Canada Squamish BC Royal Roads University Victoria BC Ryerson University Toronto ON Cont’d…

1 Canadian Institutions having high NSSE Satisfaction ratings (i.e., EXCELLENT only) in 2009 and low research intensity as reported in Maclean’s November 2010. These are considered 'aspirational' institutions for the purpose of comparison with MRU. 2 For all NSSE 2011 participating institutions, please visit: http://nsse.iub.edu/pdf/2011_comparison_group3.pdf.

2 2011

Institution Name City Province

Saint Mary's University Halifax NS Simon Fraser University Burnaby BC St. Francis Xavier University Antigonish NS St. Thomas University Fredericton NB Thompson Rivers University Kamloops BC Trent University Peterborough ON Trinity Western University Langley BC Tyndale University College and Seminary Toronto ON Université d'Ottawa / University of Ottawa Ottawa ON Université de Montréal Montréal QC Université de Sherbrooke Sherbrooke QC Université du Québec à Chicoutimi Chicoutimi QC Université du Québec à Montréal Montréal QC Université du Québec à Rimouski Rimouski QC Université du Québec à Trois-Rivières Trois-Rivières QC Université du Québec en Abitibi-Témiscamingue Rouyn-Noranda QC Université du Québec en Outaouais Gatineau QC Université Laval Quebec city QC University of Alberta Edmonton AB University of British Columbia Vancouver BC University of British Columbia Okanagan Kelowna BC University of Calgary Calgary AB University of Guelph Guelph ON University of Lethbridge Lethbridge AB University of Manitoba, The Winnipeg MB University of New Brunswick - Fredericton Fredericton NB University of New Brunswick - Saint John Campus Saint John NB University of Ontario Institute of Technology Oshawa ON University of Prince Edward Island Charlottetown PE University of Regina Regina SK University of Saskatchewan Saskatoon SK University of Toronto Toronto ON University of Waterloo Waterloo ON University of Western Ontario London ON University of Windsor Windsor ON University of Winnipeg, The Winnipeg MB Wilfrid Laurier University Waterloo ON York University Toronto ON a Mount Royal University is not represented in this group for the comparative analyses. b In 2010 there were 22 participating Canadian institutions compared to 2011 where 67 Canadian institutions took part. By agreement with the provincial ministry and the Higher Education Quality Council of Ontario, Ontario institutions participate in NSSE on a rotational schedule for inclusion of results in their Multi-year Accountability Reports. 2011 was a reporting year accounting for the higher NSSE participation rate in Canada.

3 2011

RESPONDENT CHARACTERISTICS MRU’s sample included applied, transfer, and degree seeking first-year (FY) and senior-year (SR) students. The sample size for SR students is small compared to FY due to small senior cohorts in new degree programs.

MRU Canada CA High NSSE 2011 Satisfaction

FY SR FY SR FY SR FY SR Response Rate Overall 41% 32% 38% 28% By class 40% 44% 31% 33% 36% 40% 26% 29% Sample size 1,919 715 190,067 157,808 23,193 14,579 980,113 1,001,791 Enrollment Status Full-time 96% 94% 94% 83% 94% 87% 94% 84% Part-time 4% 6% 6% 17% 6% 13% 6% 16%

Gender Female 70% 67% 64% 63% 64% 65% 64% 63% Male 30% 33% 36% 37% 36% 35% 36% 37%

Place of Residence On-campus 12% 2% 35% 5% 54% 5% 59% 14% Age 24 or older 9% 45% 9% 28% 6% 20% 8% 34% Less than 24 91% 55% 91% 72% 94% 80% 92% 66%

4 2011

HOW DOES MRU COMPARE ON FIRST-YEAR BENCHMARK SCORES*?

MRU Canada CA High NSSE 2011 Satisfaction Level of Academic 53.8 51.1 51.6 53.2 Challenge

Active and 45.4 36.2 36.8 41.8 Collaborative Learning

Student-Faculty 29.8 22.8 24.9 32.1 Interaction

Enriching Educational 25.2 24.1 24.5 27.0 Experiences

Supportive Campus 60.4 56.9 60.6 61.6 Environment

*Note: the more stars, the higher the mean score on NSSE Benchmarks.

HOW DOES MRU COMPARE ON SENIOR BENCHMARK SCORES*?

MRU Canada CA High NSSE 2011 Satisfaction Level of Academic 55.8 55.5 57.7 57.1 Challenge

Active and 54.2 45.3 48.6 50.3 Collaborative Learning

Student-Faculty 39.3 32.2 36.5 40.4 Interaction

Enriching Educational 37.4 34.3 35.1 39.5 Experiences

Supportive Campus 59.7 52.9 57.8 58.1 Environment

*Note: the more stars the higher the mean score on NSSE Benchmarks.

5 2011

HOW DOES MRU COMPARE ON FIRST-YEAR SATISFACTION? How would you evaluate your entire educational experience at this institution?

100% 92% 86% 86% 82% 80%

60%

Negative 40% Positive

20%

0% MRU Canada CA High NSSE 2011 Satisfaction

If you could start over again, would you go to the same institution you are now attending?

100% 91% 86% 85% 85% 80%

60%

No 40% Yes

20%

0% MRU Canada CA High NSSE 2011 Satisfaction

6 2011

HOW DOES MRU COMPARE ON SENIOR SATISFACTION? How would you evaluate your entire educational experience at this institution?

100% 94% 86% 85% 79% 80%

60%

Negative 40% Positive

20%

0% MRU Canada CA High NSSE 2011 Satisfaction

If you could start over again, would you go to the same institution you are now attending?

100% 93%

81% 82% 78% 80%

60%

No 40% Yes

20%

0% MRU Canada CA High NSSE 2011 Satisfaction

7 2011

LEVEL OF ACADEMIC CHALLENGE (LAC) “Challenging intellectual and creative work is central to student learning and collegiate quality. Colleges and universities promote high levels of student achievement by emphasizing the importance of academic effort and setting high expectations for student performance” (NSSE 2011).

Mean Comparisons Mount Royal University compared with: MRU Canada CA High Satisfaction NSSE 2011 Effect Effect Effect Class Mean a Mean a Sig b Size c Mean a Sig b Size c Mean a Sig b Size c First-Year 53.8 51.1 *** .21 51.6 *** .17 53.2 .05 Senior 55.8 55.5 .03 57.7 ** -.14 57.1 -.09 a Weighted by gender and enrollment status (and by institution size for comparison groups). b * p<.05 ** p<.01 ***p<.001 (2-tailed). c Mean difference divided by the pooled standard deviation.

Distributions of Student Benchmark Scores

First-Year Senior 100 100

75 75

50 50

25 25

0 0 MRU Canada CA High NSSE 2011 MRU Canada CA High NSSE 2011 Satisfaction Satisfaction

Note: Each box and whiskers chart plots the 5th (bottom of lower bar), 25th (bottom of box), 50th (middle line), 75th (top of box), and 95th (top of upper bar) percentile scores. The dot shows the benchmark mean.

8 2011

Mean Comparisons on LAC Benchmark Items

Level of Academic Challenge Items Mount Royal Compared with…

Canada CA High NSSE 2011 Satisfaction Coursework emphasizes: Analysis of the basic elements FY of an idea, experience or theory. SR

Coursework emphasizes: Making judgments about the FY value of information, arguments, or methods. SR

Coursework emphasizes: Applying theories or concepts FY to practical problems or in new situations. SR

Number of written papers or reports of 20 pages or FY more. SR

Number of written papers or reports between 5 and 19 FY pages. SR

Worked harder than you thought you could to meet an FY instructor’s standards or expectations. SR

Coursework emphasizes: Synthesis and organization of FY ideas, information, or experiences into new, more complex, interpretations and relationships. SR

Number of written papers or reports of fewer than 5 FY pages. SR

Spent significant amounts of time studying and on FY academic work. SR

Number of assigned textbooks, books, or book-length FY packs of course readings. SR

Hours spent preparing for class (studying, etc.). FY

9 2011

ACTIVE AND COLLABORATIVE LEARNING (ACL) “Students learn more when they are intensely involved in their education and are asked to think about and apply what they are learning in different settings. Collaborating with others in solving problems or mastering difficult material prepares students to deal with the messy, unscripted problems they will encounter daily during and after college” (NSSE 2011).

Mean Comparisons

Mount Royal University compared with: MRU Canada CA High Satisfaction NSSE 2011 Effect Effect Effect Class Mean a Mean a Sig b Size c Mean a Sig b Size c Mean a Sig b Size c First-Year 45.4 36.2 *** .58 36.8 *** .57 41.8 *** .22 Senior 54.2 45.3 *** .52 48.6 *** .33 50.3 *** .22 a Weighted by gender and enrollment status (and by institution size for comparison groups). b * p<.05 ** p<.01 ***p<.001 (2-tailed). c Mean difference divided by the pooled standard deviation.

Distributions of Student Benchmark Scores

First-Year Senior 100 100

75 75

50 50

25 25

0 0 MRU Canada CA High NSSE 2011 MRU Canada CA High NSSE 2011 Satisfaction Satisfaction

10 2011

Mean Comparisons on ACL Benchmark Items

Active and Collaborative Learning Items Mount Royal Compared with…

Canada CA High NSSE 2011 Satisfaction Asked questions in class or contributed to class FY discussions. SR

Made a class presentation. FY

Worked with other students on projects during class. FY

Worked with classmates outside of class to prepare FY class assignments. SR

Discussed ideas from readings or classes with others FY outside of class (students, family members, co-workers, etc.). SR

Participated in a community-based project as part of a FY regular course. SR

Tutored or taught other students (paid or voluntary). FY

11 2011

STUDENT-FACULTY INTERACTIONS (SFI) “Students see first-hand how experts think about and solve practical problems by interacting with faculty members inside and outside the classroom. As a result, their teachers become role models, mentors, and guides for continuous, life-long learning” (NSSE 2011).

Mean Comparisons

Mount Royal University compared with: MRU Canada CA High Satisfaction NSSE 2011 Effect Effect Effect Class Mean a Mean a Sig b Size c Mean a Sig b Size c Mean a Sig b Size c First-Year 29.8 22.8 *** .43 24.9 *** .30 32.1 *** -.13 Senior 39.3 32.2 *** .36 36.5 * .14 40.4 -.05 a Weighted by gender and enrollment status (and by institution size for comparison groups). b * p<.05 ** p<.01 ***p<.001 (2-tailed). c Mean difference divided by the pooled standard deviation.

Distributions of Student Benchmark Scores

First-Year Senior 100 100

75 75

50 50

25 25

0 0 MRU Canada CA High NSSE 2011 MRU Canada CA High NSSE 2011 Satisfaction Satisfaction

12 2011

Mean Comparisons on SFI Benchmark Items

Student-Faculty Interactions Items Mount Royal Compared with…

Canada CA High NSSE 2011 Satisfaction Received prompt written or oral feedback from faculty FY on your academic performance. SR

Discussed grades or assignments with an instructor. FY

Talked about career plans with a faculty member or FY advisor. SR

Discussed ideas from your readings or classes with FY faculty outside of class. SR

Worked with faculty members on activities other than FY coursework. SR

Worked on a research project with a faculty member FY outside of course or program requirements. SR

13 2011

ENRICHING EDUCATIONAL EXPERIENCES (EEE) “Complementary learning opportunities inside and outside the classroom augment the academic program. Experiencing diversity teaches students valuable things about themselves and other cultures. Used appropriately, technology facilitates learning and promotes collaboration between peers and instructors. Internships, community service, and senior capstone courses provide students with opportunities to synthesize, integrate, and apply their knowledge. Such experiences make learning more meaningful and, ultimately, more useful because what students know becomes a part of who they are” (NSSE 2011).

Mean Comparisons

Mount Royal University compared with: MRU Canada CA High Satisfaction NSSE 2011 Effect Effect Effect Class Mean a Mean a Sig b Size c Mean a Sig b Size c Mean a Sig b Size c First-Year 25.2 24.1 * .09 24.5 .05 27.0 *** -.14 Senior 37.4 34.3 *** .19 35.1 * .14 39.5 * -.12 a Weighted by gender and enrollment status (and by institution size for comparison groups). b * p<.05 ** p<.01 ***p<.001 (2-tailed). c Mean difference divided by the pooled standard deviation.

Distributions of Student Benchmark Scores

First-Year Senior 100 100

75 75

50 50

25 25

0 0 MRU Canada CA High NSSE 2011 MRU Canada CA High NSSE 2011 Satisfaction Satisfaction

14 2011

Mean Comparisons on EEE Benchmark Items

Enriching Educational Experience Items Mount Royal Compared with…

Canada CA High NSSE 2011 Satisfaction Used an electronic medium to discuss or complete an FY assignment. SR

Had serious conversations with students of a different FY race or ethnicity. SR

Did a practicum, internship, field experience, co-op FY experience, or clinical assignment. SR

Had serious conversations with students of different FY religious beliefs, political opinions, or personal values. SR

Participated in a learning community or some other FY formal program where groups of students take two or more classes together. SR

Participated in an independent study or self-designed FY major. SR

Encouraging contact among students from different FY economic, social, and racial or ethnic backgrounds. SR

Participated in community service or volunteer work. FY

Culminating senior experience (senior project or thesis, FY comprehensive exam, etc.). SR

Completed foreign language coursework. FY

Studied abroad. FY

Participated in co-curricular activities (organizations, FY campus publications, etc.). SR

15 2011

SUPPORTIVE CAMPUS ENVIRONMENT (SCE) “Students perform better and are more satisfied at colleges that are committed to their success and cultivate positive working and social relations among different groups on campus” (NSSE 2011).

Mean Comparisons

Mount Royal University compared with: MRU Canada CA High Satisfaction NSSE 2011 Effect Effect Effect Class Mean a Mean a Sig b Size c Mean a Sig b Size c Mean a Sig b Size c First-Year 60.4 56.9 *** .20 60.6 -.01 61.6 -.06 Senior 59.7 52.9 *** .37 57.8 .11 58.1 .08 a Weighted by gender and enrollment status (and by institution size for comparison groups). b * p<.05 ** p<.01 ***p<.001 (2-tailed). c Mean difference divided by the pooled standard deviation.

Distributions of Student Benchmark Scores

First-Year Senior 100 100

75 75

50 50

25 25

0 0 MRU Canada CA High NSSE 2011 MRU Canada CA High NSSE 2011 Satisfaction Satisfaction

16 2011

Mean Comparisons on SCE Benchmark Items

Supportive Campus Environment Items Mount Royal Compared with…

Canada CA High NSSE 2011 Satisfaction Quality of relationships with other students. FY

Quality of relationships with faculty members. FY

Quality of relationships with administrative personnel FY and offices. SR

Campus environment provides the support needed to FY succeed academically. SR

Campus environment helps to cope with non-academic FY responsibilities (work, family, etc.). SR

Campus environment provides the support needed to FY thrive socially. SR

17 2011

FIRST-YEAR MRU MULTI-YEAR BENCHMARK CHARTS

Level of Academic Challenge (LAC) Active and Collaborative Learning (ACL)

85 85

75 75

65 65 53.8 53.4 53.8 55 55 45.9 45.6 45.4 45 45

35 35

25 25

15 15 '04 '05 '06 '07 '08 '09 '10 '11 '04 '05 '06 '07 '08 '09 '10 '11

Student-Faculty Interaction (SFI) Enriching Educational Experiences (EEE)

85 85

75 75

65 65

55 55

45 45

35 30.1 30.6 29.8 35 25.7 25.1 25.2 25 25

15 15 '04 '05 '06 '07 '08 '09 '10 '11 '04 '05 '06 '07 '08 '09 '10 '11

Supportive Campus Environment (SCE)

65 61.9 61.0 60.4

15 '04 '05 '06 '07 '08 '09 '10 '11

18 2011

SENIOR YEAR MRU MULTI-YEAR BENCHMARK CHARTS

Level of Academic Challenge (LAC) Active and Collaborative Learning (ACL)

85 85

75 75

65 65 56.5 56.9 55.8 54.2 55 55

45 45

35 35

25 25

15 15 '04 '05 '06 '07 '08 '09 '10 '11 '04 '05 '06 '07 '08 '09 '10 '11

Student-Faculty Interaction (SFI) Enriching Educational Experiences (EEE)

85 85

75 75

65 65

55 55

45 45 37.9 39.3 37.2 37.4 35 35

25 25

15 15 '04 '05 '06 '07 '08 '09 '10 '11 '04 '05 '06 '07 '08 '09 '10 '11 Supportive Campus Environment (SCE)

65 59.6 59.7

15 '04 '05 '06 '07 '08 '09 '10 '11

19 2011

APPENDIX Interpreting Mean Comparisons & Distributions of Student Benchmark Scores

Class and Sample Statistical Significance Effect Size Means are reported Benchmarks with mean differences that are larger than would be expected by Effect size indicates for first-year students chance alone are noted with one, two, or three asterisks, denoting one of the practical and seniors. three significance levels (p<.05, p< .01, and p<.001). The smaller the significance of the Institution-reported significance level, the smaller the likelihood that the difference is due to mean difference. It is class levels are used. chance. Please note that statistical significance does not guarantee that the calculated by dividing All randomly selected result is substantive or important. Large sample sizes (as with the NSSE the mean difference or census- project) tend to produce more statistically significant results even though the by the pooled administered magnitude of mean differences may be inconsequential. Consult effect sizes standard deviation. In students are included to judge the practical meaning of the results. practice, an effect in these analyses. size of .2 is often Students in targeted Mean Comparisons considered small, .5 Mount Royal University compared with: or locally MRU Canada CA High Satisfaction NSSE 2011 moderate, and .8 administered Effect Effect Effect Class Mean a Mean a Sig b Size c Mean a Sig b Size c Mean a Sig b Size c large. A positive sign oversamples are not First-Year 53.8 51.1 *** .21 51.6 *** .17 53.2 .05 indicates that your Senior 55.8 55.5 .03 57.7 ** -.14 57.1 -.09 included. a Weighted by gender and enrollment status (and by institution size for comparison groups). institution’s mean b * p<.05 ** p<.01 ***p<.001 (2-tailed). c was greater, thus Mean difference divided by the pooled standard deviation. showing an Mean Distributions of Student Benchmark Scores affirmative result for The mean is the the institution. A weighted arithmetic First-Year Senior 100 100 negative sign average of the student indicates the level benchmark scores. institution lags 75 75 behind the comparison group, 50 50 suggesting that the Box and Whiskers Charts student behavior or A visual display of first-year institutional practice 25 25 and senior benchmark score represented by the dispersion for your institution item may warrant and your selected comparison 0 0 attention. MRU Canada CA High NSSE 2011 MRU Canada CA High NSSE 2011 Satisfaction Satisfaction or consortium groups.

95th Percentile Box and Whiskers Key A box and whiskers chart is a concise way to summarize the 75th Percentile variation of student benchmark scores. This display compares the distribution of scores at your institution, in percentile terms, with 50th Percentile/Median (Bar) that of your comparison groups. The ends of the whiskers show Mean (Dot) the 5th and 95th percentile scores, while the box is bounded by 25th Percentile the 25th and 75th percentiles. The bar inside the box indicates the median score, and the dot shows the mean score. 5th Percentile