Alberta Advanced Education Alberta Graduate Outcomes Survey Class of 2013/2014 Quantitative Fieldwork and Analysis Graduate Outcomes Survey Technical Report February 2017 Barry Davis Associate Vice-President, Alberta 10080 Jasper Avenue Edmonton, Alberta T5J 1V9 Tel: 780.423.0708 ext.242 Fax: 780.425.0400 Email: [email protected] 1 Contents 1. Executive Summary ......................................................................................................................3 1.1 Background and Objectives ....................................................................................................3 1.2 Methodology ...........................................................................................................................4 1.3 Questionnaire .........................................................................................................................4 1.4 Questionnaire Specifications ..................................................................................................4 1.5 Summary of Call Outcomes (Telephone Survey) ...................................................................4 1.6 Summary of Web Outcomes ..................................................................................................5 2. Survey Methodology .....................................................................................................................6 2.1 Sample Preparation ................................................................................................................6 2.2 Sampling Plan and Accuracy Targets ....................................................................................7 2.3 Survey Programming ..............................................................................................................8 3. Questionnaire Preparation ............................................................................................................9 3.1 Questionnaire Length .............................................................................................................9 3.2 Questionnaire Content ............................................................................................................9 3.3 Questionnaire Versions ..........................................................................................................9 3.4 Questionnaire Complexity ......................................................................................................9 3.5 DCaR Verification ...................................................................................................................9 3.6 Questionnaire Programming ................................................................................................ 10 4. Pre-Test ..................................................................................................................................... 11 4.1 Pre-Test Objectives ............................................................................................................. 11 5. Data Collection .......................................................................................................................... 13 5.1 Call Centre / Web ................................................................................................................ 14 5.2 Briefing of Interviewers ........................................................................................................ 15 5.3 Achievement of Targets ....................................................................................................... 15 5.4 Maximization of Response Rate .......................................................................................... 15 5.4.1 Language Barrier Cases ............................................................................................... 16 5.4.2 Directory Sourcing ........................................................................................................ 16 6. Data Cleaning/Analysis and Reporting ...................................................................................... 17 6.1 Data Cleaning ...................................................................................................................... 17 6.2 Preparation of the Final Dataset .......................................................................................... 17 6.3 Coding Open-Ended Responses ......................................................................................... 18 6.4 Final Data File ...................................................................................................................... 18 7. Survey Results ........................................................................................................................... 19 7.1 Publicly-Funded Post-Secondary Institution Survey Results .............................................. 19 7.2 Private Vocational Training Institution Survey Results ........................................................ 25 8. Initial Analysis ............................................................................................................................ 32 2 8.1 Non-Respondent Demographics ......................................................................................... 32 8.2 Gender ................................................................................................................................. 32 8.3 Age ...................................................................................................................................... 32 8.4 Credential Types.................................................................................................................. 33 8.5 Field of Study ....................................................................................................................... 33 8.6 Legal Status ......................................................................................................................... 34 8.7 Conclusions ......................................................................................................................... 34 9. Telephone vs. Online Completions............................................................................................ 35 9.1 Demographics ...................................................................................................................... 35 9.2 Satisfaction Levels ............................................................................................................... 36 9.3 Debts at Time of Graduation ............................................................................................... 36 9.4 Employment Outcomes ....................................................................................................... 37 9.5 Income ................................................................................................................................. 37 9.6 Conclusions ......................................................................................................................... 37 10. Effect of Data Weighting .......................................................................................................... 38 10.1 Gender ............................................................................................................................... 38 10.2 Age .................................................................................................................................... 38 10.3 Satisfaction Levels ............................................................................................................. 38 10.4 Debts at Time of Graduation ............................................................................................. 38 10.5 Outcomes .......................................................................................................................... 39 10.6 Income ............................................................................................................................... 39 10.7 Conclusions ....................................................................................................................... 39 11. Observations and Recommendations ..................................................................................... 40 11.1 Improve Sample Quality .................................................................................................... 40 11.2 Notify Graduates of Study ................................................................................................. 40 11.3 Continued Growth of Private Vocational Training Institution Participation ........................ 40 APPENDIX A: RESPONSE RATE AND MARGIN OF ERROR BY PROGRAM – BY PUBLIC INSTITUTION ................................................................................................................................ 41 APPENDIX B: RESPONSE RATE AND MARGIN OF ERROR BY PROGRAM – BY PRIVATE VOCATIONAL TRAINING INSTITUTIONS ................................................................................... 89 APPENDIX C: INITIAL QUESTIONNAIRE TESTING ................................................................. 115 A.1 Leger Comments ............................................................................................................... 115 APPENDIX D: SURVEY INSTRUMENT – GRADUATES ........................................................... 116 3 1. Executive Summary 1.1 Background and Objectives In December 2015, Leger Marketing Alberta Inc. (Leger)