Bridging an Education- Gap: A Case Study of Technical and Vocational in Taiwan

Szufang Chuang, Indiana State , USA

ABSTRACT

A disconnection between education, skills, and was concluded in many study reports around the world. This research was an exploratory study to develop and assess the effectiveness of a systematic, practice-oriented, course model to narrow an education-employment gap in Taiwan. A five- course evaluation model was developed and implemented for a course in Human Resource Management at a technical and vocational university in Taiwan. Enterprises, subject matter experts, and students were involved to ensure that the course was redeveloped successfully to cultivate the technical workforce the leisure and recreation industry requires. The study revealed increased student satisfaction and a potential to narrow the education-employment gap. In addition, the results suggested that the course evaluation model achieved positive results and is practical to implement in further studies. This study also offers new insights for scholars and practitioners involved in workforce development and training practice in Taiwan.

INTRODUCTION

Globalization has created a demand for a higher quality labor force with professional knowledge and work-related abilities and skills (Chiu & Chuang, 2016). Advances in technology changes employment and the nature of work and creates long-term structural changes. Yet, many graduates are unemployed or underemployed without proper skills that employers are looking for, which shows a disconnection between education, skills, and jobs (Chartered Global Management Accountant® 2014). Both the United Nations Educational, Scientific and Cultural Organization (2016) and the Chartered Global Management Accountant® (2014) recognize the employability issue where and skills shortage are occurring around the world and commit to address the global employability crisis. International Labour Organization (2013) estimated a global jobs gap of 67 million due to a result of economic crisis and expressed concerns over growing skills mismatch between supply of skills from institutions and demand of skills in the labor market. To overcome a skills shortage, International Labour Organization (2013) suggested and training enhancement and sufficient new employment opportunities. Government, education institutions, industry, and students themselves should work collaboratively to address the global issue of employability. Building an effective human capital plan requires trainers, adult educators, and human resource professionals to capture trends in industrial development and changes in the workforce in a timely manner. Technical and vocational higher education (TVHE) cultivated the workforce required for national economic construction and led Taiwan to become one of the Four Asian Dragons in the 1970s and 1980s. Currently, an increasing gap between employer needs and workforce skills has been a notable trend in Taiwan because of a lack of integration of vocational training in TVHE institutions (Siao, 2013). Further there are differences in equipment and materials utilized between and industry which also

150 The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue

contributes to the education-employment gap (Siao, 2013). Accordingly, Taiwan has been criticized for its lack of a technical workforce and a mismatch between employer’s expectations and university graduates’ competencies (Chang & Wang, 2017; Ho, 2015). University graduates who do not have appropriate competencies required by employers tend to remain unemployed longer than graduates with lower degrees do (Ho, 2015). A higher education diploma is insufficient unless it provides evidence of skills, competencies, and readiness to be trained in the specifics of a particular industry in Taiwan. In the same vein, the culture has shifted, such that higher education is expected to do more to train a student to be prepared for a . Narrowing the education-employment gap is an urgent issue and an important task for academic institutions and organizations. To address this issue, educational and vocational training systems must be improved with a strong connection between academic institutions and industries (Ho, 2015). The Taiwanese government has proposed national development plans designed to promote cooperative networks among enterprises, schools, and vocational training institutions (National Development Council, 2015). Specifically, industrial-academic collaboration is highly encouraged to ensure a greater link between education, skills, and jobs. Because advances in technology has increased people’s accessibility to higher education and demand for quality learning, a systematic course evaluation is critical to develop and deliver effective education programs or training that meet students’ and organizational needs. Therefore, the purpose of this study was to seek a way to narrow the education-employment gap between TVHE and industry in Taiwan. Specifically, the study sought to develop a systematic and practice-oriented course evaluation model and verify its effectiveness to narrow the gap between academic learning and industrial needs.

Discussion of Course Evaluation Course evaluation is an action that systematically investigates instructional design, course implementation, learning activity, and involvement to improve learning outcomes (Marsh & Willis, 1995). It is a series of evaluations of the course objectives and the effect of the course development process. It serves the purpose of improvement and should be more than simply a questionnaire on instructional assessment (Stufflebeam, 1983). According to Tyler (1991), the purposes of evaluation are to: (1) monitor current programs; (2) select a better program to replace the previous one; (3) assist with new program development; (4) identify a program’s effects; (5) estimate a program’s costs and effects, and (6) test a program’s relevance and validity. Based on Tyler’s notion (1980), the evaluation process should address various issues, including identifying objectives and outcomes, developing valid and reliable forms of assessment and methods of data analysis, and using the results of evaluations. The Tylerian evaluation rationale is a more traditional, objectives-based evaluation approach that is concerned primarily with achieving a specific set of objectives (Zhang et al., 2011), and training practitioners and educators have used it widely. For example, the Tylerian objectives-based approach has been used to examine levels of competencies accepted in the nursing profession (Maclean, 1992). It is important to note that many objectivist -based models of instructional design follow the Tylerian approach (Dick et al., 2002; Gagné, Briggs, & Wager, 1992; Smith & Ragan, 2005; Vrasidas, 2000). In addition to the Tylerian view of evaluation, Stufflebeam’s (2003) Context Input Process Product (CIPP) evaluation model is one of the evaluation models used most widely in education settings (Zhang et al., 2011). The principal concepts of the CIPP model include: context (are the mission and program goals being met?); input (does the quality and quantity of resources meet the needs?); process (to what extent were the program components implemented?), and product (what outcomes resulted from this program?:

The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue 151

Stufflebeam 2003). The model is “…a comprehensive framework for conducting formative and summative evaluations of projects, personnel, products, organizations, and evaluation systems” (Stufflebeam & Shinkfield, 2007, p.325). Many scholars and practitioners have used this model as a systematic framework to guide the conception, design, implementation, and assessment of training programs and to address the accountability of educational programs (Ho et al., 2011; Khalid et al., 2012; Rezaee & Shokrpour, 2011; Singh, 2004; Zhang et al., 2011). In one example, Rezaee and Shokrpour (2011) used the CIPP model as a conceptual framework to conduct performance assessments of academic departments and identify appropriate performance indicators, the results of which had a comprehensive effect on the students in the department’s programs. In another successful example, Ho et al. (2011) used a modified CIPP evaluation model to conduct a three-year evaluation project on a suicide prevention program in Taiwan; the study proved that the implementation of the program was valid and worthy of continued financial support. These examples provide strong evidence that the CIPP evaluation model helps organizations address performance evaluations of projects, personnel, and organizations. As Stufflebeam and Shinkfield (2007) stated, the CIPP evaluation model “…is configured especially to enable and guide comprehensive, systematic examination of social and educational projects that occur in the dynamic, septic conditions of the real world” (p. 351). One of the critical factors in course evaluation is instructional design (ID), which serves to promote learning’s effectiveness and minimize its difficulty (Merrison et al., 2007). According to Smith and Ragan (2005), ID requires four elements: (a) implementing instructional analysis; (b) designing instructional strategies; (c) evaluating instructional design and implementation, and (d) modifying instruction. A systematic ID involves analyzing instructional problems and student characteristics, setting learning goals, designing learning contents and strategies, mapping out learning sequences and outcomes, implementing course evaluation, and modifying the instruction (Merrison et al., 2007). Specific learning objectives and activities, course content, teaching methods, time, learning environment, assessments, supporting resources, and other facilitators are critical elements in ID. Similar to Smith and Ragan’s model, Branson’s (1975) Interservice Procedures for Instructional Systems Development (IPISD) model and Dick, Carey, and Carey’s (2002) instructional design model also were developed based on a systems approach to develop effective instruction and a successful learning environment. Interestingly, in a review of instructional designers’ work, Kenny et al. (2005) discovered that the designers did not follow the process-based ID models rigidly, but established other tasks instead in their professional activities that did not reflect ID models. There is no one-size-fits-all ID model, and an effective model should be modified based on purposes of instruction. Moreover, the 4-P model (problem clarification, planning, program action, and program evaluation) also can be used to facilitate teachers’ professional development and course evaluation (Lee et al., 2008). This was developed based on a two-year project in a university and Partnership for quality education. An ideal 4-P model is a spiral process and a continuing journey of action study. However, there is limited information regarding the application of the 4-P model in the instructional design literature. Research underpins course evaluation. It is an educational management strategy, as well as an educational research tool that ensures that teaching quality and learning objectives have been met. All of these models differ in their levels of complexity and specificity, but they are similar in the major phases of instructional design: analysis, design, development, implementation, and evaluation (Dick et al., 2002). Each model has its own strengths and emphases on instructional design and course evaluation that could benefit the development of a systematic and practice-oriented course evaluation model for a study like this.

152 The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue

CASE STUDY APPROACH AND FINDINGS

For the purpose of this study, a systematic, practice-oriented course evaluation model (see Figure 1) was developed and tested in a three-credit-hour undergraduate level course entitled Human Resource Management (HRM) at a TVHE university in Taiwan. The five-stage model was developed based on the Tylerian approach of evaluation, Stufflebeam’s CIPP evaluation model, Smith and Ragan’s instructional design model, and the 4-P model. The involvement of subject matter experts (SMEs), as well as Mager’s instructional objectives and Gagné’s Nine Events of Instruction, were incorporated in this model in an effort to enhance the quality of the course and the practicality of this practice-orientation model. The five stages of the model are described and discussed below.

Figure 1: A Practice-Oriented Course Evaluation Model

The first stage (problem clarification) assessed the needs, opportunities, and resources available for the course by seeking feedback from students and practitioners in the industry to identify instructional and learning problems and determine whether there is a gap between what is taught normally and industrial needs. It is necessary to confirm and clarify whether or not there is an education-employment gap at this stage. The second stage (course planning) entailed revising the course’s learning objectives, sequences, strategies, content, and outcomes based on the problems identified in the previous stage. The rules of Mager’s instructional objectives were used in conjunction with Gagné’s Nine Events of Instruction to design an engaging and meaningful instruction. The rules of the Mager-style Objectives, which historically have been used by instructional designers (Morrison, Ross, Kemp, & Kalman, 2010), were applied to revise the learning objectives. Each learning objective requires cognitive, psychomotor, or affective verbs with a specific criterion statement. Multiple instructional methods (e.g., lectures with and examples, class discussions, case studies, individual/group projects) were included and planned accordingly. The instructional strategies followed Gagné’s Nine Events of Instruction (i.e., gain the learners’ attention; inform them of objectives; stimulate recall of prior learning; present the content; provide learning guidance; elicit performance/practice; provide feedback; assess performance, and enhance retention: Gangé, Briggs, & Wager, 1992). This is a series of events that focuses on outcomes or behaviors of instruction that “…should be designed into any training program so as to facilitate the

The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue 153

trainee’s learning process” (Blanchard & Thacker, 2013, p. 76). According to Gangé, Briggs, and Wager (1992), course goals and objectives based on the content and students’ level of knowledge should be developed before implementing the nine events. To promote career readiness, the learning content should be both theoretical and practical, and consistent with a professional license. Any problems or issues identified in this stage required returning to Stage One for clarification. The revised course was implemented in the third stage (implementation). The implementation began with a pre-test to identify students’ current level of knowledge and skills. Multiple types of formative assessments (e.g., paper-pencil test, project, oral report, class participation, and class activity) were used to identify the learners’ progress in cognitive and skill development. During the learning process, the instruction was adjusted and revised as needed. Any problems or issues identified in this stage required returning to Stage One for clarification and/or Stage Two for modification. Stage Four (course evaluation) involved assessing the course’s effectiveness and learning outcomes. Summative evaluations, including a post-test, learner’s satisfaction survey, SMEs meeting/survey, and instructor self-evaluation, were employed at the end of the course. The results of the pre- and post-tests then were compared to determine learning effectiveness. To draw an objective conclusion in the course evaluation, additional feedback from students, SMEs, and instructor was collected to assess the course outcomes overall. Finally, the fifth stage (course revision) involved revising the course based on any problems or issues identified during the previous stages for continuous improvement. Following the course evaluation model, multiple datasets were collected from students, SMEs, and instructor. The findings of the five stages are reported in chronological order as follows.

A Gap Was Identified between Students’ Learning and Industry’s Expectations Learner analysis. The students who participated in this study were full-time undergraduates who enrolled in the HRM course and majored in Leisure and Recreation Industrial Management. They were required to receive at least one professional license and complete an before graduation for career readiness. The majority of the students already had an internship in the leisure and recreation industry for professional practice before taking the HRM course. They also had taken several management-related courses (e.g., Marketing Management, Accounting Management, Psychology, Fundamental of Leisure, Recreation Planning, and Development of Cultural Creative Industry). Learner focus groups. To identify the effectiveness of learning in the HRM course, previous students were contacted to obtain their feedback about their learning experience in the course. After the purpose of this study was explained, a total of 24 students (out of 48 students) agreed to participate by joining student focus groups (four to six people in one group). Each group spent approximately three hours sharing and discussing their experiences in the course. Many recommendations for course improvement also were collected with respect to instructional design, teaching methods, and learning materials and environment. The discussions and findings were divided into four categories (course design, instruction, the overall learning outcome, and feedback/suggestions) and are reported as follows: With respect to course design, approximately 50% (n = 12) of the previous students expressed a good understanding of the subjects. One learner said: “At the beginning of this class, I couldn’t understand it at all and had no idea what to expect. Yet, I later began to see the importance of HRM after a few lessons. I found the course was very useful.” Approximately 20% (n = 5) of the students indicated they did not understand the course material. For example, one student reported: “I tried to read it word- by-word, but I still could not understand the subjects. I guess it’s because I wasn’t really interested in this course.” In addition, over 70% (n = 17) of the students found the course very challenging. Although they

154 The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue

liked the idea and creativity of the major project, the grading criteria made it much more difficult than other courses. Lack of real-life assignments and practically-oriented case studies in a Taiwanese context also reduced their motivation to learn. In terms of the instruction, the students indicated they felt overwhelmed by the information covered and the teaching pace was too fast. Overall, the students believed that the course helped them understand the importance and significance of HRM to business, and the instruction contained a great level of detail that helped them absorb the information. They also felt their writing and oral presentation skills improved greatly because of the challenging project. The extra-credit activities also stimulated them to read before the class and enhanced their memory of the material. They also liked the option of using e-learning (an online learning system developed by the university) as a supporting learning tool even though the students’ participation rate in online discussion was low. Students’ recommendations of ways to improve the course included: using more examples, stories, and news when discussing learning subjects, adding more learning resources in the departmental library, offering more discussions (on policies/laws related to HRM, issues related to workplace safety, communication skills in the workplace, work ethics, etc.), and incorporating students’ work experiences in class discussions and activities. In addition, they indicated that the learning environment also needed to be improved because noises outside the classroom often created distractions. SMEs focus group. Four HRM practitioners (in the leisure and recreation industry) and university professors participated and met to evaluate the course outline, instructional design, and learning outcomes. A folder that contained information of the HRM course and students’ feedback from the focus groups was sent to them two weeks prior to the meeting. Their feedback and recommendations were synthesized and are reported in Table 1. In short, the results indicated that the course was not practice-oriented and also exhibited a disconnection between industrial needs and student technical skills.

Table 1: SMEs’ Feedback and Recommendations Concerning the Previous HRM Course Hospitality HRM practitioners scholars in HRM The learning purpose seemed to be mainly about The learning purpose was clearly stated, and the Learning acquiring knowledge. Adding statements of acquired design of a preview mechanism seemed to effectively purpose technique and attitude with more focus on industrial increase the learning outcomes. practice was recommended It was appropriate developed, and the fundamental It was appropriately planned and covered each area knowledge and concept of HRM was sufficient. More Learning of HRM. However, considering the students’ needs lessons in the national context (e.g., Taiwan Training scope for career and professional development, some Quality System), leisure or recreation related subjects adjustments could be made for non-HR managers. and practice oriented examples could be included. The course was systematically and creatively The instruction and evaluation activities corresponded designed. The proportion of practical exercise, case Course to the curriculum objectives. Yet, practical assignment study, group discussion, and project reports could be design was needed to increase learners’ interest and technical increased. It might be a good idea to divide students ability. into groups for field study. The learning subjects and activities were well Each subject was systematically arranged, and the designed. More case studies and class discussions learning activities were multi-faceted and practical. Learning were encouraged. Subjects such as foundation of However, more student-centered learning methods subjects, HRM, analysis, employee training, , were encouraged. Other suggestions included inviting activities leadership management, performance evaluation and HRM practitioners to the class to share their and course employee career planning could be emphasized experiences, arranging at least one field trip and hours rather than HR theories. A student reading list could incorporating learners’ interests into class assignment be provided to enhance the learning outcomes. or project.

The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue 155

The selections of textbook and teaching aids were Textbook appropriate. Even though using a translated textbook The instructor should be careful in the differences and might expand learners’ perspective and develop their between the translated textbook and the actual teaching global vision, the adaptability to Taiwanese industry practices in Taiwan. Case studies could be used to aid needed to be considered. Taiwanese culture and labor address this problem. laws could be included. Practically The fundamental knowledge was appropriate. Yet, Writing a recruitment announcement was a very designed there were some complicated issues included in the practical exercise. Similar class exercise could also be and text that could cause resistance to learning. Asking applied to other lessons. Inviting the industrial meeting students to conduct a field interview would enhance professionals in HR field to participate in the class the needs the connection between academic teaching and project or assignment could provide more practical of the industrial practice. feedback to the learners. industry

All in all, these findings confirmed Siao’s (2013) arguments about TVHE in Taiwan. Specific problems identified at this stage included: ● Problems of understanding learning objectives. ● Excessive course materials. ● Lack of real-life assignments and practically-oriented case studies in a Taiwanese context. ● Lack of industrial practice-oriented techniques and attitudes. ● Disconnection between academic teaching and industrial practice. ● Fail to address the differences between translated (western) textbook and the reality of workplace in Taiwan. ● Lack of student involvement in online discussions to promote peer learning. Recommendations for adjustment included: using field examples and stories in class discussions, arranging a field trip, inviting HRM practitioners to join the class and/or participate in class projects, providing more local and national case studies, and providing a reading list.

Learning Theories and Professional Licenses Can be Integrated in Instructional Design To address the problems identified in the first stage, the learning objectives of the HRD course were revised in accordance with the rules of Mager Style Objectives, which indicated a clearly observable and measurable action. The learning content was also revised to be consistent with the learning objectives and the Ability Training System (ATS) HRM license, a professional license the department encouraged students to obtain before graduation. This license attests to a human resource professional’s administrative ability and is issued by the International Professional Management Assembly’s Asian Pacific Region. The pyramidal hierarchical structure was adopted for the course content sequence. Based on behavioral psychology principles, this approach requires learners to demonstrate mastery of the content (Morrison, Ross, Kemp, & Kalman, 2010). Further, to improve reading comprehension, the textbook selection was based on the students’ reading level and knowledge about HRM, learning objectives, and other learning functions provided in the book (e.g., practice questions, chapter summary). Although not required by the university, an online learning system (called e-learning) was adopted to help students familiarize themselves with the online learning process and enhance their learning outcomes. Lesson plans, teaching methods, and learning materials (including textbooks, cases, examples, teaching aids) also were modified to be practically-oriented and relate to real-life situations. A total of eighteen lesson plans was redeveloped, each of which included specific learning objectives (using Mager- style learning objectives), learning sequence, instructional methods (e.g., lecture, online, and face-to-face

156 The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue

discussion, field trips, brainstorming, case studies, individual assignments, practical project, or group project) and instructional strategy (using Robert Gagné and Leslie Briggs’ The Nine Events of Instruction). In addition, instructional questions for the students also were developed in the lesson plan to ensure the students were engaged and followed the materials. Several case studies were selected based on the subject and the students’ possible interests. Cases in the Taiwanese context were emphasized, and long-winded, tedious, and complicated cases were avoided. After an extensive revision, the course was ready to be offered.

Creative Teaching Methods & Tools Should be Used to Facilitate Learning A total of 45 students enrolled in the revised HRM course. To determine their current level of knowledge related to HRM, they were given a pre-test, contained 16 true-false questions, 14 multiple choice questions, and 20 matching questions, in the first class. The result of the pre-test was used to compare with a post-test to measure the students’ improvement which was discussed in the next section. Various innovative teaching approaches and class activities were conducted as planned to increase students’ engagement in learning and enhance skills such as collaboration, creativity, and critical thinking. The e-learning system was used for timely class announcements, discussions, teaching aids, assignments, etc. In the system, students could submit their assignments, questions, feedback, and ideas, as well as download teaching aids, documents, pictures of class activities, and individual grades. They also used the chat room in the system for class discussions and the discussion board for group discussions. To promote peer learning and class involvement, a class learning portfolio was developed to collect all students’ projects and class activities (e.g., recruitment announcement, creative resume, career map, etc.). The portfolio was passed out during each class for everyone to review and discuss. Formative evaluations (class discussions, assignments, and exams) were conducted during and at the end of the course and indicated that the students had made their learning progress as planned. Overall, no major issues were identified when implementing this course.

Practice-Oriented Course Enhanced Learning Quality and Student Satisfaction At the end of the course, a summative evaluation was conducted to assess the quality of the course, students’ satisfaction, and the course effectiveness—whether the course was developed and delivered in a way that meets both students’ and industry’s needs. The feedback of students and SMEs, as well as the instructor’s self-evaluation and reflection, were gathered in this evaluation process. Comparing pre- and post-test results. A post-test that contained the same questions as the pre-test was conducted at the end of the course to assess the students’ improvement and the effectiveness of the course. Students were informed that the purpose of this test was to provide the instructor with a better understanding of their current level of knowledge of HRM, and the test results would not affect their academic score in the course. A total of 41 (91%) students completed both tests. A paired t-test indicated a significant difference in the pre-test (M = 39.93, SD = 8.22) and post-test scores (M = 47.27, SD =

10.29: t(40) = -4.68, p = 0.000). Approximately 76% (n = 31) of the students had higher scores on their post-tests, indicating a significant improvement in their HRM knowledge and skills. Over half (n = 21) of the students’ scores increased more than 10 points, and close to one-third (n = 12) of them scored more than 15 points higher on the post-test. Students’ satisfaction survey. An anonymous questionnaire was distributed in the last class to measure the students’ satisfaction and collect their feedback on the course. The questionnaire consisted of 25 questions, including one screening device question (to exclude invalid questionnaires), two open-

The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue 157

ended questions for recommendations and feedback, and 22 quantitative questions. A total of 42 (93%) of the students completed the satisfaction survey. The data collected were analyzed and are reported in Table 2. No dissatisfaction was found; however, the respondents reported that the midterm and final exams were quite challenging because too much information was covered in the exams. Students expressed their interests to learn more in subjects of human relations, communication skills, self-potential measurement, and finding the right job. Fewer paper assignments, more field trips, less learning content, and more practical examples, were recommended for future classes.

Table 2: Learner’s Satisfaction Survey: Satisfaction Level Items Ma SD Satisfaction levelb 1. Teaching quality 4.5 .55 Very satisfied 2. Instructor professionalism 4.4 .50 3. Class interaction 4.4 .59 4. Guidance provided by the instructor 4.3 .63 5. Course professionalism 4.2 .55 6. Course syllabus 4.1 .52 7. Learning content 4.1 .69 8. Course challenge 4.1 .82 9. Teaching methods 4.1 .69 10. Learning atmosphere 4.1 .79 11. Timely adjust the teaching methods based on learners’ needs 4.0 .70 12. Learning activities (recruitment announcement, creative resume, 30 most common Satisfied 3.9 .65 interview questions, etc.) 13. Practical exercises 4.0 .66 14. The overall course outcomes 4.0 .70 15. Class activities 4.0 .64 16. Course hours 3.9 .81 17. Semester project (i.e., HRM plan) 3.8 .88 18. The use of e-learning (an online classroom) 3.8 .87 19. Understandability of the course 3.8 .72 20. Individual assignments 3.7 .69 21. Textbook 3.7 .89 22. Test challenge 3.3 1.04 Neutral a 5 = very satisfied; 4 = satisfied; 3 = neutral; 2 = dissatisfied; 1 = very dissatisfied b Interpretation: 5-4.5= very satisfied; 4.4-3.5=satisfied; 3.4-2.5=neutral; 2.4-1.5=dissatisfied; 1.5-1=very dissatisfied.

SME survey. After the class finished, the practitioners were invited to assess the revised course plan and the overall learning outcomes again. The survey included seven open-ended questions (regarding learning purpose, scope, and subjects, textbook/teaching aids, class hours and activities, and course design) and one rating question to indicate to what degree the course had met the industry’s needs and expectations. A hotel manager with HR background in leisure industry reviewed feedback from previous SMEs and students and examined the improvement in the problems identified in this course. The manager’s feedback indicated that the learning purpose was stated clearly, the planning of the learning scope was appropriate, and the course design fit the needs of learners and the industry. In addition, the manager indicated that this HRM course prepared the students to meet at least 85% of industrial needs and expectations, although there was still room for improvement. For example, providing theories and industrial practice-oriented cases and assignments may be too much for a three-credit hour undergraduate

158 The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue

course. It is important to note that only one SME was able to complete the review and survey because the course ended during the leisure industry’s peak season. This is a limitation to this study. Instructor self-evaluation. The instructor conducted self-evaluations during and after the course. The self- evaluation was designed to elicit thoughts on the approach to teaching. The evaluation items included instructional objectives and strategies, teaching materials and methods, learning content, evaluation instruments, class interaction, student involvement, class preparation, and other components of teaching. According to the instructor, the relationship and interaction between the instructor and students improved greatly. The course became entertaining, and the students were more actively engaged in learning. The lesson plans were fully developed which saved class preparation time and allowed the instructor to pay more attention on the students’ learning pace and performance. A greater confidence of providing a practical-oriented course that meets the industrial needs was also reported. Finally, during the course evaluation process, it was interesting to discover that these young Taiwanese adult students tended to prefer active learning strategies (e.g., field trips, case studies, and discussion) and expected to receive more detailed explanations of learning subjects and assignments. They also were passive, learned at a slow pace, and depended on memorization for learning.

Course Should be Revised and Reevaluated for Continuous Improvement Based on the results of the summative evaluation in the previous stage, the course was revised slightly. For example, two practical examples were added to a student project, the learning subjects were edited, and one more field trip was planned. As the HRM course will be continually offered, another round of course evaluation process should be made for continuous improvement. In short, educational development has to be continuous as the course evaluation model indicates to catch up the current industry trends.

DISCUSSION AND IMPLICATIONS FOR EDUCATION AND TRAINING PRACTICES

Global higher education is faced with challenges related to employability of graduates. It is important for the higher education institutions to enhance their cooperation with industry and address societal needs. In an effort to address the global employability issue, a systematic, practice-oriented course evaluation model was developed and tested in a Human Resource Management course to bridge an education-employment gap between the leisure industry and TVHE in Taiwan. The course evaluation sought not only to provide substantial value for the students but also to bridge the gap between employer needs and workforce skills. This model sought enterprises’ and SMEs’ participation in the evaluation process specifically to ensure that the course helps cultivate the technical workforce required. Two key findings are discussed as follows. First, the five-stage course evaluation model was applicable and effective in upgrading an academic course that meets the industry’s needs. By adopting the course evaluation model, I first identified that the primary issue in the HRM course was the lack of incorporation of practice-oriented techniques and practical assignments and examples in learning content. Obviously, there also was a disconnection between academic teaching and industrial practice. In addition, there were other issues with respect to course objectives and the selection of learning materials. After following the model to redevelop the course, the results of formative and summative evaluations revealed a significant improvement in these issues. Approximately 76% of the students increased their professional knowledge and skills in the subject area successfully, and the students expressed no dissatisfaction with the course overall. Evidently, a systematic, practice-oriented course evaluation model is an effective tool to identify

The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue 159

learning problems and students’ needs and enhance a course’s quality and effectiveness. Thus, this course evaluation model can achieve significantly positive results in course development and is practical to deploy. Second, the education-employment gap can be narrowed through practical-oriented academic courses. The SME survey indicated that the course was designed effectively in a way that prepares students to meet the needs of the hospitality industry by at least 85%, which offers a great potential to narrow the gap between academic teaching and industrial practice. This study has implications for training and education on several levels. First, specific learning traits of young adults in leisure and recreation professions in Taiwan were identified—such as a preference for a particular active learning method and a tendency to memorize in learning. The identified traits may help training professionals and educators better understand the young adults in the leisure and recreation industry in Taiwan. Second, the conceptual framework proposed in this study can be used as a strategic tool to evaluate training programs and detect influential factors that enhance or hinder Taiwanese adults’ learning and professional development. Third, because this course evaluation model was developed based on models that address the accountability of education programs and a comprehensive examination of social and education projects, it may be applicable as well to training and development programs in the workplace. Training professionals and educators may adopt this model and associated instructional approaches to enhance employee learning outcomes, job performance, and return on investment. The information, concepts, and procedures presented in this study can also help educators, instructional designers, and training professionals develop effective and appealing instruction that meets employees’ and organizational needs.

LIMITATIONS AND RECOMMENDATIONS FOR FUTURE RESEARCH

The results of this study contributed to new knowledge about bridging an education-employment gap by developing practical academic courses in the Taiwanese context. The results indicated the possibility of closing the gap between TVHE and industry in Taiwan by using a systematic and practice- oriented course evaluation model to enhance graduates’ career readiness. Despite these contributions, the study had limitations that suggest several avenues for future research. First, one may raise concerns about the applicability of the results in today’s rapidly changing workplace. The literature has indicated that bridging the gap between university graduates’ skills and employees’ expectations has been and will continue to be a focus of Taiwanese educators and researchers (Chang & Wang, 2017; Chiu & Chuang, 2016; Ho, 2015; Wang & Tsai, 2014). In particular, Taiwanese hospitality graduates reported lack of confidence in their ability to find employment (Wang & Tsai, 2014). Education and skills/performance gap is a contemporary issue in Taiwan, just as in India, the United States, and the United Kingdom (Unni, 2016; Cappelli, 2015; Webb & Chaffer, 2016). Although the results of this study are relevant to today’s employment and education environment, future researchers are encouraged to replicate this study to validate the model. Follow-up research on the employment of the students involved in this study, as well as their employers’ satisfaction, also is needed to obtain a clear indication of the program’s success in narrowing the education-employment gap. Second, because of time constraints, only one SME participated in the final review. The generalizability of the course evaluation model could be an issue. However, it was clear that the model can enhance the overall outcome of the course evaluated. Nonetheless, more SMEs and the graduates’ employers should be involved in the final stage of evaluation to obtain objective feedback and reach reliable conclusions.

160 The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue

A third limitation is related to generalize the findings of this study. Because the education- employment gap is such a broad and dynamic topic, neither qualitative nor quantitative studies can provide a comprehensive overview, especially when conducted in only one academic institution in a country. Future researchers are encouraged to conduct the study in other cultural contexts or different academic settings using action research methods. This would advance both theories and practices in creating an effective human capital plan and allow researchers to judge whether the model is beneficial in a general setting, or just in the setting studied here. The approach of the course evaluation could be useful for program evaluation. An evaluation on the degree program would also benefit the overall outcome of the course evaluated and bridging the education-employment gap. Special attention should be paid to students’ cultural backgrounds and learning preferences when assessing the course evaluation model in different countries. Finally, studies in the roles of governmental intervention, social and work structures, and the potential and challenge of technology on bridging the education-employment gap are also needed to bring in national perspective into the discussion.

REFERENCES

Blanchard, P.N. & Thacker, J.W. (2013). Effective training: Systems, strategies, and practices (5th ed.). Upper Saddle River, NJ: Pearson Education. Branson, R.K. (1975). Interservice procedures for instructional systems development: Executive summary and model. Tallahassee, FL: Center for Educational Technology, Florida State University. Cappelli, P. H. (2015). Skill gaps, skill shortages, and skill mismatches: Evidence and arguments for the United States. ILR Review, 68(2), 251-290. Chang, H. & Wang, W. (2017). Breakthrough on technical and vocational education of Taiwan: Take oriental as an example. In AIP Conference Proceedings 1836, 020035. Chartered Global Management Accountant®. (2014). Addressing the Employability Crisis. Retrieved from https://www.cgma.org/resources/reports/addressingtheemployabilitycrisis.html Chiu, S. & Chuang, H. (2016). Employability and compensation in an Asian economy: Evidence for female graduates in Taiwan. Emerging Markets Finance & Trade, 52, 853–868. Dick, W., Carey, L. & Carey, J.O. (2002). The systematic design of instruction (5th ed.). New York, NY: HarperCollins College Publishers. Gagné, R.M., Briggs, L.J. & Wager, W.W. (1992). Principles of instructional design (4th ed.). Forth Worth, TX: Harcourt Brace Jovanovich College Publishers. Ho, H. (2015). Matching university graduates’ competences with employers’ needs in Taiwan. International Education Studies, 8(4), 122-133. Ho, W.-W., Chen, W.-J., Ho, C.-K., Lee, M.-B., Chen, C.-C. & Chou, F. H.-C. (2011). Evaluation of the suicide prevention program in Kaohsiung City, Taiwan, Using the CIPP evaluation model. Community Mental Health Journal, 47(5), 542-550. International Labour Organization. (2013). Global employment trends 2013: Recovering from a second jobs dip. Retrieved from http://www.ilo.org/wcmsp5/groups/public/---dgreports/---dcomm/---publ/documents/ publication/wcms_202326.pdf Kenny, R.F., Zhang, Z., Schwier, R.A. & Campbell, K. (2005). A review of what instructional designers do: Questions answered and questions not asked. Canadian Journal of Learning and Technology, 31(1), 9-26. Khalid, M.M., Ashraf, M. & Rehman, C.A. (2012). Exploring the link between Kirkpatrick (KP) and context, input, process and product (CIPP) training evaluation models, and its effect on training evaluation in public organizations of Pakistan. African Journal of Business Management, 6(1), 274-279.

The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue 161

Lee, C.J., Yin, H. & Zhou, X. (2008). Facilitating teacher professional development through ‘4-P model’: Experiences from Hong Kong’s partnership for improvement of learning and teaching (PILT) project. Journal of Educational Research and Development, 4(2), 17-48. Maclean, B.L. (1992). Technical curriculum models: Are they appropriate for the nursing profession. Journal of Advanced Nursing, 17, 871-876. Marsh, C.J. & Willlis, G. (1995). Curriculum: Alternative approaches, ongoing issues. Englewood Cliffs, NJ: Merrial. Morrison, G. R., Ross, S. M., Kemp, J. E. & Kalman, H. K. (2010). Designing effective instruction (6th ed.). Hoboken, NJ: John Wiely & Sons. National Development Council. (2015). National development plan, 2015. Retrieved from http://www.ndc.gov.tw/en/cp.aspx?n=72DB7DAE729FA530&s=C7C822667A8F469F Rezaee, R. & Shokrpour, N. (2011). Performance assessment of academic departments: CIPP model. European Journal of Social , 23(2), 227-236. Siao, Y.-J. (2013, June 7). Policy discourse: Building practical technical and vocational education. Higher Education, Technological & Vocational Education Newsletter, 78. Retrieved from http://www.tekezgo.com/index.php?do=news&act=detail&id=77. Singh, M. D. (2004). Evaluation framework for nursing education programs: Application of the CIPP model. International Journal of Nursing Education Scholarship, 1(1), 1-16. Smith, P. L. & Ragan, T. J. (2005). Instructional design (3rd ed.). New York, NY: Merrill/Macmillan. Stufflebeam, D. L. (1983). The CIPP model for program evaluation. In G. F. Madams, M. S. Screven & D. L. Stufflebeam (Eds.), Evaluation models: Viewpoints on educational and human service evaluation (pp.117-141). Boston, MA: Kluwer-Nijhoff. Stufflebeam, D. L. (2003). The CIPP model for evaluation. Proceedings of the 2003 Annual Conference of the Oregon Program Evaluation Network (OPEN), October 3, Portland, Oregon. Stufflebeam, D. L. & Shinkfield, A. J. (2007). Evaluation theory, models, & applications. San Francisco, CA: Jossey-Bass. Tyler, R. W. (1980). A brief overview of program evaluation. Journal of , 16(3), 7-15. Tyler, R. W. (1991). General statement on program evaluation. In M. W. McLaughlin & D. D. Phillips (Eds.), Evaluation and education: At quarter century (pp.1-17). Chicago, IL: University of Chicago Press. United Nations Educational, Scientific and Cultural Organization. (2016). Education 2030 incheon declaration and framework for action: For the implementation of sustainable development goal 4 ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. Retrieved from http://unesdoc.unesco.org/images/0024/002456/245656e.pdf Unni, J. (2016). Skill gaps and employability: Higher education in India. Journal of Development Policy and Practice 1(1), 18-34. Vrasidas, C. (2000). Constructivism versus objectivism: Implications for interaction, course design, and evaluation in distance education. International Journal of Educational Telecommunications, 6(4), 339-362. Wang, Y. & Tsai, C. (2014). Employability of hospitality graduates: Student and industry perspectives. Journal of Hospitality and Tourism Education, 26(3), 125-135. Webb, J., & Chaffer, C. (2016). The expectation performance gap in accounting education: A review of generic skills development in UK accounting degrees. Accounting Education 25, (4), 349-367. Zhang, G., Zeller, N., Griffith, R., Metcalf, D., Williams, J., Shea, C. & Misulis, K. (2011). Using the context, input, process, and product evaluation model (CIPP) as a comprehensive framework to guide the planning, implementation, and assessment of service-learning programs. Journal of Higher Education Outreach and Engagement, 15(4), 57-84.

162 The Journal of Human Resource and Adult Learning, Vol. 15, Num. 1, June, 2019 issue