An Update of the Collaborative Leadership Model

Competency development and validation 73

Competency Development and Validation: An Update of the Collaborative Leadership Model

Abby Swanson Kazley, PhD, Edward J Schumacher, PhD, Jami Dellifraine, PhD, Dolores Clement, DrPH, Randa Hall, MBA, Steve O’Connor, PhD, Haiyan Qu, PhD, Rick Shewchuk, PhD, & Mary Stefl, PhD

Abstract Competency-based education has taken hold in the health administration field, and accreditation from CAHME now requires programs to adopta competency-based model when developing curriculum and assessing student and program success. This paper describes the development and validation of an updated and revised competency model used by four MHA programs in the Southeast. The programs first described the development of a model in 2010 and here present the updated and simplified model based on faculty, alumni, and preceptor feedback. The group used a survey to gather input from each of the stakeholders and analyzed the responses to validate the model in a parsimonious way. Surveys were sent via email with 603 complete and usable survey respondents included in the analysis. The results provide rankings and agreement between programs and constituencies within five domain areas including communications and relationship management, leadership, professionalism, knowledge of healthcare environment, and business knowledge and skills.

Please address correspondence to: Abby Swanson Kazley, PhD, Associate Professor, Depart- ment of Health Care Leadership and Management, Medical University of South Carolina, 151 Rutledge Ave., Charleston, SC 29425, Email: [email protected], Phone: 843-792-0012 74 The Journal of Health Administration Education Winter 2016

Introduction and Background There has been an increased push for competency-based education in the health professions for the past two decades (Calhoun et al., 2008). Competency- based education in health administration and management is now a standard for the Commission on Accreditation of Healthcare Management Education (CAHME)-accredited programs. Accredited programs are required to have a competency model, periodically evaluate the relevancy of the model, and use competency assessments to make program improvements (CAHME, 2015). Competencies must align with the program’s mission, be integrated into the curriculum, and measured and assessed at the course and program levels (CAHME, 2015). Health administration programs, with the input of alumni and other stakeholders, must periodically evaluate each competency in their respective model to ensure that the competencies attained by their graduates are relevant for professional practice. In 2007, four CAHME-accredited graduate programs in health administration – Medical University of South Carolina (MUSC), Trinity University, University of Alabama at Birmingham (UAB), and Virginia Commonwealth University (VCU) – collaboratively developed and validated a competency model based on the Healthcare Leadership Alliance (HLA)1 Competency Model and other program-specific competency sets (Clement et al., 2010). The validated model was adopted and subsequently fully implemented in each of these programs. However, the healthcare industry is changing at a very fast pace, and it is unclear if the competencies needed for early careerists identified seven years ago are still relevant for today’s recent graduates. The purpose of this study is to collectively re-evaluate and update the Collaborative Leadership competency model developed to drive curriculum design and student development in each program. This paper will describe which domain and competency items have become more or less important to alumni, preceptors, and faculty during the last six years. Furthermore, this study will demonstrate an approach and method to competency evaluation and revalidation that can be used by other programs to assist them as they re-evaluate their own competency model to ensure ongoing relevance and prepare for their site visits and CAHME reaccreditation.

1The HLA is a consortium of major professional associations: American College of Healthcare Executives (ACHE), American College of Physician Executives (ACPE), American Organization of Nurse Executives (AONE), Healthcare Financial Man- agement Association (HFMA), Healthcare Information and Management Systems Society (HIMSS), and Medical Group Management Association (MGMA). Competency development and validation 75

The purpose of this study is to collectively re-evaluate and update the Collaborative Leadership competency model developed to drive curriculum design and student development in each program. This paper will describe which domain and competency items have become more or less important to alumni, preceptors, and faculty during the last six years. Furthermore, this study will demonstrate an approach and method to competency evaluation and revalidation that can be used by other programs to assist them as they re-evaluate their own competency model to ensure ongoing relevance and prepare for their site visits and CAHME re-accreditation. During the last 20 years, graduate professional programs have moved toward competency-based curricula in higher education, with an increased focus on the assessment of educational outcomes (Calhoun et al., 2008). Compe- tency based education is intended to engender student capabilities – including knowledge, skills, attitudes, and behaviors (Westera, 2001) – that positively affect performance upon graduation. A number of healthcare management programs, organizations, and initiatives have focused on identifying and understanding appropriate competencies for recent graduates in healthcare management education programs (Shewchuck, O’Connor, & Fine, 2006; Gar- man & Johnson, 2006; Calhoun et al., 2002; Stefl, 2008; Campbell et al., 2006). These studies suggest that there is a common core set of competencies for healthcare management graduates (Shewchuck, O’Connor, & Fine, 2006; Stefl, 2008). However, because there is variation in program mission, design, curricula, and students, there is some variation in program competency model development. CAHME accreditation requires that programs “adopt a set of competencies that align with the mission and types of jobs graduates enter (and) … use these competencies as the basis of the curriculum, course content, learning objectives, and teaching and assessment methods” (CAHME, 2015). Because the field of healthcare is not static (and curricula are not static), healthcare management programs should periodically re-validate their own competency models to ensure they align with the program’s mission, curriculum, and the evolving healthcare environment.

Program competency history In 2007, MUSC, Trinity University, UAB, and VCU’s health administration programs were all independently attempting to develop a competency model. Because these programs were similar in many ways (i.e., they are all in the South/Southeast, were of similar size, had similar missions and educational formats), the programs decided to collaborate and develop and validate a common competency model. The competency items were compiled into a survey and administered to recent program graduates, faculty, and preceptors of the 76 The Journal of Health Administration Education Winter 2016 four programs. Survey respondents were asked to rank the competencies in order of importance for early careerists in health administration. Results from the survey showed significant consistency in the rank-ordered importance of each competency, indicating strong consensus among stakeholders from all four programs. These results were then used to implement a validated competency model in each of the programs (Clement et al., 2010). However, the healthcare environment has changed with the passage of the Affordable Care Act (ACA) and the start of its implementation over recent years. This analysis was started to evaluate the collaborative competency model developed in 2008 to drive curriculum design and assess student development, and to examine its current relevance and validity.

Methods To assess the initial competency model after six years, the group from the four university programs convened for a day and a half in Nashville to discuss how it had been used in each of the programs, the metrics for measurement, how to proceed with an evaluation of the reliability and validity of the model, and how it might be improved. The group included representatives from each of the four programs with program directors, faculty, and department chairs participating. Given the nature of the dynamic and changing field of healthcare, the group believed that an update was necessary. The group reviewed the initial model, measures and results over the years since adoption, and feedback from faculty on commonalities among several of the competencies. The group also discussed the complexity and burden of measuring competency assessment at multiple points throughout the programs, including the fact that some competencies are not easily observed (i.e., commitment to lifelong learning, time management), and that parsimony would be a goal for the new model. In performing the review, it became clear that some of the initial competencies overlapped (such as Human Resources and Healthcare Personnel) when aligning learning objectives with specific competencies. The discussion concerning overlap and clarity of competency content led to an updated version of 25 competencies instead of the initial 30. The group was clear that the five competencies that had been “eliminated” were still important and the substance was indeed included within the remaining competencies. The descriptions of the remaining competencies were updated to ensure incorporation of the knowledge, skills, and attributes in a general way while seeking parsimony in the model. The group was also concerned whether items may have been initially omitted from the original model, and Competency development and validation 77 further refined definitions to ensure clarity of understanding of the model. Overall, there was pronounced agreement among the faculty, and discussions were held until consensus was reached. Table 1 provides a comparison of the original and revised models. The competencies are divided into the five domains that resulted from the initial factor analysis (Clement et al., 2010) and align with the overall HLA model. Under the Communications & Relationship Management domain, the only change was to move Working in Teams into the Professionalism domain. The label Planning and Implementing Change was changed to Change Manage- ment. In the Professionalism domain, Continuing Education and Lifelong Learning was eliminated as a separate competency as it is subsumed within Professional and Community Contribution. Time Management was eliminated as an individual competency, as it was viewed to be incorporated within Project Management and other competencies such as Change Management. The competencies Standards and Regulations and Risk Management, were combined with Legal Principles. Healthcare Personnel was subsumed by Human Resources. In addition, the competencies Health Policy Formula- tion, Implementation and Evaluation as well as Population Health and Status Assessment were moved out of Business Knowledge & Skills and into the Knowledge of the Healthcare Environment domain.

Table 1 Original vs. Revised Competency Model

Domain Original Model Revised Model I. Communications Interpersonal Communica- Interpersonal Communi- & Relationship tion cation Management Working in Teams Presentation Skills Presentation Skills Writing Skills Writing Skills

II. Leadership Leading and Managing Leading and Managing Others Others Systems Thinking Systems Thinking Planning and Implement- Change Management ing Change Ability for Honest Self- Ability for Honest Self- Assessment Assessment 78 The Journal of Health Administration Education Winter 2016

Table 1, cont.

Problem-Solving and Decision-Making

III. Professionalism Personal and Professional Personal and Professional Ethics Ethics Continuing Education and Lifelong Learning Professional and Commu- Professional and Com- nity Contribution munity Contribution Working in Teams

IV. Knowledge of Problem-Solving and Healthcare Envi- Decision-Making ronment Healthcare Issues and Healthcare Issues and Trends Trends Healthcare Personnel Standards and Regulations Health Policy Formula- tion, Implementation, and Evaluation Legal Principles Develop- ment, Application, and Assessment Population Health and Status Assessment

V. Business Knowl- Planning and Managing Planning and Managing edge and Skills Projects Projects Quantitative Skills Quantitative Skills Time Management Financial Management Financial Management Quality/Performance Im- Quality/Performance provement Improvement Strategic Planning Strategic Planning Organizational Dynamics Organizational Dynamics and Governance and Governance Competency development and validation 79

Information Management/ Information Manage- Understanding and Using ment/Understanding and Technology Skills Using Technology Skills Economic Analysis and Economic Analysis and Application to Business Application to Business Decisions Decisions Human Resources Human Resources Risk Management Marketing Marketing Legal Principles Develop- ment, Application, and Assessment Health Policy Formula- tion, Implementation, and Evaluation Population Health and Status Assessment

The group agreed that to validate the model, input was needed from stakeholders. An online survey was designed to collect information about the importance of each of the competencies for early careerists in health administration. The study was IRB approved. The survey was sent to alumni of each program, preceptors of fellowships or internships that had been held by each of the programs’ graduates or students, and faculty from each of the programs. Alumni were included in the sample if they graduated from either the executive or residential format programs within the previous 10 years. Each group also submitted the names and contact information of preceptors from the previous 10 years who were not alums, and the group verified that there were unique preceptors without overlap due to the fact that preceptors at some of the organizations had supervised students from multiple programs among the four universities. Any alumni from the past 10 years who also had been a preceptor were sent the alumni survey. All full-time faculty at each of the programs received the survey. An invitation from the program director at each respective program was sent to the alums, preceptors and faculty of his or her program, and a reminder email about the survey was sent within two weeks to nonrespondents. The surveys asked stakeholders to rank the importance of each of the updated 25 competencies for an early careerist MHA on a five-point Likert scale. Respondents also had the opportunity to provide feedback in an open- 80 The Journal of Health Administration Education Winter 2016 text box to advise the team whether they believed anything had been left out of the model. Once the surveys were closed, the data were analyzed in the collective and by individual program. Univariate t-tests were used to examine whether the change in competency score from 2007-2013 was significant overall, by program, and by respondent. Descriptive data and rankings were calculated to identify competencies that were viewed as the most important for new graduates in health administration.

Results There were 603 complete and usable surveys received from the 2013 survey, as compared to 327 from the 2007 survey. The response rates for 2007 and 2013 were 47.3% and 40.3%, respectively. Table 2 compares characteristics of the two surveys. Alumni are the largest category of respondents, accounting for almost 80% of the 2013 survey and 70% of the 2007 survey. The increase in the number of alumni between surveys accounts for most of the increase in the sample size. There were 228 usable surveys from alumni in 2007 compared to 481 in 2013. Notable is the high level of experience held by preceptors. Almost 80% of the 2007 respondents and over 70% of the 2013 respondents have more than 20 years of healthcare management experience. The number of faculty respondents stayed constant over the two surveys, but there is a slight increase in the level of experience among the faculty.

Table 2

Characteristics of survey participants

2007 2013 N % N % Total 327 100 603 100 Faculty 41 12.5 41 6.8 UAB 15 36.6 12 29.3 MUSC 11 26.8 7 17.1 Trinity 6 14.6 10 24.4 VCU 9 22.0 12 29.3

1-9 years 15 36.6 10 24.4 10-20 years 12 29.3 15 36.6 >20 years 14 34.1 16 39.0 Competency development and validation 81

Table 2, cont.

Preceptors 58 17.1 81 13.4 UAB 16 27.6 13 16.0 MUSC 8 13.8 11 13.6 Trinity 11 19.0 44 54.3 VCU 23 39.7 13 16.0

1-9 years 3 5.2 2 2.5 10-20 years 9 15.5 21 25.9 >20 years 46 79.3 58 71.6

Alumni 228 69.7 481 79.8 UAB 74 32.5 165 34.3 MUSC 33 14.5 137 28.5 Trinity 61 26.8 78 16.2 VCU 60 26.3 101 21.0

Survey data were analyzed by first calculating the mean and standard deviation for each category overall as well as within each stakeholder group (i.e., alumni, preceptor, and faculty). Similar to the 2007 survey, results from 2013 showed strong statistically significant correlations among stakeholders, indicating that they view the relative importance of these competencies in a similar fashion. Table 3 displays means and standard deviations for each competency by year combining stakeholders. The results presented here combine the four programs. Separate analysis by university reveals highly similar conclusions. In addition, for competencies that moved across domains, we use the revised model designation for the comparative analysis. The table also shows the overall competency rankings for each survey year. The table identifies where there was a statistical difference in the score given to the competency; a single asterisk indicates a significant difference at the p<.05 level and two asterisks indicate a significant difference at the p<.01 level. 82 The Journal of Health Administration Education Winter 2016

Table 3 Comparison of Competency Survey Evaluations Between 2007 to 2013: All Respondents

2007(n=327) 2013 (n=603) Rank Competency Mean SD Rank Mean SD Rank Change Communication & Rela- A. 4.71 0.38 4.61 0.44 tionship Management** Interpersonal Communi- 4.91 0.34 1 4.77 0.51 1 0 cation** Presentation Skills 4.65 0.54 7 4.58 0.56 9 2 Writing Skills 4.56 0.6 11 4.46 0.65 14 3 B. Leadership 4.62 0.41 4.57 0.45 Ability for Honest Self- 4.44 0.72 15 4.33 0.75 16 1 Assessment* Leading and Managing 4.74 0.53 5 4.63 0.65 6 1 Others** Change Management 4.57 0.61 10 4.53 0.67 10 0 Problem-Solving and 4.76 0.48 4 4.73 0.52 2 -2 Decision-Making Systems Thinking 4.58 0.63 8 4.64 0.59 4 -4 C. Professionalism** 4.56 0.42 4.4 0.51 Personal and Professional 4.82 0.46 2 4.63 0.63 5 3 Ethics Professional & Commu- 4.08 0.72 21 3.88 0.83 24 3 nity Contribution** Working in Teams* 4.77 0.48 3 4.68 0.59 3 0 D. Knowledge of the Healthcare Environ- 3.94 0.57 4.14 0.60 ment** Health Policy Formula- tion, Implementation, 3.71 0.8 24 4.12 0.72 20 -4 and Evaluation** Healthcare Issues and 4.51 0.6 13 4.48 0.66 13 0 Trends Legal Principles Develop- ment, Application and 3.88 0.78 22 4.14 0.76 19 -3 Assessment** Population Health and 3.65 0.77 25 3.83 0.9 25 0 Status Assessment** Competency development and validation 83

Table 3, cont.

E. Business Knowledge 4.32 0.42 4.31 0.45 and Skills Economic Analysis and Applications to Business 4.18 0.76 19 4.02 0.81 22 3 Decisions* Financial Management 4.53 0.6 12 4.6 0.6 8 -4 Human Resources 4.11 0.71 20 4.07 0.77 21 1 Information Manage- ment/Understanding and 4.19 0.7 18 4.19 0.73 18 0 Using Technology Marketing 3.88 0.72 23 3.97 0.76 23 0 Organizational Dynamics 4.21 0.68 17 4.2 0.76 17 0 and Governance Planning and Managing 4.68 0.53 6 4.6 0.59 7 1 Projects* Quality/Performance 4.45 0.65 14 4.46 0.69 15 1 Improvement Quantitative Skills* 4.58 0.57 9 4.5 0.64 12 3 Strategic Planning 4.36 0.64 16 4.52 0.65 11 -5 Overall 4.39 0.36 4.38 0.40

*indicates the mean for 2007 is statistically different from the mean in 2013p <.05 ** indicates p < .01

The broad ranking of the competency domains remained the same over the two surveys. These domains are ranked from highest to lowest as fol- lows: (1) Communication and Relationship Management; (2) Leadership; (3) Professionalism; (4) Business Knowledge & Skills; and (5) Knowledge of the Healthcare Environment. Communication and Relationship Management saw a significant decrease in ranking while Leadership stayed constant. Thus the difference between the two domains in relative score is much smaller in the 2013 survey than in the 2007 survey. In fact, the average rank of the competencies in the Communication and Relationship Management Domain in 2013 is 8.0 (1st, 9th, 14th) while the average rank of the Leadership competencies is 7.6 (16th, 6th, 10th, 2nd, and 4th). Increases in the ranking of the Problem Solving and Decision Making competencies, as well as Systems Thinking, have elevated the Leadership Domain. 84 The Journal of Health Administration Education Winter 2016

Looking at the individual competencies, remarkably, there is consider- able stability in the rankings between survey years. Interpersonal communication remained the top ranking competency. It ranks first in both survey years when results are aggregated, as well as with separate analysis within the key stakeholders. Other competencies ranking high in both years were Problem-Solving and Decision-Making, Working in Teams, and Personal and Professional Ethics. Statistically significant increases occurred in three of the four competencies within the Knowledge of the Healthcare Environment domain. While the overall ranking for these categories remained low, these competencies’ ratings were found to be significantly more important to respondents. Health Policy Formulation, Implementation and Evaluation increased from 24th to 20th, while Legal Principles Development and Application and Assessment increased from 22nd to 19th. Although the ranking for Population Health and Status Assessment remained last, its score increased significantly. Under the domain of Business Knowledge & Skills, there was a significant increase in the rating of Strategic Planning with its rank increasing from 16th to 11th. While the increase in the rating of Financial Management is not statistically significant, its ranking increased from 12th to 8th. Likewise, Systems Think- ing under the Leadership Domain increased in ranking from 8th to 4th even though its score did not increase statistically. Rankings by faculty show similar results and are found in Table 4. Since the sample sizes are smaller, there are fewer statistically significant differences, but the results are quite similar for the combined results. Health Policy For- mulation, Implementation and Evaluation, and Population Health and Status Assessment both saw increases in rating and rankings. Strategic Planning and Financial Management remained relatively constant in faculty rating. Writing Skills increased from 7th to 3rd in faculty ratings even though the mean score did not change statistically. Faculty are the only stakeholders to rank Writing Skills in the top five, and to increase the relative rank of that competency. Competency development and validation 85

Table 4 Comparison of Competency Survey Evaluations Between 2007 to 2013: Faculty

2007(n=41) 2013 (n=41) Rank Competency Mean SD Rank Mean SD Rank Change Communication & Rela- A. 4.78 0.30 4.72 0.43 tionship Management Interpersonal Commu- 4.93 0.26 1 4.78 0.47 1 0 nication Presentation Skills 4.73 0.45 5 4.63 0.58 7 2 Writing Skills 4.68 0.52 7 4.73 0.5 3 -4 B. Leadership 4.55 0.41 4.56 0.44 Ability for Honest Self- 4.46 0.67 14 4.34 0.73 17 3 Assessment Leading and Managing 4.66 0.53 8 4.66 0.53 6 -2 Others Change Management 4.37 0.62 17 4.41 0.74 15 -2 Problem-Solving and 4.76 0.43 4 4.76 0.49 2 -2 Decision-Making Systems Thinking 4.49 0.75 13 4.61 0.59 10 -3 C. Professionalism** 4.59 0.33 4.41 0.52 Personal and Professional 4.93 0.26 2 4.66 0.62 4 2 Ethics* Professional & Commu- 4.07 0.72 22 3.93 0.79 24 2 nity Contribution Working in Teams 4.78 0.42 3 4.66 0.53 5 2 D. Knowledge of the 4.09 0.41 4.28 0.50 Healthcare Environment Health Policy Formula- tion, Implementation, 4.00 0.59 23 4.28 0.64 18 -5 and Evaluation* Healthcare Issues and 4.71 0.56 6 4.59 0.63 11 5 Trends Legal Principles Develop- ment, Application and 3.78 0.65 25 4.05 0.71 23 -2 Assessment Population Health and 3.88 0.71 24 4.22 0.61 21 -3 Status Assessment* 86 The Journal of Health Administration Education Winter 2016

Table 4, cont.

E. Business Knowledge 4.42 0.35 4.35 0.49 and Skills Economic Analysis and Applications to Business 4.29 0.64 20 4.07 0.75 22 -2 Decisions Financial Management 4.63 0.54 9 4.63 0.58 8 -1 Human Resources 4.34 0.66 19 4.24 0.73 20 1 Information Manage- ment/Understanding and 4.39 0.63 16 4.39 0.7 16 0 Using Technology Marketing 4.1 0.62 21 3.88 0.75 25 4 Organizational Dynamics 4.37 0.58 18 4.24 0.7 19 1 and Governance Planning and Managing 4.59 0.5 10 4.63 0.54 9 -1 Projects Quality/Performance 4.54 0.5 11 4.56 0.59 12 1 Improvement Quantitative Skills 4.54 0.5 12 4.44 0.63 13 1 Strategic Planning 4.41 0.71 15 4.41 0.67 14 -1 Overall 4.46 0.30 4.43 0.43

*indicates the mean for 2007 is statistically different from the mean in 2013p <.05

Table 5 shows the rating by preceptors. Consistent with the overall results, there are significant increases in the ratings for competencies around the Knowledge of the Healthcare Environment domain. And while the scores do not change significantly, relative rankings increase for Systems Thinking, Financial Management, and Strategic Planning. Competency development and validation 87

Table 5 Comparison of Competency Survey Evaluations Between 2007 to 2013: Preceptors

2007(n=58) 2013 (n=81) Rank Competency Mean SD Rank Mean SD Rank Change Communication & Rela- A. 4.69 0.56 4.58 0.53 tionship Management Interpersonal Commu- 4.88 0.56 1 4.73 0.61 1 0 nication Presentation Skills 4.64 0.67 6 4.53 0.65 6 0 Writing Skills 4.55 0.68 7 4.47 0.67 9 2 B. Leadership 4.57 0.59 4.45 0.60 Ability for Honest Self- 4.4 0.72 14 4.21 0.86 17 3 Assessment Leading and Managing 4.67 0.69 5 4.47 0.78 8 3 Others Change Management 4.5 0.73 11 4.37 0.83 14 3 Problem-Solving and 4.76 0.63 3 4.63 0.7 3 0 Decision-Making Systems Thinking 4.53 0.73 9 4.58 0.71 5 -4 C. Professionalism 4.56 0.57 4.37 0.60 Personal and Professional 4.86 0.58 2 4.65 0.65 2 0 Ethics Professional & Commu- 4.09 0.78 19 3.88 0.86 24 5 nity Contribution Working in Teams 4.74 0.64 4 4.58 0.65 4 0 D. Knowledge of the Healthcare Environ- 3.74 0.62 4.06 0.59 ment** Health Policy Formula- tion, Implementation, 3.55 0.78 24 4.01 0.68 19 -5 and Evaluation** Healthcare Issues and 4.47 0.73 13 4.38 0.7 13 0 Trends Legal Principles Develop- ment, Application and 3.6 0.77 23 3.99 0.8 21 -2 Assessment** Population Health and 3.34 0.71 25 3.84 0.89 25 0 Status Assessment** 88 The Journal of Health Administration Education Winter 2016

Table 5, cont.

E. Business Knowledge 4.20 0.53 4.23 0.50 and Skills Economic Analysis and Applications to Business 3.9 0.85 21 4.02 0.84 18 -3 Decisions Financial Management 4.33 0.71 15 4.52 0.69 7 -8 Human Resources 3.91 0.71 20 4 0.69 20 0 Information Manage- ment/Understanding and 4.16 0.74 18 3.95 0.76 22 4 Using Technology Marketing* 3.69 0.65 22 3.94 0.7 23 1 Organizational Dynamics 4.24 0.71 16 4.22 0.76 16 0 and Governance Planning and Managing 4.48 0.71 12 4.32 0.69 15 3 Projects Quality/Performance 4.55 0.71 8 4.47 0.73 10 2 Improvement Quantitative Skills 4.52 0.71 10 4.41 0.67 11 1 Strategic Planning 4.22 0.73 17 4.4 0.72 12 -5 Overall 4.30 0.51 4.29 0.49

*indicates the mean for 2007 is statistically different from the mean in 2013p <.05 ** indicates p < .01

Since alumni make up a large portion of the survey, the results in Table 6, which show alumni responses, are very similar to those in Table 3. Strategic Planning saw the largest increase in absolute and relative score, increasing from 16th to 11th between surveys. Also, while the score decreased slightly between surveys, note that Planning and Managing Projects increased in rank from 6th to 5th, higher than its ranking among other stakeholders. There is also a decline in ranking from 2nd to 7th for Personal and Professional Eth- ics with a significant decrease in score. This likely represents the increase in importance of other competencies more than a decline in the importance of Personal and Professional Ethics. Competency development and validation 89

Table 6 Comparison of Competency Survey Evaluations Between 2007 to 2013: Alumni

2007(n=228) 2013 (n=481) Rank Competency Mean SD Rank Mean SD Rank Change Communication & Rela- A. 4.70 0.33 4.60 0.42 tionship Management** Interpersonal Communi- 4.92 0.27 1 4.78 0.49 1 0 cation** Presentation Skills 4.64 0.52 7 4.58 0.55 9 2 Writing Skills* 4.54 0.6 12 4.44 0.66 15 3 B. Leadership 4.64 0.36 4.59 0.42 Ability for Honest Self- 4.45 0.73 14 4.35 0.73 16 2 Assessment Leading and Managing 4.77 048 4 4.65 0.63 4 0 Others* Change Management 4.62 0.56 8 4.56 0.63 10 2 Problem-Solving and 4.75 0.44 5 4.75 0.49 2 -3 Decision-Making Systems Thinking 4.61 0.58 9 4.65 0.57 6 -3 C. Professionalism** 4.55 0.39 4.40 0.49 Personal and Professional 4.79 0.45 2 4.63 0.63 7 5 Ethics** Professional & Commu- 4.08 0.7 21 3.87 0.83 24 3 nity Contribution** Working in Teams 4.78 0.45 3 4.7 0.58 3 0 D. Knowledge of the Healthcare Environ- 3.96 0.57 4.15 0.60 ment** Health Policy Formula- tion, Implementation, 3.69 0.83 24 4.13 0.73 20 -4 and Evaluation** Healthcare Issues and 4.49 0.56 13 4.48 0.66 13 0 Trends Legal Principles Develop- ment, Application and 3.97 0.79 22 4.17 0.76 19 -3 Assessment** Population Health and 3.69 0.78 25 3.8 0.92 25 0 Status Assessment* 90 The Journal of Health Administration Education Winter 2016

Table 6, cont.

E. Business Knowledge 4.33 0.40 4.33 0.43 and Skills Economic Analysis and Applications to Business 4.24 0.74 17 4.02 0.82 22 5 Decisions** Financial Management 4.57 0.56 11 4.61 0.59 8 -3 Human Resources 4.12 0.71 20 4.06 0.79 21 1 Information Manage- ment/Understanding and 4.16 0.69 19 4.21 0.72 17 -2 Using Technology Marketing 3.89 0.75 23 3.99 0.77 23 0 Organizational Dynamics 4.18 0.69 18 4.19 0.77 18 0 and Governance Planning and Managing 4.75 0.46 6 4.65 0.57 5 -1 Projects* Quality/Performance 4.4 0.65 15 4.45 0.7 14 -1 Improvement Quantitative Skills 4.61 0.55 10 4.52 0.64 12 2 Strategic Planning** 4.39 0.61 16 4.55 0.63 11 -5 Overall 4.40 0.32 4.39 0.38

*indicates the mean for 2007 is statistically different from the mean in 2013p <.05 ** indicates p < .01

Discussion The Collaborative Leadership Model was originally crafted to identify early careerist competencies drawn from the Healthcare Leadership Alliance competency model, which served as a guide for the breadth of knowledge, skills, and attributes needed for healthcare management and administration roles and responsibilities. The HLA model is viewed to be applicable for hospital and nursing administration, medical practice administration, healthcare financial management, and healthcare information management. Programs beyond the original four have begun to use and adapt this model to their respective program missions and designs. This analysis was undertaken to assess that the model remained relevant, adjust it where needed and to ensure its validity and reliability among key stakeholders. Competency development and validation 91

The results of the analysis reveal high consistency over time, and across stakeholders and institutions. There is a strong correlation between the ratings of the 2007 survey and the revised 2013 survey further validating the Collaborative Leadership Model. While all of the competencies were rated as important for students as they launch their careers in health administration, there were some interesting movements in the relative ratings of the individual competencies. First, within the Business Knowledge & Skills domain there were large increases in the relative rankings of both Financial Management and Strategic Planning. The relative increases for Financial Management and Strategic Planning from 2007 to 2013 likely represent increased uncertainty in the industry, placing greater importance on financial acumen as well as strategic thinking. Next, Knowledge of the Healthcare Environment saw an increase in relative rankings. Presumably, the implementation of the Affordable Care Act means that healthcare managers need to have a deeper understanding of the healthcare environment. While the individual competencies within this domain remain ranked relatively low as part of the model, their importance increased substantially for Health Policy Formulation, Implementation and Evaluation, Legal Principles Development, Application and Assessment, as well as Population Health and Status Assessment. It is important to place this shift in the context of the changing healthcare industry. These changes in ranked emphasis are consistent with the movement toward a more regulated market and an emphasis on population health management subsequent to the passage of the Affordable Care Act. Finally, there is an increased emphasis on leadership skills, with specific increases in the rankings of Problem Solv- ing and Decision Making, and Systems Thinking. These leadership traits are especially valued in an increasingly uncertain environment that is moving toward reducing silos and increasing interprofessional integration of care delivery. Our study does have limitations. First, the survey included faculty and alumni responses from only four institutions in a limited geographic area. However, the graduates of these programs and their preceptors are located throughout the country and are not believed to be systematically different from other similar groups at other schools. Second, the survey was offered online, and thus, the response rate was not 100%. We made every effort to be inclusive and send appropriate reminders. Though there are limitations of this four- university study, there is more recent indication of the validity of this model. To guide their educational offerings, a Professional Development Task Force of the American College of Healthcare Executives (ACHE) analyzed the HLA 92 The Journal of Health Administration Education Winter 2016 model and has drafted a Leadership Competency Framework divided into four career stages: early, mid-level, senior, and top leaders. The early career stage core competencies broadly track very closely with the 25 competencies in the collaborative model described here. Given the dynamic field of healthcare, preparing future leaders can be a challenging and moving target. It is essential to consistently validate and update MHA curricula to ensure that graduate students are being well prepared for their careers. Given that various stakeholders have a shared interested in the “products” of our programs, the opportunity for evaluation of competencies and curricula is an important exercise for programs to do on a regular basis.

References Calhoun, J. G., Davidson, P. L., Sinioris, M. E., Vincent, E. T., & Griffith, J. R. (2002). Toward an understanding of competency identification and assessment in healthcare management. Quality Management in Healthcare, 11(1), 14-38.

Calhoun, J. G., Vincent, E. T., Calhoun, G. L., & Brandsen, L. E. (2008). Why competencies in graduate health management and policy education? Journal of Health Administration Education, 25(1), 17-35.

Campbell, C. R., Lomperis, A. M., Gillespie, K. N., & Arrington, B. (2005). Competency-based healthcare management education: The Saint Louis University experience. Journal of Health Administration Education, 23(2), 135-168.

Clement, D. G., Hall, R. S., O’Connor, S. J., Qu, H., Stefl, M. E., & White, A. W. (2010). Competency development and validation: A collaborative approach among four graduate programs. Journal of Health Administration Education, 27(3), 151-173.

Commission on Accreditation of Healthcare Management Education. (2015). Criteria for accreditation. Retrieved from http://cahme.org/Resources/Fall2013_CriteriaForAccreditation.pdf

Garman, A. N., & Johnson, M. P. (2006). Leadership competencies: An introduction. Journal of Healthcare Management /American College of Healthcare Executives, 51(1), 13. Competency development and validation 93

Shewchuck, R. M., O’Connor, S. J., & Fine, D. J. (2006). Bridging the gap: Academic and practitioner perspectives to identify early career competencies needed in healthcare management. Journal of Health Administration Education, 23(4), 366.

Stefl, S. E. (2008). Common competencies for all healthcare managers: The healthcare leadership alliance model. Journal of Healthcare Management, 53(6), 360-373.

Westera, W. (2001). Competencies in education: A confusion of tongues. Journal of Curriculum Studies, 33(1), 75-88.