Purpose Program review ensures the effectiveness and currency of established degrees. The review process helps strengthen the quality of UVU’s degree programs and ensure that they are cost- effective and address regional work force needs. Program review and possible redesign of degrees is essential in maintaining vibrant and responsive academic programs. According to USHE policy 411, all programs are reviewed every 7 years. See http://higheredutah.org/wp- content/uploads/2013/08/R411.pdf. Process
Report Preparation General Guidelines Program reviews are submitted in Insight – go to the planning module and then the program review tab. Criterion 1, 8, and 9 should be addressed at the department level and reflect all degree programs – bachelor, associate, minors, emphases, etc. Criteria 4-7 should be completed for each degree program within the department. Write a succinct response for each of the criterion. Responses should not be more than 250-300 words. Use the guiding questions in the i-buttons for each criterion to direct your response. Be sure to respond specifically to the questions. If there is a reason you are unable to respond to an area within a criterion, please explain why. Program reviews cover a time period of 5 years. Note the rubric by which each criterion will be evaluated. Specific data sources are indicated for some criterion. In other cases, a variety of choices for supporting evidence exist. These data sources are listed by criterion at http://www.uvu.edu/iri/academicprograms/program_review.html Additional sources can be used to support responses as appropriate. IRI is available to help with additional data. In cases in which data tables are used, the written response should summarize and analyze the data. Avoid making claims for which no evidence is provided. Keep in mind that peers outside of your department will be reviewing this report. As such, be clear and specific, and remember that you are communicating to those outside of your field. Associate deans and Academic Effectiveness Committee members are available for training and guidance.
2 | P a g e Checklist The checklist below provides specifics for each of the criterion.
General: Insight – include Criterion 1, 8, 9 at the department level; complete fields for criterion 4- 7 for each degree program. Responses generally concise Responses summarize and analyze data Data referenced with links (tables not included in the text of the report)
Criterion 1: History, Development, and Expectations Reference to program history, original intent; no numerical data included unless briefly to demonstrate program growth or development Reference to adaptation to change, evolution of program, anticipation of change/future
Criterion 2 – External Demand 2-3 external data sources used/referenced – highlight main/important points Summary/analysis of evidence - what does it mean? For example – trends for the degree, not just the subskills of the degree
Criterion 3 – Internal Demand Reference to connection of program to other degrees Summary/analysis of these connections in terms of service to other areas in the institution
Criterion 4: Degree Profile Description of department dashboard statistics referred to/relevant points summarized o Student credit hours and FTE o Number of majors by class level o Number of awards granted o Faculty headcount and composition o Faculty teaching load (FTE, SCH, ICHE, sections by full-time, overload/adjunct) o Graduation rates Analysis/interpretation of what this means – course and program/degree strengths and weaknesses, etc.
Criterion 5: Quality of Degree Program Outcomes Reference to student learning outcomes for department, summarize main points Reference student placement/alumni data Analyze the above per findings and resulting actions (acknowledge significance of findings based on number of respondents, etc.) Reference faculty output/productivity (% of faculty publishing/presenting, etc.), summarize highlights If areas requiring data are missing or the department is unable to provide, a statement of how this is in process/plan to get should be given
3 | P a g e Criterion 6: External Revenue and Resources Generated Reference to highlights of grants, fundraising, etc., include totals, external relationships providing benefits Analyze these resources, describing significance to department/school /institution, and resulting actions/plans
Criterion 7: Costs and Other Expenses Reference to finance dashboard populated by the Budget Office Summarize/analyze the data given (comment on efficiencies, investments provided and needed, how the program is supported in terms of the resources – students, degrees, labs, faculty, etc.); Refer to DFTE data.
Criterion 8: Internal Impact, Justification & Overall Essentiality Summation of essentiality of program – impact, benefits, connection to institutional mission, relation to internal factors Summation of how the degree program should continue, be strengthened/altered
Criterion 9: External Opportunity Analysis Analyze opportunities for improvement, strengthening, change, etc. – refer to how these could be capitalized on/benefitted from, factors that affect the program, etc.
4 | P a g e Criterion Guidelines The following guidelines are available in the i-buttons in Insight and are included here for your reference.
Criterion 1 – History, Development, and Expectations (Department Level) Guiding Questions: What was the original intent of the department’s degree programs? How have the degree programs evolved over the years? How have they adapted to meet change?
Criterion 2 – External Demand Guiding Questions: What external indicators show the need for and attractiveness of the degree program? Consider national and local statistics and trends over time. Consider employer demand for broad educational outcomes.
Criterion 3 – Internal Demand Guiding Questions: What is the relationship of the degree program to other degrees or curriculum? How does the degree program serve other degrees or institutional needs? How would alterations in the degree program affect other degrees?
Criterion 4 – Degree Profile Guiding Questions: What are the strengths, weaknesses, efficiencies, and needs of the degree program? How effective is the program in graduating students? What issues might need to be addressed in terms of the viability, health, or size of the programs? What trends are indicated by the data?
Criterion 5 - Quality of Degree Program Outcomes Guiding Questions: What evidence exists for congruence between intended and actual student learning outcomes? How does placement data provide evidence for program effectiveness? How does faculty research/creative work contribute to the program?
Criterion 6: External Revenue and Other Resources Generated Guiding Questions: What external sources of data (e.g., research grants, fundraising, equipment grants, etc.) does the program have? Indicate the source, amount, and timeframe for the funding as well as total amounts. What potential revenue sources exist? What external relationships or joint partnerships with other educational institutions, corporations, businesses, and governments exist of might be nurtured?
Criterion 7: Costs and Other Expenses Guiding Questions: What demonstrable efficiencies are associated with the degree program? What investments are needed to bring the degree programs to a higher level of quality?
Criterion 8: Internal Impact, Justification, and Overall Essentiality (Department Level) Guiding Questions: What impact have the degree programs had or are likely to have? What are the benefits to the institution? How do the degree programs help achieve the institutional mission? How are these degrees related to the success of other degrees? This is a summative measure of why the degree programs should continue or be strengthened.
5 | P a g e Criterion 9: External Opportunity Analysis (Department Level) Guiding Questions: What opportunities exist for improvement and strengthening the degree programs (cf. curriculum, student achievement, competitiveness, innovation)? How might these be capitalized upon? What external environmental factors affect the degree programs? Would a change in degree program formats be beneficial?
6 | P a g e Rubric
Program Prioritization Scoring Rubric
Adapted from Robert C. Dickeson
The purpose of the scoring rubric is to ensure inter-rater reliability. AEC members should rate degree programs based on the information provided. The scores 1, 3, and 9 are intended to force differentiation among program review results.
Criterion 1 - Minimal/Limited 3 - Moderate 9 - Exceptional/Significant History, Development, The program meets the The program meets the The program meets the and Expectations original expectations of original expectations original expectations of the the of the University, and University, has demonstrated University has demonstrated the the ability to adapt to the ability to adapt to the changing needs of the changing needs of the University and its internal and University and its external stakeholders, and internal and demonstrates exceptional external stakeholders ability to anticipate change and build for the future External Demand Demand for the program Demand for the Demand for the program is is limited; trends are flat program is moderate exceptional; it enjoys a or declining, which raises positive trend; it meets a questions about its efficacy variety of external expectations, and is seen as central to the University’s future Internal Demand The program provides The program provides The program provides minimal or no service to moderate service to exceptional service to other other programs other programs programs; such programs could not flourish without the service provided by this program Degree Profile Enrollments and awards Enrollments and Enrollments and awards granted are minimal awards granted are granted are exceptional given compared to dedicated satisfactory compared the dedicated resources resources (faculty, to dedicated program (faculty, sections, class size, sections, class size, etc.) resources (faculty, etc.) sections, class size, etc.) Quality Outcomes Evidence of exemplary Evidence of exemplary Evidence of exemplary student student performance and student performance performance and results of results of student learning and results of student student learning outcomes outcomes assessment and learning outcomes are assessment and related related actions are satisfactory; placement actions are exceptional; limited; placement data is data is acceptable; faculty productivity is weak; faculty productivity faculty productivity is commendable is minimal adequate Revenue Program generates little Program generates Program generates or no revenue on its own moderate revenue on exceptional revenue, sufficient
7 | P a g e [Less than $______] its own to sustain the program without [From $_____ to draining institutional $_____] resources Costs Program costs are not Program costs are Program costs are minimal commensurate with quality appropriate for quality compared to quality outcomes outcomes such as outcomes such as such as enrollments, awards enrollments, awards enrollments, awards granted, student performance, granted, student granted, student etc. performance, etc. performance, etc. Impact There is minimal evidence There is moderate There is an exceptional that the program is evidence relationship between the mission-critical that the program is program/service to the mission-critical University’s mission. The program is integral to the University’s future Opportunity Analysis Projections for the future Projections for the Projections for the future of of future of this this program are unknown this program indicate a program are exciting and hold or tenuous; additional moderate potential for great promise resources may be needed improvement and to maintain this program enhancement of the University’s overall portfolio
8 | P a g e