Section 4 Institutional Responsibilities

SECTION 4 – INSTITUTIONAL RESPONSIBILITIES

POLICY NO: 4.4: Annual Program Evaluation - Institutional Policy ------I. Purpose The educational effectiveness of a program must be evaluated at least annually in a systematic manner as required by the Section V.C. of ACGME Common Program Requirements effective July 1, 2013. This policy provides guidelines as to how the Annual Program Evaluation (APE) is to be completed, reviewed and presented to the GMEC.

II. Scope Applies to all sponsored graduate medical education programs.

III. Program Evaluation Committee (PEC): A. PEC formed by the program director is responsible for  Planning, developing, implementing, and evaluating educational activities of the program,  Reviewing and making recommendations for revision of competency-based curriculum goals and objectives;  Addressing areas of non-compliance with ACGME standards; and,  Reviewing the program annually using evaluations of faculty, residents, and others B. The committee membership must include at least two faculty members and one resident representative. C. The leader of the committee will present a report to the GMEC.

IV. Responsibilities/Requirements: A. The program director is responsible for an APE to be completed and submitted in a timely manner. B. The APE report must be discussed and approved by the core faculty members. C. The chair of the department is required to be involved in the APE. D. The GMEC is responsible for reviewing the reports as submitted.

V. Procedure for APE i. The APE is to be completed by the program evaluation committee, functioning in accordance with ACGME requirements. The APE must be completed even if there are no residents currently enrolled in the program. ii. The following items are to be reviewed as part of the APE: 1. Action items from previous APE, including the status of the item (closed, ongoing) 2. Board Pass Rate for past three years a. Include first time pass rate b. Overall pass rate 3. Resident Duty Hours (review compliance, note rate of violations) and Fatigue management 4. ACGME Resident & Faculty Surveys- review with trend analysis 5. Scholarly Activity for core faculty and residents- table from webADS (as applicable) should be included with report. Any other scholarly activity that PEC feels is appropriate should be included 6. Quality/Performance Improvement- list resident, PI project/committee, note whether planned, ongoing or completed and mentor. 1 7. Curriculum Analysis a. Verify that curriculum has goals and objectives specified by level of training b. Summary of resident and faculty evaluation of the curriculum c. Recommended changes to the curriculum d. Recommended educational activities for the upcoming year 8. Faculty Development Activities (Institutional & Departmental) 9. Attrition in residency program or core faculty, including analysis of impact, if any 10. Resident Milestone Data- review resident performance in comparison to national data if available 11. Program Letters of Agreement- reviewed and current 12. Departmental House Staff Policies- reviewed and current 13. Procedure and Case Logs- review procedure and case logs of most recent graduating class to ensure meeting ACGME requirements 14. Participating Sites- program director has met with site director within past year; note any comments 15. Identified Areas of non-compliance with ACGME standards- corrective plan should be listed as part of action plan 16. Residency Size- list educational rationale (including workforce analysis), educational and financial resources (in place, planned, needed) and timeline 17. Program Director Concerns

iii. The report will conclude with an action plan which includes the item, proposed action, leader and target date of completion. iv. The report is to list the members of the PEC that participated in the review, the date of the review and the date it was presented to the faculty and the GMEC. v. The Program will be responsible to note: 1. That the review is complete and meets all requirements. 2. The strengths and opportunities for improvement. 3. Overall recommendation for the program including action items and needed resources.

Attachment: Annual Program Evaluation Report Presented & Approved by GMEC: Oct 2013

2 Annual Program Evaluation Report Per ACGME Common Program Requirement (V.C.1.) effective July 1, 2013

Residency Program: ______PART I - SUMMARY Program Information: Program Director Program Coordinator Total number of trainees approved vs enrolled Approved Enrolled Date of last internal review (if applicable) Date of next accreditation site visit Date of Annual Program Evaluation Date action plan presented to faculty and reviewed, approved and documented in minutes Date presented/reviewed by GMEC

Annual Program Evaluation (APE) Information: Did the PEC review and document the following at the meeting? 1 Status of last year’s action items (closed, ongoing) Yes No 2 Board Pass Rate (First attempt and overall) Yes No 3 Resident Duty Hours / Fatigue Management Yes No 4 ACGME Resident & Faculty Surveys Yes No 5 Scholarly Activity: Resident & Faculty Yes No 6 Quality Improvement Yes No 7 Curriculum (Goals & Objectives, Evals) Analysis Yes No 8 Faculty Development Activities Yes No 9 Attrition (Resident & Faculty) Yes No 10 Analysis of Resident Milestone Performance Yes No 11 Program Letters of Agreement Yes No 12 Residency Program Policies Yes No 13 Procedure/Case Logs Yes No 14 Participating Sites Yes No 15 Identified Areas of Non-compliance with required standards Yes No 16 Change in Residency size (expansion/decrease/closure) Yes No 17 Program Director Concerns Yes No

Program Evaluation Committee (PEC) Members Name Title

3 PART II - ITEMS REVIEWED

1. Status of last year’s action items (closed, ongoing)

Action Item Status (Closed/Ongoing) Select one Select one Select one

2. Board Pass Rate for past 3 years (as available):

Academic Year Graduates (#) 1st time takers (#) Passed (#) Pass Rate (%) July 20 -June20

July 20 -June20

Overall

3. Resident Duty Hours/Supervision and Fatigue Management: (use internal data as well as ACGME survey data if available)

 How does the department monitor work hours? Does program meet all duty hour standards?

 How does the department monitor supervision?

 How does the program educate residents about Fatigue Management?

4. ACGME Survey: Resident and Faculty

 Have you discussed the findings with your residents and faculty?

4  Please discuss briefly trends if any for the Resident and Faculty Surveys. Also include a corrective plan for concerning trend/s.

 What areas of non-compliance were documented, and what is the correction plan?

5. Scholarly Activity:

 Please indicate scholarly activities during the last academic year (July to June):

Scholarship activity Residents Faculty

 Please attach the table from ACGME web-ADS (if available)

6. Quality Improvement

Resident Project/Committee Status Mentor planned, ongoing, completed Select one

7. Curriculum Analysis

 Does the Curriculum has Competency based goals and objectives (G&O) specified by level of training and by rotation?

 Are G&O distributed to all trainees and faculty at beginning of the academic year?

5  Are G&O reviewed with trainees at beginning of each rotation? Are residents evaluated for all required ACGME competencies on each rotation?

 Summary of resident/faculty evaluation of curriculum & Recommended changes, if any.

 Describe with one example how program used the aggregated results of residents’ competency evaluation to improve the program.

 How does the program document educational effectiveness?

 Recommended Educational Activities for upcoming academic year (Identify activity leader or suggested leader as possible:

8. Faculty Development Activities

Activity Target Audience Date (core faculty, departmental/divisional faculty) Select one

9. Attrition (Resident and Faculty)

Yes No

Impact:

10. Analysis of Resident Milestone Performance (as compared to national data if available)

6 11. Program Letters of Agreement

 Current and reviewed: Yes No

 Proposed changes:

12. Residency Program Policies

 Current and reviewed: Yes No

13. Procedure/Case Logs

 Most recent graduates meet procedure/case log requirements: Yes No N/A

If no, include corrective plan as action item

14. Participating Sites:

 PD has met with site director in past year: Yes No

15. Identified Areas of Non-compliance with required standards and Plan of correction (include as action item).

16. Change in Residency size (expansion/decrease/closure)

• Educational rationale (include workforce analysis):

• Educational resources (in place, planned, needed):

• Financial resources (in place, planned, needed):

• Timeline

7 17. Program Director Concerns (Please include the concerns as part of action plan)

PART III – ACTION PLAN

Summary of program strengths and opportunities for improvement Strengths:

Opportunities:

Action Plan (Also include items is carryover from previous APE)

Item Proposed Action Leader Target Date

Submitted:

______Program Director Date

______Chair of the PEC Date (if different from the Program Director)

Attachments:

1. Last ACGME accreditation letter

2. Competency based goals and objectives specified by level of training and by rotation

3. Program Policy: Resident Work Hours & Fatigue Management

4. Program Policy: Resident Supervision

8 5. List of Residents on Committees (Departmental & Institutional)

6. ACGME Survey Reports (Resident and Faculty)

7. Scholarly Activity: ACGME web-ADS table (if available)

Note: Program can include other attachments as it sees appropriate.