Consensus-Based Training and Assessment Model for General Surgery
Total Page:16
File Type:pdf, Size:1020Kb
Original article Consensus-based training and assessment model for general surgery P. Szasz1, M. Louridas1, S. de Montbrun1, K. A. Harris2 and T. P. Grantcharov1 1Department of Surgery, University of Toronto, Toronto, and 2Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada Correspondence to: Dr P. Szasz, Department of Surgery, St Michael’s Hospital, 30 Bond Street 16CC-056, Toronto, Ontario, M5B 1 W8, Canada (e-mail: [email protected]) Background: Surgical education is becoming competency-based with the implementation of in-training milestones. Training guidelines should reflect these changes and determine the specific procedures for such milestone assessments. This study aimed to develop a consensus view regarding operative procedures and tasks considered appropriate for junior and senior trainees, and the procedures that can be used as technical milestone assessments for trainee progression in general surgery. Methods: A Delphi process was followed where questionnaires were distributed to all 17 Canadian general surgery programme directors. Items were ranked on a 5-point Likert scale, with consensus defined as Cronbach’s of at least 0⋅70. Items rated 4 or above on the 5-point Likert scale by 80 per cent of the programme directors were included in the models. Results: Two Delphi rounds were completed, with 14 programme directors taking part in round one and 11 in round two. The overall consensus was high (Cronbach’s =0⋅98). The training model included 101 unique procedures and tasks, 24 specific to junior trainees, 68 specific to senior trainees, andnine appropriate to all. The assessment model included four procedures. Conclusion: A system of operative procedures and tasks for junior- and senior-level trainees has been developed along with an assessment model for trainee progression. These can be used as milestones in competency-based assessments. Presented to the Annual Clinical Congress of the American College of Surgeons, Chicago, Illinois, USA, October 2015; published in abstract form as JAmCollSurg2015; 221(Suppl 1): S45 Paper accepted 10 December 2015 Published online 22 March 2016 in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.10103 Introduction account11–16. It is also important to identify procedures that can be used for these technical milestone assessments Surgical education is changing as a result of a number within a competency-based framework. As surgical train- of complex and interdependent factors, including trainee ing evolves, the goal of training programmes should be to working-hour restrictions, decreased operating room time, optimize the available operating time with procedures and increased patient complexity, the growing use of technol- ogy and the recent focus on surgeon volume–outcome tasks that trainees will perform on a regular basis, in order 16–18 data1–7. to increase their competence and confidence .Train- As a result, educational bodies, including the Royal Col- ing should be directed away from focusing on procedures lege of Physicians and Surgeons of Canada (RCPSC), the and tasks that trainees may not perform after the comple- Intercollegiate Surgical Curriculum Programme (ISCP) in tion of training, particularly complex surgical procedures the British Isles, and the Accreditation Council on Gradu- that are nowadays directed to high-volume centres in order 19,20 ate Medical Education (ACGME) based in the USA, are to achieve improved patient outcomes . implementing competency-based education frameworks The first objective of this study was to outline a consensus with a focus on milestones8–10. At this critical juncture, on the procedures and tasks that are appropriate for junior- it seems an appropriate time to re-evaluate current gen- and senior-level trainees in general surgery. The second eral surgery training guidelines and see how to implement objective was to outline a consensus on the procedures that more progressive guidelines that take recent changes into can be used for technical milestone assessments, in order © 2016 BJS Society Ltd BJS 2016; 103: 763–771 Published by John Wiley & Sons Ltd 764 P. Szasz, M. Louridas, S. de Montbrun, K. A. Harris and T. P. Grantcharov to determine whether junior-level trainees can progress to Table 1 Participating programmes and programme director’s senior-level trainees. specialization The Delphi technique was used to achieve consensus Institution Programme director’s specialization among a group of content experts while minimizing inter- University of British Columbia General surgery, MIS, surgical oncology personal interactions, often used in areas where there is University of Alberta General surgery, trauma/acute care 21 26 a lack of evidence – . The major cornerstones of the surgery Delphi technique include: the use of experts, participant University of Calgary General surgery, surgical oncology, anonymity, iterations with interim feedback and statistical trauma/acute care surgery University of Saskatchewan General surgery, CCM 21,24,27 aggregation of results . Although the classical Delphi University of Manitoba* General surgery, trauma/acute care technique has been used for over 50 years, multiple varia- surgery, CCM tions have been developed, each of which has modified one Western University General surgery, colon and rectal 27,28 surgery, trauma/acute care surgery or more of the cornerstones described above .There University of Toronto General surgery, trauma/acute care 29 is agreement, however, as described by Cuhls , that the surgery, CCM Delphi technique is a survey administered to experts in McMaster University* General surgery, two or more iterations, in which after the first round the hepatopancreaticobiliary surgery Queen’s University General surgery, answers (or results) are returned to the experts as a method hepatopancreaticobiliary surgery, MIS of interim feedback. Experts in the context of a Delphi University of Ottawa General surgery, colon and rectal technique are individuals who understand a topic of study surgery 28 University of Montreal* General surgery, surgical oncology more than others in the same field . Programme direc- Sherbrooke University General surgery, MIS tors were chosen as the panellists in this Delphi technique Dalhousie University General surgery, as they are experts in training general surgery residents, hepatopancreaticobiliary surgery, they are up-to-date on changes within their specialty, and transplant, trauma/acute care surgery Memorial University General surgery, MIS are aware of the evolving educational initiatives that have placed an emphasis on achieving specific competencies and *These programmes completed part one of the Delphi technique only. milestones8. Anonymity refers to the participants being MIS, minimally invasive surgery; CCM, critical care medicine. unaware of which experts provided which answers28.The benefits of anonymity include a lack of an overly influential participant and the ability of participants to change their described as the median (i.q.r.) value, for each response , , opinions without the other participants’ knowledge24 27 28. item24,27,28. The survey was administered using the online platform SurveyMonkey™ (Palo Alto, California, USA). All pro- Methods gramme directors were sent an e-mail to their secure The research ethics board at the University of Toronto university-affiliated addresses with instructions and a link approved the study. to participate. Participation was voluntary. The first round closed 6 weeks after the initial contact, and after three e-mail reminders had been distributed. The second round Delphi methodology was returned only to those programme directors who The experts employed in this study were the general responded in the first round. Procedures and tasks sug- surgery programme directors across Canada within an gested to be out of the scope of practice of general surgery accredited RCPSC postgraduate training programme. trainees, by both a lack of first round responses and current Anonymity was maintained as all interactions between guidelines11–13 (such as vascular, thoracic and transplanta- participants took place via an online platform. tion surgery), were not included in the second round. The Two rounds of the Delphi technique were completed second round closed 6 weeks after it was distributed, and to achieve consensus. The content of the first round was after a subsequent three e-mail reminders. generated by the programme directors, then supplemented The first objective of the study was accomplished byask- with a review of the literature and current guidelines11–13. ing the programme directors which procedures and tasks, All responses and associated comments were redistributed characterized by anatomical region categories (upper gas- to the programme directors for ranking in the second trointestinal tract, lower gastrointestinal tract, breast, etc.), round. they considered a junior trainee (end of postgraduate year Responses of all participating experts were combined (PGY) 2) and a senior trainee (end of PGY-5) should be to give an overall opinion of the entire group28.Thiswas competent to perform. Competence in the context of this © 2016 BJS Society Ltd www.bjs.co.uk BJS 2016; 103: 763–771 Published by John Wiley & Sons Ltd Consensus-based training and assessment model for general surgery 765 study was defined for the programme directors as ‘suf- Table 2 Junior-level procedures and tasks included in the final ficiency of qualification’ by the Oxford English Dictionary consensus training model,