Undergraduate medical education at the Royal Free and UCL :

Implementation of the London Medicine recommendations

th London Medicine - BMA House - 24 April 2018

Matthew Camilleri, Elvin Chang, Nikesh Devani, Samantha Maloney, James Murray, Joseph Newman Clinical Teaching Fellows

Paul Dilworth

UCL Sub-Dean

Introduction In June 2017 London Medicine published a key set of medical education guidelines entitled: “Providing effective undergraduate medical clincial placements in London: Recommendations from the student perspective” At UCL Medical School’s Royal Free London campus, we set out to evaluate how the modules we provide already meet these recommendations and where indicated, we have designed new projects to either fill the defecits or enhance the teaching already in place. We are a diverse team of six Clinical Teaching Fellows from a range of clinical backgrounds under the leadership of Dr Paul Dilworth, UCL sub-dean, and liaison with Prof Deborah Gill, Director of UCL Medical School. We work closely with the administrative team and the wider medical school to design, deliver and evaluate a large number of medical education initiatives. In this document we aim to outline the work that we have carried out over the past year as a case study of how the London Medicine recommendations could be implemented further afield. Whilst the projects are based on local curricula and infrasturcute, we hope that the principles have a wider applicability. This work was presented to the London Medicine forum at BMA House on 24th April 2018.

Royal Free Clinical Teaching Fellows 2017/8

Contents

1. The Perfect Placement Project – mapping implementations to recommendations

2. The Perfect Placement Project – outcome measures for each new project

3. Innovation in medical education: 3.1 Teach Me campaign 3.2 The Patient Journey

4. Documentation skills in medical education: 4.1 Videotaped simulated ward rounds 4.2 Structured clerking feedback

5. Technology in medical education:

5.1 Simulation e-module

5.2 Slack app – “community of practice”

5.3 Electronic feedback system

6. The next steps

Appendix

A) Index of abstracts published or presented 2017/8 B) Structured clerking feedback proforma C) “Teach Me” example materials

1. The Perfect Placement Project – mapping implementations to recommendations The Perfect Placement Project aimed to map the existing teaching practice at the Royal Free against each of the 10 recommendations made by London Medicine. The existing practice was evaluated to ensure that it did comply with the recommendations and changes were made where needed.

In addition to this, a number of new projects were designed and implemented specifically to address each recommendation in a strive towards creating the “perfect placement”.

This is summarised in the table below. Please note that the highlighted* new CTF projects are expanded upon later in this document.

Recommendation Existing practice New CTF projects i. Ensure students receive a 1. Site induction 1. The Patient Journey* positive and appropriate 2. Speciality specific induction at the beginning of induction (e.g. resp, acute each placement med) ii. Provide students with access 1. Module tutors 1. Module Tutor Guide to a named supervisor during 2. GP tutor each placement 3. Personal tutor iii. Ensure both medical schools 1. Personalised outlook 1. The Patient Journey* and placement providers calendars with timetable provide timely and accurate 2. Site and speciality information to students before induction and during placements iv. Provide the students with 1. Site/module 1. Preparation for Practice week sufficient learning support and introduction 2. The use of Apps in MBBS facilities to create a positive 2. Personalised outlook 3. Teach Me* placement experience calendars with timetable 4. Acute sim “flipped classroom” e- 3. Access to medical module* school facilities on site v. Provide students with 1. Local feedback 1. Module tutor guide appropriate mechanisms for mechanisms 2. Network of trained Twilight Tutors raising concerns about all 2. Tutor-ship aspects of their experience 3. Pastoral care during placements vi. Ensure educators have 1. Site lead 1. Teach Me* enough information about communication with 2. Medical Education website intended learning outcomes of educators 3. Curriculum guide the placement, as well as the 4. Doctor’s as educators time, resources and capacity to 5. Training to Teach support students and their 6. Journal Club learning during placements 7. Slack* 8. PGCert / FHEA

vii. Ensure students are 1. Existing teaching 1. The Patient Journey* supported and given structures: ward, lecture 2. Year 4 Preparation for Practice appropriate opportunities to based, bedside teaching 3. ABG training gain practical experience in an 2. Twilight teaching 4. Teach Me* optimum learning environment 3. Module tutors 5. High fidelity simulation 6. Teaching ward rounds 7. Buddy system for on call 8. Year 6 Preparation for Practice 9. Ward round documentation skills* 10. Situational Judgement Test viii. Provide access to a truly 1. Acute simulation 1. Teach Me* multi-professional learning course with nursing 2. Increased sim facilitators from environment, with students across the MDT – critical care nurses, collaboration to support 2. Pharmacology tutorial physiotherapists learning between professions with pharmacy students ix. Ensure robust feedback 1. Electronic placement 1. Teach Me* systems are in place for both feedback 2. ‘You said, we did’ – ABG teaching students and educators 2. Face-to-face feedback feedback with tutors 3. Instant anonymised electronic 3. Feedback following Twilight feedback system* lectures delivered during 4. Structured clerking feedback* PfP x. Ensure robust systems are in 1. Obtaining student 1. Teach Me* place to measure a placement feedback as above 2. Doctor’s as educators crib sheet provider’s performance and to 2. rewarding / 3. Teaching awards provide feedback on their acknowledging 4. The Patient Experience performance against defined individuals or standards, particularly departments providing rewarding good performance high quality experience for the students

2. The Perfect Placement Project – outcome measures for each new project

Each of the new medical education initiatives was evaluated with a range of outcome measures including student feedback, peer review processes and presentations at national medical education conferences.

The below table summarises this.

Project Outcome  Positive feedback from students, patients and staff involved  Changes and improvements to rotations based on feedback  Improvement in Official student feedback – increase in % satisfaction as well as Teach Me notable change in culture form free text comments  Feedback from Trust Executive with consideration for official trust wide role out.  Accepted for poster presentation at RCP Conference, June 2018  Excellent feedback from students involved  Feedback from colleagues – discussed at RFH forum and piloted. Discussed with Patient Journey Head of Year 4 and for cross site roll out  Accepted for poster presentation at ASME conference, July 2018  Positive feedback from students and staff involved Teaching ward rounds  Disseminated to other specialities across module A  Accepted for oral presentation at ASME conference, July 2018  Preliminary evaluation of student feedback is very positive  Specifically mentioned in Year 4 Module A Block 2, 2017-18 Student Evaluation Structured feedback on Questionnaire (SEQ) report student clerking  Planned roll-out throughout Module A and discussed with head of year 4 for consideration of cross-site roll out.  Overhaul of course structure based on 2016/7 feedback  Positive feedback from students tracked over time, clinical skills tutors and module lead Arterial Blood Gas  Increased numbers and success rates of ABGs in practice  Increased awareness of and adherence to BTS guidelines re local anaesthesia  Accepted for poster presentation at ASME conference, July 2018  Positive feedback from students at session and end of module  Noted innovative session which gives UCL students a 2 year head start and co- aligns with clinical objectives Situational Judgement  Adds to medical education literature re whether candidates score higher if SJT Test attempted individually or with group discussion  Senior medical student involvement in data analysis and publication  Cohort’s results can be followed over next 2 years to see effect on national level

 Positive feedback from students involved  Specifically mentioned in Year 4 Module A Block 2, 2017-18 Student Evaluation Video-assisted Questionnaire (SEQ) report documentation  Planned inclusion into the Introductory Orientation Module (IOM) next year  Accepted for oral presentation at ASME conference, July 2018  >100 junior doctors formally trained on highly acclaimed UCL course  Doctors reasons for attending educational training captured Training to Teach  Course commissioned by the Royal Free showing investment in education  Alongside medical education team undertaking PGCerts, FHEA etc.  Overall ensures students taught by trained educators  Over 50 junior doctors trained to deliver regular bedside teaching  Excellent feedback from students via innovative instant-capture electronic Twilight Tutoring system which automatically collates feedback  Ensures continuity of education and a system to provide pastoral support  Community of practice via Slack app to share ideas and resources  Consistently positive feedback from students  Continual improvements to course made based on feedback  Objectives met of continuity of care between primary and secondary care and Preparation for Practice practical elements of junior doctor work Year 4  Opportunity to showcase technology (apps, googledocs) and new teaching sessions (video ward rounds, SJT)  Peer-reviewed by GP team and medical education colleagues  Four bespoke days built into year six final placement March/April 2018  Structure based on previous feedback and focus groups of FY doctors/current students Preparation for Practice  Activities including high fidelity simulation scenarios and on-call experience, Year 6 communication skills, clinical skills and practical prescribing  Initial feedback extremely positive and acted to make adjustments next placement June 2018  Consistent highly-rated feedback from interprofessional group of candidates, including medical students, nurses, and allied health professionals  Specifically mentioned in Year 4 Module A Block 2, 2017-18 Student Evaluation Hi-fidelity simulation Questionnaire (SEQ) report as a highlight of the acute medicine rotation and e-module  Presented as a poster at UCLP Education conference December 2017  Piloting new interactive e-module as part of ‘flipped classroom’ initiative; initial feedback from learners very positive  Project lead by UCL medical students with senior support  Questionnaire answered by 77 inpatients at the 2017  Almost all patients felt that interaction with medical students was important  A significant proportion feel that their clinical care and time in hospital are The Patient Experience positively influenced by the presence of students  Results will be fed back to UCL students from September to encourage clinical interaction with patients  Accepted as an ePoster presentation at the RCP Conference 2018

3. Innovation in medical education Two examples of innovation in medical education are the “Teach Me” campaign and the “The Patient Journey” programme.

3.1 Teach Me Campaign The Teach Me Campaign is a series of weeklong events run in a particular area or department. The campaign aims to promote an organisational culture that values teaching and developing others as a key lever to meeting the Trust’s mission of excellence in education, training and development. Additionally it aimed to highlight issues with placements and potential steps to improve the teaching and learning experience for all.

The objective is to support all clinicians and students as learners, while encouraging inclusion, enthusiasm and involvement in learning activities. This was achieved by a multimedia promotion campaign, as well as interactive sessions with teachers and learners to demonstrate what support is available, what they should aim to achieve during the week and generally getting to know each other. Materials with, curriculum information, tips on clinical education in the workplace and important organisational and contact information were provided. Students were encouraged to wear special student badges, in order to be more easily identified. Additionally we encouraged cross-disciplinary sharing of education resources and training that may align with inter-professional learning objectives. Feedback from the various campaigns demonstrated various positive outcomes.

Staff “It’s good to Know who the students are” Patients “It’s a great idea, it’s very reassuring” Students “The best teaching experience I have had on a ward so far. Staff were encouraging and helpful” “The teaching ward round made us feel included” “Badges made us more visible to staff”

Outcomes of Teach Me Campaigns have included placement timetable overhauls, changes to the structure and organisation of placements, and increased training for staff. These were made possible by bringing medical education under the microscope in each placement, highlighting issues and creating enthusiasm around improvement. The campaign has the potential to be exportable to any area, applied to any speciality or discipline and applicable to any level of undergraduate or postgraduate education. For some of the materials used in the campaign see Appendix B.

3.2 The Patient Journey The Patient Journey is a novel teaching session designed to allow small groups of first year clinical students to have a comprehensive introduction, induction and familiarisation with the environment, processes and expectations of medical students in an acute hospital.

A facilitator will take students on a physical tour of the hospital using a theoretical patient vignette and in each area have an informal conversation around the various aspects of the learning environment and demonstration on the workings of the hospital environment.

Feedback for this session piloted on one module at the Royal Free Hospital was resoundingly positive with the majority of students strongly agreeing that the session improved the following: • Increased knowledge of the multidisciplinary teams • Increased confidence in clerking on a ward • Increased confidence in clerking in A&E • Increase in understanding of the admissions process • Increase in understanding of the discharge process • Increase in understanding of the patient journey through an acute hospital

Additionally some of the free text comment demonstrated that students found this session extremely useful: • “(Should) have in every hospital at beginning of placement” • “(I) wish I had this in September” • “It’s perfect, change nothing”

Moving forward it has been agreed for this session to be provided for all students at the beginning of each module across the UCL campuses.

4. Documentation skills in medical education An area that seemed to be under-represented in the existing teaching practice was that of clinical documentation skills. Two projects were initiated to address this. Firstly, medical students were taught best practice of documenting entries in the medical notes through the use of videotaped simulated ward rounds. Secondly, they were encouraged to document thorough and structured medical clerkings which were objectively appraised by placement tutors using a structured feedback proforma.

4.1 Videotaped simulated ward rounds During their structured teaching week, students were provided with guidance on record keeping and effective clinical documentation. They were then tasked with observing two simulated ward-round encounters and asked to document ward-round entries in real-time. At the conclusion of this session, the students engaged in a collaborative discussion regarding their documented entries and received feedback. 90 80 Post 70 Pre 60 50 40 30

20 10 0 1 2 3 4 5

Screenshot of simulated ward round Self-reported confidence for ‘real’ WR documentation before and after session. (Likert Scale where 1 = not confident at all, 5 = very confident)

Surveying the students after the session revealed that almost half of the students (n=118) had not documented in medical notes before. Overall, the session was highly valued and considered relevant to their training needs. Furthermore, assessment of their self-reported confidence scores for future ward-round documentation revealed a significant increase following this session.

In summary, we have developed a well-received intervention which addresses an area currently under-represented in the curriculum. The use of videotaped simulated ward rounds is low-cost and reproducible; teaching can be scaled up and delivered to large cohorts of learners.

4.2 Structured clerking feedback During their attachment to Acute Medicine, students were required to submit at least one full medical clerking which was objectively appraised using a structured clerking feedback proforma (Appendix A). The aims of this were to:

a) Encourage undertaking of a ‘complete clerking’

b) Promote the development of clinical reasoning skills

c) Consolidate and further students’ understanding of good clinical documentation

Feedback was obtained before and after implementation of the proforma. This revealed that only 46% of students had previously received any feedback on their clerkings; this feedback was mostly verbal and was of moderate utility. The comparisons with the feedback following receipt of the structured appraisal have been demonstrated in the tables below:

Likert scale values Ranging from 1 -10, with 1 representing least effect e.g. not useful, not influential; and 10 representing greatest effect Previously received feedback Structured feedback proforma Usefulness of feedback 6 15

Mean = 9.1

Mean = 6.8 4 10

Frequency 5 Frequency 2

0 0 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

Likelihood feedback will influence approach to future clerkings 4 15

3 Mean = 6.9 10 mean = 9.0 2

Frequency 1 5

0 Frequency 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

Likert scale values Ranging from 1 -10, with 1 representing least effect e.g. not useful, not influential; and 10 representing greatest effect Previously received feedback Structured feedback proforma Extent to which feedback reinforces message of complete clerking 4 15 mean = 5.9 3 mean = 9.5 10 2

Frequency 1 5

0 Frequency 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

In summary, students rated the utility of structured written clerking feedback highly with an increased likelihood that it would positively impact on future clerkings undertaken. The structured clerking is a simple, effective tool which promotes completion of a full clerking and clinical reasoning. It also addresses the need to continually provide our students with feedback on their performance. We believe that it is a model which is easily exportable across medical schools.

5. Technology in medical education

We have used free and widely available technology to enhance various elements of our projects. These could be easily adapted for use elsewhere.

5.1 Simulation e-module

Using the concept of the “flipped classroom”, we re-designed the acute medicine hi-fidelity simulation course by taking the knowledge content out of the session and structuring it within a pre-course e-module. For example, the knowledge here is understanding the components of the “ABC” assessment and “SBAR” handover.

The aim of this was to allow the focus of the teaching session to then be on the assimilation and application of knowledge to a clinical scenario.

We filmed video clips with voiceovers and text to produce an interactive and engaging platform.

This allowed the medical and nursing students to work through the knowledge- based learning at their own pace and perform formative self-assessments prior to attending the course.

The initial feedback of the e-module has been very positive with one 4th Year Medical Student commenting: Screenshots from the acute medicine simulation e-module “Very well structured and helpful resource. Simple and easy to navigate, clear & succinct.”

This concept could be applied to many other teaching programmes and is familiar to clinicians who have undertaken eALS or local mandatory and statutory training.

It was created using software from Cardiff University but many other readily accessible platforms exist, such as moodle which is used by UCL.

5.2 Slack app – “community of practice”

Doctors who teach often do so informally and in isolation. We were keen to create a “community of practice”, bringing together clinicians of various specialties and grades but who all share an interest in medical education.

To do this, we used Slack, a free communication app which allows users within a group to follow particular themes that interest them, such as simulation or bedside teaching. They can then share ideas and resources.

Slack can be used on smartphones or desktop for ease and instant access. It is often used in the business or research settings, but we have found it very effective within the medical education field. For example, we used it to draw together all of the junior doctors who regularly deliver bedside teaching to troubleshoot problems and share teaching materials. This has the secondary benefit of keeping them engaged and enthused as educators. We use Slack to offer incentives such as advertising relevant conferences, giving them priority on UCL’s Training to Teach course and distribute certificates and feedback.

The app’s metrics can be tracked to gauge the level of engagement of those within the group over time. Again, this is freely-available, easy to use and transferable to other institutions.

Screenshot from the #twilight (near-peer bedside teaching) channel within the “RFL-education” group with an example sharing the London Medicine recommendations document with over fifty junior doctors with an interest in medical education. Please also note the pinned item on the right-hand side of the screen which links to the feedback system, see next section 5.3.

5.3 Electronic feedback system

Feedback from medical students regarding individual sessions or tutors can be difficult to capture without causing “feedback fatigue” or poor return rates. We devised a system in which students provide their near-peer junior doctor tutors with brief, structured and instant anonymised feedback via an online system illustrated below. Link to the googleforms feedback as pinned on the Slack group for ease of access

Firstly, the tutors copy a web link, for example from Slack (see previous section 5.2) and send this round to their students via email, text etc at the end of each session.

Then the students click on this link which brings them to a googleforms survey which they are invited to complete on their smartphones prior to leaving the session. (We have found from experience that once students have left they are far less likely to complete this feedback form.)

The survey questions are easy to compile in this very user-friendly system. We opted for a simple survey which captures the overall theme of the teaching session, the date it occurred, the name of the tutor and feedback on the teaching. The students complete this entirely anonymously. It takes only a couple of minutes to complete and the students report finding it easy to use leading to high return rates. Example googleforms feedback form

This generates both quantitative and qualitative data which automatically files into a secure master database which can be monitored by the education faculty.

Feedback can then be filtered by topic, date or tutor for example, depending on what information is required.

Secure master spreadsheet which automatically collates the student feedback response

At the end of every term, the data from the spreadsheet is viewed for each tutor and then mail- merged into two documents to present to them. Firstly, they are given a highly-personalised summary record (right) of all of their collated feedback both in terms of Likert scales and comments. This helps them to develop as teachers proves to be a useful document for portfolio purposes for junior doctors. Secondly, they are awarded a formal certificate (below) in recognition of their teaching commitments. As an incentive to commit the scheme, we award bronze for one term’s teaching, silver for two terms and gold for the whole academic year. Furthermore, a “Tutor Champion” award is given to the tutor with the best overall feedback, which this system makes easy to identify.

Example end of term named bronze teaching certificate Example end of term personalised collated feedback

This is an advantageous system for three different groups. Firstly, the medical students can provide anonymised feedback which can be rapidly acted upon week to week, enhancing the teaching they receive. Secondly, the junior doctors are able to develop their teaching skills and receive robust evidence for their portfolios in the form of collated feedback and evidence-based certificates. Lastly, the education faculty can monitor how often the sessions are happening, how many students are attending each session and the overall quality of the teaching provided, so that action towards the students and tutors can be taken, as indicated.

At the time of writing, the new GDPR is coming in to place which may limit the use of googleforms for this use unless individual tutor consent is obtained and the data held securely. Other options we are exploring include UCL’s moodle platform which would provide a very similar system.

The concept has been proven to be highly effective with good student return rates, constructive comments which are then put into practice and mutual benefits to both student and tutor.

6. The next steps…

All of these projects have been mapped to the London Medicine recommendations for clinical placements, in addition to re-evaluating and quality improving existing teaching programmes.

Having embedded these projects within the medical education practice at the Royal Free London, we are in the process of rolling these out to the other UCL Medical School hospitals, particularly:

 The Patient Journey  Video Documentation  Patient Experience  Module Tutor Guide

We are also keen to share these projects across London more widely and are would like to hear of other innovations in neighbouring medical schools, in an exchange of ideas.

“Easy Wins”, which have a wide applicability and high feasibility, include:

 Electronic feedback systems  Structured response to student feedback (e.g. the ABG programme re-design)  Patient Experience  Clerking proforma guide  Patient Journey  Video Documentation

“Tough Nuts”, which can be logistically harder to implement or more site-specific, but have proven to be highly effective, include:

 Module Tutor Guide  Clerking Proforma Guide  6th Yr Prep for Practice  Teach me  Teaching Ward Round

Please do get in touch regarding any of these projects and our approach to implementing the London Medicine guidelines via:

[email protected] – Clinical Teaching Fellows  [email protected] – UCL Sub-Dean, Royal Free Hospital

Appendix A) Index of accepted abstracts 2017/8

A number of the new initiatives this year have been accepted as abstracts at a number of national medical education conferences as posters, eposters, worskshops or oral presentations.

UCLPartners Education Conference – December 2017

1) Technology taking up the 'Slack' in medical education communities of practice POSTER  Shah N, Newman J, Camilleri M, Chang E, Smith P, Murch N, Kwong E

2) Inter-Professional Simulation – Which profession is it helping? POSTER  Dr M. Camilleri, Dr J. Newman, Dr E. Chang, Dr N. Shah, Dr N. Murch, Dr P. Smith, Mr E. Ng

TASME Conference – April 2018

3) Getting Started in Simulation LEADING WORKSHOP  Elvin Chang, Joseph Newman

4) In Situ Simulation LEADING WORKSHOP  Matt Camilleri

ASME Conference – July 2018

5) The Patient's Journey, a Student's Path POSTER  M. Camilleri, E. Chang, J. Newman, J.P. Dilworth

6) You said, we did: Responding to medical student evaluation feedback to enhance arterial blood gas teaching. POSTER  J.Newman, M Camilleri, E Chang, JP Dilworth, J Andrews

7) Learning clinical documentation with simulated patient encounters ORAL PRESENTATION  E.Chang, M.Camilleri, J.Newman, J.P.Dilworth

8) Maximising the learning opportunities for undergraduate medical students from medical ward rounds ORAL PRESENTATION  N. Devani, J. Taube, R. Hall, R. Gell, M. Mackenzie, P. Dilworth

RCP Conference – June 2018

9) The Teach Me Campaign - a different kind of education initiative EPOSTER  Matthew Camilleri*, Elvin Chang, Joseph Newman, Paul Dilworth

10) Medical students on the hospital ward: what do patients really think? EPOSTER  R. Jeyaraj, A. Kunnumpurath, E. Soh, R. Trotman, A. Huckstep, H. Sran, E. Chang, J. Newman, M. Camilleri, J.P. Dilworth, B. Fernando

Appendix B) “Teach Me” example materials

These are a selection of the posters used in the Teach Me promotional campaign. These include (clockwise from top left) posters, badges identifying roles, a large free-standing banner for the ward entrance and patient leaflets.

Appendix C) Structured clerking feedback proforma

This is a copy of the Acute Medicine Structured Clerking Feedback form for tutors to provide written feedback to their students on full medical clerkings.