Intelligent Automated Triage

Using robotic and cognitive automation to improve the triage and referral management pathway in Gastroenterology in NHS Lothian

Ian Arnott, Clinical Lead Gastroenterology, NHS Lothian Peter Lock, Healthcare Director, Deloitte Consulting NHS Lothian: Overview

• NHS Lothian provides a comprehensive range of primary, community-based and acute hospital Scotland services for circa 800,000 people NHS Lothian • It provides services for the City of , , Mid Lothian and areas as well as regional tertiary services Northern • It employs over 28,000 staff Ireland • Edinburgh is one of the fastest growing regions in the UK with significant population growth putting pressure on service provision England Wales NHS Lothian: Gastroenterology

• NHS Lothian receive c16,000 new Gastroenterology referrals per annum • Pilot focused on site in Edinburgh which has six Gastroenterology consultants who triage around 30-40 referrals a day • There are significant waiting list pressures in the specialty, with waiting times for those booked into a routine appointment of up to 52 weeks • Over the past three years the department has seen an increase in Urgent Suspected Cancer referrals • Until now there has been relatively little focus on the triage process per se with infrequent use of downgrade options Triage Process

• Scotland has had an electronic referral and triage system whereby GPs submit a referral using a standard letter template which is triaged by a vetting clinician in secondary care. • We analysed over 25,000 GP referral letters over a four year period. It demonstrated that the current triage process was complex with over 120 outcomes permutations created by combining an urgency classification with a clinic or diagnostic pathway classification. • It also became clear that there was significant variability in how clinical triage decisions were made. A sample of referrals were re-triaged and demonstrated low retest validity.

Triage Retest Validity

Retest Validity Doctor 1 Doctor 2 Doctor 3 Average

Urgency classification 28% (14/50) 52% (26/50) 44% (22/50) 41% (62/150)

Clinic/diagnostic classification 48% (24/50) 70% (35/50) 60% (30/50) 59% (89/150)

Full triage outcome 14% (7/50) 36% (18/50) 30% (15/50) 27% (40/150) Automating the triage process

• We undertook an experiment to AI Classifier 1 (AI-1): Referral Urgency assess whether an AI algorithm using Natural Language Processing could more accurately triage referral letters. USOC Urgent Routine

• The triage process requires the + consultant to make a decision about AI Classifier 2 (AI-2): Referral next step the urgency status of the referral and what the next step of the pathway General GI clinic Colonoscopy should be and this was replicated using a two stage AI model OGD Colonoscopy & OGD

Other category

TRIAGE OUTCOME

5 Automating the triage process

• We have seen promising results, particularly for urgent suspected cancer patients, and we are in the process of refining the models.

AI-1: Urgency AI-2: Triage outcome

62% of USOC patients 90% accurate at were triaged into the same detecting the cases outcome as the triaging historically triaged as USOC clinician

Out of the AI’s total USOC However, 82% of the AI’s predictions - 65% were triage decisions were concordant with the triaging deemed clinically consultant’s decision appropriate

6 Referral Triage Improvement

• The AI algorithm is currently running in ‘shadow mode’ and we are collecting prospective training data comparing AI vs clinician triage decisions • It has been integrated with the local PAS system using robotics and APIs. The ultimate aim will be to implement ‘auto-triage’ starting with a cohort of Urgent Suspected Cancer referrals subject to clinical risk management processes • The full triage improvement programme includes two further components: − Analytics: a real time analytics dashboard illustrating GP referral patterns − Assist: an on-screen attended robotic solution that allows the vetting consultant to more easily downgrade GP referrals and also presents back the AI decision in real time

Outcomes

• Faster triage • More appropriate triage by reducing variation and increasing transparency • Freeing up medical administration time • Reducing waiting lists by actively managing the triage process • Supporting a broader programme of work to transform triage processes • Facilitated joint working across primary, secondary care as well as senior management and technology colleagues