Volume 110, Number 3, March 2013

British Journal of Anaesthesia 110 (3): 325–8 (2013) doi:10.1093/bja/aet006

EDITORIAL I Downloaded from https://academic.oup.com/bja/article/110/3/325/250090 by guest on 30 September 2021 National Institute for Clinical Excellence guidance on measuring depth of anaesthesia: limitations of EEG-based technology

J. J. Pandit1* and T. M. Cook2 1 Nuffield Department of Anaesthetics, Oxford University Hospitals, Oxford, UK 2 Royal United Hospital, Bath, UK * E-mail: [email protected]

Accidental awareness during anaesthesia, especially with The available evidence on the impact of the technologies on pain and subsequent of the event, is a terrifying pros- reducing the likelihood of intraoperative awareness is limited. Overall, [EEG-base monitors are] not associated with a statistic- pect. A high proportion of those who experience it are ally significant reduction in intra-operative awareness in reported to go on to develop symptoms similar to post- patients classified as at higher risk ... traumatic stress disorder.1 The reported incidence of recall after , as ascertained by later question- What is the anaesthetist reader (or patient) to make of these ing using the Brice questionnaire, is high, 1–2 per 1000 contemporaneous and conflicting conclusions? cases (although the majority of these do not involve painful The bodies providing these two sources of advice had very experiences, are very brief recollections, or both).2 –8 There- different remits, which may explain the opposing conclu- fore, a monitor to help the anaesthetists identify those sions. The Technology Assessment Report was a review (in- patients who are awake during surgery would be extremely cluding a meta-analysis which incorporated results of a 11 useful. recent Cochrane review). The NICE guidance, on the other The publication of Diagnostic Technologies Guidance from hand, is a consensus opinion which takes into account the the National Institute for Clinical Excellence (NICE) on the findings of the Technology Report but also includes expert role of EEG-based in reducing the risk of aware- and individual views, perhaps importantly including that ness during general anaesthesia may lead patients, politi- from companies which manufacture the devices under con- cians, and planners (and some anaesthetists) to conclude sideration. The broad aim of the NICE Diagnostics guidance that we now have an answer.9 The guidance concludes that: programme is explicitly to promote rapid adoption of clinic- ally innovative and cost-effective diagnostic technologies, The use of EEG-based depth of anaesthesia monitors is recom- based on their potential (not necessarily proven ability)to mended as an option during any type of general anaesthesia in improve care (see: http://www.nice.org.uk/media/A0B/97/ patients considered at higher risk of adverse outcomes. DAPManualFINAL.pdf).

However, that is not quite the full story. The guidance was itself On the ‘probability of unconsciousness’ 10 based on a Technology Assessment Report (commissioned In several places, the NICE guidance appropriately stresses by National Institute of Heath Research Health Technology As- that a monitor’s output reflects the probability of (un)con- sessment programme) which concluded (in contrast to NICE’s sciousness in a given patient. It is repeatedly stated that a final guidance): BIS (and also E-entropy) output reading of 40 (or 40–60)

& The Author [2013]. Published by Oxford University Press on behalf of the British Journal of Anaesthesia. All rights reserved. For Permissions, please email: [email protected] BJA Editorial I indicates a ‘low’ probability of awareness. However, it is not indicated otherwise (i.e. the false negatives, FN) and the specified what ‘low’ means. Also not explained is how the number of cases of appropriate anaesthesia in which the probability of awareness varies as the monitor reading monitor indicated otherwise (the false positives, FP). These increases: 40 may be a low probability, but how much are then combined to yield quantities such as the sensitivity, higher is, say, 50 or 65? In other words, what is the shape specificity, and positive (and negative) predictive values of of the relationship between monitor output and probability, the diagnostic instrument. In the B-Aware trial, results if plotted graphically? were not presented in this way (and in the group of patients We already have answers to these questions, if they are whose anaesthesia was not BIS-guided, the monitor was posed in relation to expired volatile agent (end-tidal) concen- turned off, so no data were available). Nonetheless, the trations. This is the basis of the concept of the MAC data can be re-arranged to show the relevant discriminatory

(minimum alveolar concentration), which is an ED50; the con- capacity of the monitor (Supplementary Table S1). Examined centration at which 50% of patients move in response to in this manner, the monitor’s utility is far from encouraging, a standard stimulus. The ED50 (or EDx; points when x%, with the sensitivity and specificity of 50% and the positive Downloaded from https://academic.oup.com/bja/article/110/3/325/250090 by guest on 30 September 2021 95%, 99% of patients do not move, etc.) has a constant predictive value of just 0.2%. value in arbitrary units, regardless of the agent used and In contrast to the B-Aware trial, the B-Unaware trial (with the concept can be used for a variety of endpoints. For a similar study design) did not find a reduction in awareness example in MAC, the endpoint is response to a standard inci- with BIS monitoring.2 This time, BIS records were available sion, while in MAC-awake, the endpoint is response to verbal for both groups. Again, re-arranging the data to explore diag- stimulus. In this way, MAC (or any EDx) ‘normalizes’ the popu- nostic efficacy of the monitor does not suggest great utility lation response across agents, and indicates the probability (Supplementary Table S2), with the sensitivity 25%, specifi- of no-response at any given dose, regardless of agent.12 – 15 city 45%, and positive predictive value of just 0.1%. A third

Therefore, at 1 MAC (the ED50), there is always the same trial (BAG-Recall) had limited information of this type (it (50%) probability of no-response with halothane, isoflurane, also found no difference between BIS-guided anaesthesia or any other volatile anaesthetic. If a DOA monitor accurately and alternative groups), but the supplementary data reflected this fact, then its output should be the same at 1 appear to suggest the BIS remained ,60 in 12 of 27 patients MAC of any agent. Were the monitor to read, say, 35 at 1 who experienced awareness.3 MAC sevoflurane but 60 at 1 MAC halothane, it would not NICE did not analyse the data in this way. In using the trial be a very useful monitor. Yet this is exactly what is reported data in this manner, we have assumed (as did the trial inves- with EEG-based monitoring.16 Thus, a BIS value of just below tigators) that a BIS value of .60 is always indicative of 60 (i.e. within the recommended target values for both BIS awareness during anaesthesia, regardless of how brief this and E-entropy monitors) might in fact indicate an unaccept- period is. It is in fact not known what exactly the BIS criteria able risk of with some agents.17 NICE recom- are that suggest awareness (nor does NICE guidance help on mends that DOA monitoring should be considered when total this point). Is it the highest BIS in a certain time interval, or i.v. anaesthesia with propofol is used, but the data of Glass the average value, or a certain percentage increase over and colleagues18 clearly demonstrate similarly disparate time, etc.? In other words, should an anaesthetist be con- probabilities at equi-BIS values: at, say, a BIS value of 70, cerned during an anaesthetic if the BIS value is 40 for the probability of unconsciousness is 50% for isoflurane many hours but jumps to 75 for just 10 s? Or much more con- but just 15% for propofol. cerned in a situation where the BIS value is steady at 61 for 30 min? Or are these situations equivalent? We also do not The interpretation of existing clinical know if in these studies the anaesthetist (or anaesthetic agent) acted sufficiently promptly when the monitor indi- trial data cated that the risk of awareness was high.19 Both the NICE guidance and Technology Assessment Report To complicate matters, BIS appears to be ‘blind’ to several use data from large clinical trials to inform their conclusions. anaesthetic agents. is very well established to The B-Aware trial reported two of 1225 cases of awareness add to the effect of other volatile agents. Yet, the BIS value when anaesthesia was guided using a BIS monitor (to is unchanged (indeed, in one study, removing nitrous oxide achieve a BIS reading of 40–60) and 11 of 1238 without a caused BIS paradoxically to decline).20 21 Ketamine is BIS monitor (P¼0.022).5 The attractive conclusions are that popular with some anaesthetists, especially for emergency using the monitor has significantly reduced the occurrence surgery,22 but does not influence (or can even paradoxically of awareness and, furthermore, that maintaining the BIS in increase) BIS values.23 24 this range (i.e. ,60) will prevent consciousness. However, a monitor is not an intervention like a drug, Problems in applying the NICE guidance whose efficacy can be measured as the simple incidence of beneficial outcome. Rather a monitor is a diagnostic tool, into practice whose efficacy is usefully measured as the proportion of Even those enthusiastic about EEG-based monitoring may correct diagnoses it makes. The relevant data are therefore find the NICE guidance vague and difficult to apply. NICE the numbers of cases of awareness in which the monitor states that these monitors are ‘options’ in patients deemed

326 Editorial I BJA to be at ‘high risk’ of awareness during anaesthesia. Anaes- more consistent with, and encouraging of, the scientific thetists have long known that these monitors are options, method might have been: so this is nothing new, but what they really need to know ...the Committee considered that the current uncertainty in is how these monitors compare with other available the evidence base justifies further research into both the scien- options (other long-established options include end-tidal tific basis and clinical utility of these devices so that we can monitoring of anaesthetic vapour, or the isolated forearm better determine whether uptake of technology (that some technique). Indeed, a very recent, large (21 000 patients) believe beneficial) is appropriate. trial has reported no difference in the incidence of accidental While welcoming the interest of NICE in this technology, both awareness during anaesthesia between a BIS-guided anaes- the Royal College of Anaesthetists and the Association of thesia protocol vs one guided by end-tidal monitoring Anaesthetists of Great Britain and Ireland (see: http://www. 25 alone. rcoa.ac.uk/news-and-bulletin/rcoa-news-and-statements/ NICE specifies patients at ‘high risk’ of accidental aware- nice-recommends-depth-of-anaesthesia-monitors) empha- ness during anaesthesia to be: patients with high or sized the overriding need for specific research (including Downloaded from https://academic.oup.com/bja/article/110/3/325/250090 by guest on 30 September 2021 high use, patients with airway problems, patients the result of a current national audit on accidental aware- with previous experience of accidental awareness during ness during anaesthesia)7827before the use of these moni- anaesthesia, and those in whom neuromuscular blocking tors can be meaningfully incorporated into any clinical agents are used. Older patients and those with co- guidelines. morbidities are also judged at higher risk of accidental This is not the first time that NICE guidance will have awareness during anaesthesia because their haemodynamic proved to be controversial among anaesthetists: both the instability often results in a lower dose of anaesthetic admi- guidance on the use of ultrasound in central venous cannu- nistered. Although plausible, it is not known whether these lation and oesophageal Doppler monitoring to guide fluid groups are in fact at high risk, or what that relative magni- therapy have been strongly questioned.28 – 30 Among tude of risk is. It is not clear what age group is to be regarded responses has been the suggestion that failure to heed the as ‘older’. Some readers may be surprised by the omission of guidance should result in referral of the practitioner to the obstetric and cardiac surgical patients from this explicit list General Medical Council.31 There is also a common as these groups are traditionally considered high risk. requirement for Trusts to confirm to commissioners that they are complying with NICE recommendations, or The role of evidence explain why they are not (see: http://www.institute.nhs.uk/ Perhaps the most intellectually troubling statements within commissioning/pct_portal/cquin.html). Clearly, NICE state- the NICE guidance concern the role of evidence in decision- ments are taken very seriously by both readers and making. The scientific method involves rigorously adopting regulators. equipose to a hypothesis and constructing experiments Recent evidence indicates that EEG-based DOA monitors that seek to disprove the initial prejudice. The well- are only used by a very small fraction of anaesthetists, with established emphasis on evidence-based medicine means one-third of hospitals having no such monitors and fewer 78 that practitioners are urged only to adopt those treatments than 2% of anaesthetists using such monitors routinely. or diagnostic tools which are supported by high levels of evi- For the current NICE guidance to be implemented would dence.26 This seems inconsistent with NICE’s statement: require a dramatic change in practice across the UK. The no- ticeable inconsistencies between the conclusions of the ...the Committee considered that the current uncertainty in Technology Assessment and NICE guidance reports may the evidence base does not justify a potentially long delay in the uptake of what is likely to be a beneficial technology. not be sufficiently persuasive to achieve that change. We are therefore left with a situation where current At worst, this statement might be read as an abandonment enthusiasts of EEG-based monitoring will likely use the of the need for evidence. As if to underline this, it is notable NICE guidance to justify their decision to use EEG-based that the evidence for Narcotrend and Entropy monitors is DOA monitoring, while those of the opposite view will use reported by the Technology Report as almost absent, yet the Technology Report to justify not using it. Perhaps most the guidance explicitly gives these monitors an equal importantly, this NICE guidance should not mislead patients, strength of recommendations. If evidence is what really politicians, or planners into the erroneous belief that we cur- matters, then why regard these as equivalent? rently understand the problem of awareness during anaes- Some readers may be surprised at such reasoning from an thesia, or that we have a monitor which reliably tells an organization created to promote the best clinical practice anaesthetist whether the patient is anaesthetized. It is the based on evidence. NICE is an influential organization and probability of unconsciousness that matters and this we do implementation of its guidance is one metric on which NHS not know from any of the available monitors. Trusts are judged in terms of performance. NICE recommen- dations are also likely to influence research funders. NICE’s statement above may risk discouraging the much needed Supplementary material work to answer some of the questions quite correctly Supplementary material is available at British Journal of raised by the Technology Assessment Report. A statement Anaesthesia online.

327 BJA Editorial I

Declaration of interest 13 Eger EI 2nd, Saidman LJ, Brandstater B. Minimum alveolar anes- thetic concentration: a standard of potency. Anesthe- J.J.P. is the Clinical Lead and T.M.C. is the Advisor to the Fifth siology 1965; 26: 756–63 National Audit Project on Accidental Awareness during 14 de Jong RH, Eger EI 2nd. MAC expanded: AD50 and AD95 values of General Anaesthesia, of the Royal College of Anaesthetists common inhalation in man. 1975; 42: and Association of Anaesthetists of Great Britain and 384–9 Ireland. The authors contributed to the Royal College’s feed- 15 Stoelting RK, Longnecker DE, Eger EI II. Minimum alveolar con- back to the draft NICE Guidance, which included some of the centrations in man on awakening from methoxyflurane, halo- points raised in this article. The views expressed are their thane, ether and fluroxene : MAC awake. Anesthesiology 1970; 33:5–9 own, and not that of the NAP5 Project or of the Royal 16 Schwab HS, Seeberger MD, Eger EI II, Kindler CH, Filipovic M. College of Anaesthetists or of the Association of Anaesthe- Sevoflurane decreases values more than does tists of Great Britain and Ireland. Although much of the com- halothane at equal MAC multiples. Anesth Analg 2004; 996: mentary above focuses on the bispectral index monitor, this 1723–7 Downloaded from https://academic.oup.com/bja/article/110/3/325/250090 by guest on 30 September 2021 is for no other reason than (as the NICE Guidance and Tech- 17 Ibrahim AE, Taraday JK, Kharasch ED. Bispectral index monitoring nology Report state) because most of the existing research during with sevoflurane, , and propofol. concerns this monitor. Anesthesiology 2001; 95: 1151–9 18 Glass PS, Bloom M, Kearse L, Rosow C, Sebel P, Manberg P. Bispec- tral analysis measures sedation and effects of propofol, References midazolam, isoflurane, and alfentanil in healthy volunteers. 1 Bruchas RR, Kent CD, Wilson HD, Domino KB. Anesthesia aware- Anesthesiology 1997; 86: 836–47 ness: narrative review of psychological sequelae, treatment, 19 Cook TM. Anesthesia awareness and the bispectral index. N Engl J and incidence. J Clin Psych Med Set 2011; 18: 257–67 Med 2008; 359: 430 2 Avidan MS, Zhang L, Burnside BA, et al. Anesthesia awareness and 20 Rampil IJ, Kim J, Lenhardt R, et al. Bispectral EEG index the bispectral index. N Engl J Med 2008; 358: 1097–108 during nitrous oxide administration. Anesthesiology 1998; 89: 3 Avidan MS, Jacobsohn E, Glick D, et al.; BAG-RECALL Research 671–7 Group. Prevention of intraoperative awareness in a high-risk sur- 21 Puri GD. Paradoxical changes in bispectral index during nitrous gical population. N Engl J Med 2011; 365: 591–600 oxide administration. Br J Anaesth 2001; 86: 141–2 4 Sandin RH, Enlund G, Samuelsson P, Lennmarken C. Awareness 22 Morris C, Perris A, Klein J, Mahoney P. Anaesthesia in haemo- during anaesthesia: a prospective case study. Lancet 2000; 355: dynamically compromised emergency patients: does ketamine 707–11 represent the best choice of induction agent? Anaesthesia 5 Myles PS, Leslie K, McNeil J, Forbes A, Chan MT. Bispectral 2009; 64: 532–9 index monitoring to prevent awareness during anaesthesia: the B- 23 Hirota K, Kubota T, Ishihara H, Matsuki A. The effects of nitrous Aware randomised controlled trial. Lancet 2004; 363: 1757–63 oxide and ketamine on the bispectral index and 95% spectral 6 Sebel PS, Bowdle TA, Ghoneim MM,et al. The incidence of aware- edge frequency during propofol–fentanyl anaesthesia. Eur J ness during anesthesia: a multicenter United States study. Anesth Anaesthesiol 1999; 16: 779–83 Analg 2004; 99: 833–9 24 Vereecke HEM, Struys MMRF, Mortier EP. A comparison of 7 Pandit JJ, Cook TM, Jonker WR, O’Sullivan E. A national survey of bispectral index and ARX-derived auditory anaesthetists (NAP5 Baseline) to estimate an annual incidence index in measuring the clinical interaction between of accidental awareness during general anaesthesia in the ketamine and propofol anesthesia. Anaesthesia 2003; 58: United Kingdom. Anaesthesia, in press 957–61 8 Pandit JJ, Cook TM, Jonker WR, O’Sullivan E. A national survey of 25 Mashour GA, Shanks A, Tremper KK, et al. Prevention of intrao- anaesthetists (NAP5 Baseline) to estimate an annual incidence perative awareness with explicit recall in an unselected surgical of accidental awareness during general anaesthesia in the population: a randomized comparative effectiveness trial. Anes- United Kingdom. Br J Anaesth,inpress thesiology 2012; 117: 717–25 9 National Institute for Clinical Excellence. NICE Diagnostics Guid- 26 Rawlins MD. De testimonio. On the Evidence for Decisions about ance: Depth of anaesthesia monitors—Bispectral index (BIS), the Use of Therapeutic Interventions. The Harveian Oration. E-Entropy and Narcotrend Compact M. November 2012. Available London: Royal College of Physicians, 2008 from www.nice.org.uk/dg6 (accessed 26 November 2012) 27 Cook TM, Pandit JJ. NAP5: accidental awareness during general 10 Shepherd J, Jones J, Frampton G, Bryant J, Baxter L, Cooper K. anaesthesia. Bull Roy Coll Anaesth 2012; 72:29–31 Depth of anaesthesia monitoring (E-Entropy, Bispectral Index 28 Ghosh S, Arthur B, Klein AA. NICE guidance on CardioQTM oe- and Narcotrend). Health Technology Assessment, 2012. Available sophageal Doppler monitoring. Anaesthesia 2011; 66: 1081–3 from http://www.nice.org.uk/nicemedia/live/13504/59370/59370. 29 Clutton-Brock TH. Status of national guidelines in dictating pdf (accessed 26 November 2012) individual clinical practice and defining negligence. Letter 1. 11 Punjasawadwong Y, Phongchiewboon A, Bunchungmongkol N. Br J Anaesth 2012; 109: 284 Bispectral index for improving anaesthetic delivery and post- 30 Hutton P, Bailey D, Green D, Hill A, Karim A, Bindal N. Letter 2. operative recovery. Cochrane Database Syst Rev 2007; 4 Br J Anaesth 2012; 109: 284–5 12 Aranake A, Mashour GA, Avidan MS. Minimum alveolar concentra- 31 Bodenham AR, Fearnley RA, Bell MDD. Reply from the authors. Br J tion: ongoing relevance and clinical utility. Anaesthesia, in press Anaesth 2012; 109: 285–7

328