Assessing the effectiveness of new technologies

Adrian K Dixon

Introduction • Technical performance; Rapid changes in the way in which • Diagnostic performance; healthcare is delivered continually • Diagnostic impact; necessitate different methods of • Therapeutic impact; and assessing new technology. Economical • Impact on health. issues have become paramount. Perhaps even more than mere fi scal assessment comes the need to avoid Technical performance hospitalisation and reduce the number The fi rst level is that of technical of patient contacts with expensive performance, which assesses whether secondary care. Whether these fi nancial the new equipment or technique does imperatives are perceived by the patient deliver what it is expected to do on as optimal often remains unanswered, the technical front. For a new piece even in these days of supposed ‘patient of diagnostic imaging equipment, this choice’. The main methods of new might assess whether or not the new There should be no let technology assessment are discussed machine yields anatomical images of in this article, along with possible future spatial (and/or contrast) resolution equal assessments in the light of recent to or better than existing equipment3. For up in the quest for even changes in the way in which diagnostics nuclear medicine and other functional are being introduced into the United imaging techniques this might assess better techniques for Kingdom (UK). the additional physiological data that is obtained. For a new interventional identifying women at New technology stent, it might be a more mechanical assessment about tensile strength and risk from this dreaded assessment biocompatibility. The original hierarchical fi ve level method For the average radiology department, disease (breast cancer). of assessing new technology within the there is little involvement in such imaging fi eld is now well established with assessments as manufacturers only minor variations1,2. These are will not bring novel techniques or

12 IMAGING & ONCOLOGY | 2007 technologies to the market place numbers of true negative fi ndings in without all such technical performance normal women. The predictive value of information available. However, research a positive result is not all that high and departments become involved with this means that a fair number of normal assessments of prototypes, and some of women still have to undergo a traumatic these may have been developed by their biopsy. The sensitivity for some of the staff members. more aggressive lesions is rather lower and MRI may be better, especially in younger women6. Consequently, despite Diagnostic performance the considerable advances which have Diagnostic performance, namely how arisen as a result of the NHSBSP, there well the new technique fares with should be no let up in the quest for even regards to making the diagnosis, is better techniques for identifying women often regarded as the be all and end at risk from this dreaded disease. all of technology assessment. It is The assessment of diagnostic often regarded, erroneously, as being performance is relatively straightforward synonymous with the diagnostic in breast screening where a fi nal accuracy of the new technique. It is diagnosis (cancer or no cancer) is now realised that studies describing defi nitively established. Many workers overall accuracy are signifi cantly have pointed out how diffi cult this infl uenced by the prevalence of disease becomes when the diagnosis is elusive in the population under scrutiny. In and the patient may not necessarily fact, the markers of most importance undergo early biopsy or surgery – viz are the sensitivity and specifi city of magnetic resonance imaging (MRI) in the new investigation and these are the diagnosis of multiple sclerosis7. Even often combined to produce a receiver for something relatively straightforward operator curve for the new investigation4. such as MR of the knee, the fact that Again, many prestigious research only a selection of patients undergo Whilst the overall departments become involved in such arthroscopy hinders the assessment of early assessments. But of course, the diagnostic performance8. accuracy is very high prevalence of various disease processes in these institutions may be far removed (NHSBSP), this is based from the real world of a typical general Diagnostic impact hospital. Hence, there is need for If a new technique successfully passes on the very large numbers early, large, multicentre studies of new through the above two stages of methodologies, which will include a assessment, it should be possible to spectrum of different practices. prove that it helps make an impact of true negative fi ndings in The data now available from the very on the clinician’s diagnosis, either by high quality UK National Health Service providing a new, unexpected diagnosis normal women. Breast Screening Programme (NHSBSP) or by improving the clinician’s confi dence provides interesting conclusions: whilst in their working clinical diagnosis. Such the overall accuracy is very high5, information is extremely diffi cult to obtain this is based mainly on the very large unless the confi dence in the working

2007 | IMAGING & ONCOLOGY 13 diagnosis is established before the Therapeutic impact a randomised trial14,15. However, fi rst imaging investigation. Historically, Now that the investigation under scrutiny randomised trials are notoriously diagnostic confi dence information has has passed the fi rst three levels of the diffi cult to perform in diagnostic been obtained in patients who were technology assessment hierarchy, it radiology and ethical review referred for VQ scintigrams for possible must be shown to have therapeutic boards may refuse to pulmonary embolus (PE) where the a value. If, after an investigation has sanction a study where one priori clinical probability was necessary been performed, the clinician ends arm of patients is ‘denied’ in order to provide a defi nitive report. up doing what he or she would have access to the diagnostic This made it relatively simple to compare done anyway, it could be argued that test under scrutiny, no and prove the benefi cial diagnostic the test was unnecessary. In the days matter how new or impact of computed tomography (CT) when radiologists had to scrutinise experimental. pulmonary angiography9 and has led requests assiduously to avoid too to general acceptance in the UK that many referrals and a departmental Impact on nuclear medicine (NM) referrals for overspend, they frequently asked ‘will possible PE should now be limited to the result of this investigation change health those patients with a relatively normal your management?’. Indeed, this is It should follow that any chest radiograph, with all other patients still a very valid question, particularly investigation of proven directed towards CT. when radiation exposure is at stake11,12. technical and diagnostic Relatively few clinicians state their Again, clinicians must be encouraged performance with positive possible differential diagnosis in general to state before an investigation details clinical and therapeutic impact referrals for imaging examinations, let about their proposed management; would be associated with a alone their confi dence in the leading and they must also be asked what their benefi cial impact on health. But diagnosis. Only when this information management will be once the result of this may be diffi cult to prove: the side is demanded at the outset can the true the investigation is apparent. Only by this effects of surgery may mask the health diagnostic impact of a new investigation method can the true therapeutic impact gain in the immediate post-operative be measured. Some might argue that all be measured. This can be achieved by period and sometimes the diagnosis may imaging is aimed at reducing diagnostic means of a prospective observational have such an unfavourable outcome (for dangerous investigations. The use of uncertainty10 and that any investigation study13 but a purer study is where the example, pancreatic cancer) that the MRI instead of Endoscopic Retrograde which improves diagnostic confi dence new investigation is assessed alongside benefi cial infl uence of the diagnostic Cholangiopancreatography (ERCP) is a ‘good thing’. But an investigation a conventional investigation on the test is masked by the natural history of is one example; the development which merely confi rms the clinician’s same patient population by means of the disease. Even for benign disease, of fl uid sensitive techniques diagnosis may be an unnecessary luxury. it is diffi cult to prove the benefi t of allowed the introduction of MR Alternatively, an investigation which high technology diagnosis in terms cholangiopancreatography (MRCP)18, completely alters the clinician’s diagnosis of improvement in quality of life16. It which has virtually replaced diagnostic (say from benign disease to probable is much easier to prove the benefi cial ERCP. Interestingly, this change in cancer) may completely change the infl uence of interventional radiological practice has come about with no patient’s clinical course – and save the techniques, where great advances have full health technology assessment clinician the embarrassment of going been made recently17. As a result, several – merely common sense! In these days up the wrong diagnostic alley. And such surrogate markers have been used to of patient choice, it may be that we changes in diagnosis (diagnostic impact) assess impact of new techniques on should ask the patients which test they must be measured as radiologists and health. Clearly, the avoidance of ionising would prefer – all other information clinicians need to justify the considerable radiation is one marker; and another being equal; the presumed preference expense of diagnostic certainty. may be the avoidance of potentially for MR over myelography was never

14 IMAGING & ONCOLOGY | 2007 formally assessed but can be assumed. in imaging capacity, fi rst on the back subjective clinical examination, even for However, the preference for MR over of various cancer initiatives and now something simple such as the presence some other techniques may not be as from additional independent sector or absence of an abdominal mass14. apparent as expected; for example, provision, the UK still performs far fewer Furthermore, the newer generation of not all patients prefer conventional CT and MR examinations than other clinicians relies much more on the results MR of the shoulder over conventional developed nations. For example, the rate of imaging to guide their management arthrography – despite the perceived for CT examinations in the USA is now decisions and will frequently insist on invasiveness of the latter19. around 250 per 1000 of the population high technology imaging before offering per annum; many times the rate in the a fi nal clinical diagnosis. Additionally, UK. Numerous experts have tried to most modern clinicians prefer the newer Societal impact as a decide on the optimal level of provision investigations to the old21. Indeed, technology assessment for all such examinations, with scant there is good evidence that modern evidence available. However, it is worth imaging can optimise the surgical measure considering some of the societal benefi ts approach in many conditions, such as Because of the diffi culties in proving of increased imaging capacity which are rectal carcinoma22. Such advantages the benefi ts of high technology now being addressed: need to be quantifi ed in many more investigations, health economists The patient. Patients do not like waiting clinical situations. Interestingly, defence and others responsible for planning for investigations and would prefer organisations have started to realise and purchasing healthcare to avoid unnecessary and additional the importance of preoperative imaging have looked to yet other visits to clinics, hospitals, etc. So, if and there are now some cases coming surrogates. Increasingly, health there is adequate capacity to offer an through where surgery is regarded as economists point out that investigation (eg Chest CT) on the day inappropriate in the light of the imaging the really expensive bits of the clinic attendance, the patient is fi ndings (or the absence thereof). of healthcare relate to saved an unnecessary second visit for The community. CT and MR were both secondary care and the diagnostic imaging required, and developed at times when healthcare in-patient stays. If it the whole investigation is cheaper – no costs were under very close scrutiny and can be shown that bookings or letters, less car parking, etc. both came to be regarded (erroneously) If it can be shown the judicious use of There is considerable evidence that as expensive investigations. Of course, imaging can make MR is a better investigation for lumbar when these machines could only that the judicious use secondary care more spine problems than plain radiography handle one patient an hour they were of imaging can make effi cient and shorten yet many patients are still referred for expensive and many of the original cost- hospital admissions, conventional lumbar spine radiographs20. effectiveness studies were based on secondary care more then the case for Because of this a small number of very high costs per procedure. However, effi cient and shorten the greater use patients suffer a delay in the diagnosis both techniques can now offer very and increased of serious disease (metastatic deposits, rapid, high volume functionality for most hospital admissions, expenditure on disc space infection, major disc routine referrals. The costs of CT and imaging is made. herniation, etc). When multiplied, the MR are often lower than the alternatives then the case for And this is very much cost of such delays in diagnosis may they have replaced23,24. For example the greater use and behind the recent justify the increased expenditure. unenhanced CT of the abdomen is UK NHS initiatives in The referring clinician. Only recently cheaper than even a short intravenous increased expenditure providing increased have clinicians started to accept that urogram (IVU); the cost of an MR of the on imaging is made. access to imaging. Even the objective fi ndings of imaging are, lumbar spine pales into insignifi cance with the recent expansion in many situations, superior to their compared with the cost of a myelogram.

2007 | IMAGING & ONCOLOGY 15 But the real gain for high technology Addenbrooke’s Hospital, Cambridge comes with reduced hospital admissions. University Hospitals NHS Foundation It has been shown in numerous studies Trust and professor of radiology at the that early and judicious use of a single, University of Cambridge. high technology investigation can provide full diagnostic information (thereby avoiding a lengthy sequence of other References tests) which, in turn, is related to shorter 1. Fineberg HV, Wittenberg J, Ferrucci hospital stays14,15. Indeed, for abdominal JT Jr, Mueller PR, Simeone JF, Goldman conditions, a prompt CT examination J. The clinical value of body computed may assist the emergency physician to tomography over time and technologic decide not to admit the patient. Again, change. AJR 1983;141:1067-72. much work is still required to prove 2. Mackenzie R, Dixon AK. Measuring that overall costs can be reduced by the effects of imaging: an evaluative increasing the availability of and access framework. Clin Radiol 1995;50:513-8 to appropriate imaging. 3. Mackenzie R, Logan BM, Shah NJ, Keene GS, Dixon AK. Direct anatomical- MRI correlation: the knee. Surg Radiol Conclusion Anat 1994;16:183-192 Radiologists, radiographers and 4. Doubilet PM. Statistical techniques others allied to imaging have, hitherto, for medical decision making: been satisfi ed merely in showing the applications to diagnostic radiology. marvellous images produced by the AJR 1988; 150:745-50. increasingly sophisticated imaging 5. Department of Health. Breast devices now available and basking in Cancer Screening. www.dh.gov. the collective, refl ected glory. Although uk/PublicationsAndStatistics/ these new techniques obviously assisted Statistics/StatisticalWorkAreas/ the referring clinician and often saved StatisticalHealthCare/ the patient more invasive tests, there StatisticalHealthCareArticle/ has still been relatively little effort made fs/en?CONTENT_ to prove that they contribute to the ID=4104034&chk=mjJ%2Bof totality of healthcare. Only by proving the 6. Leach MO, Boggis CR, effectiveness of pounds/dollars/euros Dixon AK, Warren RM, et al. spent on imaging will we be able to Screening with magnetic obtain the real and sustainable growth resonance imaging and in imaging which many of us consider mammography of a UK necessary. And, we will have to be on population at high familial very fi rm ground, because increased risk of breast cancer: a expenditure on imaging will, almost prospective multicentre certainly, mean cuts elsewhere. cohort study (MARIBS). Lancet 2005; 365: 1769-78. Adrian Dixon is an honorary consultant 7. Harbord R, Main C, Deeks, at the Department of Radiology, JJ et al. Accuracy of magnetic

16 IMAGING & ONCOLOGY | 2007 resonance imaging for the diagnosis of unknown cause: prospective randomised Radiology 1999; 211:215-22. AK, Antoun NM, Moffat DA, Todd multiple sclerosis: systematic review. study. British Medical Journal 2002; 23. Moore AT, Dixon AK, Rubenstein CJ. Measuring the effects of medical BMJ 2006; 332:875-84. 14:325 (7377): 1387 D, Wheeler T. Cost-benefi t evaluation imaging in patients with possible 8. Mackenzie R, Palmer CR, Lomas DJ, 16. Mackenzie R, Hollingworth W, of body computed tomography. Health cerebellopontine angle lesions: a four- Dixon AK. Magnetic resonance imaging Dixon AK. Quality of life assessment in Trends 1987;19:8-12. center study. Acad Radiol 1998;5 (Suppl of the knee: diagnostic performance the evaluation of magnetic resonance 24. Hollingworth W, Bell MI, Dixon 2):306-309. statistics. Clin Radiol 1996;51:251-7. imaging. Qual Life Res 1994;3:29-37. 9. Cross JJL, Kemp PM, Walsh CG, 17. Molyneux AJ, Kerr RS, Clarke Flower CDR, Dixon AK. A randomized M, et al. International subarachnoid trial of spiral CT and ventilation aneurysm trial (ISAT) of neurosurgical perfusion scintigraphy for the diagnosis clipping versus endovascular coiling in of pulmonary embolism. Clin Radiol 2143 patients with ruptured intracranial 1998;53:177-82. aneurysms: a randomised comparison 10. Hobby JL, Tom BD, Todd C, of effects on survival, dependency, Bearcroft PW, Dixon AK. Communication seizures, rebleeding, subgroups, of doubt and certainty in radiological and aneurysm occlusion. Lancet. reports. Br J Radiol. 2000; 73:999-1001. 2005;366:809-17. 11. Royal College of Radiologists. 18. Bearcroft PW, Lomas DJ. Magnetic Making the best use of a department of resonance cholangiopancreatography. clinical radiology: guidelines for doctors. Gut 1997; 41: 135-7 2007 RCR, London. 19. Blanchard TK, Bearcroft PWP, 12. IRMER The Ionising Dixon AK, Lomas DJ, Teale A, Radiation (Medical Exposure) Constant CR, Hazleman BL. Regulations 2000 www.dh.gov.uk/ Magnetic resonance imaging or Increased PublicationsAndStatistics/Publications/ arthrography of the shoulder: which PublicationsPolicyAndGuidance/ do patients prefer? Br J Radiol PublicationsPolicyAndGuidanceArticle/fs/ 1997;70:786-90. expenditure on en?CONTENT_ID=4007957&chk=FkG/YB 20. Hollingworth W, Todd CJ, 13. Mackenzie R, Dixon AK, Keene GS, King H, Males T, Dixon AK, Hollingworth W, Lomas DJ, Villar RN. Karia KR, Kinmonth AL. Primary imaging will mean Magnetic resonance imaging of the knee: care referrals for lumbar spine assessment of effectiveness. Clin Radiol radiography: diagnostic yield and cuts elsewhere. 1996;51:245-50. clinical guidelines. British Journal of 14. Dixon AK, Kelsey Fry I, Kingham General Practice 2002; 52: 475-80. JGC, McLean AM, White FE. Computed 21. Southern JP, Teale A, Dixon AK, tomography in patients with an Freer CEL, Rubenstein D, Wilkinson abdominal mass; effective and effi cient? IMS, Hall LD, Sims C, Williams A. An Lancet 1981;i:1199-1202. audit of the clinical use of magnetic 15. Ng CS, Watson CJ, Palmer CR, resonance imaging of the head and See TC, Beharry NA, Housden BA, spine. Health Trends 1991;23:75-9. Bradley JA, Dixon AK. Evaluation of early 22. Brown G, Richards CJ, Newcombe abdominopelvic computed tomography RG, et al. Rectal carcinoma: thin-section in patients with acute abdominal pain of MR imaging for staging in 28 patients.

2007 | IMAGING & ONCOLOGY 17