EVIDENCE-BASED PSYCHIATRY How to appraise an article on diagnosis

Hagen Rampes, Jomes P. Warner and Robert Blizard

In our evidence-based journal club we appraised an Question article investigating the use of the CAGE questions in What is the evidence to support the use of the screening psychiatric populations for alcohol misuse. CAGE questionnaire in detecting alcohol depen We calculated a likelihood ratio of four for a score of dence or misuse in psychiatric in-patients? two or more positive CAGE questions, suggesting the It is helpful to formulate a question in three CAGEis a moderately useful screening instrument. parts. The first part concerns the population (i.e. the type of patient that you are interested in), here we are interested in psychiatric in-patients. We could narrow the field further, for example to The importance of evidence-based medicine in young men with schizophrenia, although this psychiatry is gradually becoming apparent would reduce our chances of finding a relevant (Geddes, 1996). We continue our series of study. The second part concerns the intervention articles, based on the experience of a journal or manoeuvre. Here the manoeuvre is the use of club, introducing the principles that are utilised the CAGE questionnaire. The third part of the in critical appraisal. The present article deals question is the outcome. We are interested in a with a paper on a diagnostic test. This article is diagnosis of alcohol dependence and or misuse. intended to give a brief, focused appraisal, not a comprehensive critique of the paper selected for Literature search appraisal. It would be useful, but not essential, One of us remembered seeing a discussion of the to read the paper appraised in conjunction with merits of the CAGE questionnaire in Bandolier this article. (Moore et al, 1997). In the discussion in Bandolier, a paper on the use of CAGE was appraised. However, the population was from an out-patient medical practice in an urban teach ing hospital from Virginia, USA (Buchsbaum et Use of the CAGEquestionnaire in al, 1991). Clearly it would be preferable to find a detecting alcohol dependence paper concerning the patient in our vignette (i.e. a psychiatric in-patient). A Mediine search Vignette (WinSpirs 2.0) using the term 'CAGE' for a A 36-year-old unemployed male presented with a textword search was carried out from 1966 to two-week history of worsening of auditory hallu 1987. This identified 4662 articles. Second, a cinations. He had a past psychiatric history of textword search for articles containing the word paranoid schizophrenia. He scored two out of 'hospital' was carried out, revealing 657 016 four questions on the CAGE questionnaire (May- articles. Third, a textword search for articles field et al. 1974). The CAGE questions are a brief containing the words starting with 'psychiat' was screening tool widely used to detect problem conducted by truncating the word with an drinking, consisting of: Have you ever felt you asterisk (Psychiat*) (revealing 142 424 articles). should cut down on your drinking; Have people Combining records one, two and three by using annoyed you by criticising your drinking?; Have the boolean term 'and' refines our search and you ever felt bad or guilty about your drinking?; identifies a manageable number of relevant Have you ever had a drink first thing in the articles. The abstracts of these 19 articles were morning to steady your nerves or get rid of a scanned and the one that appeared most hangover (eye-opener)? relevant was extracted. In the ensuing discussion of this case, the The article chosen was entitled 'Comparison of significance of the score of the CAGE question questionnaire and laboratory tests in the detec naire was discussed. It was decided to investi tion of excessive drinking and alcoholism' (Ber- gate this further. nadt et a/, 1982).

506 Psychiatric Bulletin (1998). 22, 506-509 EVIDENCE-BASED PSYCHIATRY

Getting the article the Bethlem and Maudsley Hospitals in a 10- The article was easily retrieved from the Chelsea month period. Although the Maudsley is a and Westminster Hospital Library and obtained tertiary referral centre, taking some specialist immediately. patients, we feel that it is unlikely this will influence the diagnosis of alcoholism to any large degree. Brief outline of the article The aim of the article was to compare the use of Did the results of the test being evaluated different questionnaires and laboratory tests in influence the decision to perform the reference detecting 'excessive drinking' and 'alcoholism'. standard? No. It appears that all admissions Three hundred and eighty-five (198 male) psy had the screening test and the reference chiatric in-patients (aged 16-65) at the Maudsley standards. and Bethlem Hospitals were recruited in a 10- month period in 1980. A research nurse admin Were the methods for performing the test des istered a structured interview that included cribed in sufficient detail to permit repli the CAGE test and two other screening tests; the cation? Yes. The authors report the method Brief Michigan Alcohol Screening Test (Selzer. is considerable detail. 1971) and the Reich interview (Reich et al, 1975). Within 48 hours of admission, blood was taken What were the results? for mean corpuscular volume, gamma glutamyl transpeptidase test, aspartate transaminase, The authors only present data for patients who alkaline phosphatase, and other tests thought scored two or more on the CAGE questionnaire. For this cut-off, we are given a sensitivity of 0.91 to be influenced by alcohol consumption. A score of two or more on the CAGE was taken as and specificity of 0.77 for the detection of indicating 'alcoholism'. Consumption of more alcoholism. From this we can reconstitute the than 16 'drinks' per day over the year before results table (see Table 1). Therefore, using this criterion, 91% of 'alcoholics' and 77% of 'non- admission was taken as drinking a hazardous amount or 'excessive drinking'. Case notes were alcoholics' will be correctly classified. perused after discharge to determine whether a primary or secondary diagnosis of 'alcoholism' Are likelihood ratios for the test results presented was recorded. Out of 385 patients. 371 were or data necessary for their calculation pro included in the data analysis (185 male). Forty- vided? Likelihood ratio is the ratio of the like two were categorised as 'excessive drinkers' and lihood of the disease being present given a 49 had primary or secondary 'alcoholism'. positive test result, to the likelihood of no disease being present given a positive test result. In other words, it provides an index of increased like Critical appraisal of the article on lihood of a disease being present given a positive diagnosis test result. The likelihood ratio for a positive result is calculated simply by: sensitivity/ This followed the recommendations of the Evi (1 —¿specificity) dence-Based Medicine Working Group (Jaeschke Likelihood ratios may also be calculated for a et al, 1993, 1994). There are three main negative test result. In this case, the likelihood components to the appraisal: Are the results ratio for a negative test provides an index of how valid? What are the results? Will the results help likely a disease is to be absent given a negative me in patient care? result (see Sackett. 1997). Using the sensitivity and specificity provided, Are the results of the study valid? the likelihood ratio for a score of two or more on the CAGE is approximately four. The usefulness Was there an independent, blind comparison with a reference standard? The reference standards used were clinician's diagnosis, Murray's Screen Table 1. Reconstituted results table for CAGE performance in detecting alcoholism (Adapted ing interview (Murray, 1977) and Research from Bernadt et al. 1982) Diagnostic Criteria for alcoholism (Spitzer & Endicott, 1978). Bernadt et al do not state whether these were conducted independently or blind to the result of the screening test. CAGE2+CAGEscore 0TotalAlcoholismpresent45449Alcoholismabsent74248322119252371score 1 or Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice? The patients Sensitivity=0.91; specificity=0.77; likelihood ratio for a were in a group of 385 consecutive admissions to positive test=0.9170.23 ^ 4.

How to appraise an article on diagnosis 507 EVIDENCE-BASED PSYCHIATRY

Table 2. Likelihood ratio values and their rele 0.666/11+0.666)=0.4. So if our patient scores vance to diagnosis two or more on the CAGE, the probability he has alcoholism has risen from 15% to 40%. Likelihood ratio Diagnostic impact 10 Very positive Will the results change my management? Quite 3 Moderately positive possibly. The CAGE is not a diagnostic tool, but a 1 Neutral positive result should alert clinicians to a greater 0.3 Moderately negative possibility of alcohol misuse. Furthermore, 0.1 Very negative searching questions could then be asked in order The higher the likelihood ratio the more likely a positive to confirm or refute the diagnosis. If alcohol test will accurately indicate the presence of a disease. misuse is present this could have a significant A test with a likelihood ratio of one will not help impact on symptomatology and outcome. Co- clinicians decide whether a disease is present or morbidity is a major problem that has important absent. Likelihood ratios for negative tests are implications for treatment and risk assessment. calculated by (1-sensitlvity)/specificity. A likelihood ratio of less than one will reduce the post-test odds. Will patients be better off as a result of the The lower the likelihood ratio for a negative test, the test? This is more difficult to answer. The more likely that negative test excludes the disease prevailing clinical model: test-»diagnosis-treat (after Sackett et al. 1997). ment-»improved outcome, represents an ideal which, in practice, may not be achievable. A positive test should result in a further evaluation of the patient. Our patient's current symptoms of likelihood ratios, given in Table 2, shows this is in the region of a moderately helpful result. In may be due to alcohol alone, in which case other words, a positive result of a test with a detoxification may lead to a resolution of his likelihood ratio of four moderately increases our symptoms. Alternatively, he may have schizo certainty of the disease being present. phrenia and problem drinking. In this event, tackling his drinking may help his psychotic symptoms, or may lead to better adherence to his Will the results help me in caring for my treatment regimen. In this scenario, whether the patients? appropriate treatment for his alcohol misuse Will the reproducibility of the test result and its leads to improved outcome would merit a further interpretation be satisfactory in my setting? Yes, evidence-based medicine exercise. we feel it is. The CAGE questionnaire is simple to administer and interpret.

Are the results applicable to my patient? If a Comment patient in a similar setting to that of the study Alcohol misuse is common in psychiatric prac scores two or more on the CAGE, then the tice. The CAGE is a well established screening likelihood ratio of a diagnosis of alcoholism is tool for alcohol misuse, and the article reviewed 0.91/0.23*4. This result would be about four here investigates its utility in a psychiatric times as likely to be seen in someone with setting. The results of the article suggest a score alcoholism, as opposed to someone without of two or more on the CAGE should alert alcoholism. By using the likelihood ratio in clinicians to the possibility of alcohol misuse. conjunction with the probability of a disease The article was quite old and terminology in this being present before the test was carried out, the domain has changed. Many people no longer find probability of the disease being present with a the term 'alcoholism' acceptable, but Bernadt et positive result can be derived. This may be done al and clinicians today will have similar inter by multiplying the pre-test odds by the likelihood pretations of the term. ratio to get the post-test odds. The likelihood ratio of four for a score of two or Odds are related to probability and are more on the CAGE in the study population calculated by dividing the probability by suggests that it is a reasonable test. However, 1—probability. For example, the probability of a the likelihood ratio for the Brief Michigan Alcohol pregnant woman having a boy is 50%. The odds Screening Test, using a cut-off of six, was eight; are 0.5/(1 —¿0.5)or 1 (evens). In our example the double that of the CAGE. Other tests frequently pre-test probability of a patient having alcohol used to diagnose alcohol misuse perform less ism was around 15% (a figure that accords to our well; the likelihood ratio of an abnormal mean clinical practices). The odds are 0.15/0.85=1/6. corpuscular volume is two, and a raised gamma If we then multiply 1/6 by the likelihood ratio (4), glutamyl transpeptidase test 2.5. Therefore, the we get 4/6 (or 0.666). This is the post-test odds. CAGE appears to be a much better discriminator Converting this back to the post-test probability than either of these biochemical tests.

508 Rampes et al EVIDENCE-BASED PSYCHIATRY

It is unfortunate that the authors used an a GEDDES. J. (1996) On the need for evidence-based psychiatry. Evidence-Based Medicine, 1, 199-200. priori cut-off of two in the CAGE. Previous studies JAESCHKE.R.. GUYATT.G. H. & SACKETT.D. L. (1993) Users' have found much higher likelihood ratios for a guides to the medical literature. III. How to use an score of two, three or four. Bush et al (1987), in a article about a diagnostic test A. Are the results of the study of 518 medical patients found a likelihood study valid? Journal of the American Medical Association, 271. 389-391. ratio of 19 for a score of two, 170 for three and —¿,—¿& —¿(1994) Users' guides to the medical literature. III. infinity for a score of four. Incidentally, they found similarly disappointing utility of mean How to use an article about a diagnostic test. B. What corpuscular volume and gamma glutamyl trans- are the results and will they help me in caring for my patients? Journal of the American Medical Association, peptidase test. Sackett et al (1991). in their 271. 703-707. analysis of the interpretation of the use of the MAYFIELD.D.. McLEOD, G. & HALL. P. (1974) The CAGE CAGE, found a cut-off of two or more provided a questionnaire: validation of a new alcoholism screening instrument. American Journal of Psychiatry. 131. likelihood ratio of seven. The current article 1121-1123. suggests a lower value. This may be because the 'gold standard' diagnosis was different, or MOORE.A.. McQUAY.H. & MUIRGRAY.J. A. (1997) Clinical scoring for alcohol abuse. Bandolier, 4. 5-6. that a psychiatric population responds differ MURRAY. R. P. (1977) Screening and early detection ently to the questions. For example, psychiatric instruments for disabilities related to alcohol patients' responses may be contaminated by consumption. In Alcohol-Related Disabilities (eds G. Edwards et ai). Geneva: World Health Organization. disturbance of affect. In particular, two CAGE REICH. T.. ROBINS, L. N.. WOODRUFF.R. A., et al (1975) items expressly refer to guilt and annoyance. The Computer assisted derivation of a screening interview suggestion that the CAGE performs less well in a for alcoholism. Archiues of General Psychiatry. 32. 847- psychiatric population raises some questions 852. about its applicability in this group. SACKETT,D. L.. HAYNES.R. B., GUYATT,G. H., et al (1991) The We found this a particularly rewarding ex interpretation of diagnostic data. In Clinical ercise. The use of likelihood ratios helps to put Epidemiology: A Basic Science for Clinical Medicine (2nd 'meat on the bones' of screening and diagnostic edn). Boston. MA: Little Brown and Company. —¿,RICHARDSON,W. S.. ROSENBERG, W.. et al (1997) tests. Although the maths may appear complex, Evidence-Based Medicine: How to Practice and Teach it is relatively easily mastered, and readers are EBM. New York: Churchill Livingstone. SELZER, M. L. (1971) The Michigan Alcoholism Screening referred to Sackett et al (1997) for a clear Test. American Journal of Psychiatry. 127. 1653-1658. explanation and example. SPITZER,R. L. & ENDICOTT.J. (ÃŒ978)Medical and mental disorder: proposed definition and criteria. In Crm'cal ¡ssues in Psychiatric Diasgnosis (eds R. L. Spitzer & References D. F. Klein). New York: Raven Press. BUCHSBAUM.D. G.. BUCHANAN,R. G.. CENTOR.R. M., et al (1991) Screening for alcohol abuse using CAGE scores Hagen Rampes, Senior Registrar, South and likelihood ratios. Annals of Internal Medicine, 115. Kensington and Chelsea Mental Health Centre. 774-777. Chelsea and Westminster Hospital London and BERNADT,M. W.. MUMFORD.J.. TAYLOR.C., et al (1982) *James P. Warner, Lecturer, Robert Blizard. Comparison of questionnaire and laboratory tests in the Medical Statistician. University Department of detection of excessive drinking and alcoholism. Lancet, i. 325-328. Psychiatry, Royal Free Hospital School of BUSH, B.. SHAW,S.. CLEARY,P., et al (1987) Screening for Medicine. Rowland Hill Street. London NW3 2QG alcohol abuse using the CAGE questionnaire. American Journal oj Medicine. 82. 231-235. 'Correspondence

How to appraise an article on diagnosis 509