DOCSLIB.ORG

Estimating and Comparing Diagnostic Tests' Accuracy When the Gold

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Estimating and Comparing Diagnostic Tests' Accuracy When the Gold Estimating and Comparing Diagnostic Tests’ Accuracy When the Gold Standard Is Not Binary1 Nancy A. Obuchowski, PhD Rationale and Objectives. Investigators often need to assess the accuracies of diagnostic tests when the gold standard is not binary-scale. The objective of this article is to describe nonparametric estimators of diagnostic test accuracy when the gold standard is continuous, ordinal, and nominal scale. Materials and Methods. A nonparametric method of estimating and comparing the area under receiver operating charac- teristic (ROC) curves, proposed by DeLong et al, is extended to situations in which the gold standard is not binary. Two examples illustrate the methods. Results. Measures of diagnostic test accuracy, their variance, and tests for comparing two diagnostic tests’ accuracies in paired designs are presented for situations in which the gold standard is continuous, ordinal, and nominal scale. These summary measures of diagnostic test accuracy are analogous in form and interpretation to the area under the ROC curve. Conclusion. Dichotomizing the outcomes of a gold standard so that traditional ROC methods can be applied can lead to bias. The methods described here are useful for assessing and comparing summary test accuracy when the gold standard is not binary scale. They have limitations similar to other summary indices. Key Words. Diagnostic accuracy; gold standard; nonparametric statistics; receiver operating characteristic curve; ROC analysis. © AUR, 2005 Receiver operating characteristic (ROC) analysis has be- racy (ie, sensitivity and specificity) at various cutpoints of come the standard method for characterizing a diagnostic the diagnostic test’s results. test’s accuracy and for comparing accuracies of compet- Some examples of binary-scale gold standards are sur- ing diagnostic tests (1–4). In traditional ROC analysis, gery for determining whether or not a cancer has spread there exists a gold or reference standard, independent of to the lymph nodes, invasive catheter angiography for the diagnostic test(s), which provides a binary result as to determining if a significant stenosis is present or absent, the presence or absence of disease. Patients undergo both and biopsy for determining if a lesion is benign or malig- the gold standard procedure and the diagnostic test(s). nant. There is a vast literature on parametric, nonparamet- The diagnostic test results are then compared with the ric, and semiparametric statistical methods for estimating results of the binary-scale gold standard to estimate accu- and comparing ROC curves and their associated indices when the gold standard is binary scale (5–12). The application of ROC curves has expanded to many Acad Radiol 2005; 12:1198–1204 fields. Even in the diagnostic radiology setting, there are 1 From the Department of Quantitative Health Sciences/Wb4, The Cleve- many applications in which we want to characterize and land Clinic Foundation, 9500 Euclid Ave, Cleveland, OH 44195. Received compare the accuracies of diagnostic tests, but the gold March 3, 2005; revision received and accepted May 17, 2005. Address correspondence to: N.A.O. e-mail: [email protected] standard is not binary scale. One approach might be to © AUR, 2005 construct a binary scale from the nominal-, ordinal- or doi:10.1016/j.acra.2005.05.013 continuous-scale gold standard so that traditional ROC 1198 Academic Radiology, Vol 12, No 9, September 2005 NON-BINARY GOLD STANDARD Table 1 Comparison of Summary Measures of Diagnostic Test Accuracy Gold Standard Estimator of Accuracy 1 nt ns ␪^ ϭ ⌿͑ ͒ Binary ͚ ͚ Xit,Xjs where: ⌿ϭ1if Xit Ͼ Xjs ntns͑i ϭ 1͒ ͑j ϭ 1͒ ⌿ϭ0.5if Xit ϭ Xjs ⌿ϭ0if X Ͻ X . 1 N N it js ␪^ ϭ ⌿͑ ͒ Continuous ͚ ͚ Xit,Xjs where: ⌿ ϭ 1if tϾs and Xit Ͼ Xjs or s Ͼ t N͑NϪ1͒͑i ϭ 1͒ ͑j ϭ 1͒ and Xjs Ͼ Xit ⌿ ϭ 0.5if t ϭ sorXjs ϭ Xit ⌿ ϭ T T 0 otherwise. ␪^ ϭ Ϫ ͑ ͒ ͑ Ϫ ␪^ ͒ Ordinal 1.0 ͚ ͚ wts·L t,s · 1 ts where: is the binary-scale estimator ͑t ϭ 1͒ ͑s Ͼ t͒ nt ns ^ 1 Nominal SAME AS ORDINAL where ␪ts ϭ ͚ ͚ ⌿͑D͑tϪs͒tj,D͑tϪs͒sk͒ ͑nt·ns͒͑j ϭ 1͒ ͑k ϭ 1͒ ⌿(D(t Ϫ s)tj,D(t Ϫ s)sk) ϭ 1ifD(t Ϫ s)tj Ͼ D(t Ϫ s)sk ⌿(D(t Ϫ s)tj,D(t Ϫ s)sk) ϭ 1/2 if D(t Ϫ s)tj ϭD(t Ϫ s)sk ⌿(D(t Ϫ s)tj,D(t Ϫ s)sk) ϭ 0ifD(t Ϫ s)tj Ͻ D(t Ϫ s)sk. methods can be used. This approach, however, often di- gold standard is not binary-scale. Table 1 summarizes the lutes the measure of accuracy, can conceal important rela- various related measures of accuracy described in this tionships between the gold standard and diagnostic test, article. and can lead to biased estimates of accuracy (13,14). In this article, we describe nonparametric methods for Binary-Scale Gold Standard estimating and comparing the accuracies of diagnostic Let Xit denote the result of the diagnostic test for the tests when the gold standard is not binary scale. Accuracy ith patient with disease status t (as determined by the gold is defined using indices analogous to the traditional area standard), and let Xjs denote the result of the diagnostic under the ROC curve. test for the jth patient with disease status s. We can think We illustrate the methods using two examples. In the of status t as being “disease present” and status s as “dis- first example magnetic resonance imaging (MRI) is used ease absent.” X can be a rating-scale confidence score to score regions of the heart according to the amount of assigned by a reader according to the perceived likelihood visible scarring. The amount of scar is compared to the that disease is present (eg, 1 ϭ “definitely no disease,” 2 ϭ results of the gold standard, positron emission tomogra- “probably no disease,” 3 ϭ “possibly disease,” 4 ϭ “proba- phy (PET), where each region was read as normal, hiber- bly disease,” or 5 ϭ “definitely disease”), a percent confi- nating, ischemic, or necrotic (ordinal-scale gold standard). dence score assigned by a reader (ie, a value between 0 and In the second example, computed tomography (CT) is 100, where 0% means no confidence in the presence of dis- used to measure the size of renal tumors. Patients also ease and 100% is complete confidence in the presence of underwent surgery during which the renal tumor was disease), or an objective measurement (eg, attenuation value, measured. The surgical measurement is considered the serum iron concentration). gold standard (continuous-scale gold standard). A nonparametric estimator of diagnostic test accuracy is ESTIMATING AND COMPARING nt ns ^ 1 DIAGNOSTIC TEST ACCURACY ␪ ϭ ͚ ͚ ⌿(Xit, Xjs) (1) ntns(i ϭ 1) (j ϭ 1) We first review the nonparametric estimation of diag- nostic test accuracy when the gold standard is binary. We where nt and ns are the number of patients with disease use the methods of DeLong et al (6) to estimate the vari- status t and s in the study sample, respectively. We as- ance and covariance of the estimated accuracy because sume that higher diagnostic test results are indicative of we can easily extend the method to situations where the greater suspicion for disease status t; then the kernel is 1199 OBUCHOWSKI Academic Radiology, Vol 12, No 9, September 2005 defined as higher gold standard outcome (eg, larger renal tumor) has a higher diagnostic test result than the patient with a lower ⌿ϭ Ͼ gold standard outcome (eg, smaller tumor). 1ifXit Xjs The estimator of the variance of ^␪ ⌿ϭ ϭ 0.5 if Xit Xjs 1 N ␪^ ϭ Ϫ ␪^ 2 ⌿ϭ ϭ Vaˆr( ) ͚ [V(Xit) ] (3) 0ifXit Xjs (N ⁄ 2)((N ⁄2Ϫ 1)(i ϭ 1) ␪ˆ is an estimator of the area under the ROC curve (5,6). where the structural components are now defined as It is interpreted as: of two randomly chosen patients, one with disease and one without, ␪ is the probability that the 1 N patient with disease will have a higher result on the diag- V(Xit) ϭ ͚ ⌿ (Xit, Xjs) ͑i j͒ (4) nostic test than the patient without disease (5). (N Ϫ 1)(j ϭ 1) DeLong et al (6) used structural components to esti- mate the variance of ␪ˆ and the covariance between the Note that the variance estimator in Eq 3 is similar to the accuracies of two diagnostic tests. Details are given in the variance estimator for binary-scale gold standards (see Eq appendix. A2), but the estimator in Eq A2 has two sums of squares terms, one for diseased and one for nondiseased patients. Continuous-Scale Gold Standard To compare two diagnostic tests’ accuracies, ␪=1 and Suppose now that the gold standard is continuous ␪=2, we use the test statistic given in Eq A6, where the scale. For example, tumor diameter in centimeters is mea- estimator of the covariance is sured noninvasively with CT (the diagnostic test) and compared with the diameter measured at surgery (the con- 1 N ^1 ^2 1 ^1 2 ^2 Coˆv(␪ , ␪ ) ϭ ͚ [V(Xit) Ϫ ␪ ][V(Xit) Ϫ ␪ ] (5) tinuous-scale gold standard). Here, Xit denotes the result (N ⁄ 2)((N ⁄2)Ϫ 1)(i ϭ 1) of the diagnostic test for the ith patient who has a contin- uous-scale gold-standard outcome of t. As with a binary- Ordinal-Scale Gold Standard scale gold standard, X can be a subjective confidence With an ordinal-scale gold standard, we assume that score assigned by a reader or an objective measurement.
Load more

Recommended publications
  • Should the Randomistas (Continue To) Rule?
    Should the Randomistas (Continue to) Rule? Martin Ravallion Abstract The rising popularity of randomized controlled trials (RCTs) in development applications has come with continuing debates on the pros and cons of this approach. The paper revisits the issues. While RCTs have a place in the toolkit for impact evaluation, an unconditional preference for RCTs as the “gold standard” is questionable. The statistical case is unclear on a priori grounds; a stronger ethical defense is often called for; and there is a risk of distorting the evidence-base for informing policymaking. Going forward, pressing knowledge gaps should drive the questions asked and how they are answered, not the methodological preferences of some researchers. The gold standard is the best method for the question at hand. Keywords: Randomized controlled trials, bias, ethics, external validity, ethics, development policy JEL: B23, H43, O22 Working Paper 492 August 2018 www.cgdev.org Should the Randomistas (Continue to) Rule? Martin Ravallion Department of Economics, Georgetown University François Roubaud encouraged the author to write this paper. For comments the author is grateful to Sarah Baird, Mary Ann Bronson, Caitlin Brown, Kevin Donovan, Markus Goldstein, Miguel Hernan, Emmanuel Jimenez, Madhulika Khanna, Nishtha Kochhar, Andrew Leigh, David McKenzie, Berk Özler, Dina Pomeranz, Lant Pritchett, Milan Thomas, Vinod Thomas, Eva Vivalt, Dominique van de Walle and Andrew Zeitlin. Staff of the International Initiative for Impact Evaluation kindly provided an update to their database on published impact evaluations and helped with the author’s questions. Martin Ravallion, 2018. “Should the Randomistas (Continue to) Rule?.” CGD Working Paper 492. Washington, DC: Center for Global Development.
    [Show full text]
  • Getting Off the Gold Standard for Causal Inference
    Getting Off the Gold Standard for Causal Inference Robert Kubinec New York University Abu Dhabi March 1st, 2020 Abstract I argue that social scientific practice would be improved if we abandoned the notion that a gold standard for causal inference exists. Instead, we should consider the three main in- ference modes–experimental approaches and the counterfactual theory, large-N observational studies, and qualitative process-tracing–as observable indicators of an unobserved latent con- struct, causality. Under ideal conditions, any of these three approaches provide strong, though not perfect, evidence of an important part of what we mean by causality. I also present the use of statistical entropy, or information theory, as a possible yardstick for evaluating research designs across the silver standards. Rather than dichotomize studies as either causal or descrip- tive, the concept of relative entropy instead emphasizes the relative causal knowledge gained from a given research finding. Word Count: 9,823 words 1 You shall not crucify mankind upon a cross of gold. – William Jennings Bryan, July 8, 1896 The renewed attention to causal identification in the last twenty years has elevated the status of the randomized experiment to the sine qua non gold standard of the social sciences. Nonetheless, re- search employing observational data, both qualitative and quantitative, continues unabated, albeit with a new wrinkle: because observational data cannot, by definition, assign cases to receive causal treatments, any conclusions from these studies must be descriptive, rather than causal. However, even though this new norm has received widespread adoption in the social sciences, the way that so- cial scientists discuss and interpret their data often departs from this clean-cut approach.
    [Show full text]
  • Assessing the Accuracy of a New Diagnostic Test When a Gold Standard Does Not Exist Todd A
    UW Biostatistics Working Paper Series 10-1-1998 Assessing the Accuracy of a New Diagnostic Test When a Gold Standard Does Not Exist Todd A. Alonzo University of Southern California, [email protected] Margaret S. Pepe University of Washington, [email protected] Suggested Citation Alonzo, Todd A. and Pepe, Margaret S., "Assessing the Accuracy of a New Diagnostic Test When a Gold Standard Does Not Exist" (October 1998). UW Biostatistics Working Paper Series. Working Paper 156. http://biostats.bepress.com/uwbiostat/paper156 This working paper is hosted by The Berkeley Electronic Press (bepress) and may not be commercially reproduced without the permission of the copyright holder. Copyright © 2011 by the authors 1 Introduction Medical diagnostic tests play an important role in health care. New diagnostic tests for detecting viral and bacterial infections are continually being developed. The performance of a new diagnostic test is ideally evaluated by comparison with a perfect gold standard test (GS) which assesses infection status with certainty. Many times a perfect gold standard does not exist and a reference test or imperfect gold standard must be used instead. Although statistical methods and techniques are well established for assessing the accuracy of a new diagnostic test when a perfect gold standard exists, such methods are lacking for settings where a gold standard is not available. In this paper we will review some existing approaches to this problem and propose an alternative approach. To fix ideas we consider a specific example. Chlamydia trachomatis is the most common sexually transmitted bacterial pathogen. In women Chlamydia trachomatis can result in pelvic inflammatory disease (PID) which can lead to infertility, chronic pelvic pain, and life-threatening ectopic pregnancy.
    [Show full text]
  • Memorandum Explaining Basis for Declining Request for Emergency Use Authorization for Emergency Use of Hydroxychloroquine Sulfate
    Memorandum Explaining Basis for Declining Request for Emergency Use Authorization for Emergency Use of Hydroxychloroquine Sulfate On July 6, 2020, the United States Food and Drug Administration (FDA) received a submission from Dr. William W. O’Neill, co-signed by Dr. John E. McKinnon1, Dr. Dee Dee Wang, and Dr. Marcus J. Zervos requesting, among other things, emergency use authorization (EUA) of hydroxychloroquine sulfate (HCQ) for prevention (pre- and post-exposure prophylaxis) and treatment of “early COVID-19 infections”. We interpret the term “early COVID-19 infections” to mean individuals with 2019 coronavirus disease (COVID-19) who are asymptomatic or pre- symptomatic or have mild illness.2 The statutory criteria for issuing an EUA are set forth in Section 564(c) of the Federal Food, Drug and Cosmetic Act (FD&C Act). Specifically, the FDA must determine, among other things, that “based on the totality of scientific information available to [FDA], including data from adequate and well-controlled clinical trials, if available,” it is reasonable to believe that the product may be effective in diagnosing, treating, or preventing a serious or life-threatening disease or condition caused by the chemical, biological, radiological, or nuclear agent; that the known and potential benefits, when used to diagnose, treat, or prevent such disease or condition, outweigh the known and potential risks of the product; and that there are no adequate, approved, and available alternatives. FDA scientific review staff have reviewed available information derived from clinical trials and observational studies investigating the use of HCQ in the prevention or the treatment of COVID- 19.
    [Show full text]
  • Randomized Controlled Trials, Development Economics and Policy Making in Developing Countries
    Randomized Controlled Trials, Development Economics and Policy Making in Developing Countries Esther Duflo Department of Economics, MIT Co-Director J-PAL [Joint work with Abhijit Banerjee and Michael Kremer] Randomized controlled trials have greatly expanded in the last two decades • Randomized controlled Trials were progressively accepted as a tool for policy evaluation in the US through many battles from the 1970s to the 1990s. • In development, the rapid growth starts after the mid 1990s – Kremer et al, studies on Kenya (1994) – PROGRESA experiment (1997) • Since 2000, the growth have been very rapid. J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 2 Cameron et al (2016): RCT in development Figure 1: Number of Published RCTs 300 250 200 150 100 50 0 1975 1980 1985 1990 1995 2000 2005 2010 2015 Publication Year J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 3 BREAD Affiliates doing RCT Figure 4. Fraction of BREAD Affiliates & Fellows with 1 or more RCTs 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1980 or earlier 1981-1990 1991-2000 2001-2005 2006-today * Total Number of Fellows and Affiliates is 166. PhD Year J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 4 Top Journals J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 5 Many sectors, many countries J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 6 Why have RCT had so much impact? • Focus on identification of causal effects (across the board) • Assessing External Validity • Observing Unobservables • Data collection • Iterative Experimentation • Unpack impacts J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 7 Focus on Identification… across the board! • The key advantage of RCT was perceived to be a clear identification advantage • With RCT, since those who received a treatment are randomly selected in a relevant sample, any difference between treatment and control must be due to the treatment • Most criticisms of experiment also focus on limits to identification (imperfect randomization, attrition, etc.
    [Show full text]
  • Evaluating Diagnostic Tests in the Absence of a Gold Standard
    Evaluating Diagnostic Tests in the Absence of a Gold Standard Nandini Dendukuri Departments of Medicine & Epidemiology, Biostatistics and Occupational Health, McGill University; Technology Assessment Unit, McGill University Health Centre Advanced TB diagnostics course, Montreal, July 2011 Evaluating Diagnostic Tests in the Absence of a Gold Standard • An area where we are still awaiting a solution – We are still working on methods for Phase 0 studies • A number of methods have appeared in the pipeline – Some have been completely discredited (e.g. Discrepant Analysis) – Some are more realistic but have had scale-up problems due to mathematical complexity and lack of software (e.g. Latent Class Analysis) – Some in-between solutions seem easy to use, but their inherent biases are poorly understood (e.g. Composite reference standards) • Not yet at the stage where you can stick your data into a machine and get accurate results in 2 hours No gold-standard for many types of TB • Example 1: TB pleuritis: – Conventional tests have less than perfect sensitivity* Microscopy of the pleural fluid <5% Culture of pleural fluid 24 to 58% Biopsy of pleural tissue ~ 86% + culture of biopsy material – Most conventional tests have good, though not perfect specificity ranging from 90-100% * Source: Pai et al., BMC Infectious Diseases, 2004 No gold-standard for many types of TB • Example 2: Latent TB Screening/Diagnosis: – Traditionally based on Tuberculin Skin Test (TST) • TST has poor specificity* due to cross-reactivity with BCG vaccination and infection
    [Show full text]
  • Screening for Alcohol Problems, What Makes a Test Effective?
    Screening for Alcohol Problems What Makes a Test Effective? Scott H. Stewart, M.D., and Gerard J. Connors, Ph.D. Screening tests are useful in a variety of settings and contexts, but not all disorders are amenable to screening. Alcohol use disorders (AUDs) and other drinking problems are a major cause of morbidity and mortality and are prevalent in the population; effective treatments are available, and patient outcome can be improved by early detection and intervention. Therefore, the use of screening tests to identify people with or at risk for AUDs can be beneficial. The characteristics of screening tests that influence their usefulness in clinical settings include their validity, sensitivity, and specificity. Appropriately conducted screening tests can help clinicians better predict the probability that individual patients do or do not have a given disorder. This is accomplished by qualitatively or quantitatively estimating variables such as positive and negative predictive values of screening in a population, and by determining the probability that a given person has a certain disorder based on his or her screening results. KEY WORDS: AOD (alcohol and other drug) use screening method; identification and screening for AODD (alcohol and other drug disorders); risk assessment; specificity of measurement; sensitivity of measurement; predictive validity; Alcohol Use Disorders Identification Test (AUDIT) he term “screening” refers to the confirm whether or not they have the application of a test to members disorder. When a screening test indicates SCOTT H. STEWART, M.D., is an assistant Tof a population (e.g., all patients that a patient may have an AUD or professor in the Department of Medicine, in a physician’s practice) to estimate their other drinking problem, the clinician School of Medicine and Biomedical probability of having a specific disorder, might initiate a brief intervention and Sciences at the State University of New such as an alcohol use disorder (AUD) arrange for clinical followup, which York at Buffalo, Buffalo, New York.
    [Show full text]
  • Adaptive Platform Trials
    Corrected: Author Correction PERSPECTIVES For example, APTs require considerable OPINION pretrial evaluation through simulation to assess the consequences of patient selection Adaptive platform trials: definition, and stratification, organization of study arms, within- trial adaptations, overarching statistical modelling and miscellaneous design, conduct and reporting issues such as modelling for drift in the standard of care used as a control over time. considerations In addition, once APTs are operational, transparent reporting of APT results requires The Adaptive Platform Trials Coalition accommodation for the fact that estimates of efficacy are typically derived from a model Abstract | Researchers, clinicians, policymakers and patients are increasingly that uses information from parts of the interested in questions about therapeutic interventions that are difficult or costly APT that are ongoing, and may be blinded. to answer with traditional, free- standing, parallel- group randomized controlled As several groups are launching APTs, trials (RCTs). Examples include scenarios in which there is a desire to compare the Adaptive Platform Trials Coalition was multiple interventions, to generate separate effect estimates across subgroups of formed to generate standardized definitions, share best practices, discuss common design patients with distinct but related conditions or clinical features, or to minimize features and address oversight and reporting. downtime between trials. In response, researchers have proposed new RCT designs This paper is based on the findings from such as adaptive platform trials (APTs), which are able to study multiple the first meeting of this coalition, held in interventions in a disease or condition in a perpetual manner, with interventions Boston, Massachusetts in May 2017, with entering and leaving the platform on the basis of a predefined decision algorithm.
    [Show full text]
  • Common Types of Questions: Types of Study Designs
    Evidence-Based Medicine Cochrane Database of Systematic Reviews Resources ACP Journal Club Meta- is as PubMed (MEDLINE) – “AND” meta-analysis into your search. This will pick up Meta-Analys analysis s a MeSH term and as a publication type. Cochrane Database of Systematic Reviews / ACP Journal Club Systematic review PubMed (MEDLINE) – Use the ”Find Systematic Reviews” filter under Clinical Queries, or “AND” systematic[sb] into your search. (Note: the systematic reviews subset includes meta- analyses.) tting | Reduction of bia ACP Journal Club / Cochrane Central Register of Controlled Trials (CCTR) Randomized PubMed (MEDLINE) – Use the “Therapy” filter under Clinical Queries, or “AND” controlled trial randomized controlled trial into your search. This will pick up Randomized Controlled Trials as a MeSH term and Randomized Controlled Trial as a publication type. PubMed (MEDLINE) – “AND” cohort study into your search. This will pick up Relevance to clinical se Cohort study the MeSH term Cohort Studies and the more specific MeSH terms Follow Up Studies, Longitudinal Studies, and Prospective Studies under it. PubMed (MEDLINE) – “AND” case control study into your search. This will Case control study pick up the MeSH term Case-Control Studies and the more specific MeSH term Retrospective Studies under it. Case Series / Case Report PubMed (MEDLINE) – “AND” case report into your search. This will pick up Case Reports as a publication type. COMMON TYPES OF QUESTIONS: TYPES OF STUDY DESIGNS: Therapy -- how to select treatments to offer patients that do more good A Meta-analysis takes a systematic review one step further by than harm and that are worth the efforts and costs of using them combining all the results using accepted statistical methodology.
    [Show full text]
  • Efficient Adaptive Designs for Clinical Trials of Interventions for COVID-19
    Efficient adaptive designs for clinical trials of interventions for COVID-19 Nigel Stallarda1, Lisa Hampsona*2, Norbert Benda3, Werner Brannath4, Tom Burnett5, Tim Friede6, Peter K. Kimani1, Franz Koenig7, Johannes Krisam8, Pavel Mozgunov5, Martin Posch7, James Wason9,10, Gernot Wassmer11, John Whitehead5, S. Faye Williamson5, Sarah Zohar12, Thomas Jakia5,10 1 Statistics and Epidemiology, Division of Health Sciences, Warwick Medical School, University of Warwick, UK. 2 Advanced Methodology and Data Science, Novartis Pharma AG, Basel, Switzerland 3 The Federal Institute for Drugs and Medical Devices (BfArM), Bonn, Germany. 4 Institute for Statistics, University of Bremen, Bremen, Germany 5 Department of Mathematics and Statistics, Lancaster University, UK. 6 Department of Medical Statistics, University Medical Center Göttingen, Germany. 7 Section of Medical Statistics, CeMSIIS, Medical University of Vienna, Austria. 8 Institute of Medical Biometry and Informatics, University of Heidelberg, Germany. 9 Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK. 10 MRC Biostatistics Unit, University of Cambridge, Cambridge, UK. 11 RPACT GbR, Sereetz, Germany 12 INSERM, Centre de Recherche des Cordeliers, Sorbonne Université, Université de Paris, France * Corresponding author. [email protected] a These authors contributed equally Abstract The COVID-19 pandemic has led to an unprecedented response in terms of clinical research activity. An important part of this research has been focused on randomized controlled clinical trials to evaluate potential therapies for COVID-19. The results from this research need to be obtained as rapidly as possible. This presents a number of challenges associated with considerable uncertainty over the natural history of the disease and the number and characteristics of patients affected, and the emergence of new potential therapies.
    [Show full text]
  • Assessing the Accuracy of Diagnostic Tests
    Shanghai Archives of Psychiatry: first published as 10.11919/j.issn.1002-0829.218052 on 25 July 2018. Downloaded from Shanghai Archives of Psychiatry, 2018, Vol. 30, No. 3 • 207 • •BIOSTATISTICS IN PSYCHIATRY (45)• Assessing the Accuracy of Diagnostic Tests Fangyu LI, Hua HE* Summary: Gold standard tests are usually used for diagnosis of a disease, but the gold standard tests may not be always available or cannot be administrated due to many reasons such as cost, availability, ethical issues etc. In such cases, some instruments or screening tools can be used to diagnose the disease. However, before the screening tools can be applied, it is crucial to evaluate the accuracy of these screening tools compared to the gold standard tests. In this assay, we will discuss how to assess the accuracy of a diagnostic test through an example using R program. Key words: AUC; gold standard test; ROC analysis; ROC curve; sensitivity; specificity [Shanghai Arch Psychiatry. 2018; 30(3): 207-212. doi: http://dx.doi.org/10.11919/j.issn.1002-0829.218052] 1. Introduction time-consuming. Because of the constraints of SCID, copyright. Mental disease is a significant cause of worldwide some easily administered screening tools, such as the morbidity, second to cardiovascular disease, based Hamilton Depression Rating Scale (HAM-D), the Beck on the estimates from the Global Burden of Disease Depression Inventory (BDI), and even simpler screening Study.[1] Among mental diseases, depression is now the tools such as the Patient Health Questionnaire (PHQ- 2, PHQ-9) were developed and admitted to patients leading cause of the global disability burden.
    [Show full text]
  • Evaluation of Diagnostic Tests When There Is No Gold Standard Vol
    Health Technology Assessment Health Technology Health Technology Assessment 2007; Vol. 11: No. 50 2007; 11: No. 50 Vol. Evaluation of diagnostic tests when there is no gold standard Evaluation of diagnostic tests when there is no gold standard. A review of methods AWS Rutjes, JB Reitsma, A Coomarasamy, KS Khan and PMM Bossuyt Feedback The HTA Programme and the authors would like to know your views about this report. The Correspondence Page on the HTA website (http://www.hta.ac.uk) is a convenient way to publish your comments. If you prefer, you can send your comments to the address below, telling us whether you would like us to transfer them to the website. We look forward to hearing from you. December 2007 The National Coordinating Centre for Health Technology Assessment, Mailpoint 728, Boldrewood, Health Technology Assessment University of Southampton, NHS R&D HTA Programme Southampton, SO16 7PX, UK. HTA Fax: +44 (0) 23 8059 5639 Email: [email protected] www.hta.ac.uk http://www.hta.ac.uk ISSN 1366-5278 HTA How to obtain copies of this and other HTA Programme reports. An electronic version of this publication, in Adobe Acrobat format, is available for downloading free of charge for personal use from the HTA website (http://www.hta.ac.uk). A fully searchable CD-ROM is also available (see below). Printed copies of HTA monographs cost £20 each (post and packing free in the UK) to both public and private sector purchasers from our Despatch Agents. Non-UK purchasers will have to pay a small fee for post and packing.
    [Show full text]