Randomized Controlled Trials and 21St Century Epistemology

926

Commentary Journal of the National Comprehensive Cancer Network Randomized Controlled Trials and 21st Century Epistemology

In the 20th century, empiricism, in the form of personal, hands-on experimentation in science and medicine, was paramount. As an example, Barry reports of Oswald Avery, the Rockefeller Institute scientist who established DNA as the genetic material, “For he always did everything, down to washing the glassware, with precision and discipline.”1 In the twenty-first century we are witnessing what I would suggest is a step backward from the principles of empiricism. Direct, personal, clinical, and research Frederic W. Grannis Jr, MD experience in medicine is increasingly considered an impediment rather than a virtue Frederic W. Grannis Jr, MD, is in the search for scientific truth. The methods espoused by the McMaster University Clinical Professor of Thoracic evidence-based medicine (EBM) group and, most recently, in the pages of JAMA Surgery at City of Hope National by Otis Brawley, MD, and the American Cancer Society (ACS), relegate clinical Medical Center in Duarte, CA. He has a long-term clinical and experts to a comment-only role, and delegate collection, grading, and interpretation research interest in management of data, as well as writing of clinical practice guideline manuscripts, to EBM of pleural and pericardial practitioners.2 A key assumption in this process is that EBM experts are the correct effusions in cancer patients group to harvest pertinent evidence, cull the strongest evidence, consolidate and and has authored the chapter on “Fluid Complications” in 13 analyze the data contained in this evidence, and write clinical practice guidelines. editions of Cancer Management: In selection of the strongest evidence, EBM principles place a special cachet on A Multidisciplinary Approach: evidence collected in prospective randomized clinical trials (RCTs) despite the fact Medical, Surgical and Radiation that RCTs sometimes, perhaps even often, contain flaws that weaken the strength Oncology. of that evidence. An example of this can be seen in the CALGB 9334 trial discussed in this issue of JNCCN. Multiple problems can be seen in study design, management, statistical analysis, and interpretation for this RCT, and the article’s authors have been responsible in candidly discussing these weaknesses. The concern, however, is that those gathering evidence and writing guidelines on the subject of malignant pleural effusions at some later date, who will typically not have substantial clinical experience in the treatment of malignant pleural effusions—and may not even be physicians—could incorrectly consider the data in this publication to be stronger evidence than it truly represents. For example, EBM experts without clinical experience may not recognize that the study design was faulty, in that it randomized individuals with trapped lung to a treatment, talc pleurodesis, that is often ineffective in these patients. Additionally, it used percent reexpansion as a criterion for success, when such reexpansion cannot be expected in patients with a trapped lung. Guideline authors without clinical experience might also miss the implications of skewed (> 60%) accrual of patients with lung cancer into the study—that survival in these patients is typically much shorter than in patients with breast cancer, for example. They might also fail to recognize that most patients were treated at institutions with very low volume (20 participating centers enrolled between 1 and 7 cases) when many practitioners were just learning to use indwelling pleural catheters effectively. Furthermore, very low accrual (12%) of projected numbers randomized to bedside talc pleurodesis versus tunneled catheter drainage, and a long unexplained delay in publication of the study further weaken the strength of conclusions drawn from study The ideas and viewpoints data. An important variance in statistical analysis is also seen. The original primary expressed in this editorial are end point, lung reexpansion of 90%, was changed to expansion of 70% or greater those of the author and do not necessarily represent any policy, because only 36% of patients achieved greater than 90% reexpansion. Such post hoc position, or program of NCCN. redefinition of a primary study end point is problematic.

Randomized Controlled Trials

The authors conclude that the combined success rate was higher with tunneled catheter drainage than with pleurodesis (62% vs. 46%; P = .064). This appears to be true when considering a mixture of cases containing both trapped and untrapped lungs, but evidence did not show that indwelling catheter drainage is superior in patients with full lung reexpansion. Furthermore, because the rates of success reported in this study are inferior to those published in prior RCTs and large single-institution retrospective series, selection of the “strongest” RCT data values in mathematical modeling studies might yield misleadingly poor results in terms of effectiveness, cost- effectiveness, and value.3–6 One observation that deserves emphasis is that, although no differences were seen between baseline dyspnea score and lung expansion, patients with better expansion had lower dyspnea scores and quality-of-life measures. This suggests that avoidance of trapped lung through earlier intervention may be important in patients with malignant pleural effusions. In summary, data collected from multiple, low-volume centers with clinicians who are new to the studied technique may be less valid than similar data produced by experienced clinicians in high-volume centers. And yet, because the low-volume evidence is collected in a RCT, EBM will rank the results as superior. The CALGB study discussed illustrates this point and the potential deficiencies of randomized control studies.

References 1. Barry JM. The Great Influenza: The Epic Story of the Deadliest Plague in History. New York, NY: Penguin Group. 2. Brawley O, Byers T, Chen A, et al. New American Cancer Society process for creating trustworthy cancer screening guidelines. JAMA 2011;306:2495–2499. 3. Dresler CM, Olak J, Herndon JE II, et al.; Cooperative Groups Cancer and Leukemia Group B; Eastern Cooperative Oncology Group; North Central Cooperative Oncology Group; Radiation Therapy Oncology Group. Phase III intergroup study of talc poudrage vs talc slurry sclerosis for malignant pleural effusion. Chest 2005;127:909–915. 4. Tremblay A, Michaud G. Single-center experience with 250 tunnelled pleural catheter insertions for malignant pleural effusion. Chest 2006;129:362–368. 5. Steger V, Mika U, Toomes H, et al. Who gains most? A 10-year experience with 611 thoracoscopic talc pleurodeses. Ann Thorac Surg 2007;83:1940–1945. 6. Warren WH, Kalimi R, Khodadadian LM, Kim AW. Management of malignant pleural effusions using the Pleur(x) catheter. Ann Thorac Surg 2008;85:1049–1055.