Randomize Evaluations to Improve Health Care Delivery Amy Finkelstein and Sarah Taubman

INSIGHTS | PERSPECTIVES desired pathways ( 3). The benefit of such dicting enantioselectivities based on simple HEALTH CARE POLICY an approach, which is typical of enzyme molecular descriptors (such as vibrational catalysts, is that it should lead to greater frequencies and dipole moments) that char- overall catalytic activity while retaining se- acterize the reactants ( 7). These data are Randomize lectivity. Moreover, recent advances in our easily obtained, obviating the need to com- understanding of noncovalent interactions, pute all possible transition states for each including π-stacking and cation-π interac- catalyst-substrate combination. Moreover, evaluations tions, appear to have laid the groundwork they showed that classical physical organic for the exploitation of these interactions in techniques can be effectively combined rational catalyst design ( 4). with modern data analysis tools to yield to improve However, harnessing the power of non- insights into the mechanisms of catalyzed covalent interactions for enantioselective reactions and the role of noncovalent inter- health care catalysis has proved difficult. Chief among actions in enantioselectivity. the reasons is the relatively weak, nondi- To demonstrate the power of their ap- rectional nature of these interactions, ne- proach, Milo et al. tackled a particular ex- delivery cessitating the introduction of numerous ample of chiral anion catalysis, in which enantioselectivity is induced by Administrative data and the noncovalent association of experimental designs lead a cationic intermediate with a “To demonstrate the power of their chiral, anionic catalyst (see the the way approach, Milo et al. tackled a figure) ( 8, 9). To understand Downloaded from these reactions, they synthe- By Amy Finkelstein 1, 2, 3* particular example of chiral anion sized and tested a library of cat- and Sarah Taubman 2 alysts exhibiting a broad range catalysis, in which enantioselectivity of enantioselectivities. These he medical profession has long recog- is induced by the noncovalent experiments provided a wealth nized the importance of randomized of data regarding the impact of evaluations; such designs are com- http://science.sciencemag.org/ association of a cationic intermediate steric and electronic factors on monly used to evaluate the safety and with a chiral, anionic catalyst.” enantioselectivity, which was efficacy of medical innovations such then distilled into predictive as drugs and devices. Unfortunately, interactions that must operate in concert mathematical models through multivariate Tinnovations in how health care is delivered to effectively stabilize the desired reaction regressions. These models unveiled subtle (e.g., health insurance structures, interven- pathway ( 3). Rationally designing cata- factors that control the enantioselectivity tions to encourage the use of appropriate lysts that achieve such coordinated effects of these reactions and, ultimately, lead to care, and care coordination approaches) are is fraught with difficulties. Indeed, even the design of better catalysts. rarely evaluated using randomization. We identifying the noncovalent interactions re- By embracing modern data analysis tech- consider barriers to conducting sponsible for selectivity in existing catalytic niques to enhance the more traditional randomized trials in this setting POLICY on April 13, 2021 reactions, which is a prerequisite for ratio- tools of physical organic chemistry, Milo and suggest ways for overcoming nal catalyst design, is often not straightfor- et al. have provided a way to harness the them. Randomized evaluations of fundamen- ward based only on experimental data. power of noncovalent interactions for the tal issues in health care policy and delivery Computational quantum chemistry, in design of enantioselective catalysts. Impor- should be—and can be—closer to the norm which quantum mechanics is used to pre- tantly, their approach is general and should than the exception. dict molecular properties by describing the be applicable to a wide range of catalytic There is particular interest in improving electronic motion, has proved invaluable for reactions. This expands the power of the delivery of health care in the United States, understanding chemical reactions and even simple linear free-energy relations that where the health care sector accounts for designing catalysts ( 5, 6). It is routinely used have long been the workhorse of physical almost one-fifth of the economy. The newly to understand enantioselectivities by pre- organic chemistry, and provides a key step created Patient-Centered Outcomes Re- dicting the structures and energies of the toward a future in which big data can be search Institute is providing an estimated operative transition states, while also quan- used to design small catalysts. ■ $3.5 billion in research grants, and the lat- tifying the impact of noncovalent interac- est round of Center for Medicare and Med- tions on these structures. Unfortunately, for REFERENCES icaid Innovation Health Care Innovation many catalytic reactions, there are simply 1. A. Milo, A. J. Neel, F. D. Toste, M. S. Sigman, Science 347, Awards provides about $1 billion in research 737 (2015). too many potential transition-state struc- 2. E. H. Krenske, K. N. Houk, Acc. Chem. Res. 46, 979 (2013). grants—much of it aimed at improving the tures (possibly hundreds) for such analyses 3. R. R. Knowles, E. N. Jacobsen, Proc. Natl. Acad. Sci. U.S.A. delivery of U.S. health care. to be practical. For example, in noncovalent 107, 20678 (2010). Studies of U.S. health care delivery typi- catalysis, the catalyst and substrate can in- 4. S. E. Wheeler, J. W. G. Bloom, J. Phys. Chem. A 118, 6133 cally rely on a range of observational and (2014). teract in a myriad of ways, and many such 5. K. N. Houk, P. H.-Y. Cheong, Nature 455, 309 (2008). quasi-experimental methods. These can be transformations are not amenable to com- 6. K. N. Houk, P. Liu, Daedalus 143, 49 (2014). extremely valuable for learning as much as putational study with this direct approach. 7. A. Milo, E. N. Bess, M. S. Sigman, Nature 507, 210 (2014). possible from existing historical data and for Milo et al. have effectively circumvented 8. M. Mahlau, B. List, Angew. Chem. Int. Ed. 52, 518 (2013). studying questions that are not amenable to 9. A. J. Neel, J. P. Hehn, P. F. Tripet, F. D. Toste, J. Am. Chem. this problem by providing a means of pre- Soc. 135, 14044 (2013). randomized designs. For prospective evaluation of new interventions, however, it is often Department of Chemistry, Texas A&M University, College possible to use a randomized design without Station, TX 77842, USA. E-mail: [email protected] 10.1126/science.aaa5624 adding substantially to the cost or difficulty 720 13 FEBRUARY 2015 • VOL 347 ISSUE 6223 sciencemag.org SCIENCE Published by AAAS of the study. When feasible, randomized DATA AND DESIGN. To understand why ran- A corollary of this labor-intensive approach designs have an unparalleled ability to pro- domized trials in U.S. health care delivery is that randomized evaluations frequently vide credible evidence on an intervention’s have been rare, we turn to some of the chal- focus on very specific patient populations. impact. This can be seen in the outsized lenges in conducting such studies. We then Of the 31 randomized health care delivery and enduring influence of the 1970s RAND propose practical approaches to managing studies from top medical journals included Health Insurance Experiment, a randomized these challenges. in the table, 77% were convenience samples evaluation of the impact of health insurance We begin with potential ethical consider- (for example, patients at a single hospital). in the United States ( 1, 2). More recently, the ations. For medical innovations, randomized This raises important concerns about their attention paid to the 2008 Oregon Health In- trials are considered essential in determin- generalizability. surance Experiment (OHIE), a randomized ing both safety and efficacy. In health care This expensive, time-consuming, and evaluation of the impact of Medicaid ( 3– 6), delivery, safety concerns tend to be less convenience-sample approach may be nec- underscores the continued power and influ- strong. However, there is often equipoise essary in most medical trials, where there ence of such randomized evaluations in both regarding effectiveness. Moreover, it is com- are often real risks to participants. However, the academy and public discourse. mon in health care delivery for promising in most health care delivery interventions, To explore how commonly randomization programs to reach only a small fraction of there is usually only minimal risk of harm to is used in health care delivery studies, we the individuals who might benefit. Where participants. As a result, an alternative ap- examined papers published in a limited set of top journals in medicine, economics, and health services between 2009 and 2013 [see ( 7) for details on data and methods]. We included papers designed to study causal Downloaded from effects of an intervention (using either randomized or other methods). We focused on a handful of top journals to capture an illus- trative set of high-profile studies; the pic- ture may be different across all published (and unpublished) studies. We did, how- http://science.sciencemag.org/ ever, observe similar patterns in reviews of trials registered with clinicaltrials.gov and of reports from major contract research organizations ( 7). On average, 18% of studies of U.S. health care delivery interventions used randomization (see the table). By comparison, 79% of studies of U.S. medical interventions were randomized (P-value for comparison < 0.001). Medical studies involving drugs were on April 13, 2021 very likely to be randomized (86%), but randomization was also common in nondrug medical studies (66%). Of course, regulatory and funding environ- there are capacity constraints, random as- proach to randomization can produce valid ments in medicine are quite different from signment can be the most equitable way to causal estimates at substantially reduced those in the social sciences. However, we allocate limited slots. Indeed, the random cost.

Load more