Journal of Perinatology (2010) 30, 163–169 r 2010 Nature Publishing Group All rights reserved. 0743-8346/10 $32 www.nature.com/jp ORIGINAL ARTICLE Variation among institutional review boards in evaluating the design of a multicenter randomized trial AR Stark1, JE Tyson2 and PL Hibberd3 1Department of Pediatrics, Section of Neonatology, Baylor College of Medicine, Houston, TX, USA; 2Center for Clinical Research and Evidence Based Medicine, University of Texas Health Science Center at Houston Medical School, Houston, TX, USA and 3Center for Global Health Research, Tufts University School of Medicine, Boston, MA, USA appropriate because institutional IRBs reflect local values in Objective: The objective of the study was to examine the variation among assessing potential risks and benefits. However, less desirable institutional review boards (IRBs) in evaluation of the study design of a reasons for variation may be due to the difficulty for local IRBs to multicenter trial. include the expertise needed to review the full range of research Study Design: We assessed the first written response of local IRBs to questions that might be asked or study designs that might be used. each site investigator for a multicenter trial of vitamin A supplementation Variation among IRBs is of particular concern in large in extremely low birth weight (ELBW) infants performed by the National randomized multicenter trials, especially those involving high-risk, Institute of Child Health and Human Development Neonatal Research vulnerable populations in which the assessment of the risk–benefit Network. Each author of this paper independently reviewed and ratio may be most important. To understand the causes categorized IRB concerns as major, minor or none, according to the contributing to variation in evaluating the design of such trials, we predefined criteria. systematically examined the variation in IRB review of a multicenter randomized trial performed in extremely low birth Result: Initially, 9 of 18 IRBs withheld approval because of at least one weight (ELBW) newborns by the National Institute of Child Health major concern. These concerns reflected difficulties in evaluating specific and Human Development Neonatal Research Network.13,14 scientific issues for the design of the trial, including its justification, Although this analysis was completed during the trial, we present enrollment criteria, control and experimental therapies, co-interventions, this information because of the increasing interest in promoting toxicity assessment, outcome monitoring and informed consent. rigorous testing of interventions used in high-risk populations, Conclusion: The difficulty in assessing appropriate trial design for the because of the paucity of data regarding IRB review of trial design, specific hypothesis under investigation resulted in considerable variability and also because recent publications of IRB variation in the review in the evaluation by local IRBs. of observational or minimal risk studies6–11 indicate that the type Journal of Perinatology (2010) 30, 163–169; doi:10.1038/jp.2009.157; of difficulties in the IRB review illustrated by this trial continue published online 1 October 2009 today. Keywords: multicenter clinical trial; institutional review board; study design Methods Introduction Description of the trial evaluated by IRBs At the time the trial was proposed, vitamin A was not widely used in Local institutional review boards (IRBs) vary considerably in their 1–11 preterm infants, although a meta-analysis of several small trials review of the same proposal for a multicenter study. Variation identified an improved survival without chronic lung disease.15 The in the informed consent form has received considerable 8,12 primary hypothesis was that supplementation with vitamin A would attention. However, little information is available regarding increase survival without chronic lung disease among ELBW other differences in the review process and their importance, infants who need early respiratory support.13,14 The eligible infants especially in assessing study design. Some variation may be weighed 401 to 1000 g and required assisted ventilation or Correspondence: Professor AR Stark, Department of Pediatrics, Section of Neonatology, Baylor supplemental oxygen at 24 h of age. Exclusion criteria were major College of Medicine, Texas Children’s Hospital, 6621 Fannin, Mail code WT6-104, Houston, congenital anomaly, congenital nonbacterial infection, terminal TX 77030, USA. illness, vitamin A administration in lipid emulsion or in amounts E-mail: [email protected] Received 31 March 2009; revised 8 August 2009; accepted 13 August 2009; published online 1 exceeding that in standard doses of multivitamins or parental October 2009 refusal of consent. Infants were randomly assigned at 24 to 96 h of Institutional review board variation AR Stark et al 164 age to intervention or control groups after stratification by birth study background and rationale, study design, inclusion or weight and center. Infants in the intervention group received exclusion criteria, interventions, sample size, analysis, data safety 5000 IU (0.1 ml) vitamin A intramuscularly with a 29-gauge monitoring or other issues that required further justification or needle three times weekly for 4 weeks, a dose established by a clarification. We considered concerns about the protocol to be preceding pilot study.15 Infants in the control group received minor if they involved questions about administrative issues or simulated injections. To maintain masking, the research nurse further clarification of the protocol. We also reviewed changes administered treatment with a screen around the bed, used a requested in the consent form. To assess concordance, a third pacifier to minimize crying and placed a band-aid over the actual reviewer (PLH), who was not associated with the network and who or simulated injection site that the research nurse removed at the was masked to the identity of the sites, evaluated the letters and time of the next dose. Measurement of serum vitamin A levels and classified responses using the same criteria. Any differences in the retinol dose-response testing were performed in the first 300 evaluation were resolved through discussion and consensus subjects, and masked investigators performed weekly physical agreement. examinations on all infants to assess potential clinical toxicity. An independent data safety and monitoring committee appointed by Results the National Institute of Child Health and Human Development In all, 7 of the 18 IRBs expressed no concerns about the study reviewed data and adverse events at specified intervals. The final results indicated that the vitamin A regimen tested had no protocol. Initially, nine raised at least one major concern that needed to be addressed by the local investigator before the trial discernible toxicity and decreased the risk of chronic lung disease (relative risk 0.89, 95% confidence interval 0.80 to 0.99, number could be approved and initiated. Two IRBs had only minor needed to treat 14 to 15). concerns, requesting clarification about sample size or drug dosing. All except two IRBs required modifications of the consent Process of protocol review before IRB review form to address major or minor concerns. The protocol was Implementation of the trial followed a standardized process that subsequently approved at all sites without any changes in study involved a much more extensive critique of study design than for design. most studies submitted to IRBs.16 Network investigators evaluated There was complete concordance among the three reviewers the proposal and provided written critique to the study principal regarding classification of the concerns as major, minor or related investigator (PI). A subcommittee was formed to assist the PI, and to the consent form. The major IRB concerns were related to basic additional critique was obtained from the member neonatology scientific issues in the design of clinical trials (see Table 1). divisions and from external reviewers with expertise in trial design, neonatal disease and treatment and vitamin A research. Before Discussion submission to any IRB, the protocol was approved by the entire Despite extensive previous review, IRBs varied substantially in the network steering committee and an external advisory board. concerns expressed about the study. Although we cannot entirely Before implementation, site PIs at the 14 network centers exclude the possibility that lack of clarity persisted despite this submitted the protocol to 18 IRBs responsible for the research in review and contributed to variation in responses, we think that participating hospitals. The protocol included an extensive these concerns may reflect the need for additional expertise to assist discussion of the rationale for the study design related to in evaluating the trial design for the specific intervention, disorder, background information about vitamin A, the results of previous population and outcome under investigation. neonatal studies of vitamin A, the need for an effectiveness trial, the selection of the primary and secondary outcomes, the basis for What distinguishes ‘experimental’ from ‘standard’ therapies? the vitamin A regimen, the assessment of safety and the avoidance In considering whether the trial was justified, three IRBs required of caregiver and assessor bias. The study PI prepared a general the site investigator to specify whether vitamin A supplementation consent form describing the protocol and potential risks and was the standard