Policy for Scientific Review of Clinical Protocols Utilizing the NIH Intramural Program

Submission Requirements and Review Criteria

PURPOSE

The purpose of this policy is to describe the minimum requirements for NIH Institutes and Centers (IC) to integrate policies related to clinical protocols into their intramural scientific review.

DEFINITIONS AND SCOPE

“Clinical protocol” means a document outlining human subjects studies requiring IRB approval. Clinical protocols may include clinical trials1, non-interventional observational (natural history) studies, screening protocols, repository protocols and teaching and training protocols. The scientific review process includes the initial protocol review, annual and quadrennial review of the ongoing protocol, and review of substantive amendments to the protocol that pose new scientific questions or substantially alter the scientific approach. In addition, the scientific review process includes prioritization of clinical protocols utilizing scarce resources at the NIH Clinical Center (CC) to assure the best science is being supported. These reviews become part of the official protocol record and are made available to the IRB and NIH leadership.

The scientific review process includes assessment of resource requirements for the Clinical Center, as well as anticipated service needs from other ICs. If an IC wants to waive scientific review, the reasons for the waiver must be documented. IC leadership and the NIH Associate Director for Clinical Research/Chief Scientific Officer CC must sign off on the scientific review before protocols are sent for Institutional Review Board (IRB) review.

Full scientific review is not the purview of NIH IRBs. However, should an IRB have concerns about the quality of the scientific review, or its absence, the Chair should speak with the IC Clinical Protocol Scientific Review Committee Chair (see below) to resolve these concerns prior to IRB review.

The Process of Initial Scientific Review ICs are encouraged to conduct a protocol concept review via initial discussions by principal investigators (PIs) with their laboratory/branch chief, Clinical Director and/or Scientific Director, or central IC concept review committee for ICs that have such a committee. The purpose of concept review is to address feasibility, fit within the mission of the IC’s intramural division, availability of lab/branch resources and additional resources outside the lab/branch, and requirements for a CRADA or clinical trials agreement (CTA). Concept reviews can consider and modify objectives, design, eligibility, and statistical analysis. The concept review is not required in the official protocol record although notation of whether it occurred will be recorded on the NIH Scientific Review of Clinical Protocols form in the Integrated Research Information System (iRIS).

1 The NIH definition of a clinical trial is “A research study in which one or more human subjects are prospectively assigned to one or more interventions (which may include placebo or other control) to evaluate the effects of those interventions on health-related biomedical or behavioral outcomes.” [https://grants.nih.gov/policy/clinical- trials/definition.htm] ICs will conduct scientific review of complete protocols by a Scientific Review Committee within the office of the Clinical Director or Scientific Director. At an IC’s discretion, the Scientific Review Committee may include members from the protocol’s originating laboratory/branch. Other reviewers, who will not be members of the laboratory/branch from which the protocol originates, will include the chair and co-chair, at least two additional IRP reviewers, and a statistician. To avoid conflict of interest for protocols with the Lab or Branch chief, Clinical Director, Scientific Director or Institute Director as the principal investigator, a central IC office, or that of another IC, should perform the review. The Scientific Review Committee Chair may use ad hoc representatives from CC departments or consult services on the review committee. ICs may use external subject matter experts to enhance the scientific review, or the IC’s extramural staff with subject matter expertise . IC’s have discretion in the composition of the Scientific Review Committee as long as the appropriate expertise is included. If ICs elect to use external reviewers, appropriate clearance of conflict of interest for participation must be obtained using a recommended short certification form.

Scientific reviews should be completed as expeditiously as possible, including initial review, response to stipulations and suggestions, and final approval. There should be an attempt to achieve consensus in all decisions. If the protocol is returned to the PI with stipulations, at the Chair’s discretion, the PI response can receive expedited review by the Chair or the committee could be reconvened for further discussion.

ICs will provide a summary rating of the scientific review for each clinical protocol using verbal descriptors (Outstanding, Excellent, Good, Fair, Poor) or a numerical score (0=Poor, 10=Outstanding) (see Appendix 1- Guidelines for Reviewer Comments for Initial Scientific Review of Protocol).

Records should be kept of the outcome of the scientific review process. For each IC’s intramural program documentation of what protocols were put forward, whether there was a concept review, whether the protocol went forward to a full protocol review, what were the results of the review (approved with or without stipulations, or not approved) and was the protocol re-reviewed after stipulations were met? The record, with dates for all the steps of the scientific review process, will be recorded by ICs on the standard NIH Scientific Review of Clinical Protocols form in the Integrated Research Information System (iRIS).

Required Materials for Initial Scientific Review

1. Protocol Elements

An NIH intramural clinical protocol must contain the following information to provide evidence that the investigator(s) has planned a feasible and scientifically excellent clinical study. General elements for all protocols include: • Official protocol title. • Study Population: A description of the study population, including the sample size, gender, age, demographic group, required health status, and geographic location. • Where study subjects will be seen. • If a multi-site study: o Clinical sites participating. o Describe governance of protocol and the coordinating center if one is planned. • Recruitment and plans: o Availability of potential participants, in compliance with NIH policy, as appropriate, for women and minorities and inclusion across the lifespan. o Ability of participating sites to recruit and retain the proposed target number of participants. 2 o Recruitment milestones over the duration of the study. • Resource availability: o For protocols with significant impact on the CC (e.g., ICU, cell processing, surgery) or on IC consult services, individuals from the CC departments and or IC consult services should provide written confirmation of resource availability. o For all protocols being conducted at the CC the Protocol Resource Impact Assessment (PRIA) must be completed and reviewed by the CC prior to Institutional Review Board (IRB) review. • Statistical analysis plan: A statistical justification pertinent to design and power calculations as a statistical analysis plan must be included (see section 7 below “Statistical Analysis Plan”). • Study Duration: Estimated time (in months) from when the study opens to enrollment until: (a) completion of data collection and (b) final data analyses.

For Clinical trials: • Define phase of the clinical trial (phase 0-IV). • Describe the intervention to be tested: Interventions include drugs/small molecules/compounds; biologics; devices; procedures (e.g., surgical techniques); delivery systems (e.g., telemedicine, face- to-face interviews); strategies to change health-related behavior (e.g., diet, cognitive therapy, exercise, development of new habits); treatment strategies; prevention strategies; and diagnostic strategies. • If applicable, describe the dose, frequency, and route of administration of the intervention(s). • Describe methods used to assign participants to study groups (treatment arms) and randomization, if applicable. • Assure compliance with the “NIH Policy and Guidelines on The Inclusion of Women and Minorities as Subjects in Clinical Research and inclusion across the lifespan.” • Describe the procedures to be followed in each arm of the trial, if applicable. • Provide the availability of Investigational Product (IP), IND/IDE status and whether the investigators have had any interactions with the Food and Drug Administration (FDA). • Specify primary and important secondary endpoints: In a Clinical Trial, the primary endpoint is the pre-specified goal(s) or condition(s) that reflects the effect of one or more interventions on human subjects’ biomedical or behavioral status or quality of life. Examples include: positive or negative changes to physiological or biological parameters (e.g., improvement of lung capacity, gene expression); positive or negative changes to psychological or neurodevelopmental parameters (e.g., mood management intervention for smokers; reading comprehension and /or information retention); positive or negative changes to disease processes; positive or negative changes to health-related behaviors; and positive or negative changes to quality of life. • Describe preclinical data and replications and the level of attention to preclinical gender studies in animals or cell lines.

For Observational (Natural History) protocols: • Define the objective of the study and the level of care provided to subjects.

3 2. Clinical Monitoring Plan and Data and Safety Monitoring Plan2

The Clinical Monitoring Plan verifies that the conduct and documentation of the clinical study comports with the Protocol, its Standard Operating Procedures (SOPs), Good Clinical Practice (GCP), and the applicable NIH human subjects research and federal regulatory requirement(s). The Clinical Data Monitoring Plan includes:

• The persons/entity responsible for conducting the monitoring (e.g., Data Coordinating Center, Clinical Research Associate, study monitor from the Coordinating Center). • The type and frequency of planned monitoring activities (e.g., Study Initiation, Interim Visits, Study Close Out), locations where monitoring will occur (e.g., participating clinical sites, data center, coordinating center) and data to be reviewed (e.g. % of subjects to be monitored). • An overall description of the monitoring plan to ensure adherence to the protocol, adequate documentation of the consenting process, and the quality and consistency of the study intervention(s), including fidelity of monitoring for behavioral interventions. • The system to record and manage protocol deviations, unanticipated problems, Serious Adverse Events (SAEs) and noncompliance. • If applicable, describe the monitoring of facilities, such as labs or pharmacies, for handling and storage of Investigational Products and specimens. Describe how Investigational Product(s) accountability and reconciliation are assured during and at the end of the trial per regulatory requirements. • Plans for handling any deficiencies that are uncovered and in cases of serious deficiencies the appropriate reporting to relevant authorities, including but not limited to the IRB, Data Safety Monitoring Board (DSMB) and/or FDA if applicable, institutional officials and the NIH. • Plans for the Data and Safety Monitoring oversight.

The Data and Safety Monitoring Plan (DSMP)2 ensures that validated systems and controls are in place to assure the integrity of the clinical research data being collected for the proposed study:

• Describe methods and systems for data collection (e.g., the research database being used and whether it is 21CFR11 compliant, Case Report Forms (CRFs)), data entry, data verification and data validation. Describe the data monitoring process and frequency and any planned mitigation strategies in the event of noncompliance. • Describe methods and systems to ensure data confidentiality and subject privacy. The NIH Intramural Policy for Data and Safety Monitoring is available at: SOP 17 • Describe the process for locking the final data which refers to how data can be preserved by applying electronic security procedures such as ‘locking’ the data (see Close Out/Step 4 SOP #3 “Final Database Lock and Final Data Delivery”).

______2The NIH Intramural Policy for Data and Safety Monitoring is available at SOP17 (https://ohsr.od.nih.gov/public/SOP_17_v2_3-8-2016_508.pdf)

4 3. Data Access and Sharing Plan Planned data access and sharing need to be defined and meet applicable data sharing and public/open access requirements. Consideration may include but is not limited to (a) the NIH intramural human data sharing policy, (b) ClinicalTrials.gov results reporting requirements, found at 42 CFR 11 and in the 2016 NIH Policy Dissemination of NIH-Funded Clinical Trial Information, (c) the 2014 NIH Genomic Data Sharing (GDS) policy, and (d) journal, partner institution, and/or sponsor data sharing requirements, or other relevant agreement terms.

4. The Milestone Plan Investigators should provide project performance and timeline objectives, especially for interventional studies. For natural history studies this should include a description of anticipated patient accruals. This section must include an overview of the anticipated project timeline for the following general milestones, as applicable:

• Completion of regulatory approvals. • Enrollment of the first subject. • If applicable, enrollment and randomization of 25%, 50%, 75%, and 100% of the projected study population, including women, minorities, children, and the aged (as appropriate). • Define a plan of action if the study fails to meet its accrual milestones. • Completion of data collection time. • Completion of primary endpoint and secondary endpoint data analyses. • If applicable, completion of final study report. • Reporting of results in https://clinicaltrials.gov/ as required.

These milestones must be reviewed and addressed at annual and quadrennial scientific reviews described below.

5. Common Data Elements Applicability NIH encourages the use of common data elements (CDEs) in clinical research, patient registries, and other human subject research to facilitate broader and more effective use of data and advance research across studies. NIH has established a “Common Data Element (CDE) Resource Portal" (http://cde.nih.gov/) to help investigators identify CDEs when developing protocols, case report forms, and other instruments for data collection. Investigators should describe whether CDEs will be used in the proposed study; if CDEs will not be used, the investigator should explain why.

6. Clinical Protocol Schedule of Events The clinical protocol schedule of events provides protocol timelines and procedures to be completed by a participant during the trial (see section 7.37 Schedule of Events Table in the NIH Clinical Trials Protocol Template.

For example: a. Week 1 Screening/Baseline Visit (4 hours) - eligibility criteria, obtain informed consent, screening assessment(s), labs, etc. b. Week 2, 4, 6, 8 Study Visits (3 hours) - intervention(s), assessment(s), labs, scan(s), etc. c. Week 12 and 18 Follow-up Visits (3 hours) - assessment(s), labs, scan(s), etc.

5 d. Week 24 End of study visit (2 hours) - assessment(s), labs, scan(s), etc. e. This document should be provided in a tabular or graphic format.

7. Statistical Analysis Plan

A detailed statistical analysis plan must be submitted (see Section 10 in Statistical Considerations in the NIH Clinical Trials Protocol Template).

Specify the number of subjects to enroll, the expected effect size, power and the statistical methods (per protocol, intent-to-treat) used to assess the primary outcome measure. A more detailed analysis plan, including a valid subgroup analysis, should be included for a Phase III Clinical Trial.

8. Disease Community Engagement and Study Design • When appropriate specify the process for including disease community engagement (research participants ) in study design. For studies with clinical outcome endpoints, specify whether research participant perspectives were included in determining clinical outcome measures (e.g., surveys distributed at patient conferences, direct meetings with patients and support group representatives, patient representation on any study design group). When disease community engagement is used it is necessary to define the NIH process used for engagement (e.g., use of NIH Conflict of Interest short certification form). If no disease community engagement has been used explain why. • Describe the process for timely feedback to research participants of study results and conclusions.

9. Investigator Qualifications Include the curriculum vitae or relevant biography of the Principal Investigator(s). Also include evidence to address the following considerations: Do the PIs, collaborators, and other researchers have appropriate training to participate in the project? If the project is collaborative, do the investigators have complementary and integrated expertise?

Prioritization of Protocols Using Scarce Resources at the NIH Clinical Center

NIH has developed a process to ensure that scarce resources at the Clinical Center are utilized to support the best scientific opportunities. • The process is overseen by a subcommittee of NIH IC Directors and chaired by the NIH Associate Director for Clinical Research/Chief Scientific Officer of the Clinical Center. • Prioritization of protocols is done after the initial Scientific Review is complete. Priority development will balance needs for new protocols with on-going protocols. • When a scarce resource prioritization committee for a particular resource (appointed by the Chief Scientific Officer of the CC) is convened, Clinical Directors whose ICs utilize the scarce Clinical Center resource make the initial prioritization within each IC of protocols that use the scarce resource using a prioritization tool provided. • The Clinical Directors or IC representatives utilizing the scarce resource then meet to harmonize priorities across ICs and establish a prioritized list for use of the scarce resource. • The Clinical Directors or IC representatives utilizing the scarce resource then work with the Clinical Center’s department chief and section chief responsible for the scarce resource to accommodate as many protocols as possible, with attention to patient recruitment goals, recent

6 and projected usage data and assuring that new user ICs and early career investigators get special consideration. • Final priority for on-going studies will also ensure that ethical issues are respected, particularly with regard to the potential termination of trials in which research participants have already been enrolled. • Protocols that receive a priority score from the resource prioritization committee below the resource capacity will be put in a queue for future review and ICs will be encouraged to use off-site resources to support these protocols. • The IC Directors’ subcommittee on protocol prioritization will adjudicate issues that may arise and make recommendations for funding mechanisms to expand (or decrease) as appropriate.

Evaluation of the prioritization process will be done by surveying the ICs after two years to assure highly meritorious protocols have been supported, tracking access to scarce resources by tenure track and other early career investigators, and tracking access by new user ICs.

Ongoing Scientific Merit Reviews

Most clinical protocols remain active for more than a year and some for many years. To assure that the quality of science remains high as the environment of science evolves, it is necessary to have annual and quadrennial merit reviews. Ongoing reviews should be tailored to the nature of the protocol (e.g., clinical trial, natural history, training, etc.).

Annual Merit Review Annual merit reviews evaluate protocol progress, identify problems, assure that the project is on course, and if not, why not and what is being done to get it back on track. These reviews should assess resource commitment, and study progress (e.g., recruitment of subjects, toxicity, and continued importance) and be conducted in concert with the annual human subject’s review when feasible. Annual merit reviews may be lab/branch based, IC-based or by external reviewers. To avoid conflict of interest, a central IC office, or that of another IC should perform review of a lab chief's protocol. Participation by members of the lab/branch and IC is encouraged to foster local engagement in the clinical research enterprise and to increase awareness of possible collaborations among intramural investigators. A written report signed by the Chair of the reviewing entity (who may be the Clinical Director), should summarize each review and become part of the protocol documentation and provided to the IRB for their continuing review of protocols.

Quadrennial Merit Review Quadrennial merit reviews will be detailed, in-depth scientific reviews at least every four years, which consider the scientific justifications for continuation of the protocol. These scientific reviews are in addition to ongoing data monitoring and review of patient recruitment and are analogous to the extramural competitive scientific reviews of ongoing clinical trials and long-term observational studies that occur every 3-5 years. Quadrennial merit reviews are based on principles such as: Is this still the right study to do? Is this the right group to do it? Are the investigators using the best methods? Should it be continued? The quadrennial merit reviewers may include external subject matter experts, appropriately constituted with clinical investigators; clearance of conflict of interest of external consultants must be obtained using a recommended NIH Conflict of Interest short certification form. A written report should summarize each review and be made available to the relevant Board of Scientific Counselors as well as the IRB.

7 General Principles for Annual and Quadrennial Reviews • Scientific timeliness and merit for continuation based on innovation, impact, significance and scientific approach. • Determination of whether the study has reached its endpoints or shown futility. • Is recruitment occurring and appropriate? • Are the methods and approaches still appropriate? • Is the project feasible, achievable and appropriate given the levels of IC and CC resource allocation? • Is this project still aligned with the laboratory/ branch, IC and NIH research goals? • Have discoveries in the field significantly altered the direction or need for this protocol? • For multi-center studies are the plans to add or drop enrollment centers, as needed, appropriate?

Routing of Scientific Reviews

Scientific reviews must be routed through the NIH Integrated Research Information System (iRIS) and become part of the protocol package maintained by the NIH Office of Protocol Services. The reviews are advisory to the IC leadership and the NIH Associate Director for Clinical Research.

Opportunity for Appeal of Scientific Review Outcome

Investigators may appeal the outcome of scientific review. Appeals will be directed to the sponsoring IC's leadership (Clinical Director/Scientific Director). If necessary, subsequent appeal can be made to the NIH Associate Director for Clinical Research.

Original Approval

John I. Gallin, MD Date Francis S. Collins, MD, PhD Date Associate Director for Director Clinical Research/Chief Scientific Officer

Revisions Approval

______John I. Gallin, MD Date Francis S. Collins, MD, PhD Date Associate Director for Director Clinical Research/Chief Scientific Officer

8 Appendix 1– Guidelines for Scientific Reviewer Comments for Initial Scientific Review of Protocols

Each protocol review memo will include a narrative addressing the required elements of protocol’s scientific merit. In addition to the protocol review decision memo, which may address stipulations and other actions of the review committee including Approval, Tabling, or Rejection, institutes need to provide an overall assessment of the scientific merit of the protocol which can assist in the allocation of scarce Institute or Clinical Center resources and can be informative to institute leadership and individual investigators. Guideline criteria for reviewers to use when evaluating the scientific merit of a protocol are listed below.

SCIENTIFIC MERIT ASSESSMENT TOOL

Protocol Title: Principal Investigator: Reviewer (Optional): ____ Scientific Merit/Background and Rationale. The protocol addresses an important problem or critical barrier to progress in the field.

Comment:______

Objectives. Clearly stated specific aims aligned with well-defined endpoints and appropriate study design.

Comment:______

Design. Clearly describes how study objectives will be achieved, methods to recruit patients, acquire data, and strategies to overcome anticipated barriers are defined. Addresses randomization, minimization of bias, patient follow up and masking/blinding (if applicable).

Comment:______

Subject Eligibility Criteria. Specific inclusion/exclusion requirements and stratification factors (if applicable). Study populations (size, gender, age, and demographic group) are well defined and justified.

Comment:______

9 Disease Community Engagement. Process for including disease community engagement (research participants ) in study design and process for timely feedback to subject volunteers of study results and conclusions presented.

Comment:______

Outcome Characteristics and Endpoint Definitions. Clearly defined primary and secondary endpoints/outcomes. Note that for some natural history studies this element may not be appropriate.

Comment:______

Statistical Analysis and Sample Size. Appropriate and adequate study design statistical analysis plan. Prospective analysis plan, including sample size justification to achieve study objectives to minimize missing data. Note that for some natural history studies this element may not be appropriate.

Comment:______

Data monitoring. Practices and procedures to manage data analysis, quality, cleaning and storage well defined. Process for sharing data defined.

Comment:______

Multi-Center studies. Protocol describes capability of conducting study at the proposed sites with an appropriate organizational structure and, for studies with international sites, there is adequate description of the potential complexities for executing the clinical trial in these sites.

Comment:______

Principal Investigator Qualifications. Has the necessary skills, experience, time and resources to ensure that the study can be successfully completed, including identification of personnel to provide statistical computations and statistical expertise.

Comment:______

10 Plan to register protocol and report outcome data in clinicaltrials.gov.

Comment:______

OVERALL ASSESSMENT. A verbal score (Poor, Fair, Good, Excellent, Outstanding) or a numerical score (0 poor-10 outstanding may be used).

Comment:______

RECOMMENDATION: o Return to PI with comments o Forward to IRB for consideration o Forward to IRB with comments

SUMMARY STATEMENT: Summarize below, at the end of the committee discussion, what changes you request or questions you want conveyed to the PI.

Comment:______

11