Overview of Analytical Validation of Donor Screening Tests Krishna (Mohan) V. Ketha, Ph.D. Division of Emerging and Transfusion Transmitted Diseases Office of Blood Research and Review CBER Presentation Outline • Overview of analytical validation – Why, When, and What • General requirements for IVD analytical validation – Analytical sensitivity, specificity, precision, reproducibility and repeatability, interference, etc. • Considerations and review issues for IVDs used for infectious disease screening – Study design, controls, standards 2 Presentation Outline • Overview of analytical validation – Why, When, and What • General requirements for IVD analytical validation – Analytical sensitivity, specificity, precision, reproducibility and repeatability, interference, etc. • Considerations and review issues for IVDs used for infectious disease screening – Study design, controls, standards 3 Why: To demonstrate that the manufactured product meets its prescribed requirements for safety and effectiveness – Preclinical and analytical both are being discussed here – Analytical performance is as critical as clinical performance When: Preferably before the IND Definitely before the final submission! 4 “What” to Perform (1) • Setting the blank • Setting the cut-off • Demonstration of the dynamic range • Setting the calibration curve • Setting the Positive and Negative Controls 5 “What” to Perform (2) • Rationale and demonstration of a re-test algorithm, where applicable • Linearity (quantitative and semi-quantitative assays) • Establishment of gray zone where applicable – To demonstrate true positives/negatives • Sample and matrix suitability, including analyte stability 6 Regulations, Guidance, and Standards • 21 CFR 58 – Good Laboratory Practice for Nonclinical Laboratory Studies. • Guidance for Industry and FDA Staff – Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests. March 2007. • CLSI EP05-A3 – Evaluation of Precision of Quantitative Measurement Procedures; Approved Guideline. Third Edition. October 2014. • CLSI EP06-A – Evaluation of the Linearity of Quantitative Measurement Procedures: A Statistical Approach; Approved Guideline. April 2003. • CLSI EP07-A2 – Interference Testing in Clinical Chemistry; Approved Guideline, Second Edition. November 2005. • CLSI EP09-A3 – Measurement Procedure Comparison and Bias Estimation Using Patient Samples; Approved Guideline, Third Edition. August 2013. • CLSI EP12-A2 – User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline, Second Edition. January 2008. • CLSI EP17-A2 – Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures; Approved Guideline, Second Edition. June 2012. (Replaced CLSI EP17-A2 Protocols for Determination of Limits of Detection and Limits of Quantitation; Approved Guideline. October 2004.) • CLSI EP25-A – Evaluation of Stability of In Vitro Diagnostic Reagents; Approved Guideline. September 2009. • ICH Guideline: Validation of Analytical Procedures: Text and Methodology Q2(R1), November 2005. 7 Presentation Outline • Overview of analytical validation – Why, When, and What • General requirements for IVD analytical validation – Analytical sensitivity, specificity, precision, reproducibility and repeatability, interference, etc. • Considerations and review issues for IVDs used for infectious disease screening – Study design, controls, standards 8 Analytical Validation Establishes Device Performance • Precision • Reproducibility • Analytical sensitivity & specificity • Cross reactivity and interference • Matrix comparison • Measuring range, reference range • Stability studies 9 Precision and Reproducibility • Evaluates how well the assay yields the same result on repeated determinations. • Statistically valid approach to evaluate multiple aliquots, multiple lots at multiple sites, via multiple runs on multiple days, etc. • Intra- and intra-assay variability • Intra- and inter-lot variability • Inter-operator variability • Inter-instrument variability, if needed • Other assay-critical, or system-critical variables (e.g., plate A/plate B when there is a specific order recommended) • Total variability 10 Analytical Sensitivity Definition: “slope of the calibration curve”; capacity of a test method to differentiate between two very close concentrations of an analyte (CLSI EP17-A2) Study design • End-point dilutions • Contrived specimens as needed • > 3 concentrations of the analyte/panels • Multiple replicates Analytical Sensitivity ≠ Limit of Detection (LoD) 11 Limit of Detection (LoD) L(LoD): the lowest concentration of an analyte that can be consistently detected (typically in ≥ 95% of samples tested) (CLSI EP17-A2) Study design • Known-positive or standards/panels • 5 dilutions/panel • At least 20 replicates/panel (include non-reactives) • Statistical analysis: > 95% reactivity 12 Analytical Specificity/Interference Testing/Cross-Reactivity Cause of significant difference in the test result due to the effect of another component or property of the sample (CLSI EP07-A2) • Samples to test for cross-reactivity • Other species/serotypes/genetic variants • Other disease conditions (autoimmune, infections, etc.) • Sample size – variable for different conditions • Interference testing • Endogenous (albumin, bilirubin, hemoglobulin, lipid, IgG) • Exogenous interferents (various drugs/supplements) 13 Matrix Comparison • Matrices claimed: whole blood, plasma, and serum • Lysed/prepared specimen vs neat/diluted • Different anticoagulants • Cadaveric claims • Analytical sensitivity • Analytical specificity • Reproducibility 14 Stability • Samples – Room temperature, refrigerated, or frozen – Neat, pooled, prepared (lysed), on-board • Kit – Calibrators, and controls – Real-time: basis for shelf-life claims – Open kit on-board • Labeling claim – Test one time point beyond the proposed claim for shelf-life – Based on data, not extrapolated or interpolated points – Based on real-time stability, not accelerated studies 15 Presentation Outline • Overview of analytical validation – Why, When, and What • General requirements for IVD analytical validation – Analytical sensitivity, specificity, precision, reproducibility and repeatability, interference, etc. • Considerations and review issues for IVDs used for infectious disease screening – Study design, controls, standards 16 Complex Donor Screening IVDs • Diverse IVDs with different final analytes – Antigen, antibody, nucleic acid • Technology – EIA, chemiluminescence, PCR, transcription-mediated amplification (TMA) • Different limit of detection (LoD) parameters – g/mL, IU/mL, copies/mL, iRBC/mL • Different clinical samples/sample size – Genetic variants, prevalence/risk of transfusion-transmission (TT) 17 Precision and Reproducibility • Panel of 6-10 well-characterized specimens, representing a clinically relevant range: • Minimum of one positive and negative sample near assay cut-off • Assay controls and calibrators • Different group/panel of specimens • Each type of specimen matrix • Each analyte • Genotype (or variant) • Tested using at least three kit lots 18 Analytical Sensitivity (1) • Each specimen group, genotype or strain • Each sample matrix (e.g., serum, EDTA-plasma) • Approaches – End-point dilution – Earliest time of reactivity in serially-collected specimens – Comparison to reference standards – Comparison to an independent method – Quantitative biochemical characterization • Direct comparison to a FDA-licensed, approved, or cleared test • Controls targeted to clinical decision points – Low positive between 1-3 S/CO or 1-3 x LoD • Validation of assay’s gray zone(s) 19 Analytical Sensitivity (2) • Appropriate standards or CBER reference panels e.g., HIV, HCV, HBsAg, Babesia • Seroconversion panels, when available e.g., multiple specimens from at least 10 subjects undergoing seroconversion) • Low-titer panels for each strain/analyte/matrix, if applicable e.g., 6-10 specimens per panel • Dilution series e.g., at least 10 specimens from 10 subjects for each strain/matrix • Known-positives from relevant populations e.g., samples from an HIV-1 high risk group • NAT: confirm sequence identity for strains/genotypes claimed 20 Analytical Sensitivity (3) • FDA prepares and provides panels of samples – Different panels for all analytes – Analytes at various levels • Not to be confused with lot release panels • Number of samples correctly detected is evaluated 21 Seroconversion Panels • Panels collected from plasmapheresis donors who are in the process of seroconverting • Real clinical samples from blood donors with values near the cut-off are rare • Seroconversion panels are real samples with analytes at relevant clinical concentrations 22 Analytical Specificity • Samples to include – Other strains/variants – confirm identity – Other disease/medical conditions (autoimmune, infections) – Potentially interfering substances – Endogenous (albumin, bilirubin, hemoglobulin, IgG, etc.) – Exogenous interferents (various drugs/supplements) – Different anticoagulants/collection tubes • Sample size – variable for different conditions • Labeling claim: include interference/cross-reactivity 23 Device Performance - What to Submit • Summaries of study designs – Materials, procedures, analysis, and oversight – Sample collection, selection criteria, handling, and storage – Statistical and clinical considerations – Documentation that all testing performed at an approved facility using Good Laboratory Practice (GLP) • Summaries of results and line data for all studies –