Best Practices for Maintaining Quality in Molecular Diagnostics

Gyorgy Abel, MD, PhD Director, Clinical Chemistry Molecular Diagnostics / Immunology Department of Laboratory Medicine Lahey Clinic Medical Center [email protected]

OUTLINE

• Establishing quality in the molecular laboratory • QC, QA, QI – Total Quality Mangement • Practicing continuous QC/QA • Assay improvement/optimization (new methods, technology, etc.) • CAP recommendations/requirements MOLECULAR PATHOLOGY An Interdisciplinary Field in Laboratory Medicine

HEMATOLOGY INFECTIOUS DISEASE

Molecular SOLID Diagnostics TUMORS

GENETIC IDENTITY TESTING TESTING Available Genetic Tests and Laboratories Performing Molecular Genetic Tests in the USA

Data source: GeneTests database / www.genetests.org Practical Applications of Molecular Diagnostics

Detection, quantitation, genotyping of infectious agents Viruses, bacteria, fungi, and parasites

Detection of defective genes and variations in the genome Molecular genetics: Genetic disorders (neuromuscular, endocrine, cardiovascular, etc., diagnosis of existing disease or predisposition) Molecular oncology: Hematological malignancies, and gene defects, expression profiles, etc., related to cancer Pharmacogenetics: Identification of metabolic gene variants (slow, average, fast metabolizers, non-responders) to optimize drug therapy Genomics: Uses genomic information (gene expression and gene pattern) for disease susceptibility, diagnostic classification, prognosis, and optimal therapy

Identification and characterization of individuals Paternity, forensic medicine, pathology, transplantation

Special considerations for molecular diagnostics laboratories

• Clinical validity and utility of nucleic acid tests • Providing clinical interpretation of test results • Staffing and technical expertise • Regulatory oversight and best practice considerations

Total Laboratory Testing Process

Assay Preanalytical Analytical Postanalytical Validation

• Clinical • Test ordering • Order • Interpretation and • Informed consent verification • Verificartion analytical • Specimen • Nucleic acid • Resulting to LIS validation isolation collection • Reporting to • Establish • Accessioning, • Calibration EMR and barcode scanning • Controls document • Treatment • Processing • Perform test clinical decision validity • Storage, • QC/QA • Data retention, transportation • Result storage verfication

Quality Management across all phases of testing!

Quality Control vs. Quality Assurance

Quality control emphasizes testing of products to uncover defects and reporting to management who decides whether to allow or deny product release.

Quality assurance attempts to improve and stabilize production and its associated processes to avoid, or at least minimize, issues which led to the defect(s) in the first place.

Quality Assurance and Quality Management

• Establish, verify, maintain performance specifications

• Total testing process, including the pre-analytical, analytical, and post-analytic phases of testing

Regulatory considerations

• CLIA 1988 (42 CFR Part 493): establishes requirements for non-waived testing and the personnel requirements for high- complexity testing • U.S. FDA – regulates MDx tests that qualify as products developed by industry as medical and IVD devices, including test kits, quality-control materials, and analyte-specific reagents (ASRs) (21 CFR Part 809; 21 CFR Part 820 • State requirements – Many states use CLIA requirements to regulate genetic testing laboratories. Certain state programs, e.g., the New York State Clinical Laboratory Evaluation Program (CLEP), have specific requirements for laboratories that test specimens obtained from New York state residents.

Quality Improvement

• Considerations for laboratory personnel • Qualifications and responsibilities for all aspects of laboratory functions that impact quality testing services • Assessment and maintenance of personnel competency

Establishment of performance specifications

• Accuracy • Precision • Analytic sensitivity • Analytic specificity, to include interfering substances • Reportable range of test results • Reference intervals or normal values (if applicable) • For quantitative assays: - Linearity - Dynamic range - Limit or Detection (LOD) - Lowest Limit of Quantitation (LLOQ) • For genetic tests: establish clinical usefulness and validity, including clinical sensitivity and specificity, and positive and negative predictive value CLIN. MICRO. REV. 2010; 23: 550–576 If the test is a lab-developed human genetic test:

• Conduct a review of available scientific studies and pertinent references • Define appropriate patient populations for which the test should be performed • Select the appropriate test methodology for the disease or condition being evaluated • Establish analytic performance specifications and determine quality-control procedures using the appropriate number, type, and variety of samples • Ensure that test results can be interpreted for an individual patient or family and that the limitations of the test are well defined and reported

Supplement 69, Current Protocols in Human Genetics CLSI MM01 Molecular Diagnostic Methods for Genetic Diseases and CLSI MM17 Verifica- tion and Validation of Multiplex Nucleic Acid Assays

Specimen selection for test validation

• Evaluate the adequacy of the specimens for the prevalence of the disease and the mutations or variants • Specimen types may include blood, buccal swabs, dried blood spots, fresh or frozen tissue, paraffin- embedded tissue, or prenatal specimens • For a multiplex genetic test, all the mutations or variants to be detected should be included in performance establishment • For rare genotypes/mutations, alternative samples acceptable Documentation of data • Completeness • Consistency • Accuracy • Reconstructability

“All data generated during the conduct of a study, except those that are generated by automated data collection systems, shall be recorded directly, promptly, and legibly in ink. All data entries shall be dated on the date of entry and signed or initialed by the person entering the data. Raw data includes the worksheets, calibration information, records and notes of original observations, and activities of a study that are necessary for the reconstruction and evaluation of the study. May include photographic materials, computer printouts, automated and hand recorded datasheets.” FDA Good Laboratory Practices QUALITY ASSURANCE FOR THE TOTAL TESTING PROCESS Standard Operating Procedure (SOP) Manual

• Procedures for reagent receipt, storage and preparation • Instrument operation and calibration logs • Instrument maintenance and repair logs • Freezer logs • Inventory logs • Methods for taking and recording data • Accession forms • Results and report forms • Many protocols and procedures are regulated: FDA and EPA GLPs,

Provide test information to users

• Intended use of the test • Test method to used • Current Procedural Terminology (CPT) codes as appropriate • Analytical and clinical validity information • Limitations of the test • Whether the test is FDA-cleared • Specimen collection, handling, transport, and submission information • Required patient information, consent if applicable • Availability of laboratory consultations regarding test selection, ordering, specimen submission, results interpretation Test requisition form – by CLIA

• Name and other suitable identifiers of the authorized person requesting the test • Patient name and any other unique identifiers • Indication for testing and relevant clinical information • Patient’s gender and date of birth • Patient racial/ethnic information, if applicable to test methods and result interpretation • Information on patient family history or pedigree, or both, that is pertinent to the disease or condition being evaluated or the testing to be performed • The tests to be performed • Source of the specimen • Date and time of specimen collection • International classification of diseases (ICD code) The analytical testing phase

• CLIA requires laboratories to have procedures in place to monitor and minimize contamination during the testing process and to ensure a unidirectional workflow for amplification procedures that are not contained in closed systems (42 CFR §493.1101) • SEPARATE AREAS FOR: - Reagent preparation - Sample preparation - Amplification area (neg. pressure, if PCR) - Post-PCR area (if amplicons are manipulated) - Use no-template control to detect contamination 

Standard Operating Procedure (SOP) Manual

• Critical element of the QMS process! • Provides all the required elements including the step-by- step procedure/process • Must be available to laboratory personnel for reference • Also used for training and competency assessment • When possible, quality-control samples should be similar to patient specimens in order to monitor the quality of all analytic steps of the testing process.



Standard Operating Procedures

• A brief summary of the assay and its purpose. The agents or genes and/or mutations tested, reference sequences, primer sequences, probe sequences, etc. • Scope • Responsibility • Definitions and acronyms • Policy • Material and equipment • Specimen • Standard safety precautions • Quality-control procedures and control materials

Standard Operating Procedures – cont’d

• Interfering substances • Reagents and consumables • Step-by-step procedures for performing the assay • Guidance on result interpretation • Method limitations • Procedure notes • Appendix: forms • List of related internal documents • References • “Modular” SOP

CLSI guideline Laboratory Documents: Development and Control (GP02-A5; CLSI, 2006)

Importance of Quality Control

• Ensures accuracy and reproducibility of lab test results used in patient care; • Ensures integrity and confidence in test results; • Provides safety in management of patients; • Complies with local, regional, and national regulations and lab accreditation requirements; • Improves staff morale and reputation of laboratory

Quality Control samples

• Control samples should be taken through the (1) extraction phase when appropriate and practical, (2) amplification phase, and (3) detection phase of the assay

• When possible, quality-control samples should be similar to patient specimens in order to monitor the quality of all analytic steps of the testing process.



Quality Control samples

• Commercial • In-house: remainder patient specimens or synthesized • For rare genetic diseases positive controls are hard to obtain • CLSI guideline Verification and Validation of Multiplex Nucleic Acid Assays (MM-17A; CLSI, 2008) • CDC’s Genetic Testing Reference Materials Coordination Program (GeT-RM) • Positive and negative controls should be tested each time patient samples are assayed



CAP Microbiology Checklist

MIC.63277 QC Statistics Phase II

For quantitative assays, quality control statistics are performed monthly to define analytic imprecision and to monitor trends over time.

The laboratory must use statistical methods such as calculating SD and CV monthly to evaluate variance in numeric QC data.

Evidence of Compliance:

• Written procedure for monitoring of analytic imprecision including statistical analysis of data



CAP Microbiology Checklist Qualitative Molecular Assays MIC.64915 Qualitative Cut-Off Phase I For qualitative tests that use a cut-off value to distinguish positive from negative, the cut-off value is established initially, and verified with every change in lot or at least every 6 months. The limit of detection (LoD) that distinguishes a positive from a negative result should be established or verified when the test is initially placed in service, and verified with every change in lot (e.g. new master mix), instrument maintenance, or at least every six months thereafter. Note that a low-positive control that is close to the limit of detection can satisfy this checklist requirement, but must be external to the kit (e.g. weak- positive patient sample or reference material prepared in appropriate matrix).

Commercial Sources of NAT Control Materials

AcroMetrix Corp., Benicia, CA (Life Technologies Inc.) http://www.invitrogen.com/site/us/en/home/brands/Produ ct-Brand/ acrometrix.html?CID=AcrometrixLife SeraCare Life Sciences, Milford, MA Somagen Diagnostics, Inc., Edmonton, AB www.somagen.com ZeptoMetrix Corp., Buffalo, NY www.zeptometrix.com



Proficiency testing (PT) and alternative performance assessment

• Materials from the CAP, when available, enroll in PT program • Review PT results, investigate and correct discrepant results • Reevaluate previous patient test results • PT samples should resemble patient specimens when possible, but some are provided as purified DNA • Alternative PT is optimally performed through interlaboratory exchange or using external materials

The post-analytical testing phase

• Molecular diagnostics test reports: by CLIA Language should be understandable by nongeneticist health professionals and other specific users of the test results. The reports should contain: • Patient’s name and identification number or a unique patient identifier and identification number • Name and address of laboratory where the test was performed • Indication for testing • Test performed Specimen source (when appropriate) Test results and (if applicable) units of measurement or interpretation • Result interpretation, recommendation or guidance

Retention of records

CLIA requires laboratories to retain or be able to retrieve copies of original test reports (including final, preliminary, and cor- rected reports) for at least 2 years after the date of reporting (42 CFR §493.1105).

Retention of molecular genetic test results for 25 years, an approximate entire generation, is recommended (CDC, 2009). QUALITY MANAGEMENT AND IMPROVEMENT

Quality Systems Assessment

An ongoing review process encompassing all facets of the laboratory’s technical and nontechnical functions

Laboratory directors must show evidence that they actively participate in this process.

QUALITY MANAGEMENT AND IMPROVEMENT

Preanalytical systems quality assessment

• Test request forms • Appropriate specimen collection and handling • Appropriate criteria for rejection of specimens • Informed consent forms QUALITY MANAGEMENT AND IMPROVEMENT

Analytical systems quality assessment

• Procedure manual • Nucleic acid extraction and specimen storage • Laboratory design • Laboratory practices • Controls (positive, negative, amplification, sensitivity; external) • Test validation • Maintenance of equipment • Competency of personnel • Proficiency testing and accreditation QUALITY MANAGEMENT AND IMPROVEMENT

Post-analytical systems quality assessment

• Laboratory test reports • Timeliness of reporting • Correction of errors • Patient confidentiality College of American Pathologists (CAP)

CAP Accreditation Program

• Provides leadership in establishing laboratory testing standards • Proficiency testing materials for most analytes • Inspections • CHECKLISTS: - Lab General - Molecular Pathology - Microbiology - Next Generation Sequencing “Improve constantly and forever the system of production and service to improve quality and productivity, and thus constantly decrease costs.”

(W. Edwards Deming)