NEON Science Data Quality Plan Date: 08/23/2018
Total Page:16
File Type:pdf, Size:1020Kb
Title: NEON Science Data Quality Plan Date: 08/23/2018 NEON Doc. #: NEON.DOC.004104 Author: C. Sturtevant et al. Revision: A NEON SCIENCE DATA QUALITY PLAN PREPARED BY ORGANIZATION DATE Cove Sturtevant TIS 01/30/2018 Cody Flagg TOS 01/30/2018 Nathan Leisso AOP 01/30/2018 Sarah Elmendorf DPS 01/30/2018 Josh Roberti TIS 01/30/2018 David Garcia SCI 10/30/2017 Claire Lunch DPS 01/30/2018 Elizabeth Knapp EDU 01/30/2018 Nora Catolico AQU 01/30/2018 Tanya Chesney DPS 01/30/2018 Kate Thibault SCI 08/22/2018 APPROVALS ORGANIZATION APPROVAL DATE Mike Stewart SYS 08/09/2018 Laura Leyba-Newton ENG 08/10/2018 RELEASED BY ORGANIZATION RELEASE DATE Judy Salazar CM 08/23/2018 See configuration management system for approval history. The National Ecological Observatory Network is a project solely funded by the National Science Foundation and managed under cooperative agreement by Battelle. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Template_NEON.DOC.004243 Rev H Title: NEON Science Data Quality Plan Date: 08/23/2018 NEON Doc. #: NEON.DOC.004104 Author: C. Sturtevant et al. Revision: A Change Record REVISION DATE ECO # DESCRIPTION OF CHANGE A 08/23/2018 ECO-05580 Initial release Template_NEON.DOC.004243 Rev H TABLE OF CONTENTS 1 DESCRIPTION ..............................................................................................................................1 1.1 Purpose ............................................................................................................................................ 1 1.2 Scope ................................................................................................................................................ 1 2 RELATED DOCUMENTS AND ACRONYMS .....................................................................................2 2.1 Applicable Documents ..................................................................................................................... 2 2.2 Reference Documents...................................................................................................................... 2 2.3 External References ......................................................................................................................... 3 2.4 Acronyms ......................................................................................................................................... 3 2.5 Terminology ..................................................................................................................................... 4 3 INTRODUCTION TO THE QA/QC FRAMEWORKS AND ASSURANCE PLANS .....................................4 4 TERRESTRIAL AND AQUATIC OBSERVATIONAL SYSTEM (OS) QUALITY FRAMEWORK .....................5 4.1 Training for TOS and AOS ................................................................................................................. 6 4.1.1 OS Training Materials .............................................................................................................. 6 4.1.2 Training for OS ......................................................................................................................... 7 4.1.3 Documentation of Completion of Training ............................................................................. 8 4.2 Field Data Planning and Tracking ..................................................................................................... 8 4.2.1 Field Collection Planning ......................................................................................................... 8 4.2.2 Tracking and Reporting of Sample Collection Bouts ............................................................... 9 4.2.3 OS Issue Tracking and Resolution ........................................................................................... 9 4.3 Field Data Entry Validation and Review ........................................................................................... 9 4.3.1 Field Data Collection ............................................................................................................... 9 4.3.2 Transcription and Storage of Field and Lab Data Recorded with Paper Datasheets .............. 9 4.3.3 Data Entered Directly into Data Entry Applications .............................................................. 10 4.3.4 Data Review with the Fulcrum Data QC Application............................................................. 11 4.4 External Laboratory Quality ........................................................................................................... 11 4.4.1 External Laboratory Contracts and Audits ............................................................................ 11 4.4.2 Sample Tracking and External Laboratory Data Validation ................................................... 12 4.5 Post-Collection Data Validation with the OS Parser (LO Data) ...................................................... 12 4.6 Data Ingest Validation .................................................................................................................... 13 4.6.1 OS Parser and Data Quality Control Ingest Workbook (DIWB) ............................................. 13 i 4.7 OS Data Transitioning .................................................................................................................... 14 4.7.1 Assigning Taxonomic information ......................................................................................... 15 4.7.2 Fuzzing Taxonomy for RT&E .................................................................................................. 15 4.7.3 Testing and Verification of Transition ................................................................................... 15 4.8 Data Publication ............................................................................................................................. 16 4.8.1 OS Publication Workbook ..................................................................................................... 16 4.8.2 Publication of data to the NEON Portal ................................................................................ 16 4.9 OS L0 Data Editor ........................................................................................................................... 16 4.9.1 The Editor .............................................................................................................................. 17 4.9.2 The Deleter ............................................................................................................................ 17 4.9.3 The Sample Editor ................................................................................................................. 17 4.9.4 Post-Edit Transitions ............................................................................................................. 17 4.10 OS Quality Assurance Improvements under Development and Future Considerations ........... 18 4.10.1 OS Training ........................................................................................................................ 18 4.10.2 Planning and Tracking of Field Collections ....................................................................... 18 4.10.3 External Laboratory Data Tracking ................................................................................... 19 4.10.4 OS Data Versioning and Revision ...................................................................................... 19 4.10.5 Manual Flagging of Data ................................................................................................... 19 5 AQUATIC AND TERRESTRIAL INSTRUMENT SYSTEMS (IS) QUALITY FRAMEWORK ........................ 21 5.1 Training for IS ................................................................................................................................. 21 5.2 Sensor Preventative Maintenance ................................................................................................. 22 5.3 Sensor Calibration Program ........................................................................................................... 23 5.4 Data Transitioning .......................................................................................................................... 23 5.4.1 Algorithm Quality Assurance................................................................................................. 24 5.4.2 Automated Data Quality Control .......................................................................................... 27 5.4.3 Uncertainty Estimation ......................................................................................................... 28 5.5 Data Publication ............................................................................................................................. 29 5.5.1 Preparation for Publication ................................................................................................... 29 5.5.2 Publication of L1 Test Data to CERT Portal............................................................................ 30 5.5.3 Publication of L1 Test Data to PROD NEON Data Portal ....................................................... 30 5.6 Sensor Problem Tracking and Resolution ...................................................................................... 31 5.6.1 Sensor Health Status Warnings and Errors