Science Based Human Reliability Analysis: Using Digital Nuclear Power Plant Simulators for Human Reliability Research
Total Page:16
File Type:pdf, Size:1020Kb
Science Based Human Reliability Analysis: Using Digital Nuclear Power Plant Simulators for Human Reliability Research DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Rachel Elizabeth Shirley Graduate Program in Nuclear Engineering The Ohio State University 2017 Dissertation Committee: Dr. Carol Smidts, Advisor Dr. Tunc Aldemir Dr. Ronald Boring Dr. Catherine Calder Copyrighted by Rachel Elizabeth Shirley 2017 Abstract Nuclear power plant (NPP) simulators are proliferating in academic research institutions and national laboratories in response to the availability of affordable, digital simulator platforms. Accompanying the new research facilities is a renewed interest in using data collected in NPP simulators for Human Reliability Analysis (HRA) research. An experiment conducted in The Ohio State University (OSU) NPP Simulator Facility develops data collection methods and analytical tools to improve use of simulator data in HRA. In the pilot experiment, student operators respond to design basis accidents in the OSU NPP Simulator Facility. Thirty-three undergraduate and graduate engineering students participated in the research. Following each accident scenario, student operators completed a survey about perceived simulator biases and watched a video of the scenario. During the video, they periodically recorded their perceived strength of significant Performance Shaping Factors (PSFs) such as Stress. This dissertation reviews three aspects of simulator-based research using the data collected in the OSU NPP Simulator Facility: First, a qualitative comparison of student operator performance to computer simulations of expected operator performance generated by the Information Decision Action Crew ii (IDAC) HRA method. Areas of comparison include procedure steps, timing of operator actions, and PSFs. Second, development of a quantitative model of the simulator bias introduced by the simulator environment. Two types of bias are defined: Environmental Bias and Motivational Bias. This research examines Motivational Bias— that is, the effect of the simulator environment on an operator’s motivations, goals, and priorities. A bias causal map is introduced to model motivational bias interactions in the OSU experiment. Data collected in the OSU NPP Simulator Facility are analyzed using Structural Equation Modeling (SEM). Data include crew characteristics, operator surveys, and time to recognize and diagnose the accident in the scenario. These models estimate how the effects of the scenario conditions are mediated by simulator bias, and demonstrate how to quantify the strength of the simulator bias. Third, development of a quantitative model of subjective PSFs based on objective data (plant parameters, alarms, etc.) and PSF values reported by student operators. The objective PSF model is based on the PSF network in the IDAC HRA method. The final model is a mixed effects Bayesian hierarchical linear regression model. The subjective PSF model includes three factors: The Environmental PSF, the simulator Bias, and the Context. The Environmental Bias is mediated by an operator sensitivity coefficient that captures the variation in operator reactions to plant conditions. The data collected in the pilot experiments are not expected to reflect professional NPP operator performance, because the students are still novice operators. However, the iii models used in this research and the methods developed to analyze them demonstrate how to consider simulator bias in experiment design and how to use simulator data to enhance the technical basis of a complex HRA method. The contributions of the research include a framework for discussing simulator bias, a quantitative method for estimating simulator bias, a method for obtaining operator- reported PSF values, and a quantitative method for incorporating the variability in operator perception into PSF models. The research demonstrates applications of Structural Equation Modeling and hierarchical Bayesian linear regression models in HRA. Finally, the research demonstrates the benefits of using student operators as a test platform for HRA research. iv Acknowledgements This research was funded by the Department of Energy’s Nuclear Engineering University Program (NEUP) Graduate Fellowship, the OSU Distinguished University Fellowship, and the US Nuclear Regulatory Commission Graduate Student Fellowship. The initial funds for the OSU NPP Simulator Facility were provided by Battelle Energy Alliance. This research is only possible because of the students who participated in our experiments. I want to thank you for your enthusiasm, your dedication, your willingness to try something new. Working with you was a wonderful part of my experience at OSU. To my advisor, Dr. Carol Smidts. You taught me how to keep pushing past dead ends and to recognize small but valuable insights. Thank you for always asking for more, and for sharing the vision and intuition that has guided my work, and for your encouragement and patience. Dr. Yuandan Li, thank you for your support in this research. Dr. Ron Boring, thank you for your levity and encouragement, and for teaching me about HRA. Dr. Yunfei Zhao, I am so grateful you joined this project near the very end; I’m not sure we would have made it this far without you. I also want to thank my labmates, Mike Pietrykowski, Qingti Guo, Atul Gupta, Chetan Mutha, Jatin Gupta, Matt Gerber and Meng Li. And finally, my husband and my parents: thank you for going on this journey with me, all the way to the end. v Vita June 2001 .......................................................Shaker Heights High School May 2005 .......................................................B.A. Physics and English, Case Western Reserve University 2005-2006 ......................................................English as a Foreign Language Instructor, Hubei Polytechnic Institute May 2008 .......................................................M.A. English, Case Western Reserve University 2008-2011 ......................................................Knowledge Management Specialist, NASA Safety Center/ARES Corporation 2011-Present ..................................................Graduate Research Fellow, Department of Nuclear Engineering, The Ohio State University vi Publications Shirley, Rachel Benish, Carol Smidts, Meng Li and Atul Gupta. "Validating THERP: Assessing the scope of a full-scale validation of the Technique for Human Error Rate Prediction." Annals of Nuclear Energy, v77-6 (2015) Fields of Study Major Field: Nuclear Engineering vii Table of Contents Abstract ............................................................................................................................... ii Acknowledgements ............................................................................................................. v Vita ..................................................................................................................................... vi Table of Contents ............................................................................................................. viii Table of Tables ................................................................................................................. xii Table of Figures ................................................................................................................ xv Acronym List ................................................................................................................... xix 1 Introduction ......................................................................................................... 1 1.1 Overview of this Dissertation ............................................................................... 3 1.1.1 Chapter 1, Introduction ................................................................................. 3 1.1.2 Chapter 2, The OSU NPP Simulator Facility ............................................... 3 1.1.3 Chapter 3, Qualitative Comparison of the OSU Student Operators and an IDAC Simulation ........................................................................................................ 4 1.1.4 Chapter 4, A Method for Quantifying Bias in NPP Simulator Experiments 4 1.1.5 Chapter 5, Development of a Bayesian Subjective PSF Model .................... 5 1.1.6 Chapter 6, Summary ..................................................................................... 5 1.2 Background .......................................................................................................... 5 1.2.1 The Great Challenge in Human Reliability Analysis: Reliable Data ........... 6 1.2.2 Integrating Data into HRA Models: A Bayesian Approach ....................... 14 1.2.3 A Test Platform: The Information-Decision-Action Crew (IDAC) HRA Model 18 2 OSU NPP Simulator Experiment Overview ..................................................... 31 2.1 OSU NPP Simulator Facility.............................................................................. 31 2.1.1 OSU Simulator Facility setup ..................................................................... 32 2.1.2 Human Behavior Data Collection ............................................................... 34 viii 2.2 OSU NPP Systems and Operations Course........................................................ 37 2.2.1 Using Students as Research Subjects .......................................................... 38 2.3 Simulator Sessions