Appendices, Draft NUREG/CR-XXXX, "Approaches for Using Traditional
Total Page:16
File Type:pdf, Size:1020Kb
APPENDIX A SUMMARY REPORT OF THE EXTERNAL REVIEW PANEL MEETING ON RELIABILITY MODELING OF DIGITAL SYSTEMS (MAY 23-24, 2007) TABLE OF CONTENTS Page A.1. INTRODUCTION .................................................... A-1 A.2. PRESENTATION OF PRELIMINARY COMMENTS BY EACH PANEL MEMBER ................................................................... A-3 A.3. TRADITIONAL METHODS AND THEIR APPLICATIONS ................ A-13 A.3.1 Defining and Identifying “Traditional” Methods ..................... A-13 A.3.2 Conclusions Stated versus Applications of the Criteria Presented in the Report ............................................................ A-15 A.3.3 Criteria Applied to Methods or to Models/Applications ................ A-15 A.4. COMMENTS ON REVIEW CRITERIA ................................. A-17 A.4.1 General Comments on Review Criteria ............................. A-17 A.4.2 Category 1: Level of Detail of the Model ........................... A-17 A.4.3 Category 2: Identification of Failure Modes of the Components of Digital Systems ............................................................ A-18 A.4.4 Category 3: Software Failures ................................... A-19 A.4.5 Category 4: Modeling of Dependencies ............................ A-20 A.4.5.1 Subcategory 4.1: Communication Networks/Buses . A-21 A.4.5.2 Subcategory 4.3: Support Systems .............. A-21 A.4.5.3 Subcategory 4.4: Sharing Hardware ............. A-22 A.4.5.4 Subcategory 4.5: Interactions of Digital Systems with Other Systems .............................. A-22 A.4.5.5 Subcategory 4.6: Modeling of Fault Tolerance Features .......................................... A-23 A.4.5.6 Subcategory 4.2: Common Cause Failures ........ A-24 A.4.6 Category 7: Probabilistic Data ................................... A-24 A.4.7 Criteria 8.4 and 8.5: Uncertainty ................................. A-25 A.4.8 Category 6: Ease of Integration with a PRA Model ................... A-26 A.4.9 Category 5: Human Errors ...................................... A-27 A.4.10 Criteria 8.1 - 8.3: Documentation and Results ....................... A-27 A.5. CONCLUDING REMARKS........................................... A-28 Attachment A Agenda of External Review Panel Meeting on Selection of Traditional Methods for Reliability Modeling of Digital Systems .................................. A-30 A-i Attachment B List of Documents Sent to Panel Members................................ A-32 Attachment C Expert Panel Meeting Attendees........................................ A-33 Attachment D Biographies of Panel Members ......................................... A-34 Attachment E Written Comments Provided by Reviewer A .............................. A-36 Attachment F Written Comments Provided by Reviewer C ............................... A-46 Attachment G Written Comments Provided by Reviewer D .............................. A-48 Attachment H Order for Addressing Review Criteria Categories/Subcategories............... A-52 A-ii A.1. INTRODUCTION This summary report documents the discussions that took place at an external review panel meeting on traditional methods for modeling digital systems as part of a probabilistic risk assessment (PRA). The meeting was held at Brookhaven National Laboratory (BNL) on May 23 and 24, 2007. A.1.1 Background The U.S. Nuclear Regulatory Commission (NRC) is currently performing research on digital system risk assessment. Their objective is to identify and develop methods, analytical tools, and regulatory guidance to support (1) using risk information on digital systems in nuclear power plant (NPP) licensing decisions, and, (2) including models of digital systems into NPPs’ PRAs. The NRC is exploring, in parallel, both the dynamic and traditional methods of modeling digital- system reliability. For this research, the latter can be thought of as the more well-established and commonly used methods of NPP system reliability modeling (e.g., fault tree modeling). Dynamic methods can be thought of as methods of NPP system-reliability modeling that attempt to explicitly model the coupling between a digital system and the plant’s physical processes. (Note: The distinction between traditional and dynamic methods was further discussed during the meeting, as documented in Section 3.1 of this report.) Under a contract with the NRC, BNL conducted the following work as part of the research on traditional methods of modeling digital system reliability: 1. Identified for further exploration two traditional methods that represent a spectrum of capabilities for modeling and quantitatively assessing the reliability of digital systems. 2. Developed criteria for evaluating reliability models of digital systems, which could eventually provide input to the technical basis for risk-informed decision-making. 3. Reviewed reliability models developed using traditional methods, such as fault tree and Markov methods, against the criteria to assist in determining the capabilities and limitations of the state-of-the-art of traditional reliability models. The findings of this work are documented in a draft letter report (T. L. Chu, G. Martinez-Guridi, M. Yue, and J. Lehner, “Probabilistic Modeling of Digital Systems at Nuclear Power Plants: Traditional Methods Selection,” Brookhaven National Laboratory, Draft Letter Report, April 2007). A.1.2 External Review Panel Meeting Process To more fully involve the technical community in identifying the most promising traditional methods for reliability modeling of digital systems, and in developing criteria for evaluating such reliability models, an external review panel was established to review the findings from the BNL activities. The panel was comprised of six members, all of whom have expertise in modeling and A-1 quantifying digital-system reliability, as well as in PRA. The objectives of the external review panel were to assess the following: 1. The identification of traditional methods and their application. 2. The draft criteria used to review reliability models of digital systems. 3. The limitations of the state-of-the-art in modeling digital systems. The responsibilities of the panel members were to (1) study BNL’s draft letter report before the meeting of the expert panel, and (2) attend the two-day l meeting to satisfy the objectives listed above. Attachment A of this report contains the agenda for the meeting. Panel members received a set of background information before the meeting. In addition setting out the objectives of the meeting and describing the process to be followed during the meeting, the background information contained the BNL draft letter report, and other related papers and reports. The materials sent to the panel members are listed in Attachment B. Furthermore, each member was asked to judge beforehand whether the draft criteria for evaluating reliability models of digital systems were appropriate and to identify any additions, deletions, or modifications to them. The meeting took place at BNL on May 23 and 24, 2007. It was conducted with the help of a facilitator who was responsible for aiding the discussions and keeping them focused. In addition to the panel members and the facilitator, the meeting was attended by the authors of the BNL report and the NRC Project Manager. Their role was to provide answers and clarifications in response to the panel members’ questions. Attachment C of this summary report list the attendees’ names; brief biographies of the experts are given in Attachment D. The remaining sections of this report summarize the discussions of the meeting roughly in chronological order. At the beginning of the meeting, each panel member gave a short presentation of his preliminary thoughts on the BNL report (documented in Section 2 of this report). The next major topic for discussion was the identification of traditional methods and their relevant applications, as summarized in Section 3. Section 4 of this report summarizes the discussions on the proposed criteria, in the same order that their categories were reviewed. The concluding remarks are summarized in Section 5. It should be noted that it was not a goal of the meeting to obtain a consensus among the panel on any particular aspect of the work under review. Rather, the goal was to receive feedback from a broad spectrum of individuals who have significant experience in the subject. Accordingly, while the following sections report any points of general agreement among the panel members, most of the information is in the form of comments from individual panel members. A-2 A.2. PRESENTATION OF PRELIMINARY COMMENTS BY EACH PANEL MEMBER During the morning of May 23, 2007, each member of the external review panel gave a short presentation of his preliminary thoughts on using traditional methods for reliability modeling of digital systems, such as suggestions for alternative methods/applications, and on the draft criteria used for reviewing the digital system reliability models. Points of general agreement among the panel members include the following: • The BNL report contains valuable information about traditional methods. • A substantial amount of probabilistic data of digital components/systems has been generated, but most of it is not publically available. An important and difficult issue is how to obtain this data. • The term “criterion” should be used instead of the term “requirement” in the report due to the regulatory implications of the word “requirement.” The comments by individual members of the panel are presented next, in the order