A Study on the Quantitative Evaluation for the Software Inciuded in Digital Systems of Nuclear Power Plants DISCLAIMER

A study on the quantitative evaluation for the software incIuded in digital systems of nuclear power plants DISCLAIMER

Portions of this document may be illegible in electronic image products. Images are produced from the best available original document. 2002. 3.

Summary

Recently, newly being developed nuclear power plants (NPPs) accept digital instrumentation and control (I&C) systems because the limitations such as mrrectness, maintainability, enhancement of the operational reliability and complexity of conventional analog systems arise. In addition, in the case of currently being operated nuclear power plants, the tendency of adopting digital I&C systems is increasing because it is difficult to prepattdestablish spear parts of installed analog I&C systems. In general, probabilistic safety analysis (PSA) has been used as one of the most important methods to evaluate the safety of NPPs. The PSA, because most of NPPs have been installed and used analog I&C systems, has been performed based on the hardware perspectives. In addition, since the tendency to use digital I&C systems including software instead of analog I&C systems is increasing, the needs of quantitative evaluation methods so as to perform PSA are also increasing. Nevertheless, several reasons such as software did not aged and it is very perplexed to estimate software failure rate due to its non-linearity, make the performance of PSA difficult. In this study, in order to perform PSA including software more eficiently, test-based software reliability estimation methods are reviewed to suggest a preliminary procedure that can provide reasonable guidances to quantifj, software failure rate. In addition, requisite research activities to enhance applicability of the suggested procedure are also discussed.

... 41 1 5) Ad ...... 1

41 2 3 4--PEaqq]cflQ PSA 4-g-A ...... 4

4 3 3 AIpI713 & 44s qq %kg ...... 7 x{] 1 4 'Select inpub' ...... 9 1. *==qq A]gq *zq g *=3qzq 4%...... 9 2 . A] gq+ @% ...... 10 3 . Alga 481 g.Q.Q Alpl qqa (test inpubs9 4%...... 15 41 2 4 'Perfom tests' 341 ...... 22 1. ;r! %b+%&* ...... 22 2 . &sE8J1qx%E 8% ...... 23 41 3 3 'Collect failure data to obtain software reliability' 941 ...... 25 1. Omcleq +lqs g coverage ...... 26 2 . 4- -P test coverage ...... 27

4 4 3 c]XI%!%%! %!XIS =41%4 PSA +qEf-%+lQ *-"E44AlgBA) 9 ++S&4 S3 ...... 29 41 1 @%Is -439 71% ...... 29 41 2 4 qX1l@+%! @XIS YS4]g.n 2EqM &nE4]qq+lqE4+ *.---.31 1. kEEq1qq 34~34. A]%* 4+4 g +3€ ~S)JZS!.) 3%...... 31 2 . A] 4%...... 34 3 . A]% QS?aq8% ...... 34 4 . ?2JqJ-+%&* ...... 35 5 . 4-=qq x%bs3% ...... 35 6. Watchdog A]&@q 44E coverage ...... 35 7 . &mE$Jqq test coverage ...... 36 8 . PSA +q-&+1Sfl &qq& &-ILEq]q oJ1G @A) ...... -37

4) 5 %b ...... 39

xd.-;?. s* ...... 42

P. .i . .. - I1 -

I xg 3-1. &32-#4 A]%& -&e+!qE a%q934 3 wok flow ...... 2% 3-2. qB 43ilE 3- 481 4aQ A]%qL;= a%*&%...... 10 3%3-3. A]qaS)**& xqQ A]%*% ...... - ...... -.12 ! 2%3-4. A] % CqOlq .?r1ap1qq ...... 16 3%3-5. 44 cJQ S++s...... 19 3%3-6. Qfqqgqs+ Aq] CAP,, A]% Q] qa- @q q] ...... 21 3%3-7. Oracle A]&q& AI-9-Q A1 ...... 26 2%3-8. 4sz Eflqs) test covemge 9 input coverage9 tj]a ...... 27 2.a 4-1. 7]e4 @X)Z ...... 29 ZLg 4-2. ClX] sZ)S ‘;1A)-& ...... 30 32 4-3. A]q& 9%3S)E 4]+ *&% ...... - ...... - .... 38

...

! -C. -1 - -2 - -3 - .... -4 - - modeling the multi-tasking of digital systems - modeling the features of phased mission systems - digital system induced initiating events including human errors

* -5 - -6 - (test-based software reliability estimation methods)

ANSI(American National Standards 1nstitute)oft qS$ 4= E aq dq2Z-k "the probability of failure-fiee software operation for a specific period of time or demand in a specified environment" e) 301 348C) [ANSIgl]. 4714A1, hZEqq 3% (failure= cM1) %-& %qS 2-k &*(enor), ag(fau1t) 9 aB(defect)q 481 %+jQc)~ 34 8~) [PfleegerE].

L. -7 - n

I I

-- I I

J 4 1 3 ‘Select inputs’ @a

-9 - Laplace'. mk of number of tests point cstlmation succession

Assuming Binnomhl Independent trial distribution

The t-dSstributSon method

-10- e=- 1 N+2

F 1-c Ncj(i-e)N-jej2 C, ~=0,1,2,... j==

F i-z N+,c,(i-e)N+l-jej2 C, ~=0,1,2,... ]d

-1 1-

ln(1- c) - ln(1- 0) N21+ > In( 1- 0 + p.0) (4 4)

2NAH = X: (1 - C) AH=H,-H k=n-m fi = frequency of the occurrence of the i”’ outcome = -*i N 6 H=-~(’In(fa) (4 5) i=l

-13- 6 Cf, = 1.0 i=l

x:(O.OS) - 11.07 N= -- 2.m 2.m

-14- -i.. -15- psiiirk Random sampling selectlon method Input sampling Bin (urn) sampling

execution coverage -1 7- a is integer, b = a/2; if (b 2 2.5) then @ else @I a is integer, b = a + 1; if (b 2 5) then @ else @

-L. -18- No yes A, B, C, D = deckbn points , '6', affected by input data t + Perform PerfOlIll segment i segment 2

-19- A: integer; 0 < A < 20

.. .. -20- B: Integer; -20 < B < 0

Binl: 2 Bin2: 3 I 1 Bin3: 5

I I I I I I I 1 1 I 0246 000 20 b A

-21-

I -22-

I -L. -23- I

-PZ- -_ -25- 6enerating correct outputs for ghn input data

......

-26- e Bug included in software

2%3-8. AZE4q4 test coverage 9 input coverage4

-27-

I .

-28- Analog/ Digital Comparator Rndog/Diglhl - trfp QDM Input moduk 0 (to check trip outputmodule % (contmlekmant (plant parameters) parameters) - (generating trip rlgnal) driving mechanism)

- Trip Manual trip signal slgnst (from manual trip buttons) 1% 4-1. 7]Sq +k3&

-29- input module -- (to check trip __c outputmodule S (wntmlahnt (plant parametars) parunatan) (generating Mp signal) drMng mchanlm) L

I J

-30-

i -31-

-33- -<. -34- -35- I. -36- -37- ...... ^...... -...... -...... -.-...... -...... -...... -...... -...... -.

Considering 'safety' perspective software reliablli

Considering ccrfK7ation test

Define re/ative target reliability

Assuming independent triak between successive tests Considering conffdence level

Sekcfing inpuk based on 'safety' perspecfive Input cases Sekcfing test inpub based on fhe combination of whife-box approach- L .-...... _...... -...... -...... ^...... __...... -...... -. -. ..-...... -.-...... -..-...... Perform I tests

...... -...... _...... -...... -...... -...... -...... -..... -.-.-.... -...... -.... i

Dennc Assuming ai/ failures from processor i falluro modes of moduk were due to soRware faub processor module Assuming no masking effects

Quantify software reliaQllity

-3 8- -. -39- -40- -29- [Cannon011 R. M. Cannon. Sense and sensitiivty - designing surveys based on an imperfect test. Preventive Veterinary Medicine 2001;49. p. 141- 163. [Chen96) S. Chen and S. Mills. A binary Markov process model for random testing. IEEE Transactions on Software Engineering 1996;22(3). p. 2 18-223. [ChoiO 13 J. G. Choi and P. H. Seong. Dependability estimation of a digital system with consideration of software masking effects on hardware faults. Reliability Engineering and System Safety 2001;71. p. 45-55. [Choi98] J. K. Choi and P. H. Seong. Software dependability models under memory faults with application to a digital system in nuclear power plants. Reliability Engineering and System Safety 1998;59. p. 32 1-329. [COOPRA97] COOPRA working document. What PRA needs from a Otgital I&C systems analysis: an opinion. www.coopra.org, 1999. [Council971 National Research Council. Digital instrumentation and control systems in nuclear power plant: safety and reliability issues. National Academy Press. 1997. [FeckoOO] MA. Fecko, M.U Uyar, P.D. Amer, A.S. Seth, T. Dzik, R. Menell and M. McMahon. A success story of formal description techniques: Estelle specification and test generation for MIGSTD. Computer Communications 2000;23. p. 1196-1213. [Friedman951 M. A. Friedman and J. M. Voas. Software assessment - reliability, safety, testability. John Wiley & Sons. 1995. [Graae96] T. Graae and L. Engdahl. The reliability of the software of the digital control system nuclear advantage. Kerntechnik 1996;61(5-6). p. 236-23 8. [Gutjahr95] W. J. Gutjahr. Optimal test distributions for software failure cost estimation. IEEE Transactions on Software Engineering 1995;2 l(3). p. 2 19-228.

[Harrold99] ** M. J. Harrold. Testing evolving software. The Journal of Systems and Software 1999;47. p. 173-181. [Hecht97] M. Hecht, D. Tang and H. Hecht. Quantitative reliability and availability assessment for critical systems including software.

A. -43- Proceedings of the 12th Annual Conference on Computer Assurance. June 16-20 1997. Gaitherburg. USA. [Hierons97] R. M. Hierons and M. P. Wiper. Estimation of failure rate using random and partition testing. Software Testing, Verification and Reliability 1997;7. p. 153-164. [Hofmann95] H. Hofmann and H. J. Sauer. Effect of fieldbus technology on digital instrumentation and control for nuclear power plants. Kerntechnik 1995;60(5-6). p. 245-247. [Jaynes82] E. T. Jaynes. On the rationale of maximum-entropy methods. Proceedings of the IEEE 1982;70(9). p. 939-952. [Jeng99] B. Jeng. Toward an integration of data flow and domain testing. The Journal of Systems and Software 1999;45. p. 19-30. [Kapur94] P. K. Kapur, S. Agarwala, S. Younes and A. K. Sinha. On a general imperfect debugging software reliability growth model. Microelectronics and Reliability 1994;34(7). p. 1397-1403. [Kapur95] P. K. Kapur and S. Younes. Sohare reliability groeth model with error dependency. Microelectronics and Reliability 1995;35(2). p. 273-278. [Kapur96] P. K. Kapur and S. Younes. Modelling an imperfect debugging phenomenon in software reliability. Microelectronics and Reliability 1996;36(5). p. 645-650. (Kunze951 U. Kunze and V. Streiche. Advanced monitoring systems for preventive maintenance of mechanical systems and components. Kerntechnik 1995;60(5-6). p. 238-241. [Lee951 I.H. Lee and R. K. Iyer. Software dependability in the tandem GUARDIAN system. IEEE Transactions on Software Engineering 1995;21(5). p. 455-467. [Littlewood97J B. Littlewood and D. Wright. Some conservative stopping rules for the operational testing of safety-critical software. IEEE Transactions on Software Engineering 1997;23( 1 1). p. 673-683.

[Lyu96] -* M. R. Lyu (editor). Handbook of software reliability engineering. Mcgraw-Hill. 1996. [Ma991 Y. Ma and K. S. Trivedi. An algorithm for reliability analysis of phased-mission systems. Reliability Engineering and System Safety

-44- 1999;66. p. 157-170. [Miller921 K. W. Miller, L. J. Morell, R. E. Noonan, S. K. Park, D. M. Nicol, B. W. Mumll and J. M. Voas. Estimating the probability of failure when testing reveals no failures. IEEE Transactions on Software Engineering 1992;18( 1). p. 33-42. [Miller981 G. Miller and D. Horn. Probability density estimation using entropy maximization. Neural computation 1998;lO. p. 1925-1938. [Musag61 J. D. Musa. Software reliability engineered testing. Computer 1996;29(11). p. 61-68. [Offitt99] A. J. Offutt and S. Liu. Generating test data from SOFL specifications. The Journal of Systems and Software 1999;49. p. 49-62. [Ohba84] M. Ohba. Software reliability analysis models. IBM Journal of Research and Development 1984;28(4). p. 428-443. [Panas911 D. L. Parnas, G. J. K. Asmis and J. Madey. Assessment of safety- critical software in nuclear power plants. Nuclear Safety 1991;32(2) p. 189-198. [PapadopoulosO11 Y. Papadopoulos, J. McDermid, R. Sasse and G. Heiner. Analysis and synthesis of the behaviour of complex programmable electronic systems in condition of failure. Reliability Engineering and System Safety 2001;71. p. 229-247. [PasquinigS] A. Pasquini and E. DeAgostino. Fault seeding for software reliability model validation. Control Engineering Practice 1995;3(7). p. 993-999. [Pfleeger92] S. L. Pfleeger. Measuring software reliability. IEEE Spectrum 1992;29(8). p. 56-60. [Pressman921 R. S. Pressman. Software engineering. McGraw-Hill. 1992. [Profeta961 J. A. Profeta, N. P. Andrianos, B. Yu, B. W. Johnson, T. A. Delong, D. Guaspari . and D. Jamesk. Safety-critical systems built with COTS. IEEE Computer 1996;November. p. 54-60. [Raghavan991 V. Raghavan, M. Shakeri and K. R. Pattipati. Test sequencing -.. problems arising in test planning and design for testability. IEEE Transactions on Systems, Man and Cybernetics-Part A: Systems and Human 1999;29(2). p. 153-163. [Ravikumar99] C. P. Ravikumar, G. S. Saund and N. Agrawal. A functional-level

-45- testability measure for resigter-level circuits and its estimation. Microprocessors and Microsystems 1999;22. p. 535-542. IBopefi71 M. Roper, M. Wood and J. Miller. An empirical evaluation of defect detection techniques. Information and Software Technology 1997;39. p. 763-775. [Smidts99] C. Smidts and D. Sova. An architectural model for software reliability quantification: source of data. Reliability Engineering and System Safety 1999;64. p. 279-290. [Speranskiy96J On a synthesis method for random test for checking digital circuits. Engineering Simulation 1996: 13. p. 673-680. [Tal2000] 0. Tal, A. Bendell and C. McCollin. A comparision of methods for calculating the duration of software reliability demonstration testing, particularly for safety-critical systems. Quality and Reliability Engineering International 2000; 16. p. 59-62. [Trier94J H. Trier. Centre for siftware reliability. Reliability Engineering and System Safety 1994;43. p. 22 1-23 1. (USNRC97J USNRC. Digital instrumentation and control systems in nuclear power plants - safety and reliability issues. Final report. Washington D.C., 1997. weevers94J A. Veevers and A. C. Marchall. A relationship between software coverage metrics and reliability. Software Testing, Verification and Reliability 1994;4. p. 3-8. woas953 J. M. Voas, and K. W. Miller. Software testability: the new verification. IEEE software 1995;12(3). p. 17-28. [WaltonOO J G. H. Walton and J. H. Poore. Information and Software Technology 2000;42. p. 859-872. [Welbourne97J D. Welbourne. Safety critical software in nuclear power. The GEC Journal of Technology 1997;14(1) p. 33-40. [W hittaker00 J J. A. Whittaker and J. Voas. Toward a more reliable theory of software reliability. IEEE Computer 2000;December. p. 36-42. [Wong99J -.. W. E. Wong, J. R. Horgan, A. P. Mathur and A. Pasquini. Test set size minimization and fault detection effectiveness: A case study in a space application. The Journal of Systems and Software 1999;48. p. 79-89.

-46- fYamada931 S. Yamada, K. Tokuno and S. Osaki. Software reliability measurement in imperfect debuging environment and its application. reliability Engineering and System Safety 1993;40. p. 139-147. ~~951 M. C. K. Yang and An. Chao. Reliability-estimation and stopping-rules for software testing, based on repeated apperances of bugs. IEEE Transactions on Reliability 1995;44(2). p. 3 15-321. [ah0941 M. Zhao and M. Xie. EM algorithms for estimating software reliability based on masked data. Microelectronics and Reliability 1994;34(6). P. 1027-1038. [Zeephongsekul96] P. Zeephongsekul. Reliability growth of a software model under imperfect debugging and generation of errors. Microelectronics and Reliability 1996;36( 10). p. 1475-1482. [Zemva98] A. Zemva and B. Zajc. Functionality fault model: a basis for technology-specific test generation. Microelectronics and Reliability 1998;38(4). p. 597-604.

-47-

BIBLIOGRAPHIC INFOMATION SHEET

Perfonning Org. Sponsoring Org. Standard Report No. INIS Subject Code ReDort No. ReDort No. I I I I I I KAE~-2091/2o(n I I A study on the quantitative evaluation for Title / Subtitle I the software included in digital systems of nuclear power plants

~~ ~ Project Manager J.K. Park (Integrated Safety Assessment team) and Department

Researcher and T.Y. Sung (ISA team), H.S. Eom (ISA team), H.S. Jeong (Hanaro), Department H.G. Kang (ISA team), K.Y. Lee (ARTD team), J.K. Park (NTC) I Publication Publication Taejon Publisher KAERI 2002.3. Place Date 21 x Page 55 p. Ill. & Tab. Yes( 0 1, No ( 1 Size 29.7cm

Open( 0 1, Restricted( 1, Classified Report Type Technical Report - Class Document sponsoring Contract No. Org. In general, probabilistic safety analysis (PSA) has been used as one of the most important methods to evaluate the safety of NPPs. The PSA, because most of NPPs have been installed and used analog I&C systems, has been performed based on the hardware perspectives. In addition, since the tendency to use digital I&C systems including software instead of analog I&C systems is increasing, the needs of quantitative evaluation methods so as to perform PSA are also increasing. Abstract Nevertheless, several reasons such as software did not aged and it is very (15-20 Lines) perplexed to estimate software failure rate due to its non-linearity, make the performance of PSA difficult. In this study, in order to perform PSA including software more efficiently, test-based software reliability estimation methods are reviewed to suggest a .preliminary procedure that can provide reasonable guidances to quantify software failure rate. In addition, requisite research activities to enhance - applicabdity of the suggested procedure are also.discussed. Subject Keywords digital I&C system, probabilistic safety assessment, software, reliability, (About 10 quantitative evaluation

words) 2.