Risk Assessment Based on Maximum Allowable Damage

Risk Assessment based on Maximum Allowable Damage

Jaime Eduardo Cadena Gomez Master of Science (Chemical Engineering) Bachelor of Science (Chemical Engineering)

0000-0003-0036-3015

A thesis submitted for the degree of Doctor of Philosophy at The University of Queensland in 2021 School of Civil Engineering

Abstract

Demonstrating the adequate fire safety performance of a building design is a requisite part of the approvals process in the construction industry. Risk assessments provide a powerful tool to quantify fire risk, which is often used as a proxy for fire safety performance. However, the nature of the risk assessment process means that it may also open the door for unacceptable losses to be present in the form of latent risks. This research analyses this problem and puts forward key elements needed for better fire risk assessments and an inherently safer built environment.

In a performance based environment, fire safety engineering can take advantage of frameworks and methodologies to explicitly demonstrate performance through qualitative or quantitative risk assessments. The latter are preferred as they evoke a sense of objectivity and scientific rigor and due to their extensive use in high-risk settings, e.g. chemical and nuclear safety. Quantitative risk assessments can be deterministic or probabilistic. The former focuses on specific consequences, while the latter quantifies the likelihood of a range of undesired consequences.

Probabilistic risk quantification can result in significant consequences appearing to be acceptable conditional on the fact that their likelihood is low enough. However, three problems arise from this: i) public responses to fires which results in large loss of life completely invalidate the premise of acceptable risk based on negligible likelihood; ii) the subjective basis and lack of structured approaches to identify scenarios that can actually challenge the performance; and iii) the reliance of probabilistic risk assessments on scarce statistical data is a topic of great concern in other fields, with some practitioners suggesting that they be reserved for specialized applications. As probabilistic risk assessments are often promoted as the default choice for demonstrating fire safety performance, the danger of providing a false sense of safety is significant. This concern has motivated this research, which has the overarching objective of proposing an alternative methodology for fire risk assessment that promotes an inherently safer built environment.

This research is divided in two parts. The first part details the limitations associated to risk assessments, to probabilistic risk assessments and to the existing guidance for fire risk assessments. These limitations include data availability, the subjective basis for scenario identification and an unknown degree of trustworthiness of the results. The second part proposes a consequence-driven fire risk assessment methodology, applies it to three distinct case studies and evaluates both its advantages and limitations. The proposed methodology focuses on the potential range of fire consequences. The upper end of this consequences range should be below a stakeholder-predefined maximum allowable damage for the performance to be adequate. This concept is the basis of the methodology, correspondingly named Maximum Allowable Damage (MAD). The consequence range is estimated for a ‘bare-bones’ design, which excludes those active safety measures that cannot be ensured to work over the life of a building (e.g. sprinklers). This estimation allows identifying possible modifications to its constitutive elements as to achieve adequate performance. This approach is aligned with the inherently safer design philosophy which designs out hazards and introduces safety measures only on an as needed basis.

MAD uses a high-level representation to describe how the fire damage materializes, i.e. damage model. This representation provides an initial bounding of the scenarios. The variables, their values, and underlying assumptions within the damage model are logged, allowing to judge their trustworthiness. Trustworthiness in MAD is a function of the strength of knowledge and output sensitivity of each variable and assumption. To characterize the upper end of the damage potential, a deductive reasoning approach is implemented, where the central question is which conditions are necessary (and plausible) to exceed the maximum allowable damage. Hence, scenarios are iterated and both the damage model and the information registry are updated as needed.

MAD allowed obtaining insight on the fire safety performance of three distinct case studies. This insight was formulated as actionable recommendations that enabled an adequate performance. The trustworthiness of each assessment was a key component in achieving and defining these recommendations. Both the damage model and the information registry serve as supports for the trustworthiness of the assessment and as a basis for third party reviews and future reassessments. As observed in all case studies, the acceptance criteria largely defines the degree of conservatism required in the assessment, addressing one of the main criticisms to consequence-driven analyses.

While MAD does not produce a metric to be used as a universal yardstick for fire risk, it is a means to trustworthy insight into fire risk. This insight can be used to ensure that a building is inherently safe before other methods are applied to optimize the fire safety design.

Declaration by author

This thesis is composed of my original work, and contains no material previously published or written by another person except where due reference has been made in the text. I have clearly stated the contribution by others to jointly-authored works that I have included in my thesis.

I have clearly stated the contribution of others to my thesis as a whole, including statistical assistance, survey design, data analysis, significant technical procedures, professional editorial advice, financial support and any other original research work used or reported in my thesis. The content of my thesis is the result of work I have carried out since the commencement of my higher degree by research candidature and does not include a substantial part of work that has been submitted to qualify for the award of any other degree or diploma in any university or other tertiary institution. I have clearly stated which parts of my thesis, if any, have been submitted to qualify for another award.

I acknowledge that an electronic copy of my thesis must be lodged with the University Library and, subject to the policy and procedures of The University of Queensland, the thesis be made available for research and study in accordance with the Copyright Act 1968 unless a period of embargo has been approved by the Dean of the Graduate School.

I acknowledge that copyright of all material contained in my thesis resides with the copyright holder(s) of that material. Where appropriate I have obtained copyright permission from the copyright holder to reproduce material in this thesis and have sought permission from co-authors for any jointly authored works included in the thesis.

Publications included in this thesis Cadena J. E., Osorio A. F., Torero J. L., Reniers G., Lange D., Uncertainty-based decision-making in fire safety: Analyzing the alternatives, Journal of Loss Prevention in the Process Industries, vol. 68, 2020 (https://doi.org/10.1016/j.jlp.2020.104288). – Incorporated as Annex 1. Contributor Statement of contribution % Jaime Cadena Gomez Conception and design 70 Analysis and interpretation 75 Drafting and production 80 Andres F. Osorio Conception and design 10 Analysis and interpretation 10 Drafting and production 10 Jose L. Torero Conception and design 5 Analysis and interpretation 5 Drafting and production 0 Genserik Reniers Conception and design 10 Analysis and interpretation 5 Drafting and production 0 David Lange Conception and design 5 Analysis and interpretation 5 Drafting and production 10

Cadena J. E., Hidalgo J. P., Maluk C., Torero J. L., Osorio A. F., Overcoming Risk Assessment Limitations for Potential Fires in a Multi-Occupancy Building, Chemical Engineering Transactions, vol. 77, 2019 (https://doi.org/10.3303/cet1977078). – Incorporated as part of Chapter 6 (section 6.5). Contributor Statement of contribution % Jaime Cadena Gomez Conception and design 80 Analysis and interpretation 80 Drafting and production 85 Juan P. Hidalgo Conception and design 0 Analysis and interpretation 3 Drafting and production 0 Cristian Maluk Conception and design 0 Analysis and interpretation 2 Drafting and production 0 Jose L. Torero Conception and design 10 Analysis and interpretation 5 Drafting and production 5 Andres F. Osorio Conception and design 10 Analysis and interpretation 10 Drafting and production 10 Submitted manuscripts included in this thesis Cadena J. E., McLaggan M., Osorio A. F., Torero J. L., Lange D., Fire risk assessment for a non- compliant façade in a high-rise building, Fire Safety Journal, Under Review. – Incorporated as part of Chapter 7 (section 7.1). Contributor Statement of contribution % Jaime Cadena Gomez Conception and design 75 Analysis and interpretation 80 Drafting and production 75 Martyn McLaggan Conception and design 0 Analysis and interpretation 5 Drafting and production 5 Andres F. Osorio Conception and design 10 Analysis and interpretation 5 Drafting and production 5 Jose L. Torero Conception and design 10 Analysis and interpretation 5 Drafting and production 5 David Lange Conception and design 5 Analysis and interpretation 5 Drafting and production 10

Cadena J. E., Gupta V., Majdalani A., Torero J. L., Hidalgo J. P., Characterising uncertainty in the modelling of fully-developed compartment fires, Journal of Building Simulation, Submitted. – Incorporated as Annex 2. Contributor Statement of contribution % Jaime Cadena Gomez Conception and design 70 Analysis and interpretation 45 Drafting and production 50 Vinny Gupta Conception and design 10 Analysis and interpretation 40 Drafting and production 30 Agustin Majdalani Conception and design 0 Analysis and interpretation 0 Drafting and production 5 Jose L. Torero Conception and design 15 Analysis and interpretation 10 Drafting and production 5 Juan P. Hidalgo Conception and design 5 Analysis and interpretation 5 Drafting and production 5

Other publications during candidature

Peer-reviews papers Cadena J. E., Muñoz F., Complexity and Uncertainty Management in Process Safety Education, Chemical Engineering Transactions, vol. 77, 2019 (https://doi.org/10.3303/cet1977073). – Not incorporated in the thesis. Contributor Statement of contribution % Jaime Cadena Gomez Conception and design 95 Analysis and interpretation 95 Drafting and production 95 Felipe Muñoz Conception and design 5 Analysis and interpretation 5 Drafting and production 5

Contributions by others to the thesis

No contributions by others.

Statement of parts of the thesis submitted to qualify for the award of another degree

No works submitted towards another degree have been included in this thesis.

Research involving human or animal subjects

No animal or human subjects were involved in this research.

Acknowledgments

Writing these words signal the culmination of one of the major goals I have set for myself. As most goals that are achieved, now it just looks like a stepping-stone to start the journey towards new challenges and goals. And as in any journey to achieve something, I was not alone. Throughout my life and specially the last four years I have been surrounded physically and in spirit by many who have supported me and helped me get here. I wish to acknowledge them in the next few lines.

To my mother. Ma, tenerte en mi vida es una bendición por la que cada día doy gracias. Tu espíritu amable, tu amor sin límites y tu constante optimismo han nutrido mi lado más humano. De entre los privilegios de los que gozo, tu amor y tu fe en mí son incomparables e irremplazables. Con estas palabras te quiero agradecer nuevamente, ya que sin tu apoyo hubiese sido increíblemente difícil culminar este proyecto. Mientras escribo estas palabras sonrío y te abrazo a la distancia.

To my father. Pa, no puedo recordar un solo instante en que hayas dudado de mis capacidades, lo cual me ha impulsado a alcanzar todo lo que me he propuesto y a soñar con cada vez más. A pesar de que somos tan diferentes, nuestro vínculo ha resistido incluso las más recias tormentas y se ha hecho más fuerte. Ad portas de la culminación de este proyecto de cuatro años y al igual que en todos mis logros, te agradezco por tu inagotable dedicación y por todo lo que me has enseñado.

To Alex. By your side I have learned more than what many PhD could teach anyone. Я даже выучил немного русский! You have helped me unlock possibilities that only existed in my imagination and now I am proud to be building a future with you. I want to thank you for supporting and –at the hardest times- standing by me. Having us both completing the journey of a doctoral research during an unprecedented moment in modern times has put tremendous pressure on both of us. I am thankful we have managed to carry on, overcome adversity and reach this point together. Thank you for believing in me and in what we can accomplish together.

To my mentor. From the people that I admire and have inspired me to pursue this project, Dr Felipe Muñoz will always stand out. Felipe, you have been my process safety and risk mentor and a friend with the most exquisite taste and compassionate heart. I thank you for your teachings but most of all for your sincere advice, which allowed me to enjoy this project for what it is, a wonderful life changing experience.

To my friends. As years pass, much of you has become a part of me and despite distance I always feel you close by. The brotherhood and joy of Carlos, Jorge, Mao and Jacobo. Luis’ humor, support and advice, Andrea’s relentless good vibes and Mateo’s contagious lust for life; my dear Cotorras. Carmen’s friendship and mastery, Vinny’s humour and commitment to our never ending paper and Einat’s willingness to have many great discussions. Amanda’s motherly care, Maria Antonia’s wisdom, Laura’s kindness, Sergio’s insight, and my brother Andres’ steadfast support. I am thankful for calling you all my friends, as well as many I fail to mention. To my colleagues at UQ Fire, thank you all for welcoming me to a group of extraordinary individuals who motivated and inspired me. You all inspired me to make the best of this experience and I am thankful for having met such wonderful people and engineers; keep up the good work!

Last but not least, to my advisors. So many brilliant people have given me their time and their attention to help me widen my perspective and to question what it is that I was doing in this project and for what purpose. I thank you all, but particularly those that I have been fortunate enough to guide me throughout this project. I want to thank Prof Jose Torero for encouraging me to challenge what I understood and what scaped my perspective, as well as for all your patient feedback. Without the trust and effort of Dr Andres Osorio it would have been extremely hard to build the foundations of this work. Thank you Andres for helping me build up the foundations of this research and for believing that in my capability to figure out a problem that at times looked like a dead end. Making sense of the most relevant subtleties and the connection of this research to the greater scheme of things was possible largely due Dr David Lange. David, your dedication, patience and commitment to what you do is something I look up to and I am proud of having the opportunity to work with you.

I would also like to thank Prof Genserik Reniers for his collaboration, Prof Ruben Van Coile for his unique insight on fire risk, Dr James O’Neill for mentoring me and helping me prepare for the next stage of my carrier. I am also thankful for the time and insight of Dr Juan P. Hidalgo and Dr Cristian Maluk, as well as for the very important conversations with Dr Henrik Bjelland, Dr Ben Seligmann, Prof Drew Rae, Chris Corless and Michael Beaumont.

I am fortunate to have had your support throughout this stage of my life and for pushing me to keep learning, growing and having a positive impact on this wonderful and insane world we live in.

Financial support

This research was supported by an Australian Government Research Training Program Scholarship.

Keywords

Fire risk assessment, Performance-based Design, Consequence analysis, Inherently Safer Design, Trustworthiness, Probabilistic Risk Assessments, Uncertainty

Australian and New Zealand Standard Research Classifications (ANZSRC)

ANZSRC code: 091507 Risk Engineering, 60% ANZSRC code: 090599 Civil Engineering not elsewhere classified, 40%

Fields of Research (FoR) Classification

FoR code: 0915, Interdisciplinary engineering, 60% FoR code: 0905, Civil engineering, 40%

Table of Contents An initial reflection ...... 1 Bibliography ...... 5 1 Introduction ...... 6 1.1 Risk assessment ...... 7 1.2 Decision-making ...... 10 1.3 Risk management strategies ...... 13 1.4 Biases ...... 17 1.5 Conclusions...... 22 Bibliography ...... 24 2 Research approach ...... 27 2.1 Research questions and claims ...... 27 2.2 Research outputs ...... 32 Bibliography ...... 33 3 Probabilistic risk assessment in FSE ...... 34 3.1 Deterministic and probabilistic quantification ...... 35 3.2 Probabilistic fire risk assessments ...... 38 3.3 Verifying safety with PRAs ...... 46 3.4 ‘Adequate safety’ ...... 49 3.4.1 Fire likelihood...... 52 3.4.2 Large loss events ...... 54 3.5 Risk acceptance criteria ...... 56 3.5.1 Types of criteria ...... 56 3.5.2 Societal risk in FSE ...... 58 3.5.3 Summary ...... 61 3.6 Limitations of scenario identification ...... 63 3.6.1 Fire scenarios ...... 64 3.6.2 Design fires and scenario clusters...... 67 3.7 Supporting data ...... 69 3.8 Guiding the design process ...... 75 3.9 Advantages and challenges ...... 78 3.10 Conclusions ...... 80 Bibliography ...... 83 4 Comparing contexts: Major Hazardous Facilities and Fire Safety Engineering ...... 93 4.1 The MHF experience ...... 93 I

4.2 Objectives and purpose ...... 96 4.3 Stakeholders ...... 100 4.4 Risk ownership ...... 102 4.5 Design process ...... 103 4.6 Classification ...... 106 4.7 Scenario identification ...... 109 4.8 Scenario use ...... 112 4.9 Uncertainty description ...... 114 4.10 Acceptance criteria ...... 116 4.11 Risk as a function of time ...... 117 4.12 Conclusions ...... 120 Bibliography ...... 122 5 Evaluation of fire risk assessment methodologies ...... 130 5.1 Objective ...... 132 5.2 Methodology ...... 132 5.2.1 Peer review ...... 134 5.2.2 Limitations ...... 135 5.3 NFPA 551 ...... 135 5.4 SFPE guide ...... 137 5.5 PD 7974-7 (2019) ...... 140 5.6 ISO 16732-1 (2012) ...... 142 5.7 TR 951 (2019) ...... 144 5.8 Evaluation results ...... 147 5.9 Discussion ...... 150 Hazards identification ...... 150 Gradual approach ...... 152 Required precision ...... 153 Quality of the assessment ...... 155 5.10 Conclusions ...... 156 Bibliography ...... 158 6 Maximum Allowable Damage ...... 160 6.1 Basis for an alternative methodology ...... 160 6.1.1 Focus on consequences ...... 160 6.1.2 Scenario identification ...... 161 6.1.3 Representing complex systems ...... 162

6.2 Maximum Allowable Damage (MAD) concept ...... 163 6.2.1 Inherent risk and layers of protection ...... 166 6.2.2 Deterministic risk analysis ...... 169 6.3 Objective ...... 172 6.4 Scope ...... 172 6.5 Process ...... 173 6.5.1 Safety objectives ...... 174 6.5.2 Acceptance criteria ...... 175 6.5.3 Damage model ...... 175 6.5.4 Damage estimation ...... 180 6.5.5 Performance assessment ...... 185 6.5.6 Trustworthiness ...... 187 6.6 Evaluation ...... 191 6.7 Conclusions...... 194 Bibliography ...... 197 7 Case studies ...... 202 7.1 Case-study: High-rise residential building with non-compliant façade ...... 202 7.1.1 Context to façade fires ...... 202 7.1.2 Building description ...... 203 7.1.3 Safety objective ...... 204 7.1.4 Acceptance criterion ...... 205 7.1.5 Damage model ...... 205 7.1.6 Damage estimation ...... 209 7.1.7 Performance assessment ...... 212 7.1.8 Identifying remediation actions ...... 215 7.1.9 Façade ignition and external flame spread ...... 220 7.2 Case-study: Chemical storage warehouse ...... 221 7.2.1 Safety objective ...... 223 7.2.2 Acceptance criteria ...... 223 7.2.3 General damage model ...... 224 7.2.4 Scenario 1 - Damage model ...... 225 7.2.5 Scenario 1 - Damage estimation ...... 227 7.2.6 Scenario 1 – Performance assessment ...... 236 7.2.7 Scenario 2 – Damage model ...... 236 7.2.8 Scenario 2 - Damage estimation ...... 238 7.2.9 Scenario 2 – Performance assessment ...... 241 7.2.10 Risk treatment & uncertainties ...... 241

III

7.2.11 Introducing variability to Scenario 2 ...... 245 Bibliography ...... 249 8 Lessons learnt ...... 254 8.1 MAD enables gaining insight, not producing numbers ...... 254 8.2 Fire risk assessments per se are useless if their output is not trustworthy ...... 255 8.3 Identifying scenarios in FSE requires a unique iterative approach ...... 256 8.4 Transparency and accountability require explicit evidence ...... 258 8.5 Using inherent safety to produce a safer built environment ...... 260 8.6 The degree of conservatism is largely dictated by the acceptance criteria ...... 260 Bibliography ...... 263 9 Conclusions and future work ...... 264 9.1 Future work ...... 265 9.1.1 Defining trustworthiness ...... 265 9.1.2 Acceptance criteria resolution ...... 265 9.1.3 Organizational and human factors in the damage model ...... 266 9.1.4 Damage models library ...... 266 9.1.5 Best practices for PRA ...... 266 9.2 Conclusions...... 267 Appendix 1 ...... 270 Appendix 2 ...... 328 Appendix 3 ...... 376 Appendix 4 ...... 383

List of Figures Figure 1-1. Risk assessment process as part of the risk management process [3] ...... 8 Figure 1-2. Principles for engineering safety [10] ...... 11 Figure 1-3. Frameworks to decision-making ...... 11 Figure 1-4. Adapted model for risk informed decision model [17] ...... 12 Figure 1-5. ‘Problem space’ defined by the type of problem and applicable science ...... 15 Figure 1-6. Main components of risk management plotted in the ‘problem space’ ...... 16 Figure 3-1. Simple representation of the result of a deterministic and a probabilistic analysis ...... 36 Figure 3-2. Simple representation of the result of a deterministic and a probabilistic analysis ...... 37 Figure 3-3. Probability consequence diagram for the example ...... 37 Figure 3-4. Event tree for the example ...... 39 Figure 3-5. Inputs, outputs and convergence for the PRA example ...... 41 Figure 3-6. Cumulative probability distribution for the probability of failure ...... 41 Figure 3-7. Graphical representation of the building ...... 44 Figure 3-8. Event tree and scenarios for the example (only carpark) ...... 44 Figure 3-9. Risk profile for the building plotted as an F-N curve ...... 45 Figure 3-10. Risk profiles for the alternatives ...... 46 Figure 3-11. Paths from regulation to decision-making in safety. Adapted from Bjelland [42] ...... 50 Figure 3-12. Risk definition adapted from (left) ISO 31000:2018 [47] and (right) Aven [48]...... 51 Figure 3-13. Fatality rate in residential fires in the UK, period 2009-2019 [57] ...... 54 Figure 3-14. MAD acceptance criteria within the generalized frequency-consequence diagram. Image adapted from [79] ...... 60 Figure 3-15. Probability consequence diagram; taken from Ale et al. [12] ...... 61 Figure 3-16. Key elements of a typical scenario ...... 64 Figure 3-17. Number of years required for observation for different risk levels ...... 72 Figure 3-18. Failure trees for a passive and an active measure performing the same function ...... 74 Figure 3-19. Typical safety features of a fire safety strategy...... 76 Figure 4-1. Trend of major accidents reported in the European Union and their relation to MHF regulation [15]...... 95 Figure 4-2. Trend of hydrocarbon releases in the UK's offshore industry and its relation with the COMAH regulation [16]...... 96 Figure 4-3. General structure for MHF regulation ...... 97 Figure 4-4. Typical steps in the plant design process and safety elements; Based on [43-45] ...... 105

Figure 4-5. Stages of building design and process management (stages 5 through 7 are omitted here). Based on [47] ...... 106 Figure 4-6. Relation between techniques, their applicability and the proportion of scenarios they cover. Adapted from [57]. HRA: Human Reliability Analysis...... 110 Figure 4-7. Process for the knowledge base update ...... 113 Figure 5-1. Evaluation results – NFPA 551 ...... 137 Figure 5-2. Evaluation results – SFPE guide ...... 140 Figure 5-3. Evaluation results – PD 7974-7 (2019) ...... 142 Figure 5-4. Evaluation results – ISO 16732-1 (2012) ...... 144 Figure 5-5. Evaluation results – TR 951 (2019) ...... 146 Figure 5-6. Overall results of the guidance document evaluation ...... 147 Figure 6-1. Consequence scale...... 163 Figure 6-2. Graphical representation of MAD ...... 164 Figure 6-3. Risk assessment process applied to inherently safer design ...... 167 Figure 6-4. Layers of protection concept in chemical process safety ...... 168 Figure 6-5. Layers of protection concept applied to building design ...... 169 Figure 6-6. Consequence-probability plot for the cleaning cabinet fire scenarios ...... 172 Figure 6-7. MAD process...... 173 Figure 6-8. Graphical representation of the building ...... 174 Figure 6-9. Evolution of accident modelling ...... 176 Figure 6-10. Risk assessment model, adapted from Beck [39] ...... 177 Figure 6-11. Causal loop of a simple compartment fire [46] ...... 178 Figure 6-12. Input classification flowchart ...... 181 Figure 6-13. Graphical representation of the building ...... 185 Figure 6-14. ASET/RSET as a function of soot yield and fire growth rate ...... 185 Figure 6-15. Performance assessment for basement-2 ...... 186 Figure 6-16. Heat map for the trustworthiness of the assessment ...... 189 Figure 6-17. Fire growth rate for inadequate performance and its cumulative probability ...... 189 Figure 6-18. Sensitivity to a higher stairwell density ...... 190 Figure 6-19. Evaluation results – MAD ...... 191 Figure 6-20. Overall results of the guidance document evaluation ...... 193 Figure 7-1. Floorplan for a typical residential level; numbers correspond to each flat ...... 204 Figure 7-2. Schematic representation of a façade system ...... 204 Figure 7-3. Safety measures in FSE highlighting those included in this case study ...... 206 Figure 7-4. Damage model for the case study ...... 206 VI

Figure 7-5. Staged evacuation scheme ...... 209 Figure 7-6. Maximum Damage Potential as a function of ASET/RSET for the lobby at each level – fire starting at the 1st, 4th, 10th and 15th level ...... 213 Figure 7-7. Maximum allowable upward flame spread ...... 213 Figure 7-8. Damage as a function of ASET/RSET for the flats above the level of fire origin – fire starting at 1st, 4th, 10th and 15th level. Slow fire growth above, ultra-fast fire below...... 215 Figure 7-9. Comparison of tenability criteria for ultra-fast fire originating at the kitchen ...... 219 Figure 7-10. Time for window breakage using B-Risk model, plotted against the temperature of upper and lower layers in the kitchen compartment; red line denotes window breakage ...... 219 Figure 7-11. Graphical description of the warehouse areas; not to scale ...... 222 Figure 7-12. Graphical description of (left) Area 5 – plan view and (right) IBCs distribution – elevation view; not to scale ...... 223 Figure 7-13. General damage model for business continuity ...... 224 Figure 7-14. Specific damage model for scenario 1 ...... 226 Figure 7-15. Comparison of the number of cells spanning the characteristic fire diameter for different mesh sizes ...... 228 Figure 7-16. Snapshot of the simulation for scenario 1.3 in FDS ...... 229 Figure 7-17. Roof structural design (not to scale) ...... 230 Figure 7-18. Gas temperature from the pool fire and steel temperature (unprotected and with insulation)...... 232 Figure 7-19. Free body and bending moment diagrams ...... 232 Figure 7-20. Reduction of the plastic moment capacity for scenario 1.3 ...... 233 Figure 7-21. Exposure profiles for the scenarios of the pool fire. a) Scenario 1.1, b) Scenario 1.2 and c) Scenario 1.3...... 235 Figure 7-22. Position of the initial and second pool fires (not to scale) ...... 237 Figure 7-23. Specific damage model for scenario 2 ...... 237 Figure 7-24. Snapshot of the simulation for scenario 2.0 in FDS ...... 239 Figure 7-25. Steel temperature as a function of fire duration ...... 239 Figure 7-26. Plastic momentum as a function of time ...... 240 Figure 7-27. Failure time as a function of the applied insulation ...... 240 Figure 7-28. Trustworthiness heat map; the number in the cells indicate the number of variables 242 Figure 7-29. Scenario S1.1 snapshot at 60 s, presenting incident heat fluxes on IBCs (kW/m2) .... 243 Figure 7-30. Effect of beam and insulation (0.5 cm) ...... 244 Figure 7-31. Effect of beam and insulation (1.0 cm) ...... 245 Figure 7-32. Process to sample the yield strength reduction factor ...... 246 VII

Figure 7-33. Convective heat transfer coefficient reported by FDS and corresponding histogram (taken from 6 devices directly over the fire within the timeframe between 30 s and 300 s) ...... 247 Figure 7-34. (Above) Probability density and (below) cumulative probability for the failure time of the steel beam. Red vertical line indicates the result of the deterministic analysis...... 248

VIII

List of Tables Table 1-1. Definition of the risk assessment steps ...... 8 Table 1-2. Approaches of Risk Assessment and their outcomes ...... 9 Table 1-3. Biases relevant to fire risk assessments ...... 18 Table 2-1. Papers produced in the course of this research ...... 32 Table 3-1. Annual occurrence rates for different events ...... 53 Table 3-2. Annual fatality rates for different activities ...... 54 Table 3-3. List of fires with unacceptable loss ...... 55 Table 3-4. Types of acceptance criteria ...... 62 Table 4-1. Classification for the type of construction [24] ...... 107 Table 4-2. Classification for consequence class [51] ...... 108 Table 4-3. Classification for consequence class [52] ...... 108 Table 5-1. Guidance documents analysed in this chapter ...... 131 Table 5-2. Evaluation aspects and specific criteria ...... 133 Table 5-3. Detailed results of the evaluation ...... 148 Table 5-4. Results per evaluation criteria ...... 149 Table 6-1. Trustworthiness criteria and number of entries ...... 187 Table 6-2. Detailed results of the evaluation including MAD. F: fully meets the criterion; P: partially meets the criterion; N: does not meet the criterion...... 193 Table 7-1. Representative quantities associated to the ASET calculation ...... 210 Table 7-2. Tenability criteria for ASET calculation ...... 210 Table 7-3. Representative quantities associated to the RSET calculation ...... 211 Table 7-4. ASET/RSET ratio for the flat and level of fire origin ...... 214 Table 7-5. Trustworthiness criteria and number of entries ...... 216 Table 7-6. Number of entries for each level of SoK and OS ...... 216 Table 7-7. Key assumptions and trustworthiness ...... 217 Table 7-8. Treatment options based on key assumptions ...... 220 Table 7-9. Typical storage quantities and types per area. As reported in official documents and observed during visits. NI: No information ...... 222 Table 7-10. Quantities involved in the damage estimation for the initial pool fire ...... 228 Table 7-11. Scenarios for the initial pool fire ...... 229 Table 7-12. Quantities involved in the damage estimation for structure ...... 231 Table 7-13. Structural damage for the initial pool fire ...... 233 Table 7-14. Scenarios for the initial pool fire ...... 235

Table 7-15. Quantities involved in the damage estimation for the second pool fire ...... 238 Table 7-16. Scenarios for the second pool fire ...... 238 Table 7-17. Stochastic variables for the probabilistic analysis ...... 246

List of Abbreviations used in the thesis

ASET: Available Safety Egress Time CCPS: Center for Chemical Process Safety FSE: Fire Safety Engineering HAZID: Hazard Identification MAD: Maximum Allowable Damage MHF: Major Hazardous Facility OS: Output Sensitivity PRA: Probabilistic Risk Assessment RSET: Required Safety Egress Time SoK: Strength of Knowledge QRA: Quantitative Risk Assessment

An initial reflection

“This just goes to show that risk is all in the mind. That is to say, risk is (also) a notion of observation, and not just an object to be observed. As a notion of observation, it is a kind of lens through which we see the world. What we see as risk is not about absolute reality, but instead depends on the kind of lens and the way in which we look through it. Different disciplines use different kinds of lens, and so they may see different things even when looking at the same object.” (Holzheu & Wiedemann, 1993, p.9-10)

How can we make sure something works? This is an intriguing question. Our modern world has been constructed using both empirical knowledge and theories that partially address the question. Both empirical knowledge and theories have led to engineering achievements since the XX century such as skyscrapers, automobiles and planes, nuclear power and the digital era. Some of these achievements have also led to tragic and unrepairable failures, showing that even well-accepted theories and useful empirical rules are sometimes dangerously flawed. The failure of these systems and their review leads to improved knowledge and better designs… and then new failures occur, prompting the cycle to continue. As a result we improve our capabilities and our naïvetés are revealed. So, how can we make sure something works?

To try to answer this I will take a small detour. I ran into the question of professional competencies several times as I conducted this research, making me question what it is that constitutes a competent engineer. Partly, it relates to having a particular qualification, usually a bachelors’ degree. The degree is necessary, but largely insufficient. Nevertheless, the results obtained as part of the degree do make a difference in the job market and therefore on the path towards becoming a competent engineer. One of these results is the grade point average or GPA. Since students with higher GPAs might be favoured for job offers, this makes me wonder is this means they are more competent that a student with an average GPA.

Consider a brilliant engineering student. What features would describe them? Perhaps one would expect a high GPA, consistent assistance, high engagement rate, and a good relationship with both lecturers and peers. However, as me, many engineers have crossed paths with brilliant students who might not check all those boxes. Some of these people are brilliant because of the way they understand and explain problems, others because of their mathematical abilities, and others because their unparalleled creativity; however, these qualities might not prevent them from failing to achieve the necessary GPA to graduate [1]. Now the flipside; most 1 An initial reflection

of us know some graduates with remarkable GPAs that are far from reliable. It seems unnatural to encompass so many dimensions that constitute engineering competency into a number such as the GPA, and yet it remains an immovable criterion to graduate, to classify how ‘good’ a graduate is and in many instances to offer or deny very promising job opportunities.

By itself, the GPA does not tell you how good any student is. This is reinforced by the study of Bernold et al. [2] relating GPA bands and learning styles, which points out that some students asking the questions an engineer should, might not hit the target needed to become one. The deterioration of current grading systems [3] and the inflation of grades [4] make the GPA even less reliable in defining if a student is competent. So, a simple benchmark does not cut it. The immense complexity of engineering competency cannot be reduced to such a simple number. To a large degree, this is the problem with most benchmarks on which engineering relies to guarantee a particular result, such as system performance.

Ticking boxes and verifying thresholds are met do not ensure a desired outcome. A very seasoned corrosion engineer once told me: “the reason why companies hire me is not because I have a lot of experience or certificates, it is simply because I am good at understanding and solving problems”. Engineering provides practitioners with knowledge, frameworks, tools and benchmarks to understand problems and figure out a way to solve them. However, none of these benchmarks or tools are solutions by themselves, they are means to devise a solution and cannot ensure something will work.

Yardsticks and mechanized procedures tend to be suitable solutions only for well-structured and well-known problems. For all other engineering problems, their suitability is questionable. In the context of safety engineering, past experience has taught us repeatedly that simple solutions are not enough and that solving problems require understanding them thoroughly. The offshore disaster of Piper Alpha showed that filling up safety paperwork does not produce real safety. The Fukushima nuclear disaster showed that even several redundancy layers can be defeated given a particular set of conditions. Complying does not ensure safety, even if it does provide a false sense of it.

As I now begin to discuss safety engineering, I consider it reasonable to provide some context about my professional background. About 16 years ago, I was a chemical engineering student beginning a path towards the professional practice of engineering. As most students, I was focused on lectures, homework and projects and I did not realize that being a competent engineer requires doing things well, but also knowing why they are done. I started becoming

2 An initial reflection

aware of this during my first job, which was plagued of mundane tasks and involved supervising and answering to people with very different perspectives on what it takes for something to work. This experience helped me realize I had a fascination for high-risk industrial facilities and for the potentially large events like fires and explosions that could take place in them. Despite being very young, I felt compelled to work in this field and contribute to prevent such disasters.

During my Master’s degree, I stepped into the world of fire safety engineering. For the next six years, I gained experience and knowledge between professional practice and educational settings, with my work focused on process safety. With some background on compartment fires from my bachelor’s thesis, I was able to structure my Master’s thesis around quantifying the uncertainty associated to compartment fire modelling. Thanks to that experience I was able to create connections and to learn more about a fascinating discipline, with many remarkable similitudes and stark differences with chemical process safety. When I finished an internship at the University of Edinburgh under the supervision of Prof Jose Torero, I had the feeling that if I ever pursued a doctoral research, this would be a very interesting field to focus on.

After three years of further professional experience as a consultant in chemical process safety, I was growing comfortable as my work network grew and project flow became steadier. I felt that I was only applying what I knew, but I was not contributing anything new to the field of safety science. Motivated by my mentor, Dr Felipe Muñoz, I knew that I would have to make a decision on whether to pursue a doctoral research or to stay home and keep building my name in consultancy. There was almost no decision at all, the research seemed like a challenge in all aspects of life and the ultimate departure from the comfort zone. This led me to coming to Australia and to The University of Queensland on 2017; I was curious about the only common element in all of my professional experience: risk. All readers will have an intuition about what risk means, but I have discovered that trying to put it into words or into numbers, is an incredible challenge. Therefore, using risk as a yardstick seemed mind-boggling.

Over the years leading up to my research, I became acquainted with the practice of translating risk into numbers and using them to gauge how ‘safe’ a system is. This approach always seemed to me something like a silver bullet. I never used such an approach to tell a client whether a chemical process was safe or not, and I always kept my focus on understanding how the processes could turn unsafe and proposing ways to prevent, control or mitigate it. To me it has seemed obvious that many good engineering decisions are made without an explicit

3 An initial reflection

quantification of risk. Quantifying risk requires significant knowledge and resources, both of which are strictly limited in engineering practice. Therefore, this approach is far from the go-to approach to solving problems in safety engineering. Even if limited, understanding how things work and how they might fail, can help devise suitable solutions. Hence, the answer to how can we make sure something works is not set in stone. It is context-dependent and evolves with time as conditions and our understanding of the problem change.

When conditions are known and remain relatively steady, a simple benchmark might be enough to ensure something works. When conditions change, it may no longer apply. This is the case when using risk measures as yardsticks, including the values representing ‘acceptable’ and ‘tolerable’ risks. However useful these values may be, they are just a construct and not an absolute number. Would you rather live in a building with a risk of one in a million fatalities per year or in one with one in a billion fatalities per year? Do any of these rates mean the building is safe?

It might seem trivial to acknowledge that yardsticks such as GPA grow less useful with time, as we live in an ever evolving world where standards quickly become obsolete. In other fields, it is not trivial at all, as standards and acceptance criteria only change over many years and even decades. An example is fire safety engineering, a discipline driven by collective experience and societal response. Building codes evolve slowly, and in some countries they stand like monoliths, until large tragic events manage to rock their foundations, spark social outcry, and produce change. However, the most advanced codes have been looking into fire risk quantification as a means to demonstrate that a building is ‘safe’. If you can show that the risk metric of your design is under certain threshold, voila, the design is ‘safe’. One of the conclusions of my research is that such an approach is the product of either naivety or negligence. No yardstick, checklist or mechanized procedure will ensure a building will be ‘safe’ when a fire occurs. Such metrics can have a positive impact on performance and optimize costs, but believing that such an approach can demonstrate absolute ‘safety’ requires one to be dogmatic, not professional.

I am convinced that it is the understanding of the problem that yields the solution. Throughout my research, contained in this thesis, I argue that it is by understanding what fire safety performance means that you can show whether it is enough or not. Such understanding is unique for each building, which makes generic solutions unreliable and

4 An initial reflection

susceptible to misuse. Used for compliance, generic solutions like benchmarks can obscure the solutions.

My research proposes a solution to the problem of demonstrating fire safety performance. This solution is based on good engineering practices and focuses on understanding the problem, as well as in gaining useful insight; including those things we are not able to predict nor control. The solution, framed as a fire risk assessment methodology, aims at gaining insight. By gaining insight on potential fire risks, the problem can be better understood and solutions can arise.

So I return to my initial question: how do we make sure something works? I do not have a universal answer for this, so I will need to narrow it down to fire safety: how do we make sure a building is fire safe? Through understanding. Understanding how and to what extent the system can fail when a fire occurs is the gateway to a safer design. Through this understanding solutions that prevent, control or mitigate a fire can be identified. Understanding what we can and cannot control from those solutions, is also a key element to achieve safer designs. Since there are no limits to understanding, absolute safety results uncertain. As Cassius J. Keyser put it “absolute certainty is a privilege of uneducated minds and fanatics. – It is, for scientific folk, an unattainable ideal”. It is only through increasing our understanding, that we can achieve a safer built environment, even if it is not absolutely safe.

I thank you for your patience and for allowing me to try an articulate the immense value this project has for me, which has not only transformed me as a professional, but also as a person.

Bibliography 1. Bernold, L.E., Paradigm shift in construction education is vital for the future of our profession. Journal of Construction Engineering and Management, 2005. 131(5): p. 533-539. 2. Bernold, L.E., J.E. Spurlin, and C.M. Anson, Understanding our students: A longitudinal‐study of success and failure in engineering with implications for increased retention. Journal of Engineering Education, 2007. 96(3): p. 263-274. 3. Boleslavsky, R. and C. Cotton, Grading Standards and Education Quality. American Economic Journal: Microeconomics, 2015. 7(2): p. 248-79. 4. Chan, W., L. Hao, and W. Suen, A signaling theory of grade inflation. International Economic Review, 2007. 48(3): p. 1065-1090.

1 Introduction

Increasing complexity of buildings and the need to ensure they achieve fire safety objectives place larger responsibility on Fire Safety Engineering (FSE) practitioners. FSE practitioners are called to provide a science-based understanding of the building’s fire safety performance. Performance-based design in building design has increasingly required that FSE practitioners use risk assessments to demonstrate adequate performance and therefore safe buildings. Explicitly demonstrating this performance through scientific rigor allows overriding pre- defined rules for building design, dictated by prescriptive construction codes. In the context of this research, a risk assessment in FSE is understood as the procedure used to explicitly demonstrate an adequate performance. This is not the same use given to risk assessments in other disciplines, and this will be explored along this study as necessary. However, the central focus in the study is the role of risk assessment in providing an explicit demonstration of ‘adequate’ performance.

In FSE practice the concept of risk has been used as a proxy to quantify performance, which constitutes an important axiomatic assumption. This has led to the use of risk assessments to provide insight on fire risk and ultimately, guide decision-making. This work does not question this axiomatic assumption, but it does question the manner in which it is employed in practice, i.e. how risk assessments are conducted and for which purpose. Other pathways exist in FSE to demonstrate that a particular building design has adequate performance, but this scape the aforementioned assumption that risk enables quantifying performance. These include Deemed to Satisfy solutions and comparative studies; these are outside the scope of this work, but have been addressed in detail to show the dangers they introduce in ensuring the ‘safety’ of a design [1, 2].

Formally, risk assessment is a structured process to gain insight on the effects of uncertainty on achieving one’s objectives [3]. This process informs stakeholders about such uncertainty and its potential effects in several fields of engineering, as well as in public health, chemistry, governance, finance and emergency management. In the context of FSE, the insight produced by a risk assessment can represent the effect of fire on a building and its vulnerable elements (its operation, occupants and surroundings), and devise concrete design or operation actions to manage it.

The risk assessment methodologies used in FSE to demonstrate ‘adequate performance’ have been adopted from other disciplines (nuclear safety, chemical process safety, reliability

Chapter 1. Introduction

engineering) and are not directly fit for the purpose of demonstrating fire safety performance. In particular, a largely used approach to risk assessments is the quantitative or probabilistic approach. This approach centres on the use of metrics for the likelihood of consequences of different magnitudes to define an acceptable risk. A highly structured system where variability is closely monitored and controlled allows constructing a wealth of statistical information. This information is the basis that supports estimations about likelihood of diverse consequences and guide decision-making. However, the systems of interest in the built environment depart significantly from this description, i.e. buildings and infrastructure. Risk assessments, however, are not a compliance tool. Literature and history have demonstrated that conducting risk assessments does not ensure safety, but provide an understanding of the system, the way it performs and the alternatives to maintain an adequate performance as a function of the relevant objectives. It is relevant to notice that in other disciplines (e.g. chemical process safety) risk assessments do not constitute compliance tools and are not used to ensure, guarantee or demonstrate safety, but to provide insight on important risks.

This research work analyses the role of risk assessments in FSE and put forward the thesis that indeed, probabilistic risk assessments are not the best-suited methodology to characterize fire safety performance. This work systematically analyses the key elements of the probabilistic approach and identifies the key features impeding an adequate characterization of fire performance. Based on those features, this research proposes an alternative methodology and illustrates its application in a set of case-studies. The overarching objective of this research and the guiding research questions are introduced in Chapter 2. Before introducing the research questions, this chapter provides a general context on the role of risk assessments within the decision-making process. This context reveals some inherent limitations of risk assessments, which serve as the basis to understand the problematic of fire risk assessments and propose an alternative.

1.1 Risk assessment The process of understanding risk is known as risk assessment and lies at the core of the risk management process as defined by ISO 31000:2018 [3], see Figure 1-1. The process of risk assessment is divided in the processes of Hazards Identification, Risk Analysis and Risk Evaluation, which are then the basis for constructing risk treatment plans (also referred to as risk management actions); these plans are then subject of records and reports. The supporting elements of monitoring and revision allow ensuring the process is ongoing and to measure its effectiveness, while communication and consulting allow establishing dialog with relevant

Chapter 1. Introduction stakeholders. Figure 1-1 frames this process as a continuous cycle, starting at early design stages (pre-feasibility, feasibility, basic engineering) and finishing at decommissioning or even beyond that. This continuous process is necessary to improve the understanding of the risks that enable increasingly effective risk treatment actions. Understanding that all elements of the risk management process are essential, this research focuses on the risk assessment process. Table 1-1 provides a general definitions of each step of the risk assessment process.

Scope, context, criteria Monitoring Monitoring Risk Assessment consultation

Hazards identification & &

Risk Analysis Revision

Risk Evaluation

Communication Risk treatment

Recording & reporting

Figure 1-1. Risk assessment process as part of the risk management process [3]

Table 1-1. Definition of the risk assessment steps Step Description Systematic and comprehensive identification elements with the inherent potential to Hazards and cause harm, i.e. hazards, and development of scenarios through which these can lead to scenario undesired consequences. A scenario is understood as a sequence of event with a defined identification timeframe that end with the undesired consequences. Understanding of the risks and its characteristics, based on the risk definition chosen. Risk analysis Typically includes judging (qualitatively) or estimating (quantitatively) likelihood and consequences of particular scenarios. Process of comparison of the results of the risk analysis against pre-defined criteria to Risk evaluation determine whether they are acceptable, tolerable or inacceptable.

In practice, a large variety of risk assessment methodologies exist, all of which pose advantages and limitations [4-6]. In general, risk assessments can be classified as qualitative, semi- quantitative or quantitative. Each type of risk assessment is described and its main advantages and limitations are presented in Table 1-2.

Chapter 1. Introduction

Table 1-2. Approaches of Risk Assessment and their outcomes Type Description Advantages Limitations This approach describes -Typically less resource -Lack of supporting data and risks through subjective demanding methodological rigor for judgements categories such as -Allows screening hazards of probabilities and consequences ‘high’ or ‘low’ and and risks Qualitative -Uncertainty of the results can be relies in engineering -Provides a basis for more large and hard to describe judgement and expert detailed analyses such as -Results cannot be easily knowledge of those probabilistic risk benchmarked involved. assessments (PRA) [7] -Typically much faster and less resource demanding as Includes both qualitative a quantitative risk -Requires both expert judgement for and quantitative inputs assessment qualitative inputs and supporting and outputs, converting -Allows using data as well data for quantitative inputs Semi- subjective judgments as qualitative descriptions -The assumptions used to convert quantitative into numbers or to describe risk data into qualitative attributes and numerical data into -Consequence analysis can vice versa can significantly limit the qualitative judgments. be quantitative while applicability likelihood can be judged qualitatively -By definition, it does not use Describes risk in a -Provides the most detailed qualitative information, although in quantitative manner, description of risk practice this is impossible. typically using -Can use both -Relies on quality, appropriateness Quantitative scenarios, likelihood deterministic (single-point and completeness of supporting data and consequences values) and probabilistic -Does not describe relevant sources triplet. inputs. of epistemic uncertainty (e.g. high level assumptions)

This work particularly focuses on quantitative risk assessments, as these align with the mathematical language used in engineering, which are expected to favour a more objective understanding (measurement) of a system. Quantitative risk assessments can take either deterministic or probabilistic approaches. The former uses point-values for all variables involved and produces an exact and repeatable answer. Such an approach has been favoured for consequences quantification, while the associated likelihoods take a secondary role and are typically implicit. This has been recognized as a significant limitation of deterministic assessments, as exemplified by the Environmental Protection Agency of the US [8] and discussed in detail by Kirchsteiger [9]. To overcome this limitation, the latter approach

9 Chapter 1. Introduction

uses stochastic variables to represent key variables and parameters within a specific scenario. These stochastic variables are usually associated to the likelihood dimension, but are normally involved in consequence quantification. This stochastic approach provides a more detailed description of the variability and –to a certain degree- of the uncertainty of the assessment.

However, no risk assessment is able to fully capture the complexity of a real system, let alone one highly complex such as a building. Therefore, it is relevant to understand the context under which one would decide to perform a risk assessment to support decision-making. Ultimately, the risk insight provided by any risk assessment methodology is imperfect, but still it constitutes the basis for making real decisions that impact the design and the operation of the system. This means that it is more reasonable to understand the limitations of risk assessments and use them to support decisions, than assuming that conducting a risk assessment will guarantee adequate safety.

1.2 Decision-making Ultimately, a risk assessment derives in an action plan. In the context of Figure 1-1, this plan is the input to the risk treatment step, where stakeholders are prompted to decide on the available options. In FSE, these decisions influence the performance of the building when a fire occurs. In general, the possible decisions result in prevention, control and/or mitigation actions. Moller and Hansson [10] present 24 safety engineering practices and classify them in four principles (Figure 1-2), which characterize the expected effect of different risk treatment actions. In a building, limiting the fire load would correspond to a decision in line with inherent safety (i.e. reduction). A structural member with passive protection could increase the safety reserves. Self-closing mechanisms align with safe-fails, while the evacuation plan is part of the procedural safeguards.

Ultimately, decision-making defines the safety measures that will prevent, control or mitigate the effects of fire; in FSE these are typically framed as a fire safety strategy [11]. In this decision-making context, risk assessments can be used to provide supporting information about the nature of the risk and the effectiveness and appropriateness of the alternatives. The reason to use risk assessment is that the information it produces is expected to stem from scientific principles and professional practice. Herein lays the importance of producing fact-based risk insight and of the uncertainties involved.

10 Chapter 1. Introduction

Inherent safe design Safety reserves Fail safes Procedural safeguards

Minimization of inherent Additional resistance to Provisions for safe results Administrative means to hazards, e.g. technology design loads, e.g. safety despite failures, e.g. support safety, e.g. quality substitution factors reliability control

Figure 1-2. Principles for engineering safety [10]

For a moment, consider the risk assessment produces objective, value-free information. Stakeholders will consider this information (along with information stemming from other sources such as feasibility and compliance assessments[12]) influenced by their own values. Moreover, stakeholders will be influenced by their objectives and biases. The introduction of values enable approaching decision-making from a constructivist perspective, which contrasts with the science (fact-based) approach of positivism. In the former, human values have a central role in making decisions; in the latter, the scientific method primes and calls for the use of objective information and the decomposition of a complex problem into smaller (less complex) problems [1]. There is yet a third perspective, instrumentalism [13] (Figure 1-3). Instrumentalism uses empirical tools to further simplify complexities that typical scientific approach cannot describe, e.g. using a pre-defined temperature-time profile to analyse the effects of a fire in a building. Both positivism and instrumentalist are only applicable when the problem is value-free, which in FSE is not the case. The inclusion of values makes decision- making an inherently soft problem, one in which risk assessments are just one of many inputs considered.

Instrumentalism Useful predictions through selected (Not-science based) theory/experiments

Constructivism Positivism Decision-making (Values based) (Values-free) Risk as a social Technical reduction construct – ever of the complexities evolving solution of risk

Figure 1-3. Frameworks to decision-making

A typical decision process can be represented as in Figure 1-4, where an analyst team is in charge of constructing a fact-based risk picture that then informs decision-makers. From a positivist perspective, the success of the analysts’ task is to construct an objective risk picture that informs the stakeholders. However, such a framing can lead to overconfidence in the

Chapter 1. Introduction

analysts’ subjective judgments and underlying assumptions, which are not always clearly stated. Any ‘objective’ risk picture is actually influenced by the analysts’ values and biases. In a constructivist perspective, these potential issues are recognized from the start, which enable identifying and treating them along the decision-making process. Regardless of the approach, a successful outcome would produce a risk picture based on facts and clearly stated assumptions that best informs the decision process. This result is dependent on the competence of the analysts, their biases and the quality and completeness of information. Particularly in the FSE field, professional competence is a complex topic. Competence in FSE has been addressed by researchers [14-16], finding agreement on the need for further efforts to ensure competence of those involved in the risk assessment process.

Analysts Stakeholders

Risk definition Risk criteria

Knowledge Risk Decision Evidence Decision base assessment maker’s review

Fact-based Value-based

Figure 1-4. Adapted model for risk informed decision model [17]

The stages in the process of Figure 1-4 called ‘fact-based’ and ‘value-based’ are not entirely so. In practice, these stages lay somewhere in the spectrum between these two, as not purely objective knowledge can be produced, and decisions could –in theory- be made purely based on technical aspects. This process also introduces two key components of any decision-making process based on risk assessments: risk definition and criteria. The former sets the basis for what analysts understand as ‘risk’ and define the expected quantities from the assessment. In practice, choosing the risk definition could be one of the most important steps of the process, but it is seldom acknowledged. As shown in Chapter 5, risk assessment guidance tend to pre- define a risk definition without proper consideration of its implications. By defining risk and the expected metrics, the evaluation criteria also take a particular format, i.e. risk criteria. It is impossible to understate the importance of selecting a risk definition and criteria, as these structure the risk assessment and define the information to be communicated to the stakeholders (and that which is not). For now, the discussion of decision-making can be maintained independent of these two elements, which will be discussed in detail in Chapter 3.

Chapter 1. Introduction

The previous indicates that decision-making relies on both the risk picture constructed by the analysts and its interpretation by the stakeholders. Both of these are subject to biases when producing and interpreting trade-offs between risk treatment approaches [18]. The specific biases associated to the risk assessment process are presented and analysed in section 1.4. The question then is if risk assessment should be approached as a technical and even mechanical process, or as a value-based process. The strategies associated to these extremes are introduced in the following section.

1.3 Risk management strategies Managing risk can take on different strategies. The one supported mainly by risk assessments is a risk-informed strategy, which also considers other inputs such as analyses of economic or policy factors. In contrast, risk-based strategies focus on the risk results with little or no consideration of other inputs. An example of the latter would be risk based inspections for mechanical integrity management in process industries [19] or land-use planning around major hazard facilities [20]. These two strategies are supported by the outputs of risk assessments, requiring from it an objective and science-based insight. Risk-informed and risk-based are best suited for well-structured problems where a methodological rigor is expected from the analysis, with particular emphasis on objective and measurable knowledge.

Not all problems are structured enough to enable purely objective analyses. Consider the distinction of hard and soft systems. This distinction is usually applied in the field of systems thinking to distinguish between well-defined problems where the main concern is achieving objectives (i.e. Hard systems) and “ill-defined, dynamic multifaceted” problems (i.e. Soft systems) [21]. The former are usually addressed by exact sciences (e.g. mathematics, physics, chemistry) to obtain objective and exact answers, while the latter addresses complex multidimensional issues such as those involving human and societal components. In hard systems, the use of exact sciences allow for the analysts to be impersonal and to demonstrate rigor in their analysis, developing and improving models to support them. In contrast, soft systems involve components and relations that are hard to model following the same rigor used exact sciences. Soft systems address problems where an objective and exact answer might not only be hard to obtain, but might in fact fail to satisfy the parties involved. Ultimately, all systems are a mix of both hard and soft elements, which presents important challenges for risk- informed and risk-based strategies.

Chapter 1. Introduction

The relationship between the type of problem and the applicable sciences is conceptualized as a four quadrants or a ‘problem space’ (see Figure 1-5). This space characterizes soft problems where the goal is building trust and the solutions are provided by social sciences. Key characteristics of this quadrant is the relevance of values and biases, subjective judgements and ambiguity. Nevertheless, solutions to soft problems typically include the use of tools stemming from exact sciences and engineering. Hard problems stem from well-structured systems, where the goal is producing reliable information that often takes a quantitative form. Here, the elements constituting risk are well defined and engineering tools and exact sciences support addressing the problem. Nevertheless, hard problems can incorporate soft components such as stakeholder interaction, which if unrecognized or ignored can jeopardize the goals. The importance of this problem space is to help identify the mix of soft and hard elements present in most engineering problems, and realizing that no single approach will likely solve all of them. This is illustrated through two examples.

Complex large-scale problems like oil marine transportation in a vulnerable environment are a perfect example of the relevance of such a problem space. A review conducted by an expert panel on the risk assessment for oil operations in the Prince William Sound in Alaska, US highlighted several shortcomings [22]. One of the key observations was the lack of a general model, particularly one that included elements beyond the technological elements involved in oil transportation and analysed the influence of human error and managerial decisions in the operation. Oil operations in a vulnerable environment with the presence of multiple stakeholders, cannot be approached only as a hard problem and through engineering solutions. Another relevant example where an approach solely based on engineering does not suffice is community perception of nuclear risk close to power plants [23]. In this context, the results of technical analyses will rarely support successful communication between the operator and the community, while open discussions and inclusion of concerned parties in the decision and policy-making can result in a more effective approach.

Chapter 1. Introduction

Figure 1-5. ‘Problem space’ defined by the type of problem and applicable science

In answer to the challenges posed by soft elements in the decision-making process, alternative risk management strategies have been proposed. Liaropoulos et al. [24] present an analysis of the main risk management strategies beginning with the previously presented risk-informed. The alternative for ‘soft’ systems is known as risk governance, where the objective is no longer to aim at obtaining an objective measure to guide decision-making, but to build awareness, confidence and enhance communication across stakeholders to achieve consensus. The more soft elements are involved and the reliance on social sciences increases to manage them, discursive strategies tend to be favoured. Problems requiring technical decisions where the influence of soft elements in minimum or reasonably managed, tend to favour risk-informed strategies. Furthermore, the typical elements of risk management as defined by ISO 31000:2018 [3] can be plotted in the ‘problem space’ (Figure 1-6). The proportion of each one of these elements and their location in the problem space depends on the context and the disciplines involved, with those of Figure 1-6 being a generalized representation. Here, hazards identification is shown as an element sharing both hard and soft features, but treated through engineering tools; this relates to the subjective nature of the identification and the difficulties in defining scenarios, as previously mentioned and further discussed in Chapters 4 and 5. On the other end of the problem space is risk communication, which involves socio-technical

Chapter 1. Introduction discussions in which the aim is to build awareness and trust and effectively support the risk treatment plan.

Soft Discursive strategies

Risk communication Social Risk Hazards Engineering sciences evaluation identification Risk treatment

Risk analysis

Risk-informed Hard strategies

Figure 1-6. Main components of risk management plotted in the ‘problem space’

The exact position of each of the risk management components, as presented in Figure 1-6, depends on the application context and the unique features of the system being studied. Risk assessment tends to be mostly treated as a ‘hard’ problem, despite key elements like scenario identification remain composed by largely ‘soft’ components based on individual judgements, i.e. biases. Risk communication and treatment tend to have much larger ‘soft’ components when the system involves a large variety of stakeholders with different values and objectives. Understanding that risk assessment is not a purely ‘hard’ problem can help identify shortcomings of current methodologies.

An alternative risk management strategy exists, particularly relevant for contexts such as FSE. Given the complex fire dynamics and the uncertain behaviour of the physical component of the building and of the occupants, FSE poses large uncertainties when performing a risk assessment. In the absence of adequate data and background knowledge, a risk-informed strategy can be inadequate and a risk-governance one can be insufficient. Under these conditions it is relevant to consider cautionary or precautionary strategies. These are defined as those in which due to the presence of large (often not quantifiable) uncertainties or the

Chapter 1. Introduction

potential for large –unacceptable- impacts, robust controls are implemented. These controls can be safety measures, which is exemplified by the requirement of maintaining fire doors free of obstructions or fuel load of any kind. The precautionary strategy can also call for more drastic measures in the face of imminent large loss, exemplified by the shutdown of nuclear power plants in the wake of the Fukushima disaster [25]. Following this strategy is typically underpinned by the inability to provide stakeholders with a comparative assessment of the alternatives (e.g. through a risk assessment) [26].

In the built environment, a risk-informed strategy is preferred by default and so it is reflected by the requirements of the construction codes. However, this is not the only option, and some unique conditions could require FSE practitioners to contemplate alternative strategies. An example of such a situation is the worldwide problem of combustible facades, which is not only a technical problem, but it involves a wide range of vulnerable stakeholders. This problem cannot be solved solely through an engineering approach and require a vast investment in risk communication and policy-making.

This research is framed within the risk-informed strategy; however, it recognizes the importance of the precautionary principle in the presence of large uncertainties or unacceptable potential loss. The way to implement these strategies in a fire risk assessment is described in Chapter 6, where a new fire risk assessment methodology is proposed. To arrive at the proposed methodology, it is necessary to return to the inherent challenges of performing risk assessments in an engineering context. A key challenge is the –often-unaware- introduction of biases (subjective preferences) into the risk assessment, which has the potential to negatively affect the credibility and reliability of its results. The next section discusses how biases may impact the goal of fire risk assessments and sets the basis for formulating a series of research questions in Chapter 2.

1.4 Biases The technical nature associated to the design of a building and the need to comply with construction code requirements favours the use of the risk-informed strategy. However, there are important ‘soft’ elements that have a significant impact in the decisions made in the design of a building, particularly in the case of fire safety requirements. One of these elements is the role of biases of the FSE practitioners carrying out the risk assessments, as well as those of the stakeholders who ultimately decide on the alternatives.

Chapter 1. Introduction

The Merriam-Webster dictionary defines ‘bias’ as “a personal and sometimes unreasoned judgment” and several biases are continuously present in the day to day decision-making of all individuals. Subjective judgements can influence stakeholders’ interpretation of technical information resulting from a risk assessment, but are also relevant in the actual risk assessment process. The relation of biases with personal views and choices (opinion) make it difficult to explicitly identify them and address them in a highly technical process such as a risk assessment. This section focuses on understanding the potential effect of biases in the fire risk assessment process.

FSE practitioners have their own values and understanding of the world, which is conditioned by their knowledge and previous experience. When performing a risk assessment, these practitioners bring their biases into the process, possibly affecting it. Kinsey et al. [27] identified a set of relevant biases in the FSE practice, which serve as a basis to identify those that can have an active role in the risk assessment process. The relevant biases of conducting fire risk assessments are presented in Table 1-3, identifying how they can affect each of the steps of the process. These steps include those introduced in section 1.1 and add peer review, which is a formal process in which a third party checks for the quality and credibility of the assessment.

As observed from Table 1-3, the identified biases are mostly associated to the assumptions made when identifying hazards as well as setting up the basis for the risk analysis. Particularly, most of these biases affect the practitioner’s ability to identify and judge the applicability, advantages and limitations of scenarios, methods, models and/or values. Since these biases are inherent to human nature, they need to be acknowledged so that their effect on the risk assessment can be recognized and managed. Failure to do so can produce seemingly objective results that will not ensure achieving the objectives, e.g. ensuring life safety or maintaining structural integrity during a fire.

Table 1-3. Biases relevant to fire risk assessments *List adapted from Kinsey et al. [27] unless otherwise referenced Hazards and scenario Bias* Risk analysis Risk evaluation Peer review identification Identifying hazards Selecting methods, and scenarios based on models or values Familiarity past experience and based on past previous knowledge experience and

Chapter 1. Introduction

Hazards and scenario Bias* Risk analysis Risk evaluation Peer review identification previous knowledge Using previously Selecting typically known methods, used scenarios without models or values Default consideration of without further options consideration of further options Accepting the risk assessment Selecting methods, based on trust models or values Authority on the based put authority organization or or credibility individual that conducted it Selecting methods, models or values based on a judgement of how Risk aversion conservative they are or on how likely they are to yield a desired result Prioritizing scenarios or their components Probability neglect (e.g. initiating event, failure modes) based solely on consequences Disproportionately Selecting methods, Selecting risk focusing on particular models or values acceptance Availability scenarios or failure due based on criteria based on modes based on recency recency recency Selection of subjective Anchoring [28] probabilities or values based on

Chapter 1. Introduction

Hazards and scenario Bias* Risk analysis Risk evaluation Peer review identification adjustment of a representative value, e.g. expected value Sample sizes for Quantity of previous databases can Representativeness events can influence influence the use of [28] selection of scenarios the statistics reported Selecting methods, Selecting scenarios models or values based on a belief of based on belief of Control [28] control over its control, e.g. components, e.g. building design failure modes features Selecting risk Omitting issues Selecting methods, Limiting the acceptance with the risk models or values Incentive identification due to criteria based on assessment due due to time/cost time/cost constraints time/cost to time/cost constraints constraints constraints Unduly limiting the Limiting the information that information that supports the Confirmation supports scenario selection of selection methods, models or values Selecting Selecting traditionally typically used risk Selecting used scenarios, acceptance traditionally used Status quo omitting a systematic criteria without methods, models or and comprehensive considering their values hazards identification applicability or limitations Selecting methods, Adjusting risk Including or excluding models or values acceptance scenarios based on an Optimism/pessimism based on an criteria (for optimist or pessimist optimist or example using attitude pessimist attitude risk conversion

Chapter 1. Introduction

Hazards and scenario Bias* Risk analysis Risk evaluation Peer review identification factors, see Chapter 5.3) based an optimist or pessimist attitude Focus on the conditions and actions that have triggered Survival recorded fires, while ignoring other potential contributors. Defining scenario clusters based on arbitrary attributes, Clustering without considering other relevant attributes

How are biases introduced in risk assessments? Risk assessments often require reducing the complexity of the system by decomposing it into sub-systems and these further down to components. This (positivist) approach can result in loss of important information related to the context of the problem, introducing uncertainties which are hard to recognize and describe [29]. Assumptions of tractability [30] and independence between variables exemplify this approach. Tractability enables decomposing the system into more manageable sub-systems, which in the context of risk assessments is used to model the system. In the context of FSE and fire risk assessments, tractability allows reducing the complexity of the interface between fire dynamics, the building and its occupants into less complex phenomena. Ultimately, this exercise in modelling the system by reducing its complexity leads to the identification of hazards and fire scenarios. This is alike to the process of generating alternatives, which Lindell [31] identifies as one of the key steps in decision-making affected by biases.

Another path for biases to enter a risk assessment is the process of estimating the likelihood of a particular scenario or of its components. When significant statistical data or experimental observations are not available, practitioners are called to quantify their degree of belief in form of probabilities. Lindell [31] details different mental processes (such as memories retrieval) involved in producing these degrees of belief. The selection of these values can be influenced

Chapter 1. Introduction

by the frequency with which they occur, recency, salience and memorability, as well as by the time constraints (see Table 1-3).

Biases can also be embedded in the limitations and assumptions of models used to describe the physical phenomena, i.e. fire modelling [14, 32]. Fire is one of the most complex processes known to man and the available models to describe its behaviour present significant assumptions and limitations which are not always identified [33]. However, it is still the responsibility of competent FSE practitioners to identify such limitations and understand their potential impact on the results of a risk assessment. A relevant study carried out in the context of this work analysed the uncertainty of fire simulations using the Fire Dynamics Simulator (FDS v6) as it is currently one of the most versatile and popular fire models available and used by practitioners – further information is provided in Appendix 2.

The varying degree of competencies of the analysts can also introduce biases into the risk assessment. This is particularly relevant to FSE, where the level of competencies practitioners has been called into questions [34], along with the basis of the discipline as a whole [1]. Competencies in FSE are built upon a wealth of knowledge of fire dynamics, as well as expertise on the guidance, methodologies, techniques, tools and data used in technical analyses. However, due to the complexity of the fire phenomena itself and the disparity in education and professional experiences of FSE practitioners, biases can make their way into the assessment. The issue of competencies can be partially managed by setting an adequate educational framework [35, 36].

1.5 Conclusions

This introduction highlights that a direct relation between safety and risk is not self-evident. What this context does support is that risk insight can support decision-making, i.e. inform key stakeholders in the light of key decisions. In theory, the more rigorous and objective this insight is, the better decisions could be supported. This makes it a challenge to categorically state that a favourable risk assessment signifies an adequate performance. To a large extent, this will depend on the definition of risk used, the acceptance criteria used and the key elements of the process to assess risk briefly mentioned in this section.

The apparent objectivity and the methodological rigor associated to quantitative and probabilistic risk assessments have been key points for their proliferation across disciplines, including FSE (section 1.1), and it is a key point supporting its use to demonstrate performance. However, numerical results by themselves are not inherently objective nor demonstrate

Chapter 1. Introduction methodological rigor. The results of a quantitative risk assessment can be knowingly or unknowingly affected by the biases discussed in section 1.4, as it requires important subjective judgements from FSE practitioners [37]. The fact that this type of risk assessment produces detailed quantification does not ensure safety. It also does not ensure that decision-makers can adequately interpret the resulting numbers and their underlying uncertainty. Reniers [38] discusses how risk assessment outputs “cannot be considered as tools for accurate prediction” except in systems with a high degree of simplicity. The author agrees and also puts forward the challenge of defining what ‘accurate’ and ‘simple system’ mean.

The previous calls for an approach to risk assessment that does not focus on getting ‘the right answer’ but on gaining insight and understanding its limitations. Such an approach seems to fit the reflection of Spinardi [39] about the use of fire risk assessments to define the management needs of a building. Risk assessments should not reveal that a system is ‘safe’ but account for the ways in which it is known to be unsafe and the necessary actions to manage them. This clashes with typical practice in FSE, exemplified by Dame Judith Hackitt’s statement about the main motivation of the UK’s: “to do things as quickly and cheaply as possible” [40]. This often means to focus on compliance with the regulatory regime, rather than in producing useful risk insight, which is not only an issue in FSE but in other disciplines such as chemical process safety [41].

An approach based on compliance and maximizing economic savings, does not benefit from a critical analysis of risks. Instead, such an approach can benefit from a mechanistic approach to risk assessments that helps achieve compliance. The danger of mechanized approaches in producing risk insight will be exposed throughout this study. The first step in doing so was introduced in this first chapter, which describes how risk assessments are inherently subjected to soft elements and biases. This is particularly relevant to FSE, where the soft components introduced by occupants and their response to fire can only be quantified conditioned by large and very relevant uncertainties.

Chapter 1. Introduction

Bibliography 1. Gehandler, J., The theoretical framework of fire safety design: Reflections and alternatives. Fire Safety Journal, 2017. 2. Bjelland, H. and A. Borg, On the use of scenario analysis in combination with prescriptive fire safety design requirements. Environment Systems & Decisions, 2013. 33(1): p. 33-42. 3. Standardization, I.O.f., ISO 31000:2018: Risk management - Principles and guidelines. 2018: Geneva, Switzerland. 4. Tixier, J., et al., Review of 62 risk analysis methodologies of industrial plants. Journal of Loss Prevention in the Process Industries, 2002. 15(4): p. 291-303. 5. Marhavilas, P.K., D. Koulouriotis, and V. Gemeni, Risk analysis and assessment methodologies in the work sites: On a review, classification and comparative study of the scientific literature of the period 2000–2009. Journal of Loss Prevention in the Process Industries, 2011. 24(5): p. 477-523. 6. Khan, F.I. and S.A. Abbasi, Techniques and methodologies for risk analysis in chemical process industries. Journal of Loss Prevention in the Process Industries, 1998. 11(4): p. 261-277. 7. (CCPS), C.f.C.P.S., Guidelines for chemical process quantitative risk analysis. 2000: The Center. 8. (USEPA), U.S.E.P.A., Risk Assessment Forum White Paper: Probabilistic Risk Assessment Methods and Case Studies, U. Office of the Science Advisor, Editor. 2014: Washington, D.C., United States. 9. Kirchsteiger, C., On the use of probabilistic and deterministic methods in risk analysis. Journal of Loss Prevention in the Process Industries, 1999. 12(5): p. 399-419. 10. Möller, N. and S.O. Hansson, Principles of engineering safety: Risk and uncertainty reduction. Reliability Engineering & System Safety, 2008. 93(6): p. 798-805. 11. Cowlard, A., et al., Fire Safety Design for Tall Buildings. Procedia Engineering, 2013. 62: p. 169-181. 12. Thierry, M., R. Genserik, and C. Valerio, Risk Management and Education. 2019, Berlin, Boston: De Gruyter. 13. Bjelland, H., et al., The Concepts of Safety Level and Safety Margin: Framework for Fire Safety Design of Novel Buildings. Fire Technology, 2015. 51(2): p. 409-441. 14. Beard, A.N., Requirements for acceptable model use. Fire Safety Journal, 2005. 40(5): p. 477-484.

Chapter 1. Introduction

15. Meacham, B.J., Fire safety engineering at a crossroad. Case Studies in Fire Safety, 2014. 1: p. 8-12. 16. Maluk, C., M. Woodrow, and J.L. Torero, The potential of integrating fire safety in modern building design. Fire Safety Journal, 2017. 88: p. 104-112. 17. Aven, T., Risk assessment and risk management: Review of recent advances on their foundation. European Journal of Operational Research, 2016. 253(1): p. 1-13. 18. Lindell, M.K., Chapter 18 - Judgment and Decision-Making, in Laboratory Experiments in the Social Sciences (Second Edition), M. Webster and J. Sell, Editors. 2014, Academic Press: San Diego. p. 403-431. 19. Institute, A.P., Recommended Practice for Risk-based Inspection. 2016: Washington D.C., USA 20. Cozzani, V., et al., Application of land-use planning criteria for the control of major accident hazards: A case-study. Journal of Hazardous Materials, 2006. 136(2): p. 170- 180. 21. Checkland, P., Systems Thinking and Soft Systems Methodology. The Oxford Handbook of Management Information Systems: Critical Perspectives and New Directions, 2011. 22. Review of the Prince William Sound, Alaska, Risk Assessment Study. 1998, Washington, DC: The National Academies Press. 78. 23. Xiao, Q., H. Liu, and M.W. Feldman, How does trust affect acceptance of a nuclear power plant (NPP): A survey among people living with Qinshan NPP in China. PLOS ONE, 2017. 12(11): p. e0187941. 24. Liaropoulos, A., K. Sapountzaki, and Z. Nivolianitou, Adopting risk governance in the offshore oil industry and in diverse cultural and geopolitical context: North Sea vs Eastern Mediterranean countries. Safety Science, 2019. 120: p. 471-483. 25. Kramm, L., The German Nuclear Phase-Out After Fukushima: A Peculiar Path or an Example for Others? Renewable Energy Law and Policy Review, 2012. 3(4): p. 251- 262. 26. Smith, C. and C. Curtis, The precautionary principle and environmental policy: science, uncertainty, and sustainability. Int J Occup Environ Health, 2000. 6(4): p. 263- 5. 27. Kinsey, M., M. Kinateder, and S. Gwynne, Burning Biases: Mitigating Cognitive Biases In Fire Engineering. 2019.

Chapter 1. Introduction

28. Abrahamsson, M., Uncertainty in Quantitative Risk Analysis – Characterisation and Methods of Treatment in Division of Fire Safety Engineering. 2002, Lund University: Lund, Sweden. 29. Armstrong, J., Positivism and Constructivism, in Improving International Capacity Development. 2013, Springer. p. 28-37. 30. Reason, J.T., Human error. 1990, Cambridge [England]; New York: Cambridge University Press. 31. Lindell, M., Chapter 18: Judgment and Decision Making. 2014. p. 403-431. 32. Lundin, J., Model Uncertainty in Fire Safety Engineering. 1999, Lund University. 33. Torero, J.L., Scaling-Up fire. Proceedings of the Combustion Institute, 2013. 34(1): p. 99-124. 34. D. Lange, J.T., C. Maluk, J. Hidalgo, Fire Safety Engineering - Competencies Report, T.W. Centre, Editor. 2019: Sydney, Australia. 35. Woodrow, M., L. Bisby, and J.L. Torero, A nascent educational framework for fire safety engineering. Fire Safety Journal, 2013. 58: p. 180-194. 36. Spinardi, G., Fire safety regulation: Prescription, performance, and professionalism. Fire Safety Journal, 2016. 80: p. 83-88. 37. Rozell, D.J., Values in Risk Assessment, in Dangerous Science. 2020, Ubiquity Press. p. 29-56. 38. Goerlandt, F. and G. Reniers, Prediction in a risk analysis context: Implications for selecting a risk perspective in practical applications. Safety Science, 2018. 101: p. 344- 351. 39. Spinardi, G., J. Baker, and L. Bisby, Post construction fire safety regulation in England: shutting the door before the horse has bolted. Policy and Practice in Health and Safety, 2019. 17(2): p. 133-145. 40. Hackitt, J., Building a Safer Future, Independent Review of Building Regulations and Fire Safety: Final Report. 2018: London, UK. 41. Rowe, R., et al., How Safety Cases and Bowties Can be Used to Improve Safety, in SPE African Health, Safety, Security, Environment, and Social Responsibility Conference and Exhibition. 2016, Society of Petroleum Engineers: Accra, Ghana. p. 10.

2 Research approach

Chapter 1 presented a series of limitations associated to the use of risk assessments as a mean to explicitly quantify and evaluate the fire safety performance of a building. These limitations undermine the use of quantified risk assessments to justify the approval of a building design. Such an idea seems to be a pragmatic way of dealing with the problem, particularly given the large body of research and practice supporting it found in disciplines such as nuclear and chemical process safety. Nevertheless, risk assessments are not an entirely objective description of the system, as it is based on subjective judgments that differ between analysts and stakeholders. The introduction also established that generating insight on risk and communicating it to relevant stakeholders is not straightforward. Chapter 1 put forward a set of high-level conclusions associated to the subjective basis of any risk assessment, which has been clearly recognized by research dealing in risk perceptions [1].

The focus of this research work is the technical methodology to perform risk assessments in fire safety engineering (FSE). This work has the overarching objective of constructing an alternative methodology for fire risk assessment from first principles that addresses some of the limitations associated with existing approaches. This objective has led the author to take an exploratory journey from the conceptual basis of risk assessment set in the 1970s, passing through the technical developments of quantitative and probabilistic risk assessments between 1980 and the present date, and finally through the specific challenges found in the context of FSE. In order to guide this exploratory journey a set of case studies were used, which will be introduced in Chapters 6 and 7. These case studies allowed using currently accepted fire risk assessment practices, identifying their limitations and using them as a basis to build and evaluate the new methodology. Although in practice the case studies guided the research process, conceptually it is framed by a series of research questions.

2.1 Research questions and claims In general, a risk assessment aims at providing insight on two dimensions: probability and consequences of a defined set of scenarios. Practitioners with critical thinking should stop and think about the capabilities available to gain insight on each one of these dimensions. The former is probability, which can be addressed as likelihood, i.e. the chances of a scenario taking place or a risk materializing. Gaining insight on likelihood largely relies on information about past behaviour or in its absence, on the degree of belief judged by competent professionals.

27 Chapter 2. Research approach

This reliance on data or subjective judgement presents a challenge, as the insight gained from likelihood will not produce a prediction if the future conditions differ from those recorded in the past or if there are relevant limitations with the data (quality, completeness, age). When determining the likelihood of a particular event (typically quantified in terms of frequencies, i.e. occurrence per unit of time), predictions will only be as good as the supporting data is.

Modern engineering systems are inherently complex and their correct operation is the result of decisions made during throughout its whole life cycle. These decisions involve uncertainty and the need to make a judgement on both the likelihood and the consequences of a failure of the systems. Nevertheless, there is an important difference when it comes to these two dimensions. Likelihood of undesired rare events are unpredictable as it can be attributed to foreseen and unforeseen conditions, as well as to a combination of the two. These conditions can be intrinsic or external and a judgment on their occurrence and the effects they have on the system is not always known a priori. An unlikely failure can take place at any point of the life cycle as a product of unforeseeable set of conditions and actions. It is granted that highly structured and controlled systems allow characterizing their failure likelihood as in the case of safety instrumented systems, which are design, produced and tested to provide a known level of reliability [2]. But even highly controlled systems can fail unpredictably, despite high reliability rates and redundant administrative and physical layers of protection, as exemplified by the issues surrounding the design and operation of the Airbus 737 Max airplane [3].

In the construction sector, there are also examples of mistakes, errors, miscalculations or simple omissions that lead to latent hazards which eventually lead to the realization of risks. If systems can fail, despite how reliable they are deemed, it is imperative that the consequences of their failure are properly characterized and managed. Evidently, failure is inherent to technology and furthermore, to human nature. Society is willing to accept some risks and their consequences, as these bring about benefits and advancement to civilization. One of the most important risks in the built environment is fire risks. Probabilistic risk assessments (PRA) have been identified as an appropriate method to manage fire risks, in which acceptance of risk is given by the magnitude of consequences and of likelihoods. In this probabilistic framework, catastrophic loss can be accepted if its associated likelihood is deemed negligible (see Chapter 3). However, recent fire events have made it clear that large consequences such as fires involving multiple fatalities are unacceptable and lead to outcry and the scrutiny of the applicable technical and legal frameworks. Although multiple small fire losses are accepted by society, single events

Chapter 2. Research approach resulting to catastrophic loss is categorically rejected by society (see section 3.4.2). The previous leads to the first research question of this work:

Research question 1. Are PRAs the adequate method for explicitly assessing fire safety performance?

Chapter 1 sets a broad conceptual basis to answer this question, which then leads the way to present how PRA works in the context of FSE. Chapter 3 presents a typical PRA and its use to verify an ‘adequate fire safety’ allowing to explore its conceptual basis as well as each of its procedural steps. The analysis presented in both of this chapters aim at providing evidence to support the following claim:

Claim 1. Demonstrating adequate fire safety in the built environment cannot be based on both consequences and likelihoods, as this can lead to accepting intolerable consequences.

As any tool, a particular risk assessment methodology can be useful for the right application when applied by competent users. In FSE, the requirements of the construction codes define the need to evaluate fire safety performance on an absolute quantitative basis and demonstrate compliance against a set of performance objective. This need implies that the output of a PRA must be not only precise enough to demonstrate an adequate performance, but that it should also be trustworthy. The uncertainty involved in any risk assessment is a key challenge to demonstrate both precision and trustworthiness. However, the uncertainty present in a risk assessment does not necessarily reduce its quality or its capacity to inform decision-making [4]. Understanding and managing uncertainty is particularly relevant in the FSE context, where the capacity to monitor and control many variables affecting fire risk is simply not available as in the case of a process plant. The key then is understanding uncertainty as an additional layer of information resulting from the risk assessment and capitalizing on it once it has been reduced to the largest extent possible within the constraints of the assessment. This raises a second research question:

Research question 2. What are the main challenges of characterizing uncertainty in fire risk assessments?

This research work has addressed different elements of uncertainty associated to a fire risk assessment to answer this question. First, the alternatives available to characterize uncertainty, particularly those related to likelihood, have been addressed in a review paper published in the

29 Chapter 2. Research approach

Journal of Loss Prevention in the Process Industries, see Appendix 1. The uncertainty associated to fire dynamics is quite relevant, as it directly affects the precision of the consequence judgments. In FSE practice computational models are typically used to characterize fire dynamics, resorting to important simplifications and assumptions. These often translate into model parameters that need careful consideration and in certain conditions become unknown (e.g. under ventilated fires in which the heat of combustion, radiative fraction and soot yield are conditioned by the incomplete combustion). The uncertainty associated to input parameters in fire modelling was studied using the most popular fire model publicly available and a robust experimental dataset. The model is a computational fluid dynamics (CFD) model and uncertainty resulting from compartment fires has been studied in a paper (submitted). This paper can be found in Appendix 2 to this document.

The second research question has been further explored in the context of performing a PRA in Chapters 4 and 5. Chapter 4 explores the implications of trying to adopting (instead of adapting) a PRA to fulfil the requirements of FSE, taking into account that it is originally found in disciplines such as chemical process safety, particularly for major hazardous facilities. Chapter 5 comparatively evaluates the available guidance on fire risk assessments with a particular focus on uncertainty management. This body of work has led the author to the following claims:

Claim 2. Probabilistic risk assessments pose inherent limitations such as the dependency on data and on a subjective basis for scenario identification, which could impede producing trustworthy insight on the fire safety performance.

Claim 3. It is necessary to describe the maximum damage potential of a building fire, i.e. consequences, and characterize the associated uncertainty to promote inherently safer designs. Moreover, insight on inherent risk can be the basis for progressing into a PRA for selecting safety measures.

Focusing on the consequences is a necessity of the application, i.e. the need to ensure an adequate fire safety performance. Likelihood information can contribute to the decision- making process, particularly when it comes to alternative evaluation and selection. However, insight on consequences has the outmost priority in guiding the design process of a building. Understanding fire consequences helps designing building features that directly influence the fire dynamics (e.g. ceiling height, natural ventilation, construction materials and compartmentation) and its effect on building occupants, the surroundings and the fire service. Regardless of the likelihood, fire safety performance is ultimately a matter of how the build

30 Chapter 2. Research approach

respond to the occurrence of a fire. This is the central argument for why a consequence-driven risk assessment is necessary, i.e. Claim 3.

A risk assessment focused on consequences can be deterministic or probabilistic, as the only difference is the type of inputs and the number of realizations made. A deterministic consequence-driven risk assessment does not provide a full picture of fire risk, but it does provide clear insight on the limits of fire safety performance. Furthermore, such an analysis provides a robust basis to perform probabilistic analyses focused on providing insight on likelihood and on the impact of different safety measures. Based on the previous, a third and final research question emerges:

Research question 3. How can a consequence-driven fire risk assessment guide the design process and achieve an adequate performance?

To answer this final research question, Chapter 6 presents the basis for an alternative methodology. This basis is used to structure the concept of Maximum Allowable Damage, as an acceptance criterion guiding the design of a building towards an inherently safer design. This concept is then presented as a series of steps that make up the fire risk assessment. A case study of a multi-occupancy office building is used in Chapter 6 to exemplify each one of the steps and the outputs of the methodology. The proposed methodology is then evaluated based on the same criteria used in Chapter 5 to identify shortcomings, limitations and improvement opportunities. Finally, Chapter 7 presents the application of the methodology to two additional systems. The first one a high-rise residential building with a non-compliant façade for which a remediation strategy is required. The second one a chemical storage warehouse where the focus shifts to property protection. After proposing the alternative methodology and applying it to case-studies, the following claims are made by the author:

Claim 4. Recognizing the boundaries of validity of a fire risk assessment enhances trustworthiness and provides a basis for peer review, improvement and auditing.

Claim 5. A building’s fire performance changes throughout its life cycle, requiring monitoring it and feeding it back to the fire risk assessment in order to effectively manage it over the life cycle.

The research will provide arguments and evidence to answer the three research questions. Furthermore, these arguments and evidence will also provide support to the claims. Concerning the claims, it is acknowledged that no risk assessment methodology is perfect, and that much

31 Chapter 2. Research approach

depends on how it is implemented. Therefore, Chapters 8 and 9 will present the conclusions of this work, as well as describe the degree to which each claim has been supported. For any claim that is not fully supported, the author will propose an action plan for future research. The conclusions will also provide the opportunity to reflect on what this work as a whole has been able to accomplish and what is needed from future research.

2.2 Research outputs This research work produced relevant products, in the form of scientific communications and journal papers. The author was invited to present (poster format) in the 16th Loss Prevention Symposium, which was hosted by TU Delft at Delft, The Netherlands on June 2019. This participation led to the first journal paper and to an invitation to publish an additional paper in the Journal of Loss Prevention in the Process Industries.

The full list of papers produced during the course of this research is presented in Table 2-.

Table 2-1. Papers produced in the course of this research Paper Title Authors Type Place published Accounting for Uncertainty in V. Gupta, J. 12th International Computational Fluid Cadena, J. P. Performance-Based Codes Conference 1 Dynamics Modelling of Hidalgo, A. Osorio, and Fire Safety Design proceedings Fully-Developed C.Maluk, A. H. Methods Conference Compartment Fires Majdalani (SFPE) Overcoming Risk Assessment J. Cadena, J. Limitations for Potential Fires Hidalgo, C. Maluk, Chemical Engineering 2 Journal paper in a Multi-Occupancy D.Lange, J. L. Transactions Building Torero, A. F. Osorio Uncertainty-based decision- J. Cadena, A. Journal of Loss Prevention 3 making in fire safety: Osorio, J. L. Torero, Journal paper in the Process Industries analysing the alternatives G. Reniers, D. Lange J. Cadena, V. Uncertainty analysis in the Gupta, A. H. Building Simulation 4 modelling of fully-developed Journal paper Majdalani, J. L. (submitted) compartment fires using FDS Torero, J. P. Hidalgo J. Cadena, M. Fire risk assessment for a McLaggan, A. F. Fire Safety Journal 5 non-compliant façade in a Journal paper Osorio, J. L. Torero, (under review) high-rise building D. Lange

32 Chapter 2. Research approach

Bibliography 1. Tancogne-Dejean, M. and P. Laclémence, Fire risk perception and building evacuation by vulnerable persons: Points of view of laypersons, fire victims and experts. Fire Safety Journal, 2016. 80: p. 9-19. 2. Brazendale, J. Iec 1508: Functional safety: Safety-related systems. in Proceedings of Software Engineering Standards Symposium. 1995. IEEE. 3. Herkert, J., J. Borenstein, and K. Miller, The Boeing 737 MAX: Lessons for Engineering Ethics. Science and Engineering Ethics, 2020. 4. Roberts, L.E.J. and M.R. Hayns, Limitations on the Usefulness of Risk Assessment. Risk Analysis, 1989. 9(4): p. 483-494.

33 3 Probabilistic risk assessment in FSE

Everything in nature continuously changes and experiences natural variations. This extends to complex technological systems, a description that could be applied to many (if not most) buildings. Fire safety performance of buildings is therefore also susceptible to these permanent variations. Fire safety performance is understood the degree to which the safety objectives required by the construction code and other stakeholders are achieved when a fire occurs, e.g. ensuring life safety after the occurrence of a compartment fire.

In the performance-based design process of a building, technical analyses are employed to judge fire safety performance (section 4.5 describes this process in detail). The outcome of this judgement is critical not only for obtaining an eventual approval from the Authority Having Jurisdiction (AHJ), but to determine the investments required for the fire safety strategy of the building. To support such a judgment, risk assessments are often employed, particularly for complex or unique designs. As discussed in Chapter 1, risk assessments can range between qualitative and quantitative. Although qualitative judgements are unavoidable in the process of assessing risk, this research work is mainly focused on the methodologies to quantify risk in the context of FSE. Quantitative analyses can be further divided in deterministic and probabilistic analyses, i.e. based on their quantification approach.

This chapter focuses on the use of probabilistic safety assessments in the context of FSE. It is not the premise of this chapter to present the workings of a PRA in detail, although this is done to a large extent. Ramachandran [1] provides an accounting of the basis to perform PRAs in FSE, Sabapathy et al. [2] provides an example of a PRA to an apartment building, Johansson [3] uses PRA to quantify the risk associated to deemed-to-satisfy designs and Gernay et al. [4] provides an example of PRA in the context of fire structural engineering. In general, these applications are all of PRAs, but this chapter will present that subtle differences in risk definitions, criteria and purpose of the risk quantification, make of PRAs a complex methodology.

The reader is directed to the work of Aven [5, 6], who presents a robust and scientific basis for PRAs. This chapter will focus on the ability and the limitations of using PRAs to demonstrate an adequate fire safety performance. The complexity of this topic is broken down in a number of levels, which ultimately remain interconnected. These levels include:

• Deterministic and probabilistic quantification (section 3.1)

34 Chapter 3. Probabilistic risk assessment in FSE

• Probabilistic fire risk assessments (section 3.2) • Verifying safety through PRA (section 3.3) • ‘Adequate safety’ (section 3.4) • Risk acceptance criteria (section 3.5) • Limitations of scenario identification (section 3.6) • Supporting data (section 3.7) • Guiding the design process (section 3.8)

After discussing these elements, the advantages and limitations of PRAs in FSE are summarized in section 3.9. This chapter discusses that the outputs of a PRA can potentially reflect the performance of the system or at least aspects of it, but that it could also produce misleading risk insight. Such misleading insight could secure compliance, but would not ensure adequate performance. Partly, this is a result of the imperfect nature of the models, assumptions and data used in the risk assessment. However, intentional or unintentional misuse of the tool and the available data can also result in this undesired and dangerous result. This chapter aims to provide the evidence for why the previous is a valid concern and what it implies for the future of fire risk assessments.

3.1 Deterministic and probabilistic quantification A probabilistic analysis is based on the availability of statistical information about a relevant variable, which is then used to produce a probability distribution of the output of interest. An example is the number of casualties and fatalities resulting from a fire, and a key input to estimate it is the fuel load in the compartment. If statistical data is available on typical fuel in compartments similar to the one being studied, then this data can be used to generate a probability distribution of the fuel load and then be sampled to obtain a probabilistic expression of the output, as illustrated in Figure 3-. Such an output provides stakeholders with statistical confidence that the decisions made are adequate. However, the trustworthiness of the resulting probabilistic expression is conditioned by how complete, valid and relevant the underlying data employed is.

Chapter 3. Probabilistic risk assessment in FSE

Figure 3-1. Simple representation of the result of a deterministic and a probabilistic analysis

In situations where statistical data on key variables does not exist, are significantly limited or inadequate, deterministic analysis present an alternative. A deterministic calculation omits statistical variations and instead typically uses conservative values for key variables to define a scenario an estimate the fire safety performance. If recalculated, the result will be the same, differing from probabilistic analyses (as the sampled values can vary between calculations). The previous could indicate that deterministic analyses do not provide insight on variability. It also leads to the idea that deterministic methods are simplistic and lack certainty, robustness and completeness [7] and that their degree of conservatism cannot be strictly defined nor compared [8]. This criticism could impede using deterministic analyses as a means to choose between alternatives and even optimal solutions. However, deterministic analyses can be coupled with sensitivity analyses to construct a picture of variability and to provide an understanding of how conservative the results are. The issue is defining what variables to explore and to what degree; this will be further explored in Chapter 6. In the context of FSE, the result of a deterministic analysis typically quantifies the effects of fire (e.g. radiative heat from a fire) or its consequences on a particular target (e.g. degree of burn to a human due to exposure to heat). This can be represented in the consequence scale of Figure 3-2, in which the selection of the fuel load and its limits (a, b) needs to compensate for the lack of statistical information.

36 Chapter 3. Probabilistic risk assessment in FSE

Figure 3-2. Simple representation of the result of a deterministic and a probabilistic analysis

In a probabilistic analysis in which the distributions of key inputs are modelled as stochastic variables, each realization will lead to a unique result. The whole of realizations will provide a probabilistic distribution for the results. Returning to the presented example, if the range for the fuel load is assumed to be uniformly distributed between values a and b, then the resulting distribution would be as in Figure 3-3. This figure shows that now the consequence range is covering results between one and multiple fatalities and that no other result is possible. Hence, this sampling allows to form a more complete understanding of the possible consequences, providing a range and also a judgement of its likelihood. In this particular example, any of these results is equally likely within the range (a, b) for fuel load. In this case, the probabilistic analysis yields the same result as a deterministic calculation, as all results are equally likely. When the input distributions are not uniform, this is no longer the case and careful fitting or selection of the distributions and their sampling is required. Ultimately, if appropriate statistical data exists, it is relevant to consider the use of probabilistic analyses; section 3.7 discusses what constitutes appropriate supporting data.

Figure 3-3. Probability consequence diagram for the example

37 Chapter 3. Probabilistic risk assessment in FSE

In the absence of statistical data, deterministic analyses (coupled with sensitivity analysis) can provide an adequate insight on variability, which is key to fire risk assessments. However, probabilistic quantification provides insight not only on the result itself, e.g. a measure of the consequences, but also on its likelihood. This is the main difference between these two types of quantification. By providing insight on likelihood, events with low frequencies or probabilities could be deemed negligible based on a pre-defined criterion. This has a significant effect on the decision-making process, one that can be beneficial, but also misleading. Before further understanding this issue, the application of probabilistic analysis to fire risk assessments is discussed in the next section.

3.2 Probabilistic fire risk assessments Probabilistic risk assessments (PRA or quantitative risk assessment, QRA) are based on the quantitative risk definition, where risk is a function of the triplet scenario, probability and consequences [9]. Eq. 1 represents the mathematical approach to risk quantification in a PRA in the context of FSE, where a frequency of fire occurrence ( ) multiplies a function made up of the probability of a particular scenario occurring ( ) and𝐹𝐹𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 its consequences ( ), for all identified scenarios (i = [1, N]). 𝑝𝑝𝑖𝑖 𝑐𝑐𝑖𝑖 The scenarios are derived from hazard identification analyses (e.g. event tree analysis). These techniques are typically based on workshops where a facilitator guides subject matter experts in the identification of causes and conditions leading up to undesired results [10]. At the time of writing, it is not clear how these techniques are thoroughly implemented in the context of FSE, despite being recommended [11].

Estimating probabilities and consequences for each identified scenario require diverse inputs. Those inputs for which variability is inherent such as occupant behaviour, fire growth rate or suppression activation are described as stochastic variables represented by probability distributions. Sampling these distributions allows estimating the consequences for each scenario, which if aggregated describe a probability distribution.

= ( ) (1)

𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 𝑖𝑖 𝑖𝑖 The outputs of a PRA are a 𝑅𝑅𝑅𝑅𝑅𝑅measure𝑅𝑅 𝐹𝐹 of loss/time∙ 𝑓𝑓 𝑝𝑝 ∙ 𝑐𝑐, which result in a simple and easy to communicate result for comparison and evaluation purposes. This format is achieved due to the involvement of the term, which must be constructed from statistical analysis of the

building/system under study𝐹𝐹𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 or similar ones. Depending on the form of the function to account

38 Chapter 3. Probabilistic risk assessment in FSE

for the probabilities and consequences of scenarios, PRAs can provide a measure of individual risk (i.e. loss per unit of time for a single unidentifiable person) or frequency-consequence diagrams (also known as F-N curves, i.e. cumulative frequency of occurrence of a given loss). An insightful description of how these curves originated is provided by Ale at al. [12]. To construct these curves, each point represents a cluster of scenarios with a similar level of consequence. Each cluster has a frequency equivalent to the sum of the frequencies for each scenario. For simplicity, these curves represent the expected frequency of exceeding a particular consequence level and therefore are cumulative.

The more information there is available for describing the distributions of the variables involved, and the more variables are incorporated to estimate risk, the more resolution the result will have. This aligns with the most important advantage of a PRA, its capacity of gathering knowledge. This knowledge constitutes a detailed accounting of the system, its hazards, vulnerabilities and threats [13, 14] and the basis for the assessment. Building up this knowledge is a resource intensive effort and its quality and completeness conditions the credibility of the assessment’s results [15]. Due to the scientific rigor expected from the quantification process, PRA is often deemed as an objective way to inform decisions, yet it remains an exercise of expert judgment to a larger or lesser degree [6]. Hence, there is a direct relation between the validity of the PRA results and its credibility. Ultimately, no risk assessment is perfect, but results can be valid within a well-defined set of boundaries that are not inherent to the PRA outputs.

Box 3.1 – PRA example

A probabilistic fire risk assessment can be exemplified through the life-safety objective. Consider the occurrence of a compartment fire in an office building. A very simplified event tree of this occurrence can be represented as in Figure 3-4. In this case, two scenarios are identified, and an assumption is made that the evacuation is certain to begin.

Evacuation begins 1 Scenario 1

Compartment fire

0 Scenario 2

Figure 3-4. Event tree for the example

39 Chapter 3. Probabilistic risk assessment in FSE

To inform stakeholders about the likelihood that the fire will cause at least one fatality, the available time to egress can be compared to the required one (ASET vs. RSET approach). The former is conditioned by the fire development. The latter is a function of the design of the building and the occupants’ behaviour. In this example these two quantities are considered stochastic variables, normally distributed with means of 13 and 15 minutes, and standard deviations of 5 and 2 minutes, respectively. These distributions are selected just as an example and do not represent any real system.

Implementing a Monte Carlo procedure in Matlab to generate random values from the input distributions, the results in Figure 3-5 are obtained. The results of the procedure are sensitive to the sampling of the values from each input. Kroese et al. [16] present options to control Monte Carlo variance through different sampling techniques such as stratified, Latin Hypercube and importance sampling. The example presented here opts for a uniform random sampling (normrnd in Matlab R2018b). In complex multivariate problems where the focus of the Monte Carlo simulation is estimating a limit state function (such ASET – RSET = 0), consideration of the sampling technique is relevant and can have an effect on the uncertainty and the trustworthiness of the results.

40 Chapter 3. Probabilistic risk assessment in FSE

Figure 3-5. Inputs, outputs and convergence for the PRA example

A feature of the Monte Carlo technique is that the number of iterations must allow for convergence of the results. This convergence can be defined by the change in the mean value of the result or its standard deviation (as well as other distribution moments like skewness and kurtosis). Other criteria can be defined, but this convergence provides an initial outlook on the convergence of the results. Devaney [17] introduces different criteria for defining the number of iterations required in the context of structural fire safety analyses, yet seems to default to the convergence criteria. Robert and Casella [18] provide a detail accounting of these convergence criteria. In practice, this is useful, but expensive criteria in terms of computing times. In a strict sense, the error of the Monte Carlo simulation can be modelled as an uncertainty distribution to obtain a confidence interval for the number of iterations [19].

In the present example, convergence criteria suffices. Observing both mean and standard deviation in the example indicates convergence at around ten thousand iterations. The resulting distribution is produced for the maximum number of iterations (i.e. 2x104) and indicates that the performance is unacceptable with a probability of 65.24%. This result can be verified by checking the cumulative probability distribution of the probability of failure for all the iterations ran, as shown in Figure 3-6.

Figure 3-6. Cumulative probability distribution for the probability of failure

Based on this result, scenario 1 poses an important probability (>60%) of an unacceptable result. Following Eq. 1, the consequence has been characterized as well as its probability. In order to make a judgement about the building’s fire safety performance, the last element required is a fire occurrence frequency. If a fire frequency is available based on reliable data, the resulting individual risk can be obtained from the product said frequency and the probability of scenario 1 resulting in a fatality. If the acceptance criteria for individual risk is of 10-4

41 Chapter 3. Probabilistic risk assessment in FSE

fatalities/year, this would require the fire occurrence frequency to be 1.53 x 10-4 fires/year at most.

Estimations like the one portrayed in this example can provide valuable insight of risk, as long as the main quantities (ASET, RSET) are modelled from first principles approach. The alternative would require enough complete and valid data to model them as stochastic variables. Each one of these quantities represent complex functions encompassing a large set of variables that can be represented by deterministic values stochastic distributions.

The result of a PRA by itself is a description of uncertainty, providing stakeholders with an understanding of the most and less likely consequences of a building fire. Furthermore, performing sensitivity analysis on the parameters and shapes of the distribution of the quantities allow for an explicit quantification of epistemic uncertainty that can be reflected as a family of F-N curves, deemed by Paté-Cornell as the highest level of uncertainty treatment in a risk assessment [8]. In the context of FSE this can be done by using different distributions and different consequence models, but this can seldom capture the uncertainty associated to inadequate knowledge or underlying assumptions. In the unfortunately popular case of façade fires, it is still unknown for large segments of the population, how combustible materials came to wrap thousands of buildings around the globe without consideration of their fire risks. Allegedly, many engineers, including fire safety engineers, were involved in the design process leading to this decision. However, if ‘low combustibility’ materials are assumed to present the same risk as ‘non-combustible’ materials, the reasoning leading to this situation may be quite reasonably inferred. Assumptions such as these are not clearly described by PRAs and communicating them constitutes a key challenge of this approach.

By trying different sets of assumptions (i.e. assumptions) and models and then weighing the different results, a more complete description of uncertainties could be achieved, analogue to Paté-Cornell’s two-tier lottery machine [20]. As trying out different key assumptions, models and even distributions can lead to prohibitive analysis time and cost, these are prioritized by the leading analysts, taking down the argument that a typical PRA is enough to represent in detail the risks of a system. As all methodologies, PRA contributes to gain insight on risk, but no insight is ever complete. Van Coile [21] argues that in a risk assessment some of the variables “may be well known and can consequently be modelled by a single deterministic value… Other variables may be less clearly defined and should be modelled as stochastic variables”. The distinction between these two is a matter of expert judgement and therefore

Chapter 3. Probabilistic risk assessment in FSE

problematic for the credibility of a PRA. Expert judgement can hardly be quantified except when elicited from a set of experts, requiring performing a calibration stage which on its own is based on a model [22].

An alternative to the selection of probability distributions is the use of frequencies, subjective probabilities and exact probabilities estimated from a database. Such process is more akin to a deterministic analysis that includes both consequences and likelihoods. As analysed in Chapter 5, existing guidance on fire risk assessment hardly makes the difference between the different types of PRAs, despite their significant implications on the result. Box 3.2 presents an example of such assessment, in which scenarios are developed using an event tree analysis and the probabilities of its branches are based data reported in literature. The example provides a comparison between the results of two different analyst teams, showing the effects of different assumptions on the risk profile in the F-N format. This example also indicates how a deterministic analysis could be easily portrayed as a probabilistic one, and how the uncertainty associated to the likelihood figures are easily omitted in the result.

Box 3.2 – Quantitative fire risk assessment example

A simple building is used to exemplify the application of a PRA in FSE. The example consists of a 10 storeys office building to be constructed in an urban area of the state of Queensland in Australia. This is a multi-occupancy building, comprising carparks, retail areas and office areas, or classes 5, 6, 7a, respectively as shown in Figure 3-7. To implement the PRA on this building, it is assumed that a fire will start with a frequency determined by the type of occupation (based on the approach proposed by Tillerson [23]) at any of the three occupancies of the building (carpark, retail, offices).

43 Chapter 3. Probabilistic risk assessment in FSE

Figure 3-7. Graphical representation of the building

Following the workflow of a typical PRA, an event tree is constructed to identify the fire scenarios, for which it consequence can be estimated using a two-zone model (CFAST [24]).

The event tree is presented in Figure 3-8 for the carpark levels. The probabilities used for this assessment are those presented without brackets and are based on available data [25-27] and subjective probabilities assigned by the author after consultation in the literature and with the advisory team to this research project. The event tree was constructed using the available safety measures in the building and considering the outcomes of their failure or correct functioning. The safety measures considered are 1) early detection by occupants, 2) early suppression of the fire by occupants, 3) fire smoulders (i.e. does not produce flaming combustion and it is assumed to be controlled) 4) activation of the heat/smoke detector, 5) evacuation starts and 6) fire doors operate adequately.

Early detection of Extinguishment of Fire smoulders Heat detector / Evacuation Smok e doors fire by occupants fire by occupants fire alarm protocol begins operate properly activation promptly Fire extinguished 0.9 0.45 (0.8) (0.3) 0.7* La rge Fire

0.9 La rge fire with (0.8) open doors 0.9 0.1 (0.7) 0.3 (0.2) Impaired 0.755 evacuation (NS) 0.9 (0.8) 0.7* La rge fire without alarms

0.55 0.1 La rge fire without (0.7) (0.2) alarms a nd open doors 0.3 0.1 Fire starts @ (0.2) carpark Impaired 1.50E-03 evacuation events/year 0.245 (NS) Fire smoulders

Early detection fails (P=0.1, mirrored tree)

Figure 3-8. Event tree and scenarios for the example (only carpark)

The scenarios described with acronyms are consistent with those employed in the Verification Method for Fire Safety of the National Construction Code of Australia [28]. The base frequencies and consequences reported in this examples are based on the study carried out by FPA [29], but the use that is made of them differs significantly to the original study. In the

44 Chapter 3. Probabilistic risk assessment in FSE original study, the same fire ignition frequency was used for all occupancies, while here it was recalculated following the cited method for each one of the three occupancies.

It is worth noting that the subjective probabilities and evidence-based probabilities used in the even tree are static, i.e. no random sampling was done that enables the construction of a family of risk curves. The result of the risk profile in an F-N curve is presented in Figure 3-9, where the acceptance criteria is taken to be the societal risk acceptance curve of the UK [30].

1.E-01

1.E-02

ALARP (R2P2 UK) 1.E-03 Baseline

1.E-04 Frequency [1/year] Frequency 1.E-05 1 10 100 1000 Loss [casualties]

Figure 3-9. Risk profile for the building plotted as an F-N curve

The result of the PRA indicates that the risk profile fails to comply with the R2P2 criteria and therefore does not reflect an ‘adequate performance’. This result can be compared to the one obtained by testing different sets of assumptions regarding the available safety measures. A first alternative assumes that early detection by occupants does not occur and that evacuation certainly does not commences promptly, while a second alternative considers the same issues with evacuation due to the heat/smoke detectors not activating and the inadequate operation of the fire doors. These assumptions translate into alternative event trees in which these safety measures are assumed to fail (Pfail = 1). Testing these two alternatives the F-N curve results in the profiles presented in Figure 3-10, which also includes the result of the original study.

45 Chapter 3. Probabilistic risk assessment in FSE

1.E-01

1.E-02 ALARP (R2P2 UK) 1.E-03 Baseline Assumption 1 Assumption 2 1.E-04

Frequency [1/year] Frequency Original study

1.E-05 1 10 100 1000 Loss [casualties]

Figure 3-10. Risk profiles for the alternatives

The results show the effect of the assumptions made regarding the reliability of safety measures, where assuming the failure of selected features result in a more onerous risk profile. From this result it is possible to understand the importance of each safety measure and guide an alternative selection or an investment prioritization decision. Furthermore, the result also shows that a seemingly straightforward exercise of constructing a risk profile has the potential for significant disparity when the baseline result is compared to the original study. Notwithstanding the differences in the calculation of frequencies and the values for subjective probabilities, the resulting risk profiles differ in about an order of magnitude towards the upper end of the consequence scale. Such situation can arise in practice when a PRA is conducted in parallel by two different analyst teams or when the original study is reviewed as in this case.

3.3 Verifying safety with PRAs PRAs have been largely applied in several engineering (and non-engineering) fields with notable success and growing popularity as in the cases of land-use-planning for major hazardous facilities [13] and nuclear Safety [31]. However, adequate performance cannot be guaranteed a priori by any known mean in engineering [32] which calls for a continuous monitoring, understanding and managing of the risks throughout the life-cycles of the systems. This is constantly expressed by the different catastrophic events that occur both in the built environment and in very specific engineering fields such as oil & gas and the chemical industry.

For the built environment, Meacham [33] argues that quantifying the requirements of construction codes can help them achieve their goals, of which a particular set pertains to fire safety. In contrast, Bjelland [34] suggest that the process of risk quantification allows circumventing its goal, which is demonstrating that a building is safe enough. In other words, focusing on estimating acceptable risk results can miss the objective of achieving adequate

46 Chapter 3. Probabilistic risk assessment in FSE

safety is missed. In practice, focusing on an achieving an acceptable risk criterion can push FSE practitioners to focus on the desired result, regardless of whether it reflects the reality of the system or not.

The mechanizing of risk assessments is not a new concept. In 1982, Beard [35] presented an example of the use of the event tree analysis technique to investigate a hospital fire, analysing the extent of application of such a tool. His conclusion was that using such a model to answer a precise question (e.g. is the fire risk acceptable?) is limited by the model’s conceptual or numerical construction and that a quantitative model could lead to misrepresentation of the system’s complexity [35]. Currently, there are important knowledge bases for performing PRAs in the context of FSE, such as the one provided for England dwellings by Hopkin et al. [36], where the distribution of several relevant quantities are formulated as the result of a review of existing statistical analyses. This work constitutes a significant effort to understand the probability distribution of key variables, but fails to provide a basis for assessing buildings that differ from those of the study. Defining whether a building is similar or not to those of the study would also require criteria. How different should the occupancy and structural features be so that the inputs still apply? This is not an easy question to answer and being strict, most innovative building design will be unique.

As mentioned in Chapter 1, the more structured and controlled the system is, the easier it is to account for variability. However, no methodology can account for all possible variability, regardless of discipline or application. In contrast to many disciplines where standardization and recommended practices are abound, the variability of the features between any two buildings could be unquantifiable. This variability stems from changes in the environment, regulatory requirements, individuals’ decisions and the way in which the safety features of the building are managed, among many others. In contrast to other engineering systems, buildings are not inherently designed to monitor any deviation from ‘normal operation’. This partly has to do with the nature of the built environment, in which privacy and property draw important limits on what can be monitored. This caps the capacity to monitor and to manage variability in the built environment and presents a challenge to generate and improve data that supports probabilistic assessments (see section 3.7 for a detailed discussion on supporting data)

Demonstrating that the performance of the building is adequate through a PRA requires making assumptions about the specific occupancy, the specific behaviour of the occupants during a fire, the response of the safety measures in place, the fire service intervention and about the

47 Chapter 3. Probabilistic risk assessment in FSE

fire itself. Some of these can be accurately described, but some cannot. As presented in the previous section, the output of a PRA does not clearly communicate which variables have been accurately described, which are based on assumptions and which are supported by mere educated guesses. There could be significant uncertainty sources to a PRA realization that are not easily conveyed in its output and which require a clear communication between the analysts and the stakeholders. Due to the limitations of both of these parties (such as biases), this communication might not always lead to a clear understanding of the output.

Providing a clear quantitative definition of ‘adequate safety’ is a challenge that gets circumvented by assessing risk. As discussed here, the link between safety and risk is not self- evident. In cases when certain scenarios are not identified, the link is inexistent. The issue of scenario identification will be discussed in the following section. Furthermore, until the capacity to monitor and manage variability in the built environment is at par with that of high- risk industries, it is unreasonable to argue that the probabilistic quantification or fire risks can demonstrate adequate performance and verify safety. As exemplified by Yin and Jiang [37] and Cheng et al. [38], this will be gradually solved using the internet of things (IoT) and building information modelling (BIM) to have a detailed, real-time monitoring of the environment within a building, as well as of the sub-systems and components on which fire safety performance relies on. However, the cost and technological complexity of such systems make it only a possible solution in the future. The previous does not mean that PRAs should not be applied in FSE, as these support important decisions such as alternative prioritization. What this means is that the role of the PRA is not to be a yardstick to verify safety, but a tool to gain insight on particular hazards and risks.

Attempts have been made in the context of FSE to ‘prescribe’ fire risk assessments and their scenarios through ‘verification methods’ in order to reduce the complexity of a fire risk assessment, while providing a common procedural basis that enables results benchmarking. However, these administrative procedures aimed at diminishing the complexity of the analyses can be counterproductive (see section 3.6.2 for a detailed discussion of verification methods). Such administrative approach to complex risk assessments has been discussed in the context of dam risk: “Complex, hazardous tasks like dam operation are rightly guided by procedures, yet in rare cases, events will take place that are at their limit or may not be covered strictly. If operational decisions could be completely specified in advance, then they would not require professional staff (or indeed staff at all). Equally, experts may be following a procedure, but as a tool which should represent the culmination of expert knowledge, they may not frame their

48 Chapter 3. Probabilistic risk assessment in FSE

decisions explicitly in these terms. A focus on testing procedural adherence alone with view to assessing culpability, while an easier task for non-experts, is fundamentally misguided” [39]. This is yet another warning regarding mechanized risk assessments.

3.4 ‘Adequate safety’ Building codes define objectives and specific performance requirements, which are meant to provide boundaries for the decision-making process undertaken by relevant stakeholders. The stakeholders make decisions on the design features and investment on safety measures of a building based on the available information on alternatives, potential risks and benefits, as well as their own objectives and values. The objectives of construction codes are usually formulated in function of achieving ‘adequate safety’.

To understand to what extent a risk assessment (and a PRA in particular) can be used to verify that ‘adequate safety’ is achieved it is convenient to formulate what the term itself means. Despite the lack of consensus, safety has two predominant definitions, one claiming it is the absence of harm (Safety-I), another claiming it is the condition of –actively- being protected from harm (Safety-II) [40]. Levenson [32] argues that these definitions fail to match current practices of safety engineering and proposes a definition more akin to the approach of this work. Safety-III is based on systems safety and is defined as “freedom from unacceptable losses as identified by the system stakeholders. The goal is to eliminate, mitigate, or control hazards, which are the states that can lead to these losses”. Notice that this final definition places an important emphasis on the process of continuously identify and manage potential loss, which begins at the conception of the project. With this lack of consensus, ‘adequate safety’ remains qualitative and vague, open to subjective interpretation.

A way to define ‘adequate safety’ is formulating performance objectives and requirements, which help to articulate more clearly what it is that the building should do or not in case of a fire. An example from the National Construction Code of Australia is performance requirement CP1, which requires demonstrating that “a building must have elements which will, to the degree necessary, maintain structural stability during a fire” appropriate to its features. This particular requirement hints the need to demonstrate how long a structure will resist a fire before minor or major collapse but does not help clarifying what duration is acceptable. To help quantify performance, a proxy is used. The proxy is risk, which in turn is quantified through risk assessments. Other proxies have been used, such as classifications for occupancies and building importance; these classifications are discussed in section 4.6.

49 Chapter 3. Probabilistic risk assessment in FSE

Figure 3-11 formulates the link between the performance based approach, risk assessments and decision making. It also introduces the prescriptive approach in which decisions gravitate around compliance. In the performance based approach risk assessments can be used either to produce absolute or comparative analyses. The latter are typically done using reference designs, which are deemed-to-satisfy the regulatory requirements. Comparing designs to prescriptive solutions (deemed-to-satisfy or DtS) is excluded from this work and its validity is discussed by Bjelland [34, 41]. In this scheme, ‘adequate safety’ is explicitly demonstrated through the achievement of a desired level of performance, which in turn is demonstrated by a risk assessment.

Regulations Regulatory level

Performance Prescriptive based Design principle

Risk assessment Analysis principle

Compliance Absolute Comparative Evaluation principle

Decision-making

Figure 3-11. Paths from regulation to decision-making in safety. Adapted from Bjelland [42]

Linking ‘adequate safety’ to ‘acceptable performance’ and to ‘acceptable risk’ is not self- evident, although existing literature in FSE do so [3]. Assuming that the relation between performance and a risk measure is properly defined, this is a viable solution. However, formulating this relation is also not self-evident and poses important challenges. First, defining what ‘risk’ means. Second, proposing a construct that relates performance and risk. Third, defining a threshold for an acceptable risk and therefore for ‘adequate performance’.

First, risk has a myriad of definitions from which the common denominator is that it is rooted in uncertainty, i.e. lack of knowledge, vagueness and unpredictable variation [43-45]. Selecting these definitions has profound and practical implications for decision-making in an engineering context [46], as well as legal. The popular definition of risk as the product of consequence and likelihood has been replaced by a larger and more comprehensive set of definitions, in which

50 Chapter 3. Probabilistic risk assessment in FSE likelihood is not always present. In Figure 3-12 two definitions of risk are presented, in which likelihood is not a constituent element. Risk as a function of an undesired event, its consequences, the associated uncertainty and the background knowledge is deemed a useful definition, as it can easily accommodate PRAs and deterministic analyses.

Uncertainty Risk = (C, Q, K) Uncertainty

Objectives Action Consequences

Figure 3-12. Risk definition adapted from (left) ISO 31000:2018 [47] and (right) Aven [48].

Second, the link between performance and risk should be formulated. This is not a straightforward step, and guidance is virtually inexistent. An option is to assume that a risk measure directly reflects performance. This is a dangerous assumption. The danger is exemplified by choosing the definition ‘Risk = Consequence x Probability’. In this definition, both consequences and probabilities have the same weights, i.e. they are equally important. Therefore, a high consequence could be turned into an acceptable risk as long as its probability is deemed negligible based on some pre-defined criteria. It is key to ensure that the stakeholders are satisfied with this link between performance and risk definition, as the risk assessment will produce insight and support decisions based on this definition. Understanding the shortcomings of using risk as a proxy can help produce a transparent decision-making process. Currently, no guidance exist that highlights any of the two previous points. Instead, guidance on risk assessment methodologies typically assume a definition and present the process as a function of it.

Finally, once ‘adequate safety’ has been boiled down to demonstrating an acceptable risk measure, it is necessary to define what acceptable means. In general, defining what an acceptable risk measure is will depend directly on the chosen definition as well as the motivations, objectives and values of the stakeholders (see section 1.2).This is far from a technical process, although a positivist approach to risk criteria has been developed using decision theory, particularly expected utility. Before presenting the existing options for risk criteria (see section 3.5), it is necessary to discuss how likely fires are if the quantitative definition of risk used in PRAs is a function of likelihood. Moreover, that definition equally

51 Chapter 3. Probabilistic risk assessment in FSE

weights likelihood and consequence. The following sections will discuss if this is reasonable in the context of FSE.

3.4.1 Fire likelihood A key argument for using PRAs is based on the idea that fire is an unlikely event. Therefore, fire risk can be deemed acceptable if the frequency of occurrence is negligible. Recalling Eq. 1, this means that the frequency set by has a particularly important role in the PRA and it is worth examining what is known about𝐹𝐹𝑓𝑓 it.𝑖𝑖𝑓𝑓𝑓𝑓 In the USA, 379.600 dwelling fires [49] and census reported 105 million dwellings were reported in 2018 [50]; this yields an annual rate of 3.6 x 10-3 fires/dwelling. In the UK, 27.240 fires and 24 million dwellings were reported in 2016 [51]; this yields an annual rate of 1.24 x 10-3 fires/dwelling. From this data, it is not possible to distinguish those dwellings with and without particular fire safety measures, which introduces an important bias to what these rates signify. From the rates alone it is not possible to understand how likely fires are, as some basis for comparison is required.

Table 3- presents the fire occurrence statistics alongside with those of earthquakes. Consider a highly active area for earthquakes such as California in the USA; here justifying a particular structural design based on a low frequency of occurrence for a high magnitude earthquake takes place (e.g. 1994 Northridge earthquake in California). Evidently, a building cannot be design to withstand any earthquake, but it must withstand the largest feasible earthquake. Here, defining what feasible means requires a combination of statistical knowledge and professional judgement. Once it is set, the probability of an earthquake the chosen intensity is taken to be one, conditioning the performance of the building to withstand it. Adequate performance is defined in the seismic setting through reliability targets that make use cost-optimization techniques such as the life quality index [52]. In fire, such an approach cannot be applied in a straightforward manner. The ‘intensity’ of a fire is not simply defined by a handful of quantities, but by a large set of variables that must be defined based on the unique characteristics of each system (see section 3.6). The ‘intensity’ of the fire might only be understood after it occurs or after careful analysis of all the variables in a fire scenario.

Statistically, fires as a general category of events seem less frequent than large earthquakes (Table 3-, based on data from 1960 to 2016 [53]). Based on a frequency scale used in the SFPE Handbook for Fire Protection Engineering [54] most fire entries in Table 3- can be labelled ‘unlikely’, while some are deemed ‘expected’. These are subjective labels, for which no basis is provided and therefore without a real meaning of how likely the fires are. Only based on

52 Chapter 3. Probabilistic risk assessment in FSE these rates it can be said that hospitals in the UK are more likely to experience fires than those in the US, while the opposite can be said about offices. In relative terms, these rates are quite useful. In absolute terms, these rates only indicate all these settings are susceptible to fire.

Table 3-1. Annual occurrence rates for different events Activity Annual occurrence rate Earthquake (Any) [53] 2.00E+04 Earthquake (4.0-4.9) [53] 3.52E+02 Earthquake (5.0-5.9) [53] 2.73E+01 Earthquake (6.0-6.9) [53] 1.63 Earthquake (7.0-7.9) [53] 2.60E-01 Fires @ hospitals (UK) [55] 5.86E-02 Fires @ schools (USA) [56] 5.51E-02 Fires @ assembly entertainment (USA) [56] 5.45E-02 Anticipated, expected [54] > 1.00E-02 Fires @ offices (USA) [55] 4.23E-03 Fires @ hospitals (USA) [55] 3.63E-03 Fires @ dwellings (USA, 2018) [49, 50] 3.60E-03 Fires @ offices (UK) [55] 1.67E-03 Fires @ dwellings (England, 2016) [51, 57] 1.24E-03 Unlikely [54] > 1.00E-04 Extremely unlikely [54] > 1.00E-06 Beyond extremely unlikely [54] < 1.00E-06 Earthquake (>8) [53] 0

Another possible tool to understand how likely fires are is the fire fatality rate. This rate can be estimated from the reported fatalities in different countries caused by fire, which can be compared to the fatality rates from different activities. These activities include high-risk activities such as agriculture and manufacture, and can help construct a better idea. The fatality statistics provided by the Health & Safety Executive of the UK [30] show that indeed fire fatality rates are found towards the lower end of the range (Table 3-2). From these statistics it can be seen that fire fatalities are at around the same level than manufacturing fatalities, which shows that it is far from a negligible rate. Furthermore, it is only one order of magnitude lower than mining and road accidents. This comparison indicates that fire cannot be deemed unlikely and it is very much a part of human life in the built environment. This has a serious implication for the design of a building, as the consequences of fire risk can be devastating even if the likelihood estimation yields a very low number.

Chapter 3. Probabilistic risk assessment in FSE

Table 3-2. Annual fatality rates for different activities Activity Annual fatality rate Cancer [30] 2.58E-03 Road accident [30] 3.19E-04 Mining [30] 1.09E-04 Agriculture/Construction [30] 5.81E-05 Fire (39 countries, 1993-2016)*** 1.50E-05 Manufacture [30] 1.30E-05 Dwelling fire (England) [51, 57] 2.98E-06 Dwelling fire (USA) [49, 50] 8.53E-06 Gas incident [30] 6.62E-07 Non-dwelling fire (USA) [49, 50] 2.60E-07 Lightning strike [30] 5.35E-08

3.4.2 Large loss events Fischoff et al. [58] discusses how the presented data reflects society’s revealed acceptance of loss and therefore of risk. The fatality rates provided for the UK and the USA allow to compare the consequences of fires for different type of buildings and in different jurisdictions, but fail to explain how likely fires are or what a negligible fire risk would be. However, the argument of a negligible likelihood breaks down when significant events occur. A tragic example is the Grenfell tower fire in 2017 which left a toll of over 70 fatalities. That event by itself is three to four orders of magnitude higher than those in the USA and four orders of magnitude more than the average rate reported for residential fire casualties in the UK in the period 2009-2019. This single incident sparked outcry from society and triggered the independent review of building safety by Hackitt [59], while highlighting the aversion of society for high consequence events. The effects of this single incident is reflected in Figure 3-13.

1.00E-02

9.50E-03

9.00E-03

8.50E-03 Fire fatality rate 8.00E-03 (England) 7.50E-03 Without Grenfell 7.00E-03 Fatality rate [fatalities/fire] 6.50E-03 2009 2011 2013 2015 2017 2019 Year Figure 3-13. Fatality rate in residential fires in the UK, period 2009-2019 [57]

Chapter 3. Probabilistic risk assessment in FSE

Averaging the data over long periods of time can convey positive trend, but the fact remains that events like Grenfell are unacceptable. Society clearly rejects them and their consequences are severe, spanning well beyond the event itself and triggering legal and political procedures aimed at prosecuting the responsible and preventing such a tragedy reoccurs. So this leads to ask if the likelihood of an event such Grenfell really matters when it comes to ensuring adequate performance in the built environment. Considering the World Trade Center collapse, the Grenfell Tower and even the Lacrosse building fire (which resulted in zero casualties), it should be clear to the reader that the likelihood of these events is irrelevant as long as they are feasible. This can be easily said of many fires that have sparked global outrage such as those of nightclubs [60], historical structures such as Notre-Dame [61] and the National Museum of Brazil [62], the Kemerovo fire [63], among many more (see Table 3-3). What a priori probability would have made the consequences of any of these fires acceptable? The answer seems to be none. This serves as a counterargument for using likelihood as acceptability criteria.

Table 3-3. List of fires with unacceptable loss Building Fire Date Location Loss type 50 fatalities Summerland [64] Sports 1973 Isle of Man, UK 80 injured 48 fatalities Stardust [65] Nightclub 1981 Ireland 128 injured 2726 fatalities [67] World Trade Center [66] Office 2001 Ney York, USA US ~$178b (direct costs) [68] Station [60] Nightclub 2003 Rhode Island, US 100 fatalities 6 fatalities Lakenal House [69] Residential 2009 London, UK >20 injured Melbourne, Lacrosse [70] Residential 2014 >US $9m [71] Australia 72 fatalities Grenfell [72] Residential 2017 London, UK >70 injured National museum of Brazil Rio de Janeiro, >20 million historical/cultural Historical 2018 [62] Brazil items >64 fatalities Kemerovo [63] Mall 2018 Russia 52 injured Historical structures and Notre-Dame [61] Historical 2019 Paris, France contents

Chapter 3. Probabilistic risk assessment in FSE

3.5 Risk acceptance criteria As mentioned in the previous section, the fire data does not capture the extreme aversion to large loss events described in the previous section. This indicates that identifying acceptance criteria that allow deeming a particular design ‘safe’ is not an easy task. Many approaches exist to address risk acceptance, and this chapter presents the most relevant alternatives.

3.5.1 Types of criteria A first type of criteria traditionally used in fire safety are rules, i.e. prescriptive-codes. The rules of prescriptive construction codes for typified buildings are an implicit acceptance criteria, therefore any building design following them is deemed acceptable. Such approach has been the result of centuries of trial and errors and lessons learnt by the society as many devastating fires have occurred. However, innovation and complexity associated to modern buildings goes out of the applicability of this type of acceptance criteria and a more flexible approach is required, i.e. performance-based design.

With the incorporation of performance-based provisions in construction codes, a second type of acceptance criteria has emerged. These are quantifiable criteria that have as purpose ensuring that the effects of a fire would be low enough for the targets (e.g. people, structure) not to be affected in a significant manner. Such criteria can be defined in function of physical properties of the fire or the damage it causes, for example the maximum radiation of toxic species concentration. Criteria of this type have been used mainly in the context of verification methods. This approach has been combined with the definition of design fires described in section 3.6.1, and together have shown to be insufficient in demonstrating adequate levels of safety [73]. Outside of the context of verification methods, this type of phenomenological criteria based on harmful effects is widely used in chemical process safety [74] and in FSE to perform deterministic analyses, but usually fail to account for the consequences on a specific target. The latter typically requires a more probabilistic approach as analytical relations are scarce to express the relation between effects and consequences.

The third type of acceptance criteria is closely related to performance-based design and makes use of comparative analyses that indicate –typically in a qualitative manner- that a design poses as much risk as a deemed-to-satisfy design, i.e. acceptance through demonstrations of equivalence. This particular type of acceptance criterion is discussed in detail by Bjelland [42], who argues that such approach is only valid for very slight deviations from the prescriptive code and that mechanizing it can also lead to untrustworthy assessments and unsafe designs.

Chapter 3. Probabilistic risk assessment in FSE

A fourth type of acceptance criteria are reliability targets. Reliability is mathematically defined as the inverse of the probability of failure [75]. This type of criteria defines an absolute quantitative measure for failure per unit of time, where failure means whatever level of damage or performance that is considered unacceptable. Reliability targets can be defined for individual components, set of components and even complex systems made of multiple sets of interconnected components and are typically used in design of structures. This type of criteria is meant for reliability analysis, which in essence are probabilistic, therefore they account for significant variation in the design variables, including those that define resistance and load [75]. In few words, a system with a resistance capacity X and a load Y fails when X ≤ Y or X – Y ≤ 0, i.e. a limit state function. Both X and Y can be defined as probabilistic functions and typically make use of models of varying complexity to represent the resistance of the system and the load that is imposed (this is exemplified in Box 3.1). Reliability targets are established for well-defined systems (e.g. structural design at ambient temperature [52]) where an abundant knowledge of the limit state functions and the variability of its constituent variables and parameters is available. In the context of structural fire safety, reliability targets can be used but have not been developed so far as fire dynamics considerations are unique and must be incorporated from the formulation of the target [76].

The final type of acceptance criterion is the one associated to most PRA and QRAs, individual and societal risk criteria. These were defined in earlier sections of this chapter as the typical outputs of a PRA and have units of loss over time. Individual risk adds up risk contributions from all assessed scenarios and represent the ‘amount of risk’ associated to a single person, usually at a defined distance from the hazardous event, e.g. fire. In different countries there are different values for the maximum individual risk, which in the UK takes 1x10-3 worker fatalities/year and 1x10-4 public fatalities/year [30]. This approach is relevant for highly hazardous facilities, as risk increases as people get closer to the hazard source, which allows defining iso-risk contours around the facility and support land-use-planning decisions [13, 77]. However, this approach requires the multiplication of the frequency and consequences of each scenario, which can ‘hide’ large consequence scenarios and provide a partially informing number. If large loss can still take place, this is not a reliable acceptance criteria.

Societal risk builds up on this need and allows to express and evaluate risk using the already mentioned F-N curves. Here, both consequence and frequencies for scenarios are disaggregated, though frequencies of scenarios with the same consequences are added up, allowing to grasp how likely it is for the fire to yield a large unacceptable loss. In theory, this

Chapter 3. Probabilistic risk assessment in FSE

approach seems ideal for evaluating fire safety performance and it is one of the reasons for its application in many engineering disciplines. However, the societal risk criteria required for the evaluation is not straightforward and touches on complex topics that can easily scape the realm of FSE and even engineering itself. In the field of structural engineering, societal risk criteria has been considered for defining whether a particular area represents an acceptable collapse risk or not. This application is conditioned by the existence of building-specific fragility curves, population census and hazard data (earthquake magnitude records) [78]. This approach seems to be useful for Authorities Having Jurisdiction to support land-use-planning decisions but not to make decisions for the risk of a particular building design.

So far, the options for an acceptance criterion can seem overwhelming as it is and do imply a high degree of responsibility for the stakeholders that define it for a particular application. The range goes from prescriptive codes to societal risk, passing through absolute phenomenological criteria. All of these have advantages and disadvantages and it is the aim of the rest of this section to present and objective picture of why it is not straightforward to use a societal risk approach for FSE. Before expanding on societal risk, it is relevant to understand why a large loss event is significant and what its relation is with FSE.

3.5.2 Societal risk in FSE The use of F-N curves is underpinned by the underlying argument that an infrequent fire loss is appropriate to evaluate fire safety performance [33, 79]. The criteria used for F-N curves set a limit to the frequency of a range of loss, which is typically expressed in human fatalities. When this is done, the scale easily captures several hundreds or thousands of fatalities, which indicates that the risk considered can cause a severe consequence to society. This is also known as societal risk criteria and the acceptance criteria for loss vs. frequency for a single person is defined as individual risk.

Existing criteria for individual and societal risk are largely based on fatality records for different productive activities and largely vary with country and application, such as chemical process safety [80] or nuclear power plants [31]. This is consistent with Starr’s analysis of revealed preference applied to risk perceptions [81] and with the trend expected for the distribution of the scenarios making up an F-N curve. Existing societal criteria has evolved from the notion derived of accidents in nuclear power plants releasing radioactive material, traced back to Farmer’s curve back in 1967 in the UK. In Farmer’s curve the main premise was ensuring that events with more than 10 fatalities would not take place at most once every 10

Chapter 3. Probabilistic risk assessment in FSE

thousand years [77], i.e. an anchor points with a value of 1x10-3 fatalities/year. This semi- empirical threshold was the basis for further adjustment, including the modification of the slope of the curve to account for risk appetite/aversion. The Health & Safety Executive (HSE) of the UK [30] made it clear that “these criteria merely guidelines to be interpreted with common sense and are not intended to be rigid benchmarks to be complied with in all circumstances.”

The warning of the HSE is pertinent, as there is no clear rational for using the same criteria to achieve compliance with the construction codes in the context of FSE. Applying the same approach to acceptance criteria in the context of FSE would require a detailed understanding of the consequence axis (as with the Farmer’s curve), initially formulating it not as a function of fatalities, but as a function of performance of the building once the fire starts, e.g. Percentage of untenable conditions throughout the egress paths at fixed times. This aligns with an absolute phenomenological acceptance criterion focused on the physical effects of the fire, rather than on a societal measure. However, a phenomenological criterion cannot capture the risk profile of complex systems such as a high-rise multi-occupancy building built with composite materials, as it will be focused on limiting the effect of a fire in a particular scenario.

Acceptance criteria for PRAs in FSE are formulated in the basis of frequencies of occurrence of different magnitudes of consequences, as discussed in detail by Van Coile et al. [79]. In the work of Van Coile, an F-N curve is used to conceptualize acceptance and tolerability criteria, see Figure 3-14. To understand the limitations of this approach, consider the four points plotted in the diagram of Figure 3-14 where a fire scenario (or cluster of scenarios) could result in, particularly the role of uncertainty at each one of these points. At point 1) likelihoods are judged as high with low consequences and the slope of the tolerability criteria is almost zero, implying that large uncertainties in the frequency axis may have a large impact on acceptance but not for the consequence axis. At point 2) both likelihood and consequences are judged as low to medium and the slope on the acceptability threshold is mainly diagonal, implying that uncertainty might affect both variables in the same proportion. Moving towards point 3) consequences increase while likelihoods drop and the acceptance is increasingly skewed on the likelihood side. Uncertainties on the likelihood axis could have the benefit of erring on the acceptable region, while on the consequences axis these could lead towards point 4). Here, in point 4) there are several remarkable features, beginning with the cut-off point for the tolerance criteria, which is marked by a dotted line. The significance of this cut-off point is not discussed in the main body of literature for quantitative risk criteria, but as in the Hong Kong example, it has a direct impact on decision-making. Such a boundary in the consequence axis is a direct

Chapter 3. Probabilistic risk assessment in FSE

statement of unacceptable consequences regardless of likelihood. Furthermore, notice the tolerability threshold displays an almost vertical slope, meaning a shift in Point 4) due to uncertainties in the analysis could move the system from the ALARP region towards intolerability.

Figure 3-14. MAD acceptance criteria within the generalized frequency-consequence diagram. Image adapted from [79]

The analysis of these four points indicates that there is a skew in risk acceptance/tolerance towards consequences, i.e. the weights for likelihood and consequences are not the same. In addition, there is a cut-off point signifying a maximum allowable level of consequences. Considering that uncertainties increase towards the higher end of the consequences axis [20], the previous observations highlight the dangers of using societal risk as an absolute criteria for evaluating fire safety performance. It can be a useful tool, but the generation of an F-N curve and that it is located underneath the tolerability/acceptance criteria does not grant the design is ‘adequately safe’. At most, such a design can be deemed adequately safe within the boundaries of validity of the risk profile and of the societal risk criteria.

Current frameworks for PRA in FSE such as PD 7974-7 [82] require setting the maximum level of acceptable loss and indirectly set its frequency by defining an anchor point and a slope (F- N curve). The anchor point is located in the left upper part of the diagram, and sets an acceptable frequency for a low magnitude loss. The slope sets how stringent the acceptability line will be and in the intersection with the maximum level of acceptable loss, it defines its frequency. It is relevant to question the relevance of this upper limit for consequences, shown

Chapter 3. Probabilistic risk assessment in FSE

in Figure 3-14 as a vertical line. Such a limit is found explicitly in some societal risk curves such as Hong Kong’s [77] but has not been formalized as an acceptance criteria by itself; using this maximum allowable level of consequences is part of the proposal of an alternative fire risk assessment methodology, as presented in Chapter 6. This is consistent with the reflection done by Ale et al. [12] with respect to major events that cannot be tolerated under any circumstance, as represented by the probability consequence diagram of Figure 3-15. Currently there is no practical guidance on how to define these acceptability/tolerability criteria for FSE taking into account the unique characteristics of a building or its occupancy.

Figure 3-15. Probability consequence diagram; taken from Ale et al. [12]

3.5.3 Summary If enough statistics were available for typical types of buildings and occupancies, the case for estimating a negligible fire frequency could be reasonable. This is seldom the case. As discussed in the next section, the purpose of gaining insight on the fire risk through a PRA should focus on the conditions that can lead to large and unacceptable consequences and that ultimately guide the design process. This requires assuming that fire will indeed occur, i.e.

Pfire=1, instead of using statistics to justify it being negligible. Evidently, likelihood information can contribute to constructing a risk picture that effectively supports decision-making. With large fires having a particular relevance for FSE, it is evident that societal risk criteria can provide assurance that the design is ‘safe’. In reality, no acceptance criteria can ensure this, which is why risk assessments are only one of many pieces of information used by stakeholder to reach a decision. This decisions can also include significant compromises from the designer (affecting economics) and from the authority and the community (accepting larger risks).

Chapter 3. Probabilistic risk assessment in FSE

This section has referred to the work by Van Coile et al. [79] who argue the need to define a hierarchy for acceptance criteria. However, this work was clearly framed within the context of structural fire safety, as seen by the application examples of each one of the proposed criteria. Being far from general, this hierarchy can produce a bias in the way acceptance criteria is understood. Instead, this work has presented acceptance criteria from a more organic point of view, introducing five different type of acceptance criteria that are in fact used in quantitative risk assessments in different engineering fields (also consistent with the levels of risk assessment proposed by Paté-Cornell and introduced in section 3.2).

The resulting classification of risk acceptance criteria is presented in Table 3-4 where the five types of acceptance criteria are summarized and cross-referenced with those presented by Van Coile. The summary table also frames each criterion within an application and highlights its main limitation. It is the intention that this result further improves the understanding of the possibilities for how to judge fire safety performance. It is vital for the author to clearly state that frequency-based criteria is not the only option to assess performance and depending on the application, it could be far from the recommended option.

Table 3-4. Types of acceptance criteria Equivalence in Acceptance hierarchy of Application Main limitation criteria acceptance [79] Only applies for designs that deviate to a Evaluate minor minor degree from an accepted solution Comparative with deviations from or design. ‘Minor degree’ must be AC1 DtS design prescriptive code defined by the competent practitioner provisions and can be pushed outside of reasonable boundaries [42] Guide inherently safer Fails to explicitly account for variations Phenomenological design and the basis for N/A and can be deemed overconservative to based conducting a PRA, see an unknown degree section 3.8 Refered to as de minimis in [79], provides an Relies on supporting statistics or stringent lower boundary Likelihood based AC2 benchmarks to define it and can be for the occurrence of an overconservative undesired event or failure

Chapter 3. Probabilistic risk assessment in FSE

Equivalence in Acceptance hierarchy of Application Main limitation criteria acceptance [79] Provides a performance Requires a robust underlying knowledge measure for assessing of the limit state function and of the the reliability of a statistical data that represents the Reliability target AC5 system or its variability of key variables of both components based on an resistance and load; under lack of data it unsafe (known) state, i.e. is reasonable to default to the use of probability of failure deterministic (conservative) values Relies on socio-technical discussions to tune the criteria as based solely on Produce a risk profile fatality statistics, as these account only that links a wide range Individual and for the revealed risk acceptance, and not AC3-AC4 of consequences and societal risk the expressed one. Current literature is their associated not unified on the construction of these frequencies criteria and it is susceptible to adjustments based on practicality [77]

3.6 Limitations of scenario identification PRAs are scenario-based, which means that they decompose the risk of a system into the risk contributed by a set of individual scenarios. There are virtually infinite sequences of events and conditions that can take place in a building leading to undesired consequences that cannot be known a priori. Therefore, in a PRA a finite set of scenarios is defined. The process of identifying scenarios is supported by Kaplan’s theory of Structured Scenario Theory (SST) [83] and relies on the application of systematic techniques such as What-if, Failure Mode and Effects Analysis or FMEA, Hazards and Operability Analysis or HAZOP, Checklists, Failure and Event Tree Analysis (FTA and ETA, respectively) and Safety reviews among others [84]. A typical scenario is made up of three key elements: an initiating event, a hazardous event and the final event. A brief explanation of each element is provided in Figure 3-16, which are set within the conditions provided by the organization (administrative), the workplace and the culture, as detailed by Marks et al. [85].

Chapter 3. Probabilistic risk assessment in FSE

Administrative conditions Workplace conditions Safety culture conditions Initiating Hazardous Final event event event

Action that combined with First event of a sequence in Last event of a sequence in system’s conditions takes it which damage can occur which damage can occur outside its normal within the system or its within the system or its operating range (deviation) surroundings surroundings e.g. short-circuit e.g. compartment fire e.g. flashover fire

Figure 3-16. Key elements of a typical scenario

As scenario identification and development is one of the main challenges in disciplines where mature risk assessments have been employed [86], it is without a doubt one of the main challenges to apply PRAs in FSE.

3.6.1 Fire scenarios Can the scenarios for a fire risk assessment be fully known a priori? Fire events are a result of the complex interaction between the initial fire, additional fuel loads, the geometry, ventilation and structure of a building. Although variables such as fuel load might provide an indication of the potential damage resulting of the fire, this result is conditioned by many variables. ‘Minor’ changes in material selection due to aesthetic or building use requirements can produce failure modes and fire scenarios that cannot be identified on the basis of previous knowledge. This suggests that scenarios selection cannot be prioritized before starting the analysis (e.g. using pre-defined lists), nor before considering the building’s specific features into account. This is a major difference with other phenomena such as fires in open industrial facilities, explosions, dispersions or spills. This leads to question how to narrow down and prioritize a potentially infinite set of fire scenarios. Consider the heat release rate (HRR) profile of a fire. Usually fire models require the user to input such profile to estimate smoke production and movement, and the thermal-boundary condition affecting the structure. However, the HRR cannot be known a priori as it is a function of the fuel and of the ventilation and compartmentation characteristics of the building. This requires iterating the scenarios and, again, a critical expert judgement from the competent FSE practitioners involved. This complex relation between the features of the fire and those of the environment where it takes place, challenge the use of conventional hazard and scenario identification techniques.

Chapter 3. Probabilistic risk assessment in FSE

In chemical process plants variability is carefully and continuously controlled, which leads to consequences being a function of variables that are generally well-known a priori. Nevertheless, structured, systematic and comprehensive hazard identification techniques applied in this industry can miss over 40% of scenarios that end up materializing [14]. For atypical or novel systems, scenario identification in process plants can be done through existing methodologies which rationalize pre-defined scenarios as in the case of bio-refineries [87]. However, knowledge regarding scenarios is dynamic and depends on continuous monitoring of failure modes of components and systems, as exemplified in the nuclear power industry [88, 89]. The previous puts forward an important limitation to any risk assessment, particularly to fire risk assessments in the built environment where control of variability is considerably less.

A solution is narrowing down the scope of the fire scenarios considered. For example, the scenarios for structural fire analyses can be narrowed down to specific fire loads imposed to the structured and then a PRA can provide an insight on the adequate of the structural design. This is exemplified by Van Coile et al. [4, 21] and Zhang [90] who use the Maximum Entropy (MaxEnt) theory to obtain a probability distribution function that reflect a structural system response to fire without structural biases. Such approaches rely on a mathematical handling of limited observations of the system’s response. Typical PRAs require manually (and often subjectively) selecting a distribution, which can result in a biased result. MaxEnt is an example of a method that explicitly accounts for output uncertainty, with the trade-off of technical resources and required user competences being prohibitive for a large range of systems. Mathematically defining a large enough set of scenarios is relevant for such a narrow scope, but will fail to reveal all scenarios as variables are incorporated in the assessment and the potential for new failure modes is introduced. Such variation in failure modes and the resulting scenarios cannot be captured by probability distributions, unless statistical data exists that also supports analysing new potential outcomes.

The result is that identifying overall fire scenarios remains a major challenge [87, 91, 92]. The statistical analysis of Hasofer and Thomas [93] for apartment fires in 1993 presents a comprehensive picture of fire causes and the factors influencing consequences. This information is useful in constructing scenarios for buildings that resemble those from which the study data was gathered. Even in ‘similar’ buildings, human behaviour and decisions cannot be predefined and this significantly conditions the fire safety performance. In today’s FSE context, studies such as the one by Hasofer and Thomas are not widely available and make the construction of predefined scenarios a challenge. If the available data is current and

65 Chapter 3. Probabilistic risk assessment in FSE

comprehensive, PRA can be a powerful tool to support cost-benefit analyses such as that performed by Van Coile et al. [94] using the Life Quality Index methodology.

The absence of robust datasets can be partly remediated by systematically building and updating such studies for a wide variety of occupancies and types constructions and it could provide a baseline of PRA application for typical buildings. However, this does not guarantee that the focus of the PRA is placed on all the relevant scenarios. To overcome this challenge, fire safety performance-based codes have resorted to the use of predefined scenarios to provide a ‘similar’ basis of comparison between different buildings. These are known as design fires and are meant to represent challenging conditions to the performance of the building [36, 54, 95]. Using known techniques to identify scenarios such as event tree analysis, the applicable design fires are identified and then their consequences on the different vulnerable elements tested, e.g. occupants, structure. However, the basis for these design fires is their features and not necessarily the range of consequences they produce. This is places a significant cap on the level of insight these scenarios provide.

As previously established, consequences are not known a priori, therefore the PRA must be run to obtain an estimate of the consequences. When an assessment covers all aspects and variability of the building, a large proportion of scenarios would be expected to point towards the lower range of consequences. On the other hand, scenarios with large magnitude consequences should be a reduced set, which is often seen in F-N curves by a drastic (expected) drop towards the higher end of the consequence axis. This is reflected by the study of Shpilberg [96] in 1977 analysing the loss associated to 130 thousand fires, in which lognormal and pareto distributions are used to fit the data. Although the data fits the shape of these distribution which are consistent with the large amount of scenarios on the lower consequence end and a few large consequence scenarios, there is no conclusive evidence of a distribution better fitting the data. The study by Hasofer [97] in 2003 confirms this trend, while also pointing out the danger of the low-resolution at the high-consequence end of the distribution.

Considering the previous challenges to identify fire scenarios, what does an F-N curve tell stakeholders about the resolution of its individual points (each one representing a cluster of scenarios)? What does such a curve say about the tail of the distribution and its implications on design? A single point in the lower end of consequences could be made up of tens or hundreds of scenarios, while the high-consequence end could be populated by a handful. This highlights the relevance of the resolution of the results. What is the resolution of the scenarios in the tail

66 Chapter 3. Probabilistic risk assessment in FSE

of the distribution? These questions are formulated without a clear answer being found in the existing literature. From a rational point of view, it make sense to have a large set of scenarios describing the whole risk range, but this information might overlook important insight on the tails of the distribution. Furthermore, if the assessment is meant to guide the design, a high level of resolution is desirable at the tails. In lieu of this, a PRA can become a simple exercise of calculation and estimations that might easily overlook important knowledge gaps that could lead to unknown and catastrophic consequences.

3.6.2 Design fires and scenario clusters The complexity of fire and its interaction with the building and its occupants challenges our understanding of a well-structured causal relation, i.e. cause-consequence. As in many disciplines, this is tackled by adopting a positivist attitude towards the problem and decomposing it into smaller ‘chunks’ for which it is easier to formulate causal relations (i.e. tractability; section 1.2). This is evidenced in FSE with the proposition to decompose fire safety into sub-systems [11, 98]. In his book, Gano [99] challenges the traditional causal relations used to describe accidents in retrospective, highlighting the favourable position that retrospective gives to analysts. He further shows that any effect, e.g. fire, is the result of pre- existing passive conditions and a set of triggering actions. The former can be ignored, as the latter constitute the causal elements that tend to draw most attention during investigations, by the media and in hazards identification workshops. Such complex causality relationships challenges the usefulness and effectiveness of design fires in demonstrating performance. Understanding causal relations involved in a fire scenario remain a key challenge of any fire risk assessment, as it introduces a large degree of subjectivity.

Shergold-Weir [100] and Hackitt [59] enquiries have shown that pre-existing conditions in FSE were a major contributor to the consequences of the Grenfell Tower fire [72] and the issue of pre-existing conditions can be extended to other relevant events such as the Lacrosse Fire [70], the WTC collapse [101] or the Live Oak/Milstar complex fire [102]. Immediate causes in the Grenfell tower disaster did not prevent the spotlight on the underlying conditions that introduced fire hazards without proper management. The triggering actions in the Lacrosse building fire associated to high fuel loads and ignition sources management did prevent the focus of the enquiries on the selection of the façade material and the lack of proper measures to stop vertical spread [70]. The fire in 2005 at the Live Oak/Milstar complex left the U.S. Fire Administration investigators to a set of lessons learnt which are relevant to this discussion; a few relevant lessons are listed as follows:

Chapter 3. Probabilistic risk assessment in FSE

• “Automatic fire sprinkler systems will not control all fires involving preheated combustible liquids nor will they prevent the rapid collapse of lightweight unprotected construction exposed to rapidly developing fires” • “Process controls must include procedures to effectively manage expected equipment failures” • “Maintenance and repair of fire doors must be performed on a regular basis and damage that affects the door’s operation should be promptly corrected” • “Regularly scheduled fire prevention inspections and review of plans for construction and major remodelling should be conducted”

Assuming safety elements will behave as expected or that they will do so with a certain probability, could lead to conditions such as those in the Live Oak/Milstar complex to materialize. Acknowledging that fires are the product of underlying conditions and triggering actions represents a challenge for fire safety, means demonstrating performance cannot be limited to checking the system’s response to a predefined set of design fires. These predefined scenarios fail to capture the plausible range of consequences and provide a pathway for compliance, therefore endangering adequately informing decision-making.

A practical example of the subjectivity and the effect in biases in scenarios is found in the verification method for fire safety in New Zealand. Comparing the 2012 [103] and 2017 [104] versions of the same fire safety verification methods, important differences are found in the ‘vertical spread’ scenario, numbered 4.6. In 2012, laboratory scale tests were accepted to demonstrate a façade was compliant with the performance requirement for vertical fire spread, while in 2017 this was removed and replaced by quantitative requirements that would require large scale testing or validated modelling. The considerations in 2017 included the low roof exposure, adding on to fire spread from openings and over façade materials. These modifications effectively change the scenario considerations, many of which are quantified but are essentially product of considerations related to the practice of FSE, recent experience and knowledge.

Hall and Sekizawa [105] claim that all the scenario universe can be covered by ‘scenario clusters’ from which representative ‘design fire scenarios’ can be chosen for consequence estimation. Furthermore, they criticize the intensive use of professional judgment for scenario identification and the lack of techniques like those used in the chemical process industry [10]. However, the examples provided above show that even in the process industries the use of

Chapter 3. Probabilistic risk assessment in FSE these techniques does not provide a solution. In fact, Cameron et al. [106] state how hazard and scenario identification in the chemical process safety is subject to failure, while in FSE, Bjelland [41] highlights the crucial and necessary role of competent professionals. ‘Scenario clusters’ are referred to by a large body of FSE guidance such as BS 7974-7 [82] and ISO 16733-1 [95], in which a clearly qualitative process is used to generate the groups of scenarios based on variables deemed ‘more’ relevant by the fire safety engineers. This qualitative process relies on classifying variables and judging their configurations and values (prior to quantification). This subjective process then relies on the practitioners’ capacity to define the relevant parameters for a cluster, as the existing literature only provides examples in lieu of a structured procedure. This ends up conditioning the quality of the scenarios on the experience, knowledge and the capacity for screening the scenarios (e.g. using simplified calculation tools) of the practitioner. If the practitioner does not have the competencies or experience to do so, the assessment is then conditioned to his intuition and his personal biases.

Taking into account that the result of creating ‘scenario clusters’ yields the basis for performing the PRA, it is worth considering its advantages and disadvantages. It is indeed vital to construct a conceptual representation of what constitutes a scenario, based on a set of variables; this becomes one of the basis of the proposed methodology of this work, presented in Chapter 6. However, relying on qualitatively classifying all possible states of the world imply the assumption that these naturally are distributed into ‘critical’ and ‘non-critical’ states. Cox et al. [107] analysed that when this distribution is not guaranteed (as in the case of FSE), qualitative analyses can result in erroneous results. The scenario clustering guidelines suggest focusing on the ‘critical’ scenarios, which also serve as a proxy for a wider set of ‘similar’ scenarios that do not need to be explicitly analysed. This constitutes a weak basis for demonstrating in an absolute and quantitative manner that a design is ‘safe’ or its performance ‘adequate’.

3.7 Supporting data A recurring theme of the previous sections has been the dependence of PRA on statistical data to support the likelihood estimations. This section focuses on the general limitations of using this data (even in disciplines where PRA is applied extensively) and points out the particular limitations in the FSE field.

The statistics for fire frequency ( in Eq. 1) are based on ad hoc studies such as those carried out in Japan for non-residential 𝐹𝐹buildings𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 [108]. Other studies report the probability of a fire occurrence instead of its frequency, as in the case of Swiss retail buildings [109]. However,

Chapter 3. Probabilistic risk assessment in FSE

these studies are rapidly outdated and the estimates no longer capture the fast-paced development of the built environment around the world. This is where structured and continuously maintained fire databases become essential, such as NFIRS in the US [49] or the data recorded by the Home Office in England [57]. In the case of the US, NFIRS is not alone and has another eleven databases that overlap and complement it to an unknown degree. This was precisely the topic of conversation during the “Today and Tomorrow’s Fire Data” workshop held by the Fire Analysis and Research Division of the National Fire Protection Association (NFPA) in 2014 [110]. In this workshop, there was a clear acknowledgement that NFIRS does not cover all fires, and that those covered do not all possess specific and relevant information useful for further analysis. Further issues include the effort required from fire services in registering the data, which often lacks validation through benchmarking, and can include a varying degree of subjective errors. It is worth keeping in mind that the structures on which these databases are constructed might not be intuitive nor reflect the variability to which fire services are exposed when attending to a fire.

The large amount of statistical data and analysis for safety measures like doors [111], smoke detectors [112] and sprinklers [113, 114] are also constructed from recorded failures, i.e. they capture events in which both a fire took place, the safety measure failed to be effective and there was a successful recording. The resulting failure rates from this approach have to be carefully considered as the data does not consider all the failures occurred when a fire did not take place. Failures in individual components of safety measures could provide a better understanding of reliability, but these are seldom monitored and registered in a continuous basis. It is naïve to think that the reliability of a fire safety measure is given only by its behaviour when a fire occurs. The probability of failure on demand is more adjusted to what the current data reports, but cannot be tested as is done for safety instrumented systems. These are components or systems that are guaranteed (by the manufacturer) to provide a range of probability of failure on demand. Such an approach is unfeasible for safety measures that encompass a whole building.

Although data is indeed available and it should be tempting to use the figures it produce to support a PRA, a careful consideration of the shortcomings of data is necessary. The following points help illustrate the limitations of datasets. Some of these points also illustrate how data can be unreliable to justify adequate performance and could actually produce misleading information:

Chapter 3. Probabilistic risk assessment in FSE

• How reliable is FSE reliability data? - Smith [115] describes a best practice for recording failures, which go beyond failures on demand (as in the case of a sprinkler not working when a fire occurs) and require a detailed accounting of time between failures, associated causes, cost, repair length (if applicable). For chemical process safety reliability data, Smith [116] identifies different issues to be considered with existing reliability databases which have been constructed over decades, concluding that these result in failure rates with an uncertainty range of two to three orders of magnitude. Existing fire datasets do not follow this best practice and since fire data does not capture all failure data but only that associated to a recorded fire, it is reasonable that the issues identified by Smith will be present. The analysis of nuclear power stations considers the evolution of failure rates (and consequences) as a function of time. This can account for a reduction of reliability in over three orders of magnitude after 40 years of operation in critical systems such as water pumping [117]. Such an understanding of the failure statistics throughout time can largely affect the results of a risk assessment, and existing fire data does not enable such a study.

• How complete is fire data? - Often guidance documents for fire risk assessment mention the lack of data issue [118] but seldom these discuss the validity of data as a function of their sample size. Klara [119] considered the issue of the required sample size –in terms of test drive distance- to ensure that the failure rate of autonomous vehicles would be at most 1.09 fatalities per 100 million miles with a 95% confidence. The result was a test drive without failures of 12.5 years. The estimation was repeated in order to ensure that the real failure rate would fall within 20% of the stated rate with a 95% confidence, resulting in a 400 years test drive – This new estimates account for Type I error in statistics, i.e. rejecting the null hypothesis when it is in fact true. The estimation is repeated once more to account for Type II error, i.e. not rejecting the null hypothesis when it is false, typically associated to sample size issues. The final estimation yields a 500 years test drive. This example shows the significance of sample size and its impact on failure rates and therefore reliability. The previous considerations have not been introduced to judge the appropriateness of data associated to fire occurrence and causes. Following Klara’s [119] approach to defining statistical significant sample size to FSE a key reflection can be attained. Assume that the probabilistic acceptance criteria is taken to define adequate performance, particularly the individual risk with a value of 1 fatality per 10 thousand years (1 x 10-4 fatalities/year); this value is taken as the desired reliability of the building design. A confidence level (ranging from 50% to 99%) is defined and an assumption made that a building based on this design

Chapter 3. Probabilistic risk assessment in FSE

will not experience a fire while observed. Then, the number of years required for observation of a single building can be estimated based on the binomial distribution and yield the results of Figure 3-17. For a confidence level of 95%, a single building would need to be observed for 30 thousand years, which results impossible. An alternative would be observing a thousand buildings based on the same design for 30 years, which although feasible, raises the question if these buildings could actually be comparable. Unless construction practices, maintenance regimes and occupation remain within a narrow margin of difference, the buildings could not be comparable even when originating from the same design. The previous exemplifies the challenges of assuming a statistical approach to reliability in FSE.

Figure 3-17. Number of years required for observation for different risk levels

• Adequate performance through redundancy? – It could be argued that high consequence fires have low occurrence rates, therefore the focus should be on the reliability of safety measures. This seems to resonate with failures in the aviation industry. Downer [120] discusses the strict and independent failure rate required by authorities on components and the responsibility of manufacturers to demonstrate their products meet it, usually through redundant safety measures. Downer [121] draws attention to the use of redundancy as a way to demonstrate acceptance of technological risks and the problems it does not solve. The complexity of problems like engine failure is such that testing of the engine cannot

Chapter 3. Probabilistic risk assessment in FSE

provide accurate reliability measures thus scenarios that could potentially alter the performance of the engine are also included in the reliability assessment. This is the case of the assessment of bird impact on the performance of an engine by means of an artificial chicken shot onto operating engines; defining the chicken and its impact parameters reflects the same issues mentioned earlier about scenario identification and the impossibility to exhaust them. Downer [121] indicates that acceptance has to address complexity, independence, unforeseen failure modes (as in the case of the Boing 737 MAX [122]) and human factors. Redundancy by itself is not a good guarantee for adequate performance, and Downer [121] puts forward diversity in design as an alternative. In FSE, justifying a building design based on the performance of a particular safety measure requires this system being available and reliable when needed, otherwise being backed up by a suitable redundancy. With statistics not guaranteeing reliability and stakeholders pushing for cost reduction, diversity in design is rather an uncommon practice in FSE. Instead, it is not uncommon that performance is dependent on a single safety measure which despite highly reliable can be rendered ineffective. The ultimate design diversity in FSE is using the building design features (e.g. natural ventilation, compartment sizes and geometry, construction materials) to achieve an adequate performance and then introduce backup systems that lower the probability of this performance not being achieved (e.g. detection, suppression).

• What matters is what happens when a fire occurs, not if it occurs - The analysis done by Manes et al. [56] on the fire frequencies in the UK, the US and those indicated by the PD 7974-7 [82], indicates that these differ between 4 to 82 times, i.e. almost two orders of magnitude. This has a severe influence not only in the validity of the ‘unlikely’ nature of fire, but also introduce a source of variability to the . In the context of a risk

assessment, estimating for a particular set of scenarios𝐹𝐹𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 requires accounting for the frequency of ignition, as𝐹𝐹 well𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 as of the probability of failure of the different safety measures involved. The fact that statistics might support a low frequency for a specific scenario does not make it less likely if the right conditions are present in the operation of the building. Within the design process of the building it can be assumed that a fire event will occur and thus its probability is unity, shifting the focus of the risk assessment to the effects of the fire instead of focusing on its likelihood. This is the fundamental reason behind the fact that all buildings (and furnishings) include some level of fire protection measures, as it is

Chapter 3. Probabilistic risk assessment in FSE

impossible to design out all possibilities for fire occurrence.

• Safety measures on which performance relies are not fully reliable - There is evidence for the failure of key safety measures [123], which creates a very real ‘potential for surprise’ if they happen fail to on demand. This is the case of sprinklers, which are commonly deemed highly reliable and effective. As shown by Long et al. [124], data on sprinklers show that in fires large enough to activate them, 1 out of 10 sprinkler systems fail to be effective. Those are significantly concerning odds if the whole adequacy of fire safety performance relies on this single safety measure. One of the key causes for sprinkler ineffectiveness reported by Long et al. [124] is improper maintenance; this is important, as poor maintenance for sprinklers might reflect poor maintenance overall and therefore reduce the odds of a successful redundancy or back up being in place. In contrast, passive elements do not require a specific action in order to provide the intended function. If an active and passive measures perform the same function, e.g. containing fire spread, the passive measure might fail if there are flaws in its design or installation. On top of these, the active measure could also fail due to activation issues. In a comparison where design and installation are done at the same standard of quality, the additional failure cause for the active measure renders it less reliable than the passive one; the failure trees in Figure 3-18 illustrate this point. Reliability is also a time dependant function that requires frequent reassessment to account for deterioration and new failure modes. This requires adequate recording of inspection and maintenance activities throughout the life-cycle of the building, as well as a comprehensive data recording for failures associated to fires.

Passive Active measure fails measure fails

Installation Installation Activation Design flaw Design flaw flaw flaw issues

Figure 3-18. Failure trees for a passive and an active measure performing the same function

One possible way to circumvent the issues described here is limiting the use of PRAs to understand variability and to prioritize alternatives in a cost benefit-basis; this has already been called for in the context of chemical process safety and is a controversial topic within safety

74 Chapter 3. Probabilistic risk assessment in FSE science [123]. This would require ensuring that the building design provides and acceptable performance and narrow the focus of the PRAs to provide high levels of reliability for particular failure modes or scenarios. This research work advocates for this possibility and introduces a consequence-driven fire risk assessment methodology in Chapter 6 to assess the performance of the building design without relying on the estimation of likelihoods. In order to introduce such an alternative, the role of risk assessments in informing the design process needs to be clarified.

3.8 Guiding the design process Defining design features and safety measures of a building is a clear outcome of the design process, one that largely defines the costs of the design, construction and operation. Based on the discussion of section 3.4, it is evident that the performance of a building should cope with potentially devastating fires that a PRA can deem highly unlikely or completely miss. Evidently, PRAs with a comprehensive scenario set, adequate supporting data and that fully communicate uncertainty, could help identify these scenarios and inform the design process. At this point it should be clear that these are an ideal rather than a common practice. More importantly, scenario development in PRAs tends to include safety measures from the beginning. This impedes understanding the inherent risk of the system, i.e. when only the constitutive features of the design are in place and no additional safety measures have been provided. This can be done through PRA, but the outputs will still condition the performance assessment on likelihood estimates.

The current practice of PRA in FSE does not allow clearly characterising the consequence space that the design produces and obscures the process by which safety features are selected. Instead, PRAs tend to focus on likelihood or cost-benefit analysis to guide this process. This is exemplified by the work of Van Coile et al. [94] and by De Sanctis and Fontana [125] who use the Life Quality Index method to guide the selection of design features such as fire door width. The examples as application of a methodology are sound, but the underlying data for the presented case studies is susceptible to the issues presented in the previous section. Additionally, some of this data is only applicable to the presented case studies, which at the time of this writing dates back between 5 to 21 years. Moreover, the application of the cost- benefit approach of the Life Quality Index completely ignores the reaction of society to particularly large losses, despite these being highly unlikely based on the averaged data used.

Chapter 3. Probabilistic risk assessment in FSE

Each element of a fire safety strategy (see Figure 3-19) has a specific function aimed at preventing, controlling or mitigating the effects of a fire. These are effectively safety measures, which condition the scenarios that can take place in a building, as well as its consequences and likelihood. The effective or ineffective role of each specific safety can lead to different scenarios and fire damage, as well as affect other safety measures. For example, detection will primarily address the effects on people by establishing the onset of the evacuation process while fireproofing addresses the effect of heat on structural performance. Nevertheless, detection might be called to influence structural behaviour by enabling fire suppression, while fireproofing might support egress by providing a protected means of egress. This reflects the highly interconnected nature of the fire safety performance, which is the object of study in a fire risk assessment. When performing a PRA, each safety measure further conditions the likelihood of the outcome but their effects are considered in isolation. This can bring about unrealistic results, which in the best case lead to additional costs and in the worst, decreased performance.

Detection & Compartm- Smoke Structural Fire fighters Management Suppression notification entation management integrity facilities strategy

Detection Fire rated Staircase Manual Evacuation Robust design Fire panel system walls pressurization extinguishers strategy

Alarm Smoke Mechanical Sprinkler Structural fire Risers & Training & drills system lobbies ventilation system protection connections

Communication Service Inert gas Sprinkler Sprinkler system penetrations flooding valves valves

Curtain Firefighters Risk system plugs lift assessments

Façade Communication Audits projections system

Fire dampers Conventions

Passive safety measure

Active safety measure

Figure 3-19. Typical safety features of a fire safety strategy

Consider a residential building that has combustible materials in its envelope, as part of its façade system. If a fire occurs within the building and it is allowed to grow to flashover, it is feasible that the façade will be exposed to a wide range of heating mechanisms. At the design stage of the building, this scenario would lead to the designers to ask themselves about the likelihood of ignition of the façade given a compartment fire. A first step in gaining insight on

Chapter 3. Probabilistic risk assessment in FSE

this question could be modelling the heat flux to the façade. However, this is a complex phenomenon to model, as the interface between the window, the wall and the façade can include complex geometrical shapes and several discontinuous materials. A possibility would be simplifying the problem to a degree where all these details are lost and instead a single representative quantity is estimated, e.g. impinging heat flux using Abecassis-Empis’ model [126]. Despite coarse, this quantity provides a clear insight on whether the façade materials will be exposed to a heat flux higher than the one needed for ignition. If it is larger indeed, the question is then the time to ignition. If experimental data on ignition time is available, assumptions would be needed to account for the differences in the geometries, ensemble and materials. These assumptions would have to be done in a rational and deterministic manner and most likely be very conservative. In the absence of data, a deterministic analysis provides a more appropriate description of the risk. For this particular example, assuming an ignition time of zero seconds once the impingement onset would be reasonable in lieu of experimental –and specific- data.

The previous example points towards the need to identify and analyse scenarios stemming from different but interconnected hazards. Traditionally, a PRA could be implemented to ensure an adequate level of structural performance, while another PRA could focus on the overall fire performance. Partly, this is reflected by compartmentalized guidance associated to the construction code. Take the UK case, where the building regulations have the Approved Document A [127] for structural matters and Approved Document B [128, 129] for fire safety matters. Although a competent fire safety engineer might be able to identify connections between the two of them, triggering a dialogue with the structural designer, this is not always the case. Take the collapse of WTC1 & 2 [66, 101]. This event was caused by a preceding event (aircraft impact) that was considered in the structural design of the building but the other effects of which had not been accounted for in the rest of the design. Although fire scenarios had also been taken into account, it was necessary to project the domino effect of an aircraft impact to identify a scenario of multiple fires at multiple levels at the same time. The structure was considered in isolation and complied with the structural requirements, but failed to be integrated with other elements of the design. The previous consideration is of importance when selecting and designing safety measures. In the WTC event, dislodged fire proofing was deemed a significant contributor to the collapse. Dislodging was caused by the aircraft impact but had a profound implication for the fire resistance of the structure.

Chapter 3. Probabilistic risk assessment in FSE

As previously discussed, the reliability of the system as a whole is given by the use of redundant safety measures that can unknowingly be interdependent. This places a significant limitation on demonstrating adequate performance through comparative analyses such as those provided by a PRA. The comparative approach does not provide a strong basis to demonstrate that the design performs adequately, as it is dependent on the completeness of scenarios and on the validity of the supporting data. Missing supporting data, the strength of a PRA lies in providing insight on specific failure modes and alternative selection [11]. For a particular scenario set (e.g. specific failure modes) that results in inacceptable risk, a PRA can help appraise design alternatives through cost-benefit analysis that take into account variability in reliability and effectiveness of the safety measures or design features [82, 130].

Kirchsteiger [131] discusses how deterministic analyses can be used to complement PRA results in nuclear power plant, as the latter do not offer remediate the lack of understanding of potentially catastrophic consequences. Specifically, PRA influences design by incorporating probabilities of failure of the different elements of the fire safety strategy. A prerequisite to incorporate probabilities of failure is understanding the effect of a failed safety measure on the consequences, i.e. the effect on the fire damage potential. An alternative approach, such as a consequence-driven risk assessment, can focus on testing the boundaries of the system’s performance and then feedback this insight into the design process, highlighting the specific risks that require treatment through increased reliability and therefore a PRA.

Understanding the performance as a function of the consequence can identify a first set of scenarios where these are acceptable and require no further treatment. All other scenarios need to be analysed to determine their potential consequences, i.e. mapping the performance range of the building as a function of consequences. Such a deterministic process informs the design and its constitutive features in order to increase its performance to an acceptable degree. Thus, the focus of this process is focused on managing consequences and delivering an acceptable level of damage. Kirchsteiger [131] states that a consequence-oriented approach would lead to pass/fail result, which is consistent with the intended objectives within FSE.

3.9 Advantages and challenges The previous sections have dissected the essential elements of a PRA and both the advantages and limitations they carry. This section summarizes the advantages and limitations of implementing PRAs in the FSE context. The main advantages of the methodology are:

Chapter 3. Probabilistic risk assessment in FSE

• PRAs constitute a robust process to gather and generate knowledge about a particular building and the elements influencing its fire safety performance. • PRA provides a mechanism to take into account a large amount of design features and account for varying properties, conditions and actions to quantify performance in a format based on likelihoods and consequences. • Eq. 1 can be used to understand the conditions and necessary safety measures for achieving a specific outcome (i.e. consequence), while providing an estimate of its likelihood as a function of underlying statistical data or assumptions. This representation allows for constructing a detailed description of risk. • PRAs are useful to model complex system responses and allow incorporating a large body of knowledge regarding variability (aleatory uncertainty). • By incorporating variability to a high degree of detail, PRAs enable sophisticated cost- benefit analyses that allow ranking design and safety measures alternatives. • Probabilistic risk assessments can be used in a mechanistic manner to verify safety [34, 132], which aligns and expands on the intent of existing verification methods. • PRAs can be complemented by deterministic consequence analyses to better understand and communicate uncertainties associated to inputs and assumptions [133, 134]. • Significant guidance documentation exists on PRAs, as presented and evaluated in Chapter 5.

The previous advantages are associated to the popular role of PRAs and QRAs across a large spectrum of engineering disciplines. However, this chapter has also introduced some key limitations which are general to the methodology. Furthermore, implementing PRAs in the context of FSE brings about unique difficulties, which will be further explored in Chapter 4. The main limitations are:

• The design of building is focused on their functionality and aesthetics, not on ensuring acceptable consequences. This provides a platform for acceptance criteria such as societal risk to justify designs based on negligible likelihood, without actually managing unacceptable consequences. • Using PRAs as a tool to verify ‘adequate safety’ focuses on the means, not on the goal. This can turn a PRA into a prescriptive tool as analysed and argued against by Gehandler [132] and Johnson and Nobel [73].

Chapter 3. Probabilistic risk assessment in FSE

• The danger of using a mechanistic approach to verifying safety is compounded by the paucity of reliable statistics for high consequence fires and for specific reliability data for sub-system and individual components. • High consequence fire statistics and reliability data are a function of time as well as periodic inspection, maintenance and testing. This statistics have poor resolution at their tails, and therefore present an important limit to frequency estimations. • PRA outcomes such as F-N curves can be generated without a single probabilistic calculation or consideration of the probabilistic distribution of key parameters (see Box 3.2). • The PRA methodology does not account for the significant subjective base they require for defining scenarios and specifying probabilities without accounting for the full probability distribution. • Likelihood and cost-benefit based acceptability criteria, including the so called “Societal Willingness To Pay to save one statistical life (SWTP)” [94], are defined using a revelled preference approach, which fails to account for value judgments of relevant stakeholders. • Communicating the full range of uncertainties involved in a PRA requires running alternative assessment using different underlying assumptions (e.g. expanding or narrowing down the scenario set) or different models for the constituting analyses. • Fully investigating and communicating the uncertainties in a PRA increases its trustworthiness and its potential to correctly inform the design; however, this is a costly exercise, far from a common practice and no clear guidance exists to do so in the context of FSE. • The outputs of a PRA are based on the initial process of hazards identification and scenario development. Section 3.5 exposed how this basis is largely subjective and typically incomplete. • Prescribing scenarios can mechanize the PRA process and limit its ability to accurately provide a comprehensive insight on risks. This in turn detracts from the supporting information to decision-making.

3.10 Conclusions This chapter examined the key claim for PRAs in the context of FSE, which sits with the argument that a fire design can be acceptable if the likelihood of the event is negligible. This claim can be reasonable within the bounds of the available knowledge (e.g. expected failure modes), notwithstanding that a favourable PRA result does not grant a ‘safety’ guarantee.

Chapter 3. Probabilistic risk assessment in FSE

Qualifying negligible likelihood poses the need of trusting the knowledge base, including assumptions and potential biases introduced by the analysts. The lack of prediction capacity [135] and accuracy [136] of a PRA has been recognized, partly associated to the fact that historical data cannot account for future performances and risk assessments operate in a future timeframe. Magnusson [137] recognized this in 1997 and called to apply PRA from first principles, as data availability is a problem without a clear solution in sight in the context of the built environment. Some solutions like BIM and IoT have been proposed, but are unlikely to provide reliable data at a large scale in the near future. When the knowledge base is not strong enough, the precautionary principle becomes relevant (see section 1.3), calling to focus on the potential consequences rather than on the likelihood information.

In Eq. 1, the frequency of occurrence ( ) can be of the order of 10-3 fires/year as shown in

section 3.4.1. This rate can be reduced in𝐹𝐹𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 several orders of magnitude when combined with the low probabilities of failure of the safety measures considered in the fire scenario. This requires a detailed and valid knowledge base for , which is seldom the case in FSE (section 3.7).

The common practice of using simple calculations𝐹𝐹𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 from available data to obtain frequencies and probabilities ( ) worsen this issue, as exemplified in Box 3.1. This lack of credibility of

frequencies and probabilities𝑝𝑝𝑖𝑖 is particularly relevant for unique or highly complex buildings, for which scarce or none data is available. It is not easy to define what a unique or highly complex building is, making the matter of applicable data even more challenging.

Despite a favourable result of a PRA, an unacceptable loss can still occur under the correct combination of (unforeseen/overlooked) circumstances. Prescribed scenarios attempt to solve this issue by creating a common set of conditions that enables comparison of different PRAs, but do not mitigate subjective judgments nor model uncertainty. These scenarios can also bias the assessment, as additional (and relevant) scenarios might be overlooked. The Warren Centre [138] concluded that these scenarios confuse demonstrating safety with comparing levels of safety provided by deemed-to-satisfy solutions. This is a dangerous solution when time and resources are limited and compliance is favoured. Although informative, PRAs do not reflect a full picture of what society considers “unacceptable” as reflected by recent events leading to outrage and reform. The previous arguments regarding the fire frequency and the subjective basis of PRA support Claim 1.

The previous consideration limits the use of PRA as a design tool to provide insight on alternative selection and cost-benefit analysis on top of an inherently safe design process.

Chapter 3. Probabilistic risk assessment in FSE

However, this is not reflected by the neat numerical results presented in an F-N curve or as a measure of individual risk. Only expert practitioners with experience in this type of assessments could infer the quality of the scenario set by looking at the risk profile, requiring an in-depth peer review of the analysis to determine whether it is trustworthy or not. Even a peer review process might not be able to account for all the assumptions made by the analyst team and ensure their validity, unless these are clearly formulated. This translates into a risk insight that may seem complete and detailed, but could omit important hazards, failure modes and scenarios that have a critical role in quantifying fire safety performance. Overall, this supports Claim 2 related to the strength of PRAs in performing comparative assessments, instead of absolute.

As the presented limitations of PRA cannot be fully solved and pose an important danger, it is critical to conceptualize an alternative methodology. Such an alternative should aim at providing trustworthy insight on the level of fire safety performance of a building and its selected fire safety strategy. It is also clear that the abundant subjectivity and lack of knowledge make it impossible to produce a methodology free of limitations. This poses the challenge of using these limitations within the risk assessment to better understand risk and how to manage it.

Chapter 3. Probabilistic risk assessment in FSE

Bibliography 1. Ramachandran, G. and D.A. Charters, Quantitative risk assessment in fire safety Ganapathy Ramachandran and David A. Charters, ed. C. Ebooks. 2011, London ; New York: London ; New York : Spon Press. 2. Sabapathy, P., A. Depetro, and K. Moinuddin, Probabilistic Risk Assessment of Life Safety for a Six-Storey Commercial Building with an Open Stair Interconnecting Four Storeys: A Case Study. Fire Technology, 2019. 3. Johansson, U.C., Quantifying Risk for Deemed-to-Satisfy Apartment Buildings. 2010, Department of Fire Safety Engineering and Systems Safety, Lund University: Lund, Sweden. 4. Gernay, T., et al., Efficient uncertainty quantification method applied to structural fire engineering computations. Engineering Structures, 2019. 183: p. 1-17. 5. Aven, T., Foundations of risk analysis. 2nd ed.. ed. 2012: Hoboken, N.J. : Wiley. 6. Aven, T., Quantitative Risk Assessment: The Scientific Platform. 2011. 7. Lundin, J., Model Uncertainty in Fire Safety Engineering. 1999, Lund University. 8. Paté-Cornell, M.E., Uncertainties in risk analysis: Six levels of treatment. Reliability Engineering & System Safety, 1996. 54(2): p. 95-111. 9. Kaplan, S. and B.J. Garrick, On The Quantitative Definition of Risk. Risk Analysis, 1981. 1(1): p. 11-27. 10. (CCPS), C.f.C.P.S., Guidelines for Hazard Evaluation Procedures. 2010, Hoboken: American Institute of Chemical Engineers. 11. (BSI), B.S.I., BS 7974:2019, Application of fire safety engineering principles to the design of buildings - Code of practice. 2019: BSI. 12. Ale, B., P. Burnap, and D. Slater, On the origin of PCDS – (Probability consequence diagrams). Safety Science, 2015. 72: p. 229-239. 13. Pasman, H. and G. Reniers, Past, present and future of Quantitative Risk Assessment (QRA) and the incentive it obtained from Land-Use Planning (LUP). Journal of Loss Prevention in the Process Industries, 2014. 28: p. 2-9. 14. Goerlandt, F., N. Khakzad, and G. Reniers, Validity and validation of safety-related quantitative risk analysis: A review. Safety Science, 2016. 15. Aven, T., Supplementing quantitative risk assessments with a stage addressing the risk understanding of the decision maker. Reliability Engineering & System Safety, 2016. 152: p. 51-57.

Chapter 3. Probabilistic risk assessment in FSE

16. Dirk P. Kroese, T.T., Zdravko I. Botev, Variance Reduction. Handbook of Monte Carlo Methods, 2011: p. 347-380. 17. Devaney, S., Development of software for reliability based design of steel framed structures in fire. 2015, The University of Edinburgh. 18. Robert, C.P. and G. Casella, Monte Carlo Integration, in Monte Carlo Statistical Methods, C.P. Robert and G. Casella, Editors. 2004, Springer New York: New York, NY. p. 79-122. 19. Ata, M.Y., A convergence criterion for the Monte Carlo estimates. Simulation Modelling Practice and Theory, 2007. 15(3): p. 237-246. 20. M. Paté-Cornell, P.F., Rationality and risk uncertainties in building code provisions, in Sixth Conference on Applied Statistical Problems in Civil Engineering (ICASP6). 1991: Mexico City, Mexico. 21. Van Coile, R., et al., An Unbiased Method for Probabilistic Fire Safety Engineering, Requiring a Limited Number of Model Evaluations. Fire Technology, 2017. 53(5): p. 1705-1744. 22. Colson, A.R. and R.M. Cooke, Expert Elicitation: Using the Classical Model to Validate Experts’ Judgments. Review of Environmental Economics and Policy, 2018. 12(1): p. 113-132. 23. Tillander, K., Utilisation of statistics to assess fire risks in buildings, V. Publications, Editor. 2004, VTT: Espoo. 24. Richard D. Peacock, P.A.R., Glenn P. Forney, CFAST – Consolidated Model of Fire Growth and Smoke Transport, Volume 2: User’s Guide, U.S.D.o. Commerce, Editor. 2017. 25. Proulx, G. Occupant Behaviour and Evacuation. in 9th International Fire Protection Symposium. 2001. Munich, Germany. 26. G. Proulx, R.F.F., The Time Delay to Start Evacuation: Review of Five Case Studies Fire Safety Science, 1997. 5: p. 783-794. 27. E. D. Kuligowski, B.L.H., Occupant Behavior in a High-rise Office Building Fire. 2010, National Institute of Standards and Technology (NIST). 28. 2020, C.o.A.a.S.a.T., Handbook: Fire Safety Verification Method. 2020, Canberra, Australia: Australian Building Codes Board (ABCB). 29. Draft Fire Engineering Report: Building 6 – Mixed-use office building (Class 5, Class 6 and Class 7A), in ABCB Fire Safety Verification Method Calibration Study. 2017, FPA Australia: Melbourne, Australia.

Chapter 3. Probabilistic risk assessment in FSE

30. Reducing risks, protecting people - HSE’s decision-making process, H.a.S. Executive, Editor. 2001: Norwich, England. 31. Hayns, M.R., The Evolution of Probabilistic Risk Assessment in the Nuclear Industry. Process Safety and Environmental Protection, 1999. 77(3): p. 117-142. 32. Leveson, N., Safety III: A Systems Approach to Safety and Resilience. 2020, Massachusets Institute of Technology (MIT). 33. Meacham, B., Ultimate Health & Safety (UHS) Quantification: Individual and Societal Risk Quantification for Use in National Construction Code (NCC). 2016, The Australian Building Codes Board. 34. Bjelland, H., et al., The Concepts of Safety Level and Safety Margin: Framework for Fire Safety Design of Novel Buildings. Fire Technology, 2015. 51(2): p. 409-441. 35. Beard, A.N., A logic-tree approach to the St Crispin Hospital fire. Fire Technology, 1983. 19(2): p. 90-102. 36. Hopkin, C., et al., Design Fire Characteristics for Probabilistic Assessments of Dwellings in England. Fire Technology, 2019. 37. Yin, K. and J. Jiang, An application of internet of things in the field of urban building fire safety. International journal of safety and security engineering, 2014. 4(2): p. 135- 142. 38. Cheng, M.-Y., et al., BIM integrated smart monitoring technique for building fire prevention and disaster relief. Automation in Construction, 2017. 84: p. 14-30. 39. Maslen, S. and J. Hayes, Experts under the microscope: the Wivenhoe Dam case. Environment Systems and Decisions, 2014. 34(2): p. 183-193. 40. Hollnagel, E., Is safety a subject for science? Safety Science, 2014. 67: p. 21-24. 41. Bjelland, H. and A. Borg, On the use of scenario analysis in combination with prescriptive fire safety design requirements. Environment Systems & Decisions, 2013. 33(1): p. 33-42. 42. Bjelland, H. and T. Aven, Treatment of uncertainty in risk assessments in the Rogfast road tunnel project. Safety Science, 2013. 55: p. 34-44. 43. Amundrud, Ø. and T. Aven, On how to understand and acknowledge risk. Reliability Engineering & System Safety, 2015. 142: p. 42-47. 44. Aven, T., O. Renn, and E.A. Rosa, On the ontological status of the concept of risk. Safety Science, 2011. 49(8): p. 1074-1079. 45. Analysis, C.o.F.o.R., Society for Risk Analysis Glossary. 2015, Society for Risk Analysis.

Chapter 3. Probabilistic risk assessment in FSE

46. Aven, T., Perspectives on the nexus between good risk communication and high scientific risk analysis quality. Reliability Engineering & System Safety, 2018. 178: p. 290-296. 47. Standardization, I.O.f., ISO 31000:2018: Risk management - Principles and guidelines. 2018: Geneva, Switzerland. 48. Aven, T. and O. Renn, On risk defined as an event where the outcome is uncertain. Journal of Risk Research, 2009. 12(1): p. 1-11. 49. Administration, U.S.F., National Fire Incident Reporting System. 2020. 50. Selected Housing Characteristics, U.S.C. Bureau, Editor. 2020: Washington, DC, USA. 51. Dwelling Stock Estimates: 2016, England, D.f.C.a.L. Government, Editor. 2017: London, UK. 52. Rackwitz, R., The philosophy behind the life quality index and empirical verifications. JCSS Basic Documents on Risk Assessment in Engineering, 2008. 53. GeoNet. Earthquake Statistics. 2020 20/08/2020]; Available from: https://www.geonet.org.nz/about/earthquake/statistics. 54. Hadjisophocleous, G.V. and J.R. Mehaffey, Fire Scenarios, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 1262-1288. 55. Manes, M. and D. Rush, Assessing fire frequency and structural fire behaviour of England statistics according to BS PD 7974-7. Fire Safety Journal, 2020: p. 103030. 56. Manes, M. and D. Rush, A Critical Evaluation of BS PD 7974-7 Structural Fire Response Data Based on USA Fire Statistics. Fire Technology, 2018: p. 1-51. 57. Fire and rescue incident statistics, England, year ending March 2020, in Incident Recording System. 2020, Home Office of the Government of the United Kingdom: London, UK. 58. Fischhoff, B., et al., How safe is safe enough? A psychometric study of attitudes towards technological risks and benefits. Policy Sciences, 1978. 9(2): p. 127-152. 59. Hackitt, J., Building a Safer Future, Independent Review of Building Regulations and Fire Safety: Final Report. 2018: London, UK. 60. Grosshandler, W.L., et al., Report of the Technical Investigation of The Station Nightclub Fire (NIST NCSTAR 2), Volume 1. 2005. 61. Katrin Bennhold, J.G., Notre-Dame’s Safety Planners Underestimated the Risk, With Devastating Results, in The New York Times. 2019: New York, USA.

Chapter 3. Probabilistic risk assessment in FSE

62. Brazil's national museum hit by huge fire. BBC News, 2018. 63. Fire tragedy at Kemerovo shopping mall leaves at least 64 dead, in TASS. 2018. 64. Isle of Man 'shame' over Summerland fire disaster. BBC News, 2013. 65. Tribunal of Inquiry on the Fire at the Stardust, A., Dublin on the 14th February, and R. Keane, Report of the Tribunal of Inquiry on the Fire at the Stardust, Artane, Dublin on the 14th February, 1981. 1982: Stationary Office. 66. Disaster, F.B.a.F.S.I.o.t.W.T.C., Final Report on the Collapse of the World Trade Center Towers, N.N. 1, Editor. 2005, National Institute of Standards and Technology (NIST): Gaithersburg, MD, USA. 67. SP Schwartz, W.L., L Berenson, RD Williams, Deaths in World Trade Center Terrorist Attacks --- New York City, 2001, C.f.D.C.o.t.U. States, Editor. 2002. 68. S. Carter, A.C., One 9/11 Tally: $3.3 Trillion. The New York Times, 2011. 69. Sturcke, J., Fire that killed six people caused by faulty TV set, say investigators. The Guardian, 2009. 70. genco, G., Municipal Building Surveyor report - Lacrosse Building Fire. 2015, City of Melbourne: Melbourne, Australia. 71. Jonathan Newby, C.P.a.M.D., Lacrosse Judgment Summary. 2019, Colin Biggers & Paisley Lawyers: Online. 72. Torero, J., Grenfell tower: Phase 1 report. 2018, Torero, Abecassis Empis and Cowlard. 73. Johnson, P. and N. Lobel, Fire Safety Verification Method – The Australia Research Experience. Journal of Physics: Conference Series, 2018. 1107: p. 042033. 74. Reference Manual Bevi Risk Assessments. 2009, National Institute of Public Health and the Environment (RIVM): Bilthoven, The Netherlands. 75. Sánchez-Silva, M. and G.-A. Klutke, Reliability and Life-Cycle Analysis of Deteriorating Systems by Mauricio Sánchez-Silva, Georgia-Ann Klutke. 1st ed. 2016.. ed. 2016, Cham: Cham : Springer International Publishing : Imprint: Springer. 76. Van Coile, R., et al., The meaning of Beta: background and applicability of the target reliability index for normal conditions to structural fire engineering. Procedia Engineering, 2017. 210: p. 528-536. 77. (CCPS), C.f.C.P.S., Guidelines for developing quantitative safety risk criteria. 2009, New York : Hoboken, N.J.: CCPS, Wiley. 78. Crowley, H., V. Silva, and L. Martins, Seismic Design Code Calibration Based on Individual and Societal Risk. 2018.

Chapter 3. Probabilistic risk assessment in FSE

79. Van Coile, R., et al., The Need for Hierarchies of Acceptance Criteria for Probabilistic Risk Assessments in Fire Engineering. Fire Technology, 2018. 80. Pitblado, R., et al., International comparison on the application of societal risk criteria. Process Safety Progress, 2012. 31(4): p. 363-368. 81. Starr, C., Social Benefit versus Technological Risk. Science, 1969. 165(3899): p. 1232. 82. (BSI), B.S.I., PD 7974-7:2019 Application of fire safety engineering principles to the design of buildings. Probabilistic risk assessment. 2019, London, UK. 83. Kaplan, S., Y.Y. Haimes, and B.J. Garrick, Fitting Hierarchical Holographic Modeling into the Theory of Scenario Structuring and a Resulting Refinement to the Quantitative Definition of Risk. Risk Analysis, 2001. 21(5): p. 807-807. 84. Guidelines for Hazard Evaluation Procedures. 2011: Wiley. 85. Marks, L.B., T.A. Pawlicki, and J.A. Hayman, Learning to Appreciate Swiss Cheese and Other Industrial Engineering Concepts. Practical Radiation Oncology, 2015. 5(5): p. 277-281. 86. Aven, T., Ignoring scenarios in risk assessments: Understanding the issue and improving current practice. Reliability Engineering & System Safety, 2016. 145: p. 215-220. 87. Casson Moreno, V., et al., A consequences-based approach for the selection of relevant accident scenarios in emerging technologies. Safety Science, 2019. 112: p. 142-151. 88. Denning, R.S. and R.J. Budnitz, Impact of probabilistic risk assessment and severe accident research in reducing reactor risk. Progress in Nuclear Energy, 2018. 102: p. 90-102. 89. Taylor, N., et al., Updated safety analysis of ITER. Fusion Engineering and Design, 2011. 86(6): p. 619-622. 90. Zhang, X., Efficient Computational Methods for Structural Reliability and Global Sensitivity Analyses. 2013, UWSpace. 91. Baybutt, P., On the completeness of scenario identification in process hazard analysis (PHA). Journal of Loss Prevention in the Process Industries, 2018. 92. Kadlic, M. and V. Mózer, Uncertainties Associated with Tunnel Design Fire Scenarios. Procedia Engineering, 2017. 192: p. 387-392. 93. Hasofer, A.M. and I. Thomas, Analysis of fatalities and injuries in building fire statistics. Fire Safety Journal, 2006. 41(1): p. 2-14. 94. Van Coile, R., G. Jomaas, and L. Bisby, Defining ALARP for fire safety engineering design via the Life Quality Index. Fire Safety Journal, 2019. 107: p. 1-14.

Chapter 3. Probabilistic risk assessment in FSE

95. (ISO), I.O.f.S., ISO 16733-1 Fire safety engineering - Selection of design fire scenarios and design fires. 2015. 96. Shpilberg, D.C., The Probability Distribution of Fire Loss Amount. The Journal of Risk and Insurance, 1977. 44(1): p. 103-115. 97. Hasofer, A.M. and I.R. Thomas, Probability Distribution Of Fire Losses. Fire Safety Science, 2003. 7: p. 1063-1072. 98. CIBSE Guide E - Fire Safety Engineering (3rd Edition). CIBSE Guide E - Fire Safety Engineering. 2010: CIBSE. 99. Gano, D.L., Apollo root cause analysis : a new way of thinking. 2nd ed.. ed. New way of thinking. 2003, Yakima, Wash.: Apollonian Publications. 100. Peter Shergold, B.W., Building Confidence – Improving the effectiveness of compliance and enforcement systems for the building and construction industry across Australia. 2018, Building Ministers’ Forum (BMF). 101. Usmani, A.S., Y.C. Chung, and J.L. Torero, How did the WTC towers collapse: a new theory. Fire Safety Journal, 2003. 38(6): p. 501-533. 102. Miller, T.H., Live Oak/Milstar Complex and Carpet Service Center, F.E.M.A. (FEMA), Editor. 1995, U.S. Fire Administration (USFA): Emmitsburg, Maryland, USA. 103. C/VM2 Verification Method: Framework for Fire Safety Design, I.a.E. The Ministry of Business, Editor. 2012: Wellington, New Zealand. 104. C/VM2 Verification Method: Framework for Fire Safety Design, I.a.E. The Ministry of Business, Editor. 2017: Wellington, New Zealand. 105. Hall, J.R. and A. Sekizawa, Revisiting Our 1991 Paper on Fire Risk Assessment. Fire Technology, 2010. 46(4): p. 789-801. 106. Cameron, I., et al., Process hazard analysis, hazard identification and scenario definition: Are the conventional tools sufficient, or should and can we do much better? Process Safety and Environmental Protection, 2017. 110: p. 53-70. 107. Cox Jr, L.A., D. Babayev, and W. Huber, Some Limitations of Qualitative Risk Rating Systems. Risk Analysis, 2005. 25(3): p. 651-662. 108. Kobayashi, Y., Frequency of ignition of non-residential buildings in Japan. Fire Safety Journal, 2017. 91: p. 1035-1043. 109. Fischer, K., et al., Wirtschaftliche Optimierung im vorbeugenden Brandschutz. Ibk Report, 2012. 338. 110. (NFPA), N.F.P.A., Workshop Report: Today and Tomorrow’s Fire Data 2014.

Chapter 3. Probabilistic risk assessment in FSE

111. McDermott, H., R. Haslam, and A. Gibb, Occupant interactions with self-closing fire doors in private dwellings. Safety Science, 2010. 48(10): p. 1345-1350. 112. MacLeod, J., S. Tan, and K. Moinuddin, Reliability of fire (point) detection system in office buildings in Australia – A fault tree analysis. Fire Safety Journal, 2020: p. 103150. 113. Hall, J.R., U.S. experience with sprinklers. 2013, National Fire Protection Association: Quincy, MA, USA. 114. Moinuddin, K.A.M., J. Innocent, and K. Keshavarz, Reliability of sprinkler system in Australian shopping centres –A fault tree analysis. Fire Safety Journal, 2019. 105: p. 204-215. 115. Smith, D.J., Chapter 13 - Field Data Collection and Feedback, in Reliability, Maintainability and Risk (Ninth Edition), D.J. Smith, Editor. 2017, Butterworth- Heinemann. p. 209-220. 116. Smith, D.J., Chapter 4 - Realistic Failure Rates and Prediction Confidence, in Reliability, Maintainability and Risk (Ninth Edition), D.J. Smith, Editor. 2017, Butterworth-Heinemann. p. 43-57. 117. Andriy, R., K. Dana, and K. Jens-Uwe, Guidelines for Analysis of Data Related to Ageing of Nuclear Power Plant Components and Systems, in EUR - Scientific and Technical Research Reports. 2009, Joint Research Centre Institute for Energy. 118. (ISO), I.O.f.S., ISO 16732-1 Fire safety engineering. Fire risk assessment. General. 2012. 119. Kalra, N. and S.M. Paddock, Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice, 2016. 94: p. 182-193. 120. Downer, J., When the Chick Hits the Fan: Representativeness and Reproducibility in Technological Tests. Social Studies of Science, 2007. 37(1): p. 7-26. 121. Downer, J., When failure is an option: redundancy, reliability and regulation in complex technical systems. LSE Research Online Documents on Economics, 2009. 122. Herkert, J., J. Borenstein, and K. Miller, The Boeing 737 MAX: Lessons for Engineering Ethics. Science and Engineering Ethics, 2020. 123. Aven, T., On the Need for Restricting the Probabilistic Analysis in Risk Assessments to Variability. Risk Analysis, 2010. 30(3): p. 354-360.

Chapter 3. Probabilistic risk assessment in FSE

124. Long, R., N. Wu, and A. Blum, Lessons Learned From Unsatisfactory Sprinkler Performance: An update on trends and a root cause discussion from the investigating engineer's perspective. Fire Protection Engineering, 2010. 48: p. 26. 125. De Sanctis, G. and M. Fontana, Risk-based optimisation of fire safety egress provisions based on the LQI acceptance criterion. Reliability Engineering & System Safety, 2016. 152: p. 339-350. 126. Abecassis-Empis, C., Analysis of the compartment fire parameters influencing the heat flux incident on the structural façade. 2010, The University of Edinburgh: Edinburgh, UK. 127. Ministry of Housing, C.L.G., Structure: Approved Document A, in The Building Regulations 2010, H. Government, Editor. 20109: London, UK. 128. Britain, G., The Building Regulations. Approved Document B, Volume 2–Buildings other than Dwellings. 2011, Newcastle-Upon-Tyne: NBS. 129. DCLG, Fire safety approved document B, volume 1–dwellinghouses (2006 edition incorporating 2010 amendments). 2010, DCLG London, UK. 130. Norway, S., SN-INSTA/TR 951:2019 - Fire Safety Engineering - Guide for Probabilistic Analysis for Verifying Fire Safety Design in Buildings. 2019. 131. Kirchsteiger, C., On the use of probabilistic and deterministic methods in risk analysis. Journal of Loss Prevention in the Process Industries, 1999. 12(5): p. 399-419. 132. Gehandler, J., The theoretical framework of fire safety design: Reflections and alternatives. Fire Safety Journal, 2017. 133. Pasman, H. and W. Rogers, How trustworthy are risk assessment results, and what can be done about the uncertainties they are plagued with? Journal of Loss Prevention in the Process Industries, 2018. 55: p. 162-177. 134. Pasman, H.J., W.J. Rogers, and M.S. Mannan, Risk assessment: What is it worth? Shall we just do away with it, or can it do a better job? Safety Science, 2017. 135. Goerlandt, F. and G. Reniers, Prediction in a risk analysis context: Implications for selecting a risk perspective in practical applications. Safety Science, 2018. 101: p. 344- 351. 136. Aven, T., Risk assessment when the objective is accurate risk estimation, in Quantitative Risk Assessment: The Scientific Platform, T. Aven, Editor. 2011, Cambridge University Press: Cambridge. p. 51-75. 137. Magnusson, S.E., Risk assessment. Fire Safety Science, 1997. 5: p. 41-58.

Chapter 3. Probabilistic risk assessment in FSE

138. D. Lange, J.T., A. Osorio, N. Lobel, C. Maluk, J. Hidalgo, Fire Safety Engineering - The Methods Report, T.W. Centre, Editor. 2019: Sydney, Australia.

4 Comparing contexts: Major Hazardous Facilities and Fire Safety Engineering

Transfer of knowledge between disciplines is something natural in today’s world. This is particularly true for all matters related to risk and risk-based approaches [1]. And it is no less true for fire safety engineering, where the introduction of goal-based and later of risk based approaches coincided with the move of high-risk industries away from prescriptive approaches [2]. In the context of performance-based design, fire risk assessments lay in the intersection of social sciences, fire dynamics and safety science [3]. Therefore, it is natural to expect that FSE will make use of the learnings, methods and tools of other disciplines, particularly those of ‘higher’ risk. As mentioned both in Chapters 1 and 2, fire risk assessments differ from risk assessments in other disciplines in that the former are used to demonstrate adequate performance.

Implementing risk assessment methodologies from a different discipline into FSE can foster advancement. However, it can also lead to setbacks if their limitations are not carefully adapted to the specific context, i.e. the built environment. A particular example is the implementation of methods and tools developed for Major Hazardous Facilities (e.g. safety cases, PRAs, societal risk criteria) in order to support performance-based design in FSE. This chapter presents a detailed analysis of the main differences between contexts and the limitations arising when attempting to implement a PRA. Following Chapter 3, this chapter provides further evidence to answer research question 1 (“Are PRAs the adequate method for explicitly assessing fire safety performance?”).

4.1 The MHF experience Meacham [4] and Wolski et al. [5] argue that individual and societal risk criteria should be the benchmark for performance-based requirements in FSE. Meacham supported this argument with the application of this approach in the field of chemical process safety, with specific emphasis on Major Hazard Facilities (MHF). These are industrial facilities where the amount and nature of chemicals handled pose significant risks of fire, explosions and other high consequence events that can produce significant damage beyond the facility’s boundaries. As decades of application and refinement have led to robust regulatory frameworks for MHFs using individual and societal risk benchmarks, Meacham argues that it follows that such approach is applicable to Fire Safety Engineering. Furthermore, Meacham argues that such an

Chapter 4. Comparing contexts: MHF and FSE

approach acknowledges that loss is inevitable and therefore its likelihood must be understood and managed, which is in line with Kaplan’s quantitative definition of risk [6].

However, the previous arguments are debatable. Assuming that the methods and tools in MHF have been successful, one must ask why it would also work for FSE. In her work regarding uncertainties and risk analysis, Paté-Cornell [7] explained how a ‘deterministic’ risk analysis can be as valuable as a full PRA on the condition that it is consistent with the decision making that follows. In her work, Paté-Cornell presents a six level classification of the representation of uncertainty in risk analysis. A careful reading of this classification makes it clear that it does not imply that any of the levels is the ‘ideal one’. Each level presents advantages and drawbacks, e.g. deterministic analyses provide clear single-point answers with the drawback of unmeasurable conservatism. This is consistent with the hierarchy of acceptance concepts in FSE proposed by Van Coile et al. [8]. Van Coile analysed acceptance criteria used in MHF and in structural engineering, proposing a hierarchy of five different acceptance concepts, each one with a suitable use, clear limitations, and clear implications in terms of the level of responsibility being adopted by the designer. Both Paté-Cornell and Van Coile use examples of a full PRA in contexts where uncertainties can be separated into epistemic and aleatoric, and the result clearly reflects them independently. Paté-Cornell uses a seismic problem, while Van Coile uses the probability of failure for a structural design. The latter is then reinforced with further research in which the epistemic uncertainty is maximized (i.e. the probability distributions are assumed unknown) and a set of system observations are carried out to estimate the probability of failure of a structural design [9]. These examples illustrate how the right method must be chosen for the right problem, which is not an obvious selection, and provide a counter argument to the use of PRA in FSE.

One must consider if MHF regulations or PRAs have been successful in managing risks in MHF. Lack of consideration of this argument has resulted in the adoption by FSE of tools and frameworks originally developed for MHF to demonstrate a desired level of performance in the context of fire safety [10, 11]. Analysing the major accident database eMARS of the European Union it is possible to relate the occurrence of major accidents with the introduction of MHF regulation, mainly the Seveso directives. The major accident data in Figure 4-1 corresponds to (21%) accidents causing property damage (>€2M on-site, >€0.5M off-site), (33%) injury to persons (≥1 fatality, ≥6 hospitalizing injuries), (31%) large chemical release, (7%) interesting lessons learnt and (7%) immediate damage to the environment. No clear trend exists between major accidents and regulation implementation. Furthermore, Figure 4-2 shows

Chapter 4. Comparing contexts: MHF and FSE

the relation between offshore hydrocarbon releases and the MHF regulation in the UK, also known as COMAH. This relation shows that even if the regulation is considered effective, the effects on results might be delayed over several years and require continuous improvements such as the updated regulations, COMAH15. On the other hand, Goertland [12] shows that quantitative risk assessments (QRAs, equivalent to PRAs) are not a flawless tool in managing major risks in the chemical industry and that benchmarking risk assessment results from different teams result in spreads of several (2-3) orders of magnitude. This is in part explained by Baybutts’ analysis of missed scenarios during process hazards analysis [13], which constitute the backbone of a QRA [14]. The presented evidence shows that MHF regulation is not a guarantee for reducing the likelihood of major accidents, despite significant benefits being obtained (notice that no reference is made to the consequences of the events). Therefore, the argument that applying methods from MHF in FSE will help demonstrating adequate fire safety performance is not self-evident. The reader is referred to Chapter 3 for a detailed discussion of uncertainty in PRAs and QRAs.

Upper tier Lower tier Unknown/Not Seveso 40 35 30 25 20 15 10

No. No. of Major Accidents 5 0 2020 2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 2001 2000 1999 1998 1997 1996 1995 1994 1993 1992 1991 1990 1989 1988 1987 1986 1985 1984 1983 1982 1981 1980 1979

Seveso III Seveso II Seveso Directive 2012/18/EC Directive 96/82/EC Directive 82/501/EC

Figure 4-1. Trend of major accidents reported in the European Union and their relation to MHF regulation [15].

Chapter 4. Comparing contexts: MHF and FSE

Major Minor Significant 400 350 300 250 200 150 100

No. ofHC Releases 50 0 2018 2017 2016 1998 1997 1996 1995 1994 1993 1992 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 2001 2000 1999

COMAH15 COMAH

Figure 4-2. Trend of hydrocarbon releases in the UK's offshore industry and its relation with the COMAH regulation [16].

4.2 Objectives and purpose In MHFs the main objective of the safety case is to prevent the occurrence of major accidents as well as limiting their consequences as exemplified by the regulation and guidance of Australia [17], the European Union [18], the United Kingdom [19] and the United States [20, 21]. Except for the United States, all other regulations require a safety case for an MHF in which the operator demonstrates explicitly how major accidents are prevented and their consequences minimized. The previous implies that the expected benefit from the chemical operations is associated to the risk of a highly harmful event happening, and therefore the solution is to prevent it and minimize its consequences. To achieve this the regulations set out the operator to understand the risk of their operation and therefore construct a reasonable and proportional risk treatment plan that is then subject to approval by the AHJ. This is the ultimate objective of such exercise and specific objectives such as ensuring life safety, protecting the environment and ensuring business continuity are embedded within the considerations of preventing major accidents and minimizing their consequences. In order to ensure that such an objective is achieved, most of these regulations set out responsibilities for the operator and for the AHJ along with a list of minimum requirements (Figure 4-3).

Chapter 4. Comparing contexts: MHF and FSE

Major hazard facility

Potential for major accident Facility classification Generation of guidance Hazards identification Safety Case evaluation Operator’s Risk assessment … AHJ’s responsibilities responsibilities … Inspection, vigilance and control Construction, submission and scheme implementation update of Safety Case

Prevention of major accidents & Minimization of consequences

Figure 4-3. General structure for MHF regulation

To contrast the presented focus of MHF consider the FSE context, taking the SFPE Handbook [22], BS 7974:2019 [23] and the NCC of Australia [24] as key references. Fire safety objectives are one of many objectives to be achieved within construction codes, and those specific to fire safety include ensuring life safety in case of a fire amongst others (e.g. property protection) which, depending on the legislation, might be optional or up to the developer to consider. As these objectives are formulated in a qualitative manner, complying with them requires formulating performance objectives that can be translated to quantifiable and. / or verifiable criteria. The approval process then checks that all safety objectives are achieved whether through deemed-to-satisfy provisions or through the achievement of performance requirements. With this process, analogous to a verification, the design is either adjusted to comply or accepted for construction. From the previous, some key differences emerge. First, the design philosophies are quite different in MHF and in the built environment. Second, there is an assumption of tractability to fire safety, i.e. it is assumed that fire safety is the result of a set of requirements that can be independently checked. Third, the focus in fire safety is compliance while in MHF the focus is managing risk.

A first difference relates to how the regulation guides design philosophies. In principle, both an industrial facility and a residential building have the potential for unwanted events (e.g. fire) to cause large consequences, and therefore regulation should aim at understanding and managing such risks in the most appropriate manner. This is the case for MHF, as any design errors could introduce a threat (i.e. a trigger) that can then enable a major accident. This covers all aspects of the MHF including construction materials, utilities, administrative areas and amenities for employees, security devices (think electronic surveillance devices), etc. The design philosophy for MHFs is ensuring that a major accident is prevented or its consequences minimized and this requires understanding and managing the risks, beyond what the design

Chapter 4. Comparing contexts: MHF and FSE

standards and best practices require. Devastating events such as Flixborough, Piper Alpha, Texas City and Deepwater Horizon [25] helped understand that following rules is not enough and promoted the use of design philosophies such as inherently safe design [26].

The result of applying the inherently safer philosophy to all aspects of an MHF helps ensuring that the overarching objective is continuously achieved, and therefore that the operation can continue as intended. On the other hand, fire safety compliance typically relies on a set of requirements that currently can be interpreted in isolation from other requirements. The NCC of Australia points out specific dependencies between fire safety and other requirements, but does not state an explicit requirement to understand fire safety in a holistic manner. The NCC [24] does point out the importance of an holistic approach in performance solutions, but not explicitly for fire safety, nor it gives explicit guidance on how to achieve it. The recommendations proposed by Hackitt [27] respond to this, pointing out the need for well- defined responsibilities for key roles in the design process where principals designers must demonstrate their understanding and management of risk “so far as is reasonably practicable” (recommendation 2.1). The report of Shergold-Weir [28] in Australia also identifies such a need in Recommendation 8, highlighting the need for early engagement of fire authorities in the design process to aid in the fire safety design.

The second difference lays in the understanding of fire safety as a tractable element of the whole building. Such an assumption is common place throughout FSE guidelines and is consistent with the way many engineering disciplines have evolved [29]. The behaviour of a building when a fire occurs depends not only on the characteristics of the fire and of a set of safety measures, but also on a wide range of features given by the design of the building that directly impact the fire safety strategy. For example, a designers’ choice of floor-to-ceiling height has a direct impact on the time for a smoke layer to accumulate. The selection of an architectural design might lead to a structural design that in turn defines the compartmentation possibilities and the travel distances. By de-coupling fire safety from many aspects related to the design of the system, there is an assumption that it can be treated independently, which is not supported considering the simple examples presented. Assuming that fire safety is a tractable aspect of a building and that it can be achieved through compliance with a set of requirements constitutes a major divergence from MHF. While in MHF the objective cannot be achieved without a sound and comprehensive understanding of the risk, in FSE the performance requirements are treated in isolation and through simplification until compliance is achieved. Nevertheless, the complexity in MHF is such (as in many engineering systems)

Chapter 4. Comparing contexts: MHF and FSE

that assumptions of tractability are unavoidable and unavoidably lead to events and the need for continuous improvement through learning.

The third difference is rooted in the evolution of FSE as a discipline. The regulation of FSE around the world was mainly prescriptive (rule-based) until a few decades ago, when design requirements started departing significantly from the boundaries of application of these prescriptive rules, e.g. large open plans, large atria, complex geometries and innovative materials. To solve the conflict an alternative was provided (first in some countries) with the performance-based design, which requires the definition of performance requirements that in turn allow to evaluate whether safety objectives are achieved. However, the unique characteristics of the design process in FSE (see section 4.5) make this a problem of compliance rather than of risk assessment. In FSE a non-compliance with the requirements leads to restarting the process with an alternative design until a successful design is found (i.e. trial designs concept [10]). This yields a compliance exercise and it is supported by the need for strengthened peer review, as called for by Hackitt [27] and Shergold-Weir [28].

An example of the compliance approach is that of using predefined fire scenarios, which is deemed ineffective since the fire scenarios required for design are by definition a function of the building, and therefore cannot be pre-defined [30]. The previous is reinforced by Bjelland’s [31] conclusions on comparative analysis to demonstrate adequate performance. On the other hand, the regulation for MHF clearly states that the end goal is preventing major accidents and to minimize their consequences. As previously analysed, this has an impact on the design philosophy but it also sets a high-level requirement that cannot be achieved by simply complying with a set of specific requirements. This is coherent to the unique features of each industrial facility, the hazards posed by the chemicals and the operating conditions. Predefining requirements would set a template that hardly any facility would fit. For buildings with unique design features where prescriptive regulation does not apply, it becomes necessary to make the case for why the building’s design and operating conditions allow continuously achieving fire safety, i.e. a holistic approach.

Three differences haven been presented here concerning design philosophies, tractability and the focus of the discipline (compliance/safety). The comparisons show that tractability is inherent to engineering practice, even when a holistic approach is chosen, as the complexities of system failure are seldom fully understood. However, the design philosophy and discipline approach are significantly different between MHF and FSE. This is highlighted by Spinardi

99 Chapter 4. Comparing contexts: MHF and FSE

and Law [32] when identifying limitations to Hackitt’s recommendations of applying an outcome-based, safety case like, framework for FSE. Based on these differences, application of the MHF framework into FSE does not seem to constitute the ideal solution to the challenges of the latter.

4.3 Stakeholders The structure adopted in MHF regulation presented in Figure 4-3 shows that there are two clear stakeholders in the risk management process: an authority enforcing the regulation with approval powers and an operator who makes the case for why their operation is worthy of said approval. Both of these stakeholders share risk ownership as the regulator is allowing for the operation (and its risks) to be deployed in the territory, while the operator is directly benefiting from the industrial activity. Although different stakeholders can aid the operator in the design, construction and even in the operation of the facility, the ownership does not change. By involving third parties, there can be a degree of risk transference such as when the design is supported by a consultant. The consultant will share a degree of accountability in case a major event occurs as a product of a design error, but even then, it does not discharge the operator from their responsibility. This provides a clear continuity of risk ownership that enables the AHJ to implement inspection, vigilance and control and achieve the overarching goal of the MHF regulation. Furthermore, other stakeholders are considered from early stages of the design process, as neighbouring industries and communities as wells as the surrounding environment must be engaged in consultation. In FSE there is a significant large number of relevant stakeholders to take into account in the design and approval process. The stakeholder proposing the project can be a private citizen aiming to build a house for him/herself but typically, in large-scale projects it is a developer with a large financial leverage. This developer appoints designers, typically architects, to provide a conceptual design that is then the basis for early engineering designs focused on the structural aspect. At this point, the early design is the base for relevant engineering disciplines to propose designs for a range of aspects such as utilities, elevators, fire safety and others. Here two significant differences appear. First, the difference in competences between professionals involved in the design of a MHF and those involved in a fire safety design. Second, the discontinuity of risk ownership in FSE.

In an MHF there is an extremely wide range of professionals involved in the design of the facility, but typically the lead designers will be chemical or process engineers. The regulation pertaining MHF does not specify rules, nor verification methods that the engineers can use templates to achieve a safe design. These professionals use their competencies and an iterative

100

Chapter 4. Comparing contexts: MHF and FSE

process of reviews and adjustments (see next section regarding design process) to achieve a final design. All other engineering disciplines aiding the design, even at levels not directly related to the process are held accountable by their own code of ethics and are expected to provide adequate outcomes using their competences. Both process and other engineering practices are supported by large bodies of knowledge such as theory texts, guidelines and also by details standards and best practices. In sum, the MHF regulation only sets the responsibilities of the operator to demonstrate that the overarching objective can be achieved with the proposed design. This is exemplified in the Safe Work Australia guidelines for a safety case [33], where the personnel involved in the construction of the safety case are to be listed, but no guidance is given regarding their competencies. In FSE the situation is quite different, as both the Shergold- Weir [28] and Hackitt [27] identify issues with properly defining roles, responsibilities and the competencies within the design process of a building. Spinardi and Law [32] point out that architects lead the design process in the built environment (as this is an older profession that fire safety engineering) and typically involve fire safety engineers if required due to the degree of deviations from the code or if recommended by the AHJ. This is quite a significant difference, as both MHF and FSE address the explicit existence of risks with the potential for large consequence, yet in FSE fire engineering is only called upon in exceptional cases. Despite sharing the existence of significant risks, the drivers for the design process in FSE do not typically include fire safety concerns. This aligns with the analysis in the previous section regarding design philosophies and the assumption that fire safety is tractable, i.e. can be treated as an add-on made up of a list of independent requirements.

Now, consider the nature of the stakeholders in the actual risk of an MHF and a residential building. First at the design stage it is clear from the previous discussion that FSE has unresolved issues regarding competences, which impede the application of the MHF approach based on an overarching objective. The previous is analysed by Spinardi and Law [32] in relation of Hackitt’s proposal for a FSE outcome-oriented framework and reinforced by initiatives like the one of the Warren Centre [34] to have a unified definition of FSE professional competencies. Second, at the operational level the personnel in an MHF voluntarily accepts the workplace risks, requiring training and to comply with a certain level of competencies. Despite these (and many other) administrative controls, human error continues to be a significant factor of unwanted events at MHFs1. Now consider occupants

1 DNV failure frequency guidance indicates that operational error contributes to 35% of loss of primary contention events, ASME guidance on risk based inspection allocates 20% to the same cause and the eMARS database indicate 11% of the registered Major Accidents involved human error.

101

Chapter 4. Comparing contexts: MHF and FSE

behaviour: both the use of the building by the occupants as well as their response to a fire is fundamentally uncertain and designers must heavily rely on assumptions to account for it at the design stage, e.g. defining optimal evacuation paths [35]. Therefore, the level of variability expected from trained personnel at an MHF and that of occupants in a building introduce complexities that do not easily fit within the MHF regulation framework. This last point touches on uncertainty analysis and hints at the need for monitoring of fire safety throughout the building life-cycle; the evolution of risk as a function of time is analysed in section 4.11.

4.4 Risk ownership The issue of poorly defined roles, responsibilities and lack of competencies sheds light on another significant difference, risk ownership. Risk ownership defines a relationship between a stakeholder and the authority to manage said risk [36]. Chemical process safety pioneer Trevor Kletz said that “one cannot manage a risk arising from a hazard that has not been identified” [26] but risks can also go unmanaged if they do not have an owner. An extreme example of the issues that lack of risk ownership entail is seen in the bushfire context, where there is a plethora of stakeholders with high stakes and limited resources. Young et al. [37] proposed a risk ownership framework in which risk ownership is identified for the assets at stake, for the consequences of risks and for the risk management actions. Risk ownership is a key element of MHF regulation and are made explicit through the mentioned responsibilities for operator and authority. Typically, it is not an issue to identify risk ownership in a MHF, as without clarity on it the AHJ will not be able to provide a license to operate. However, change of ownership is more troublesome as major industrial facilities are acquired by new owners or start operating under new management. Lack of risk ownership in these cases has been a key factor in significant major accidents such as Texas City, Buncefield and Deepwater Horizon. In the last one, an offshore operation that resulted in eleven fatalities and incalculable environmental loss, the complexities of the large-scale multi-party project led by BP (the operator) were identified as one of the organizational factors leading to the event [38]. These events associated to MHFs illustrate the complexities of risk ownership in presence of a large array of stakeholders, even when all of those are held to high technical and administrative standards. The guidance for change of risk ownership in MHF [39] highlights these complexities and define priorities for the process, including a gradual handover between old and new owner in which potentially affected stakeholders are taken into account, e.g. neighbours and community.

102 Chapter 4. Comparing contexts: MHF and FSE

Risk ownership is not less complex in the context of the construction sector, which is a main driver of national economies (in Australia it constitutes about 9% of GDP [40]) and can introduce biases to the legislation as a result of lobbying and other effects not analysed here. In the construction sector risk ownership initially lays with the owner of the project, usually a private citizen or a company, i.e. a builder. The builder employs a designer and a constructor who support the design approval process and then manage the construction. In Australia, the constructor is liable for major defects (structural and non-structural) for a period of seven years after the construction is finalized2. At this point, the fire risk ownership should lay with the builder and those professionals involved in the design and in the construction. As Shergold- Weir point out this is not necessarily the case due to lack of well-defined roles and responsibilities. Once the construction is finalized and if the building is sold, the builder no longer has risk ownership. Such complex risk ownership is exemplified with the Lacrosse building fire where design issues were a critical factor enabling fire propagation along the façade in 2014. On 2019, the building’s owners claim for damages was resolved by a judge, showing a complex set of liabilities involving the architect, the builder and the fire engineer who together had to answer for A$12.7 million [41]. Hackitt [27] addresses the need for a clear model of risk ownership in the construction sector in the UK (Recommendation 2.3) irrespective of building’s size or complexity, which is likely to stem from decades of MHF risk management embodied in the Health & Safety Executive of the UK in which risk ownership is essential.

4.5 Design process In the previous sections there have been frequent mentions of the design processes at MHFs and in the construction sector. Major differences exist as the systems themselves, the technology involved and –mainly- the hazards involved are completely different. Construction of an MHF is infrequent due to the economic leverage required and the technical complexities involved. However, the construction of residential and non-residential buildings is common and as previously stated, a main driver of national economies. The function of chemical plants and buildings are also quite distinct, as the former aim at transforming raw material into products with added value and the latter aim at providing facilities for residential, commercial, recreational and societal means. Therefore, it is self-evident that the design processes have to

2 In case the constructor fails financially, a Home Warranty Insurance policy should cover the repair costs. However, this warranty has been removed from the requirements for constructors of buildings over three storeys high, introducing another complexity to the risk ownership issue.

103

Chapter 4. Comparing contexts: MHF and FSE

be different. The focus of this comparison is then not the design process itself, but how safety is considered in each one.

Seider’s text on product and process design [42] is a cornerstone for the formation of chemical and process engineers. In this text the foundations of the design process are established and safety is introduced along with the code of ethics in the earlier chapters. Here, we describe the design process based on the model of Tamim et al. [43] and the main chemical process safety principles proposed by Crowl [44] and the Center of Chemical Process Safety (CCPS) [45] (see Figure 4-4).

The general design process of a chemical plant begins with defining a reaction pathway for the desired product, where principles of inherently safer design can already be applied by eliminating, substituting or minimizing hazardous materials. Here, considerations of the maximum inventory can also be included in preliminary risk analyses and the same principles of inherently safer design can be applied. With a reaction pathway chosen a rough representation of the process is created based on blocks, which is then refined in the second step of the design process, where a detailed representation of the equipment and the flows is necessary. At this point hazard identification is not self-evident and cannot longer be based on simple substances inventories, requiring the implementation of more structured process hazard analysis (PHA) techniques such as the hazards and operability study (HAZOP).

Introduction of more structured PHA techniques with narrower focus becomes more relevant in order to identify potential problems and troubleshoot them. The next step involves estimating costs and here both energetic optimization alternatives as well as safety alternatives have a significant impact on the final estimation. Consistent with Paté-Cornell’s ideas about risk analysis for selection of alternatives, quantitative risk assessments are introduced at this point to evaluate the cost-benefit of different design options. The body of knowledge constructed up to this point regarding the facility, its hazards and the main risks supports constructing a safety case, with different safety measures being analogous to layers of protection. Iteration is a constant in this design process and the safety considerations can easily trigger feedback loops into the design of specific components, units or even the whole plant. After decades of accidents and lessons learnt, Seider [46] points out that the future of safer design processes rely on the extensive use of failure data to update our knowledge of failure modes and to perform dynamics risk analysis that help identify unique operation conditions that can render all layers of protection ineffective.

104

Chapter 4. Comparing contexts: MHF and FSE

Project conception

FEED Conceptual phase Pre-FEED (Front end engineering and Detailed engineering design) • Process safety knowledge • Hazardous substances base is created • Updated process safety • Updated process safety properties • Hazard identification (not knowledge base knowledge base • Inherently safer design scenario-based, e.g. what-ifs) • HAZOPs on detailed P&IDs • Hazardous area classification • Land use planning • Layers of protection • Consequence analysis • Management of change considerations identification • Layers of protection analysis • Safety instrumented systems • Best practices • QRA for alternative selection • QRA for alternative selection and safety integrity levels • Life cycle analyses and ALARP verification and ALARP verification (SIS/SIL)

To procurement and construction

Figure 4-4. Typical steps in the plant design process and safety elements; Based on [43-45]

The building design process is mainly driven by the client (not necessarily the final user) and an architect. It is the architect’s objective to transform the vision or requirement of the client into a design. The design is supported by several specialist engineering consultants and then evaluated. Currently, this process takes different paths depending on the jurisdiction and the architect, but in general can be represented as in Figure 4-5, based on the first five stages of the Plan of Work of the Royal Institute of British Architects (RIBA) [47]. However, architects tend to focus on the aesthetics and spatial features of buildings with or without awareness of potential conflicts these have with fire safety as described by Meacham [48], who exemplifies this complex relation with the TU Delft architecture building fire in 2008. Skinner [49] analyses architectural education in the state of New South Wales, Australia based on the curricula content associated to fire safety based on the ‘new’ challenges presented to architects when engaging in performance-base design. Surveys and analysis of the curricula show the need to enhance performance-based design educational content, while positioning FSE as one of the main design considerations. RIBA’s Plan of Work in Figure 4-5 is the result of a major overhaul (the largest in 57 years) making FSE considerations explicit throughout the whole process. Analysing the stages presented here, there is an important advancement in recognition of key fire safety elements, but it is clear that these are not the driving element of the design (and perhaps they are not ought to be). The lack of a risk assessment notion and of well-structured steps that support it throughout the process is a major difference with the design of a chemical process. One could overlook this and assume that FSE professionals will ensure the risk assessment is indeed carried out and used to support key design decisions. However, the

105

Chapter 4. Comparing contexts: MHF and FSE

understanding of risk assessment for FSE differs from that of MHF as highlighted by Clancy [50] when discussing the hazards identification step (this is further analysed in section 4.7).

Project conception

Strategic definition Preparation and briefing Concept design (Stage 0) (Stage 1) (Stage 2) • Stakeholders identification • Obtain feedback from stakeholders • Site suitability for high level fire • Develop fire safety requirements to (including specialists) on fire safety safety requirements include in the project brief measures • Identification of relevant fire-related • Fire strategies of existing buildings on • Align architectural design with fire trends, policy and legislation site safety strategy • Identify needs of specialist fire safety • Define fire safety specialist • Record of key fire safety design expertise responsibilities decisions

To manufacturing Spatial coordination Technical design and Construction (Stage 3) (Stage 4) (Stage 5) • Testing of the fire safety design through engineering analysis • Technical design with final • Cross-check with Building regulations specifications related to fire safety • Integrate fire safety measures into a • Review insurer requirements spatially coordinated design • Review building use (management • Record deviation from the fire safety and maintenance requirements) strategy • Nominate fire protection contractors

Figure 4-5. Stages of building design and process management (stages 5 through 7 are omitted here). Based on [47]

4.6 Classification Resources are finite. This means that resource allocation is a key issue of any major project such as designing an MHF or a building. From the risk management perspective, a unique safety case for each MHF would overrun the capacity of the AHJ of any country. This problem was solved for MHF through the classification of the facilities based on their chemical inventory and the creation of tiers with a different degree of responsibilities. The safety case is reserved for facilities surpassing the upper thresholds, i.e. those presumed to have the most important potential for major accidents, while for other facilities a reduced set of requirements are established without changing the overall objective: preventing major accidents and minimizing their consequences, even if the inventory is below the threshold for safety cases. Regardless of the classification of the facility, all chemical processes designed by chemical or process engineers are required to follow the same design process discussed in section 4.5 and there is no streamlined set of rules that allow bypassing it.

Given the larger amount of buildings constructed compared to chemical facilities, a case-by- case analysis seems unlikely due to a lack of resources (both at the design and regulator sides)

106 Chapter 4. Comparing contexts: MHF and FSE and calls for standardized methods, tools and data that enable a verification of safety, rather than an explicit demonstration of the building’s safety as in MHF. In FSE such classification has been implemented to a certain degree, not using the hazards directly (e.g. fuel load and distribution) but through proxy quantities such as total height, type of occupation and proximity to other infrastructure, e.g. see High Risk Residential Buildings (HRRB) classification proposed by Hackitt based on the number of stories [27].

The HRRB classification is not directly related to the use of performance-based design, but to the applicable construction code requisites to be complied with. This creates confusion regarding the methods and tools to use to justify a departure from the prescriptive rules, although typically it is addressed through a comparative approach. Bjelland presents the typical options in the context of the construction code of Norway and shows that there is a recognition of comparative and absolute methods to demonstrate adequate performance [31].

Other relevant classifications in the context of FSE, are those made by construction codes. The classification of the NCC of Australia [24] is divided in to dimensions, the first one being the ‘class’ (or type of occupancy) and the second one the type of construction (Table 4-). The former can be associated to the functionality of the building, its use and a general description of its intended occupants. These could be interpreted as to provide some indirect connection to the fire likelihood, but this variable if influenced by factors well beyond those captured by the classes. The classes seem to provide some insight on the level of consequences or vulnerability of the occupants, but again, these can also be characterized using a large range of variables and not just the occupancy. The type of construction is a general classification dependent on both the number of storeys and the building class. As storeys increase, it might be reasonable to expect a larger consequence, but again there can be exceptions to this and consequences cannot only be defined by such a narrow set of descriptors.

Table 4-1. Classification for the type of construction [24] Class of building Class of building Rise in storeys 2, 3, 9 5, 6, 7, 8 4 or more A A 3 A B 2 B C 1 C C

The Basis of Structural Design of the Eurocode [51] proposes a classifications that helps selecting a correspondent reliability target. These targets were discussed as one type of

107

Chapter 4. Comparing contexts: MHF and FSE acceptance criteria in Section 3.5. The Eurocode proposes three consequence classes, for which qualitative descriptors and examples are put forward, ultimately making it necessary for the practitioner to select the applicable class (Table 4-2).

Table 4-2. Classification for consequence class [51] Consequence Summary of description class CC3 “High” human loss or “very great” economic, social or environmental consequences CC2 “Medium” human loss or “considerable” economic, social or environmental consequences CC1 “Low” human loss or “small or negligible” economic, social or environmental consequences the ISO standard for the General principles on reliability for structures [52] (and its corresponding standard for the assessment of existing structures [53]) propose a similar classification to the one from the Eurocode, with two additional consequence classes. The definition of each class is slightly more detailed, including number of expected fatalities.

Table 4-3. Classification for consequence class [52] Consequence class Summary of consequences 1 Insignificant 2 Some (undefined) damage and functionality loss with “little or no societal impact” 3 Significant damage and functionality loss. Less than 50 fatalities 4 “Disastrous event” with severe damage and loss. Less than 500 fatalities 5 “Catastrophic event”. More than 500 fatalities The discussed classifications are relevant and useful for specific purposes. This is particularly clear in the case of reliability targets for structural design, which are formulated as a function of the potential consequences. The Eurocode – for example- is clear in stating that some of these targets exclude human factors and may not reflect actual failure rates [51].

None of these classifications capture all the necessary aspects to determine potential fire consequences nor likelihood. Identifying such aspects is not a general task, but one specific to each type of building, occupancy and socio-economical context. As discussed in Chapter 3 and in section 4.8, the fire consequences are typically unknown until a thorough risk assessment is done and can only be judged a priori with large uncertainties.

These classifications can be useful, but largely general and qualitative. This seems a strange basis to perform a detailed and objective risk assessment that informs decision-makers in a comprehensive manner. The classifications examples presented show a large degree of

108

Chapter 4. Comparing contexts: MHF and FSE disparity in the classification criteria and number of classes. Using these classifications as a basis for risk screening can prevent identifying relevant risks (even in low consequence class buildings), which then prevents these from being managed. Compared to the approach in MHF where an overarching objective is set an a single (simple) classification exists based on the magnitude of the (chemical) hazard, the classifications available in FSE are inadequate to support initial screening in performance-based design.

4.7 Scenario identification Typical hazard identification processes in MHF enable scenario discovery and their prioritization. Figure 4-6 presents a simplified representation of how process hazard analysis (PHA) help identifying an initial set of scenarios to be considered in the design stage, which are prioritized using further PHA techniques like failure modes and effects analysis (FMEA) and layers of protection analysis (LOPA). The use of fault and event trees is applied to about one fifth of the scenarios, yielding only 1% of the original scenario set to be analysed with quantitative risk assessment. This identification and narrow down of scenarios reflects the design process for chemical processes presented in section 4.5, in which hazards are treated at the beginning of the process and then those that remain are converted into scenarios than then are treated individually.

PHAs can be based on scenarios, but they can also be generic. Tixier et al. [54] present a review of 62 techniques used in the chemical process safety discipline to identify hazards and analyse risk, while the Centre for Chemical Process Safety [55, 56] presents the main techniques and the protocol for implementation. Typically, these methodologies require examining the facility’s engineering diagrams, as well as the inventories, and most often than not require an actual visit to the facility if it exists. This constitutes an initial stage of analysis, after which preparation of required templates takes place. The templates are key tools of the PHA process, as they record all the information related to the hazards and the associated scenarios. To carry out the PHA a facilitator runs a workshop with representatives of all relevant disciplines and areas of the plant. These sessions can span between a day and several days and require a scribe who assists the facilitator in condensing the key information into the templates. At the end of the sessions, scenarios are prioritized based on the hazards involved or in consequence or likelihood levels. Typically, consequences and likelihoods are judges qualitatively, but simplified modelling tools can be used during the session to dimension their order of magnitude. The prioritized scenarios can then be further investigated due to their consequence potential or relative likelihood.

109

Chapter 4. Comparing contexts: MHF and FSE

The PHA process yields a comprehensive understanding of the hazards involved in the processing of chemical substances and a set of scenarios that can be treated through different safety measures. Evidently, this set of scenarios is incomplete and both the characteristics of the scenarios and of the effectiveness of the safety measures include aleatory and epistemic uncertainty. At this point, the treatment of major accident scenarios by applying QRAs or PRAs supports evaluating design and safety measure alternatives, as well as comparing against individual and societal risk benchmarks. Once the facility is operational, the technical memoirs for the PHAs are used as the basis for the periodical revalidation, in which new scenarios are identified based on updated operational information such as modifications to the process.

Qualitative Simplified quantiative analysis Quantative analysis analysis 10-20% scenarios 1% scenarios 100% scenarios

Applicability of PHAs, indexing Event tree Event tree, PHAs LOPA techniques techniques quantification Fault tree, HRA

Simple issues Good Good Good Excessive Excessive

Complex Poor Poor Fair Fair Good issues

Figure 4-6. Relation between techniques, their applicability and the proportion of scenarios they cover. Adapted from [57]. HRA: Human Reliability Analysis

In FSE the role of hazards identification is reflected by the key guidance for risk assessments –bear in mind that hazards identification is supposed to be the first step of this process as established in ISO 31000:2009. In the UK the BS 7974:2019 [23] there is a requirement for a comprehensive hazards identification at the qualitative design review (QDR) stage, however it fails to state how this process is carried out, by whom, to which end and more importantly what the use of this information is. As discussed in section 3.6, this is seldom achieved. The SFPE Handbook of Fire Protection Engineering [58] deals with fire hazards in its first volume (Chapter 1-36), while the second volume deals with a mix of performance-based design, safety measures design and fire scenarios (Chapter 37-62). The final volume of the handbook deals with unusual scenarios such as vapour cloud explosions, dust explosions and a large set of chapters devoted to fire risk analysis, the tools and the data required for it (Chapter 63-90). From these 90 chapters only in chapter 38 devoted to fire scenarios there is a mention to fire

110 Chapter 4. Comparing contexts: MHF and FSE

identification, which in turn is a reference from IST/TS 16733. The role of hazards identification in existing fire risk assessment guidance is further discussed in Chapter 5.

Evidently, the hazards between a chemical plant and a residential building are quite different. However, this does not imply that the risk assessment steps should be cut short due to the lack of presence of a toxic substance or the potential for an explosion (which can in fact be plausible scenarios in a building; The Ronan Point event is an example of how this seemingly unlikely scenarios can lead to major consequences and trigger significant changes in engineering practice [59]). The potential damage of a fire is such that a comprehensive hazards identification process can help designers and fire safety engineers to design out unnecessary hazards and have a clear scope of scenarios on which to build a risk assessment. This is consistent with the claim by Clancy [50] who points out the lack of a structured hazard identification process. An example is lightweight-timber frame construction sites. The role of timber in a compartment fire is a current topic of research [60-63] given its economic, logistic and environmental benefits and its inherent combustible nature. However, the multiple fires reported in 2017 in Massachusetts, USA [64] exemplify the lack of a formalized hazard identification process at design and construction planning stages that have led to typically unacceptable fire loads to be accumulated. Another example is found in the refurbishment of historical buildings, which usually lead to the upgrade of outdated or inexistent fire safety measures. The Notre Dame fire in Paris, France exemplified how the lack of hazard awareness by engineers deciding on the fire protection measures of the building led to a detection and alarm system that did not correspond with the complexity of the structure, the need for reliable detection and of compartmentation barriers that would contain fire spread [65].

Similar issues were associated to the fire in the architecture faculty building of TU Delft, with Engelhardt et al. [66] pointing out the lack of a system-level fire structural analysis as a contributor to the resulting partial collapse. Identifying the hazard posed by large fire loads combined with the lack of vertical and horizontal compartmentation indicate can lead to the analysis of multi-level fire scenarios and the mentioned system-level fire structural analysis. Park et al. [48] notes that architects (leading the design) typically do not have an understanding of the effect of their decisions on fire safety. Although this is in line with the creative nature of the profession of architects, client requirements, functional requirements, the surroundings and the laws of nature bound the design. Another design bound should be fire hazards, which cannot be identified unless a structured identification process is established and led by competent practitioners at the earliest design stages.

111

Chapter 4. Comparing contexts: MHF and FSE

Several FSE guidance points towards the concept of scenario clustering and the selection of representative scenarios [67-69]. The SFPE guidance on risk assessment suggests using scenario clusters that represent ‘families’ of scenarios with similar characteristics, and using the consequences of a ‘representative’ scenario as the average consequences of the whole cluster. At the same time, the guidance states that “very low frequency scenarios” are very sensitive to even slight uncertainty in the frequency estimation. This is reasonable as fires are extreme events by definition and the values for the variables that help define them lay in the tails of the distributions (see section 3.4.2). The concept of scenario clustering (a highly subjective exercise for which little guidance is provided in available literature) contradicts the facts regarding the sensitivity of scenarios, both to consequences and to likelihood estimations. In conclusion, this concept obscures the need for a structured and systematic approach to scenario discovery or development.

4.8 Scenario use The previous section summarizes how scenarios are identified in both FSE and other disciplines. Disciplines like nuclear and chemical process safety have significantly systematic techniques to identify scenarios. This is a product of high levels of monitoring and recording over many decades, where detailed knowledge of failure modes has been produced for a wide range of consequences, including catastrophic ones. Through numerous lessons learnt and increasing monitoring, these disciplines have constructed a robust knowledge base for scenarios. This base is cannot ever be completed, but can been refined significantly. This is embodied by the statement of Mr Justice Haddon-Cave: “there are no new accidents, just lessons to be learnt from the ones we have already had” [70]. From this robust knowledge base, a reliable list of scenarios can be drawn to assess the safety performance of the system and support decision-making. An example are the boiling liquid evaporating vapour explosion (BLEVE), which tend to be extremely destructive and which result from uncontrolled pressurization of gas recipients3. Despite its large consequences, the conditions leading to these results were known since at least the 1960s [71] and now these can be designed out through the use of inherently safer design [72]. Instead of unknown failure modes, organizational and human factors are rather the focus or recent events such as in the case of the Fukushima nuclear disaster [73] or the Airbus MAX8 [74]. This is an inductive procedure, where the basis

3 Those containing combustible and flammable substances can generate both a mechanical and a chemical explosion, the former causing a destructive overpressure wave and the latter a large deflagration known as fireball.

112

Chapter 4. Comparing contexts: MHF and FSE

knowledge is updated and enhanced with each accident (see Figure 4-7). The inductive approach is applicable as the design of chemical processes, nuclear facilities and other industrial systems tends to be highly controlled, standardized and monitored (see section 4.11). When innovations are introduced, this knowledge base can fail to capture the key hazards and scenarios of the system. This situation can be addressed by considering the unique features of the system and elicit scenarios from them [75].

Figure 4-7. Process for the knowledge base update

Now consider the level of innovation and variability in the construction sector. Recently, the quality of the construction industry in Australia has come into the spotlight due to some significant deterioration of high-rise residential buildings. This is partially captured by the Shergold-Weir enquiry [28] as well as by the research of Love and Matthews [76]. The Hackitt enquiry [27] extend these findings to the UK construction industry and also highlights issues associated to FSE. This variability in construction quality drives the final product (i.e. the building) away from the intended design, which can lead to unexpected failure modes and factors previously considered acceptable. An example of this is the role of facades containing combustible materials in large human and economic loss [77]. Collectively, this points towards the much more fragile knowledge base in FSE to define scenarios. The high level of innovation in the construction industry can introduce both unexpected variability and unknown failure modes that cannot be captured by generic scenarios.

The previous poses a large challenge in FSE, as previously known scenarios cannot be reliably used as benchmarks for fire safety performance (e.g. design fires from the verification methods [78]). This list of scenarios can be useful as a general reference and a starting point, but any fire risk assessment will need to consider the unique characteristics of the designs. These unique characteristics may give rise to unique conditions that require careful consideration, and

113

Chapter 4. Comparing contexts: MHF and FSE

which could lead to unacceptable risks. Therefore, the scenarios necessary to understand the building performance stem from its unique characteristics. Any pre-defined or ‘typical’ scenario cannot capture the effects of these unique characteristics on the performance. This shows a break in the process described in Figure 4-7, just as in the case of other disciplines when facing unique or innovative systems. Ultimately, this open loop makes prescribed and ‘typical’ scenarios a poor choice for demonstrating an adequate fire safety performance.

4.9 Uncertainty description The larger the set of hazards (and therefore the deriving scenarios), the more challenging it is to explicitly consider them in a risk assessment. An example in MHF are the threats of mechanical integrity for process equipment that take place under different temperature, pressure and chemical concentrations inside the process equipment. Although mechanical and metallurgic engineers drive the process of managing mechanical integrity, the threats stem from the process conditions. Understanding the process and its conditions is necessary to screen the damage mechanisms reported in API 571 [79]. However, mechanical integrity seldom considers the synergy between two or more damage mechanisms and typical calculations for inspection and maintenance frequencies cannot reflect it. This is one of the two main types of uncertainty: incomplete or imperfect knowledge, i.e. epistemic uncertainty. The estimation of an inspection frequency based on a corrosion rate, the parameters affecting it can lead to variability that can only be reduced by increasing the sample size and the precision of the measures. This is the second type of uncertainty: natural (random) variability, i.e. aleatory uncertainty.

One could argue that most risk assessments are tools to represent how little or how much we know about a certain event. The result of some risk assessments could point out to the need for additional knowledge in order to reduce uncertainty. However, the willingness to acquire more knowledge (either through fundamental research or by increased sampling of a ‘random’ phenomenon) depends on the benefits it would bring about.

The need to better characterize uncertainty as part of the decision-making process was exemplified by the review of the first PRA made for commercial nuclear power plants back in 1975 (WASH-1400 [80]). The Lewis Committee report of 1978 expressed the need for the epistemic uncertainties to be explicitly stated, instead of being mathematically processed along with sources of aleatory uncertainty [81]. This study not only led to the beginning of a new era in risk assessments, but posed the question of what is the best way to represent uncertainties.

114

Chapter 4. Comparing contexts: MHF and FSE

Safety science research discusses the complexities of the quantification process [82] including uncertainty in the quantification outputs [83] and the variability of acceptance criteria [84]. The limitations of typical QRAs/PRAs have led this field to explore the use of non-probabilistic tools to analyse uncertainty [85]. These tools allow to process qualitative information and to treat probabilities that do not fit into the axioms of the classic probability theory (the application of these alternatives to FSE is described in Appendix 2). Despite the existence of diverse tools to describe, quantify and communicate uncertainty, all these remain a challenge for MHF and for engineering systems posing potential large consequences on the environment and the population [86].

The complexity of the system and the hazards involved determine the complexity of the risk assessment approach to be selected. The outcomes of the selected approach should be consistent with how they will be used. No available approach is perfect and even a fully quantitative or probabilistic approach includes epistemic uncertainty that might be relevant for decision makers. Furthermore, the results of the risk assessment are not to be the only decision- making input, and should incorporate other important ones like a peer (third party) review on the assessment. A level of analysis that does not fit the level of complexity of a system can produce misleading results through oversimplifications or omissions of key features influencing risk (e.g. failure modes, dependencies of variables). An example of such a potential inconsistency is quantifying the performance of a building’s fire strategy through PRAs. Such is the complexity of a fire safety strategy that it can only be captured through significant simplifications or if divided into components. Although a holistic approach is possible, its peer review could easily result unviable.

Lundin [87] lists the uncertainties typically found in a performance-based design in FSE and his argument aligns with that of Paté-Cornell regarding deterministic approaches. He states that a deterministic approach will not provide the uncertainty description required to properly inform a decision-maker. This is debatable, as professional engineers have the competencies to judge the values selected for individual variables and therefore can quantify the level of conservatism through the selected quantile for a particular variable; the issue of epistemic uncertainty is not so easily solved and the methodology proposed in this work provides an alternative (see MAD section). Although extensive guidance [10, 11] and applications [88-92] of PRA are found in FSE, there is not yet a clear understanding of the impact of epistemic uncertainties in the decisions of fire safety professionals in these applications. A discussion is provided on the guidance for uncertainty treatment in fire PRAs in Chapter 5.

115

Chapter 4. Comparing contexts: MHF and FSE

4.10 Acceptance criteria MHF regulations acknowledge risk assessment as the ideal tool to understand risk and therefore construct a risk treatment plan. Overarching regulation like Seveso III in the EU set as a responsibility of the operator to choose the methods, tools and data sources that they consider most convenient to support their safety case, while specific applications of Seveso in countries like the Netherlands use predefined calculation methods like Bevi for specific risk calculations. In the case of the US, both PSM and RMP programs specify the need to use hazard identification techniques recognized in the industry, i.e. process hazard analysis (PHAs), but do not specify the risk assessment methods to be employed. Although specific types of risk assessments are not typically specified, operating permits would not be issued without a supporting risk assessment; this is exemplified by the safety case contents, review and prohibition of operation within the COMAH regulations in the UK [19]. Specific countries can decide to use individual and societal risk acceptance thresholds to assess the risks posed by MHFs, but it is not the only possible pathway. While Seveso III in the EU establishes a comprehensive framework for managing risks in MHF, there is not an explicit risk criterion or preference. COMAH in the UK specifically acknowledges individual and societal risk measures as a mean to demonstrate acceptable risks. However, the decision on the operating license is based on the analysis of the whole safety case and the measures in place taken to prevent major accidents and to minimize their consequences. This not only includes the risk quantification, but the appropriateness of the method selected, the robustness of the tools employed, the validity of the data and the overall resulting uncertainty in the results, as well as its treatment through additional safety measures or an adjusted design.

Although fire engineering guidelines recommend basing the fire safety strategy and decisions on risk assessments [93], fire requirements in regulation might not require it as long as performance is demonstrated. This demonstration can be achieved through compliance with prescriptive rules, through comparison with reference designs or by an explicit quantification of performance. Lundin [87] notes how codes do not provide explicit performance criteria, as the analytical approach required in a performance-based design could result in different outcomes depending on the professionals involved. The previous yields two major differences between MHF and FSE. First, in MHF the acceptance criteria (be it individual or societal) captures the risk aversion of a particular society and is used to evaluate the major accident scenarios discussed in section 4.7. If a facility surpasses such criteria, the design of the operation can be adjusted in order to obtain a risk management strategy that supports the safety

116

Chapter 4. Comparing contexts: MHF and FSE

case and the license to operate. In this case, safety is not being verified, is being demonstrated. In the case of FSE there is an inclination for consolidating calculation packages into verification methods, which has dangerous outcomes as the understanding of risk takes a compliance nature, rather than one of supporting a particular fire safety strategy to achieve the goals. Bjelland [94] refers to this as a mechanization of the methods and discusses the potential pitfalls of this approach, having unsafe conditions being ignored (and therefore unmanaged) as one of the most disturbing outcomes. Second, in MHF the overarching objective discussed in section 4.2 is not achieved by the implementation of any particular method, as the complexity of the systems and the potential consequences cannot be managed by simple compliance. The implementation of quantitative risk criteria in FSE could be a positive aspect, driving fire safety engineers to support their designs with risk calculations that help modifying such criteria over time and to explicitly benchmark designs. However, the issues associated to the complex uncertainties of the fire dynamics, human behaviour and the response of the system under fire conditions cannot be overlooked by applying any one particular methodology or acceptance criterion.

4.11 Risk as a function of time Having safety management systems, methods and tools in MHFs to manage risk, undesired events continue to occur and improvement opportunities exist [95]. Aging is a critical issue for MHFs and for chemical process plants in general as equipment subject to the previously mentioned mechanical integrity threats deteriorates, increasing the likelihood of many scenarios, primarily loss of containment [96, 97]. Managing mechanical integrity typically relies on risk based inspection methodologies, in which risk indices are estimated for all equipment in order to prioritize resources and calibrate inspection frequencies. Recent research has pointed out the need for a dynamic approach to assessing these risks as operational changes in between periodical assessments could require shortening or reorienting inspection and maintenance resources [98, 99]. These are not stand-alone efforts as there is an underlying risk management framework.

Managing risk in MHF is led by active management (see section 4.4) and is implemented on a daily basis by the operation staff. Such efforts respond to the need to continuously manage risk and to comply with regulatory requirements such as inspection by AHJs, which has a significant cost for both the state and for the operator of the MHF. In fact, these costs can be debatable from a cost-benefit analysis depending on the assumptions made (e.g. value of avoiding life loss, effective men hours spent in safety) and on the focus of management: safety

117

Chapter 4. Comparing contexts: MHF and FSE

or productivity. Reniers [100] argues that a focus on productivity negatively impacts risk management and can lead to hazardous situations, while a focus on safety can lead to suboptimal economic situations. In light of the continuous incidents and near misses registered thanks to safety management systems required by MHF regulation, current safety expenditures are accepted and operators tend to find manners (including risk assessments) to optimize it without reaching unsafe operating conditions. A graph over time of safety-related expenditures can be one of many indicators that could point out inadequate safety levels, which could be correlated with operational events such as plant turnovers and major maintenance or changes in management. Understanding risk as a function of time, including associated costs, enables MHFs to identify effective risk management strategies based on their unique conditions. However, to achieve this there is a large reliance on measurements and data [101].

Efforts to continuously managing risk in MHFs rely significantly on failure data. Smith [102] divides failure dataset into site-specific, industry-specific and generic. His statistical analysis show that with a 95% confidence the eventual failure rate of the operation will be better than 3½, 5 and 8 times the estimated rate depending on the type of dataset. This represents a large uncertainty range and justifies the mentioned efforts to continuously monitor degradation and failure. Furthermore, this data is a snapshot in time and efforts for its update are resource intensive as demonstrated by one of the most recognized databases in the oil & gas and chemical process industries, OREDA [103]. This project spans 12 phases starting in 1983 after its foundation in Norway 1981 and is one of the main sources for reliability data for risk assessment purposes, including the mentioned risk based inspections. In MHF there are also other useful sources such as the Hydrocarbon Releases (HCR) System for offshore operations in the UK [104], the PDS data handbook [105] and the OGP Process release frequencies [106] to mention a few.

Nevertheless, Rausand [107] indicated the inherent limitations of OREDA due to the lack of explicit criteria to differentiate between failure modes when an entry is recorded, leading to subjective decisions being made by the person submitting the entry. The datasets can lead to identifying different failure modes as shown in a recent study on condition-based maintenance (CBM) for critical equipment [108] and Smith [109, 110] details how this data can lead to underestimations if the uncertainty sources are not properly managed or if the quality is poor. A large range of industrial facilities use the previously mentioned datasets, however these are constructed mainly with oil & gas industry failure data, which implies biases in the data as design and operating standards differ between this and other industries. The previous is in part

118

Chapter 4. Comparing contexts: MHF and FSE

explained by incident and fatalities rates in oil & gas operations and the scarcity of failure and incident databases for other sectors [111]. In sum, the availability of datasets is a powerful input for risk assessments but do not guarantee the estimation of ‘true’ nor accurate future likelihoods.

The differences with FSE at this point are important and touch on most of the previous sections of this chapter. For MHF we have described the effect time has in risk, the risk management efforts it imply and the associated costs, detailing the use and limitations of reliability data. In FSE the differences can be seen at all these levels, with an exception in the costs of safety. Reniers [100] discusses aversion in operational safety, pointing out that managers might feel unease by doing certain investments (an expenditure that can be seen as a loss) while obtaining in return uncertain safety levels. This aversion to safety expenditures in FSE is exemplified (to an extreme) by the lack of sound professional judgment behind the use of combustible materials in façade systems, as knowledge required from large-testing materials in several configurations represents large costs [112] and if frequently not available. Situations like this arise in the context of performance-based codes, which ironically are meant to drive design towards rational risk management and therefore economic benefits as compared to those fixed expenditures based on prescriptive codes. This can be explained by Reniers’ argument of cost aversion and the misuse of the performance-based framework by the risk owners [113].

Focusing on the differences, one must consider the influence of risk ownership. In the combustible façade materials example, one can compare the actions of a publicly owned facility of high priority such as the Princess Alexandra Hospital and the already mentioned hundreds of private high-rise residential buildings in Australia. The hospital got its non-compliant façade removed in 2017, just a few months after the Grenfell tower fire and to a cost of about ten million Australian dollars [114]. In the meantime, private owners are still going through a survey process to identify whether their building requires rectification and it is expected that this process end in 2021. The important difference between these two cases is that the state can act immediately to protect society from a hazard that they themselves did not prevent from being introduced to critical infrastructure, while in the latter the risk lays with the private owners. These might lack the resources, coordination and particularly the technical expertise to properly identify and deal with this hazard. These two situations exemplify a critical difference with MHF, the lack of an underlying continuous risk management process.

119

Chapter 4. Comparing contexts: MHF and FSE

A second important difference are the data issues. These issues have been discussed in the context of probabilistic risk assessment [Reference to Chapter – PRA], but here the focus is on the difference of issues between the data recorded in MHF and the one recorded in FSE. It was already established that such datasets are far from ideal and that a challenge of using them relies on incomplete, quality varying and context dependent data. In FSE the situation is not better, as available data for use in risk assessments is based on a mixture of laboratory, bench or large scale testing data, as well as post-fire building surveys [115-117], NFIRS [118], PD 7974-1 [10], SFPE Handbook – Engineering data [118] and Reliability [119]. These two differences point out the effects of clear risk ownership on the ability to monitor the deterioration of the building. Introduction of new hazards and recording of maintenance data are not visible through current provisions, which diverge from the management of risk over time in MHF. In particular, the lack of trustworthy or applicable reliability data affects the likelihood estimations in risk assessments and can hinder decision-making. An example of this lack of risk over time is the unforeseen excess occupancy in the compartment of fire origin in the Lacrosse fire in 2014 [120].

4.12 Conclusions The drive towards a probabilistic approach to demonstrate adequate fire safety performance is grounded on the base of accepting that certain consequences are feasible, but highly unlikely. It is a fact that society accepts loss, but it does not react the same to all loss. A single fatality in a residential building fire might not lead to massive social outcry and demand for change, unless special conditions are involved like criminal negligence. In the field of MHF, most major accidents do result in significant loss and a corresponding demand for accountability and changes that prevent recurrence. This is expected, as per definition, a major accident involves significant loss and at least one human fatality. In FSE though, it is not straightforward to identify the threshold after which society deems a loss unacceptable. Section 3.4.2 discussed the revealed and expressed approaches to risk perception, showing that despite being long standing theories, they are far from conclusive and heavily reliant on data. It is in the view of the author that the criteria itself is not what matters most, but the use of legal frameworks and risk assessment to inform the decision-making process at the right stage. This is a major divergence between the use of risk assessments in MHF and in FSE.

The comparison shows that the nature of MHF and FSE are significantly distinct, in particular those related to design philosophy, which affects the ability to identify and design out hazards, the lack of provisions to monitor, assess and manage risk over time and the lack of clearly

120 Chapter 4. Comparing contexts: MHF and FSE

defined risk ownership. These differences have the potential to impede or difficult the adoption of recognized practices from MHF into FSE. One of these practices is the use of probabilistic thresholds for risk acceptance, which is being considered for implementation within building codes such as the Australian [4]. The previous helps answer research question 1, in that PRAs should not be considered –by default- an adequate method for explicitly assessing fire safety.

The implications of adapting a quantitative risk approach to fire safety are plenty but here the focus is set on one, for which three considerations are necessary. First, the trend in FSE to mechanize methodologies in order to verify safety and obtain approval is described by Gehandler [29], which diverges from the objective of demonstrating safety. Second, there is an expected expert asymmetry between the AHJ and the FSE practitioners in terms of technical knowledge related to fire dynamics and fire performance of a building. Spinardi [113] discusses this expert asymmetry, which was an important challenge in MHF until robust inspection schemes were established and led by expert practitioners (often coming from decades of experience in the MHF operation). Third, implementing and carrying out resource intensive activities like probabilistic risk assessments can provide stakeholders with a sense of safety being achieved, despite not having any real impact on the building design, its hazards, risks or risk treatment strategy. This is what Rae [121] calls ‘probative blindness’ and although it was originally identified in the MHF context, examples such as the one by Bjelland [122] show its existence in FSE. Adopting methodologies from a different context, the potential to mechanize them and to use them with intentions that differ from the construction code intent pose an important danger to the future of performance-based design and of FSE itself.

In conclusion, the demonstration of acceptable performance is the aim of fire risk assessments and directly adopting existing methodologies from different contexts is not enough to achieve it. Based on the identified differences and the considerations presented, the hypothesis discussed at the beginning of this chapter is deemed invalid. An important reflection for FSE bodies pushing for this approach is to consider if risk management in MHF would be a failure if not for safety cases or probabilistic risk assessments. Based on the evidence presented, hazard identification is the pillar of the measured success in MHF, which is a continuous challenge rather than a goal to achieve. Therefore, FSE must tailor solutions that depart from mechanistic approaches and predefined requirements, as these cannot ensure safety. Although decades of knowledge, tools, methods and frameworks are useful, adopting (without adapting) them will carry underlying issues that may result in irreparable loss. Such loss is harshly judged by society!

121 Chapter 4. Comparing contexts: MHF and FSE

Bibliography 1. Aven, T., What is safety science? Safety Science, 2014. 67: p. 15-20. 2. Meacham, B., The evolution of performance-based codes and fire safty design methods, NIST-GCR-98-761. Nat. Inst. Stand. Technol., USA, 1998. 3. Borg, A. and O. Njå, The concept of validation in performance-based fire safety engineering. Safety Science, 2013. 52: p. 57-64. 4. Meacham, B., Ultimate Health & Safety (UHS) Quantification: Individual and Societal Risk Quantification for Use in National Construction Code (NCC). 2016, The Australian Building Codes Board. 5. Wolski, A., N.A. Dembsey, and B.J. Meacham, Accommodating perceptions of risk in performance-based building fire safety code development. Fire Safety Journal, 2000. 34(3): p. 297-309. 6. Kaplan, S. and B.J. Garrick, On The Quantitative Definition of Risk. Risk Analysis, 1981. 1(1): p. 11-27. 7. Paté-Cornell, M.E., Uncertainties in risk analysis: Six levels of treatment. Reliability Engineering & System Safety, 1996. 54(2): p. 95-111. 8. Van Coile, R., et al., The Need for Hierarchies of Acceptance Criteria for Probabilistic Risk Assessments in Fire Engineering. Fire Technology, 2018. 9. Van Coile, R., et al., An Unbiased Method for Probabilistic Fire Safety Engineering, Requiring a Limited Number of Model Evaluations. Fire Technology, 2017. 53(5): p. 1705-1744. 10. (BSI), B.S.I., PD 7974-7:2019 Application of fire safety engineering principles to the design of buildings. Probabilistic risk assessment. 2019, London, UK. 11. Norway, S., SN-INSTA/TR 951:2019 - Fire Safety Engineering - Guide for Probabilistic Analysis for Verifying Fire Safety Design in Buildings. 2019. 12. Goerlandt, F., N. Khakzad, and G. Reniers, Validity and validation of safety-related quantitative risk analysis: A review. Safety Science, 2016. 13. Baybutt, P., On the completeness of scenario identification in process hazard analysis (PHA). Journal of Loss Prevention in the Process Industries, 2018. 14. Benintendi, R., Chapter 12 - Quantitative Risk Assessment, in Process Safety Calculations. 2018, Elsevier. p. 607-628. 15. The Major Accident Reporting System (eMARS), E. Commission, Editor. 2020. 16. Offshore Hydrocarbon Releases 1992-2019, H.a.S. Executive, Editor. 2020: United Kingdom.

122 Chapter 4. Comparing contexts: MHF and FSE

17. Australia, S.W., Model Work Health and Safety Regulations, in Chapter 9, P.C.s. Committee, Editor. 2019: Camberra, Australia. p. 51. 18. Commission, E., Directive 2012/18/EU of the Eruopean Parliament and of the Council of 4 July 2012 on the control of major-accident hazards involving dangerous substances, amending and subsequently repealing Council Directive 96/82/EC 2012, Official Journal of the European Union: Strasbourg. 19. The Control of Major Accident Hazards Regulations (COMAH), in 483, P.o.t.U. Kingdom, Editor. 2015: London, United Kingdom. 20. (OSHA), O.S.a.H.A., Process safety management of highly hazardous chemicals, in Occupational Safety and Health Standards, U.S.D.o. Labor, Editor. 2013. 21. (EPA), E.P.A., Subpart G - Risk Management Plan Source, in 40. 1996. 22. Notarianni, K.A. and G.W. Parry, SFPE Handbook of Fire Protection Engineering, Fifth Edition. 2016: Springer New York. 2992-3047. 23. (BSI), B.S.I., BS 7974:2019, Application of fire safety engineering principles to the design of buildings - Code of practice. 2019: BSI. 24. Board, T.A.B.C., National Construction Code 2019, I.a.S. Department of Industry, Editor. 2019, The Australian Building Codes Board: Australia. 25. (CCPS), C.f.C.P.S., Guidelines for consequence analysis of chemical releases. 1999: American Institute of Chemical Engineers. 26. Kletz, T.A. and P. Amyotte, Process plants : a handbook for inherently safer design. 2nd ed. ed, ed. ProQuest. 2010, Boca Raton: CRC/Taylor & Francis. 27. Hackitt, J., Building a Safer Future, Independent Review of Building Regulations and Fire Safety: Final Report. 2018: London, UK. 28. Peter Shergold, B.W., Building Confidence – Improving the effectiveness of compliance and enforcement systems for the building and construction industry across Australia. 2018, Building Ministers’ Forum (BMF). 29. Gehandler, J., The theoretical framework of fire safety design: Reflections and alternatives. Fire Safety Journal, 2017. 30. D. Lange, J.T., A. Osorio, N. Lobel, C. Maluk, J. Hidalgo, Fire Safety Engineering - The Methods Report, T.W. Centre, Editor. 2019: Sydney, Australia. 31. Bjelland, H. and A. Borg, On the use of scenario analysis in combination with prescriptive fire safety design requirements. Environment Systems & Decisions, 2013. 33(1): p. 33-42.

123

Chapter 4. Comparing contexts: MHF and FSE

32. Spinardi, G. and A. Law, Beyond the stable door: Hackitt and the future of fire safety regulation in the UK. Fire Safety Journal, 2019. 109: p. 102856. 33. Australia, S.W., Guide for major hazard facilities - Developing a safety case outline. 2012. 34. D. Lange, J.T., C. Maluk, J. Hidalgo, Fire Safety Engineering - Competencies Report, T.W. Centre, Editor. 2019: Sydney, Australia. 35. Ronchi, E., Developing and validating evacuation models for fire safety engineering. Fire Safety Journal, 2020: p. 103020. 36. Standardization, I.O.f., ISO 31000:2018: Risk management - Principles and guidelines. 2018: Geneva, Switzerland. 37. Celeste Young, R.J., Margarita Kumnick, Greg Christopher, Nicholas Casey, Risk ownership framework for emergency management policy and practice. 2017, Bushfire and Natural Hazards CRC: Melbourne, Australia. 38. Invetigation Report Executive Summary - Drilling Rig, Explosion and Fire at the Macondo Well. 2016, U.S. Chemical Safety and Hazard Investigation Board. 39. (OECD), O.f.E.C.-o.a.D., Guidance on Change of Ownership in Hazardous Facilities. 2018, Environment, Health and Safety, Environment Directorate: Paris, France. 40. Expenditure on Gross Domestic Product (GDP), A.B.o. Statistics, Editor. 2020. 41. Hanmer, G., Lacrosse fire ruling sends shudders through building industry consultants and governments, in The Conversation. 2019: Sydney, Australia. 42. Seider, W.D., Product and Process Design Principles. 4th ed.. ed. 2016. 43. Tamim, N., et al., Roles of contractors in process safety. Journal of Loss Prevention in the Process Industries, 2017. 48: p. 358-366. 44. Crowl, D.A. and J.F. Louvar, Chemical process safety : fundamentals with applications. 3rd ed. ed. 2011, Upper Saddle River, NJ: Prentice Hall. 45. (CCPS), C.f.C.P.S., Guidelines for Engineering Design for Process Safety, Second Edition. Second Edition ed. 2012. 46. Seider, W.D., et al., Design for Process Safety – A Perspective, in Computer Aided Chemical Engineering, M.R. Eden, J.D. Siirola, and G.P. Towler, Editors. 2014, Elsevier. p. 795-800. 47. Architects, R.I.o.B., RIBA Plan of Work. 2020, Royal Institute of British Architects: London, UK. 48. Park, H., et al., Enhancing Building Fire Safety Performance by Reducing Miscommunication and Misconceptions. Fire Technology, 2014. 50(2): p. 183-203.

124 Chapter 4. Comparing contexts: MHF and FSE

49. Gary Skinner, Y.H., Fire Saefty Engineering and Arhcitects, in FSE11: Fire Safety Engineering International Conference - Raising the Bar. 2011. 50. Clancy, P., Advancing the implementation of risk assessment, in FSE11: Fire Safety Engineering International Conference - Raising the Bar. 2011. 51. Eurocode, B., EN 1990: 2002-Basis of Structural Design. British Standard Institution, 2002. 52. Organization, I.S., ISO 2394: General Principles on Reliability for Structures. 2014, ISO Geneva. 53. Standardization, I.O.f., ISO 13822: 2010: Bases for design of structures–Assessment of existing structures. 2010, International Organization for Standardization. International Standard …. 54. Tixier, J., et al., Review of 62 risk analysis methodologies of industrial plants. Journal of Loss Prevention in the Process Industries, 2002. 15(4): p. 291-303. 55. (CCPS), C.f.C.P.S., Guidelines for Hazard Evaluation Procedures. 2010, Hoboken: American Institute of Chemical Engineers. 56. (CCPS), C.f.C.P.S., Scenario-Based Hazard Evaluation Procedures. Guidelines for Hazard Evaluation Procedures, 2008: p. 99-174. 57. (CCPS), C.F.C.P.S., Layer of protection analysis: simplified process risk assessment, ed. D.A. Crowl. 2001, New York: American Institute of Chemical Engineers. 58. Hurley, M.J. and M.J. Hurley, SFPE handbook of fire protection engineering / Morgan J. Hurley, editor-in-chief. 5th ed.. ed. 2016, New York: New York : Springer. 59. 1968: Three die as tower block collapses. BBC News, 1968. 60. Joseph Su, P.-S.L., Matthew Hoehler, Matthew Bundy, Fire Safety Challenges of Tall Wood Buildings – Phase 2: Task 2 & 3 – Cross Laminated Timber Compartment Fire Tests. 2018, National Research Council of Canada: Ottawa, Ontario, Canada. 61. Östman, B., D. Brandon, and H. Frantzich, Fire safety engineering in timber buildings. Fire Safety Journal, 2017. 91: p. 11-20. 62. Gorska, C., J.P. Hidalgo, and J.L. Torero, Fire dynamics in mass timber compartments. Fire Safety Journal, 2020: p. 103098. 63. Emberley, R., et al., Description of small and large-scale cross laminated timber fire tests. Fire Safety Journal, 2017. 91: p. 327-335. 64. Douglas Moser, K.H., Building Risk: Large Wood Structures Present Challenges for Firefighters, in NBC Boston. 2017: Boston, USA.

125

Chapter 4. Comparing contexts: MHF and FSE

65. Katrin Bennhold, J.G., Notre-Dame’s Safety Planners Underestimated the Risk, With Devastating Results, in The New York Times. 2019: New York, USA. 66. Engelhardt, M., et al., Observations from the Fire and Collapse of the Faculty of Architecture Building, Delft University of Technology. 2013. 1138-1149. 67. (SFPE), S.o.F.P.E., SFPE engineering guide to application of risk assessment in fire protection design. 2005. 68. Hall, J.R. and A. Sekizawa, Revisiting Our 1991 Paper on Fire Risk Assessment. Fire Technology, 2010. 46(4): p. 789-801. 69. (ISO), I.O.f.S., ISO 16733-1 Fire safety engineering - Selection of design fire scenarios and design fires. 2015. 70. (IChemE), I.o.C.E., Ten things we learned at Hazards 26. 2016: Online. 71. Hemmatian, B., et al., BLEVE: The case of water and a historical survey. Journal of Loss Prevention in the Process Industries, 2019. 57: p. 231-238. 72. Gómez, G., et al., Kletz's legacy for developing countries: Simple systems based on inherently safer design. Journal of Loss Prevention in the Process Industries, 2012. 25(5): p. 843-847. 73. Denning, R.S. and R.J. Budnitz, Impact of probabilistic risk assessment and severe accident research in reducing reactor risk. Progress in Nuclear Energy, 2018. 102: p. 90-102. 74. Herkert, J., J. Borenstein, and K. Miller, The Boeing 737 MAX: Lessons for Engineering Ethics. Science and Engineering Ethics, 2020. 75. Paltrinieri, N., A. Tugnoli, and V. Cozzani, Hazard identification for innovative LNG regasification technologies. Reliability Engineering & System Safety, 2015. 137: p. 18- 28. 76. Love, P.E.D. and J. Matthews, Quality, requisite imagination and resilience: Managing risk and uncertainty in construction. Reliability Engineering & System Safety, 2020. 204: p. 107172. 77. Torero, J., Grenfell tower: Phase 1 report. 2018, Torero, Abecassis Empis and Cowlard. 78. 2020, C.o.A.a.S.a.T., Handbook: Fire Safety Verification Method. 2020, Canberra, Australia: Australian Building Codes Board (ABCB). 79. (API), A.P.I., API 571: Damage Mechanisms Affecting Fixed Equipment In The Refining Industry. 2nd ed. 2011.

126 Chapter 4. Comparing contexts: MHF and FSE

80. Commission, U.S.N.R., Reactor Safety Study: An Assessment of Accident Risks in U.S. Commercial Nuclear Power Plants. 1975. 81. M. Paté-Cornell, P.F., Rationality and risk uncertainties in building code provisions, in Sixth Conference on Applied Statistical Problems in Civil Engineering (ICASP6). 1991: Mexico City, Mexico. 82. Pasman, H. and G. Reniers, Past, present and future of Quantitative Risk Assessment (QRA) and the incentive it obtained from Land-Use Planning (LUP). Journal of Loss Prevention in the Process Industries, 2014. 28: p. 2-9. 83. A. Rae, J.M., R. Alexander, The Science and Superstition of Quantitative Risk Assessment, in Proceedings of PSAM 11 & ESREL 2012. 2012, International Association of Probabilistic Safety Assessment and Management, IAPSAM. p. 2292- 2301. 84. Pitblado, R., et al., International comparison on the application of societal risk criteria. Process Safety Progress, 2012. 31(4): p. 363-368. 85. Abdo, H., J.M. Flaus, and F. Masse, Uncertainty quantification in risk assessment - Representation, propagation and treatment approaches: Application to atmospheric dispersion modeling. Journal of Loss Prevention in the Process Industries, 2017. 49: p. 551-571. 86. Årstad, I. and T. Aven, Managing major accident risk: Concerns about complacency and complexity in practice. Safety Science, 2017. 91: p. 114-121. 87. Lundin, J., Model Uncertainty in Fire Safety Engineering. 1999, Lund University. 88. Jovanović, B., et al., Review of Current Practice in Probabilistic Structural Fire Engineering: Permanent and Live Load Modelling. Fire Technology, 2020. 89. Sabapathy, P., A. Depetro, and K. Moinuddin, Probabilistic Risk Assessment of Life Safety for a Six-Storey Commercial Building with an Open Stair Interconnecting Four Storeys: A Case Study. Fire Technology, 2019. 90. Paudel, D. and S. Hostikka, Propagation of Model Uncertainty in the Stochastic Simulations of a Compartment Fire. Fire Technology, 2019. 91. Hopkin, C., et al., Design Fire Characteristics for Probabilistic Assessments of Dwellings in England. Fire Technology, 2019. 92. Magnusson, S.E., Probabilistic analysis of fire exposed steel structures. Bulletines of Division of Structural Mechanics and Concrete Construction. Vol. Bulletin 27. 1974: Lund Institute of Technology.

127 Chapter 4. Comparing contexts: MHF and FSE

93. Plank, R., Performance Based Fire Engineering in the UK. International Journal of High-Rise Buildings, 2013. 2(1). 94. Bjelland, H., et al., The Concepts of Safety Level and Safety Margin: Framework for Fire Safety Design of Novel Buildings. Fire Technology, 2015. 51(2): p. 409-441. 95. Lindhout, P. and G. Reniers, Risk validation by the regulator in Seveso companies: Assessing the unknown. Journal of Loss Prevention in the Process Industries, 2017. 49: p. 78-93. 96. Golob, S. and M. Kožuh, Methodology for determining the risk acceptance criteria for the Seveso establishments. Process Safety and Environmental Protection, 2016. 100: p. 163-172. 97. Bragatto, P. and M.F. Milazzo, Risk due to the Ageing of Equipment: Assessment and Management. Chemical Engineering Transactions, 2016. 53: p. 253-258. 98. Bhatia, K., et al., Dynamic risk-based inspection methodology. Journal of Loss Prevention in the Process Industries, 2019. 62: p. 103974. 99. Paltrinieri, N. and G. Reniers, Dynamic risk analysis for Seveso sites. Journal of Loss Prevention in the Process Industries, 2017. 49: p. 111-119. 100. Reniers, G.L.L. and H.R.N. Van Erp, Operational Safety Economics. 2016. 101. Wood Maureen, H., Analysing Accidents and Lessons Learned: You Can’t Improve What You Don’t Measure. Chemical Engineering Transactions, 2018. 67: p. 391-396. 102. Smith, D.J. and K.G.L. Simpson, Chapter 6 - Failure Rate and Mode Data, in The Safety Critical Systems Handbook (Fifth Edition), D.J. Smith and K.G.L. Simpson, Editors. 2020, Butterworth-Heinemann. p. 133-141. 103. Technology, S., et al., OREDA : offshore and onshore reliability data handbook. 2015. 104. HSE, Offshore Hydrocarbon Release Statistics 2001, in HID Statistics Report HSR. 2001, Health & Safety Executive. 105. Handbook, P.D., Reliability data for safety instrumented systems. PDS Data Handbook, SINTEF, 2010. 106. (OGP), I.A.o.O.a.G.P., Process Release Frequencies, in Risk Assessment Data Directory. 2019, Reuters England and Wales: London, UK. 107. Rausand, M. and K. Øien, The basic concepts of failure analysis. Reliability Engineering & System Safety, 1996. 53(1): p. 73-83. 108. Jun, H.B. and D. Kim, A Bayesian network-based approach for fault analysis. Expert Systems with Applications, 2017. 81: p. 332-348.

128 Chapter 4. Comparing contexts: MHF and FSE

109. Smith, D.J., Chapter 5 - Interpreting Data and Demonstrating Reliability, in Reliability, Maintainability and Risk (Ninth Edition), D.J. Smith, Editor. 2017, Butterworth- Heinemann. p. 59-72. 110. Smith, D.J., Chapter 4 - Realistic Failure Rates and Prediction Confidence, in Reliability, Maintainability and Risk (Ninth Edition), D.J. Smith, Editor. 2017, Butterworth-Heinemann. p. 43-57. 111. Wood, M.H. and L. Fabbri, Challenges and opportunities for assessing global progress in reducing chemical accident risks. Progress in Disaster Science, 2019. 4: p. 100044. 112. Čolić, A. and I.B. Pečur, Influence of Horizontal and Vertical Barriers on Fire Development for Ventilated Façades. Fire Technology, 2020. 113. Spinardi, G., Fire safety regulation: Prescription, performance, and professionalism. Fire Safety Journal, 2016. 80: p. 83-88. 114. Caldwell, F., Flammable cladding confirmed on five Queensland government buildings, in Brisbane Times. 2018: Brisbane, Australia. 115. Manes, M. and D. Rush, A Critical Evaluation of BS PD 7974-7 Structural Fire Response Data Based on USA Fire Statistics. Fire Technology, 2018: p. 1-51. 116. Kobayashi, Y., Frequency of ignition of non-residential buildings in Japan. Fire Safety Journal, 2017. 91: p. 1035-1043. 117. Rahikainen, J. and O. Keski-Rahkonen, Statistical Determination of Ignition Frequency of Structural Fires in Different Premises in Finland. Fire Technology, 2004. 40(4): p. 335-353. 118. Gwynne, S.M.V. and K.E. Boyce, Engineering Data, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2429-2551. 119. Joglar, F., SFPE Handbook of Fire Protection Engineering, Fifth Edition. 2016: Springer New York. 2875-2940. 120. genco, G., Municipal Building Surveyor report - Lacrosse Building Fire. 2015, City of Melbourne: Melbourne, Australia. 121. John Rae, A. and R.D. Alexander, Probative blindness and false assurance about safety. Safety Science, 2017. 92: p. 190-204. 122. Bjelland, H. and T. Aven, Treatment of uncertainty in risk assessments in the Rogfast road tunnel project. Safety Science, 2013. 55: p. 34-44.

129 5 Evaluation of fire risk assessment methodologies

Around the globe, a handful of documents guide FSE practitioners in performing probabilistic risk assessments. These documents typically are framed within a larger guidance context and have specific objectives or roles within the fire safety design process, typically oriented toward the demonstration of adequate safety levels. More importantly to this work, these documents and the methodologies they present are constructed upon key assumptions. These assumptions are recognized and managed throughout these documents in different degrees and manners. The nature of these assumptions and their management has an impact on the practice of fire risk assessments, many of which have been presented in Chapter 3.

This document sets out to briefly account these guiding documents, describe them and the methodologies they offer and while doing so, identifying the key assumption on which these are formulated. Furthermore, the key elements and steps of the methodologies are compared between them. This comparison helps identifying some of the issues discussed in Chapter 3 as well as positive aspects, drawing a parallel to the methodology proposed in this work. Namely, the guidance documents described, analysed and compared here are listed in Table 5-1. This work is complementary with the evaluation of these guidance documents done by the Warren Centre and the reader is encourage to consult it for criteria related to objectives, structure, competences and compatibility with the Australian context.

The objective of the evaluation is set in section 5.1 and the methodology is presented in 5.2. Sections 5.3 through 5.7 briefly introduce each guidance document and its evaluation results, while the overall results of the evaluation are presented and analysed in section 5.8. A brief discussion section addressing the most relevant elements of the evaluation is presented in section 5.9, leading up to a short set of conclusions presented in section 5.10.

130 Chapter 5. Evaluation of fire risk assessment methodologies

Table 5-1. Guidance documents analysed in this chapter Publishing Current Part Year of Parent Name Scope organization - edition, of 1st document Country year suite edition NFPA 551 – Guide National Fire for the Evaluation of Protection General 6th, 2019 No - 2004 Fire Risk Association Assessments [1] (NFPA) - US SFPE Engineering Society of Fire Guide to Application Protection of Risk Assessment in General 1st, 2005 No - 2005 Engineers (SFPE) - Fire Protection Multiple Design [2] BS 7974:2019 PD 7974-7:2019 Application of Application of fire fire safety safety engineering engineering Specific - BSI Standards - principles to the 2nd, 2019 Yes principles to 2003 buildings UK design of buildings – the design of Part 7: Probabilistic buildings - risk assessment [3] Code of practice [4] ISO 23932- 1:2018 Fire ISO 16732-1:2012 2nd International safety Fire safety (currently Organization for engineering - engineering – Fire General under Yes 2005 Standardization General risk assessment Part revision), (ISO) - Multiple principles - 1: General [5] 2012 Part 1: General [6] SN-INSTA/TR 951:2019 - Fire Safety Engineering Guide for Specific - Standards Norway 1st, 2019 Yes - 2019 Probabilistic Analysis buildings (SN) - Norway for Verifying Fire Safety Design in Buildings [7]

131 Chapter 5. Evaluation of fire risk assessment methodologies

5.1 Objective There is not a standardized approach to evaluate the documents in Table 5-1, as they are put forward by different organizations, framed by different contexts and vary significantly in scope and level of detail. The methodologies for fire risk assessment have many common elements and divergences across documents and it is not the intention of this evaluation to judge the lack of a unified approach.

The objective of this evaluation is to gauge the completeness and appropriateness of each guidance document concerning quantification of fire risk. Chapter 3 and 4 of this work identified a set of important elements associated to the implementation of probabilistic risk assessments in FSE, which are the focus of this evaluation. These elements include the design process and the philosophy it utilizes, the procedures to identify hazards and then construct scenarios, and to quantify and describe uncertainty to fully inform stakeholders. These key elements are translated into the evaluation criteria presented in section 5.2.

This evaluation by itself does not provide a full picture of the advantages and disadvantages of all the criteria, but mainly those concerning the implementation of probabilistic risk assessments in FSE.

5.2 Methodology Given the disparity in redaction, scope and depth of the documents, a set of qualitative evaluation criteria is formulated and a score is proposed to judge whether these are achieved or not by each document. The criteria are selected based on the differences identified in the comparison between Major Hazards Facilities (MHF) and Fire Safety Engineering (FSE) presented in the Chapter 4.

The identified differences are focused on design philosophy, risk ownership, uncertainty and risk as a function of time. These four aspects are evaluated using specific criteria (Table 5-2), which is based on the main assumptions identified for the application of probabilistic risk assessments Chapter 3 and the detailed differences analysed in the previous chapter. There is a potential overlap between criteria 1.2 and 3.3, as both touch on elements of hazards identification. However, the former specifically addresses the guidance with regards the procedure to identify the hazards stemming from the building, while the latter only addresses how the scenarios derived from the hazards identification are bounded. Specifying the boundaries of the scenarios and their limitations (based on its underlying assumptions) can be done during the hazards identification, but criterion 3.3 points towards the need for guidance

132 Chapter 5. Evaluation of fire risk assessment methodologies to do this as an element of uncertainty analysis. Another potential overlap exists between criterion 1.3 and 3.6, as both address the risk assessment approach taken. The former specifically addresses the need for guidance to go from a coarse risk assessment to a more detailed one. Typically, this implies transitioning from a qualitative to a quantitative approach, where the scope of scenarios studied narrows down and focuses on those that can challenge the performance of the building. This gradual approach is described in section 4.5. The latter specifically refers to the quantification of uncertainty, which by default and tradition is done in a probabilistic manner. The criterion points towards the need for guidance indicating the different alternatives to quantification of uncertainty, including those that are non-probabilistic. A compendium of these alternatives is presented in Appendix 1.

Table 5-2. Evaluation aspects and specific criteria Aspect Specific criteria Role of risk assessment: The guidance document clearly states the role of the 1.1 methodology in supporting a specific design philosophy Hazards identification: The role of hazards identification is defined and structured in Design 1.2 the guidance document philosophy Gradual approach to risk assessment: Guidance is provided to implement a gradual 1.3 transition towards quantitative assessments, including qualitative screening of scenarios and use of deterministic analysis Stakeholder identification: Guidance is provided for a structured process to identify 2.1 stakeholders, their roles and responsibilities towards fire safety Risk owner: Guidance is provided to identify the key responsibilities of the risk Risk ownership owner in ensuring an adequate fire safety performance. Risk owner is defined as a 2.2 clearly identifiable stakeholder who has an explicit authority relationship to manage fire risk and a set of responsibilities associated to it. Epistemic uncertainty (1): Guidance is provided on the identification and 3.1 management of epistemic uncertainty. Epistemic uncertainty (2): Guidance is provided on the model uncertainty 3.2 identification and management, including how it is defined, quantified and bounded. Epistemic uncertainty (3): Guidance is provided on the limitations of scenario 3.3 Uncertainty discovery Data limitations: Guidance is provided for analysis alternatives in the absence of, or 3.4 poor quality, data to support probabilistic and reliability calculations Precision requirement: Guidance is provided to explicitly quantify the precision of 3.5 the assessment results and verify that in case of invalid assumptions the estimates do not err on unacceptable performance

133 Chapter 5. Evaluation of fire risk assessment methodologies

Aspect Specific criteria Non-probabilistic alternatives: Guidance is provided on non-probabilistic uncertainty 3.6 analysis alternatives when the application of a PRA is not granted Time dependency assumption: the guidance explicitly recognizes that a snapshot of 4.1 risk in time is not sufficient to effectively manage risk throughout the building life- Risk as a cycle function of time Data gathering: Guidance is provided to construct, obtain or estimate reliability data 4.2 specific to the system studied Key assumptions: Any key assumptions of the methodology including tractability, 5.1 application boundaries and ambiguity are clearly stated by the document, providing guidance to manage them Documentation: Guidance is provided to document the key results and assumptions Assumptions 5.2 of the risk assessment to enable peer review and auditing Peer review: Guidance is provided for a structured peer review process that checks 5.3 the underlying key assumptions of the risk assessment and therefore complement its results

Each of the criteria is judged using a three level score range. The first level corresponds to the criteria not being met (marker ‘DC’), the second corresponds to the criteria being partially met (marker ‘PC’) and the third to the criteria being met (marker ‘MC’). A comment is provided for the level selected for each criteria to justify the selected score in which direct reference is made to the applicable sections or provisions of the document.

5.2.1 Peer review Given the qualitative nature of the evaluation and the known biases that any professional or expert can introduce in such an analysis (see Chapter 1) the author acknowledges the need for peer review. The approach chosen was identifying three prominent experts in the field of fire safety engineering, experienced academics in the use of the guidance, to conduct the peer review. This process implied an initial meeting with the three experts to introduce them to the evaluation and agree on the following steps. All reviewers were sent the initial evaluation results in full, as well as a copy of the existing draft of this document and were given a two-month period to review it. At the end of this period the three reviewers provided a detailed accounting of their agreements and disagreements with the scoring of the evaluation, the supporting comments and with the draft. The author then used this feedback to update the scores (when applicable), answer the reviewers’ comments and to update this document. Without the participation of the three experts, much would be lacking from this evaluation and the author acknowledges their key role in achieving the objective set out in

134 Chapter 5. Evaluation of fire risk assessment methodologies

section 5.1. The three reviewers are Prof Ruben Van Coile from the University of Ghent, A/Prof Thomas Gernay from the John Hopkins Whiting School of Engineering and A/Prof Negal Elhami Khorasani from the University of Buffalo.

5.2.2 Limitations The guidance document significantly differ in contents, scope, intended audience and complexity. This poses an important limitation on the evaluation, as it is needed to carefully scan and interpret the provisions of each guidance document to identify the evidence (or lack of it) for meeting the criteria. It is evident that subjective judgement is fundamental to this exercise and that the conclusions reached will be debatable depending on the experience, knowledge and biases of each practitioner. To mitigate this, the structure of the evaluation and its peer review propend to attaining an objective picture of the guidance document with regards to the evaluation criteria. All considerations associated to the scoring have been provided in Appendix 3, and can be consulted to understand the rationale behind the scoring and to use it within the context of creating new guidance.

5.3 NFPA 551 The NFPA guidance differs from the rest of the documents as it does not intend to provide a framework for conducting fire risk assessments, but to perform a technical review process and construct the documentation for evaluating them. This guidance is by far the most senior among those evaluated here, with six editions so far that have been built upon the need of providing practitioners with a document that describes “the properties of an acceptable fire risk assessment” [1] for any “fire safety problem. Although not explicit, it is understood that this scope covers both buildings and other applications such as chemical processes. Over a period of 15 years, the NFPA 551 has put forward guidance for risk assessment through its “Technical Committee on Fire Risk Assessment Methods” keeping up with the development of construction codes and the increasing reliance on performance-based design. Although not part of a suite of documents, the guidance is closely related to the NFPA 101A – Guide on Alternative Approaches to Life Safety, the NFPA 550 – Guide for the Fire Safety Concept Tree and the Fire Protection Handbook.

One of the main features of the guidance is that it puts forward that the role of a fire risk assessment is to focus “attention on what is important to fire safety” and as one of the elements considered in risk-informed decision making. By doing so, it provides a large range of general and specific elements to be reviewed by competent personnel (also defining competences for

135 Chapter 5. Evaluation of fire risk assessment methodologies

it). A challenge of this approach is that the elements are method-specific and differ largely between them, except for the quantitative and cost-benefit ones, where significant overlapping occurs. Although this specificity is favourable for practitioners who are getting introduced to each type of risk assessment, it can be counterproductive in the sense that it does not provide a common structure to carry out the assessments. Chapter 4 of the guidance document provides a structure to evaluate a fire risk assessment, presenting an overview of the risk assessment process and its relation with the stakeholders, the acceptance criteria, the role of the authority having jurisdiction (AHJ), the scope of the assessment and the uncertainty involved in it. This is a general structure that can be used for evaluating any fire risk assessment, but in the context of performing a quantitative risk assessment and in light of the analysis of Chapter 3 of this work, it is not sufficiently specific nor comprehensive.

The evaluation results are summarized in Figure 5-1. A unique aspect of the guidance related to how it classifies fire risk assessment methods is needed. The guidance defines five different categories that range from qualitative to cost-benefit analyses. This classification is inconsistent with the relevant literature on risk assessments [8-10]. An example is including a technique typically used for hazards identification (What if) as a fire risk assessment method. Structuring the guidance around this classification sends mixed messages to practitioners, as many details are relevant but seemingly applicable only to specific methodologies. This is reflected in the poor score obtained by the guidance in the design philosophy criteria, where the only met criterion is 1.1 by clearly stating the purpose of a fire risk assessment. For the risk ownership criteria, the guidance provides some important basis such as the recognition of the stakeholders and their role in the process, but fails to acknowledge their responsibilities and specifically, those associated with the assessment and its validity. In terms of uncertainty the results are mixed. The guidance clearly requires a list of limitations and assumptions as part of the assessment documentation, specifically the operations and maintenance manual (Chapter 7.4.2). This implies that assumptions have an impact on the operation and future state of the building and a requirement to protect them is put forward (Chapter 7.4.2.2-3, 7.4.3 - Management of Change). The guidance recognizes the potential limitations of the data, typically used to support PRAs. Section 6.2 establishes general requirements on the data quality, including availability, type of sources, lack of data, records management, applicability and uncertainty and variability. On the other hand, key elements of epistemic uncertainty are overlooked without mention of precision and accounting

136 Chapter 5. Evaluation of fire risk assessment methodologies of uncertainty in a non-probabilistic manner is mentioned.With respect to risk as a function of time, the guidance clearly states the need for constructing and maintaining an operations and maintenance manual to identify the conditions that must be kept in order for the risk assessment to remain valid. It does not provide any guidance on the construction of datasets which can later support the update of risk assessments. Finally, the guidance mostly meets the criteria related to assumptions, with a notable exception. The exception has to do with criterion 5.1 regarding the key assumptions of the assessment. Although Chapters 5.4.4.2 and 7.4.2 mention the need to explicitly state assumptions and treat them through sensitivity, these are related to specific (limited) elements as presented in Chapter 7.3, which are associated with comprehensive project documentation. Key assumptions regarding the methodology and the limitation of their validity are not explicitly treated by the guidance.

Figure 5-1. Evaluation results – NFPA 551

5.4 SFPE guide The SFPE Engineering Guide to Application of Risk Assessment in Fire protection Design contrasts with the NFPA guidance in that it is mean to provide guidance on the selection and use of fire risk assessment methodologies. This guidance covers both fire risk assessment for buildings and processes. The guidance can be set apart from much of the literature in fire risk assessments due to its definition of risk. Typically, this is taken to be the function of likelihoods and consequences for pre-defined or selected scenarios. In the SFPE guidance, risk is defined as a function of 1) unwanted adverse consequences, 2) appropriate scenarios and 3) their associated frequencies. The key element here is the mention of appropriate scenarios (Chapter 2).

As in the NFPA guidance, the definitions provided for the different type of risk assessments are not consistent with the general definition of the process as per ISO 31000:2009. A qualitative risk assessment is taken to be no more than a mere hazards identification relying on

137

Chapter 5. Evaluation of fire risk assessment methodologies narratives and checklists, while the quantitative assessment provides explicit estimations for defined scenarios. It appears the definition of the latter ignores that quantitative assessments also rely on narratives and subjective criteria to construct the scenarios, which then becomes part of the uncertainty associated to the results (whether or not it gets explicitly stated).

The guidance document provides an unusual but important distinction between a hazard (something with inherent properties that can cause harm, e.g. combustible material) and deficiencies. The latter refers to hazards that “can be eliminated with appropriate engineering or safety management procedures.” This subtle difference echoes the need to understand fire safety as a key driver of the design philosophy, as discussed in Section 4.5. Furthermore, hazards as an element with the potential to cause harm is further classified in initiating event or hazard source (heat/fuel sources/hazardous practices and environments), enabling conditions (elements which enable consequences to escalate, particularly with respect to fire growth and spread) and vulnerabilities (special characteristics that can increase the damage potential of a hazard). Each of these can be treated to different degrees in the risk assessment and in the design process, making it important to understand their differences in order to effectively identify them.

Although this evaluation does not fully cover the NFPA 550 - Guide to the Fire Safety Concepts Tree document, it is referenced in the SFPE guidance and it is worth noting its importance in providing a qualitative model for structuring fire safety. Despite independence assumptions and lack of consideration of changes (explicitly stated), this additional resource serves as a practical basis for hazards identification and for pin pointing the effect of the decision-making along the design process. Furthermore, the guidance clearly indicates the need for a consideration of the change in time of the system (Section 5.6).

Several assumptions are laid out without addressing their potential effect, such as stating that a "full analysis combining models" results in the user not having to "worry about the links between models." Evidence like Beard's work [11-13] suggest this is an oversimplification. Other guidance (BS 7974 [4]) indicates the complexity of defining the system boundaries to then support consequence modelling. Furthermore, subjective statements are made regarding the difficulty of a consequence analysis in comparison to a hazard identification (section 11.4.1). Although some of the statements in section 11.4 can be valid in a large range of conditions, these are not general and do not provide a framework of reference.

138 Chapter 5. Evaluation of fire risk assessment methodologies

A unique feature of the SFPE guide concerns section 14.3.1.2, which calls to revisit the acceptability criteria in cases where the assessed risk is found to be intolerable or unacceptable. In other words, when the technical analysis cannot justify the adequate fire safety performance, this provision enables revisiting the criteria and considering alternative one that could justify it somehow. It is clearly stated that this does not mean that the criteria is lowered as to pass, as this would be unethical to start with, but it can indeed be further decomposed into more specific criteria to verify partial acceptance. No detailed guidance or examples are provided. This is particularly relevant for how the acceptance criteria is defined and used in the Maximum Allowable Damage methodology (Chapter 6).

Given the wide spectrum of detailed guidance provided by the SFPE document, it constitutes a key tool for any FSE practitioner and it sets important elements that should be considered in any risk assessment. This is reflected in the overall high score of the guidance, as summarized in Figure 5-2. However, it is evident in the language and redaction use that there is a significant focus on the calculation aspect of the risk assessment, rather than on its knowledge building capacity to support decision-making. This focus on calculation tends to skew the guidance towards practical tips, resorting to useful but specific examples. On the other hand, this skew affects negatively the way the document addresses epistemic uncertainty sources and the amount and specificity of guidance to recognize its importance, communicate it and treat it. First, with regards to uncertainty the document does not offer guidance on how to treat the uncertainty associated to the models used within the risk assessment. Chapter 10.5.4.3 addresses the issue of probability distribution selection and the potential issues of using a 'default' distribution, but this is a particular example, while Chapter 11 presents a simplistic representation of what a consequence analysis is in a fire risk assessment. It does not specify the need to formulate a model for performance and to define its boundaries. On the other hand, the document does provide some key basis for accounting for uncertainty in the assessment, for example as in Chapter 13.3.2.2. This entry addresses uncertainty sources that are not quantified in the analyses and the need to justify them, as well as defining an approach to treat them. Another criterion not met by the guidance has to do with data gathering (criterion 4.2). Although the document refers to the NFIRS database in Chapter 9 and guidance is provided on the role of data and how to apply it in the risk assessment, it does not address how to construct the database, the different sources to obtain it nor the methods to estimate it.

139 Chapter 5. Evaluation of fire risk assessment methodologies

Figure 5-2. Evaluation results – SFPE guide

5.5 PD 7974-7 (2019) The PD 7974-7 guidance document constitutes one of the most detailed pieces of guidance to conduct a PRA. The document is one of seven interrelated documents that are formulated under the general guidance of the BS 7974:2019. This is the Code of practice for the Application of fire safety engineering principles to the design of buildings, and it is one of the guidance documents specifically constructed for fire safety in the built environment. The suite of guidance documents under BS 7974:2019 consists of eight parts, each one addressing different sub-systems of the fire safety problem. These cover fire initiation and development (part 1), spread of smoke and toxic gases (part 2), structural response to fire and fire spread (part 3), fire detection and protection systems activation (part 4), fire service intervention (part 5), occupant evacuation (part 6), probabilistic risk assessment (part 7, concerning this evaluation) and property protection (part 8). This is the only suite of such a wide scope within all of the guidance documents evaluated here. It is noticeable that BS 7974:2019 sets the basis for all studies related to fire safety for buildings, describing a general process to address hazards and risks. This process proposes the use of qualitative (screening) methodologies that somehow provide the basis for the more detailed probabilistic risk assessments described in part 7. However, the ‘somehow’ is not clearly defined. It is also worthwhile noting that there is not a particular argument for why probabilistic assessments are favoured over deterministic analysis, or any other kind of quantification that does not directly rely on developing likelihood insights.

Another distinctive feature of the guidance is that the data it uses to support likelihood estimations has been subject of study and update [14], as well as the concepts underlying risk acceptance [15]. However, some of these data is already outdated and might not be applicable to a large set of problems. More importantly to this research, the guidance is constructed upon assumptions not clearly supported such as the inability of a deterministic evaluation to provide

140 Chapter 5. Evaluation of fire risk assessment methodologies an explicit safety evaluation. Furthermore, the PRA is portrayed as to be able to go over these drawbacks, for which no support is provided. Such important assumptions are not discussed in the document and could potentially send a biased message to practitioners, particularly those not akin to risk and safety science but only in look for tools to verify the compliance of their work.

The recommended five step process for uncertainty treatment starts with identifying uncertainty sources and then performing a qualitative review to screen them based on their potential impact on the results. Although there is no additional detail on the nature or steps involved in this qualitative review, a series of proposed treatment pathways are put forward for each type of uncertainty. Although this is a key element of the guidance, it does not provide enough detail, which effectively generates a knowledge gap.

The results for the evaluation of the guidance document are summarized in Figure 5-3. Due to the narrow scope of the document, focusing on the execution of the probabilistic risk assessment, it does not meet any of the risk ownership criteria. The clauses in the BS 7974:2019 document describing the relationship of stakeholders within the application of fire safety engineering principles is not deemed enough, as there are particularities of a PRA that must be addressed in a specific manner and that can easily scape general provisions. The particularities hereto referred are those presented in Chapter 3. In this same line, the document fails to clearly establish a connection between the hazards identification process and that of conducting the PRA (criterion 1.2). Figures 1, 2 and 3 of the guidance document establish the procedures for the PRA, and fail to indicate hazards identification as a step in the process for demonstrating adequate safety of a design or in the PRA process. In Figure 3 the question "are all possible consequences tolerable?" seems to address only the identified scenarios, but fails to discuss the limitation of the hazards identification step and the implication of having unidentified scenarios with unacceptable consequences. A strength of the document was found in the guidance it provides to address epistemic uncertainty, for which it meets all criteria. Although it does not fail to meet the criteria regarding data limitations, precision and non-probabilistic alternatives, the guidance provided is not clear enough nor connected to the overall needs to understand and communicate uncertainty. This is the same case for the criteria addressing assumptions. Finally, the document includes no mention of risk as a function time and the need to review and update the assessment as operational conditions in the building change.

141 Chapter 5. Evaluation of fire risk assessment methodologies

Figure 5-3. Evaluation results – PD 7974-7 (2019)

5.6 ISO 16732-1 (2012) The ISO 16732-1:2012 is not compatible with the ISO 31000:2018 standard on risk management, despite belonging to the same standardization organization. This translates into the use of an ad hoc language and structure for risk assessment. The diagram presented in Figure 1 of the document does not match the structure proposed by the ISO 31000 counterpart and seems to be subjectively narrowed down to the context of fire safety without considerations of the principles underlying risk management. In large extent, the ISO 16732-1:2012 seems to be a minor extension of quantitative fire risk assessments originating in ISO 23932-1:2018 Fire safety engineering - General principles. However, these documents are not aligned and do not provide practitioners with a unified approach to fire risk assessments. This lack of alignment can be extended to closely related documents and cannot be considered as a suite. The related documents are:

• ISO 23932-1, Fire safety engineering – General principles • ISO 13943, Fire safety – Vocabulary • ISO 16730-1:2015, Fire safety engineering — Procedures and requirements for verification and validation of calculation methods — Part 1: General • ISO 16732-1:2012, Fire safety engineering — Fire risk assessment — Part 1: General • ISO 16733-1:2015, Fire safety engineering — Selection of design fire scenarios and design fires — Part 1: Selection of design fire scenarios • ISO 16734:2006 through 16737:2012 – Requirements governing algebraic equations for different fire dynamic phenomena • ISO/TR 16738:2009, Fire-safety engineering — Technical information on methods for evaluating behaviour and movement of people

142

Chapter 5. Evaluation of fire risk assessment methodologies

• ISO 24678-1:2019, ISO 24678-6:2016, ISO 24678-7:2019 – Requirements governing algebraic formulae for different fire dynamic phenomena • ISO 24679-1:2019, Fire safety engineering — Performance of structures in fire — Part 1: General • ISO/TS 29761:2015, Fire safety engineering — Selection of design occupant behavioural scenarios

Although the ISO 16732-1:2012 document addresses ‘fire risk assessments’ in general, the definitions presented in section 3 and the argumentation of section 4 seem to indicate the focus of the guidance is quantitative or probabilistic risk assessments where explicit consideration and quantification of scenario likelihoods takes place. As noted in the justification for criterion 3.2, most of the argumentation and structure of the document does not match the quantitative and ‘well-structured’ nature of the proposed method. The document includes complementary examples for and office building and for an industrial facility presented in ISO 16732-2:2012 and ISO 16732-3:2012. The first example includes unstructured scenario identification and clustering, as well as the use of the comparative approach using a reference building as acceptance criteria. The uncertainty is expressed in a qualitative description which does not reflect the impact it may have on the resulting performance. The second example seems to fit chemical process safety and uses an approached based solely in frequency of occurrence of an undesired event (BLEVE). The example does not incur in a formal uncertainty analysis and seems not to directly contribute to the understanding of fire risk or fire safety performance, despite introducing the reader to useful tools. The application examples of ISO 16732-1:2012 include the use of different methods and tools available to the FSE practitioner, but do not provide a structure guidance to conduct a risk assessment and to explicitly consider the uncertainties involved.

Overall, the guidance document provides mixed elements useful to a practitioner, but the lack of a structured approach with underlying guiding principles makes it vague and does not avoid the issues identified herein. This is reflected by the significant amount of criteria which are not met by the guidance document in this evaluation. Mainly, risk ownership and risk as a function of time are not considered by the document. With regards to the former, it is worth noting that ISO 23932-1:2019 mentions affected parties or stakeholders, but limits their involvement in the fire risk assessment process to that of being informed of deviations in order to document them and seek their agreement. This diverges greatly from the concept of risk ownership and stakeholder engagement. With regards to the latter, there is no reference to changes in time and

143 Chapter 5. Evaluation of fire risk assessment methodologies

Section 6 fails to indicate the need for data gathering, including component, sub-systems and systems failure data under fire and normal operation. This section also fails to mention the necessary characteristics of the data for a reliability analysis as part of a probabilistic assessment.

Figure 5-4. Evaluation results – ISO 16732-1 (2012)

5.7 TR 951 (2019) This document is part of a suite of two different documents (TR 950 and TR 951) which address fire safety in Norway, specifically when a deviation from the prescriptive provisions occur in the design of a building. This significantly narrows the scope of the documents and their purpose, as less emphasis is put on a holistic understanding of the system and its fire risk, and more emphasis is put on achieving compliance. TR 951 concerns the use of quantitative (probabilistic) analyses to verify such achievement once a deviation has been identified.

The Standards Norway guidance provides unique guidance for the acceptance criteria to use in the risk evaluation. This guidance is based on research conducted between the 1980 and 2000 by Litai [16] and Wolski [17] and a main outcome is a set of Risk Conversion Factors, by which a practitioner can modify the acceptance criteria. These factors are associated to variables that have been shown –statistically- to have an impact on the numerical value of risk (formulated as fatalities per year). The statistical analysis and identification of these factors is based on widely accepted activities in the society, e.g. smoking, chemical plants. The factors become quite relevant when the acceptance criteria (typically based on mortality rates) has to be modified to take into account particularities of the risk under study. Examples of these factors are volition (voluntary vs. involuntary), severity (ordinary vs. catastrophic) and benefit (clear vs. not clear). Litai shows that the ratio of acceptance between –for example- voluntary and involuntary risks is of 100 times. The guidance provides limited examples in the use of these factors, but fails to clarify the potential uncertainty involved in the multiplicative

144 Chapter 5. Evaluation of fire risk assessment methodologies

factors for modifying the acceptance criteria. In the original research [16] this uncertainty is expressed with a confidence of 90% for the range (4 to 20) and taken typically as 10. Furthermore, this research was based on statistical data of 1976 for the USA and has not since then been updated.

The evaluation results for the guidance document are summarized in Figure 5-5. Based on section 4.1, a fire risk assessment is used to verify the level of safety of a design”, which based on section 1 means to "use fire safety engineering methods in order to evaluate compliance with an absolute criterion". This significantly limits the possibility of using the fire risk assessment as a design tool. This leads the guidance to not meet the criteria for design philosophy, which has been identified as a central issue of PRAs in FSE. Risk ownership is also not met by the guidance. Although the introduction mentions that selecting voluntary performance criteria involves the stakeholders, the process is not specified. The guidance does not refer to how the risk owner should participate in the risk assessment process or the role of its outcomes in the decision making process.

Despite mentions regarding uncertainty due to lack of knowledge or imperfect knowledge in section 6, section 7 dealing with uncertainty does not explicitly address epistemic uncertainty or its implications in the verification of safety. Section 7.3.3 refers to model uncertainty, addressing that form software packages and indicating the need for 'considerable margin of errors'. The previous leads to not meeting the criterion for epistemic uncertainty (3.1). Although all other criteria regarding uncertainty are addressed by the guidance, it does it only in a partial manner. No explicit consideration is given regarding the understanding of the performance of the fire safety strategy through a model, to the subjective nature of the scenario identification process nor to the implications of undiscovered scenarios. Data uncertainty is addressed in section 7.3.2, calling for the practitioners to consider the effect of sample size and other relevant parameters on the final quantification. However, no structured process or set of requirements are provided to account for these elements. Perhaps, one of the most important elements with regards to uncertainty is the statement made in Section 8 regarding the conclusions of the analysis and the effect of assumptions on them. Here, it is explicitly stated that no assumption or simplification should alter the conclusions beyond the justifications and considerations provided. This could be considered a clear requirement on the precision of the result and significantly contribute to uncertainty management. Nevertheless, this requirement is left underspecified and has not associated procedure to support it.

145 Chapter 5. Evaluation of fire risk assessment methodologies

The two criteria for risk as a function of time are met fully and not at all. With regards to data, Section 6.4.1 addresses the need to use available it and the assumptions involved in its interpretation. A few key criteria are provided to check the data is satisfactory and to discard unreliable data. Examples are not provided, although Annex B provides sample statistical data and sample size is addressed in section 7.3.2. Annex C specifically addresses reliability, providing references for the data sources as well as resources to estimate probability of failure of selected safety measures. Furthermore, Annex C.9 provides an introduction to reliability analysis, which aids practitioners in estimating reliability figures. However, with regards to understanding the changes experienced by the building throughout its life-cycle, the document provides little to no guidance. Annex B.1 addresses the validation of fire statistics to be used in the probabilistic risk assessment requires considering the changes in regulations and developments in fire safety strategies, as these influence data. This consideration calls for using the most recent available data. The guidance does not explicitly consider the dependence of all presented figures on time, nor does it consider the need to consider the implication of their change in time.

Concerning assumptions, the guidance provides a clear set of requirements. Section 7 specifies the need to state and manage key uncertainties (mainly aleatory ones), while Section 8 puts forward the need to reach conclusions which are not affected by unjustified assumptions or simplifications. This is a clear call to competent practitioners to reflect on all uncertainty sources and their effects on the conclusions. This key element is formulated only here and in general terms, limiting its potential benefits on the overall process. Source references like Wolski address significant assumptions and criticism, which are not included in the guidance and can impact decision-making. Source references like Wolski [17] address significant assumptions and criticism, which are not included in the guidance and can impact decision- making.

Figure 5-5. Evaluation results – TR 951 (2019)

146 Chapter 5. Evaluation of fire risk assessment methodologies

5.8 Evaluation results The results presented in this section specifically address the criteria of Table 5-2 and do not make a general characterization of the documents with regard to other important aspects and criteria. Figure 5-6 presents the comparative results of the evaluation, with the results for all criteria added up. The overall result shows that no guidance document meets all (15) criteria. The one that met the largest range of criteria is the SFPE guide with nine (9) met and five (5) partially met, followed by the NFPA 551 with six (6) met and five (5) partially met. All guidance documents failed to meet at least two (2) criteria and up to eight (8). The only guidance document that did not mostly met any criteria was the ISO 16732-1 (2012). Partially met criteria can contribute important elements for consideration by practitioners and for the construction of future guidance, and therefore no weighted score is provided. Instead, the overall results are further breakdown to understand their implications.

Figure 5-6. Overall results of the guidance document evaluation

Table 5-3 provides a more detailed picture of the evaluation results. This representation allows to identify major blocks of criteria being met at different degrees. By amount of fully met criteria, both NFPA 551 and the SFPE Guide present a better basis for performing fire risk assessments, particularly when applying a probabilistic methodology. Taking into account the number of criteria met or partially met, the SFPE Guide scored the best result again and is followed by PD 7974-7. Although the latter only meet three criteria fully, it does provide the more complete guidance after the SFPE guide, and the only one to meet all epistemic uncertainty criteria. The lack of specificity and connection to important references of the same organization seem to be reflected by the results for ISO 16732-1 (2012). From these overall results it is important to notice that no guidance fully meets the following criteria:

• (1.3) Gradual approach to fire risk assessments

147 Chapter 5. Evaluation of fire risk assessment methodologies

• (2.2) Risk owner identification • (3.5) Precision requirement • (3.6) Non-probabilistic alternatives

A possibility for these criteria not being met is their lack of relevance. However, all of them have been directly derived from the discussions of both Chapter 3 and 4 and are considered fully relevant for any of these documents. The differences in the FSE context with respect to others such as nuclear of chemical process safety, justify why these criteria are not met by the available guidance. This set of criteria directly speak of the improvements needed in the understanding of the role of fire risk assessments, not only as a technical tool but as an element that integrates the understanding of fire risk with the decision-making process.

Table 5-3. Detailed results of the evaluation Evaluation Specific SFPE PD 7974-7 ISO 16732- TR 951 NFPA 551 criteria criteria Guide (2019) 1 (2012) (2019) 1.1 MC MC PC PC DC Design 1 1.2 DC MC DC PC DC philosophy 1.3 DC PC PC DC DC Risk 2.1 PC MC DC DC DC 2 ownership 2.2 PC PC DC DC DC 3.1 MC MC MC PC DC 3.2 PC DC MC DC PC 3.3 PC MC MC PC PC 3 Uncertainty 3.4 MC MC PC PC PC 3.5 DC PC PC DC PC 3.6 DC PC PC DC PC Risk as a 4.1 MC MC DC DC DC 4 function of 4.2 DC DC PC DC MC time 5.1 PC PC PC PC MC 5 Assumptions 5.2 MC MC PC PC MC 5.3 MC MC PC PC PC

Table 5-4 presents the overall results broken down for each one of the five criteria, highlighting the best and worst scoring guidance document. The results show that the least met criterion was risk ownership, with only the SFPE Guide meeting one of the specific criterion (2.1) and partially meeting the second (2.2). The criteria that was met the most was related to assumptions, with all of the guidance document meeting –at least partially- all the criteria. It is

148 Chapter 5. Evaluation of fire risk assessment methodologies noticeable that the overall best scoring guidance (SFPE Guide and NFPA 551) met the specific criteria associated with documentation (5.2) and with peer review (5.3) which are traditional elements of the FSE practice. TR 951 (2019) was the only guidance that fully met the specific criterion about identifying and managing key assumptions in fire risk assessments (5.1).

The main criteria with the most diverse results was design philosophy, which was nearly fully met by the SFPE guide and which was not met at all by TR 951 (2019). This presents an interesting contrast between the intent of the documents and the effect it has on the usefulness of the risk assessment. In the first case, the SFPE Guide is a general-purpose document that covers all fire safety problems while clearly stating the role of the risk assessment and framing it within the decision-making process. As it is general and wide reaching, it addresses a large range of issues and elements that can be relevant to the practitioner, and presents different situations that can take place during the assessment along with strategies, procedures and tools to address them. On the other end, TR 951 (2019) constitutes a guidance document with a narrow scope and the clear intent of verifying safety. This highlights the limitations of conceptualizing risk assessments as a mere technical analysis that validates a design and points towards the need of understanding them as highly interconnected and dynamic elements that inform decision-making (and do not settle it).

Table 5-4. Results per evaluation criteria Design philosophy

Best: SFPE Guide Worst: TR 951 (2019)

Risk ownership

Best: SFPE Guide Worst: TR 951 (2019)

Uncertainty Best: PD 7974-7 (2019)

149 Chapter 5. Evaluation of fire risk assessment methodologies

Worst: ISO 16732-1 (2012)

Risk as a function of time

Best: NFPA 551, SFPE Guide Worst: ISO 16732-1 (2012)

Assumptions

Best: TR 951 (2019) Worst: PD 7974-7 (2019), ISO 16732-1 (2012)

5.9 Discussion

Hazards identification In general, fire hazards will encompass anything that can burn or contribute fuel to an existing fire. The systematic identification of ‘anything that can burn’ seems futile in the context of FSE, when in other contexts it leads to a finite, discrete accounting, despite imperfect. In FSE, the issue of hazards identification is indeed closely intertwined with that of scenario development. In other fields, one flows naturally onto the other. In FSE, one replaces the other. By defining reasonable scenarios (e.g. external fire spread, blocked exit) the hazards identification is superseded. The argument put forward here is that this is wrong as it disables practitioners from reflecting on the origin of the hazard, the manner in which it materializes and the array of conditions and actions that can result in scenarios worthy of analysis.

150 Chapter 5. Evaluation of fire risk assessment methodologies

Noticing the scores for the five guidance documents, the SFPE guideline is the only one achieving an ‘F’ score. The NFPA document provides guidance on hazards identification which seems to be more coherent with the complexity of the problem. Firstly, it does not provide an empty text addressing ‘hazards identification’ without any connection, as the ISO document does. Second, it clearly states the importance of the step and its complexity. Third, it provides a set of different tools, which differ in scope, complexity and ease of implementation, to carry out the hazards identification. One of these is the NFPA Fire Safety Tree, which is a concept map of interrelated conditions and actions which guide the practitioner in identifying building blocks for the scenarios.

All three independent reviewers considered the criterion for hazard identification as to be partially fulfilled with the provisions for scenario identification. Based on the previous arguments, it should be clear that these are not sufficient to account for the complexity of the hazards identification process. From the existing literature in FSE it is clear that this is ‘the way it is done’, but there is no better moment than now to recognize that many of the clearly unacceptable fires that have taken place in the last decade involve the clear omission or exclusion of key fire hazards. Perhaps, this calls for a more precise definition of fire hazards in the context of FSE which is not limited to materials with the inherent property of being combustible or flammable. Fire hazards could rather be understood as the collection of conditions and actions around a fuel that can lead to a fire. This resembles the concept of scenario cluster, but differs from it in that clusters are subjectively defined, while hazards must be systematically identified.

Due to the complex and unforeseen interactions that occur in the interface between the occupants, the building, its management and the relevant authorities, the existing hazard identification techniques such as what-if depend on brainstorming workshops and are largely unstructured. Other techniques looking into specific details to construct scenarios such as failure mode and effects analysis (FMEA) might fall on the opposite end. A hybrid approach could be a potential solution, in which a conceptual basis such as the NFPA fire safety tree is configured to the unique features of the building and then used to systematically extract all potential permutations of actions and conditions that lead to a set of scenarios. The bottom line is that traditional hazard identification techniques stemming from chemical process safety do not match the intangible nature of hazards in the built environment; although the guidance points towards these techniques, these are far from a solution to systematically and comprehensively analyse hazards in a building design.

151

Chapter 5. Evaluation of fire risk assessment methodologies

Gradual approach One of the central elements to risk management in high-risk industries is to gradually implement risk assessments from the conception of the projects, through design, construction, commissioning and operation and all the way through to decommissioning. The degree of complexity and resolution of risk assessments change at each stage and often, multiple risk assessments implementing different techniques and with complementing focuses are used. Overall, the process aims at understanding whether risks are being managed adequately and what actions are needed in case of the opposite. Accidental scenarios are constructed in all of the risk assessments, whether it is implicitly or explicitly, and the knowledge on these scenarios is refined as risk assessments are updated and complemented. The engineering tools to carry out risk assessments differ in complexity and practicality, but usually require multiple tools and multi-disciplinary teams to construct a risk picture. In order to achieve safety objectives, there is no silver bullet, no ideal methodology and no all-knowing professional. Considering the previous, it is only logical to face significant challenges when assessing fire risk, as buildings and the built environment in general, pose an infinite set of scenarios. Some of these are complex combinations of conditions and actions which cannot easily be imagined and that might not have yet occurred. Herein lies the importance of gradually building up the understanding of hazards and scenarios, making use of different techniques, tools and points of view. This was partially reflected by the guidance document through criteria 3.1 through 3.3 regarding epistemic uncertainty. In particular, criterion 3.2 regarding scenario bounding was found to be met by both the SFPE Guide and PD 7974-7 (2019).

In the SFPE Guide, the definition of "Fire scenario, representative" in Ch. 2 provides a first glance at the limitation of scenario discovery, as the representative fire scenario is assumed to have a consequence that adequately describes the consequences of a scenario cluster. However, this assumption is not clearly recognized and then fails to be managed in Section 8.3 in which a set of qualitative (subjective) statements are used to structure scenarios. These include the assumption that a single design fire calculation represents the average consequence of all scenarios in the cluster and additional considerations are built upon this assumption. Sections 8.7.5.2 and 8.7.5.3 provide important cautions to practitioners regarding exclusion of scenarios and the need for documenting and communicating it to the relevant stakeholders, particularly in the case of scenarios with high and 'unavoidable' consequences. In PD 7974-7 (2019) the situation is similar when inspecting Section 6.5.4, which addresses completeness uncertainty. This section recognizes the potential lack of knowledge concerning hazards, initiating events,

152 Chapter 5. Evaluation of fire risk assessment methodologies

failure modes, operation modes, behaviours, phenomena or other relevant factors. Although it is not stated in the guidance, all these factors are the constitutive elements of scenario construction.

The previous shows a recognition of the difficulty in structuring scenarios, yet the results for criterion 1.3 regarding a gradual approach to risk assessments show that this is not yet recognized as a potential solution. Most guidelines address the distinction between qualitative, semi-quantitative and quantitative assessments, and it is reasonable that many practitioners have their preferred methods. However, little or nothing is said regarding the multiple approaches available for quantification. In a way, it seems that quantification directly means applying a probabilistic method. This is far from the reality, as deterministic analyses can quantify risks. Although this quantification generally focuses on consequences, it can be done initially not only to get insight on hazards, scenarios and the upper end of the consequence scale, but to provide a basis for later probabilistic analyses. In some occasions, the results of the deterministic analysis might provide enough information to support decision and structure the fire safety strategy of a building, without need for a probabilistic assessment. In other occasions, such as when variability data is missing, it should be the first option.

The importance of deterministic analysis is in fact recognized by most of the guidance, which is reflected by criterion 3.6 regarding non-probabilistic alternatives. Three guidance (SFPE Guide, PD 7974-7 (2019) and TR 951 (2019)) partially meet this criterion, mostly by identifying deterministic analyses as an option. Particularly, the SFPE Guide in Section 8.7.2 contemplates using qualitative methods, which include worst-case evaluations and maximum foreseeable loss. Recognizing such alternatives is important, but does nothing in relation with the need for gradual risk assessments. This gradual approach would require not only recognizing assessment alternatives, but clearly identifying the connection between them and how the level of detail of information changes between them. PD 7974-7 (2019) recognizes deterministic analyses at the same level of importance of the probabilistic, as clearly depicted in its Figure 1. However, the options are portrayed as ‘one or the other’. This lack of recognition of the gradual approaches is a key element to develop in the future with the objective of improving the quality of the risk assessment process and its relevance in decision-making.

Required precision It is relevant to notice that acceptance criteria was not part of the evaluation criteria. This was a conscious decision, as the discussion of this topic in Chapter 3 provided a clear understanding

153 Chapter 5. Evaluation of fire risk assessment methodologies

of the existing disparity between the acceptance criteria used in different applications. Nevertheless, several of the main criteria were directly related to acceptance criteria, with the most relevant one being uncertainty. The six specific criteria for uncertainty (3.1 – 3.6) had direct connection with it, and no other more than the precision requirement (3.5). Defining the precision required by a risk assessment is a function of the margin between the result of the assessment and the position of the acceptance criteria. In terms of an F-N curve this is usually done in terms of orders of magnitude and there is evidence of how even properly ran assessments can be off by several orders of magnitude [18].

Noticeably, TR 951 (2019) provides the most guidance about precision. Section 7.2 introduces sensitivity analysis as a way to understand the impact of varying mathematical formulations (models) and inputs on the performance. This is an important element, but one which is typically captured in uncertainty quantification [19]. Section 8 makes a clear statement regarding the conclusions reached by the analysis and the effect of assumptions on them, explicitly stating that no assumption or simplification should alter the conclusions beyond the justifications and considerations provided. Such a statement cannot be clearer, however the guidance provides no procedures or tools to implement this in practice.

The SFPE Guide also addresses precision partially and in Section 13.3.6 describes the potential for switchover, which is when the assessment can err on the unacceptable performance side. No detailed guidance is provided to execute it beyond reference to the SFPE Handbook [20]. Section 14.4 also hints at the issue of erring on the unacceptable performance side and recommends (without supporting evidence) implementing detailed analysis, when the basis for decision is simple analysis. If not possible the guidance indicates that the risk should not be considered acceptable and proposes reducing risk or revisiting the acceptance criteria. This is a bald statement from the guidance, and it is then clarified that revisiting the acceptance criteria does not mean to simply lower it. Instead, it is suggested (again without practical guidance) to further decompose the acceptance criteria and perform analyses that show the degree to which it is met. Such suggestion resonates with the discussion in Chapter 3 about using consequence-oriented criteria, which enables the exclusion of highly variable and complex to model phenomena, such as occupants’ behaviour during evacuation.

Although PD 7974-7 (2019) provides a detailed discussion on a proposed hierarchy of acceptance criteria, this is a classification that is not clearly recognized in risk literature. Furthermore, the classification has underlying assumptions that directly address the validity of

154 Chapter 5. Evaluation of fire risk assessment methodologies the risk assessment. These are not laid out explicitly in the document and although the original source by Van Coile et al. [15] provides significant clarification, it also calls for continuing the discussion. This classification is particularly relevant in the structural fire engineering field, which is a subset of fire safety engineering. What might well be a perfectly working solution in structural fire calculations such as reliability targets, is not straightforward to apply when addressing the overall fire safety performance of a building. Beyond the acceptance criteria provided by PD 7974-7 (2019), the guidance does not provide any hints on what choosing a criterion implies for the precision requirement. A reliability target would require a very high degree of precision, which as discussed in Chapter 3, is hard to guarantee when quantifying the fire safety performance of a whole design.

Quality of the assessment How to define the quality of a fire risk assessment? This question about quality was not part of the criteria, as there is barely any mention of quality within guidance. However, quality is a central element of any risk assessment if its intended use is to provide accurate risk estimates as in the case of PRAs in FSE [21]. There might be a presupposition that professional, competent practitioners conducting the assessment, inherently ensure a high degree of quality. But this presupposition is weak in light of the doubts casted over the competencies of FSE practitioners and the work still needed to ensure them [22, 23] (see Sections 1.4, 3.6 and 4.3).

Competencies aside, there are other key factors that define the quality of a fire risk assessment. One of this is the explicit acknowledgement of uncertainties and their treatment. Most of the guidance evaluated here have provisions on this regard, as shown with criteria 3.1 to 3.6. Yet, none formulates a clear definition of what a high quality assessment is. One could argue that quality is given by the level of sophistication of the mathematical solution underpinning the likelihood and consequence estimations. Another possibility is the uncertainty range of the results and the breadth of uncertainties taken into account. Another way of defining quality could deal exclusively with the quality and appropriateness of the data used. And the possibilities go on, with none providing a criterion that fully embodies what a high quality assessment is.

The author believes that the answer for the quality of an assessment lies in the practical function it has, rather than in its constituent elements, i.e. adopting a pragmatic approach. In practice, risk assessments inform decisions with a potentially large economic and safety impact. As stated in Chapter 1, these decisions are not made solely based on risk assessments, but on

155

Chapter 5. Evaluation of fire risk assessment methodologies different information sources. For the risk assessment to inform decisions it must be trustworthy, i.e. the stakeholders should believe the information it provides is worth of consideration. Without the trust from the stakeholders, no other technical aspect of the assessment will matter. Therefore, a high quality risk assessment provides a basis for trustworthy information that supports decision-making. For an assessment to be trustworthy it requires technical credibility [24] as well as the recognition of its limitations.

5.10 Conclusions The evaluation has revealed a widely diverse set of guidance documents, varying in scope, structure, supporting material and approach. This diversity is reflected by the results of the evaluation, where no guidance meets all criteria. Importantly, this diversity also reflects some of the central issues of applying probabilistic assessments in FSE. The results show a lack of unified approaches to build the basis for a probabilistic assessment and to then execute it, which is compensated by the fact that as a whole, the five guidance documents provide vital knowledge and exemplification for the FSE practitioners.

It is a conclusion of this work that the understanding of the role of fire risk assessments in FSE is anything but clear. Currently it is found in the range made up by a mechanistic tool to verify safety and a process that yields insight to support decision-making. Furthermore, the role of stakeholders –and particularly that of the risk owner- in the risk assessment process is widely ignored. This cuts short the options to ensure that the risk assessment is useful, regardless of the uncertainty it carries in its results. Guidance is needed where the proactive role of the stakeholders is outlined, particularly with regards to setting the objectives and acceptance criteria, as well as the communication of uncertainty and assumptions.

The existing body of guidance does contain a wealth of provisions, statements and examples that as a whole can effectively guide a competent FSE practitioner. The understanding of what constitutes a competent FSE practitioner is a matter of debate and current work by organizations such as the Warren Centre [22] following a series of fires that have put professionals in the fire safety area under the scrutiny, resulting even in considerable liabilities. Putting the competency issue aside, the results show the need to take this (and other related evaluations) into account in any future efforts of updating the guidance.

In terms of the preference for the probabilistic risk assessment as a way to demonstrate fire safety performance, the evaluation results show the need to unify the existing elements and tools provided by the guidance into a new document. This new guidance should compile the

156

Chapter 5. Evaluation of fire risk assessment methodologies met criteria and provide practitioners with clear, practical examples of the pitfalls of these assessments and the ways in which they can be avoided or managed. Overall, the purpose of producing new guidance is to consolidate the best practices and also to incorporate the dimension of trustworthiness, which goes beyond an accounting of uncertainty.

In the context of this chapter, the discussion of quality becomes relevant as it is conspicuous in its absence within the guidance documents evaluated. Although many stand-alone elements are listed, described and guidance provided for employing them, little to no reference is made to the overall quality of the assessment or its impact on trustworthiness. So far, the inherent limitations of risk assessments have been introduced and those of the PRAs highlighted. However, it seems reasonable to explicitly address the quality of risk assessments in order to understand why it is an essential feature for producing better insight on fire risk performance.

157

Chapter 5. Evaluation of fire risk assessment methodologies

Bibliography 1. (NFPA), N.F.P.A., NFPA-551, Guide for the Evaluation of Fire Risk Assessments. 2019, Quincy, MA, USA. 2. (SFPE), S.o.F.P.E., SFPE engineering guide to application of risk assessment in fire protection design. 2005. 3. (BSI), B.S.I., PD 7974-7:2019 Application of fire safety engineering principles to the design of buildings. Probabilistic risk assessment. 2019, London, UK. 4. (BSI), B.S.I., BS 7974:2019, Application of fire safety engineering principles to the design of buildings - Code of practice. 2019: BSI. 5. (ISO), I.O.f.S., ISO 16732-1 Fire safety engineering. Fire risk assessment. General. 2012. 6. (ISO), I.O.f.S., ISO 23932-1:2018 Fire safety engineering. General principles. 2018. 7. Norway, S., SN-INSTA/TR 951:2019 - Fire Safety Engineering - Guide for Probabilistic Analysis for Verifying Fire Safety Design in Buildings. 2019. 8. (CCPS), C.f.C.P.S., Guidelines for chemical process quantitative risk analysis. 2nd ed. ed. 2000, New York: The Center. 9. Aven, T., Foundations of risk analysis. 2nd ed.. ed. 2012: Hoboken, N.J. : Wiley. 10. Standardization, I.O.f., ISO 31000:2018: Risk management - Principles and guidelines. 2018: Geneva, Switzerland. 11. Beard, A.N., Requirements for acceptable model use. Fire Safety Journal, 2005. 40(5): p. 477-484. 12. Beard, A.N., Risk assessment assumptions. Civil Engineering and Environmental Systems, 2004. 21(1): p. 19-31. 13. Beard, A.N., Fire models and design. Fire Safety Journal, 1997. 28(2): p. 117-138. 14. Manes, M. and D. Rush, A Critical Evaluation of BS PD 7974-7 Structural Fire Response Data Based on USA Fire Statistics. Fire Technology, 2018: p. 1-51. 15. Van Coile, R., et al., The Need for Hierarchies of Acceptance Criteria for Probabilistic Risk Assessments in Fire Engineering. Fire Technology, 2018. 16. Litai, D., D.D. Lanning, and N.C. Rasmussen, The Public Perception of Risk, in The Analysis of Actual Versus Perceived Risks, V.T. Covello, et al., Editors. 1983, Springer US: Boston, MA. p. 213-224. 17. Wolski, A., N.A. Dembsey, and B.J. Meacham, Accommodating perceptions of risk in performance-based building fire safety code development. Fire Safety Journal, 2000. 34(3): p. 297-309.

158

Chapter 5. Evaluation of fire risk assessment methodologies

18. A. Rae, J.M., R. Alexander, The Science and Superstition of Quantitative Risk Assessment, in Proceedings of PSAM 11 & ESREL 2012. 2012, International Association of Probabilistic Safety Assessment and Management, IAPSAM. p. 2292- 2301. 19. Gernay, T., et al., Efficient uncertainty quantification method applied to structural fire engineering computations. Engineering Structures, 2019. 183: p. 1-17. 20. Notarianni, K.A. and G.W. Parry, Uncertainty, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2992-3047. 21. Aven, T., Risk assessment when the objective is accurate risk estimation, in Quantitative Risk Assessment: The Scientific Platform, T. Aven, Editor. 2011, Cambridge University Press: Cambridge. p. 51-75. 22. D. Lange, J.T., C. Maluk, J. Hidalgo, Fire Safety Engineering - Competencies Report, T.W. Centre, Editor. 2019: Sydney, Australia. 23. Woodrow, M., L. Bisby, and J.L. Torero, A nascent educational framework for fire safety engineering. Fire Safety Journal, 2013. 58: p. 180-194. 24. Busby, J.S., R.E. Alcock, and E.J. Hughes. Credibility in risk assessment. in Probabilistic Safety Assessment and Management. 2004. London: Springer London.

159

6 Maximum Allowable Damage

The previous chapters have drawn into existing Fire Safety Engineering (FSE) and safety science literature to identify the advantages and limitations of current fire risk assessment practice. It has been shown that some of these limitations can inhibit effective decision-making and even provide misleading information. Motivated by such limitations and the need for an alternative for practitioners, this chapter introduces a proposed fire risk assessment methodology.

This methodology is the Maximum Allowable Damage (MAD) methodology. Its conceptual basis was partly inspired by the actuarial practice of estimating the maximum loss (EML). This quantity aids in determining whether a property can be insured, as well as in estimating the premium [1]. Typically, EML is only limited to the quantification of economic loss, and it is typically of interest for the insurer of a building [2]. In risk assessments, EML can be set as the upper limit of consequences, helping truncate risk estimates. In chemical process safety, this concept has been applied to truncate F-N curves [3], with N representing economic loss. However, utilizing the potential loss information has not yet been incorporated as a design tool in FSE.

First, the basis for the methodology is discussed in section 6.1. This basis then allows structuring the concept of the methodology in section 6.2. The implementation of the methodology is done through the structured process introduced in section 6.5, in which a case study is used to exemplify it. Following the evaluation done in Chapter 5, MAD is evaluated using the same criteria in section 6.6. To finalize, MAD is further exemplified in two additional case studies in Chapter 7.

6.1 Basis for an alternative methodology

6.1.1 Focus on consequences Data for performing likelihood estimations is unknown a priori for a large range of buildings. Existing data presents important limitations such as underreporting [4] and does not represent the future performance of a building with unique characteristics (section 3.6). The SFPE guidance on risk assessments [5] acknowledges the importance of this limitation by stating that “while scenarios having negligible consequences usually have negligible risk, negligible risk cannot be safely inferred from very low frequency alone. Any scenario, no matter how low its frequency, can have significant risk if its consequence is high enough.”

160

Chapter 6. Maximum Allowable Damage

The previous is consistent with approaches followed in some countries for managing the major accident risks of chemical plants, such as Germany and Korea, where consequence assessments help demonstrating the potential impacts of a major event and the effectiveness of the safety measures in place [6]. Moreover, current PRA guidance for FSE hints at the importance of consequence-driven approaches by setting cut-off thresholds for consequences in the Frequency-Loss diagrams [7, 8] (section 3.5).

As described in Chapter 3, relying on probabilistic analysis to ensure adequate performance is inappropriate unless it is restricted to well-characterized failure modes with adequate supporting data. Therefore, the MAD methodology is formulated to focus on the consequence scale and use it as a measure of performance. The Health & Safety Executive guidance on handling uncertainty aligns with this proposal [9] indicating that as likelihoods are increasingly uncertain, the focus should be on eliminating large consequences that pose societal concerns.

6.1.2 Scenario identification In the context of FSE, hazards identification and the subsequent scenario development is a flawed and poorly structured process, as discussed in detail in section 3.6 and 4.7. After the fire at SkyCity in Auckland, NZ (2019), firefighter Martin Campbell stated: “all of the worst things seemed to come together”. This quote exemplifies that fires which lead to large loss can be the result of unimaginable, yet feasible conditions. They are unimaginable either because they have not occurred yet, or because no one thought of them. This poses a large challenge, which cannot be fully solved through structured techniques of hazard identification and scenario development [10].

Typically, risk assessments use inductive reasoning to develop scenarios (Hazards  General category of scenarios  Specific scenarios  Risk analysis). Since in FSE the scenarios that matter could be an output of the assessment, rather than an input, a deductive reasoning process could be more appropriate (Maximum allowable damage  Damage model  Scenarios  Iteration based on results).

To enable a deductive reasoning process to identify key scenarios, MAD uses an abstract representation that captures known failure modes and related variables. This representation constitutes a model for how the consequences materialize, i.e. a damage model. This damage model provides a structure from which scenarios can be drawn and bounds the problem. The damage model enables exploring variations in failure modes and associated variables to

161

Chapter 6. Maximum Allowable Damage

characterize to characterize the range of potential consequences, i.e. the damage potential. This idea will be expanded in section 6.5.

6.1.3 Representing complex systems No model can capture the ‘true’ performance of a system, as required information and knowledge might not be available a priori or the available knowledge is imperfect [11]. Defining what a model represents is important, as stated by PWC [12]: “‘models are simplified representations of real-word relationships among observed characteristics, values and events’. The key phrase here is ‘simplified representations’. A model is only an approximation of reality. Understanding this is the key to understanding how to safely leverage the power of models.” British statistician George E. P. Box put it in more direct terms: "All models are wrong, but some are useful“.

The damage model introduced in the previous section is at the core of the methodology as it structures the problem and provides the scenarios to characterize the damage potential. Therefore, an alternative to typical PRA approaches needs to acknowledge the knowledge limitations of this model and account for them when interpreting its outputs. This epistemic uncertainty is introduced by simplifications and assumptions, which directly impact the usefulness and credibility it projects [13]. The alternative methodology needs to explicitly acknowledge these sources of epistemic uncertainties to effectively support decision-making, rather than trying to systematically eliminate them, estimate their quantitative effect or simply ignore them.

Graydon and Holloway [14] analyse currently available alternatives to quantify assumptions and limitations in an effort to produce a numerical measure of trustworthiness. Their discussion shows that quantifying trustworthiness can be a pyrrhic exercise, presenting the danger of over precision and potentially resulting in undue trust. Moreover, the discussion emphasizes the responsibility of analysts (e.g. FSE practitioners) with the arguments they make to support the result of the assessment. The work by Busby et al. [15] is consistent with that of Graydon and Holloway, and proposes a framework to speak about credibility in risk assessments, aiming at positively impact their quality and better inform stakeholders.

The MAD methodology would require practical tools to register the sources of epistemic uncertainty and to judge them consciously, in order to provide bounds for results validity. This is a key input for understanding assumption-deviation risks [16, 17] and to provide stakeholders with an insight into the degree of trustworthiness of the results.

162

Chapter 6. Maximum Allowable Damage

6.2 Maximum Allowable Damage (MAD) concept Taking into account the basis for the methodology, MAD operates on a consequence scale rather than on a frequency-loss scale (Figure 6-1). As MAD focuses on consequences to guide the design process, it is of particular interest to characterize the upper end of this consequence scale. However, at this end is where scenario identification tends to be more challenging. The challenge arises from the lack of knowledge of failure modes or their combination with particular conditions that enable a large loss. Even a knowledgeable and experienced practitioner could overlook a particular combination of conditions that results in a large damage potential. Although scenarios across the whole range of consequences can provide useful insight to guide the design process, it is the scenarios at the upper end that could provide the more useful insight into the fire safety performance of the building.

Figure 6-1. Consequence scale

Consider a trash bin fire in an office building resulting in the evacuation of the level, which is manually suppressed by the fire warren of the level where fire originated; this scenario can provide insight to adjust the location of manual extinguishers and to elaborate standard operating procedures for the personnel. This valuable information will not provide insight into alternative scenarios that can lead to major damage, e.g. if the bin fire spreads to its surroundings.

Therefore, MAD focuses on the scenarios at the upper end of the scale, i.e. those that can lead to unacceptable damage. Figure 6-2 illustrates the elements that allow MAD to gain insight into the upper end of the damage potential. The first element is setting acceptance criteria that clearly defines an unacceptable performance, i.e. a Maximum Allowable Damage. These criteria are a function of the safety objectives set out by the construction code (e.g. life safety) or by the stakeholders (e.g. property protection).

163 Chapter 6. Maximum Allowable Damage

Figure 6-2. Graphical representation of MAD

With acceptance criteria, a damage model can be structured to represent how the consequences materialize through specific failure modes and interconnected variables. This is the second element in Figure 6-2 and is followed by drawing values for the variables in the damage model from the set of known conditions. This is the third element of MAD and, as illustrated in Figure 6-2, it is composed of typical, optimistic, challenging and unforeseen conditions. Different combinations of these conditions allow generating initial scenarios that help characterize the damage potential (element 4a in Figure 6-2).

In other words, the damage model is used to identify scenarios, preliminary quantify their damage and use these results to update the damage model and explore alternative – potentially worse- scenarios (element 4b in Figure 6-2). This represents a deductive and iterative process to explore the damage potential and constitutes an important paradigm shift in FSE. This approach recognizes that scenarios cannot be simply specified a priori, as the level of damage they lead to is not necessarily evident without quantifying it. This iterative process is more resource-intensive (mainly in terms of time), but it accounts for the potentially unique features of the built environment. This investment can be of high return if poor-performing scenarios are found and appropriate risk prevention and mitigation measures are put in place early in the design process (see section 6.4).

164

Chapter 6. Maximum Allowable Damage

The epistemic uncertainties in the damage model could prevent from identifying potentially worse performance, which reinforces the need for iterating the damage model in order to explore potentially useful scenarios. However, the purpose of MAD is not to exhaust all the scenarios (as this is impossible). Instead, MAD focuses on gaining valuable insight from the scenarios that it identifies, which are bounded by explicitly stated assumptions. During the course of the assessment some of these assumptions may indicate that the real performance could be worse than the estimated one; in this case it would be useful to quantify how much worse the performance can be if the assumption does not hold. However, in lieu of knowledge, data or models required to quantify it, this potential for worse performance must at the very least be acknowledged in a qualitative manner. By doing so, this potential can be communicated along with the performance assessment. This focuses on improving the risk insight communicated to the stakeholders, rather than on a particular numerical result.

The final element (No. 5) in Figure 6-2 is identifying the maximum damage potential. This is the largest damage produced by the scenarios developed through the damage mode. Once the damage quantification is complete for all scenarios and sensitivities performed, the results can be compared to the acceptance criteria to quantify a safety margin (δ). Where δ≤0, the result is unacceptable, triggering the review and iteration of the design. This process can be repeated as necessary until an acceptable performance is achieved. In this way, MAD is aligned with the need for buildings to approach a safety case scheme, as proposed by Hackitt [18].

MAD’s use of the damage potential and the uncertainties involved in the assessment resonates with an approach applied in the nuclear safety field. This approach is the Quantification of Margins and Uncertainties (QMU) for the US nuclear weapons program. This methodology was initially proposed by Goodwin and Juzaitis [19] and dissected in detail by Pilch et al. [20]. To explain the idea behind QMU, consider a system will have a feasible space of operation, from which a portion is deemed safe, analogous to the consequence scale of Figure 6-. This system has a threshold after which its performance is unacceptable, analogous to the Maximum Allowable Damage or acceptance criteria in Figure 6-2. QMU aims at quantifying the margin between the safe space of operation and the threshold, as well as the uncertainty associated to this quantity; this is analogous to the quantification of the safety margin δ in MAD. This information is meant in both methodologies to better convey when the system is expected to perform adequately and when not, therefore better informing decision-making. Another key element in common between MAD and QMU is the vital role of credible estimations (section

165

Chapter 6. Maximum Allowable Damage

6.5.6) and the warnings made regarding the potential for oversimplification resulting from a PRA/QRA.

In summary, MAD quantifies fire safety performance as a function of fire damage or consequences, focusing on its upper limit, i.e. the maximum damage potential. The next sections introduce two key concepts underpinning the methodology: inherently safer design philosophy (section 6.2.1) and the deterministic approach to risk analysis (section 6.2.2).

6.2.1 Inherent risk and layers of protection The concept MAD is underpinned by the inherently safer design philosophy. The idea of understanding the inherent risk of a system refers to gaining insight on the system’s risk without any of additional safety measures beyond the basic design itself. When the inherent risk results unacceptable, design modifications and layers of protection can be introduced until the residual risk results acceptable. Inherently safer design is typically applied to chemical plants (introduced first by Kletz [21]), where the basic design is constituted by vessels, pipes, main equipment and operating conditions such as flow, temperature and pressure. This basic design can be understood as a bare-bones representation of the system, which only includes the necessary for fulfilling the desired function. The inherent risk stemming from this ‘bare-bones’ design provides insight into its performance and the need for modifications such as elimination or substitution of hazards or the addition of layers of protection. Effectively, this allows iterating the system's design and managing the risk without even considering additional safety measures (Figure 6-3).

166

Chapter 6. Maximum Allowable Damage

Figure 6-3. Risk assessment process applied to inherently safer design

The process in Figure 6-3 starts by analysing the initial design. The safety features of the design should enable hazards elimination, minimization and substitution, as well as simplification4. This initial result can be subjected to a risk assessment and, if the result is acceptable, the risk treatment will then focus on system monitoring to ensure the validity of the assessment’s assumptions over time. However, if the risk is unacceptable, an alternative design is necessary. This alternative initially focuses on adjusting the main design features. If this alternative fails to achieve an acceptable risk, safety measures can be gradually introduced in the design.

Additional safety features act as layers of protection [22], which surround the basic design and provide robustness and redundancy. This is the design philosophy known as inherently safer design [21]. Implementing this philosophy rationalizes the selection of safety measures and uses them either as enhancements or as redundancies. The former aids the initial design to achieve the necessary performance; the latter provides robustness in case a disturbance or

4 Simplification can be a useful way to reduce the interdependencies and therefore of unexpected failure modes.

167

Chapter 6. Maximum Allowable Damage

unforeseen condition takes place and resonates with the ‘design by diversity’ idea described by Downer [23] (see section 3.7).

Figure 6-4. Layers of protection concept in chemical process safety

The same concept can be applied to the design of a building in the context of FSE. The ‘bare- bones’ design (e.g. geometry, compartmentation, etc.) can be used to gain insight on the inherent risk of the building and to adjust the basic architectural or structural design to minimize it, before even considering any additional safety measures such as detection or suppression. This corresponds to a first-principles approach, in contrast to the need to understand the system with a set of pre-required safety measures that might or might not help to achieve the desired performance.

Inherent risk can help guide the basic design and reach an acceptable risk, i.e. acceptable consequences despite the occurrence of a fire. Such a feat can be achieved if the design by itself is capable of containing the fire in the room of origin, or diverting the smoke layer safely to the outside. If the inherent risk results not to be acceptable (on a consequence basis) then it would be appropriate to begin introducing additional safety features to manage it. This is not to be confused with a ‘fatalism’ bias [24] in which safety measures are regarded as ineffective or useless; all the contrary, these measures can be highly effective, but their selection must be the result of understanding the limitations of the design and providing redundancies.

168

Chapter 6. Maximum Allowable Damage

Figure 6-5. Layers of protection concept applied to building design

In inherent safety the design itself acts as a core safety measure, and its performance can be quantified. If the performance results unacceptable, further safety measures can be gradually introduced. As these additional safety measures are susceptible to failure (section 3.7), initially they can be assumed to work promptly and effectively. Later on, these assumptions would need to be tested, resulting in either alternative/more reliable safety measures or a list of building management responsibilities that ensure their adequate performance.

MAD produces insight on the inherent risk using the ‘bare-bones’ design in which only the main constitutive features of the building are taken into account. Additional safety measures are gradually introduced on a need basis. This insight provides support for performing subsequent risk assessments that focus on variability. The potential for the inherently safer design approach in FSE has been exemplified for tunnel design by Ostrem and Sommer [25]; their work highlights the different and proactive approach that this philosophy can bring into FSE, despite not providing a practicable approach to it.

6.2.2 Deterministic risk analysis Quantifying the damage in MAD requires mapping the scenarios into the consequence scale. As discussed in section 3.1, this process can be either deterministic or probabilistic. Paté-Conell [26] recognizes the usefulness of deterministic analyses, but points out that these can be conservative to a degree that cannot be easily compared. Abrahamsson [27], Lundin [28] and Frantzich [29] also recognize this limitation, and argue that deterministic analyses do not provide insight on the likelihood of the scenarios. However, Frantzich [29] recognizes the value of using deterministic values within PRAs for largely uncertain phenomena such as exposure of occupants to toxic gases.

169

Chapter 6. Maximum Allowable Damage

The previous indicates that if deterministic analyses are used, explicit consideration of the level of conservatism is necessary. The MAD methodology employs deterministic analyses to quantify the fire safety performance as a function of fire damage. In the methodology, the degree of conservatism is not necessarily unknown. Any risk assessment will be as conservative as the values and assumptions underlying it are; this means that if such values and assumptions are explicit, these can be ranked (albeit in a qualitative manner). Some of these assumptions might be conservative, but other could result to be the opposite. In the latter, a potential for worse performing results exists, which is seldom acknowledged in criticism towards deterministic analyses and in favour of PRAs.

MAD requires then an explicit description of the selected values and assumptions and a check of how conservative they are. In any risk assessment (using any given methodology) it is a central issue to define what a conservative value or assumption is. The MAD methodology does not aim to provide a solution for this problem, as it is a matter of perspective and subjective judgement. However, what the methodology does encourage is to be as conservative as necessary to gain useful insight. Consider a fire risk assessment for a residential building in which the results seem favourable under the assumption of a ‘normal’ occupancy; given that the performance is sensitive to the occupation, it would be reasonable to explore the result of an abnormal or extreme occupation, as this may reveal a feasible unacceptable performance that must be addressed. Now consider the opposite situation in which the initial performance with a ‘normal’ occupancy results unacceptable; in this case estimating the performance with an extreme occupancy would not yield further useful insight. Such calls are based on the practitioner’s own experience, knowledge and judgement. However, MAD does provide a basis for the practitioner to reflect upon their choices and the level of conservatism they have employed.

Finally, the separation between deterministic and probabilistic analysis seems radical and irrefutable when reading the definitions provided in the literature. However, the line between deterministic and probabilistic analysis is rather blurry. Every deterministic analysis is rooted in some combination of probabilities associated with the scenario, and this information is also valuable for the output of the risk assessment. Furthermore, selecting a conservative value for a key variable, e.g. fuel load, can be analogous to sampling a probability distribution at its tail. The implicit likelihood information in a deterministic analysis is exemplified as follows.

170

Chapter 6. Maximum Allowable Damage

Probabilities in deterministic analyses: Example

Deterministic analyses can provide insight into the likelihood of a scenario or its constituent elements. ‘A fire starts at a cleaning supplies cabinet in an office building at night and goes undetected until smoke reaches a detector’ is a qualitative construction describing different aspects of a scenario, including important and relevant aspects of likelihood. The probability of early detection is zero, as smoke must accumulate and leave the cabinet until reaching a detector. The probability of early extinguishment is zero as no suppression is mentioned. The probability of delayed detection is one, as a detector is assumed to go off. The deterministic nature of an analyses does not force them to be deprived of information about likelihood. Constructed carefully and systematically, deterministic analyses have a lot of information to convey.

‘A fire starts at a cleaning supplies cabinet in an office building at night and goes undetected until spreading and reaching flashover in the larger contiguous compartment’. This scenario is a variation of the previous, and also contains key information about likelihoods. Both of these scenarios can be represented graphically using once again the Probability-Consequence space (Figure 6-6), where the effect of detection can be seen in the probability of the consequence for each scenario. If the detection works with a value of Pdet between 0 and 1, scenario 1 yields a typical output from a PRA. However, if the detection is considered ineffective (e.g. local detector with no connection to the building alarm or to the fire service), scenario 2 reveals a large damage potential. Limiting the analysis to these two scenarios would make scenario 2 the maximum damage potential or MDP, which can in turn be compared to pre-defined values of the maximum allowable damage to evaluate its acceptance. In the conditions of Figure 6-6, the result would be unfavourable (fail) and the design would need to be adjusted or discarded.

171

Chapter 6. Maximum Allowable Damage

Figure 6-6. Consequence-probability plot for the cleaning cabinet fire scenarios

Such a result can be deemed conservative and incomparable. However, it does pose the question of what would happen if the detection system were unavailable or poorly designed. The question can go further and enquire about what ‘poorly designed’ would be in this situation. The particular chemicals stored in the cabinet could burn with a low soot yield and delay the response of a photoelectric detector, which would make ionisation detectors more suitable. Already, relevant knowledge to guide the design process has been gained, even in the absence of probabilistic data. It is unlikely though that in an office building the fire safety performance is defined by a detection system.

6.3 Objective The objective of the MAD methodology is to provide a framework for FSE practitioners to conduct a fire risk assessment that allows demonstrating the fire safety performance of a building design while managing potentially unacceptable consequences. This is done by characterizing the damage potential of a design and comparing its upper limit to the acceptance criteria.

6.4 Scope MAD is intended for early design stages in order to implement an inherently safer design and the gradual approach to risk assessments. The former propends to design a system that can deliver an adequate performance by itself and minimizes the dependence on additional safety measures. The latter addresses the inherent limitations of any risk assessment and acknowledges that a consequence-driver assessment can yield a conservative result.

172

Chapter 6. Maximum Allowable Damage

In the early stages of a building design, the results from MAD enable inherently safer design and intend to serve as the basis for further assessments in which variability and likelihood are the focus. The later in the stage design, the fewer possibilities exist to make changes to the constitutive features of the design and therefore achieve acceptable or lower inherent risks. Furthermore, once the design features have been set the reliance on add-on safety measures increases, as well as the reliance on a robust knowledge base to quantify their failure likelihoods.

6.5 Process The process to implement MAD is presented in Figure 6-7. In this process, the inputs are a set of objectives (e.g. preserving the life of occupants during a fire) and the outputs are a quantified performance measure in a pass/fail format, with an explicit boundary of validity. As the methodology is consequence-oriented, the performance measure is estimated by comparing the maximum damage accepted by the stakeholders and the maximum damage potential of the system, as described in the previous section.

Design modification / Operation rules

Unacceptable 1. Safety 2. Acceptance 3. Damage 4. Estimated 5. Performance objectives threshold model damage assessment Acceptable Assumptions Inputs Limitations Assumptions

Information Registry

Figure 6-7. MAD process

This section specifies the details of each step and the elements involved, notwithstanding that their implementation needs to be adjusted to the nature, magnitude and complexity of each system. For the purpose of clarity, a simple case-study is presented throughout this section to exemplify the implementation of MAD.

Case study

The case study is a 10 storeys office building to be constructed in an urban area of the state of Queensland in Australia. This is a multi-occupancy building, comprising carparks, retail areas and office areas, or classes 5, 6, 7a, respectively, as shown in Figure 6-8. The MAD methodology will be implemented step-by-step in this case study, which was previously introduced in Chapter 3 to illustrate PRAs in the context of FSE.

173

Chapter 6. Maximum Allowable Damage

Figure 6-8. Graphical representation of the building

6.5.1 Safety objectives The first step defines the safety objectives that typically include: 1) ensuring life safety of occupants, 2) reducing direct and indirect losses and 3) providing firefighters with a building that –when burning- will facilitate their operations, as this is their workplace. The Building Construction and Safety Code (NFPA 5000) [30] proposes having goals (“nonspecific overall outcome to be achieved” of qualitative nature) and objectives (a “requirement that needs to be met to achieve a goal”).

In MAD, objectives reflect the desired outcome while also enable defining an acceptance threshold as a function of fire damage, e.g. no exposure of occupants to toxic concentrations. The premise for this step is that the intended purpose of the system is established and adequate objectives are set accordingly. The objectives set the aspects on which the stakeholders seek to gain insight and then take action. This allows narrowing down the scope of the variables involved in defining the damage model. The objectives are defined by the stakeholders as a function of both regulatory requirements and their own objectives.

Case study: Safety goal

The safety goal for the office building risk assessment is achieving evacuation without loss of life in case a fire occurs in the building.

174

Chapter 6. Maximum Allowable Damage

6.5.2 Acceptance criteria Clearly defined criteria are necessary to conduct the performance assessment. As MAD is a quantitative methodology, it requires quantitative acceptance criteria. These criteria are defined by stating how much damage is acceptable in the context of the objectives set. In general, the acceptance criteria will be a function of the type of damage caused by the fire in relation to the performance objective.

In case of life safety, the damage is associated to the occupants, and the acceptance criteria can be a number of fatalities, major injuries (third-degree burns, asphyxia) or minor injuries (first, second-degree burns). For structural integrity, load bearing and non-bearing structural elements can be damaged, as well as compartmentation limiting the extent of the fire. Potential damage to fire services is based on the extent and magnitude of the fire, which can be quantified in terms of the maximum extent of fire area and the heat release rate and the degree of structural damage. Environmental damage can be formulated in terms of the smoke volume produced, its plume footprint and toxicity levels along it or the quantity of fire-fighting water.

The more specific the damage is, the more precision is required from the damage model and its quantification, which can constitute a significant challenge in the presence of important knowledge gaps or when resources (human, monetary, timeframe) are limited. Given sufficient resources, the latter can be managed, but knowledge gaps will only be remediated by investing in additional research, which in the context of FSE might imply large-scale testing.

MAD is defined through socio-technical discussions between the stakeholders and the FSE practitioners in charge of the risk assessment. Once the criteria are defined there should not be any uncertainty associated with it as it is the basis for evaluating the system’s performance. However, the value selected for MAD can reflect the stakeholders’ risk aversion or appetite, which translate directly into more demanding or lenient acceptance criteria.

Case study: Acceptance criteria

Based on the safety goal of achieving evacuation without loss of life in case of a fire, the acceptance criteria is defined as zero (0) casualties.

6.5.3 Damage model To carry out the performance assessment, it is necessary to quantify the damage caused by the fire. The damage model is a conceptual representation of the physical phenomena taking place during the fire and which ultimately lead to damage. It also provides a scope for the analysis

175

Chapter 6. Maximum Allowable Damage

as it should explicitly state what phenomena and relations it includes and which it excludes. The damage model is the evidence of the knowledge limits of the fire safety engineers and of the level of complexity introduced in the performance assessment. A simple damage model with known uncertainty sources can yield better results than extremely complex models with unaccountable uncertainties, and the selection of the model is the sole responsibility of the engineers in charge of the process.

Multiple tools can aid its construction such as fault and event tree analysis, systems dynamics representations, limit state functions, Swiss Cheese or Reason’s model [31], STAM , FRAM, ACCIMap [32] and failure mandalas [33]. Linear models for constructing scenarios such as the Swiss cheese model are easy to construct and communicate, but do not capture all the complexities of a system, the interdependencies of its internal elements and the dependencies with external ones [34, 35]. This limitation is also found in the guidelines available to construct logic trees (e.g. FTA and ETA), in which the first steps involve defining a physical and temporal scope for the analysis [36]. Furthermore, understanding of accidents is constantly evolving as shown by Waterson [37] in which a clear transition from linear to systemic models is presented (Figure 6-9). A relevant review of major accidents causation models is provided in [38], which can provide the reader with a wider set of alternatives to formulate a damage model.

Complexity Sys tems thinking

Sociotechnical levels dynamic Swiss interaction Human cheese (Accimaps) factors Root cause FMEA

1950 1970 1990 2010 Time

Figure 6-9. Evolution of accident modelling

Notice that the mentioned tools do not directly explain the physical behaviour of the fire or its interaction with the system, as they provide a higher level of understanding of the variables involved and the physical and human processes that produce the damage. Relevant examples of damage models available in the FSE literature include Beck’s [39] (Figure 6-10), Beck’s

176

Chapter 6. Maximum Allowable Damage

risk-cost model [40], the Fire Safety Concepts tree by NFPA [41] and other summarized by Johanssen [42], who proposes an adjustment of Meacham’s cost-risk model [43].

Detection Smoke management Fire Smoke spread Untenable development Untenable conditions - thermal conditions - toxic

Compartmentation Detection Risk assessment Occupant Fire spread Conventions Untenable (ERL) communication Occupant response conditions - thermal Safety measure Time for untenable Fire service action Output Fire service Occupant conditions communication avoidance Arrival time Occupant response Submodel

Figure 6-10. Risk assessment model, adapted from Beck [39]

Beck’s model in Figure 6-10 can be seen as the basis of many other damage models in FSE, including the seven sub-systems model proposed within the framework of the BS 7974:2019 [44]. The latter is a particularly useful model as it is by default a collection of sub-models that the FSE practitioner must configure according to the objectives and complexity of the building being studied. Park [45] proposed two qualitative damage models, the Generic Fire Response Model (GFRM) and the Integrated characteristic interaction model (ICIM), which in contrast to Beck’s model, exemplify the high level of complexity that fire introduces. Cadena [46] used systems’ theory to construct a causal loop of a typical compartment fire as shown in Figure 6- 11. This particular representation was useful to construct a computational model of an experimental bench-scale compartment [47], which was later used to compare the results and analyse the uncertainty throughout the two classical fire regimes. This research is presented in Appendix 2.

177

Chapter 6. Maximum Allowable Damage

Figure 6-11. Causal loop of a simple compartment fire [46]

All damage models and the scenarios they capture are conditioned by the causal relations they represent. These causal relations are typically a simplification of complex interdependencies and are susceptible to subjective judgement (see section 3.6.2 for a discussion of causal relations in FSE). Such limitation points out the need to use the assumptions and uncertainties in the damage model to enhance the informing capacity of the fire risk assessment. In line with the thoughts of Young [48], the approach of MAD is to be aware of all assumptions made and then use them either to improve the model or, when this is impractical or unfeasible, to improve the understanding of the results. The first step to manage the uncertainty sources of the damage model is to explicitly recognize and register them. To this end, an information registry is employed as presented in section 6.5.5.

Case study: Damage model

Given the need to ensure the safe evacuation, the MAD criterion is defined based on two quantities: the available safe egress time (ASET) and the required safe egress time (RSET). The damage is defined for each level of the building, i, as in Eq. 1.

= (1) 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝑖𝑖 𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝑖𝑖 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑖𝑖 ASET and RSET are defined for each level, the former as a function of the time to reach untenable conditions in level i and for the latter as a function of the different times that amount to the egress duration. Eq. 2 expresses time to reach untenable conditions, , as the time at which occupants have zero exposure to toxic gases, which in turn is defined𝑡𝑡𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈 in𝑈𝑈 Eq.3 as

178

Chapter 6. Maximum Allowable Damage the time at which the smoke layer reaches a critical height, . This is consistent with the approach proposed by Gwynne and Rosenbaum [49]. 𝐻𝐻𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶

= = ( ) (2)

𝑖𝑖 𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 𝑡𝑡 𝑓𝑓 𝑍𝑍𝑍𝑍𝑍𝑍𝑍𝑍 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 𝑒𝑒𝑒𝑒 𝑡𝑡 𝑡𝑡 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔 (3)

𝑡𝑡𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈 ≈ 𝑡𝑡𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 In order to estimate it is necessary to define a fire growth and development. This is done assuming an alpha𝑡𝑡𝑈𝑈𝑈𝑈𝑈𝑈 𝑈𝑈t𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈-squared𝑈𝑈 model, based on the main characteristics of the fuel within level i: the fire growth rate (α), the heat release rate per unit area (HRRPUA) and the soot yield

(Ysoot) as in Eq. 4. This allows estimating for each level i.

𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈 ( 𝑡𝑡 ) , = , , (4) 𝑡𝑡𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑖𝑖 𝑓𝑓 𝛼𝛼 𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻 𝑌𝑌𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 RSET is defined as in Eq. 5 as a sum of the time for detection, tdet,i, time for notification, tnot,i, pre-movement time, tpre-mov,i, and time for occupants to actually displace and exit, tmov,i. Each one of these times is dependent on different variables and parameters, including the fire detection time (either automatic or manual), whether the compartment is the fire compartment and specific characteristics of occupant movement such as walking speed.

= , + , + , + , (5)

𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑖𝑖 𝑡𝑡𝑑𝑑𝑑𝑑𝑑𝑑 𝑖𝑖 𝑡𝑡𝑛𝑛𝑛𝑛𝑛𝑛 𝑖𝑖 𝑡𝑡𝑝𝑝𝑝𝑝𝑝𝑝−𝑚𝑚𝑚𝑚𝑚𝑚 𝑖𝑖 𝑡𝑡𝑚𝑚𝑚𝑚𝑚𝑚 𝑖𝑖 The value of each variable presented in this damage model, as well as the main assumptions and limitations are explicitly considered and registered in an information registry, which is later on used to judge the trustworthiness of the results of the assessment. In particular, it is worth mentioning the limitations of the ASET/RSET approach of the damage model presented by Babrauskas [50] which are divided in i) limited capacity to reproduce human behaviour, ii) survival bias, iii) it does not provide a ‘margin of safety’ basis for evaluation of alternative designs. These assumptions are recorded in the information registry and addressed as described in section 6.5.5.

Other relevant assumptions at this point concern the safety measures included in the ‘bare- bones’ design of the building. These include the structural integrity, evacuation strategy, the self-closing doors and the detection/alarm system. Given that the assessment focuses on life safety and on the moments following the start of a fire, it is assumed that the scales of ASET and RSET are small enough to ensure that structural integrity remains throughout. The building is set to have a staged evacuation strategy and self-closing doors. The latter applies to the doors

179

Chapter 6. Maximum Allowable Damage

leading to the staircase, and it is relevant to quantify the performance of the system given that these doors do not operate correctly. This means assuming the doors will be open, as this can result from the occupants egress and due to malfunction of the mechanism. Finally, the detection/alarm system is assumed to be in place and fully reliable; this is an important assumption without which the performance would result unacceptable. Therefore, maintaining this system reliable and available becomes a key responsibility of the building management.

6.5.4 Damage estimation With a damage model formulated, it is now necessary to quantify the damage potential using engineering tools. In FSE there is a large range of tools to choose from, ranging from empirical correlations or simple tools as the compartment fire framework, all the way to computer fluid dynamics (CFD) and finite-element analysis (FEA) models. Each tool is meant to address a specific physical phenomenon associated to fire dynamics and the response of the system to them, producing specific outputs. Also, each tool has underlying assumptions and parameters, which should also be incorporated into the information registry for trustworthiness considerations.

The damage estimation process is not linear as identifying adequate inputs and tools with an adequate output precision (adequate can be understood as matching the level of resolution set by the value of the acceptance criteria; see section 6.5.2) can lead to modifications of the damage model or its underlying assumptions. This section discusses the treatment of the inputs, the selection of the engineering tools and finally the damage estimation.

6.5.4.1 Inputs The precision of the inputs as well as their correspondence to the actual system will have a significant impact on the quality, precision and trustworthiness of the results. The large range of input types in a building range from well-known quantities such as heights and distances, to typically underspecified ones such as occupation and egress speed. Classifying the inputs is helpful to understand better those for which epistemic and aleatory uncertainties are critical. A possible classification scheme for inputs is presented in Figure 6-12, which results in three types of inputs: those without a potential to influence the damage – Not relevant, those with a potential to influence it and a fixed value, and those with a range. This is a coarse classification intended to identify those inputs for which careful consideration is needed when selecting a value.

180

Chapter 6. Maximum Allowable Damage

Variable or parameter potentially influencing output

Can it influence the Yes outcomes of the event? Relevant No

Can a single value represent the Yes least conservative Not relevant but conditions? No registered

Justification Assign fixed value Assign range

• Describe its role • Reference sources • List relevant assumptions

Judge trustworthiness

Information registry

Figure 6-12. Input classification flowchart

The scheme of Figure 6-12 consists of three important elements. First, defining the criteria that makes an input relevant for the damage potential. The former must be treated in a case-by-case manner by the competent FSE professional, but it is indeed a critical element. The criteria required to deem an input irrelevant for the damage potential must be based on a sound understanding of fire dynamics, fire safety performance and the safety objectives. The safety objectives define a scope for the variables of interest, and those falling outside of it could be deemed not important after careful consideration. Consider the example of intumescent coatings. If the focus of the study is life safety, the variables associated with the action of intumescent coatings could be deemed not relevant; these would be relevant for the structural stability of the building. However, if the intumescent coating is applied to timber elements, it would still be relevant to consider them relevant as timber can start its pyrolysis process with the degradation of hemicellulose in the 120-300ºC temperature range [51]. Timber degradation (including the one with intumescent coatings) can release combustible gases that can pose a toxic hazard to humans and potentially enable an increased fire growth rate. These specific

181

Chapter 6. Maximum Allowable Damage effects resulting from the presence of timber are relevant to the damage potential for the life safety objective.

The second element is defining whether a single value (point-value) can represent the least conservative condition, which cannot always be defined a priori. The least conservative conditions are explored in order to promote identifying the maximum damage potential. Defining the least conservative condition is not a straightforward problem, as it is context- dependent and might be counterintuitive. An example is the ventilation of the compartment of fire origin. A fire that grows with a continuous fresh air source such as an open window might be deemed less conservative than one where the compartment is completely closed as it will run out of oxygen and quench the flames. However, the latter presents the possibility of a backdraft once a door or window is opened as part of rescue operations. Therefore, the least conservative conditions should be selected based on the safety objective. The third element is specifying the required information according to its classification, which includes a justification for it, a description, the value (or range of values) and any assumptions underlying its selection, as well as the information sources supporting them. This information enables performing the judgement of trustworthiness (section 6.5.6).

6.5.4.2 Models The tools used to perform the damage estimation have to be consistent with the level of detail and precision of the available inputs, the competency of the practitioners and the desired outputs. Selecting advanced computational fluid dynamics to model a fire in a building without detailed information on the fuel load distribution would not yield a more reliable result than a simpler model. The available models in FSE range from simple semi-empirical correlations like Hestestad’s flame height or Thomas’ maximum temperature for the gas layer, to computational fluid dynamic simulations for geometries and fuel load distributions. No engineering tool should be employed in the absence of a competent user who is fully aware of the uncertainties introduced by model parameters.

In MAD, the need for consistency between inputs, models and outputs is provided explicitly in the damage estimation step. Here, the practitioner is called to identify all models used and include their implicit assumptions within the information registry, as well as justifying their selection. As different models yield different results, registering these assumptions allows screening them and prioritizing those requiring additional sensitivity analyses and the use of alternative models to validate the results.

182

Chapter 6. Maximum Allowable Damage

It is of the utmost importance to avoid formulating the risk assessment (and its outputs) as a function of modelling tools available to the analysts. Instead, the tools should be selected to quantify the necessary outputs. The same is applicable to the different levels of resolution or accuracy. The availability of a highly precise or detailed tool should not influence the formulation of the damage model, and should only be used if it is required to quantify the proposed damage model.

6.5.4.3 Estimation The inputs and engineering tools can now be used to produce a damage estimation. Different combination of feasible inputs can lead to different damage estimates. It is the objective in MAD to identify the maximum damage potential, which requires understanding the fire dynamics and their interaction with the occupants, fuel load and with the structure itself. Variables such as horizontal egress speed of occupants can vary in a range of ±40% around 1.5 m/s [52], which can be easily addressed by selecting the lower limit as it would yield a worst- case scenario. However, this speed selecting has an underlying assumption of an adequate state of mind (i.e. calm evacuation) from the occupants that cannot be guaranteed. Sampling the extreme values of these ranges can be informative and can provide a range for the damage potential, but identifying those underlying assumptions that could result in differences of performance of several orders of magnitude is much more challenging. In principle, the construction of the damage model should address these underlying assumptions, and the identification of the scenarios for damage estimation can be made based on simple schemes such as upper-lower boundaries or random sampling. These are described as follows:

• Limit cases: an initial approach based on having the minimum amount of damage assessments is to select those values for which the system should provide upper and lower boundaries of performance. This approach has an underlying assumption of independence and although useful, must be treated carefully when correlations between variables and parameters exist. As the objective of MAD is to base the performance assessment on the maximum damage potential, this is the default approach as it will produce the most onerous yet feasible damage in the system. • Uniform sampling: in case there is no knowledge that favours any particular set of values within the range of possible values, a uniform distribution can be employed. This distribution assigns the same probability for each value being the actual value of the variable. This means that the damage assessment should be done for all values or for as many values as it is practical to assess, with a defined and constant delta between them.

183

Chapter 6. Maximum Allowable Damage

There are variables such as fire service intervention in which a uniform distribution will not adequately represent reality [53]. • Other probability distributions: in case there is knowledge supporting that the variable or parameter can take a set of values with more probability than the rest in the range, a probability distribution could be imposed for the sampling to use. The assignment of these distributions could induce biases in the analysis and should be reported in the information registry with the corresponding justification, strength of knowledge and output sensitivity.

Case study: Damage estimation

To estimate the damage, the classification of Figure 6-12 is carried out on the variables associated with the damage model presented in the previous section. From this, the inputs with a range (to be explored) are the location, fire growth rate (α), heat release rate per unit area

(HRRPUA) and soot yield (Ysoot) for ASET, while the density at the stairs and occupant’s movement speed are taken as least conservative in estimating the RSET.

The model selected to estimate ASET is CFAST (version 7.0.0) [54], a two-zone model that for the simple geometry of the open-plan office building allows estimating . The model is run from a Visual Basic script which allowed automating the sampling of the𝐻𝐻𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 input values for ASET. Uniform sampling is carried out for each one of the three quantities (α, HRRPUA, Ysoot) yielding a total of 405 scenarios. Selecting a compartment of fire origin allows carrying out the simulations and obtain the ASET, which are then compared to RSET in a spreadsheet and processed in MatLab for visualization purposes.

The results are plotted by fixating a HRRPUA value (taken as 400 kW/m2 for this example) and covering the full range for α and Ysoot. This allows obtaining a three dimensional surface where the performance of each level can be visualized, with the ratio ASET/RSET on the vertical axis and the variable inputs in the horizontal ones. The following plots illustrate the results for a fire starting at the Basement-2 level (B2), showcasing the results for the bottom levels B2 and B1 and the top levels O6 and O7 (Figure 6-13).

184

Chapter 6. Maximum Allowable Damage

Figure 6-13. Graphical representation of the building

The orange plane cutting the surface is equivalent to ASET/RSET=1, or the acceptance criterion. Any result above the plain (ASET/RSET > 1) indicates acceptable performance, while under the plain (ASET/RSET ≤ 1) the performance is unacceptable. These plots indicate the possibility of inadequate performance in the first two levels. This is better displayed by using two-dimensional plots as presented in Figure 6-14.

Figure 6-14. ASET/RSET as a function of soot yield and fire growth rate

6.5.5 Performance assessment The performance in MAD is a function of the difference between the acceptance criteria and the maximum damage potential, which is the consequence-oriented basis of the assessment. With this difference being positive, the performances are considered acceptable within the

185

Chapter 6. Maximum Allowable Damage

bounds of the validity of the assessment. These bounds are enquired by analyzing the information registry, which lists the key assumptions and provides insight on the necessary actions for them to remain valid during the lifecycle of the system.

In the opposite case, when the performance assessment is negative, it is useful to understand which are the variables or the assumptions underlying this result. It is possible that an overconservative assumption renders it impossible to achieve adequate performance or that the damage model does not consider a key safety measure or feature. This can be analyzed by using the information registry and the associated trustworthiness information. Producing a prioritized list of assumptions as a function of strength of knowledge or output sensitivity, allow identifying the possible design modifications to obtain a better performance. This approach is consistent with the design for change approach proposed by Bjelland [55] and with the holistic approach to fire safety advocated for by Hackitt [18].

Case study: Performance assessment

Using the results for the damage estimation, the regions for which the inadequate performance exists can be identified. This is done by comparing typical values for fire growth rates and for the soot yield associated with the fuel. Intersecting the surface with the feasible ranges and determine whether an inadequate performance can take place, which would trigger the need for design modification or the inclusion of safety measures. The result is shown in Figure 6-15 for the Basement-2 level.

PMMA Crude oil Furniture (0.022) (0.16) (0.2)

Flammable liquid (0.07)

Exterior fire (0.047) ASET/RSET ≥ 1 Car (0.017) ASET/RSET < 1

HRRPUA = 25 kW/m2

Figure 6-15. Performance assessment for basement-2

The results indicate that a very clean burning fuel (Ysoot < 5%) and a pool fire would lead to ASET/RSET < 1 and therefore to unacceptable performance. It can also be seen that a fire growth rate associated with a car fire is within the area of adequate performance, but a slight

186

Chapter 6. Maximum Allowable Damage increase could yield untenable conditions before all occupants can egress. The previous indicates a favourable result, given that adequate controls are provided to the fuel entering and being maintained in the basement.

6.5.6 Trustworthiness An explicit registry of these assumptions and limitations is required before judging how credible or uncertain the results of the performance assessment are. Such a registry implies an additional load to the analysts in charge of the assessment, but the author considers it is a necessary one. This is evidenced by the need for a clear trace of design decisions, auditability during building operation and accountability in incident investigations. Both Shergold-Weird’s [56] and Hackitt’s [18] enquiries identify such needs. Constructing an information registry with the information presented in Figure 6-12 allows making qualitative judgments that reflect the level of uncertainty in the assessment. Furthermore, systematically addressing each assumption provides a basis to construct a risk treatment plan, in which it is essential to ensure that assumptions remain valid through the life-cycle of the building.

The information registry is meant as a basis for third party reviews and for incident investigation while addressing the issue of assessment credibility [15] or completeness uncertainty [7] in risk assessment. The registry allows to systematically register all quantities involved in the assessment, including the assumptions associated with them. This document does not provide a rigid structure for the information registry, with Appendix 4 providing the information registries for the case study presented in this Chapter and those in Chapter 7. Regardless of the format, the information registry needs to account for the trustworthiness of each quantity. Trustworthiness is defined as a function of the available body of evidence (Strength of Knowledge or SoK) and the sensitivity of the damage to changes in inputs or assumptions (Sensitivity-1). Qualitative criteria are defined for each of the trustworthiness components using a Low/Medium/High scale (Table 6-), although quantitative criteria could also be employed.

Table 6-1. Trustworthiness criteria and number of entries SoK Criteria Sensitivity-1 Criteria Poor theoretical grounds, supporting Theoretical grounds for increased Low references or low consensus between Low damage in case of changes analysts leading to MAD breaches Theoretical grounds for increased Medium Neither high nor low Medium damage in case of changes

187

Chapter 6. Maximum Allowable Damage

SoK Criteria Sensitivity-1 Criteria Recent references, strong and relevant Theoretical grounds indicate an High theoretical grounds and agreement between High increase in damage is not analysts reasonable

Judging trustworthiness of individual assumptions and input values allows prioritizing them based on their impact on performance, while identifying those requiring management actions. This approach adopts the ideas of Aven [16, 17, 57-60] and addresses the issues of accountability in FSE identified by Hackitt [18] and Shergold-Weir [56]. Oberkampt et al. [61] provide a framework specifically centred in characterizing the predictive capability of computational models, which could be implemented to compare fire models. This last option is seen by its authors as a standalone element, not to be incorporated in the risk assessment process and therefore not explored as an option here. Nevertheless, it presents important elements (such as the levels of maturity and the evaluated information attributes) that can enhance the trustworthiness judgment proposed here.

The trustworthiness layer of information effectively constitutes a tool to understand the boundaries of validity of the results obtained in the assessment. An assumption with a low strength of knowledge and large sensitivity, can easily result in an inadequate performance and compromise the safety of the building. By identifying it, it can be treated or designed out. If the assumption cannot be removed, then it becomes the input for constructing a risk treatment plan that ensures it remains valid throughout the life cycle of the building or until it can be effectively managed.

Case study: Trustworthiness

The variable classification step as proposed by Figure 6-12 identified 32 variables involved in the damage estimation. From these, four variables required defining a working range (fire

location, α, HRRPUA, Ysoot). The trustworthiness information regarding these four variables indicates that two of them (fire location and α) can lead to high sensitivity in the damage, while their SoK is rated medium due to the inherent epistemic uncertainty in defining them. The uncertainty associated with these variables is managed through the systematic exploration of the damage potential as described in section 6.5.4.

The complete set of variables has been judged for trustworthiness, based on a detailed description of the variable and the selected values. This information can be communicated through a heat map based on SoK and sensitivity, as shown in Figure 6-16. For most of the

188

Chapter 6. Maximum Allowable Damage variables, there is a high strength of knowledge (21) associated with design parameters and well-characterized fire dynamics parameters. The variables with a medium SoK (9) constitute some of the key elements of the damage estimation, such as the number of open fire doors, the fire location and growth, fuel mass and parameters that define the evacuation.

Figure 6-16. Heat map for the trustworthiness of the assessment

It is of particular interest to understand the possible impact of a higher fire growth rate, which is why the results presented by Nilsson et al. [62] are used to plot the cumulative probability function of α next to the result. Based on the distribution reported by Nilsson et al. for fire growth rates in public buildings, the rate for which an unacceptable performance would result has a probability of less than 10% (Figure 6-17). This seems to be a favourable result, but without specific data pertaining to the carpark in office buildings, it only indicates both an unacceptable result is still feasible.

Figure 6-17. Fire growth rate for inadequate performance and its cumulative probability

189

Chapter 6. Maximum Allowable Damage

The evacuation variables involve the notification and pre-movement times, for which the underlying knowledge is only statistical and hardly applicable to any specific building. The uncertainty associated with these nine variables is managed through onerous assumptions such as taking all fire doors open. Although significantly conservative, the existing knowledge base requires considering these conditions and communicating their possible result.

There are two variables with low SoK. The first one is the density of occupants within the staircase during evacuation. This density effectively modifies RSET and has been characterized to be between 1 and 2 persons/m2 according to the statistics of the SFPE Handbook for Fire Protection Engineering [49, 52]. The estimation in the previous section was done based on the estimated density per level, which is around 0.1 persons/m2. This can lead to an unrealistic result; therefore the analysis is re-run using the upper limit reported by the literature. The result is shown in the figure below, where a decrease is found (see black dotted line in Figure 6-18), which makes the performance more sensitive to higher fire growths. In particular, a car fire would yield a negative performance. This calls for ensuring an evacuation procedure that avoids high density in the stairwell, or addressing a design variable (e.g. natural ventilation) to modify ASET directly.

Figure 6-18. Sensitivity to a higher stairwell density

The second variable with low SoK is the fuel mass. This variable determines the time to burnout and is key for estimating ASET. There is no manner of determining this variable a priori, and even available statistics could incur in a significant deviation from the particular conditions of this building.

190

Chapter 6. Maximum Allowable Damage

As this risk assessment is meant to guide early design, the fuel mass selected is onerous, and is set to allow the fire to continue well beyond the range of RSET. This can yield a very conservative result, but could easily be adjusted once that information on the real fuel load of the building is available.

6.6 Evaluation The MAD methodology has been evaluated to identify its strengths but also its shortcomings. As recognized in previous sections of this work, any risk assessment methodology is a tool and as such, presents limitations. Identifying these limitations and accounting for them in the implementation of the methodology is critical to improving its role in the decision-making process. The 16 specific criteria introduced in Chapter 5 and used to evaluate five existing fire risk assessment guidelines are used to evaluate MAD, which provides consistency and a basis for comparison. The results of the evaluation for MAD are summarized in Figure 6-19. The overall result of this figure shows that MAD does not fully meet seven criteria, from which two are not met at all. First, these unmet criteria will be analysed.

100% 0 80% 1 1 2 4 2 60% MC 0 40% PC 1 1 20% 1 2 1 DC 0% 0 0 0 Design Risk ownership Uncertainty Risk as a Assumptions philosophy function of time

Figure 6-19. Evaluation results – MAD

Concerning design philosophy, the methodology fails to provide detailed guidance to perform hazard identification, although it does provide an important basis for it by requiring the definition of the damage model. Currently, the literature in FSE does not provide for a structured and systematic technique that allows constructing a comprehensive list of fire hazards. Typically, techniques stemming from chemical process safety such as the hazards and operability analysis (HAZOP) are cited, but these do not match the particular characteristics of a building or infrastructure. The take of MAD on hazards is different, as the hazards identification is scoped to those elements with the potential of generating or contributing to the resulting damage. This is done through the construction of the damage model (section

191 Chapter 6. Maximum Allowable Damage

6.5.3). As it is not necessarily exhaustive nor systematic by definition, this criterion is only partially met.

In terms of risk ownership, MAD fails to clearly identify the role of the risk owner in the risk assessment process, and only partially addresses all stakeholders. This is a common unmet criterion throughout all fire risk assessment guidance and calls for the need to further develop MAD so that it explicitly accounts for the role of the risk owner in its process. Nevertheless, the methodology does define the role of the FSE practitioners by setting their responsibilities at each step of the process. Further unmet criteria relate to uncertainty, which seems odd as the methodology has been built to directly address this aspect. The first criterion relates to the guidance related to non-probabilistic alternatives, which is only partially met as the methodology itself is non-probabilistic. However, the methodology does not directly provide guidance on other non-probabilistic approaches. This is mitigated by the wider body of research, which includes a detailed review of non-probabilistic methods, presented in Appendix 2. The second criterion not fully met with regards to uncertainty is the precision requirement. As in other guidance documents, this requirement is formulated implicitly. Section 6.5.6 defines the need to bind the validity of the assessment, which in turn requires understanding the underlying assumptions and their robustness. An implicit aspect of this exercise is verifying that the damage estimation is precise.

Another two criteria are unmet, the first related to data gathering and the second to documentation. Both of these elements are omitted from the guidance provided in this Chapter. Data gathering helps define fire frequencies and the reliability of different safety measures, both of which are handled in a unique manner in MAD. The fire frequency is irrelevant, as MAD focuses on understanding the damage once the fire occurs, not if it occurs. On the other hand, based on section 6.2.1, safety measures are introduced only if the performance is unacceptable, and no further modifications can be done to the main design features. Finally, no specific documentation guidance is provided in MAD beyond the need to provide the information record and the exemplification provided by the three case studies in Chapters 6 and 7.

The unmet criteria highlight the potential future work of the methodology, which will be further addressed in Chapter 9. Another important aspect of the evaluation is the overall results. These indicate that in terms of met criteria, MAD is at the same level as the SFPE Guide, as presented in Figure 6-20. However, this numerical result is not entirely insightful, and only indicates that

192

Chapter 6. Maximum Allowable Damage both methodologies meet the largest number of criteria. The met criteria between the SFPE Guide and MAD are not the same. The differences and similitudes can be better observed from the full table of results presented in Table 4-. Both methodologies coincide in not meeting criterion 4.2 regarding data gathering. The SFPE Guide fully meets the hazards identification criterion (1.2), which could be a potential complementary resource for constructing the damage model in MAD. The same can be said about the treatment of stakeholders, which is much clearer and explicit in the SFPE guide. On the other hand, MAD fully meets the criterion regarding model uncertainty (3.2), which is not met by the SFPE guide. MAD also addresses the key assumptions of the fire risk assessment (5.1), which is only partially met by the SFPE Guide. These differences show the potential of complementary use of the SFPE Guide and MAD.

MC PC DC 16 2 2 14 4 5 7 12 8 5 5 10

8 5 9 6 6 9 9 4 8 6 2 3 3 0 0 NFPA 551 SFPE PD 7974-7 ISO 16732- TR 951 MAD Guide (2019) 1 (2012) (2019)

Figure 6-20. Overall results of the guidance document evaluation

Table 6-2. Detailed results of the evaluation including MAD. F: fully meets the criterion; P: partially meets the criterion; N: does not meet the criterion. PD ISO TR NFPA SFPE Specific criteria 7974-7 16732-1 951 MAD 551 Guide (2019) (2012) (2019) 1.1 Role of risk assessment F F P P N F 1.2 Hazards identification N F N P N P 1.3 Gradual approach to risk assessment N P P N N F 2.1 Stakeholder identification P F N N N P 2.2 Risk owner P P N N N N 3.1 Epistemic uncertainty (1) – General F F F P N F

193

Chapter 6. Maximum Allowable Damage

PD ISO TR NFPA SFPE Specific criteria 7974-7 16732-1 951 MAD 551 Guide (2019) (2012) (2019) Epistemic uncertainty (2) – Model 3.2 P N F N P F uncertainty Epistemic uncertainty (3) – 3.3 P F F P P F Subjectivity 3.4 Data limitations F F P P P F 3.5 Precision requirement N P P N P P 3.6 Non-probabilistic alternatives N P P N P P 4.1 Time dependency assumption F F N N N F 4.2 Data gathering N N P N F N 5.1 Key assumptions P P P P F F 5.2 Documentation F F P P F P 5.3 Peer review F F P P P F

6.7 Conclusions The Maximum Allowable Damage (MAD) methodology was built upon a series of specific requirements, unique to the field of FSE. These include being deterministic, consequence- driven and placing explicit importance on the credibility of the results (i.e. trustworthiness). A key premise of the methodology is that the probability of a fire is certain to be one. This assumption is necessary for a context where the unique features of the built environment and the stakes (e.g. lives, large capital investments, and business continuity) call for designs that can perform adequately in the most adverse circumstances. The features of MAD are meant to produce valuable insight with respect to the fire safety performance, which requires focusing and narrowing in on the maximum damage potential. To do so, the scenarios are formulated as the answer to the question: which set of conditions can lead to a large damage? These scenarios could be the product of known and unknown conditions, which places particular importance in the way these are constructed. In MAD, a few scenarios can provide large insight to guide the design process, which diverges from other methodologies where a large number of scenarios are used to verify compliance but may fail to provide insight on the performance.

Typically, the most onerous results involve unforeseen conditions, which is why MAD takes the stance of assessing the performance without any safety measure except those given by the design itself (e.g. natural ventilation, staircases, and compartmentation). It is a central issue to define how conservative is conservative enough. There is no ‘correct’ value for any given variable, instead MAD is setup in a manner that practitioners are required to explicitly state

194

Chapter 6. Maximum Allowable Damage

their level of conservatism and to systematically evaluate if it is sufficient. It is a matter of professional competency to take a stance on where to draw the line between conservative and ‘too’ conservative. What MAD does provide is a platform to gain insight on the damage potential and a transparent record that evidences how conservative the assessment was.

MAD does not define specific risk acceptance criteria. By default, acceptance criteria in MAD are defined based on fire effects (e.g. smoke concentration and flows, heat fluxes, fire spread rates, size and duration) and their consequences (e.g. human loss, partial or total structural collapse). Accounting for human behaviour is without a doubt one of the key challenges in characterizing the uncertainty of a fire risk assessment and in an ideal application of MAD, human behaviour should be designed out. This answers Research Question 2, which asks about the more important uncertainties in fire risk assessments. In principle, acceptance criteria in MAD can be set in such a way that the fire safety performance results independent of human behaviour. This is exemplified by setting the maximum allowable damage of a fire to zero fire and smoke spread beyond the compartment of origin. Such criteria could remove human behaviour from the fire safety performance, but would require modifying all design parameters to accommodate for it. As presented throughout this thesis, this could result in an impossible feat due to functional and aesthetic requirements.

This Chapter introduced MAD using a simple case study. This case study was a multi- occupancy, high-rise office building. The use of the inherently safer design concept allowed obtaining effective risk treatment actions for a design that initially had failed a PRA (see Chapter 3). The case study provides key evidence to support Claim 1, as the focus on consequences allow to find conditions under which the system’s fire safety performance is adequate. Instead of focusing only on a pass/fail result, MAD is able to provide insight on when the performance can be adequate and what the requirements would be to achieve it. Contrasting with the failed PRA, this insight provides a basis for practicable actions that can ensure an adequate performance or at the bare minimum, identify the conditions under which the performance will result unacceptable. Further evidence for this claim will be presented in Chapter 7 with two additional case studies.

The approach of MAD is novel, compared to other approaches that focus on the risk estimate, i.e. the numerical result. MAD enables holistic fire risk assessments and aims at using the insight on the maximum damage potential to guide the decision-making during the design process. In a way, this approach embodies good engineering practices in the presence of large

195

Chapter 6. Maximum Allowable Damage uncertainties and limited data. Once knowledge and data about failure modes and extreme scenarios reach a high level of maturity in FSE, PRAs could be extremely insightful in guiding the design process. For the moment, the current level of maturity reflected by (catastrophic) past events and the subsequent critiques to the FSE practice, make an approach like MAD essential to adequately support the design process in the built environment.

It is worth noting that the author does not claim that MAD is inherently better than any other methodology, nor that it is not susceptible to conscious or unconscious misuse. The first claim would require third parties performing risk assessments following existing guidelines for other methodologies and the procedure presented here for MAD and then comparing results. The case study presented in this chapter allows to draw some conclusions on that regard, such as the additional insight provided by MAD compared to the pass/fail approach of the probabilistic assessment of the same system presented in Chapter 3. However, it is acknowledged that the author’s participation in this case studies could have introduced biases that can only be removed if third parties conduct similar studies. Similar applications of such studies in chemical process safety have shown the large spread of results from quantitative risk assessments [63]; round- robin studies in fire safety engineering have indicated similar large spreads from the use of fire modelling tools [64]. Notwithstanding the previous reflection, Chapter 7 will introduce two case studies in which the capabilities of MAD are put to test, revealing both advantages and needs for further development.

196

Chapter 6. Maximum Allowable Damage

Bibliography 1. Gustavsson, M., M. Shahriari, and M. Lindgren, Evaluating EML Modeling Tools for Insurance Purposes: A Case Study. International Journal of Chemical Engineering, 2010. 2010. 2. Marchant, E.W., Fire safety systems–interaction and integration. Facilities, 2000. 3. Cavanagh, N. and J. Linn. Process Business Risk A methodology for assessing and mitigating the financial impact of process plant accidents. in ASSE-MEC Seventh Professional Development Conference and Exhibition, Bahrain. 2006. 4. Frank, K., et al., A review of sprinkler system effectiveness studies. Fire Science Reviews, 2013. 2(1): p. 6. 5. (SFPE), S.o.F.P.E., SFPE engineering guide to application of risk assessment in fire protection design. 2005. 6. Kirchsteiger, C., On the use of probabilistic and deterministic methods in risk analysis. Journal of Loss Prevention in the Process Industries, 1999. 12(5): p. 399-419. 7. (BSI), B.S.I., PD 7974-7:2019 Application of fire safety engineering principles to the design of buildings. Probabilistic risk assessment. 2019, London, UK. 8. Norway, S., SN-INSTA/TR 951:2019 - Fire Safety Engineering - Guide for Probabilistic Analysis for Verifying Fire Safety Design in Buildings. 2019. 9. Reducing risks, protecting people - HSE’s decision-making process, H.a.S. Executive, Editor. 2001: Norwich, England. 10. (CCPS), C.f.C.P.S., Guidelines for Hazard Evaluation Procedures. 2010, Hoboken: American Institute of Chemical Engineers. 11. Beard, A.N., Fire models and design. Fire Safety Journal, 1997. 28(2): p. 117-138. 12. (PWC), D. The Role of Model Review, Model Risk Management and Continuous Model Monitoring in the Financial Services Industry. 2014. 13. Beard, A.N., Risk assessment assumptions. Civil Engineering and Environmental Systems, 2004. 21(1): p. 19-31. 14. Graydon, P.J. and C.M. Holloway, An investigation of proposed techniques for quantifying confidence in assurance arguments. Safety Science, 2017. 92: p. 53-65. 15. Busby, J.S., R.E. Alcock, and E.J. Hughes. Credibility in risk assessment. in Probabilistic Safety Assessment and Management. 2004. London: Springer London. 16. Khorsandi, J. and T. Aven, Incorporating Assumption Deviation Risk in Quantitative Risk Assessments: A Semi-Quantitative Approach. Vol. 163. 2017.

197

Chapter 6. Maximum Allowable Damage

17. Berner, C. and R. Flage, Strengthening quantitative risk assessments by systematic treatment of uncertain assumptions. Reliability Engineering & System Safety, 2016. 151: p. 46-59. 18. Hackitt, J., Building a Safer Future, Independent Review of Building Regulations and Fire Safety: Final Report. 2018: London, UK. 19. Goodwin, B.T. and R. Juzaitis, National certification methodology for the nuclear weapons stockpile. 2006, Lawrence Livermore National Lab.(LLNL), Livermore, CA (United States). 20. Pilch, M., T.G. Trucano, and J.C. Helton, Ideas underlying the Quantification of Margins and Uncertainties. Reliability Engineering & System Safety, 2011. 96(9): p. 965-975. 21. Kletz, T.A. and P. Amyotte, Process plants : a handbook for inherently safer design. 2nd ed. ed, ed. ProQuest. 2010, Boca Raton: CRC/Taylor & Francis. 22. (CCPS), C.F.C.P.S., Layer of protection analysis: simplified process risk assessment, ed. D.A. Crowl. 2001, New York: American Institute of Chemical Engineers. 23. Downer, J., When failure is an option: redundancy, reliability and regulation in complex technical systems. LSE Research Online Documents on Economics, 2009. 24. Tancogne-Dejean, M. and P. Laclémence, Fire risk perception and building evacuation by vulnerable persons: Points of view of laypersons, fire victims and experts. Fire Safety Journal, 2016. 80: p. 9-19. 25. Østrem, L. and M. Sommer, Inherent fire safety engineering in complex road tunnels – Learning between industries in safety management. Safety Science, 2021. 134: p. 105062. 26. Paté-Cornell, M.E., Uncertainties in risk analysis: Six levels of treatment. Reliability Engineering & System Safety, 1996. 54(2): p. 95-111. 27. Abrahamsson, M., Uncertainty in Quantitative Risk Analysis – Characterisation and Methods of Treatment in Division of Fire Safety Engineering. 2002, Lund University: Lund, Sweden. 28. Lundin, J., Model Uncertainty in Fire Safety Engineering. 1999, Lund University. 29. Frantzich, H., Uncertainty and Risk Analysis in Fire Safety Engineering, in Department of Fire Safety Engineering and Systems Safety. 1998, Lund University. p. 206. 30. (NFPA), N.F.P.A., NFPA 5000: Building Construction and Safety Code, in Goals and Objectives. 2018: Quincy, MA, USA.

198

Chapter 6. Maximum Allowable Damage

31. Reason, J.T., Human error. 1990, Cambridge [England]; New York: Cambridge University Press. 32. Yousefi, A., M. Rodriguez Hernandez, and V. Lopez Peña, Systemic accident analysis models: A comparison study between AcciMap, FRAM, and STAMP. Process Safety Progress, 2018. 0(0). 33. Hatamura, Y. and K. Lino, Decision-Making in Engineering Design: Theory and Practice, ed. R. Roy and Y. Hatamura. 2006, London: Springer London. 34. Hollnagel, E. Understanding accidents-from root causes to performance variability. in Proceedings of the IEEE 7th Conference on Human Factors and Power Plants. 2002. 35. Larouzee, J. and J.-C. Le Coze, Good and bad reasons: The Swiss cheese model and its critics. Safety Science, 2020. 126: p. 104660. 36. Guidelines for Hazard Evaluation Procedures. 2011: Wiley. 37. Waterson, P., et al., ‘Remixing Rasmussen’: The evolution of Accimaps within systemic accident analysis. Vol. 59. 2016. 38. Yang, X. and S. Haugen, Implications from major accident causation theories to activity-related risk analysis. Safety Science, 2018. 101: p. 121-134. 39. Beck, V., Fire Safety System Design Using Risk Assessment Models: Developments In Australia. Fire Safety Science, 1991. 3: p. 45-59. 40. Beck, V.R. and D. Yung, A Cost-Effective Risk-Assessment Model for Evaluating Fire Safety and Protection in Canadian Apartment Buildings. Journal of Fire Protection Engineering, 1990. 2(3): p. 65-74. 41. (NFPA), N.F.P.A., Guide to the Fire Safety Concepts Tree. 2017: Quincy, MA, USA. 42. Johansson, U.C., Quantifying Risk for Deemed-to-Satisfy Apartment Buildings. 2010, Department of Fire Safety Engineering and Systems Safety, Lund University: Lund, Sweden. 43. Meacham, B.J., et al., Building Fire Risk Analysis, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2941-2991. 44. (BSI), B.S.I., BS 7974:2019, Application of fire safety engineering principles to the design of buildings - Code of practice. 2019: BSI. 45. Park, H., et al., Conceptual Model Development for Holistic Building Fire Safety Performance Analysis. Fire Technology, 2015. 51(1): p. 173-193. 46. Cadena Gómez, J.E. and F. Muñoz Giraldo, Uncertainty analysis of fire simulations through FDS. 2014, Uniandes: Bogotá.

199

Chapter 6. Maximum Allowable Damage

47. Majdalani, A.H., et al., Experimental characterisation of two fully-developed enclosure fire regimes. Fire Safety Journal, 2016. 79: p. 10-19. 48. Young, C., Model Uncertainty and the Crisis in Science. Socius, 2018. 4: p. 2378023117737206. 49. Gwynne, S.M.V. and E.R. Rosenbaum, Employing the Hydraulic Model in Assessing Emergency Movement, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2115-2151. 50. Babrauskas, V., J.M. Fleming, and B. Don Russell, RSET/ASET, a flawed concept for fire safety assessment. Fire and Materials, 2010. 34(7): p. 341-355. 51. Bartlett, A.I., R.M. Hadden, and L.A. Bisby, A Review of Factors Affecting the Burning Behaviour of Wood for Application to Tall Timber Construction. Fire Technology, 2019. 55(1): p. 1-49. 52. Gwynne, S.M.V. and K.E. Boyce, Engineering Data, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2429-2551. 53. Manes, M. and D. Rush, A Critical Evaluation of BS PD 7974-7 Structural Fire Response Data Based on USA Fire Statistics. Fire Technology, 2018: p. 1-51. 54. Richard D. Peacock, P.A.R., Glenn P. Forney, CFAST – Consolidated Model of Fire Growth and Smoke Transport, Volume 2: User’s Guide, U.S.D.o. Commerce, Editor. 2017. 55. Bjelland, H., et al., The Concepts of Safety Level and Safety Margin: Framework for Fire Safety Design of Novel Buildings. Fire Technology, 2015. 51(2): p. 409-441. 56. Peter Shergold, B.W., Building Confidence – Improving the effectiveness of compliance and enforcement systems for the building and construction industry across Australia. 2018, Building Ministers’ Forum (BMF). 57. Flage, R. and T. Aven, Some brief concluding remarks in relation to the discussion with Floris Goerlandt and Genserik Reniers about strength of knowledge (strength of evidence) judgments in semi-quantitative risk analysis. Safety Science, 2018. 108: p. 237. 58. Askeland, T., R. Flage, and T. Aven, Moving beyond probabilities – Strength of knowledge characterisations applied to security. Reliability Engineering & System Safety, 2017. 159: p. 196-205.

200

Chapter 6. Maximum Allowable Damage

59. Goerlandt, F. and G. Reniers, Evidence assessment schemes for semi-quantitative risk analyses: A response to Roger Flage and Terje Aven. Safety Science, 2017. 98: p. 12- 16. 60. Aven, T., Supplementing quantitative risk assessments with a stage addressing the risk understanding of the decision maker. Reliability Engineering & System Safety, 2016. 152: p. 51-57. 61. Oberkampf, W.L., M. Pilch, and T.G. Trucano, Predictive capability maturity model for computational modeling and simulation. 2007, Sandia National Laboratories Albuquerque, NM. 62. M. Nilsson, N.J., P. Van Hees, A New Method for Quantifying Fire Growth Rates Using Statistical and Empirical Data – Applied to Determine the Effect of Arson Fire Safety Science-Proceedings of the Eleventh International Symposium, 2014: p. 517-530. 63. Pasman, H. and G. Reniers, Past, present and future of Quantitative Risk Assessment (QRA) and the incentive it obtained from Land-Use Planning (LUP). Journal of Loss Prevention in the Process Industries, 2014. 28: p. 2-9. 64. Rein, G., et al., Round-robin study of a priori modelling predictions of the Dalmarnock Fire Test One. Fire Safety Journal, 2009. 44(4): p. 590-602.

201

7 Case studies

This chapter presents the implementation of MAD to two different case studies. In the first one, MAD is implemented to a residential building in which life safety is essential, but where uncertainties severely limit the capacity to predict the performance. This case study was essential in understanding the advantages of the methodology in formulating a risk treatment plan, while it also helped uncovering important limitations about the identification of the maximum damage potential. The second case study is focused on the potential loss related to property and business continuity in a chemical storage warehouse, diverging from focus on life safety of the case study introduced in Chapter 6 and the one introduce here in section 7.1. The lessons learnt from the set of three case studies are discussed in Chapter 8.

7.1 Case-study: High-rise residential building with non-compliant façade A high-rise residential building with a layout and façade similar to the Grenfell tower has been selected for this case study. In the case study, the façade of the building was found to be non- compliant due to the flammable hazard it introduces. The aim of implementing MAD to this case study is to understand the damage potential of a fire and propose a remediation strategy for this non-compliant facade. Before introducing the case study itself, it seems necessary to provide context on the topic of façade fires and the large problem they represent across many jurisdictions around the globe.

7.1.1 Context to façade fires Largely, the fire concerns associated with façade stems from their dramatic success in increasing building energy efficiency. Such effects are exemplified by the retrofitting of 44 existing buildings in Copenhagen [1] that led to a reduction of the buildings annual energy consumption of between 31% to 67%. Facade systems are significant and relevant solutions as 36% of global energy demand is associated to building construction and use [2].

However, the implications of using combustible materials in a façade are not addressed extensively in FSE. Existing studies use laboratory-scale flame spread to assess tenability in rooms over the fire of origin [3], while others qualitatively describe the complex behaviour of the burning façade noting the effect that elements like sealants and tapes have on the overall behaviour [4]. In 1990, Oleszkiewicz [5] described the complexities of evaluating flame spread in façade systems, claiming that a full-scale approach is the most reasonable and that escalation from laboratory-scale is not linear. Current design methods [6] and standardized large-scale

202 Chapter 7. Case studies

testing [7] evaluate façades without taking into account key variables like wind loads, installation defects, complex geometries and other factors directly affecting flame spread. Studies exist on particular façade issues like the effect of the insulation layer thickness and [8] but do not provide an overall understanding or measurement of flammability at large-scale. Bonner and Rein [9] point out the usefulness of an index that reflects façade materials flammability, while also highlighting that this is not attainable under current testing protocols.

Considering recent fire events in Australia, Qatar, England, Scotland, China and United Arab Emirates [3, 10] that led to significant human, economic and legal consequences, these systems constitute a major challenge for the built environment, for FSE practitioners and for the FSE discipline itself. Consider residential buildings where combustible materials have been used in facades at a large scale. In Australia there are reports of expected 2000 affected buildings in New South Wales, while in Victoria about 800 privately owned (>400 deemed as ‘high risk’ [11]) and 400 government owned buildings have been identified [12, 13]. In England the situation is similar, where 155 high-rise residential and public buildings have already been remediated and more than 360 residential buildings remain to be treated (about half of these belong to the social housing sector) [14]. Noticeably, the Grenfell fire embodied the damage potential of a fire involving combustible cladding in a building with a single staircase, an intricate smoke extraction system and a stay-put evacuation strategy. Fu [15] discusses how a compliant building was stuck in time and was not updated to incorporate safety measures that could have helped providing a better performance during a fire. However, design decisions such as the staircase number or key components to compartmentation and redundancies are hard to update and typically will not be justified solely by an economic assessment.

7.1.2 Building description The building is a residential building of 20 levels comprised of a concrete frame and a single core containing the only emergency staircase. Each residential level (levels 1 to 20) has four large and two small flats and a connection to the lift lobby area as shown in Figure 7-1. In the lobby area of each level there is access to the elevators (not suitable for evacuation purposes) and to the emergency staircase, which is the sole evacuation path of the building. The building has an occupancy that can range between 494 people (normal occupancy) and 950 (maximum expected occupancy).

203

Chapter 7. Case studies

22.4 m

Lobby B1 B2 L 1 2 Door W [m] 0.76-1.7 H [m] 2 S Large flat W [m] 1.6 t B K Window H [m] 1.33 S t L E: Elevator (x2) S S: Staircase 21.3 m 6 E t 3 S K St: Store Small flat B B1 B1: Bedroom 1 B2: Bedroom 2 L: Living 5 4 K: Kitchen B: Bathroom

Figure 7-1. Floorplan for a typical residential level; numbers correspond to each flat

The existing façade system currently achieves a ten-fold reduction of the U-value of the building and significantly increases energy efficiency; at the time this was one of the main drivers for the design of the system. The materials chosen for the façade are Polyisocyanurate (PIR) for thermal insulation (100-160 mm thickness) and a 4 mm thick sandwich panel of aluminum layers with a 3 mm thick polyethylene (PE) core, as shown in Figure 7-2.

Insulation Air Wall Gap

Metal composite panels • Copper, zinc, aluminium (ACP) • 2-6 mm thick • Core material - Polyethylene (PE) - Fire retardant (FR, 30% polymer) - A2 (< 10% polymer) - Honeycomb (No polymer core) Metallic structure

Figure 7-2. Schematic representation of a façade system

7.1.3 Safety objective One of the purposes of the building is to provide safe living quarters for the occupants, and installing the façade system introduces hazards that may jeopardize it. Given the combustible nature of the façade system materials (e.g. PE), there is a potential for an internal compartment

204

Chapter 7. Case studies

fire spreading to the building’s exterior and affecting the current evacuation plan and overall fire safety strategy. Therefore, the safety objective selected for the risk assessment is ensuring life safety of the occupants when a fire occurs. Structural integrity and fire-service intervention considerations are beyond the scope of this assessment.

7.1.4 Acceptance criterion The stakeholders’ acceptance criterion for life safety has been set in qualitative terms: with the exception of the flat of origin, the occupants will not be in contact with smoke or fire until evacuation is completed or the fire is fully extinguished. Other acceptance criteria could be envisioned but this one was chosen for clarity, simplicity and because it meets the intent of life safety of almost all building codes. This implies that occupants will have enough time to evacuate before being in contact with smoke or fire. The damage quantification then calls to estimate the available safe egress time (ASET) and the required safe egress time (RSET). ASET refers to the time that occupants have before the conditions in the building are untenable, while RSET refers to the actual time that they need to egress. When the ratio of ASET to RSET is greater than 1 the performance is unacceptable as occupants will be exposed to untenable conditions.

7.1.5 Damage model ASET/RSET is made of quantities that reflect the damage potential. Defining each quantity is a complex problem on its own and the approach has recognized limitations [16]. Bjelland [17] discusses the relevance of this ratio in FSE as well as highlighting that currently there is no standard way of modelling it. Modelling the damage require proposing a model including the involved phenomena and the associated variables. The elements of the fire safety strategy considered for this particular case-study are those highlighted in Figure 7-3.

205

Chapter 7. Case studies

Figure 7-3. Safety measures in FSE highlighting those included in this case study

The proposed damage model results in the flow diagram presented in Figure 7-4, which provides an understanding of how ASET and RSET are estimated for each location within the building. The assumptions and limitations associated to this damage model are established in the model and are identified with the reference marker A#, where # refers to the number of assumptions, e.g. A7. These are collated in Table 7-7. The impact of these assumptions on the fire safety strategy are discussed in section 7.1.8.

Windows breakage Façade ignition and Time to expose a and external flaming external fire spread new compartment

Compartment fire Fire spread, smoke ASET for Location i starts layer descent compartment

Lobby smoke filling ASET for lobby

Smoke enters ASET for staircase staircase & exit Next ASET/RSET l ocation RSET for compartment

Evacuation Egress Occupants’ Fire detection RSET for lobby strategy paths behaviour

RSET for staircase & exit Figure 7-4. Damage model for the case study

206

Chapter 7. Case studies

ASET describes the time to untenable conditions defined by the smoke layer height (2 meters; A0). Tenability is assessed for (i) the compartment where the fire starts (location i), (ii) the contiguous lobby and (iii) staircase. Tenability is a function of several variables, including fire growth, compartmentation and safety barriers. Being consistent with the acceptance threshold (section 7.1.4), tenability is defined based on the time occupants have before being in contact with the smoke. The required assumptions (A1-A12) supporting this damage model are described as follows:

• A1 - Given the significant uncertainty associated to defining the fire growth due to fuel load and distribution variability, fire growth is assumed to behave as an alpha t-squared fire. Furthermore, as fuel load is unknown and impossible to fully control during the building operation, an onerous condition is selected. First, the fuel selected is polyurethane foam, typical of residential upholstery. Second, the area covered by the fuel is the total area of the compartment selected for the fire to start. Third, the fuel density is fixed at 26 kg/m2, which is typical for a residential setting [10]. • A2 - Compartmentation is the physical ability to stop smoke and fire spread, which can be broken due to lack of physical barriers or their failure due to occupants’ behavior or material properties. Based on the previous and following the MAD rationale, compartment doors are assumed open, as well as fire safety doors leading to the emergency staircases. • A3 - Compartmentation is also relevant for the involvement of the façade. Window breakage is defined by the difference between the temperature of the smoke layer and that of the window in the unexposed side. Keski-Rahkonen [18] suggests that breakage is possible with differences larger than 100 K. A conservative breakage criterion of 80°C on the exposed side is selected, given that the smoke layer covers the windows fully. • A4 - Broken windows enable external flaming, as the compartment is ventilation controlled (opening factors for the living room, bedrooms and kitchen range between 19.5 m-½ and 23.8 m-½). External flaming is assumed to begin immediately after window breakage, given that the criteria for flashover is achieved (heat flux of 20 kW/m2 on the floor of the compartment). • A5 - External flaming will lead to an impinging heat flux on the façade, which if above a critical heat flux will cause the ignition of the combustible materials present in the facade. This critical heat flux is set at 18 kW/m2, taking into account the particular materials of the proposed façade [10]. The delay between flame impingement and ignition given a heat flux higher than the critical one is assumed to be of zero seconds.

207

Chapter 7. Case studies

• A6 - Given that the façade has ignited, the damage model assumes that only upwards propagation will occur and that the flame spread rate will be similar to those recorded in real events, e.g. 4 m/min [10]. • A7 - No wind effect is considered, besides the consideration of using real fire spread rates (A6). Although wind can significantly increase damage by helping flame spread horizontally, current knowledge limits the possibility of modelling its potential contribution to vertical and horizontal flame spread. • A8 - As flame spreads up the façade, re-entry is expected to occur once the spread reaches the next set of windows and cause their failure. Experimental setups of ACM cladding [19] resulting in a temperature range of 296-915˚C above the window sill (2.5 meters) and a temperature of 526˚C without flame spread barriers (most similar to this case-study). Based on A3 and the previous experimental results, it is assumed that the time delay between the flames reaching the windows and the fire re-entry is zero seconds. • A9 - RSET is a function of the detection time, notification time, pre-movement time, the time required by the occupants to reach a safe place and the evacuation strategy, i.e. evacuation order. Detection is modelled based on smoke obscuration, with a criterion of 24 %/m. • A10 - Notification is assumed to occur in 30 seconds after detection, while pre-movement time can vary largely (A9). • A11 - Modelling occupants’ behavior is done assuming no erratic/panic behavior and a homogenous demography for the occupants. • A12 - The evacuation strategy is staged, with the initial stage (level where fire originates and levels above and below) evacuating first, followed by the upper and lower stages (see Figure 7-5).

208

Chapter 7. Case studies

Upper stage Evacuate after initial stage

Initial stage First to evacuate STAIRCASE Lower stage Evacuate last

EXIT

Figure 7-5. Staged evacuation scheme

This model enables the estimation of ASET and RSET for location i, as well as for contiguous lobbies and staircase sections. If the initial compartment fire triggers external flame spread, the model restarts at locations i + 1. This enables estimating ASET/RSET for the whole building and assess its overall performance. The quantification of this model requires inputs and engineering tools that are described in detail in the next section.

7.1.6 Damage estimation Some quantities that directly feed the damage model have been already defined, for example, the use of critical heat fluxes for façade ignition or its vertical flame spread rate. To quantify the whole model, the following set of tools are selected: a. CFAST v7 [20] to model the compartment fire and model tenability b. Abecassis-Empis [21] model to estimate heat flux from the fire to the façade c. Smoke detector model embedded in CFAST v7 [20] d. Hydraulic model for modelling the occupants evacuation [22, 23]

From the selected tools, a and b allow estimating the fire dynamics while c and d address the detection and evacuation. Assuming the detection will behave exactly as in CFAST is not realistic, as some very fast fire could yield detection times of 0 seconds. Given uncertainties associated to the detection model implemented in CFAST, a minimum detection time of 50 seconds has been established (A9). This value of 50 seconds corresponds to the maximum detection delay presented in the validation data for the detection model implemented in CFAST [57]. The façade ignition is modelled using the critical heat flux for the façade materials and comparing them with the heat flux generated from the fully developed fire using the Abecassis- Empis correlations. The latter yields an estimated heat flux of 66 kW/m2 impinging on the

209

Chapter 7. Case studies bottom of the façade (closest distance to the window opening), which based on known critical heat flux [10] ensures ignition. The criteria used for ASET calculation, including smoke layer height, toxic species concentration, critical heat flux for the façade ignition and window breakage are presented in Table 7-1.

Table 7-1. Representative quantities associated to the ASET calculation Value / Quantity Units Justification Range Heat release rate per unit area The reported value for PU foam tests [24] is within this 400 kW/m2 (HRRPUA) range, associated to residential values [25] Value associated to an expected fire load of 26 kg/m2 and Fuel load for all flats 650 MJ/m2 the ideal heat of combustion of PU foam of 25 MJ/kg, within the bounds presented by [25] [0.0029, Fire growth rate kW/s2 Bounding limits on t-squared fire growth 0.1876] This value is computed as the product of the HRRPUA Peak heat release rate Variable MW and the total area of the compartment Kitchen, These locations have direct contact with windows, Living Location of initial fire - yielding the shortest times for the external fire spread to room, begin Bedroom [Open, Door status - Bounding limits on ventilation Closed] This value corresponds to the rate registered in the Upwards fire spread rate 4 m/min Grenfell tower fire [10]

Table 7-2. Tenability criteria for ASET calculation Value / Quantity Units Justification Range Compartment and lobby At this height occupants can start inhaling the toxic gases tenability: Smoke layer 2 m of the hot gas layer [26] height Staircase tenability: HCN 150, ppm Criteria based on Purser [26, 27] and CO concentration 7000 Critical heat flux for façade 18 kW/m2 Criterion based on Torero [10] ignition Window breakage criteria: Typical commercial glass fails at around this temperature; upper layer/external flames 80 ˚C furthermore uPVC (used in the window frame) loses 80% temperature of its stiffness by this temperature value [10, 28]

210 Chapter 7. Case studies

RSET was calculated using the quantities for detection, notification, and displacement and queuing within the level. The time for displacement in the staircase and until the exit is estimated for each level. The associated values and ranges for the application of the hydraulic model are presented in Table 7-3, with the rest of the parameters of the queuing model taking values as reported in [22].

Table 7-3. Representative quantities associated to the RSET calculation Representative Value / Units Justification quantity Range This value represents the upper limit of occupation, Occupants 26 People/level equivalent to three occupants in each small flat and 5 occupants in the large ones Detection criterion 24 %/m NIST CFAST Technical guide [29] This value is unknown and could be expected to be at least Notification time [30, 600] s 30 seconds [22]. An onerous upper limit is selected (10 minutes) Expected to be at least 30 seconds for the room of fire origin Pre-movement time [30, 60] s and 60 seconds for other rooms [22] Horizontal distance 20.2 m Maximum distance from a flat door to the staircase door Stairs path from floor to floor taking into account the steps; Vertical distance 3.61 m distance between floor is 3 m This velocity is used as the basis of the estimation of the Walking speed occupants speed within the hydraulic model [22] and is 1.4 m/s (horizontal, vertical) based on statistical information of occupants egress speed [23] Occupants density in Purser [30] reports that a density of 2.1 people/m2 yields [1, 2] People/m2 staircase no movement in stairs

In total, 66 representative quantities were used as part of the damage estimation; the trustworthiness information for each one of these quantities can be found in Appendix 4. In the context of a traditional quantitative or probabilistic risk assessment, these variables define a range of possible ‘scenarios’ which could yield different performances (see quantities in Table 7- and Table 7-3). In this particular application, these quantities are a direct result of the damage model proposed in section 7.1.5, and could have been different if it was constructed using an event tree analysis or other appropriate tool.

211 Chapter 7. Case studies

7.1.7 Performance assessment

Using the values presented in Table 7- and Table 7-3, a wide range of conditions or ‘scenarios’ are possible for which the performance is assessed. Some of the inputs to the damage estimation may paint an extremely conservative approach. However, major events experience demonstrates that all these ‘conservative’ conditions are feasible and unfortunately, recurrent. The performance assessment presented in this section focus on discrete scenarios aimed at identifying the upper limit of the damage potential, i.e. the maximum damage potential (MDP). Exploring alternative scenarios, e.g. effective fire doors, yields a more complete picture of the damage potential as exemplified by Cadena [31].

7.1.7.1 Maximum Damage Potential

In this case-study the focus is first put on identifying the MDP and then exploring the damage potential through additional (presumably less conservative) scenarios. To identify the MDP the bold quantities highlighted in Table 7- and Table 7-3 are used to quantify the damage model presented in section 7.1.5, corresponding to an ultra-fast fire and a vertical flame spread rate of 4 m/min. The results for the flat of fire origin result in an ASET/RSET ratio of zero for a fire starting in the kitchen (ASET = 0 s, RSET = 80 s) and of 0.12 starting at the living room or at the bedroom (ASET = 10 s, RSET = 80 s). These results are independent of the level at which the fire starts. The overall performance of the building depends on the level on which the fire originates.

The results for the lobby and the staircase consider fire origin at level 1, 4, 10 and 15 and a fire starting at the kitchen (lowest ASET with 90 s) are presented in Figure 7-6. For fires starting at these levels the resulting evacuation time using the staged evacuation ranges between 64 and 67 minutes (~1 hour), with the longest times for fires closer to the top of the building. Although the variation is not large in the evacuation time, Figure 7-6 shows the significant impact on the ASET/RSET due to the involvement of the façade and the resulting fire re-entry.

212 Chapter 7. Case studies

Fire - 1 Fire - 4 Fire - 10 Fire - 15 3.5 3 2.5 2 1.5 ASET/RSET 1 0.5 0 1 2 3 4 5 6 7 8 9 10Level11 12 13 14 15 16 17 18 19 20

Figure 7-6. Maximum Damage Potential as a function of ASET/RSET for the lobby at each level – fire starting at the 1st, 4th, 10th and 15th level

7.1.7.2 Inherently safer solution The results of the previous section are based on the mean vertical flame spread registered in the Grenfell tower fire (4 m/min). Although a maximum rate of 6 m/min was registered, an average rate of 4 m/min produces an unacceptable performance and is used to formulate a remediation strategy. This strategy would need to ensure a maximum allowable flame spread as a result of the involvement of the façade, assuming no other variable can be modified.

The maximum allowable flame spread was obtained by iteration, modifying the results presented in Figure 7-6 until only the level of origin has an ASET/RSET<1. The results are presented in Figure 7-7 and indicate that a rate of 0.1 m/min should be ensured if the façade is involved in the fire in order to yield an acceptable result. Such a result can be ensured by using a non-combustible façade, as ensuring such a low rate would imply experimental and analysis uncertainties that cannot be managed [5].

0.4 0.35 0.3 0.25 0.2 0.15

Spread [m/min] 0.1 0.05

Maximum Allowable Flame 0 1 3 5 7 9 11 13 15 17 Level of fire origin

Figure 7-7. Maximum allowable upward flame spread

213 Chapter 7. Case studies

7.1.7.3 Damage potential In order to better characterize the damage potential and to confirm the MDP presented in section 7.1.7.1, this section explores different scenario configurations based on the variables and values of Table 7- and Table 7-3. The performance for the flat where fire originates and the corresponding level is summarized in Table 7-4. Although slow growth fires result in an ASET>RSET for the rooms of origin, the ASET for the flat corridor is less than the RSET for the lobby and therefore in unacceptable performance. In the lobby, a slow growth fire performance is acceptable, while ultra-fast fires yield unacceptable performance (Table 7-4). Results indicate that even a slow growing fire provides a potential for unacceptable performance.

Table 7-4. ASET/RSET ratio for the flat and level of fire origin Fire growth Room of origin ASET/RSET (flat) ASET/RSET (level) Kitchen 0.8 1.5 Slow Living 0.8 1.5 Bedroom 0.8 1.6 Kitchen 0.2 0.6 Ultra-fast Living 0.2 0.6 Bedroom 0.2 0.9

At the building level the scenarios evaluated consider effective compartmentation barriers (flat and staircase doors), which yield lobbies free of smoke. However, due to the vertical fire spread, flats above the flat of fire origin will be affected. Figure 7-8 presents the results for the performance at each level, displaying unacceptable performance for all scenarios except for the top levels (18th, 19th and 20th). Although these results seem worse than the MDP presented in section 7.1.7.1, in these scenarios the fire is contained at the level of origin and where it re- enters the building. Here, the ASET/RSET quantity fails to fully capture the performance of the building, despite providing insight on the evolution of fire spread through it.

214

Chapter 7. Case studies

Fire - 1 Fire - 4 Fire - 10 Fire - 15 3 2.5 2 1.5

ASET/RSET 1 0.5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Level

3 2.5 2 1.5

ASET/RSET 1 0.5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Level

Figure 7-8. Damage as a function of ASET/RSET for the flats above the level of fire origin – fire starting at 1st, 4th, 10th and 15th level. Slow fire growth above, ultra-fast fire below.

7.1.8 Identifying remediation actions A remediation strategy is required as the damage potential and the MDP indicate an unacceptable performance. The inherently safe option has been identified in section 7.1.7.2 and to provide further alternatives the information registry is explored. The registry contains the variables and assumptions that affect performance and that can be used to formulate actions to improve it.

The information registry is meant as a basis for third party reviews and for incident investigation while addressing the issue of assessment credibility [32] or completeness uncertainty [33] in risk assessment. In the context of this case study this issue is referred to as trustworthiness and it is defined as a function of the available body of evidence (Strength of Knowledge or SoK) and the sensitivity of the damage to changes in inputs or assumptions (Sensitivity-1). Qualitative criteria are defined for each of the trustworthiness components using a Low/Medium/High scale (Table 7-5). Judging trustworthiness of individual assumptions and input values allows prioritizing them based on their impact on performance, while identifying those requiring management actions. This approach adopts the ideas of Aven [34-39] and

215 Chapter 7. Case studies addresses the issues of accountability in FSE identified by Hackitt [40] and Shergold-Weir [41].

Table 7-5. Trustworthiness criteria and number of entries SoK Criteria Sensitivity-1 Criteria Poor theoretical grounds, supporting Theoretical grounds for increased Low references or low consensus between Low damage in case of changes analysts leading to MAD breaches Theoretical grounds for increased Medium Neither high nor low Medium damage in case of changes Recent references, strong and relevant Theoretical grounds indicate an High theoretical grounds and agreement between High increase in damage is not analysts reasonable

The information registry contains 49 quantities employed in the damage estimation, with the distribution of SoK and Insensitivity presented in Table 7-6. The six quantities with low strength of knowledge pose the potential for the results of the assessment not to be trustworthy, while the 16 quantities with high output sensitivity could lead to different performances.

Table 7-6. Number of entries for each level of SoK and OS SoK No. of entries Insensitivity No. of entries Low 6 Low 16 Medium 7 Medium 12 High 28 High 13

Each key assumption is associated to several of the quantities involved in the assessment. As each quantity has a SoK and Precision classification, this allows identifying the assumptions with associated low SoK and low Precision. These are the results with the lowest certainty and with the greatest impact on the outputs. This is presented in Table 7-7, highlighting those assumptions with a significant potential for improving or worsening the performance. This allows identifying the assumptions with the potential for a better performance (fire growth, compartmentation, ignition of façade and the fire re-entry criterion) and the those that could lead to worse outcomes (external flame spread; wind effect; notification and pre-movement times; and ordered evacuation).

In general, evacuation strategies can be optimized to improve the performance through the implementation of reliable and sophisticated notification systems through the building [42]. On the other hand, a worse performance could result from issues in the evacuation management (e.g. confusing orders, miscommunication) resulting in increased notification and pre-

216 Chapter 7. Case studies movement times, as well as in disorderly behaviour from the occupants, e.g. oversaturation of staircase, increased que time. For the case study analysed here there is a minimum margin for optimization of the strategy, which combined with the assumption of a calm and orderly evacuation (A12) do not justify exploring it as a remediation action.

Table 7-7. Key assumptions and trustworthiness Sensitivity- Key assumption SoK Discussion 1 An alternative criterion such as smoke layer temperature [26] at 200˚C is verified using CFAST output, indicating that A0 Tenability criteria H H smoke layer height yields a more onerous result (Figure 7- 9). Onerous fuel conditions were chosen assuming polyurethane as the fuel. This is consistent with fuel loads in residential settings [43]. An alternative fire model (B-Risk from A1 Fire growth H L BRANZ [44]) was used to verify the original simulations, showing consistency in temperatures within the compartment of origin and times for untenable conditions. Past events reflect that loss of compartmentation is feasible. Unless specific evidence exist for considering A2 Compartmentation H L compartmentation is maintained during the fire, this scenario is valid for assessing the performance. B-Risk model was used to estimate the temperature at which the glass breaks [45], yielding a time of 76 seconds, A3 Window breakage H H corresponding to an upper layer temperature of 75˚C (Figure 7-10). Based on the simulation results, all configurations achieve flashover in the compartment of origin, which are ventilation-controlled fires. This justifies the assumption of A4 External flaming H M external flaming. A delay on external flaming could be incorporated, but it would not reflect current knowledge nor represent an onerous scenario. An optimistic estimation for this complex phenomenon would result in increased ASET. Under current knowledge A5 Ignition of façade M M and available resources, such modelling would not be accurate nor reliable. A zero seconds’ delay is recognized as an onerous but valid condition for the damage potential. External flame Despite the large variability of the upwards flame spread A6 H M spread (upwards) rate, the building of this case-study is similar to the Grenfell

217

Chapter 7. Case studies

Sensitivity- Key assumption SoK Discussion 1 tower, for which the rate of 4 m/min was the mean value during the fire [10]. Real fires have shown that downwards vertical spread is External flame possible and actively contributes to fire spread and to A6 spread L L increase fire damage. Modelling downward spread could (downwards) lead to re-entry and to faster untenable conditions at the stairs, i.e. worse performance. Wind can influence external flame spread, reduce vertical flame spread and promote horizontal flame spread [21]. Current knowledge for assessing wind effects on external A7 Wind effect L M fire spread is limited, but its potential for a worse performance is recognized [46]. A theoretical model has been proposed by Bai et al. [47], although its applicability is limited to reduced scale setups with HRR < 18 kW. Gandhi et al. [4] performed a façade resistance test using similar ACP configurations to the ones in the case-study, Fire re-entry resulting in a 67 seconds delay between window exposure A8 L M criterion and fire re-entry. Including such a delay once windows are exposed to flaming would yield a better performance, but ensuring it with current available evidence is not supported. The 24 %/m obscuration criterion is given by default in the Smoke detection CFAST detection model, however it corresponds to a A9 H H criterion reasonable setting based on the analysis by Schifiliti et al. [48]. These quantities depend on non-observable variables such as Notification and the state of mind of occupants. The assessment used the least A10 pre-movement L L onerous values and yielded an unacceptable performance; times increasing these times would yield even worse performances. Assuming a homogenous demography (adults around 40 Ordered behaviour years old) is a simplification. Panic effects or physical A11 L L during evacuation difficulties of particular occupants could significantly increase the RSET and therefore yield worse performances.

218

Chapter 7. Case studies

500 2.5

400 2 Gas layer C]

˚ temperature 300 1.5 Tenability temperature 200 1

Height [m] Smoke layer height Temperature [ 100 0.5 Tenability height 0 0 0 10 20 30 40 50 Time [s]

Figure 7-9. Comparison of tenability criteria for ultra-fast fire originating at the kitchen

Figure 7-10. Time for window breakage using B-Risk model, plotted against the temperature of upper and lower layers in the kitchen compartment; red line denotes window breakage

The unacceptable performance of the existing façade system could be remediated through a series of actions associated to the key assumptions. These actions are presented in Table 7-8 and provide flexibility in the potential remediation actions to be defined by the stakeholders, considering that their cost-effectiveness largely varies. These actions are proposed on the basis that occupancy cannot be modified. Furthermore, the active elements of the fire safety strategy (detection, notification) need to function adequately during the lifecycle of the building as an inadequate management would not only invalidate the performance assessment but could also lead to disastrous consequences, e.g. failed evacuation due to lack of detection.

219

Chapter 7. Case studies

Table 7-8. Treatment options based on key assumptions Management Description Key assumption actions Replacing combustible elements like carpets and Control: fire load A1 Fire growth combustible wall finishes could improve performance, manage limited to the common areas. Alternative door design can lead to better performance. Fire-rated doors with a gap of 3 mm could enhance smoke Control: redesign of A2 Compartmentation containment [49], although self-closing mechanisms door system would be needed in addition [50]. However, failed compartmentation scenarios are not fully eliminated [51]. Prevention: Use less The flame spread rate can be iterated to find a maximum Ignition of façade or non-combustible A5 allowable flame spread, found at 0.1 m/min (see section criterion materials for the 7.1.7.2). cladding Giraldo et al. [52] studies the impact of flame spread barriers on timber facades, including building’s External flame Mitigation: flame A6 geometry. The Lacrosse fire showed how an architectural spread spread barriers decision prevented horizontal flame spread (under specific wind conditions) [53]. Control: Increase Nguyen et al. [46] suggests insulated and laminated glass Fire re-entry time for external performs better than regular glass despite its increased A8 criterion flaming and fire re- cost. This substitution would yield an increased time for entry breakage and re-entry.

7.1.9 Façade ignition and external flame spread The performance assessment results can be deemed conservative but the proposed actions align with the intent of a performance-based design by providing flexibility in decision-making. This is achieved through the holistic approach of the MAD methodology. As fire research provides data and models to deal with some of the complex phenomena involved, the damage model employed can be updated and conservatism reduced while accuracy increased. If fire research does not develop substantively then the obtained information remains a valid base for decision- making without compromising the professional ethics of the engineers nor shutting down the stakeholder motivations.

Professional ethics require consideration of stakeholder motivations and never compromising the trustworthiness of technical studies. For example, a parameter such as the flame spread rate is critical in a fire risk assessment and it will have considerable influence on the outcome. In

220 Chapter 7. Case studies

this case study, 4 m/min was used based on a series of full-scale real building fires and is independent of the materials used. Refining this value by considering specific façade materials is desirable but must be done carefully to ensure that it is realistic and representative. Bench- scale flammability data from the Cladding Materials Library [54] quotes a rate of 0.1 m/min to be used only as part of correlations for flame spread theory. Applying this value directly would be tempting as it is a drastically lower flame spread rate and will thus often lead to an acceptable performance for a given building. However, a scaling analysis is required to be able to obtain a realistic value to apply to a full building, such as one by Chung and Drysdale [55].

Technical considerations which may appear unimportant can in reality have significant impact on the overall performance of a building. The consideration above of an alternative flame spread exemplifies this as using it would lead to a false safety sense and results which are not credible. The implications of this are relevant in Australia and worldwide where tens of thousands of buildings await remediation after their facades have been deemed non-compliant. The proposed methodology adds significant value to the technical information in these situations and helps potentially prevent incalculable losses.

7.2 Case-study: Chemical storage warehouse The warehouse operation consists on storing raw and finished chemicals mainly used as additives for construction purposes. The raw materials are mixed at atmospheric conditions on- site to produce a defined range of products that are then stored in situ. The warehouse is divided in five main areas as shown in Figure 7-11 including the office area, finished product storage, flammable materials storage, acids and salts storage and bulk storage & mixing area. The chemical products storage is done using ICB containers on timber pallets, which are placed on five (5) stack storage racks. The structure is made of lightweight pre-cast concrete walls and horizontal I beams, which go across the warehouse and onto the neighbouring warehouse (these share a common wall).

221 Chapter 7. Case studies

39.2 m

5. Bulk storage 1. Offices & mixing

Forklift 62 m 2. Finished product charging 39.5 m area

4. Acids and salts

Off-spec Sliding door 30 m Main door Storage racks

ICB 3. Flammables 7.5 m Mechanical ventilation 14.58 m

Figure 7-11. Graphical description of the warehouse areas; not to scale

Table 7-9. Typical storage quantities and types per area. As reported in official documents and observed during visits. NI: No information Area Storage class Packaging group Approx. quantity (L) 8 - Corrosive II (medium danger) 14000 2 8 - Corrosive III (low danger) 7000 9 - Miscellaneous III (low danger) 5000 3 3 – Flammable NI 600 4 NI NI 3000-6000 8 - Corrosive II (medium danger) 2000 5 8 - Corrosive III (low danger) 2000 9 - Miscellaneous III (low danger) 2000

It is of particular interest to the management of the warehouse to understand the fire risk. This allowed the author to inspect the premises and conduct a preliminary hazards identification, noting the combustible, oxidizing and corrosive materials being stored and processing in area 5 (Figure 7-12). The processing of the materials mainly involve their transfer through pneumatic pumps to a large mixing container, where using specific proportions and additional (inert) materials, the final product is obtained and packed in empty IBCs. This process presents the potential for leak scenarios and subsequent pool fires, particularly those in which chemicals with over 90% ethanol content are involved. From the preliminary hazard analysis, this scenario was prioritized and is the subject of this case-study.

222 Chapter 7. Case studies

Figure 7-12. Graphical description of (left) Area 5 – plan view and (right) IBCs distribution – elevation view; not to scale

7.2.1 Safety objective In this case-study the focus is on the operation, therefore the safety objective is defined as avoiding major economic damage from a fire and ensuring business continuity. This objective excludes life-safety, assuming that the low occupancy of the facility will allow for a prompt and quick evacuation. Understanding whether this assumption is valid is not part of the scope of this analysis.

7.2.2 Acceptance criteria Translating the safety objective into an acceptance criteria requires understanding how to quantify ‘fire damage’ and business continuity. The former can be defined in terms of the degree of damage caused to the warehouse due to the fire effects, including the amount of IBCs lost to the fire and can be formulated as a percentage of the total of chemicals stored in the warehouse, e.g. 10% of ICBs. The latter, business continuity could be ensured as long as the warehouse remains fit for purpose, i.e. its structural integrity remains. The time for structural failure of the beams on the ceiling can be taken as a criterion for evaluating business continuity. In this application, the failure of the steel beam is taken to occur when it loses all its plastic moment capacity.

223 Chapter 7. Case studies

7.2.3 General damage model The acceptance criterion is formulated in function of property damage. This means that the damage model should aid in identifying scenarios that 1) affect the integrity of the IBCs stored in the warehouse and 2) damage the roof steel beams. Such scenarios can provide the necessary insight to support decision-making in the warehouse and lead to increased safety levels. The question is which scenarios should be modelled.

The answer can be found by trying to answer a more phenomenological question: which scenarios can raise the temperature of the steel to the point that it loses its plastic moment capacity. In other words, this becomes an exercise in iterating the fire scenarios possible in the warehouse. Since it is a fairly simple system with very well defined contents and activities, an initial set of scenarios of interest can be formulated.

The first scenario of interest is associated to the activity that could likely produce a loss of containment and a subsequent spill of a flammable liquid and therefore a pool fire. The second scenario is one in which the stored IBCs catch fire; given the practices in this facility it does not seem reasonable that the IBCs will spontaneously catch fire. One possibility for this scenario to come about is if it is triggered by a pool fire in its proximity, i.e. the first scenario. This constitutes a general damage model that will be used to explore the damage potential in terms of lost IBCs and business continuity. The general damage model for business continuity is graphically described in Figure 7-13.

Figure 7-13. General damage model for business continuity

224 Chapter 7. Case studies

The following sections will provide a specific damage model for each one of the scenarios inferred and estimate their damage potential. The results will enable to identify whether IBCs are lost (economic loss) or if structural failure is reached (business continuity failure).

7.2.4 Scenario 1 - Damage model During the drainage of IBCs into the mixing vessels of area 5, the IBCs are connected to flexible plastic hoses that allow loading the mixing vessels. This activity provides several possibilities (not explored in detail here) for loss of containment of the ethanol IBCs used to produce a particular chemical formulation. Although other scenarios could lead to loss of containment in the warehouse, this activity is routinely carried out, providing for periodical possibilities for loss of containment.

Furthermore, this activity is also subject to the presence of different ignition sources. A first ignition source is the drainage operation itself, as the flow of flammable liquids through the container and then the hoses can lead to the accumulation of static electricity [56]; this can be mitigated by earthing the system, but cannot be ensured due to the human factor involved. Another potential ignition source is the electrical forklift used to transport the IBCs and a final source is the presence of internal combustion engines of the trucks entering and exiting the warehouse in area 5.

Given the possibility for a loss of containment of a flammable liquid and the potential for ignition sources, the assessment initially focuses on a pool fire scenario taking place at the mixing vessels area. This is represented by the specific damage model illustrated in Figure 7- 14 for scenario 1.

225 Chapter 7. Case studies

Figure 7-14. Specific damage model for scenario 1

In this damage model it is assumed the leak of ethanol will occur and that an ignition source will be readily available leading to the pool fire to start. Second, the ethanol spill is limited by the available drainage system, which surrounds the mixing area. The effect of the drainage system is key as it limits the available fuel area and volume. The final pool volume is defined by the minimum thickness for a spilled layer of ethanol on concrete. The pool fire will lead to heat transfer by radiation directly from the flames to the IBCs being stored in the area, while the hot gas layer (smoke) accumulating on top will also contribute to this effect. The IBCs exposed to heat will possibly degrade and leak, contributing to a second pool fire. The number of degraded IBCs conditions the duration of the second pool fire, and this duration is the key variable to quantify the damage to the beams on the roof of the warehouse structure. The key assumptions associated to this damage model are:

• A1 - A spill of one IBC with ethanol occurs and an ignition source is readily available • A2 - The drainage is not blocked and its borders increase the expected pool thickness • A3 - The fire occurs during normal operation hours when the front door of the warehouse is fully open • A4 - The leak rate of affected IBCs is assumed to be consistent with full rupture • A5 - When considered, mechanical extraction is readily available and effective

226 Chapter 7. Case studies

7.2.5 Scenario 1 - Damage estimation Based on the specific damage model for scenario 1, the damage estimation can be carried out. This estimation requires obtaining the gas temperature near the ceiling to assess the loss of plastic moment capacity and any negative effect on the surrounding IBCs (e.g. degradation).

7.2.5.1 Pool fire The effects of the initial pool fire are quantified using the Fire Dynamics Simulator (FDS) [57], a CFD model that allows to represent the fire dynamics within an enclosure such as area 5 of the warehouse. Based on the findings of the research presented in Appendix 2, FDS can lead to severe underestimation of temperatures and heat fluxes for under-ventilated fires, and underestimation of the gas temperatures for fuel-controlled fires. However, the opening factor for area 5 (19.8 m-1/2) indicates this is an ideal compartment fire to be simulated in FDS based on the results of Appendix 2.

The FDS model constructed is a simplified representation of area 5 and includes the storage racks with the IBCs. Using the user guidelines of FDS a characteristic fire diameter (D*) was estimated [57] using the heat release rate for both the initial and secondary pool fires. Based on theoretical calculations for ethanol pool fires [58] the heat release rate (HRR) is estimated to about 90 MW. This HRR assumes a constant mass burning rate, which in real large fires can vary significantly.

Preliminary simulations of the pool fire indicate the HRR can be up to 200 MW. Based on these fire sizes, Figure 7-15 describes the relation between the numbers of cells required to cover the flame and the mesh size (δx). This graph indicates that a mesh size of 0.4 m allows to cover the flame with 12 to 17 cells, which is in the same range of the number of cells typically used in the validation tests for FDS [59]. However, the flame sizes in these tests are 3 to 4 times smaller than the one in the case study. This is taken into consideration in the trustworthiness of the performance assessment.

227

Chapter 7. Case studies

40.0

Theoreti 30.0 cal mass

x loss rate δ 20.0 D*/ Variable 10.0 mass loss rate 0.0 0.2 0.4 δx [m] 0.6 0.8

Figure 7-15. Comparison of the number of cells spanning the characteristic fire diameter for different mesh sizes

As presented at the introduction of the case study, this fire risk assessment is focused specifically in the scenario of pool fire in the mixing process of Area 5. To set up the scenario, a series of conditions and variables must be defined and their variability considered. These are summarized in the quantities recorded in the information registry as presented in Table 7-10. The values reported are used to set up the scenario in FDS to estimate the damage on the IBCs close to the initial pool fire and then to obtain the temperature profile at the ceiling to estimate the damage to the structure.

Table 7-10. Quantities involved in the damage estimation for the initial pool fire Variable Value Unit Source Main door width 8 m Main door height 7 m Facility specific Initial pool fire fuel Ethanol - Initial pool fire volume 1 m3 Time to ignition 0 s Assumption 1 Spill depth 1.0 - 5.0 mm Gottuk and White [60] Ambient temperature 25 C Facility specific IBC dimensions 1 x 1.2 x 1.2 m3 HSE report [61] IBC material High density polyethylene (HDPE) - Facility specific HDPE specific heat 1.7-2 kg/kJK Luche et al. [62] HDPE conductivity 0.35-0.43 W/mK Extraction rate 0, 3x50, 3x100 m3/s Specified by analyst FDS simulation mesh size 0.4 m Specified by analyst

The quantities presented in Table 7-10 allow setting up a range of scenarios. First, different pool fire thicknesses are considered in the range 1-5 mm, as this is an unknown quantity that directly defines the fuel volume and therefore the fire duration. Based on the reported data [60],

228

Chapter 7. Case studies

a 0.75 mm pool thickness would be expected in perfect ethanol-air equilibrium. However, the constraints caused by the drainage system and the transient nature of the spill cannot allow ensuring this value. Such effects can be reflected in the literature. For example, Lauri [63] assumed an ethanol pool thickness of 1.5 cm. Benfer [64] found thicknesses for denaturalized alcohol of less than 1 mm for spills of less than 450 ml, while reporting previous research in which the thickness was between 1 and 2 mm.

The previous considerations and values for key variables allow formulating a series of scenarios. These are presented in Table 7-11 and are the basis for the damage estimation. These six scenarios include different pool thicknesses ranging from 1 to 5 mm, different mesh sizes and the inclusion of mechanical extraction fans. The damage resulting from each scenario is estimated in the following subsections, for which one of the key inputs is the results from the simulation in FDS (Figure 7-16).

Table 7-11. Scenarios for the initial pool fire Pool depth Mesh size Extraction rate Scenario [mm] [m] [m3/s] 1.0 1 - 1.1 2.5 0.4 - 1.2 5 - 1.3 5 0.2 - 1.4 5 3 x 100 1.5 5 0.4 3 x 50 1.6 5 3 x 10

Figure 7-16. Snapshot of the simulation for scenario 1.3 in FDS

7.2.5.2 Structural damage The gas temperature near the ceiling obtained in the FDS simulation can be used to gain insight on the structural response of the warehouse. The warehouse structure is composed of pre-cast

229 Chapter 7. Case studies concrete walls and the roof is made of a series of steel rafters (IPE 140) supporting purlins and steel roof sheets, as shown in Figure 7-17. An important assumption is that the heat of the fire will not affect the concrete walls and that all the structural damage is limited to the response of the roof structure (A6).

Figure 7-17. Roof structural design (not to scale)

The structural damage to the roof structure is defined by the loss of the plastic moment capacity of the steel beam, which is reduced as its internal temperature increases. This requires first performing a heat transfer estimation from the hot gas layer to the beam. The heat transfer is done following the lumped capacitance method for unprotected steelworks given by Eq. 1 [65]. For protected steelworks, the lumped capacitance method is implemented having as main assumptions the uniform temperature throughout the beam and an insulation temperature equal to the mid-difference between the gas and the steel temperature (A7; Eq. 2). Although the roof beams in the warehouse have no protection, its effect on the damage potential is estimated to gain insight on its potential to mitigate the risk, as it was done with mechanical extraction.

= + Eq. 1 𝐹𝐹 1 4 4 ∆𝑇𝑇𝑠𝑠 𝑉𝑉 𝜌𝜌𝑠𝑠𝑐𝑐𝑠𝑠 �ℎ𝑐𝑐�𝑇𝑇𝑔𝑔 − 𝑇𝑇𝑠𝑠� 𝜎𝜎𝜎𝜎�𝑇𝑇𝑔𝑔 − 𝑇𝑇𝑠𝑠 �� ∙ ∆𝑡𝑡 = Eq. 2 𝐹𝐹 𝑘𝑘𝑖𝑖 𝜌𝜌𝑠𝑠𝑐𝑐𝑠𝑠 𝐹𝐹 𝑑𝑑 𝜌𝜌 𝑐𝑐 𝑠𝑠 𝑖𝑖 𝑠𝑠 𝑠𝑠 𝑖𝑖 𝑖𝑖 𝑖𝑖 𝑔𝑔 𝑠𝑠 ∆𝑇𝑇 𝑉𝑉 𝑑𝑑 𝜌𝜌 𝑐𝑐 �𝜌𝜌𝑠𝑠𝑐𝑐𝑠𝑠+𝑉𝑉∙ 2 � ∙ �𝑇𝑇 − 𝑇𝑇 �∆𝑡𝑡

230 Chapter 7. Case studies

Estimating the steel temperature as well as the loss of the plastic moment capacity requires specifying a series of inputs, presented in Table 7-12. The potential failure of the beam is given

by the difference between the plastic moment (Mapp) and the plastic moment capacity (Mp).

This is better represented by the limit state function Mapp ≤ Mp. The latter is a function of the plastic section modulus (Zp; defined as in Eq. 3) and the yield stress ( , ). The latter is

specified by the manufacturer for ambient temperature ( , ) and decreases𝜎𝜎𝑦𝑦 𝑇𝑇 as a function of heat transfer (Eq. 4 and 5). The yield stress reduction factor𝜎𝜎𝑦𝑦 𝐴𝐴𝐴𝐴𝐴𝐴 ( , ) is estimated following the Eurocode 3 [66]. 𝑘𝑘𝑦𝑦 𝑇𝑇

= + 0.25 ( 2 ) Eq. 3 2 𝑧𝑧𝑝𝑝 𝑏𝑏𝑡𝑡𝑓𝑓�𝑑𝑑 − 𝑡𝑡𝑓𝑓� 𝑡𝑡𝑤𝑤 𝑑𝑑 − 𝑡𝑡𝑓𝑓 = , Eq. 4

𝑀𝑀𝑝𝑝 𝑧𝑧𝑝𝑝𝜎𝜎𝑦𝑦 𝑇𝑇 , = , , Eq. 5

𝑦𝑦 𝑇𝑇 𝑦𝑦 𝑇𝑇 𝑦𝑦 𝐴𝐴𝐴𝐴𝐴𝐴 Table 7-12. Quantities involved𝜎𝜎 in the𝑘𝑘 damage𝜎𝜎 estimation for structure Variable Value Unit Source Steel protection width 0.5-1.0 cm Specified by analyst Convective heat transfer coefficient 50 W/m2K Recommended for hydrocarbon fires [65] Resultant emissivity 0.5 Typical value for carbon steel beams [65] Steel density 7850 kg/m3 Typical value for carbon steel beams [65] Steel section IPE 140 - Facility Steel yield strength 355 MPa Typical value for carbon steel beams [65] Roof sheet weight 10 kg/m Facility Purlin weight (c-section) 5.66 kg/m Facility

Equations 1 and 2 are implemented in a Microsoft® Excel spreadsheet, in which the input gas temperature profile is obtained from the FDS simulation. The results allow to estimate the steel temperature for both the unprotected case and for the cases in which the steel is protected through an insulation material. The results are presented in Figure 7-18, where the insulation is sprayed mineral fibre of 5 and 10 mm thickness.

231

Chapter 7. Case studies

Figure 7-18. Gas temperature from the pool fire and steel temperature (unprotected and with insulation)

Using the results of the heat transfer to the steel beam, the reduction in the plastic moment capacity (Mapp) can be estimated. This value is estimated as a function of the loads supported by the roof beam, which account for the steel roof sheets, the purlins on which they rest and the self-weight of the beams. The loads on the simply supported beam are presented in the free body diagram in Figure 7-19, along with its corresponding bending moment.

Figure 7-19. Free body and bending moment diagrams

= 8 Eq. 5 2 𝑎𝑎𝑎𝑎𝑎𝑎 𝑤𝑤𝐿𝐿 A relevant and conservative assumption𝑀𝑀 is that the� heat effects on the beam will be uniform and caused by the hot gas layer near the ceiling (A8). Eventually, the beam will fail if its yield

strength is exceeded at the centre, as shown in the bending moment diagram, where Mapp is estimated as in Eq. 5. Here, w is the added line load on the beam and L its length. Using the

232 Chapter 7. Case studies

values of Table 7-12 this yields a Mapp of 92.5 kNm. Without accounting for fire effects, the plastic moment is only 2.7% of the plastic moment capacity at ambient temperature.

Applying equations 3-5, Mapp is estimated. The results for both unprotected steel and protected steel are presented in Figure 7-20.

Figure 7-20. Reduction of the plastic moment capacity for scenario 1.3

These results clearly indicate that the unprotected steel beams reduce their plastic moment capacity, but stay away from the limit state and do not fail. This can be translated into a safety factor defined as the ratio between Mapp and Mp (see Table 7-13). In the case of the protected beams, the effect of insulation prevent any reduction in the plastic momentum capacity of the steel beam. Although the structural damage is negligible, the effects of this pool fire can produce damage to the rest of IBCs stored in area 5. This damage is analysed in the following section.

Table 7-13. Structural damage for the initial pool fire Resulting safety Pool depth Mesh size Scenario factor for [mm] [m] unprotected beam 1.0 1 37 1.1 2.5 0.4 25.8 1.2 5 7.4 1.3 5 0.2 7

233 Chapter 7. Case studies

7.2.5.3 IBCs’ degradation

With a favourable result from the structural damage, it is necessary to estimate the damage produced by the pool fire to the IBCs located in area 5. The damage to these IBCs represents a property loss by itself, but could also lead to secondary events that may challenge the structural integrity of the roof.

First, degradation of the IBC is defined as the loss of integrity and subsequent leak of the IBC’s content and can be defined based on two criteria. The HDPE can reach its melting temperature at around 135ºC [67] or reach its ignition temperature, 340ºC [61]. An alternative criterion refers to the degradation of HDPE based on the heat flux it is exposed to. According to the experimental study by the HSE [61], the IBCs exposed to fire tend to fail within 2 and 3 minutes and leak. This study also suggests a rule of thumb in which a heat flux under 35 kW/m2 should result in the IBC surviving the fire, while one over 60 kW/m2 will ensure degradation and leakage, although it does not specify a minimum exposure time.

An additional assumption (A9) is made with regards to this minimum exposure time setting it to be 1.5 minutes. This is a conservative value considering the 2-3 minutes range reported by the same HSE for the IBCs to fail, which is justified as the same report stated that at this heat flux the containers “are almost certain to cause fire spread” [61]. The results are presented in Figure 7-21, where a) corresponds to scenario 1.1, b) to scenario 1.2 and c) to scenario 1.3. These graphs plot the temperature and incident heat flux on the surfaces of the IBCs.

The results of Figure 7-21 indicate that a spill thickness of 1 mm (Scenario 1.0) produces no degradation, despite ~80% of the IBCs reach the melting point for less than 2 minutes. Both 2.5 mm and 5 mm pool thicknesses yield degradation of a large amount of IBCs, with the latter compromising all of them for about 42 seconds.

234

Chapter 7. Case studies

2.5 mm (T) 2.5 mm (HF) 5 mm (T) 5 mm (HF) 160 160 140 140 120 120 100 100 80 80 60 60 No. IBCs exposed 40 No. IBCs exposed 40 20 20 0 0 0 50 100 150 200 250 0 50 100 150 200 250 Time [s] Time [s] a) b)

5 mm - 20 cm mesh (T) 5 mm - 20 cm mesh (HF) 160 140 120 100 80 60

No. IBCs exposed 40 20 0 0 50 100 150 200 250 Time [s] c) Figure 7-21. Exposure profiles for the scenarios of the pool fire. a) Scenario 1.1, b) Scenario 1.2 and c) Scenario 1.3

Mechanical extraction can exert an important effect on the damage as it effectively removes heat from within the compartment and reduces the damage on the exposed IBCs. The resulting set of scenarios for the initial pool fire is presented in Table 7-14, including the results for the number of IBCs experiencing degradation. The effect of the mechanical extraction is acknowledged but not carried in the damage estimation due to the extremely high rates required to achieve a significant reduction in damage (300 m3/s for a 50% reduction).

Table 7-14. Scenarios for the initial pool fire Damage Pool depth Mesh size Extraction rate Scenario IBCs affected Exposure time [mm] [m] [m3/s] (T > 340ºC) [min] 1.0 1 0.4 - - -

235

Chapter 7. Case studies

Damage Pool depth Mesh size Extraction rate Scenario IBCs affected Exposure time [mm] [m] [m3/s] (T > 340ºC) [min] 1.1 2.5 - 92% 1.98 1.2 5 - All 3.38 1.3 5 0.2 - All 3.65 1.4 5 3 x 100 All 1.82 1.5 5 0.4 3 x 50 All 2.36 1.6 5 3 x 10 All 3.23

7.2.6 Scenario 1 – Performance assessment From a business continuity point of view, scenario 1 does not enable structural failure. However, not only does scenario 1 allow for the complete loss of the IBCs located in the mixing area and the potential damage of the mixing equipment, but it can also produce a significant damage to the IBCs stored in area 5. More importantly, this damage can trigger the fire on the IBCs rack, which can lead to a secondary pool fire with a large potential for damage.

7.2.7 Scenario 2 – Damage model

The general damage model indicates that a rack fire is another possible scenario leading to an increase of the steel beam temperature and its subsequent failure. The following sections repeat the procedure for scenario 1, assuming it will degrade the IBCs on the contiguous rack and cause a secondary pool fire. A pool fire associated to the contents of the IBCs rack would occupy the whole floor area except for the drained area and the bulk storage area (Figure 7- 22). As there is no drainage or bunds in this area, the pool is free to extend along this are and beyond; the focus of this assessment is limited to Area 5.

236

Chapter 7. Case studies

Figure 7-22. Position of the initial and second pool fires (not to scale) An updated damage model (Figure 7-23) explicitly takes into account the degradation of the IBCs and the effects of the second pool fire. In practical terms, this second pool fire exposes the roof beam system to high temperatures until its burn out. Given the uncertainty on the inventory of IBCs, the potential volume of the fuel is part of the inputs of this updated damage model, which can be determined based on the storage rules of the warehouse or current practices.

Figure 7-23. Specific damage model for scenario 2

237

Chapter 7. Case studies

7.2.8 Scenario 2 - Damage estimation Based on the specific damage model for scenario 2, the damage estimation is carried out. This estimation requires obtaining the gas temperature near the ceiling to assess the loss of plastic moment capacity and any negative effect on the IBCs in the rack (e.g. degradation).

7.2.8.1 Pool fire To set up the scenarios for the second pool fire, a series of conditions and variables are defined and their variability considered. Relevant quantities of Table 7-10 are used again and complemented by the five new ones presented in Table 7-15. It is important to consider the variability of the empirical mass burning rate suggested in literature, which significantly varies the time to burnout.

Table 7-15. Quantities involved in the damage estimation for the second pool fire Variable Value Unit Source Spill depth - mm Estimated from volume of degraded IBCs Time for IBC failure 2-3 min HSE report [61] Secondary spill area 38 x 7.6 m2 Facility specific FDS simulation mesh size 0.4 m Specified by analyst Empirical mass burning rate 0.014-0.029 Kg/m2s Gottuk [60] and Babrauskas [58]

The results of the finer mesh simulation (Scenario 1.3) indicate that approximately 100 IBCs reach the ignition temperature for a period of about 2 minutes (section 7.2.5.3). The failure of these IBCs would result in a spill of 100 m3, including both fuel (i.e. ethanol) and other chemicals stored in the mixing area. As the storage inventory changes continuously in the warehouse, a precise quantity of spilled fuel cannot be established. Instead, three scenarios can be formulated.

An onerous scenario is proposed in which the totality of the spill consists of ethanol (Scenario 2.0). The second and third scenarios are set to have 50% and 25% of ethanol and the remainder of unreactive chemicals (Scenarios 2.1 and 2.2, respectively). A summary of the scenarios is provided in Table 7-16, where times to burnout are estimated using the empirical mass burning rate suggested for ethanol pool fires of over 6 m diameter (Table 7-15).

Table 7-16. Scenarios for the second pool fire Expected time to burnout Scenario Degraded IBCs Fuel volume [m3] [s] [min] [hr] 2.0 100 100 9481 158 2.63 2.1 100 50 4741 79 1.32

238

Chapter 7. Case studies

Expected time to burnout Scenario Degraded IBCs Fuel volume [m3] [s] [min] [hr] 2.2 100 25 2371 39.5 0.66

Figure 7-24. Snapshot of the simulation for scenario 2.0 in FDS

7.2.8.2 Structural damage Based on the updated damage model and the results of the FDS simulation for the secondary pool fire, the heat transfer calculations are carried out for the unprotected system and for six different insulation materials. A sample of the results is presented in Figure 7-25 in which the insulation material for the protected beams is sprayed mineral fibre with thickness of 0.5 and 1.0 cm. The unprotected beam reaches the gas temperature at around 6 minutes (381 s), while the protected beam does so at 32 and 62 minutes for each insulation thickness.

1300 1200 1100 1000 900 ºC] 800 Gas T 700 Unprotected 600 500 Protected 0.005

Temperature [ 400 Protected 0.01 300 200 100 0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Time [s]

Figure 7-25. Steel temperature as a function of fire duration

The heat transfer results enable estimating the yield stress reduction of the beam and the time for failure. The failure due to the loss of plastic momentum capacity is graphically described

239

Chapter 7. Case studies

in the profiles of Figure 7-26. These results indicate that the unprotected beam will fail in all three scenarios, which indicates an unfavourable performance of the system.

S2.2 S2.1 S2.0 500 Mapp 450 400 350 Mp 300

250 Mp-

M [kNm] 200 Protected 0.005 150 Mp- 100 Protected 50 0.01 0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Time [s] Figure 7-26. Plastic momentum as a function of time The presence of insulation on the steel beam slows heat transfer and has a direct effect on the time for failure. As different insulation materials provide different levels of protection (based on their thermal properties), the time to failure also differs. This effect can be observed in the results of Figure 7-27, where the unprotected beam fails in any of the three scenarios. The protected beam achieves an acceptable performance for the larger insulation thickness in scenario 2 for selected materials. The effect of the insulation material is condition by the assumption that it is adequately installed and maintained (A10).

Protected - 0.01 Protected - 0.005 S2.2 S2.1 S2.0 Compressed mineral wool board

Gypsum plaster board

Fiber-silicate board

Sprayed High-density vermiculite plaster

Sprayed Vermiculite plaster

Sprayed mineral fibre

Unprotected

0 20 40 60 80 100 120 140 160 Time [min] Figure 7-27. Failure time as a function of the applied insulation

240

Chapter 7. Case studies

7.2.9 Scenario 2 – Performance assessment The summary of these results indicate that the current facility will yield an inadequate performance in scenarios S2.0 and S2.1, while thick insulation could provide enough protection for scenario S2.2. The thickness of the insulation could be increased until it provides an adequate protection. Considering an application of insulation that goes beyond 10 mm would significantly challenge the quality of the insulation and could interfere with the structure itself. This would not be an effective solution due to its increasing cost and the uncertain performance such a thick insulation would provide. The risk treatment plan presented in the following section introduces a series of alternatives to reach an adequate performance.

Overall, this performance assessment exposes a series of elements that condition the result. These are mainly 1) initial pool volume, 2) affected IBCs, 3) structural features. Under the assumptions and available background knowledge, the result of the performance assessment is unacceptable as potentially all the IBCs in Area 5 are lost to the fire.

Regarding business continuity, the structural damage results indicate that the unprotected structure fails around 7 minutes and the protected fails between 19-31 min for the 0.5 cm insulation layer and between 35-65 min for a 1 cm insulation layer. These failure times would only be acceptable for scenario S2.2 (25 IBCs with fuel) given a 1 cm insulation layer, and result unacceptable for scenarios S2.0 (100 IBCs with fuel) and S2.1 (50 IBCs with fuel). The three elements conditioning the performance and the associated assumption can be further analysed to set out a risk treatment plan.

7.2.10 Risk treatment & uncertainties The quantities and assumptions used to assess the performance (Table 7-10, Table 7-15, Table 7-12) are scrutinized to bound the validity of the results. Each of these quantities are judged based on the strength of knowledge (SoK) and insensitivity (sensitivity-1) dimensions, as introduced in section 6.5.6. The results of this analysis allow presenting a distribution of the quantities with respect to the two dimensions and provide an overall description of the trustworthiness of the results. This description is synthesized in the heat map of Figure 7-28, in which 30 quantities are captured. The full information registry is presented in Appendix 4.

241

Chapter 7. Case studies

Figure 7-28. Trustworthiness heat map; the number in the cells indicate the number of variables

From these, only one quantity presents a low SoK, associated to the ignition time of the initial pool fire. As it is not possible to make a prediction of this time and the focus of the study is to understand the consequences of the pool fire, it is deemed reasonable to assume it is instantaneous. However, it is acknowledged that significant efforts can be made in preventing the ignition occurrence such as the implementation of a hazardous atmosphere classification (see API RP 505, ATEX Directive 2014/34/EU). As long as the possibility exists for the fire to occur, the performance assessment remains relevant to support decision-making.

Figure 7-28 indicates an overall high strength of knowledge, except for four quantities. One of this is the smoke extraction rate, which was shown in section 7.2.5.3 to have a very limited effect on the resulting damage and was therefore excluded from the rest of the assessment. The second quantity is the heat flux criterion for IBC degradation, which is based on the experimental work of the HSE [61] and set at a value of 60 kW/m2. According to the research, this heat flux would make the IBC “certain to cause fire spread” (A9). As there is no time associated to this assertion, it is conservatively assumed that failure will occur at 1.5 minutes, which is less than the range for failure provided by the HSE (2-3 minutes).

Based on the work by Luche et al. [62] the heat flux suggested by the HSE translates into mass loss rates 19 g/m2s, which applied to a typical HDPE sheet with density of 0.93 g/cm3 would result in complete mass loss at 8 minutes. This shows an important uncertainty source, as varying this time has a significant effect on the resulting damage. However, the heat fluxes experienced by the IBCs in the initial pool fire (scenario S1.1) are significantly in excess of the criterion set by the HSE, leading to support the assumption that a 1.5 minutes exposure will lead to failure and leak of the IBC (see Figure 7-29). Additional experimental data could be necessary to precise this conservative assumption.

242

Chapter 7. Case studies

Figure 7-29. Scenario S1.1 snapshot at 60 s, presenting incident heat fluxes on IBCs (kW/m2)

The third quantity with a medium SoK corresponds to the spill thickness. This quantity was discussed in detail in section , showing that the selected values for this assessment are conservative but well within the order of magnitude reported by other researchers. Taking into account that no trustworthy data is available on the effect of bounding edges on the pool thickness, the selected values are deemed representative. Experimental tests could be done to enquire on the effective thickness of the pool and adjust it accordingly in this model. Moreover, the presented results indicate that a 1 mm spill thickness could lead to minimum or no damage at all.

Ensuring a particular thickness for the spill is a major challenge, as it is a function of the properties of the substance and of the surface where the spill occurs. As the former cannot be modified, this leaves a few alternatives to be considered about the spill area: 1) Modify the spill area so that its surfaces are in an angle (with respect to the ground) and propend to the transport of the spill to the drainage system; 2) The drainage system could also include additional draining channels within the mixing area, which yield a reduced spill area. These options do not guarantee a specific pool thickness and are not considered significant in reducing the risk.

The fourth quantity with a medium SoK is the number of IBCs with fuel in area 5. This variable changes with time based on the requirements of the operation and cannot be exactly predicted. However, its upper limit can be defined as an operating rule. From the results presented in the previous section, it is clear that the only scenario with possibilities of an adequate performance is S2.2, which corresponds to 25 IBCs with additional fuel. This result can help manage fire risk by imposing a maximum number of containers with contents that can contribute to the continuation of the pool fire. With this, it is extremely relevant to the performance to devise a strategy for minimizing the number of IBCs with ethanol content and those that have

243

Chapter 7. Case studies

incompatible chemicals. This effectively removes the hazard from area 5, and would require ensuring effective compartmentation between this area and the interconnected ones.

The rest of quantities all are judged to have a high SoK, but still require consideration. Most of these quantities are associated to the facility characteristics and therefore are not addressed here. Most of the remaining quantities are associated to the physical and thermal properties of the steel beam. It is hard imagining replacing the beams to mitigate risk, as this is a built and operating facility.

However, insight on how the performance changes with different beams can aid in designing a future warehouse. To gain this insight, the analysis of section 0 is repeated for a large array of standard beams, in which the cross sectional area varies. This change has a significant impact on the weight of the beam and to its heating process. The results are presented in Figure 7-30 and in Figure 7-31, indicating a high sensitivity of the result to the selected beam. Beams with failure time of zero minutes indicate that they do not fail within the considered timeframe. Adequate performance is obtained from protected IPE 180 beams or larger, while unprotected configurations would require IPE 220 beams or larger. This information should be balanced against more detailed structural analysis and a cost-benefit analysis, as a larger beam will create additional load to the supporting walls. Furthermore, it is acknowledged that beam selection can be driven by the available offer at the time of construction.

100

90 Unprotected 80 Sprayed mineral fibre 70 Sprayed Vermiculite 60 plaster Sprayed High-density 50 vermiculite plaster Fiber-silicate board 40 Gypsum plaster board Failure time [min] 30 Compressed mineral wool board 20

0 IPE 80 IPE 100 IPE 120 IPE 140 IPE 160 IPE 180 IPE 200 IPE 220 IPE 240 IPE 270 IPE 300 IPE 330 IPE 360 IPE 400 IPE 450 IPE 500 IPE 550 IPE 600 IPE Figure 7-30. Effect of beam and insulation (0.5 cm)

244

Chapter 7. Case studies

100

90 Unprotected 80 Sprayed mineral fibre 70 Sprayed Vermiculite plaster 60 Sprayed High-density vermiculite plaster 50 Fiber-silicate board 40 Gypsum plaster board Failure time [min] 30 Compressed mineral wool board 20

0 IPE 80 IPE 100 IPE 120 IPE 140 IPE 160 IPE 180 IPE 200 IPE 220 IPE 240 IPE 270 IPE 300 IPE 330 IPE 360 IPE 400 IPE 450 IPE 500 IPE 550 IPE 600 IPE

Figure 7-31. Effect of beam and insulation (1.0 cm)

Returning to the existing facility, applying insulation on the existing beam could contribute to a better performance (except for the compressed mineral wool board). However, this solution cannot cope with the requirement of scenarios 2.0 and 2.1 as these extend the fire up to 80 and 160 minutes, respectively. Applying insulation would still require redistributing the Area 5 inventory, ensuring that IBCs with potential fuel load constitute only up to 25% of the total inventory.

7.2.11 Introducing variability to Scenario 2 Based on the analysis of the information registry, the results of the performance assessment are ratified and a series of potential risk prevention and mitigation actions are proposed. The performance assessment was conducted following the guidance for the MAD methodology as introduced in Chapter 6, i.e. a deterministic approach.

The objective of this additional exercise is to explore the effect of introducing stochastic variables into the calculation and highlight any potential additional insight on the performance of the system. The variables selected are listed in Table 7-17 and the procedure selected is a Monte Carlo simulation of fifty thousand realizations. This procedure is implemented in the same Excel worksheet where the damage estimation took place, making use of the SIPMath plugin [68].

245

Chapter 7. Case studies

Table 7-17. Stochastic variables for the probabilistic analysis Variable Source Distribution Par. 1 Par. 2 Units Ambient temperature Local conditions Normal 20 5 ºC Steel beam – density Normal 7850 0.01 Kg/m3 Steel beam - Yield strength Devaney [69] Lognormal 399.2 0.07 MPa (initial) Steel beam - Yield strength Qureshi et al. Uniform 0 1 - reduction factor [70] Smoke layer - Heat transfer Whole possible W/m2 Uniform 5 50 coefficient range [65, 71] K

The density, initial yield strength (at ambient temperature) and its reduction factor were selected due to their relevance in different probabilistic studies for steel beams and the availability of well characterized distributions [69, 70]. The temperature dependent reduction factor for the yield strength is modelled by randomly sampling a standard normal distribution and using this value as the seed to determine the reduction profile, which is based on the investigation conducted by Qureshi et al. [70]. The process to select the ‘seed’ value to then define the yield strength reduction factor is illustrated in Figure 7-32.

Figure 7-32. Process to sample the yield strength reduction factor

Ambient temperature was selected as it is inherently variable and has an effect on the yield strength reduction. Finally, the heat transfer coefficient is a significant source of uncertainty, as it can take values within the range of 5 W/m2K (for free convection phenomena [71]) up to 50 W/m2K (for hydrocarbon fires [65]).

246

Chapter 7. Case studies

Analytically estimating the heat transfer coefficient can be a significant challenge scaping the scope of this assessment, therefore its expected value is estimated directly from the FDS simulation of the pool fire modelled in section 7.2.5.1, obtaining a value for the heat transfer coefficient ranging from 2.9 to 12.8 W/m2K (see Figure 7-33).

Figure 7-33. Convective heat transfer coefficient reported by FDS and corresponding histogram (taken from 6 devices directly over the fire within the timeframe between 30 s and 300 s)

Other key variables that introduce significant output sensitivity have already been discussed, such as the pool thickness. These variables cannot be readily represented as stochastic variables without making significantly unsupported assumptions; therefore this analysis is limited to the presented variables which can be appropriately characterized. However, it must be noted that the selection of these variables does not follow an explicit algorithm or procedure and it is actually beyond the scope of the guidelines for MAD. Nevertheless, it is relevant to provide a link between MAD and a potential PRA.

The results of the probabilistic analysis based on the five stochastic variables are summarized by the probability density and cumulative probability graphs of Figure 7-34. These results yield less conservative results compared to those resulting from the deterministic estimation and presented in section 7.2.8.2. The original results for the steel beam failure time are included in the results of Figure 7-34, indicating that these lay on the upper end of the distributions and a quantitative description of how conservative they are. This indicates the benefit of complementing the deterministic result of MAD with a probabilistic analysis.

247

Chapter 7. Case studies

Figure 7-34. (Above) Probability density and (below) cumulative probability for the failure time of the steel beam. Red vertical line indicates the result of the deterministic analysis.

The resulting probability distributions provide an additional insight on the performance, indicating that variability will allow for less conservative results with a high probability. Nevertheless, these results are conditioned by the selected stochastic variables, number of realizations and sampling method (in this case quasi-random). Implementing more robust sampling methods (e.g. Latin-hypercube sampling) and extending the analysis of the heat transfer coefficient results in FDS could yield different results. These possible improvements to the probabilistic analysis are out of the scope of this analysis, but the results provide a link between the results from MAD and their use in a subsequent analysis focused on alternative selection.

248

Chapter 7. Case studies

Bibliography 1. Hannoudi, L.A., J.E. Christensen, and M. Lauring, Façade System for Existing Office Buildings in Copenhagen. Energy Procedia, 2015. 78: p. 937-942. 2. Programme, I.E.A.a.t.U.N.E., 2018 Global Status Report: towards a zero‐emission, efficient and resilient buildings and construction sector. 2018. 3. Chen, T.B.Y., et al., Fire Risk Assessment of Combustible Exterior Cladding Using a Collective Numerical Database. Fire, 2019. 2(1): p. 11. 4. Gandhi, P., et al., Performance of glass-ACP façade system in a full-scale real fire test in a G+2 structure. Procedia Engineering, 2017. 210: p. 512-519. 5. Oleszkiewicz, I., Fire exposure to exterior walls and flame spread on combustible cladding. Fire Technology, 1990. 26(4): p. 357-375. 6. Bedon, C., et al., Structural characterisation of adaptive facades in Europe – Part I: Insight on classification rules, performance metrics and design methods. Journal of Building Engineering, 2019. 25: p. 100721. 7. Bedon, C., et al., Structural characterisation of adaptive facades in Europe - Part II: Validity of conventional experimental testing methods and key issues. Journal of Building Engineering, 2019. 25: p. 100797. 8. Zhou, B., et al., Experimental study of expanded polystyrene (EPS) External Thermal Insulation Composite Systems (ETICS) masonery façade reaction-to-fire performance. Thermal Science and Engineering Progress, 2018. 8: p. 83-92. 9. Bonner, M. and G. Rein, Flammability and Multi-objective Performance of Building Façades: Towards Optimum Design. International Journal of High-Rise Buildings, 2018. 7. 10. Torero, J., Grenfell tower: Phase 1 report. 2018, Torero, Abecassis Empis and Cowlard. 11. Bleby, M., 'High-risk' buildings in Victoria could be far more than expected, in The Australian Financial Review. 2019, Fairfax Media Publications Pty Limited. 12. Bleby, M., Wht NSW must follow Vic on cladding, in The Australian Financial Review. 2019, Fairfax Media Publications Pty Limited. 13. Bleby, M., Victoria removing cladding from 13 state-owned schools, in The Australian Financial Review. 2019, Fairfax Media Publications Pty Limited. 14. Woodcock, A., Grenfell tower fire: 56,000 'still at risk' from flammable cladding three years on, in The Independent. 2020: London, UK. 15. Fu, F., Chapter Two - Fundamentals of Tall Building Design, in Design and Analysis of Tall and Complex Structures, F. Fu, Editor. 2018, Butterworth-Heinemann. p. 5-80. 16. Babrauskas, V., J.M. Fleming, and B. Don Russell, RSET/ASET, a flawed concept for fire safety assessment. Fire and Materials, 2010. 34(7): p. 341-355. 17. Bjelland, H., et al., The Concepts of Safety Level and Safety Margin: Framework for Fire Safety Design of Novel Buildings. Fire Technology, 2015. 51(2): p. 409-441.

249

Chapter 7. Case studies

18. Keski-Rahkonen, O., Breaking of window glass close to fire. Fire and Materials, 1988. 12: p. 61-69. 19. Rukavina, M.J., M. Carević, and I.B. Pečur, Fire protection of facades. The Guidelines for Designers, Architects, Engineers and Fire Experts, 2017. 20. Richard D. Peacock, P.A.R., Glenn P. Forney, CFAST – Consolidated Model of Fire Growth and Smoke Transport, Volume 2: User’s Guide, U.S.D.o. Commerce, Editor. 2017. 21. Abecassis-Empis, C., Analysis of the compartment fire parameters influencing the heat flux incident on the structural façade. 2010, The University of Edinburgh: Edinburgh, UK. 22. Gwynne, S.M.V. and E.R. Rosenbaum, Employing the Hydraulic Model in Assessing Emergency Movement, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2115-2151. 23. Gwynne, S.M.V. and K.E. Boyce, Engineering Data, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2429- 2551. 24. Notarianni, K.A. and G.W. Parry, SFPE Handbook of Fire Protection Engineering, Fifth Edition. 2016: Springer New York. 2992-3047. 25. Hopkin, C., M. Spearpoint, and D. Hopkin, A Review of Design Values Adopted for Heat Release Rate Per Unit Area. Fire Technology, 2019. 55(5): p. 1599-1618. 26. Purser, D.A. and J.L. McAllister, Assessment of Hazards to Occupants from Smoke, Toxic Gases, and Heat, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2308-2428. 27. Purser, D.A., Combustion Toxicity, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2207-2307. 28. Dembele, S., R.A.F. Rosario, and J.X. Wen, Thermal breakage of window glass in room fires conditions – Analysis of some important parameters. Building and Environment, 2012. 54: p. 61-70. 29. Richard D. Peacock, P.A.R., Glenn P. Forney, CFAST – Consolidated Model of Fire Growth and Smoke Transport, Volume 1: Technical guide, U.S.D.o. Commerce, Editor. 2017. 30. Purser, D., Dependence of Modelled Evacuation Times on Key Parameters and Interactions. 2010. p. 667-675. 31. J. Cadena, J.H., C. Maluk, D. Lange, J. L. Torero, A. F. Osorio, Overcoming Risk Assessment Limitations for Potential Fires in a Multi-Occupancy Building. Chemical Engineering Transactions, 2019. 77. 32. Busby, J.S., R.E. Alcock, and E.J. Hughes. Credibility in risk assessment. in Probabilistic Safety Assessment and Management. 2004. London: Springer London. 33. (BSI), B.S.I., PD 7974-7:2019 Application of fire safety engineering principles to the design of buildings. Probabilistic risk assessment. 2019, London, UK.

250

Chapter 7. Case studies

34. Flage, R. and T. Aven, Some brief concluding remarks in relation to the discussion with Floris Goerlandt and Genserik Reniers about strength of knowledge (strength of evidence) judgments in semi-quantitative risk analysis. Safety Science, 2018. 108: p. 237. 35. Askeland, T., R. Flage, and T. Aven, Moving beyond probabilities – Strength of knowledge characterisations applied to security. Reliability Engineering & System Safety, 2017. 159: p. 196-205. 36. Berner, C. and R. Flage, Strengthening quantitative risk assessments by systematic treatment of uncertain assumptions. Reliability Engineering & System Safety, 2016. 151: p. 46-59. 37. Goerlandt, F. and G. Reniers, Evidence assessment schemes for semi-quantitative risk analyses: A response to Roger Flage and Terje Aven. Safety Science, 2017. 98: p. 12-16. 38. Khorsandi, J. and T. Aven, Incorporating Assumption Deviation Risk in Quantitative Risk Assessments: A Semi-Quantitative Approach. Vol. 163. 2017. 39. Aven, T., Supplementing quantitative risk assessments with a stage addressing the risk understanding of the decision maker. Reliability Engineering & System Safety, 2016. 152: p. 51-57. 40. Hackitt, J., Building a Safer Future, Independent Review of Building Regulations and Fire Safety: Final Report. 2018: London, UK. 41. Peter Shergold, B.W., Building Confidence – Improving the effectiveness of compliance and enforcement systems for the building and construction industry across Australia. 2018, Building Ministers’ Forum (BMF). 42. Groner, N.E., A decision model for recommending which building occupants should move where during fire emergencies. Fire Safety Journal, 2016. 80: p. 20-29. 43. T. Z. Fabian, P.D.G., Smoke Characterization Project. 2007, Underwriters Laboratories Inc.: Northbrook, IL, USA. 44. C. Wade, G.B., K. Frank, R. Harrison, M. Sprearpoint, B-Risk 2016 user guide and technical manual, B. Ltd., Editor. 2016: Judgeford, New Zealand. 45. Parry, R., C.A. Wade, and M. Spearpoint, Implementing a Glass Fracture Module in the BRANZFIRE Zone Model. Journal of Fire Protection Engineering, 2003. 13(3): p. 157-183. 46. Nguyen, K., et al., Performance of modern building facades in fire : a comprehensive review. Electronic Journal of Structural Engineering, 2016. 16: p. 69. 47. Bai, Z.P., Y.F. Li, and Y.H. Zhao, Study on characteristics of fire plume in building facade window under lateral blow. PLOS ONE, 2019. 14(11): p. e0225120. 48. Schifiliti, R.P., R.L.P. Custer, and B.J. Meacham, Design of Detection Systems, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 1314-1377. 49. Cheung, S.C.P., et al., The influence of gaps of fire-resisting doors on the smoke spread in a building fire. Fire Safety Journal, 2006. 41(7): p. 539-546.

251

Chapter 7. Case studies

50. Hopkin, C., M. Spearpoint, and Y. Wang, Internal door closing habits in domestic premises: Results of a survey and the potential implications on fire safety. Safety Science, 2019. 120: p. 44-56. 51. McDermott, H., R. Haslam, and A. Gibb, Occupant interactions with self-closing fire doors in private dwellings. Safety Science, 2010. 48(10): p. 1345-1350. 52. Giraldo, M., V. Rodríguez-Trujillo, and C. Burgos, Computer-Simulation Research on Building-Façade Geometry for Fire-Spread Control in Buildings with Wood Claddings. 2012. 53. genco, G., Municipal Building Surveyor report - Lacrosse Building Fire. 2015, City of Melbourne: Melbourne, Australia. 54. M. S. McLaggan, J. P. Hidalgo, A. F. Osorio, M. Heitzmann, J. Carrascal, D. Lange, C. Maluk, J. L. Torero, The Material Library of Cladding Materials, T.U.o. Queensland, Editor. 2019: Brisbane, Australia. 55. Chung Tsai, K.-. and D. Drysdale, Flame height correlation and upward flame spread modelling. Fire and Materials, 2002. 26(6): p. 279-287. 56. Holbrow, P., Explosion hazards and protection in the use of Intermediate Bulk Containers. LOSS PREVENTION BULLETIN-INSTITUTION OF CHEMICAL ENGINEERS, 2007. 194: p. 18. 57. K. McGrattan, B.K., S. Hostikka, J. Floyd, Fire dynamics simulator (version6), Users' Guide, N.I.o.S.a.T.R. (NIST), Editor. 2012. 58. Babrauskas, V., Heat Release Rates, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 799-904. 59. Kevin McGrattan, S.H., Randall McDermott, Jason Floyd, Craig Weinschenk, Kristopher Overholt, Fire Dynamics Simulator Technical Reference Guide, Volume 3: Validation, N.I.o.S.a. Technology, Editor. 2013. 60. Gottuk, D.T. and D.A. White, Liquid Fuel Fires, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2552- 2590. 61. Atkinson, G., Fire performance of composite IBCs. 2007, Health and Safety Laboratory: Norwich, UK. 62. Luche, J., et al., High-density polyethylene thermal degradation and gaseous compound evolution in a cone calorimeter. Fire Safety Journal, 2012. 54: p. 24-35. 63. Lauri, R., et al., Pool Fires: a Model for Assessing Meteorological Parameters Influence on Thermal Radiation. Chemical Engineering Transactions, 2017. 57: p. 229-234. 64. Benfer, M., Spill and burning behavior of flammable liquids, in Department of Fire Protection Engineering. 2010, University of Maryland: College Park, USA. 65. A. Buchanan, A.A., Steel Structures. 2nd ed. Structural Design for Fire Safety. 2016: Wiley. 154-194.

252

Chapter 7. Case studies

66. CEN, Eurocode 3: Design of steel structures ‐ Part 1‐2: General rules – Structural fire design, in 3.2 Mechanical properties of carbon steels 2005, European Commitee for Standarization: Brussels, Belgium. 67. LTD, Q.P. Safety Data Sheet - Unpigmented HDPE and MDPE. 2020 [cited 2020 07/10]; Available from: http://www.csinfosafe.com/CSIAU/SDS/SDSExternalView.aspx?ID=e627d119-4b33-4054- 9d50-b4a03f9903b7. 68. Sam L. Savage, M.T., SIP Math. 2012. 69. Devaney, S., Development of software for reliability based design of steel framed structures in fire. 2015, The University of Edinburgh. 70. Qureshi, R., et al. Effect of probabilistic strength retention factors for steel and concrete on structural reliability of columns in fire. in 3rd International Fire Safety Symposium (IFireSS 2019). 2019. 71. Drysdale, D., An introduction to fire dynamics. 1999: Wiley.

253

8 Lessons learnt

There is a drive to use fully quantitative criteria to demonstrate adequate performance, not only in fire safety but in all aspects of performance related to the built environment [1]. Chapter 3 discussed at length both the benefits and drawbacks of different types of fire safety performance quantification, particularly the one based on probabilistic risk assessments. That discussion concluded that relying on negligible likelihoods for feasible (yet unacceptable) outcomes does not constitute a valid premise nor and effective way to achieve a safer built environment.

On the basis of the identified limitations, Chapter 6 presented the Maximum Allowable Damage (MAD) methodology. MAD aims at providing an effective approach to guide the building design process by gaining insight on the potential consequences of a fire. This approach was put to test and iterated using three distinct case studies.

This chapter highlights the main lessons learnt from implementing MAD methodology to three case studies (Section 6.5 and Chapter 7). These lessons point out the advantages and challenges stemming from a consequence-driven approach to fire risk assessments (Research question 2) and discuss the role of MAD in guiding the building design process to achieve adequate performance (Research question 3).

8.1 MAD enables gaining insight, not producing numbers Risk assessments have their greatest strength in providing insight on the risks inherent to a design; this diverges from an intention to produce desired risk metrics below a certain threshold. Traditional approaches such as QRA/PRA in FSE are commonly applied for the latter purpose, at the end of the design process, and rarely used in a way that conveys real, practicable insight to guide the design process. MAD was built keeping in mind that the objective is not to produce numbers, but to gain insight.

The simplest implementation of MAD was the office building case study introduced along with the methodology in Chapter 6. The results of that case study provided an important insight into the capabilities of the methodology, as the building had failed a previous PRA. The results from implementing MAD showed that adequate performance could be achieved if the operating conditions are limited and controlled; alternatively, the design could also be modified to ensure a large enough available time for egress – ASET. The output from MAD is not just a number, but a description of the performance and the available options to enhance it or to prevent

254

Chapter 8. Lessons learnt performance from being unacceptable. This showed that MAD goes well beyond a pass/fail result and focuses on generating useful insight on building performance under fire conditions.

The usefulness of implementing MAD was confirmed with the second case study. In this case, the high-rise residential building was a more complex case, as the combustible façade posed an important challenge to the damage estimation. MAD was able to produce a detailed accounting of the variables (and the uncertainty sources) on which the performance quantification depends. The variables, in turn, were used to formulate a set of fire risk mitigation options. The results constitute a road map to deal with the complex issue of facades, were the inherently safer strategy is to use a non-combustible façade. Although ‘less combustible’ facades could be used in theory, the methodology clearly highlights the knowledge gaps to support such an action. Regardless of the remediation strategy chosen, the implementation of MAD highlights associated knowledge gaps that would require further research or monitoring.

A third case study differed from the previous in its overarching objective. In this case, the chemical storage warehouse had as objective property protection and business continuity. The failure of the warehouse roof was the limiting factor in business continuity and therefore it was the ultimate object of analysis. The results of the case study offer a detailed description on how to address the resistance of the beams to the fire by applying insulation layers or by using different beam sizes in future designs for similar warehouses. Moreover, the results indicate the possibility of mitigating fire risk by addressing simpler logistical aspects of the operation, such as the inventory of chemicals in the fire compartment. Once again, MAD provided a detailed insight on a set of (design and operative) recommendations to mitigate fire risk and attain adequate performance.

8.2 Fire risk assessments per se are useless if their output is not trustworthy When a risk assessment is limited to an exercise in number crunching, it could easily provide a false sense of safety. For risk assessments to be useful, their insight must be trustworthy, even when it cannot be accurate. MAD provides explicit evidence of the degree of trustworthiness of the fire safety performance quantification. Accounting for trustworthiness acknowledges complexities and uncertainties associated to fire dynamics and fire safety performance in general. This acknowledgement helps enhance the assessment through an iterative process, while allowing to communicate the degree of trustworthiness.

255

Chapter 8. Lessons learnt

Listing the variables involved in the damage model and classifying them provides the analyst with a tangible tool to identify assumptions and interrelated variables (sections 6.5.3 and 6.5.6). Based on the three case studies, each time the damage was estimated and a particular value or condition had to be defined, the question arose: “Is this contemplated in the information registry?.” A negative answer prompted its inclusion and the identification of any associated assumptions, while triggering the update of the damage model. Although no guarantee exists that the users of the methodology will use the information registry to achieve a trustworthy assessment, it still constitutes a tangible evidence for the trustworthiness of the results.

This iterative process drove all the three case studies presented here in a way that is hard to capture in print. Several versions of the damage model existed for each case study, and each iteration unveiled the need for more detail or the presence of uncertainties that could not be removed. A remarkable example is found in the second case study (section 7.1) in the external flame spread. This quantity largely defines the maximum damage potential and therefore the performance; however, the capacity to perform a phenomenological model of this quantity with any decent degree of precision is inexistent. This led to the decision to maintain this as a single value quantity, adopting the average and largest values recorded in a real fire for the performance assessment. Although conservative and highly uncertain, this decision provides a higher degree of trustworthiness than attempting a complex heat transfer for which little theoretical and empirical support exist.

8.3 Identifying scenarios in FSE requires a unique iterative approach Traditional hazard identification techniques originating in industrial processes have been recommended to identify fire scenarios; however these have not been formulated for the built environment nor take into account its inherent uncertainties and complexities. It is naïve and potentially negligent to assume that tools developed for a different problem apply and are adequate for FSE; in this sense it becomes necessary to redefine the way that scenarios are identified and developed within the context of quantifying fire safety performance. This has been addressed by MAD using deductive reasoning and the damage model to identify and iterate scenarios as to gain insight on the fire safety performance.

MAD does not pre-define scenarios, nor is there a unique step to define them. Instead, MAD enables them to emerge and evolve as part of an iterative process. First, the boundaries of the analysis and the actual physical phenomena are defined using the damage model (section 6.5.3). This is similar to the guidance provided by BS 7974:2019 [2], where the fire risk is divided in

256

Chapter 8. Lessons learnt

seven sub-systems, from which the analysis must make a choice. The damage model goes beyond and sets bounds for the potential scenario sets by defining the failure modes and key variables involved, i.e. the damage model provides an ad hoc structure to the problem. The scenarios elicited from the damage model enable a preliminary characterizing of the damage potential. These preliminary results provide insight on necessary adjustments and sensitivities to the damage model in order to explore more onerous scenarios. This is repeated until the upper end of the damage potential is characterized and the trustworthiness information indicates that no further analysis is practicable (section 6.5.6). Overall, this constitutes a deductive reasoning process to identify scenarios.

The deductive process allows for the evolution of the damage model and the scenarios identified. This incentives the practitioners to seek out further insight by incorporating more information or further detail in the analysis, e.g. in the third case study this resulted in the analysis of the spill thickness. In all case studies it resulted essential to use this iterative process and the feedback from the advisory team on the basis of preliminary damage estimations. In the first case study, the preliminary results indicated that the key variables defining ASET needed to be analysed systematically, leading to the exploration of hundreds of scenarios. The detailed results allowed pinpointing the necessary design and operation modifications to reach an adequate performance.

The complexity of the second case study did not allow for a systematic sensitivity analysis. Instead, preliminary damage estimation allowed identifying configurations of the damage model that resulted in low damage, e.g. effective door operation. This led to adjustment to the damage model to explore more onerous, yet feasible scenarios. In order to understand the inherent risk of the system and to quantify its fire safety performance, this exercise results in significant insight that in the case study guided the potential remediation strategies.

The third case study provided the most illustrative example of how to iterate the damage model in order to identify key scenarios that can actually challenge performance. A general damage model focused on verifying if the plastic moment capacity of the beam could be completely lost. This general model accounted for two very general scenarios, a first (short) pool fire at the chemical mixing area and a second rack/pool fire with a potentially large duration. The second scenario could be triggered by different initiating events, but in this case study, the preliminary results of the first scenario indicated that this was one of the possibilities. This result justified exploring the second scenario, which as opposed to the first, resulted in the

257

Chapter 8. Lessons learnt failure of the beam. At the end, the case study provides a comprehensive insight on fire safety performance and how to improve it both at the existing facility and potentially new ones.

In the end, the damage models used in all three case studies resulted from an iterative process, but are not perfect. This means that they can continue to be updated as additional modelling capabilities are available, or when new knowledge is incorporated. The information registry then enables a plug-and-play approach, in which coarse elements can be replaced by more detailed ones. This also opens the possibility of creating generic damage model libraries that can be continuously updated to reflect the most current knowledge (see possibilities of future work in the next chapter).

8.4 Transparency and accountability require explicit evidence As discussed in the previous point, trustworthiness is essential to effectively support decision making. The inherent uncertainties of FSE limit the capability to make accurate and precise performance calculations, which can lead to lack of transparency and accountability. MAD uses the information registry as a tool to catalyse scenario identification (see previous point) but also as a log. This log enables communicating the limits within which the assessment is valid and provides evidence for accountability.

Uncertainty sources are classified mainly as random variability (aleatory), lack of knowledge (epistemic) and vagueness; see Appendix 2 for the detailed classification. Chapter 5 concluded that a key aspect to produce high quality risk assessments (and therefore quality risk insight) is taking into account trustworthiness. The previous prompted the use of the information registry in MAD to help provide an explicit judgement of the trustworthiness of the results.

The implementation of MAD to all three case studied evidenced knowledge gaps and necessary assumptions that conditioned the results of the performance assessment; this information did not invalidate the assessments, but enhanced the obtained insight. The author considers this is a novel and extremely important advancement towards trustworthy fire risk assessments; an advancement which has already been identified in other fields [3, 4]. This differs from mechanized approaches, where the focus is put on achieving a desired numerical result, i.e. demonstration of compliance.

In the first case study, the key issue is the large range of possible fires and uses of the areas in the office building. Therefore, the key variables influencing the damage estimation were mapped using uniform distributions. This allowed plotting the limit state region as a function

258

Chapter 8. Lessons learnt of the ratio between available and required time for egress. The failure region indicated two possibilities: a modification in the design or a restriction on the operating conditions. Furthermore, a variable of concern is the potential fire growth rate of a fire, which was compared to reported data. The result indicated less than 10% probability of an unacceptable result, i.e. 1 in 10. Given this odds and an extremely limited dataset (a survey of N=10 buildings) indicate that an unacceptable result is indeed feasible. This information set as a whole provides the stakeholders with valuable information to decide over the restrictions on the building, which can then inform the fuel load limits and allow for prioritizing areas.

The second case study was a challenge on the concept of trustworthiness, as it included the complex combustion of external facades and its associated flame spread. Nevertheless, this complexity allowed to display the full potential of MAD in understanding uncertainty sources and assumptions. A key variable, the external flame spread rate, was used to estimate the resulting damage. With the original parameters of the building the result was an unacceptable performance. However, this allowed to reverse-engineer a solution, given by a maximum flame spread rate at each level. In this way, the weakness of this assessment was turned into an asset. The implementation of MAD and the estimation of the maximum flame spread rate can allow designers to prioritize materials and identify sections of the building where special safety measures are required (e.g. projections or fire stops).

The third case study allowed covering a large range of both operational and design aspects of a chemical storage warehouse. The trustworthiness information of this case study indicated the major relevance of the initial pool fire thickness and the effect of the design of the mixing area floor. Additionally, the inventory of containers in the fire compartment was found to have a significant effect on the resulting damage. This, being a variable fully in control by the warehouse management, represents a key element to prevent fire risk. Finally, the information registry also pointed out to the need to rethink the design of the roof beams, offering several possibilities that would enhance its performance even in the most onerous condition.

Overall, it is important to notice that none of the case studies, their damage models nor their estimated performances are perfect. This was never the point. Instead, the point was providing useful insight, which as discusses here requires also being trustworthy. The information registries (Appendix 4) constitute a valuable tool to assess potential improvement opportunities for future reassessments or for similar risk assessments.

259

Chapter 8. Lessons learnt

8.5 Using inherent safety to produce a safer built environment MAD is a proposal to improve the use of risk assessments for quantifying fire safety performance. However, some of its features are essential to achieve a safer built environment. From these features, the inherently safer design philosophy results essential, as it improves the fire safety performance of the base design and rationalizes the use of additional safety measures. Only what is truly needed is introduced in the design.

The author acknowledges that one of the less attractive aspects of MAD is the call for conservatism, as it can be seen as undue (see the discussion on conservatism in the next point). An example is the failure of the fire doors to work properly that results in loss of compartmentation. This can be interpreted as an extreme and unnecessary assumption. And yet, making such assumption allows evaluating the performance in case those doors do fail to work, as they have in catastrophic fire events of the recent past. Therefore, in this particular instance a seemingly ‘conservative’ assumptions results in a reasonable condition against which to test the performance of the system.

Such assumptions and even the conservative values of some variables enable MAD to characterize the damage potential of a bare-bones system. This is analogous to quantify the inherent risk of the system and it is an old and central concept in chemical process safety [5], but quite recent in FSE [6]. MAD takes important steps forward in implementing inherently safer design in FSE, as shown in all three case studies where the results address both the performance of the ‘bare-bones’ system as well as configurations in which necessary safety measures are included.

It is not the purpose of MAD to do away with safety measures, but to rationalize them and to establish that if any are necessary, the responsibilities associated to them being available and reliable need to be communicated as part of the results of the fire safety performance quantification. This results consistent with the intent of fully performance-based design, and contrasts with the current use of PRAs to quantify fire safety performance without being able to discern the inherent risk of the design. As evidenced in the first case study, by understanding the inherent risk, practicable actions can be recommended to drive the performance to an adequate level.

8.6 The degree of conservatism is largely dictated by the acceptance criteria In MAD, onerous scenarios are analysed as these tend to challenge fire safety performance (in contrast to optimistic ones). Since MAD favours a deterministic approach, the

260

Chapter 8. Lessons learnt

degree of conservatism can be deemed to be unknown. However, the acceptance criteria set by the stakeholders largely defines the degree of conservatism required. The application of MAD to the second and third case studies show that conservatism is used only as needed and is not an impediment to gain useful insight.

By setting a very stringent criterion such as ‘zero exposure to toxic gases’, MAD will require exploring very conservative conditions in order to demonstrate whether it can be accomplished or not. Accomplishing such a criterion with very optimistic or average input values will not provide insight on the performance of the system when unexpected conditions or ‘surprises’ occur. On the other hand, relaxing the criterion to several fatalities (most certainly not recommended) would allow for a lesser degree of conservatism.

Given pre-defined acceptance criteria, then the challenge is defining the conservatism of the input values and necessary assumptions. In MAD, both output sensitivity and strength of knowledge help the practitioner identifying whether the degree of conservatism is adequate; in cases where the trustworthiness of an assumption or a particular value is low, it is reasonable to explore the most onerous –yet feasible- values or conditions. However, this is also dependent on the preliminary results of the damage estimation.

The importance of preliminary results to gauge and calibrate the necessary degree of conservatism was exemplified by the second case study (section 7.1). There, the maximum damage potential corresponds to a series of assumptions, including a ‘normal’ occupation and an expected pre-movement time. In reality, these variables can take much more conservative values, as in the case of high occupancy during special dates, e.g. Christmas. However, the performance results unacceptable even under these optimistic values. Since the objective of the assessment is to provide a remediation strategy for the façade, modifying these variables would not yield an enhanced insight and are therefore not explored. From a design point of view, it would be reasonable to explore these more conservative conditions and address potential issues associated with large occupancies and evacuation times. In the case study, such conservatism was not necessary as the remediation strategy was limited to the façade.

The third case study started with a deterministic analysis, in which onerous conditions were tested to characterize the damage potential. After a deterministic performance assessment was conducted, the analysis was then extended to incorporate the variability of stochastic inputs related to the heat transfer to the steel beams (section 7.2.7). Incorporating these variables

261 Chapter 8. Lessons learnt increased the resolution of the results and supported the degree of conservatism in the initial performance assessment.

It is impossible to claim that the information registry and the trustworthiness information can ensure an adequate level of conservatism. At this stage, it also seems unreasonable to try to quantify said level. Both of these points highlight the importance of the practitioners’ role in conducting the assessment. This thesis has mentioned the challenges faced in FSE associated to competencies, and justifying the degree of conservatism is related to this issue. Ultimately, the practitioner must make an educated choice on when enough conservatism (or optimism) is granted; MAD does not address this, but does provide the evidence to judge the practitioners’ choices through third party reviews.

262 Chapter 8. Lessons learnt

Bibliography 1. Meacham, B., Ultimate Health & Safety (UHS) Quantification: Individual and Societal Risk Quantification for Use in National Construction Code (NCC). 2016, The Australian Building Codes Board. 2. (BSI), B.S.I., BS 7974:2019, Application of fire safety engineering principles to the design of buildings - Code of practice. 2019: BSI. 3. Aven, T. and V. Kristensen, How the distinction between general knowledge and specific knowledge can improve the foundation and practice of risk assessment and risk-informed decision-making. Reliability Engineering & System Safety, 2019. 191: p. 106553. 4. Aven, T., Supplementing quantitative risk assessments with a stage addressing the risk understanding of the decision maker. Reliability Engineering & System Safety, 2016. 152: p. 51-57. 5. Kletz, T.A. and P. Amyotte, Process plants : a handbook for inherently safer design. 2nd ed. ed, ed. ProQuest. 2010, Boca Raton: CRC/Taylor & Francis. 6. Østrem, L. and M. Sommer, Inherent fire safety engineering in complex road tunnels – Learning between industries in safety management. Safety Science, 2021. 134: p. 105062.

263 9 Conclusions and future work

FSE is moving increasingly fast towards the quantification of fire safety performance. This of course is a natural necessity stemming from performance-based design, an approach that provides a flexibility lacking in the more traditional prescriptive design (rule-based) approach. Still, the definition of what ‘performance’ means is unclear. This has led to the growing interest of the global FSE community on the application of risk assessments, which effectively enable risk quantification and therefore provide a measurable proxy for fire safety performance.

In the FSE context, using risk as a proxy to represent fire safety performance constitutes an axiomatic assumption. In this research, this assumption holds insofar that risk is defined only as a function of the potential fire consequences. Furthermore, it is the inherently safer design resulting from the Maximum Allowable Damage methodology that constitutes an adequately performing solution. This focuses the risk assessment process on gaining insight to guide the design process, rather than on achieving a particular numerical threshold. Chapter 1 concluded that no risk assessment is free of subjectivity, associated to biases and to uncertainty. From the latter, it is particularly important to highlight what is referred to as ‘unknown unknowns’; events, conditions and situations that practitioners simply do not have any idea that can come to occur. This conclusion makes it so that any methodology selected for risk quantification will present inherent limitations, either because of subjective judgements or simply because of knowledge that is not yet known.

Nevertheless, consulting the FSE literature and in particular its guidelines for fire risk assessments, it is clear that there is a strong drive of the discipline towards the use of probabilistic risk assessments (PRAs). This can be seen as one major paradigm for fire risk quantification and therefore fire risk performance. It is important to understand that PRAs are not just a simple risk assessment methodology, as by choosing it a risk definition is also chosen. This is a very specific definition of risk, one among many available and by far not the obvious one to choose in any engineering application, let alone in FSE. Employing PRAs, a large consequence can be tolerated or accepted given that its likelihood is lower than a pre-defined threshold, an idea largely discussed in Chapter 3.

A different paradigm for quantifying fire risk focuses only on the scenarios and the consequences. This paradigm lacks formal methodological support and is often associated to deterministic analyses. In these analyses, conservative scenarios are defined and their consequences estimated using conservative inputs and parameters. This raises the concern of

264 Chapter 9. Conclusions and future work

the degree of conservatism used, as it is not an easy feature to gauge with precision. However, when the objective is gaining insight on the effect of a fire occurring and not of its likelihood, this paradigm provides practitioner with a powerful tool. This research does not advocate for the undue use of conservatism. On the contrary, this work calls to let conservatism arise from the process of characterizing the damage potential of a fire; an example is found in the quantities used for quantifying the performance of the residential building in Section 7.2.

This thesis originated from the tension between these two fire risk quantification paradigms and the push towards the quantification of fire safety performance. Before reaching the conclusions of this research, it is worth noting that this is just a step towards a different approach to fire risk assessment. The following section offers a list of the additional work required to construct fire risk assessments that enable an safer built environment.

9.1 Future work Some of the key issues associated to fire risk assessments which have not been addressed in this thesis include stakeholder engagement, defining risk criteria and the use of fire risk assessments at late design stages. The following activities can build upon the outcomes of this research and maintain the general aim of producing trustworthy and insightful fire risk assessments.

9.1.1 Defining trustworthiness A large focus has been given to trustworthiness throughout this research work. In the context of this work, trustworthiness is understood as a synonym for credible, high-quality and reliable. In section 6.5.6, trustworthiness was defined as a function of the strength of knowledge and the output sensitivity. This was a practical choice and one supported by the existing safety science literature. However, it is clear that this concept needs to be formulated beyond the context of MAD.

As the purpose of the risk assessment is to inform decision-making, this implies that end users are the stakeholders. In FSE this means those proposing the design (client, designers) and those that approve it (AHJ). It is necessary to survey such stakeholders and elicit their understanding of what a trustworthy fire risk assessment is and where its importance lies.

9.1.2 Acceptance criteria resolution The acceptance criteria defines the performance assessment. A large discussion is presented in this work, demonstrating that acceptance criteria is far from a well grasped concept. Although

265

Chapter 9. Conclusions and future work

all guidance speak of ‘acceptance criteria’, its definition is usually oversimplified and limited to only a few of the types of criteria presented in section 3.5.1. As discussed at the end of Chapter 8, the author considers it necessary to explore the outcomes of MAD when varying the level of resolution of the acceptance criteria.

9.1.3 Organizational and human factors in the damage model In this research, fire safety performance was quantified using damage models that focus on the interface between the building, the vulnerable targets (e.g. occupants) and the fire dynamics. Nevertheless, the resulting fire damage can also be modelled as a function of different organizational factors that impact the design and/or the operation of the building. A clear example of this is poor inspection and maintenance routines for key safety measures, which directly condition the resulting damage of a fire. It is possible to study the influence of these organizational and operational factors using techniques such as Accimaps, STAM or FRAM (see section 6.5.3). The objective would be to quantify the degree to which these factors influence damage using existing case studies (e.g. incident investigations) and generate a connection between high-level and building/fire specific factors.

9.1.4 Damage models library Section 3.6 identified the limitations of scenario discovery, while sections 6.1.3 and 6.5.3 recognized the potential of formulating a damage model to enhance scenario discovery. The damage model is at the core of MAD and it is nothing but a conceptual representation of how fire damage materializes. This is where the complexities and nuances of combustion, compartment fire dynamics and heat transfer enter the risk assessment. In Section 6.3.3 it was recognized that currently there is a large range of techniques and tools that can help structure a damage model, and indeed there are several fire damage models available from literature.

An important follow up to this research would be constructing a library of damage models. This library would not only gather existing models, but would also define a taxonomy that allows classifying them based on the type of system they fit and the assumptions they hold. Such a library would be ideal for executing MAD and would also help any fire safety engineer in formulating their risk assessments.

9.1.5 Best practices for PRA The comparative evaluation of Chapter 5 provided a detailed analysis of the advantages and improvement opportunities for the existing guidance on fire risk assessments. This evaluation

266

Chapter 9. Conclusions and future work

was focused on the limitations identified in Chapters 1, 3 and 4 on current fire risk assessment methodologies. The results show that as a whole, the guidance covers most of the evaluation criteria, which is similar to results of other related evaluations. Nevertheless, there are some criteria that are not met at all such as those of stakeholder engagement, definition of a risk owner and data collection.

As different entities around the globe push for increased use of probabilistic risk assessments to demonstrate fire safety performance, there is a need for an updated and comprehensive guidance. This poses the need to formulate a new document in which the best practices for performing PRAs are gathered, and where its pitfalls are illustrated in detail.

9.2 Conclusions After presenting some of the remaining challenges in quantifying fire safety performance, this final section puts forward a series of conclusions. These conclusions address both the limitations with current practice, as well as the possibilities for a new approach.

Chapters 1 and 3-5 presented PRAs, how these can be implemented in FSE and the significant pitfalls they pose. These pitfalls greatly limit the potential of PRA to demonstrate adequate fire (Claim 1) and constitute the basis for the need to pursue a different approach. These pitfalls include:

• Assuming fire risk acceptability if the likelihood of a feasible and unacceptable consequence is low enough • Assuming that managing likelihoods is as effective as managing consequences in FSE • Ignoring the inherent subjective basis associated to fire scenario identification • Assuming that data for fire occurrence is applicable and enables predicting likelihoods of future fires • Assuming that reliability data is applicable and enables predicting the reliability of similar safety measures • Given valid reliability data existed, it would not reflect the reliability of the overall fire safety strategy • Focusing on the numerical results of the assessment, rather than using its insight to guide the design process • A PRA’s trustworthiness can be hard to gauge even for competent and experienced practitioners

267

Chapter 9. Conclusions and future work

Chapter 4 reinforced these pitfalls by presenting the fundamental differences of the built environment from that where PRAs originated (nuclear and chemical process safety). Chapter 4 also indicated how some of these pitfalls are still unsolved problems in these more mature fields. This raises the question of whether FSE is using the wrong tool, i.e. PRAs, to solve a very unique problem, i.e. demonstrating adequate fire safety performance. Past fire events resulting in unacceptable consequences support that PRAs are an inadequate tool for demonstrate adequate fire safety performance, particularly in a holistic manner.

A major pitfall for implementing PRAs in FSE are data limitations. Existing fire occurrence and reliability data in FSE might seem robust due to its magnitude, but this data only provides insight based on reported fires and none from those which are left out. This can lead to confirmation bias, as in the case of the ‘high’ reliability of some fire safety measures. Until the completeness and level of details are comparable to those in industrial processes, data will remain an impediment to achieve trustworthy estimates for fire occurrence and the reliability of safety measures. Monitoring and feeding back knowledge into the fire risk assessment can help screen unsupported assumptions and achieve more useful (and trustworthy) insight (Claim 5). Despite this limitation, the author does not claim that PRAs are not applicable in FSE. There are certainly specialized problems such as structural fire analyses in which PRAs can be successfully applied.

The Maximum Allowable Damage (MAD) methodology enables quantifying fire safety performance as a function of potential consequences while providing evidence for its trustworthiness (Chapter 6). Based on a deterministic and consequence-driven approach, MAD drives FSE practitioners to ask what scenarios can lead to an unacceptable damage, i.e. deductive reasoning. This contrast with the more typical approach of setting some initial conditions an enquiring about their consequences; if these consequences fail to challenge the performance they may provide a false sense of safety. If the fire safety performance is to be quantified, an important focus must be placed on those feasible conditions that can actually challenge it. Therefore, in MAD, there are no pre-defined scenarios. Using an abstract representation of how the fire damage materializes, i.e. the damage model, and the information registry as a log of variables and assumptions, scenarios are identified and preliminary risk quantification obtained. These enable updating the damage model, as well as the variables and models used. Such approach is open ended and propends to gain insight, rather than arriving to a particular numerical result (Claim 3).

268

Chapter 9. Conclusions and future work

Three case studies that enabled constructing, improving and testing MAD where presented in Chapters 6 and 7. The lessons learnt from these case studies were discussed in the previous chapter. These case studies showcase the importance of the structured process of MAD, as well as the steps in which creativity and iteration are required in order to characterize the damage potential. The solutions obtained in each case study characterized the inherent risk of the buildings and put forward actionable recommendations. Importantly, these recommendations accounted for knowledge gaps and important uncertainties that cannot be removed from the assessment (Claim 4). The three case studies provided valuable lessons regarding the potential for using MAD to design a safer built environment. All these lessons are consistent with a responsible approach to conducting engineering work, where risk assessments are used to gain insight rather than as a mechanized process.

The design drivers in the built environment are different from those in industrial processes. In the built environment these are functionality and aesthetics (sometimes more the latter than the former), while in industry it is safety, functionality and efficiency. The design, construction and operating practices are largely different as well. In industrial settings, for instance, a large –yet fallible- degree of monitoring and control, enables –also fallible- likelihood-driven risk assessments. Whereas in the built environment, varying degrees of quality, standards and recommended practices largely impede gathering reliable data. The existing differences between contexts do not grant the use PRAs to quantify fire safety performance in the built environment. If anything is to be adopted from the industrial experience in FSE, it is the importance of understanding consequences and implementing inherently safer design in order to avoid unacceptable losses. This research has provided and argument and evidence for why a consequence-driven approach such as MAD can pave the way to an inherently safer built environment. An added benefit to MAD is the enabling the responsible use of likelihood-based analyses such as PRAs. As a whole, this work provides the key elements to define a route-map for using risk assessments to quantify fire safety performance in FSE.

269

Appendix 1

Uncertainty analysis options for risk assessment Journal Paper

270

Appendix 1. Uncertainty analysis options for risk assessment

Uncertainty-based decision-making in fire safety: analyzing the alternatives

Jaime E. Cadenaa*, Andres F. Osoriob, Jose L. Toreroc, Genserik Reniersd, David Langee aThe University of Queensland, Crn Staff House rd, St Lucia, QLD, Australia, [email protected] bThe University of Queensland, Crn Staff House rd, St Lucia, QLD, Australia, [email protected] cUniversity College London, Gower Street, London, UK, [email protected] dTU Delft, Jaffalaan 5, Delft, The Netherlands, [email protected] eThe University of Queensland, Crn Staff House rd, St Lucia, QLD, Australia, [email protected] Keywords: fire risk, fire safety engineering, risk assessment, uncertainties, uncertainty analysis

Abstract

Large accidents throughout the 20th century marked the development of safety fields in engineering, devoted to better identify hazards, understand risks and properly manage them. While many such fields evolved rather quickly and moved from a compliance to a risk-based approach, Fire Safety Engineering (FSE) experienced a significant delay in this transition. While the current performance-based approach to FSE provides flexibility and capitalizes on learning from accidental events and other engineering disciplines, FSE is still in many respects decades behind other safety fields and a key issue for the future is how FSE can account for uncertainty in the decision making process. This work provides an overview of the main alternatives to account for uncertainty in safety studies within the context of FSE, including traditional probabilistic analyses and emerging approaches such as strength of knowledge. An example is used to illustrate the impact of the uncertainty analysis on the results of a simplified fire safety assessment. A structured evaluation is performed on each alternative to assess its ease of implementation and communication. The outcome is a compendium of advantages and disadvantages of the alternatives that constitute a toolbox for fire safety engineers to configure and use within their fire risk assessments. Process safety engineers are expected to gain an understanding of the similar and important challenges of FSE, it being directly relevant for process risk management and fire risk management in administrative buildings.

271

Appendix 1. Uncertainty analysis options for risk assessment

1 Introduction

1.1 Aim of this paper Safety engineering covers a large range of specialized disciplines, including nuclear safety, chemical process safety and security, disaster and emergency management, reliability engineering and fire safety engineering (FSE). The current approach to fire safety in the built environment places an undue focus on regulatory compliance, at the expense of demonstrating safety in the majority of cases [1]. Demonstration of safety requires a full consideration of the uncertainties inherent in a systems response as well as the response of the system to reasonable scenarios which may occur. Probabilistic risk assessments (PRA’s) are a common means to assess complex systems which are intended to account for uncertainties and evaluate the response of a system to various reasonable scenarios. Various approaches to risk assessment in performance based design in Fire Safety Engineering are evidenced throughout the literature [2-5], however their outputs can yield potentially inaccurate results due to a lack of appreciation of the extent and nature of the uncertainties involved [6].

Process safety and FSE, despite addressing hardly intersecting systems, share common challenges and potential solutions [7]. One of these challenges is how to account for and represent uncertainty in technical studies such as risk assessments. It is this challenge that is the focus of this work. Responding to a number of concerns identified in the literature, as well as concerns raised through discussions with experts in the fields of process safety and FSE this work presents a range of alternatives available for FSE practitioners to account for uncertainty.

This introduction presents the challenges of accounting for uncertainty in current risk assessment practices, having a clear precedent in those faced by chemical process safety. Section 2 presents the main uncertainty analysis alternatives found in literature and describing them, section 3 implements them to a simple fire safety example. Section 4 evaluates the alternatives on a comparative basis and section 5 discusses the FSE challenges in light of the evaluation results, where we also describe interviews with expert practitioners which support the concerns in literature. Finally, section 6 presents the conclusions of this work and its role in addressing current FSE challenges. Despite being target towards the FSE community, this work is also relevant to the chemical process industry since administrative and storage buildings constitute key elements of a chemical processing business. Fire risks in these facilities must therefore also be adequately managed to ensure process safety and business continuity.

272

Appendix 1. Uncertainty analysis options for risk assessment

1.2 Risk assessment supports decision making

Fire events in Table A1- show that complex systems can fail and lead to major loss for all involved stakeholders. Similar events have led to the development of safety engineering fields such as nuclear safety and chemical process safety. Safety engineering within its purposes to support decision-making and properly manage risk. The formal process to do so is a risk assessment, made of hazards (and scenarios) identification, risk analysis and risk evaluation. The input of this process is information about the system such as physical characteristics and hazardous elements, while the outputs depend on the definition of risk and the analysis method. In general, the outputs of a risk assessment is the prioritization of risks, identifying those which are not acceptable in relation to a pre-defined acceptance criterion. Such output depends on the risk definition choice. Diverse definitions are available [8, 9] and the one by ISO 31000 [10] states that it is the effect of uncertainty on objectives. Aven et al. [11] have stated that definitions of risk impact the way it is understood and evaluated. Detailed analyses of the ontological origin of the risk concept and its evolution throughout time are provided by Beck [12] and Blokland and Reniers [13]. A popular definition for risk which enable many traditional risk assessment methodologies is that of consequences multiplied by likelihood, i.e. expected value.

Table A1-1. Examples of catastrophic events Event Year Location Safety field Human Economic loss loss Collapse of World Trade 2001 New York, Fire, 2977 US$8B (buildings Centre [14] USA security fatalities value) [15] Fire at Grenfell Tower [16] 2017 London, Fire 72 fatalities £50M (renovations and UK enquiry) [17] New Zealand International 2019 Auckland, Fire - US$26M (liquidated Convention Centre fire [18] NZ damages over delays) Campbell chemical 2019 Melbourne, Fire, - Multi-million dollar warehouse fire Australia process clean up [19] Chemical warehouses fires in 2019 Dhaka, Fire 81 fatalities Unknown Dhaka Bangladesh [20]

1.3 Uncertainty in risk assessments

A recognized issue of risk assessment is the presence of uncertainty sources, which can not only be numerous but correlated in complex ways. Before the scientific foundations of risk

273

Appendix 1. Uncertainty analysis options for risk assessment

assessment were structured around 1945-1980 [21], many traditional engineering fields were already using its key features and managing the associated uncertainties. Structural engineering is an example relevant to FSE, where calculations needed to design a structure involve multiple complex variables and dependencies, which are modeled using well-known tools and parameters both of which involve considerable uncertainties, see Tallja et al. [22]. In this particular example uncertainties are identified and managed through ‘experience from practical analysis’ [22], i.e. expert judgment. As complexity grows, new approaches to account for uncertainties emerge, e.g. the comparison of the use of safety factors and reliability approach in structural design by Wang et al. [23].

Currently, risk assessment is recognized as a formal process and the core of the risk management process as defined by ISO 31000 [10] and Aven [24] presents a detailed analysis of its foundations, as well as of the way in which uncertainties are involved in the different steps. Using the example of the mature chemical process safety field, issues have been identified associated to the complacency of practitioners [25], the limits of prediction in risk assessments [26], the limits of the typically used probabilistic approach [27] and communicating uncertainties in the studies to the stakeholders [28]. In particular, Pasman and Rogers [29] conclude that risk analysts (engineers in charge of carrying out the risk assessment) are “haunted” by uncertainty while at the same time highlighting the vital role of risk assessments in supporting key decision making by stakeholders.

Safety science literature includes reviews on risk assessment methodologies, displaying approaches that adapt to different nature, complexity and magnitude of systems [30-32]. These reviews reflect not only the uniqueness and complexity of systems, but also the need to adapt the risk definition to better understand and manage uncertainty. Aven [27] clarifies that the purpose of risk assessment is not obtaining a risk index, e.g. Risk = Likelihood x Consequence, but obtaining ‘an objective description of unknown quantities’ or ‘a scientific judgement about the unknown quantities’ from the qualified safety engineers performing the risk assessment. A last and important consideration is that risk assessment foundations could lack coherence as they derive from reaction to catastrophic events and practical experience, rather than from a scientific approach [33]. This also poses a challenge to benchmark results and to unify guidelines. As highlighted by Aven and Kristensen [34] and the previously identified issues, it is a current priority of safety engineering fields to better understand uncertainty and its effect on risk assessments.

274

Appendix 1. Uncertainty analysis options for risk assessment

1.4 Risk assessment issues in fire safety

Most of the advancements of risk assessment have been undertaken in disciplines with a predominant performance-based approach, to which FSE has been transitioning since the appearance of the first performance-based construction codes, e.g. United States [35]. FSE has adapted risk assessment tools from other disciplines, however its framework does not provide the same basis for its structured and systematic implementation, as revealed by Hackitt [1], who in the aftermath of the Grenfell tower fire conducted an enquiry on the building safety provisions in the UK. An example is Approved Document A [36], which provides a wide range of accepted approaches to comply with the requirements of the building regulation, in particular those associated to the structural integrity of the building. Approved Document A requires an explicit and systematic risk assessment of buildings exceeding limits of area or number of stories. However, to implement such assessment there is limited guidance on the identification of fire scenarios for analyzing structural integrity [37]. The issues identified by Hackitt in the UK have also been identified in Australia [38] in the aftermath of less severe but equally concerning fires, such as the Lacrosse fire in Melbourne in 2014 [39].

The issues identified by the Shergold-Weir enquiry in Australia include lack of shared risk assessment practices, poor data collection and sharing, inadequate documentation (Recommendations 5, 8, 12, 14 [38]). The issues identified by the Hackitt enquiry in the UK include availability, completeness and updating of fire risk assessments, lack of a ‘building safety manager’, lack of a broad scope for risk assessments, management of changes and technical assumptions (Recommendations 3.2 - 3.4, 2.9, 9.3 [1]). These issues highlight a vast range of challenges, but are relevant here as many of them can be partially or largely tackled by implementing fire risk assessments where uncertainty is more clearly accounted for.

1.5 Uncertainty sources in risk assessment

The uncertainty sources in the context of performing a fire risk assessment are listed and described in Table A1-2, minding that others sources exist and can be useful in the right context, including volitional uncertainty [40], dependence [41] and the taxonomy of deep uncertainty (ranging from known-knowns to unknown-unknowns) presented by Walker et al. [42]. Such structure is a product of the recompilation of the definitions provided by different sources and many of the works referenced in this study. First, Notariani and Parry [43] highlight the role of uncertainty in a typical fire risk assessment, from which the potential for switchover arises. Switchover is the change of performance evaluation criteria due to changes in the

275

Appendix 1. Uncertainty analysis options for risk assessment

assumptions on which inputs and models are grounded. In simpler terms, when assumptions do not hold, the estimated performance might err in the unacceptable side and lead to design changes. This sets the main concern associated to poor uncertainty accounting in a fire risk assessment. Second, most PRAs are based on Kaplan’s quantitative definition of uncertainty [44], therefore the uncertainty sources in Table A1-2 were formulated to be consistent with the three variables that compose it: scenarios, probabilities and their consequences. Finally, Aven’s considerations of the different risk definitions make it clear that uncertainty accounting cannot be limited to a mathematical process and have to also incorporate ambiguity and vagueness, i.e. hard to quantify epistemic uncertainty sources [8].

It can be argued that uncertainty is only divided into epistemic and aleatoric and that linguistic elements are included in the former [45]; however, linguistic uncertainty as defined by Colyvan [46] is particularly relevant in the context of safety engineering fields. The use in FSE of risk acceptability criteria such as ALARP (as low as reasonably practicable) [4] and of performance-based requirements [47] which largely rely on linguistic elements, introduce this type of uncertainty which cannot be easily treated quantitatively. Johansen and Rausand [48] capture linguistic uncertainty within a broader category of ambiguity and provide a detailed accounting of how to identify and treat it within the process of a risk assessment.

Table A1-2. Uncertainty types in FSE Type of Epistemic Aleatory Ambiguity uncertainty Description Lack of or incomplete Natural variability Vagueness, context knowledge dependency, linguistic, under- specificity, normative Context Complex systems and Large populations Performance-based regulation, phenomena criteria and guidelines Reduction New or improved theories Statistical studies to Consensus and alternative methods based on experimental characterize probability interpretations observation distributions Residual Deep uncertainty Uncertainty associated Misinterpretation uncertainty after associated to complex fire to extreme variations reduction phenomena and interactions

A generic risk assessment process is shown in Figure A1-, along with the typical uncertainty types fed to it. Propagating such uncertainties can be a procedural issue if all are deemed epistemic and expressed using classical or Bayesian probabilities; however, the process shows

276

Appendix 1. Uncertainty analysis options for risk assessment that different uncertainty types are mixed along the process, backing up Colyvan’s [46] argument. Both Hackitt and Shergold-Weird enquiries [1, 38] highlight the need of reducing vagueness and/or ambiguity from performance requirements used to evaluate a fire safety in a building. Such ambiguity and vagueness is not communicated and it is rather carried along the risk assessment process without explicit consideration. Notarianni [49] also notices this when discussing uncertainty sources in the FSE design process by stating that ‘at present, performance criteria are not established or agreed on’. The relevance of this type of uncertainty in FSE is not clear as reflected by the uncertainty definition of the Society of Fire Protection Engineering (SFPE): ‘amount by which an observed or calculated value might differ from the true value’ [50]. This definition not only constitutes a narrow view of what risk and uncertainty are, but also greatly limits the capacity for practitioners to express uncertainty in studies where a probabilistic approach is not feasible or recommended.

Selected models Lack of knowledge Model Risk criteria parameters Lack of Subjectivity knowledge Likelihood and Variability Risk Outcomes & Inputs consequences Outputs Subjectivity evaluation recommendations estimation Lack of Variability knowledge Subjectivity

Risk communication

Figure A1-1. Mapping of uncertainties along a generic risk assessment process

As presented in Table A1-2, epistemic uncertainty can be reduced through acquiring new information, i.e. evidence. With enough additional evidence, epistemic uncertainty can be reduced enough to make analysts comfortable with the outcomes of their risk assessments. Evidence can either update or improve prior knowledge on something already known, e.g. updating probabilities, or reveal new knowledge that did not exist, e.g. new failure modes. Failure modes or faulty assumptions are typically identified when catastrophic events occur. Such uncertainty is referred to as ‘deep uncertainty’ or the highest level of uncertainty according to Walker et al. [42] and cannot be treated in a probabilistic manner as Kaplan once proposed [51]. In a practical sense, probabilities can only be estimated for events (and the associated failure modes) that analysts can foresee. Unknown scenarios product of

277

Appendix 1. Uncertainty analysis options for risk assessment

unrecognized failure modes or dependencies between existing ones, cannot be quantified and included in the risk description.

The variety of uncertainty sources and their type (including deep uncertainty) present a challenge for any risk assessment approach. Aven [52] provides a compilation of references in which risk assessments in general are deemed ‘simplistic and unrealistic’ and even misleading when the background knowledge of the analyst is poor. This draws relevance to this work and the identification of the different uncertainty analysis alternatives available for FSE practitioners.

1.6 PRAs in FSE

PRAs have their foundation in the Rasmussen study [53] and the theoretical basis provided by Kaplan [44]. Several references provide technical details of PRAs [54-57] and detailed examples include airports [58], liquid spill fires [59], natural events triggering technological events (NaTech; these are events where events such as floods interact with manmade systems to cause undesired events such as explosions) [60, 61], hydrogen refueling stations [62], land- use planning [63], railway tunnels [64], urban road tunnels [65]. In short, in PRAs risk is

defined as the triplet , where i refers to an integer number representing a given scenario (s), p refers to the probability of said scenario occurring and x to its consequences. Both Amundrud [66] and Zio [67] highlight that PRAs are useful and their major advantage being scenario identification and risk sources comparison. Both of these elements are significant challenges in FSE, as the scenario identification is conditions to the complex relationship between the fire dynamics and the building response, which is often an output of the assessment rather than an input. Furthermore, comparing risk sources require underlying statistics than enable benchmarking the contribution different hazards or threats that can lead to an unwanted event. These statistics in FSE are not comparable to the level of detail and maturity than those in chemical process safety, notwithstanding these have their own limitations [68].

The previous entails that applying PRAs to account for uncertainty in FSE is not an ideal solution. PRAs are applied in FSE through the BS 7974 [69] and rely on statistical data of fire ignition and probabilities of failure for safety equipment such as sprinklers or mechanical extraction. Similar approaches in process safety have already been analyzed after decades of implementation, highlighting the potential for large uncertainty margins which are seldom reported [70, 71] and requiring new risk assessment perspectives [34, 72-78]. Such

278

Appendix 1. Uncertainty analysis options for risk assessment

uncertainties threaten to render risk assessments useless in supporting decision-making and the new perspectives call for a need to understand them not as a mechanistic process, but as a complex evidence gathering exercise than effectively supports decision-making. FSE is not strange to this potential problems, as a probabilistic nature to fire risk is found in early fire safety engineering literature [79].

In the context of PRAs, Paté-Cornell [80] discusses how even a deterministic analysis can be useful in the right context. A deterministic analysis can provide insight on uncertainties given that it goes beyond a simple point-value result and includes variability in the form of error bars. Multiple deterministic models can provide a bound for the point-value estimate, and large random variability could call for fully probabilistic analyses. Regardless of the analysis approach, the need for uncertainty analysis is crucial and make necessary to know and weight the available alternatives to do so. These alternatives are introduced in the following section.

2 Uncertainty analysis options

The approaches available for safety engineers to describe uncertainty go well beyond probability [46, 81] and this section identifies the main approaches through a bibliographical search, using Dubois’s work [82] to guide the selection of search criteria (Table A1-3). The searches were conducted on Scopus® database which indexes key peer-reviewed literature much related to the mentioned safety engineering fields. The data was retrieved on the 13th of March of 2019 using a search timeframe between January 2000 and December 2018, limiting it only to journal articles. The table presents the search parameters, while Figure A1-2 presents the predominance of the probabilistic approach in journal papers production and the growing contribution of the other main alternatives.

Table A1-3. Searches and top results. *For the probabilistic, the results for ‘non- probabilistic’ were removed. Approach Number of articles (2000-2018) Top subject area Top author (No. publications) Probabilistic 749* Engineering (28.4%) Aven, T. (11) Bayesian 316 Engineering (26.4%) Khan, F. (6) Fuzzy 306 Engineering (27.8%) Huang, G.H. (23) Possibility 150 Environmental (21.0%) Zio, E. (5) Belief 82 Engineering (11.6%) Tesfamariam, S. (4)

279

Appendix 1. Uncertainty analysis options for risk assessment

Probabilistic Bayesian Fuzzy Possibility Belief 225

150

75 Articles per year

0 2000 2002 2004 2006 2008 2010 2012 2014 2016 2018 Year Figure A1-2. Results and annual growth for the main approaches to uncertainty analysis

This initial set of approaches to uncertainty analysis is further expanded by exploring selected papers. One of these papers is the detailed and well-structured description of the Generalized Information Theory (GIT) presented by Klir [83]. Another relevant work is the collation of imprecise-probability methods done by Beer [84], where the focus are problems where non- mutually exclusive sets are formulated (1 < , with i being the set or possible outcome).

Aven [85]provides a risk definition that accommodate∑ 𝑝𝑝𝑖𝑖 s these uncertainty analysis approaches beyond Kaplan’s original quantitative definition [44]. In Aven’s definition risk goes beyond scenarios (A) and their associated probabilities (P) and consequences (C), and replaces probability for an uncertainty descriptor (Q). This definition also accounts for the supporting background knowledge of the analysts (K), resulting in a definition of the form R = f(A, C, Q, K), which easily accommodates both probabilistic and non-probabilistic approaches. Abdo et al. [86] apply a variety of the approaches reported by the previous references to an atmospheric dispersion of a toxic gas, obtaining a direct quantitative comparison based on the likelihood measure they provide.

GIT is a useful basis to begin the introduction to the existing alternatives, as it focuses on describing the degree of evidence existing to estimate the true value of a variable of interest X within a specified set. A particular case of this is when the set is defined by the interval [0,1]. Klir calls these approaches uncertainty functions and include classical probability theory, possibility and evidence theory, and constitute the first approaches reviewed in this work, as presented in Table A1-4.

280

Appendix 1. Uncertainty analysis options for risk assessment

Table A1-4. Uncertainty analysis alternatives within this work General approach Specific approach Date Main feature Classical probability 1977 Additive, mapping onto [0,1] Possibility theory 1978 Uncertainty functions Non-additive, mapping onto Evidence theory 1976 [0,1] P-boxes 1987 Info-gap [88] 2005 Focus on robustness Robust Decision Making Strength of knowledge [34, 73, 76, 78] 2017 Assumption testing [87] Exploratory Model Analysis [89] 2003 Scenario exploration

Uncertainty functions provide a numerical outcome of uncertainty contained in all the information available -or the lack of it- and this is not always possible to map onto the prescribed range. Klir recognizes alternative theories can fit within the GIT as long as they allow mapping uncertainty into the defined interval [0,1] and highlights that the “choice of the uncertainty theory employed in dealing with each given problem should be determined by the nature of the problem itself” [83].

Beyond uncertainty functions, alternatives have arisen which focus on better supporting the decision-making despite the presence of considerable -unquantifiable- uncertainty both in the inputs used to define a scenario and in the assumptions used to model a system. Such alternatives are grouped under Robust decision making [87], which is defined is a framework designed to support decision-making under deep uncertainty. An example of these alternatives is the info-gap approach [88] in which the robustness is defined as an uncertainty horizon, i.e. a measure of how much uncertainty the system can resist. Such an approach helps engineers define satisfying performance requirements and allowing for variability in expected loads to the system.

The info-gap approach applied to structural problems with uncertain load variables [90] is in line with the performance assessment done by Cadena et al. [91] despite not specifying a robustness measure. In the latter, the assessment is accompanied by a judgment of its trustworthiness instead of a quantification of the robustness as in info-gap or of uncertainty as a probability. This trustworthiness judgment is done based on the approach formulated by Aven [34, 73, 76, 78] in which all quantities and associated assumptions within the model used are judged according to their strength of knowledge and output sensitivity. Exploratory Modelling Analysis (EMA) is another alternative that has been formulated under the Robust Decision Making framework. EMA makes use of computational power to run a large set of potential

281

Appendix 1. Uncertainty analysis options for risk assessment

realities (scenarios) and then provide the information as a whole for stakeholder to support their decision.

2.1 A note on inputs

The nature of the input is key to any risk assessment, and also to uncertainty analysis. The approaches considered in this work do not specify a particular type of input. In general, inputs can be classified as quantitative or qualitative. The former can be used in risk assessments if qualitative criteria are used to define the variables of risk, e.g. probability and likelihood. Qualitative inputs can also be transformed into the latter using fuzzy numbers, which are variables with a specific formalized language.

Fuzzy numbers can be used as inputs for most of the approaches presented in this work and have the added value of expressing the membership function of a quantity of interest within a specified range. This allows engineers to describe their knowledge about the quantity as a function of degree of belief, rather than as a crisp function. A review of the use of fuzzy numbers in safety engineering is presented by Kabir and Papadopolous [92], while Shi [93] presents a building fire risk analysis using fuzzy numbers. Fuzzy numbers also reflect the need for engineers to use additional information rather than just a crisp value.

Numeral Unit Spread Assessment Pedigree (NUSAP) [94] constitute another type of input. NUSAP is a notational scheme that presents a numerical result along with information about the measurement and systematic error (Spread and Assessment, respectively) and the quality of what the number represents (Pedigree). NUSAP aims to provide decision makers with a more complete image of what a numerical result entails by taking into account its uncertainty as presented by Ellis et al. [95]. Given that pedigrees are subjectively defined categories, this constitutes an issue in propagating uncertainty and favor approaches such as fuzzy numbers or the recent alternative proposed by Zadeh [96], Z-numbers.

In this work, none of these notations are prioritized, as we acknowledge that the type of number used depends on the knowledge and confidence of the analyst in defining a variable. For the rest of this work, typical quantitative inputs will be considered, but the previous referenced work present applications of the uncertainty analysis approaches using different inputs.

282

Appendix 1. Uncertainty analysis options for risk assessment

2.2 Probabilistic The idea of having a number that expresses how likely an event is to happen is very powerful and is in line with the intuitive meaning of likelihood [97]. The probabilistic alternative has as main characteristic that the possible values of the variable of interest X are assumed independent and associated with random variation. By specifying the evidence theory framework, more precise estimates can be obtained, but less information is provided. In particular, the information associated to knowledge -and lack of it- is not explicitly presented in the estimates and a sense of completeness of knowledge is associated to the estimates; this is not always the case though. There are three distinct alternatives for a probability analysis: classical, frequentist and subjective. The former is analytical and evaluates the expected result against all possible results, e.g. rolling a dice, usually of little use in risk assessments. However, in fire risk assessments there are variables for which a uniform probability distribution is assumed, i.e. classical approach.

Frequentist alternative is suited to problems where a fully analytical approach is not possible, introducing random variables which can be probabilistically analyzed. This approach usually begins with recorded data that represents the behavior of the system to later analyze it and estimate future states of the system. This is done through logical constructions and random sampling. An example of the databases used in frequentist approaches is loss of containment databases such as [98] in which oil & gas operators record and submit leaks and their main characteristics. Another relevant example of frequentist probabilistic analysis is the one carried out for the Cassini spaceship, where the aim was obtaining the probability of having a failure during the spacecraft’s flight and then a reentry to Earth [99, 100]. This is a case of interest given that the hazard related to the reentry to Earth was not the impact (with most materials melting in reentry) but the nuclear fuel. The report that describes the calculation [100] presents a detailed accounting of the diverse possibilities explored for a reentry to earth, which could only happen under very specific conditions which were deemed “highly unlikely”. In the Cassini case the assumptions are clear and explicit, being supported by the engineers to show they are reliable. Almost two decades ago Apostolakis [101] presented the issue of focusing on obtaining estimates of the parameters that define a probability distribution which allows to gain insight to support a decision and claimed that such practice should be avoided. Not long ago Young [97] advocated for the same argument when analyzing research results that are the product of a large number of regressions. Apostolakis concluded that researchers should not only report preferred statistics and regression, but the whole body of models that they used.

283

Appendix 1. Uncertainty analysis options for risk assessment

Evidently, all possible models cannot be considered and would not result useful either. Reporting on all models used could yield the technical analysis prohibitive from a time and economic resource point of view. In the context of risk assessment, the implications are similar when considering assumptions and associated uncertainty. The solution to this conundrum is a balance between detailed uncertainty description and professional competence to ensure the considerations of the study are adequate.

Subjective probabilities constitute a third approach, as mathematically structured by Savage [102] and also known as the Bayesian approach. This expresses the degree of belief towards a particular event from a set of states of the system, which can be ‘updated’ as new observations are obtained. Cooke [40] describes that this intuitive approach allows for ‘rational decision’ and the incorporation new evidence (e.g. additional relevant variables) from which knowledge increases and probability estimations are improved. In the context of fire risk assessments, the possibility of achieving such observations is not guaranteed.

Consider the design and construction of a mid-rise residential building which is intended for a life of no less than 40 years. Furthermore, consider that the architectural characteristics of this building do not match the boundaries of a prescriptive code and therefore a fire risk assessment must be done to explicitly demonstrate its safety level. Such assessment will depend on a large amount of variable from which information is limited and the best approach to account for them is the use of assumptions, as presented by Aven [34]. The updating of risk assessments (and validation of their assumptions) through the life-cycle of a system is addressed by Hackitt [1], who proposes a safety case approach for the approval process of high-risk buildings in the UK. In a chemical process plants requiring safety cases, dedicated stakeholders continuously work on the collection of new observations to optimize the operation and update the risk picture. This is not the case for most occupied buildings. An example is high-rise residential building where the building manager conducts administrative tasks to comply with code requirements, such as annual audits. However, there is not a figure that integrates, monitors and attend to all the components and systems that make up a fire safety strategy in a building. Throughout decades of research and unfortunate events, lessons have been learnt that improve this function, resulting in the identification and replacement of asbestos and the removal of fuel loads from critical locations such as fire doors. However, a fire risk assessment at the design stage is not typically updated later on unless significant changes occur. Even in this situation, there is not a framework that would call for the updating of the initial (a priori) probabilities embedded in a probabilistic fire risk assessment. The previous highlight that probabilistic –particularly

284

Appendix 1. Uncertainty analysis options for risk assessment

Bayesian estimates- risk assessments in fire safety should not be the only option to express uncertainty, despite highly valuable if the monitoring of the system produced the required data to update the estimates of probabilities.

Bayesian networks (BN) allow decomposing complex dependencies between multiple variables, representing them graphically through nodes and links. BN are based in Bayes theorem, using known probabilities of basic elements to estimate the conditional probabilities of dependent elements. This is a powerful use of known probabilities to model unknown probabilities in complex systems such as estimating the dynamic operational risk assessment in a chemical process setting [103] or estimating the risk of human fatality in building fires [104]. As this method is based on conditional probability theory, its weaknesses are similar. Probability distributions for the random variables in the basic nodes are required, which can introduce engineers ' biases or assumptions that do not correspond to the system's reality; their effect can only be accounted for if they are made explicit.

2.3 P-boxes This approach is the result of using probability theory and interval analysis [105] to produce probability boxes (P-boxes) which represent a class of distributions, instead of a single one. The class is a representation of the associated epistemic uncertainty and the variability (aleatory uncertainty). In typical probabilistic analysis, a variable of interest can be represented by a random variable with a given probability distribution function and its related statistics, but if this function is unknown, the interval approach can be applied to its distribution. By doing so the variable is circumscribed within upper and lower boundaries. Each boundary is also a probability distribution function and these can have a distinct shape, e.g. Exponential, or not, i.e. non-parametric. Whether the shape is well defined or not is a function of the available information to the engineers. This information is also the key of the simple but effective P-box approach, as well defined bounds for the variables of interest must be known in order to analytically obtain the upper and lower boundaries.

For a variable of interest X with known minimum and maximum bounds, e.g. [1, 2], the non- parametric p-box would be the one in Figure A1-3a, while if information about X allows assuming it distributes normally with bound on its mean and standard deviation, e.g. =[1,

1.75], σ=[0.1, 0.3] the result is as in Figure A1-3b. This evidently provides the probabilistic𝑋𝑋� approach with a higher flexibility, particularly when there is no confidence in point value estimates and the engineers prefer to express uncertainty as an interval.

285

Appendix 1. Uncertainty analysis options for risk assessment

Lower Upper Lower Upper Normal (lower ) Normal (upper) 1 1 0.9 0.9 0.8 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 Cummulative probabilityCummulative Cummulative probabilityCummulative 0.2 0.2 0.1 0.1 0 0 Value of X Value of X

a) b) Figure A1-3. a) P-box for known bounds of X, b) P-box assuming a normally distributed X with known bound on mean and standard deviation

2.4 Evidence theory This first approach, based on the Dempster-Shafer theory, allows using different sources of evidence to support an estimate of the degree of belief. In the context of risk assessment this degree of belief is associated to the true value of a quantity of interest (X). The uncertainty -or simply lack of knowledge- regarding the true value of this variable of interest is typically characterized in a probabilistic manner, as presented in section 2.2. In evidence theory the possible events -or values of the variable of interest- (x1, x2, …, xi) are not constrained to single events, but to all possible subsets of events. Each of this possible events provide the analyst

with a larger range of options to not only define a degree of belief of a single value, e.g. x3, but

to define the degree of uncertainty corresponding to combinations, e.g. (x1, x2).

The previous is a generalization of probability theory, where the latter have a probability of

zero, i.e. P(X = x1, x2) = 0. Evidence theory provides a platform to gauge the strength of

evidence by defining a basic probability assessment or masses for each subset, e.g. m(x1 x2) = 0.1. These masses are either obtained through expert elicitation or using the available data,∪ allowing to define the belief and plausibility measures (Bel(X), Pl(X)). These measures can be interpreted as upper and lower boundaries of the degree of belief, which supports making a

prediction between to possible values x1, x2 given that Bel(x1) is greater than the belief of all other possibilities.

286

Appendix 1. Uncertainty analysis options for risk assessment

Given the generalized form of this approach, it can be employed to process both probabilistic inputs and basic probability masses, as presented by Du [106] for the structural analysis of a machine. This provides an additional layer of information denoting lack of certainty regarding singleton values, which can be mapped into the same space in which probabilities are represented.

2.5 Possibility theory Possibility theory is closely related to fuzzy variables, as these allow translating qualitative expert criteria into numbers. As described by Dell’Orco [107] and Darby [108], possibility theory with its necessity and possibility measures constitute the application of Dempster-Shafer theory to a problem in which the possible sets of the variables of interest are nested, and therefore non-conflicting. This means that any possibility for the variable, includes or is included in another subset.

This alternative provides engineers with an additional tool to express uncertainty in a case in which no conflicting evidence exist, but not enough evidence is available to have completely exclusive and independent sets. The latter is the special case of probability. This has been found to be the case for a large set of studies in the field of process safety, in which possibility theory is used to compute the risk assessment of different experts for a set of scenarios.

Ouazari [109] presents the application to the Layers of Protection Analysis (LOPA), while Mandal [110] applies it to the Failure Modes and Effects Analysis (FMEA), both versatile and largely used process safety analyses. These examples show both the additional information conveyed by the results, in which risk is presented as a fuzzy number rather than a point- estimate (such a point-estimate can result from different de-fuzzification methods). In these examples the outcomes reflect the possibility and necessity measures, analogous to the previously presented belief and plausibility of belief theory.

2.6 Info-gap The approaches presented up to now rely on the concept of likelihood, degree of belief or directly to the probability. In these, the uncertainty of the analyses is provided within the output as in the case of evidence measures. Info-gap is an alternative approach which was first proposed by Ben-Haim [88]. Info-gap’s name points to the fact that there are information gaps within our models and therefore uncertainties that we need to manage, even in the extreme case in which the uncertainty level is so high that we cannot use one of the previously presented approaches.

287

Appendix 1. Uncertainty analysis options for risk assessment

Hayes [41] describes info-gap as one of the main approaches to deal with uncertainty and highlights its non-probabilistic nature. This approach analyzes uncertainty from the perspective of how much uncertainty can the system handle, as a function of variations in the inputs and the parameters of the models used. This implies a considerable knowledge of the system and a model or set of models that allow obtaining an initial estimate, which in typical design approaches is used to obtain an optimized solution. Such solution is supported by a set of assumptions that in reality might not hold which if why info-gap provides a framework to test the resistance of the system to changes in them.

Ben-Haim [111] exemplifies info-gap as a framework to establish a safety factor on the results of the system. This non-probabilistic approach -despite its requirement for considerable high levels of knowledge- provides a different approach to uncertainty. Such an approach provides a solution that acknowledges the lack of knowledge and the impact this has on the system’s response, instead of a frequency, probability or degree of belief.

2.7 Strength of knowledge The idea of designing a system that involves uncertainty without addressing it as a probabilistic problem was presented with info-gap, but this is not the only approach. Strength of knowledge is another one of these approaches. It aims at identifying and managing assumptions and limitation used in an analysis and which could significantly affect its outcomes. Based on the studies published by its authors, Strength of Knowledge can be applied in different manners and it is flexible. In fact, it can be seen as a very general idea which is then tailored to suit the specific features of each problem, as presented in [72, 74, 78, 85, 112].

In a typical risk assessment -either in chemical process or fire safety- assumptions usually are found everywhere, beginning with the information defining the system, then the construction of the scenarios, the models to estimate risk indices and to evaluate them. The general idea of Strength of Knowledge is to identify key assumptions made within a risk assessment and then judge whether they need to be addressed before communicating the results to the stakeholders. To judge the strength of knowledge a set of ordinal categories of qualitative or semi- quantitative nature are defined (e.g. low/medium/high) with their corresponding criteria, which are then used to judge each assumption.

The supporting knowledge for an assumption is not the only aspect that can impact the outcomes, as the sensitivity of these to a variation in the inputs or in the assumption is also key. This aspect is also judged for each assumption using another set of ordinal categories, which

288

Appendix 1. Uncertainty analysis options for risk assessment

can be defined with purely quantitative criteria based on sensitivity ranges on the outputs. Qualitative criteria can also be used for assumptions that cannot be quantitatively assessed, requiring a competent expert formulating them and overseeing the judgment. The results can help identify a narrow range of potentially problematic assumptions requiring treatment.

This treatment can be part of the risk assessment itself, such as further probabilistic analysis or the use of imprecise probabilities (e.g. evidence theory). If the resources are exhausted and at the end of the assessment there are still critical assumptions, these can be part of the decision making process and lead to establishing monitoring, control and mitigation measures that account for them not holding during the operation of the system.

2.8 Exploratory Model Analysis Acknowledging the presence of deep uncertainties at different levels within and around a system, there is a need for uncertainty analysis alternatives that move further away from prediction and grow closer to the concept of robustness already introduced with the info-gap alternative. Exploratory Model Analysis presents a framework in which models are no longer used as the tool to find the exact answer or in the case of risk assessments, a failure prediction. Instead, models are recognized as the flawed constructions that they are and this is instead used to explore a wide range of plausible worlds, i.e. scenarios, and assess the response to particular decisions made by the stakeholders. In a risk assessment this means exploring how a large set of possible scenarios would react to different risk management strategies and decisions.

EMA can remove the mentioned burden on engineers and offer simpler approach to make the best out of the available models. Based on the work by Kwakkel [113], EMA can be described as in Figure A1-4 where the possible worlds are tested using different policies based on a model for the system. This constitutes a single computational experiment and each variation in the scenarios or the policy used generates different outcomes of interest. This differs from a typical engineering analysis in that all uncertainty sources are explicitly considered as part of the scenario, and allows analyzing the relation between these and the outcomes to identify those that produce higher sensitivity or breakpoints. Such process is the vulnerability analysis, which is complemented by the robustness evaluation of the outcomes that comes close to the robustness concept of the info-gap theory.

289

Appendix 1. Uncertainty analysis options for risk assessment

Vulnerability Policy options analysis

L1 … LM

Robustness evaluation Policy levers (L)

States of the world, i.e. scenarios Outcomes

x1 M1

Relationships in the Uncertainties (X) … system (R) …

xN MK

Computational experiment

Figure A1-4. EMA process and main components based on [113]

EMA can be interpreted as an intelligent exploration of unknown realities using imperfect models, from which a prediction is impossible. The value of this approach is that it allows providing the key stakeholders with a comprehensive set of results that identify a policy that results in successful outcomes based on a given set of conditions within the scenarios. This is clearly and brilliantly exemplified by the implementation of EMA for the policy selection of water management of a river in the United States [114]. Given the simplicity of the compartment fire example EMA is not implemented as it would require increasing the complexity of the system to introduce the possibility of testing different risk management strategies.

3 Case study: compartment fire

To exemplify the differences between the presented alternatives a typical simple FSE example to illustrate them; both possibility theory and EMA are excluded. The former is excluded on the basis of its reliance on the additional fuzzy input from multiple practitioners, while the former is excluded as the simple nature of the example used does not support a wide range exploration as EMA intent.

The example used is the calculation of the descent of the smoke layer in a simple rectangular compartment of length 7 m, width 5 m and height 3 m (total floor area 35 m2) with a door of

290

Appendix 1. Uncertainty analysis options for risk assessment

width 1.5 m and height 2 m. The compartment is of residential nature and expected to have a representative fuel load of polyurethane (PU) which is initially taken to have a distribution of 600 MJ/m2 [115]. Once ignited, the fuel is modelled with a heat release rate of 400 kW/m2 and a medium rate of fire growth (α = 0.0117 kW/s2). These two variables set the fire area and the growth of the fire, respectively. The fire is allowed to grow following a time-squared function until reaching the peak fire size.

Using the set of equations of Appendix 1 based on energy and mass conservation, as well as some key experimental correlations, a suitable model is constructed to perform the calculation. The model is applicable for pre-flashover conditions and limits the peak fire size to 1 MW, which is one of the assumptions also registered in Appendix 1 and discussed in the next sections. With a critical height established at 2 meters, the key output of the model is the time for the smoke layer to reach this level.

Assuming the previous input values without any uncertainty and applying them to the model, the result obtained is 58 s. Such an output is key for available vs required egress time calculations in performance-based analysis and it is discussed related to the outcome provided by each approach and its advantages and disadvantages.

3.1 Probabilistic To illustrate a simple application of the probabilistic approach, we use the compartment fire example. Not enough information exists to define the exact value of the fire growth rate, which can range from a slow growing fire (α = 0.00293 kW/s2) to an ultra-fast one (α = 0.1874 kW/s2) [116]. This leads to the engineers to formulate alpha as a continuous random variable, which according to Nilsson [117] distributes as a log-normal function with a mean of 0.01924 kW/s2 and a specified 99.5th percentile of 0.219 kW/s2, which are the result of analyzing 2965 fires. By using a simple add-on to Microsoft Office Excel known as SIPmath [118], a Monte Carlo simulation is performed in order to sample the distribution as described by Figure A1-5. After trialing different number of realizations, ten thousand was chosen as the first momentum of the distribution is stable at this point, noting that additional criteria can be used such as the second momentum.

The obtained samples are the input for the compartment fire model in which the time for the smoke layer reaching 2 meters can be calculated, yielding the results presented in Figure A1- 6. The height of 2 m is chosen as it is above the average height at which occupants breath [119]. From this output it is possible to obtain the time for untenable conditions and therefore the

291

Appendix 1. Uncertainty analysis options for risk assessment available time for egress in the compartment, which for a 50th percentile of 53 seconds and a 99th percentile of 40 seconds. This indicates that there are fire scenarios in which extreme (but not impossible) scenarios can yield 18 seconds less than the previous output which did not account for uncertainty. Likewise, there are scenarios in which the time is as large as 89 seconds, which if taken into account could lead to underestimate the risk. These results do not account for particularly vulnerable occupants, such as those with movement impairment or simply those who are not awake and alert. This is part of scenario definition and becomes a limitation of the analysis, not always explicitly addressed. This is further discussed in the alternative presented in section 3.5.

Frequency Cumulative %

6000 100% 90% 5000 80% 4000 70% 60% 3000 50% Counts 40% 2000 30% 1000 20% 10% 0 0%

Range of Alpha [kW/s2]

Figure A1-5. Fire growth alphas sampled for input of the probabilistic analysis

Po 100% 90% 80% 70% 60% 50% 40% Probability 30% 20% 10% 0% 0 20 40 60 80 100 120 Time to reach 2 m [s]

Figure A1-6. Resulting distribution for descent time

292

Appendix 1. Uncertainty analysis options for risk assessment

3.2 P-boxes To construct the P-boxes for the compartment fire example, a range of distributions replaces a single probability distribution. This allows incorporating uncertainty on the distribution parameters, which for the case of the compartment fire and the time for the smoke layer to descend to 2 meters height are given by the 50th and 99.5th percentiles of the lognormal distribution that described the fire growth coefficient, alpha.

Figure A1-7 presents the resulting P-box having Po, the previous single distribution, P1 a and

P2. The new distributions respond to the analyst considering different conditions than the ones

initially introduced in the previous section. P1 rules out possible arson in the compartment, which leads to a lognormal distribution with 50th and 99.5th percentiles of 0.011 kW/s2 and 2 0.105 kW/s . P2 is a pessimist view of the analyst, based on the fact that the data used for the previous distributions do not match exactly the nature of the compartment.

Po P1 P2 100% 90% 80% 70% 60% 50% 40% Probability 30% 20% 10% 0% 0 20 40 60 80 100 120 Time to reach 2 m [s]

Figure A1-7. P-box for the descend of the smoke layer height

In order to present a more onerous scenario, the parameters of Po are doubled, resulting in a lognormal distribution with 50th and 99.5th percentiles of 0.0385 kW/s2 and 0.438 kW/s2. Here, specifying a range becomes harder, as now there are three distributions for which the results could be reported. Using the results of Figure A1-7, the original result of 58 s has a 45%

probability in the pessimistic P2 distribution, which could lead analyst to further hesitate in using such result. This of course translates into confidence boundaries and is an ideal tool to better understand the uncertainty of the calculation. With occupants’ walking speed potentially under 1 m/s given they are sleeping or have a disability, the result of the P-box could help identify potential improvements to reduce egress path length.

293

Appendix 1. Uncertainty analysis options for risk assessment

The disadvantage for the use of such results is that their communication can mislead stakeholders in believing that the worst possible conditions are covered, which might not be the case depending on the engineers’ trustworthiness on their assumptions such as the

pessimistic parameters of P2. As additional data becomes available such as that presented by Hopkin et al. [120], the possibilities for incorporating different conditions further enable the use of P-boxes.

3.3 Evidence theory Back to the compartment fire example, the evidence theory is applied to conflicting information regarding the fire growth rate. The fire engineers are unsure of the correct rate to use in the calculation as the compartment’s lining materials are not specified and can contribute to different fire growth rates. This provides a considerable large range of possible fire growth rates, leading to the use of probability masses for the slow (S), medium (m) and fast (F) rates. The frame of discernment is therefore = { , , } and a power set:

2 = {Ø, S, M, F𝜃𝜃, (S, M𝑆𝑆)𝑀𝑀, (S𝐹𝐹, F), (M, F), (S, M, F)} 𝜃𝜃 For these eight focal elements, the two experts are asked to provide the basic probability assignments with m(Ø)=0, which is presented in Figure A1-8.

1 1 0.9 0.9 0.8 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 cummulativeprobability cummulativeprobability 0.1 0.1 0 0 0 0.02 0.04 0 0.02 0.04 α [kW/s2] α [kW/s2]

Figure A1-8. Dempster-Shafer structures for fire growth rate based on two experts’ opinion

Disparity between the experts in their opinion is evident, which is common in the context of risk assessments [121] and could result in enhancement of the results [122]. Particularly, discrepancies might be found between third party reviewers and the opinion of the fire services, which might want the consideration of more onerous fire scenarios. By applying Dempster’s rule of combination, these two independent structures can be combined and the resulting

294

Appendix 1. Uncertainty analysis options for risk assessment

structure used to estimate the quantity of interest, i.e. time for the smoke layer to descend to 2 m height.

The results for the resulting structure for the quantity of interest are presented in Figure A1-9. The results indicate that this time could be in the range between 41 and 17 seconds, with the former having a 40% likelihood and the range between 23 and 17 seconds one of 24%. Although this information -as with P-box analysis- does not provide a point-value answer, it enables combining evidence provided by multiple experts and obtaining a simple but clear result which could support the need to explore further design fires.

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 cummulativeprobability 0.1 0 37 27 17 Time to reach 2 m [s] Figure A1-9. Dempster-Shafer structure resulting for the time to reach 2 m based on two experts input

One possible application of the Dempster-Shafer theory is using structures such as those in Figure A1-8 to construct probabilistic distributions and then assessing the belief and plausibility of a whole family of these. Such application can again be applied to our example, this time moving away from different engineers providing evidence, to a single one which provides a belief structure for the parameters of the lognormal probabilistic distribution considered in sections 2.2 and 2.3. The analyst defines the ranges for the parameters of the distribution as presented in Table A1-5, which are computed using the IP Toolbox add on for Matlab and yields the results of Figure A1-10. This graph provides the result for the belief and plausibility measures for the evidence provided and it is compared to the possible P-box resulting from using the extreme values of the ranges of Table A1-5.

Table A1-5. Evidence for constructing the distribution of the fire growth rate µ (kW/s2) σ (kW/s2) Basic probability assignment [0.01, 0.03] [0.03, 0.04] 0.9 [0.03, 0.05] [0.04, 0.1] 0.1

295

Appendix 1. Uncertainty analysis options for risk assessment

Figure A1-10.Belief and plausibility measures for the lognormal distribution with uncertain parameters

The comparison shows that indeed the evidence measurements provide additional insight to the possible ranges in which the ‘real’ distribution might be found, which are much narrower than those provided by the P-boxes. Using these results, the time for the smoke layer to reach 2 meters height can be estimated for the 50th and 95th percentiles, yielding the ranges [25 s, 32 s] and [22 s, 25 s], respectively.

Although the example presented does not include significant conflicting information, the Dempster-Shafer theory does allow for the inclusion of this type of conflicting evidence [107]. The main takeaway from this example is the possibility of narrowing down on specific scenario sets using subjective inputs and the providing a quantitative description of the likelihood, including an upper (Plausibility) and a lower bound (Belief).

3.4 Info-gap Ferson and Tucker [123] provide an understanding on how to implement the info-gap approach using probability bounds, i.e. P-boxes. Info-gap is implemented following the safety factor concept. Specifically, four alternatives are presented and info-gap theory does not prescribe any particular one, as the analyst must decide which approach better fits each system. The approaches presented by Ferson and Tucker are practical and require a robustness measure

(named alpha, for clarity referred here as αR) which is applied for the inputs of the system and allow identifying the level at which the system’s response is no longer acceptable. This measure is applied to confidence bounds, e.g. Kolmogorov-Smirnov for empirical distributions, proportional bounds, distribution shift and a validation metric.

296

Appendix 1. Uncertainty analysis options for risk assessment

The previous ideas are in line with that of Cadena [91] when assessing fire risk through the concept of maximum allowable damage in a building for life safety, as an alternative to typical probabilistic approaches. These approaches are applied to the fire growth rate distribution presented in section 2.2 from the compartment fire example. First, an exponential function is defined which increases the P-boxes proportionally to the value of the fire growth rate (Figure

A1-11), followed by multiplying αR and the critical value of the Kolmogorov-Smirnov test to the nominal distribution, both for a confidence of 1% and 20% (Figure A1-12); the latter provides a wider P-box.

Figure A1-11. (Left) Info-gap concept applied to P-boxes through proportional uncertainty

Figure A1-12. Application of info-gap with Kolmogorov-Smirnov confidence bounds and αR between 1 and 20

Applying the αR to the nominal distribution original range and generates a simple but effective set of inputs, given that a fire growth step is established (a value of 0.01 kW/s2 is selected). With this simple application of the robustness measure, the time for the smoke layer to reach 2

297

Appendix 1. Uncertainty analysis options for risk assessment

m can be tested until it reaches a critical value, defined as the time it takes for an occupant to egress from the room. Based on the maximum path distance of 12 m within the room and a

walking speed of 0.7 m/s, this critical time is 16 s. A value of αR > 17 is required for the fire growth rate to reach ultra-fast (~0.19 kW/s2) and yield a time of 16 seconds for the smoke layer

to descend to 2 m height. As the value of αR increases, so does the confidence on the result,

which goes from 5% for αR = 16.95 to 90% for αR = 17.17 (Figure A1-14).

Figure A1-13. Simple application of info-gap to generate fire growth rate P-boxes

This outcome would provide the designers of the building with an understanding of the robustness that can be expected from the compartment, which can then be extrapolated to the whole building. Although the result for robustness is accompanied by a notion of likelihood or chance, it is important to understand that info-gap analysis mainly focuses on the potential for the system to resist a load, rather than on the likelihood of said load actually occurring.

The application of info-gap theory and the robustness function in this example is simplified to provide the reader with a clear understanding of the potential for its use in fire safety. As complexity increases, info-gap robustness measurement allows evaluating two competing design alternatives which could imply significantly different trade-off in a fire safety strategy within a building. Such an evaluation is objective and is not based on the likelihood of an unknown event or condition, but on the amount of uncertainty that the system can tolerate and still perform adequately.

298

Appendix 1. Uncertainty analysis options for risk assessment

100%

80%

60%

40%

20% Percentage16 s time>of

0% 16.9 16.95 17 17.05 17.1 17.15 17.2 αR

Figure A1-14. Likelihood of time descending to 2 m being less than 16 seconds; 5% chance

at a value of αR = 16.95

3.5 Strength of knowledge (SoK) This alternative is applied to the compartment fire example, which begins by defining the levels and criteria for the strength of knowledge and output sensitivity. The former is adapted from previous work, as strength of knowledge criteria is largely compatible with the features of this example. For output sensitivity, the criteria are defined based on the variable of interest, i.e. the time for the smoke layer to descend to 2 meters height, compared to the time required for occupants to egress the compartment, which is 16 seconds based on the calculation of the previous section. The criteria for both aspects are presented in Table A1-6 and are applied to judge the list of assumptions within the analysis. The six assumptions and their SoK assessment are presented in Table A1-7, from which a wide range of potential issues are identified with the analysis.

Table A1-6. Strength of knowledge and output sensitivity levels and criteria Aspect Level Criteria -Updated references back up the values or assumptions High -Strong and relevant theoretical grounds -Subjective knowledge is backed up by theory or robust research Strength of Medium Neither high nor low knowledge -Poor theoretical grounds and references for the values and assumptions Low -Low consensus between personnel involved in the assessment -Knowledge sources are subjective and not validated Variations within known ranges yield shorter available times than those High Output sensitivity required Medium Neither high nor low

299

Appendix 1. Uncertainty analysis options for risk assessment

Aspect Level Criteria Large –unrealistic- variations required to yield shorter available times than Low those required

From the list, all assumptions have the potential to influence the result, but those with lowest SoK and highest output sensitivity (OS) can be prioritized. This makes assumptions 3, 5 and 6 stand out and require a close analysis. Assumption 3 questions the validity of the chosen un- tenability criterion selected as a function of smoke height layer. This potential issue with the performance criterion itself is treated by analysing the output of the compartment fire model used (Annex I) and comparing both criteria.

The outcomes of the comparison are provided in Figure A1-15, showing that smoke height layer is a more conservative criterion. Based on this result, the SoK and OS are assigned new values (High, Low) and is no longer a pressing concern for the output of the analysis. The remaining two assumptions, No. 5 and 6, point towards unreliable inputs that can be addressed in a simple and effective manner. Assumption 5 points out that no detection, notification and pre-movement times are considered for the value of 16 seconds representing the required safe egress time. A pre-movement time that cannot be exactly pre-defined. A change in occupants condition (e.g. from awake and alert to asleep) could translate to lack of recognition of the fire and detection times in the range of several minutes. This could also be the case if the flaming fire was preceded by a previous smouldering phase where toxic gases are released. In conclusion, both assumptions remain unchanged and constitute an important source of uncertainty which can increase the previously calculated 16 seconds, to times of 46 seconds or more (using a pre-movement time of 30 seconds based on the SFPE Handbook data [124]).

As pre-movement time increases due to lack of adequate means of notification or lack of understanding of the occupants due to low familiarity, the system reaches an unacceptable performance with lower fire growth rates. This evidently echoes the robustness approach of the info-gap theory, but also implies a much simpler course of action. Despite the lack of quantification, here it is easy to identify it is essential to ensure proper notification and a delivery of clear instructions to the occupants in order to maintain the system’s performance.

Table A1-7. List of assumption and SoK judgements Assumption SoK OS Justification 1. Fuel is polyurethane Medium Medium Representative of the fuel in a typical dwelling foam

300

Appendix 1. Uncertainty analysis options for risk assessment

Assumption SoK OS Justification Despite fire spread and growth having a complex 2. Alpha t-squared fire High Medium behavior, in a dwelling this assumption is reasonable as growth is valid long as different scenarios are analyzed Smoke layer temperature could reach a critical 3. Smoke layer height Low High temperature before the smoke descends to 2 m height, thus criterion is valid (High) (Medium) yielding untenable conditions before it Natural ventilation like doors and windows have a soffit 4. No ventilation effect below 2 m, therefore assuming it does not influence the High High on smoke layer descent smoke layer descent is onerous but valid for an initial analysis 5. Detection, The occupants will react to the fire within the notification and pre- Low High compartment and immediately evacuate movement times = 0 s 6. Walking speed = 0.75 Based on SFPE data [124] this is a slow speed, providing Medium High m/s an onerous scenario for this simple analysis

H [m] Hcrit Th [°C] Tcrit 3.5

2.5

1.5

Smokelayer height [m] 0.5

0 0 10 20 30 40 50 60 70 80 Time [s]

160 C]

140 ° 120 100 80

60 40 20 Smoke layer temperature [ 0 0 10 20 30 40 Time [s]

Figure A1-15. Tenability criteria comparison for slow (left) and ultra-fast (right) fire growth

301

Appendix 1. Uncertainty analysis options for risk assessment

4 Alternatives evaluation

The previous section presented a description of the main uncertainty analysis options, including a brief description. In this section an analysis of these alternatives is provided, which aims at helping risk analysis to select one of them according to their needs. To conduct the evaluation, the main steps of the Design Science Research Methodology (DSRM) [125] are adopted (Table A1-8).

Table A1-8. DSRM steps applied to this work DSRM step Application to this work Identify the problem and Better risk understanding through the use of appropriate uncertainty analysis motivate options (Section 1) Provide a clear picture of the uncertainty analysis alternative for fire safety Objectives of a solution engineers Evaluation of alternatives Use of hierarchy structure to perform the evaluation (Figure A1-16) Demonstration Example of the compartment fire applied to the alternatives Conclusions and opportunities based on the analysis are presented (section Communication 0)

This section addresses the evaluation of alternatives, for which a hierarchical structure is proposed following Prat’s guidelines [126]. The two dimensions selected for evaluation are suitability and effectiveness in the context of a fire risk assessment. These dimensions are broken down into evaluation criteria and sub criteria as shown in Figure A1-16. The dimensions chosen reflect the main challenges of analyzing uncertainty in FSE. Understanding the added value of each alternative as well as its limitations is the first challenge, without regards for the technical difficulties of its implementation. This is covered in the first dimension, suitability. Despite the advantages of a particular approach, it could still require complex tools or steps to actually be implemented. This is a major challenge, as risk assessments are often done bounded by time and capital constrains. This is captured by the second dimension, effectiveness. The evaluation criteria for both dimensions are defined in Table A1-9.

302

Appendix 1. Uncertainty analysis options for risk assessment

System dimensions Evaluation criteria

Significant advantages Suitability Evaluation of Insignificant limitations alternatives Easy to implement Effectiveness Easy to communicate

Figure A1-16. Proposed hierarchy structure for the evaluation of alternatives; based on [126]

Table A1-9. Evaluation criteria -Suitability dimension Evaluation Level Indicator of level attainment criteria The alternative is advantageous relative to the knowledge it Fully achieved (FA) provides and the potential to act upon it by relevant stakeholders. Significant Partially achieved Cannot be judged as FA nor NA advantages (PA) Does not provide significant advantages or none can be Not achieved (NA) associated to this incertitude space Limitations exits but are clearly defined and minor changes in Fully achieved (FA) inputs or model parameters or assumptions do not make the Insignificant approach inviable limitations (not Partially achieved Cannot be judged as FA nor NA related to (PA) implementation) It is possible to trespass the applicability limits with minor Not achieved (NA) variations to inputs or changes to model parameters or assumptions Clear sequence of steps in which data manipulation is clear and Fully achieved (FA) traceable; guidelines, algorithms and software are readily available Easy to Partially achieved implement Cannot be judged as FA nor NA (PA) Lengthy and complex process with no clear guidelines for Not achieved (NA) implementation. Software is not available or scarce. The output’s uncertainty is easy to communicate and compare, Fully achieved (FA) Easy to being consistent with the quantity and detail level of the inputs. communicate Partially achieved Cannot be judged as FA nor NA (PA)

303

Appendix 1. Uncertainty analysis options for risk assessment

Evaluation Level Indicator of level attainment criteria The output does not clearly represent the uncertainty involved and Not achieved (NA) can be inconsistent with the nature of the inputs.

Once the evaluation is completed for all alternatives, the results can be synthesized as presented in Table A1-10. Notice that the objective of the evaluation is to show that none of the alternatives is perfect and all of them will imply important challenges for the practitioners implementing them either in FSE or other fields. It is also important to notice that the challenges that each alternative imply vary greatly in the dimension and criteria evaluated, which means that some of them will be better suited for some problems. The detailed results of the evaluation are presented in Annex II, which includes the associated supporting comments, available software and implementation examples.

First, a general analysis of the results is done, beginning with those of the suitability dimension. The previous point of all approaches being useful is reflected by the results of the significant advantages criterion, where all alternatives partially or fully achieve it. It can be observed that a breakpoint exist in the insignificant limitations criterion, which can be decisive for practitioners exploring new options. The first four approaches score a NA for this criterion, as all of them are constructed on the basis of strong and often not justifiable assumptions such as following a particular shape of probability distribution function or on the subjective estimates of experts which can seldom be directly validated.

Although both info-gap and strength of knowledge present some important limitations, the basis of these approaches are physical phenomena and the recognition of the incomplete or imperfect knowledge, respectively. Such basis provides practitioners with imperfect but useful approaches in which uncertainty is explicitly formulated. In the effectiveness dimension, it can be observed that the more recent and alternative approaches present larger issues of implementation and communication. This can be a major obstacle for their implementation in the FSE context, as development of competences to correctly implement the alternatives might be needed. Communication is also an issue, as both the competences of the practitioners’ play a crucial role, as well as the willingness of the AHJ and other stakeholders to analyze uncertainty from a non-probabilistic approach.

304

Appendix 1. Uncertainty analysis options for risk assessment

Table A1-10. Evaluation summary P- Info- Dimension Criterion Probability Evidence Possibility SoK EMA boxes gap Significant FA FA PA PA FA FA FA advantages Suitability Insignificant NA NA NA NA PA PA PA limitations Easy to FA PA PA NA NA PA PA implement Effectiveness Easy to PA PA PA PA PA FA NA communicate

With the exception of software limitations that decrease the implementation ease for possibility theory and the opposite situation for probability, the first four approaches are considerably similar. Each one offers a unique way to understand the likelihood of an event occurring or of a condition existing, but approach the problem in a similar manner and therefore also offer similar challenges for practitioners. In particular, those different to probability have a heavy reliance on subjective estimates provided by more than one expert. This is reflected by the consistent NA level of achievement for the insignificant limitations criterion by all four approaches, which –again- does not render them useless, but impose significant challenges on the practitioners who wish to implement them. The mechanical nature of the calculations involved in these approaches can lead to believe that the results are trustworthy, but each individual problem should be treated on a case-by-case basis and the limitations explicitly stated. This treatment of the potential sources of uncertainty within the approach itself is seldom done, as the mechanical nature of the approaches usually leads practitioner to rely only on sensitivity analysis.

Info-gap and strength of knowledge approaches present two different ways to deal with uncertainty without necessarily using a probabilistic approach, however they are able to accommodate it. The results of the evaluation for the former indicate significant advantages for practitioners as it avoids dealing with uncertainty by focusing on measuring the robustness. In the case of a fire risk assessment this translates into understanding how much extra loads can a building take under fire condition before the objectives are compromised. However, there is not a standard way to do it and the available examples -although detailed and clear- need to be carefully extrapolated into each individual problem. Here lie the important limitations for this approach, as a deep analytical understanding of the problem is not always available in FSE;

305

Appendix 1. Uncertainty analysis options for risk assessment

this triggers the use of methodologies as the one previously mentioned based on the concept of the maximum allowable damage [91]. This methodology acknowledges the presence of uncertainty in the risk assessment and triggers two simultaneous processes: assessing the system’s performance as a function of the safety objectives and systematically record and judge the assumptions and limitations embedded in inputs, parameters and models employed.

The purpose of judging assumptions and limitations is not only to identify potential weaknesses of the assessment, but to identify actions that allow monitoring and controlling them during the life-cycle of the system. In the context of the ISO 31000 framework, this would be part of the risk treatment plan. The default tool to judge assumptions and limitations in the maximum allowable damage methodology is the strength of knowledge approach. The evaluation results for this approach can be regarded as the best of all, as it fully achieves significant advantages and ease of communication. The bibliography available on this approach show how the simple idea of SoK can be applied in both a qualitative or quantitative format and effectively support decision making even in situations where knowledge is recognized to be incomplete or imperfect; this represents a significant tool for fire risk assessments, where much is known but even more assumed. The challenges of this approach lie on the need to properly setting it up in a case-by-case basis and with the use of categories that can evoke the same issues of constructing a risk analysis matrix [127].

In the suitability dimension, only EMA fully achieves the significant advantages criteria given its explicit recognition of deep uncertainty and the treatment of a massive universe of possibilities in order to inform decision-makers. An approach like EMA and others belonging to Robust Decision Making (RDM) explore a wide range of possible futures based on a large scale exploration of variations of both conditions and solutions and both the theoretical construction of these futures and the complex relationships between variables involved represent a significant limitation. Even when implementation (rated PA) is supported by machine-learning or artificial intelligence, the complexity of these algorithms can be such to not allow traceability of the results or exploring the relationships between key variables. Furthermore, EMA does not fully achieve the criteria for effectiveness, as its implementation is still limited to complex problems which can be well characterized and populated with significant background information. Communicating EMA’s outputs also poses a challenge, particularly for stakeholders used to a quantitative answer in the form of probabilities or frequencies. This approach can be considered at the extreme end of the possibilities to analyze

306

Appendix 1. Uncertainty analysis options for risk assessment

uncertainty in fire safety risk assessments, and could be a powerful tool to assess the potential impact of fire safety policy changes such as proposed bans on specific materials.

In today’s big data world, the acquisition and consolidation of large databases has promoted the implementation of machine-learning algorithms that can quickly and effectively provide an output in contrast to highly analytical and complex calculations. Fire safety is not the exception and currently there are documented efforts of implementing it to support decision-making in evacuation strategies based on a database of 569 building evacuations [128]. Machine-learning has also been applied to provide faster results when simulating fires by using traditional model outputs to train the algorithm [129]. Both examples highlight the potential of an alternative not directly addressed by this work, which in the future could be used to analyze demographic and construction data to identify construction clusters with fire safety deficiencies or population groups with particular fire safety challenges. However, the implementation of machine learning is conditioned to training data. In the context of a fire risk assessment, unique fire and building features and interactions pose a major challenge for its implementation. Attempts to apply machine-learning to fire consequence modelling in nuclear facilities [130] could set the basis for integrating such models in a risk assessment, although uncertainty description might still require using the alternatives presented in this work.

5 Discussion on the practitioners’ perspective

As previously stated, the aim of the paper is to supply practitioners with a picture of the uncertainty analysis approaches and their evaluation based on ease of implementation/communication. In order to gauge and discuss the impact of the alternatives analysis an interview with two risk assessment experts were interviewed. The interviews do not intend to provide a statistical representation of the perception of all practitioners, but to incite the debate on key issues and needed changes in the role of risk assessments and their outputs in FSE. First the profile of the interviewees is presented, followed by the interview results and a brief discussion. Interviewee 1 (I1) has a doctorate in chemical engineering in the topic of systems theory applied to safety engineering, over 10 years of academic career in the process safety field and is currently a senior health and safety professional at a major oil & gas company. Interviewee 2 (I2) has a doctorate in civil engineering in the topic of structural fire, over 10 years of academic career in the FSE field and is currently a professor in a worldwide recognized university. Both interviewees were consulted due to the trajectories and recognition among peers of their contributions to their respective fields.

307

Appendix 1. Uncertainty analysis options for risk assessment

First, the interviewees were asked if they consider the probabilistic approach the best suited alternative to assess the performance of an engineering system, specifically the fire safety of an occupied building. I1 pointed out that identifying low probability scenarios is not always possible following a probabilistic approach, requiring complementary techniques. In particular, I1 drew attention to the outmost need of preliminary qualitative risk analysis that focus on identification of scenarios that then lead to refined quantitative analysis of prioritized scenarios. This is consistent with the approach presented by BS 7974:2019 [69], and differs from a more mechanistic approach as proposed by the verification methods [131]. I1 states that following a probabilistic approach or an alternative is a decision to be made as a function of the analysis objectives and the available resources. He further suggested such a decision needs to be made by stakeholders at an early stage of the project and that in particular, the probabilistic approach lacks usefulness when the reliability changes in the system (also in its sub-systems and elements) are not managed through time. I2 answer is that the probabilistic approach is the best suited alternative for some systems, as it is the alternative that provides the highest resolution, i.e. most insight in the spectrum of possible performance of the system in case of fire. Contrasting this, I2 recognizes that some systems might only need a site visit from a competent professional to effectively account for uncertainties. A key point made by both I1 and I2 is that a risk assessment cannot be turned into the probabilistic analysis of all involved variables or uncertainties. I2 reinforces this mentioning the value of guidance and of prescriptive codes, which are applicable –and adequate- for a large range of projects. I2 states that codes and guidance -particularly in FSE- have the value of providing a common benchmark and alternatives to approach different problems. This range can be seen as to go from a purely guidance based (prescriptive) to depending fully on the competence of the practitioners (performance-based). This last point made by I2 poses an interesting issue, as both extremes of practice range in FSE (rule-based and performance based) have identified the danger posed by poorly managed uncertainties. In the context of chemical process safety, risk assessment guidelines point towards the probabilistic approach and the use of Paté-Cornell’s [80] uncertainty management levels to manage uncertainty in a quantitative manner. However, safety engineering presents a wide range of implicit and explicit manners to deal with uncertainty [43] including the way in which scenarios are identified, pre-defined rules for systems fitting a taxonomy (prescriptive approach), etc. The interviewees agreed on the need for a preceding qualitative assessment that guides the probabilistic one, and on the fact that using the former is no guarantee that uncertainty is adequately accounted for and managed.

308

Appendix 1. Uncertainty analysis options for risk assessment

The previous leads to the second question regarding the obstacles of implementing the alternatives in practice. I1 states that the alternatives constitute a ‘toolbox’, from which a selection must be made early in the assessment. This selection is based on three elements: 1) assessment objectives as a function of decision-making, 2) available information and 3) available resources. Given that these elements are clearly identified, I1 finds that any obstacle in selecting an alternative will be associated to the stakeholders with authorization (approval) power and the system users. This is coherent with the difficulties in implementation found in the evaluation, which are common to all alternatives and require informed stakeholders with the ability to interpret the outputs. Furthermore, I1 emphasizes the need to prioritize the uncertainty analysis alternatives for each assessment based on an initial qualitative analysis. To address the question, I2 states that in FSE the current preferred approach to risk assessments is implicit worst credible scenarios (deterministic), which is in fact backed up by the analysis of Johansson [132] of Australian fire engineering reports and presented in a broader international perspective by Bjelland [133]. Therefore, I2 highlights that practitioners should be more aware and willing to communicate about the uncertainties involved in deterministic assessments, as well as their limitations. However, I2 recognizes that the current construction focus on the end goal (i.e. getting a design approved and built) limits the possibility of communicating the uncertainties completely and effectively. This is consistent with the Shergold-Weir enquiry [38] in Australia and with the Hackitt [1] enquiry in the UK. The former reflects it with several recommendations, including No. 9 and 10 associated to the lack of transparency and the need for a code of conduct for building surveyors. The latter referred to the issues associated to the construction regulatory system failure stating that “the primary motivation is to do things as quickly and cheaply as possible rather than to deliver quality homes which are safe for people to live in”. I2 concludes that the main obstacle in FSE to implement the presented alternatives and better support decision-making is the lack of openness about the effects of uncertainties (within risk assessments) on objectives.

6 Conclusions

This work has described the need to better understand and employ available options for uncertainty analysis in the context of a fire risk assessment, starting from the premise that a single approach is not enough to tackle current challenges for FSE. Given the variability of uncertainty sources and their nature, practitioners cannot force all of them into a single analysis alternative, despite the lack of appropriate guidelines to do so. The lack of a clear perspective

309

Appendix 1. Uncertainty analysis options for risk assessment

on the available alternatives has been addressed by this work, describing them and applying them to a simple compartment fire problem. Applying the considered alternatives to a simple fire safety problem the results vary significantly and imply supporting different decisions for the compartment. Despite its simplicity, the example illustrates the variability of the outputs based on the uncertainty analysis alternative. More importantly, no alternative provides the ‘right’ answer, while each one provides different pieces of valuable information.

The advantages and challenges of each alternative in relation to their implementation and the outputs they provide for the decision-making process were judged following the modified DSRM methodology. Using the suitability and effectiveness dimensions, each one of the eight alternatives were evaluated in the context of FSE. The results indicate –as the case study- that no alternative is the ‘best’ and each one must be considered carefully in the context of the assessment being carried out. One common challenge is the reliance on mathematical calculations that are increasingly complex, requiring competent users capable of handling them or a robust software package that aids the calculations. As with any tool used in engineering, the use of available software must be done within its applicability range and considering the assumptions that the tool itself introduces to the calculation. A versatile alternative is found in the strength of knowledge, however the calibration of its categories could also pose challenges for its implementation, i.e. how to define a ‘high’ strength of knowledge.

The work presented aims at widening the perspective of practitioners fire safety engineers involved in risk assessment, but capitalizes on the knowledge and experience of different disciplines facing the same challenges. Recognizing that very distinct disciplines share challenges and solutions, create a needed synergy that may build new, more trustworthy and effective, risk assessments. This work capitalizes on the work done by many others throughout decades and expects to provide the basis for future work, in which technically challenging alternatives such as EMA can be efficiently implemented to typical problems in chemical process and fire safety engineering.

7 Acknowledgements

The authors recognize the participation of the two interviewees Dr Felipe Muñoz Giraldo from the Colombian National Oil Company (Ecopetrol S.A.) and Prof Ruben Van Coile from Ghent University, who despite time zone differences and bustling agendas found time to engage in invaluable discussions. Through the interviews, the authors understood the meaning of this work from different perspectives, allowing for refining and important adjustments. These

310

Appendix 1. Uncertainty analysis options for risk assessment conversations also reflect the wealth of knowledge each practitioner has when it comes to uncertainty accounting and therefore the large amount of work yet to be done to provide unified, useful guidelines for its accounting in risk assessments.

References

1. Hackitt, J., Building a Safer Future, Independent Review of Building Regulations and Fire Safety: Final Report. 2018: London, UK. 2. Van Coile, R., G. Jomaas, and L. Bisby, Defining ALARP for fire safety engineering design via the Life Quality Index. Fire Safety Journal, 2019. 107: p. 1-14. 3. Gernay, T., et al., Efficient uncertainty quantification method applied to structural fire engineering computations. Engineering Structures, 2019. 183: p. 1-17. 4. Van Coile, R., et al., The Need for Hierarchies of Acceptance Criteria for Probabilistic Risk Assessments in Fire Engineering. Fire Technology, 2018. 5. Van Coile, R., et al., An Unbiased Method for Probabilistic Fire Safety Engineering, Requiring a Limited Number of Model Evaluations. Fire Technology, 2017. 53(5): p. 1705-1744. 6. Beard, A.N., Risk assessment assumptions. Civil Engineering and Environmental Systems, 2004. 21(1): p. 19-31. 7. Jaime E. Cadena, F.M., The Link between Fire Research and Process Safety: An Evolution from Specific Needs to General Concern. Chemical Engineering Transactions, 2013. 31: p. 679-684. 8. Aven, T. and O. Renn, On risk defined as an event where the outcome is uncertain. Journal of Risk Research, 2009. 12(1): p. 1-11. 9. Analysis, C.o.F.o.R., Society for Risk Analysis Glossary. 2015, Society for Risk Analysis. 10. Standardization, I.O.f., ISO 31000:2018: Risk management - Principles and guidelines. 2018: Geneva, Switzerland. 11. Aven, T., O. Renn, and E.A. Rosa, On the ontological status of the concept of risk. Safety Science, 2011. 49(8): p. 1074-1079. 12. Beck, M. and B. Kewell, Risk. 2012: WORLD SCIENTIFIC. 380. 13. Blokland, P. and G. Reniers, An Ontological and Semantic Foundation for Safety and Security Science. Sustainability, 2019. 11(21): p. 6024.

311

Appendix 1. Uncertainty analysis options for risk assessment

14. Usmani, A.S., Y.C. Chung, and J.L. Torero, How did the WTC towers collapse: a new theory. Fire Safety Journal, 2003. 38(6): p. 501-533. 15. Amadeo, K., How the 9/11 Attacks Affect the Economy Today, in The Balance. 2020: New York, USA. 16. Torero, J., Grenfell tower: Phase 1 report. 2018, Torero, Abecassis Empis and Cowlard. 17. Booth, R., Grenfell inquiry has cost 100 times amount saved on cladding, in The Guardian. 2019: London, UK. 18. Johnston, K., Inferno: the fire that brought a city to its knees, in New Zealand Herald. 2019: Auckland, New Zealand. 19. Chris Vedelago, S.I., Company behind Campbellfield fire collapses, taxpayers could foot bill, in The Age. 2019: Melbourne, Australia. 20. India, P.T.o., 81 killed as massive fire spreads quick through chemical warehouses in Dhaka. 2019. 21. Dionne, G., Risk Management: History, Definition, and Critique. Risk Management and Insurance Review, 2013. 16(2): p. 147-166. 22. Talja, H., et al., Structural safety analysis with engineering integrity assessment tools. Computers & Structures, 1997. 64(1): p. 759-770. 23. Wang, X., et al., Comparison of the reliability-based and safety factor methods for structural design. Applied Mathematical Modelling, 2019. 72: p. 68-84. 24. Aven, T., Risk assessment and risk management: Review of recent advances on their foundation. European Journal of Operational Research, 2016. 253(1): p. 1-13. 25. Årstad, I. and T. Aven, Managing major accident risk: Concerns about complacency and complexity in practice. Safety Science, 2017. 91: p. 114-121. 26. Goerlandt, F. and G. Reniers, Prediction in a risk analysis context: Implications for selecting a risk perspective in practical applications. Safety Science, 2018. 101: p. 344- 351. 27. Aven, T., On the Need for Restricting the Probabilistic Analysis in Risk Assessments to Variability. Risk Analysis, 2010. 30(3): p. 354-360. 28. Zeng, Z. and E. Zio, A classification-based framework for trustworthiness assessment of quantitative risk analysis. Safety Science, 2017. 99: p. 215-226. 29. Pasman, H. and W. Rogers, How trustworthy are risk assessment results, and what can be done about the uncertainties they are plagued with? Journal of Loss Prevention in the Process Industries, 2018. 55: p. 162-177.

312

Appendix 1. Uncertainty analysis options for risk assessment

30. Tixier, J., et al., Review of 62 risk analysis methodologies of industrial plants. Journal of Loss Prevention in the Process Industries, 2002. 15(4): p. 291-303. 31. Marhavilas, P.K., D. Koulouriotis, and V. Gemeni, Risk analysis and assessment methodologies in the work sites: On a review, classification and comparative study of the scientific literature of the period 2000–2009. Journal of Loss Prevention in the Process Industries, 2011. 24(5): p. 477-523. 32. Baecher, G., Uncertainty in dam safety risk analysis. Vol. 10. 2016. 1-17. 33. Yang, X., S. Haugen, and N. Paltrinieri, Clarifying the concept of operational risk assessment in the oil and gas industry. Safety Science, 2018. 108: p. 259-268. 34. Aven, T. and V. Kristensen, How the distinction between general knowledge and specific knowledge can improve the foundation and practice of risk assessment and risk-informed decision-making. Reliability Engineering & System Safety, 2019. 191: p. 106553. 35. Meacham, B.J. Concepts of a Performance-based Building Regulatory System for the United States. in International Association for Fire Safety Science. Fire Safety Science. Proceedings. Fifth (5th) International Symposium. 1997. Melbourne, Australia. 36. Ministry of Housing, C.L.G., Structure: Approved Document A, in The Building Regulations 2010, H. Government, Editor. 20109: London, UK. 37. Bisby, L., 'Adequacy' in structural fire engineering, in Technical Lecture Series 2019. 2019, The Institution of Structural Engineers (IStructE). 38. Peter Shergold, B.W., Building Confidence – Improving the effectiveness of compliance and enforcement systems for the building and construction industry across Australia. 2018, Building Ministers’ Forum (BMF). 39. genco, G., Municipal Building Surveyor report - Lacrosse Building Fire. 2015, City of Melbourne: Melbourne, Australia. 40. Cooke, R. and T. Bedford, What is uncertainty?, in Probabilistic Risk Analysis: Foundations and Methods. 2001, Cambridge University Press: Cambridge. p. 17-38. 41. Hayes, K.R., Uncertainty and uncertainty analysis methods. 2011, CSIRO: Tasmania, Australia. 42. Walker, W.E., R.J. Lempert, and J.H. Kwakkel, Deep Uncertainty, in Encyclopedia of Operations Research and Management Science, S.I. Gass and M.C. Fu, Editors. 2013, Springer US: Boston, MA. p. 395-402.

313

Appendix 1. Uncertainty analysis options for risk assessment

43. Notarianni, K.A. and G.W. Parry, Uncertainty, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2992-3047. 44. Kaplan, S. and B.J. Garrick, On The Quantitative Definition of Risk. Risk Analysis, 1981. 1(1): p. 11-27. 45. Zio, E. and N. Pedroni, Uncertainty characterization in risk analysis for decision- making practice. 2012. 46. Colyvan, M., Is Probability the Only Coherent Approach to Uncertainty? Risk Analysis, 2008. 28(3): p. 645-652. 47. Meacham, B.J. and I.J. Van Straalen, A socio-technical system framework for risk- informed performance-based building regulation. Building Research & Information, 2018. 46(4): p. 444-462. 48. Johansen, I.L. and M. Rausand, Ambiguity in risk assessment. Safety Science, 2015. 80: p. 243-251. 49. Notarianni, K.A. and G.W. Parry, SFPE Handbook of Fire Protection Engineering, Fifth Edition. 2016: Springer New York. 2992-3047. 50. Hurley, M., Uncertainty in Fire Protection Engineering Design. Journal of Testing and Evaluation, 2012. 40(1): p. 12-17. 51. Kaplan, S., Y.Y. Haimes, and B.J. Garrick, Fitting Hierarchical Holographic Modeling into the Theory of Scenario Structuring and a Resulting Refinement to the Quantitative Definition of Risk. Risk Analysis, 2001. 21(5): p. 807-807. 52. Aven, T., Selective critique of risk assessments with recommendations for improving methodology and practise. Reliability Engineering & System Safety, 2011. 96(5): p. 509-514. 53. Commission, U.S.N.R., Reactor Safety Study: An Assessment of Accident Risks in U.S. Commercial Nuclear Power Plants. 1975. 54. Benintendi, R., Chapter 12 - Quantitative Risk Assessment, in Process Safety Calculations. 2018, Elsevier. p. 607-628. 55. Quantitative Risk Analysis, in Handbook of Safety Principles. 56. Ramachandran, G. and D.A. Charters, Quantitative risk assessment in fire safety Ganapathy Ramachandran and David A. Charters, ed. C. Ebooks. 2011, London ; New York: London ; New York : Spon Press. 57. (CCPS), C.f.C.P.S., Guidelines for chemical process quantitative risk analysis. 2000: The Center.

314

Appendix 1. Uncertainty analysis options for risk assessment

58. Iervolino, I., et al., Quantitative risk analysis for the Amerigo Vespucci (Florence, Italy) airport including domino effects. Safety Science, 2019. 113: p. 472-489. 59. Zhao, J., et al., Quantitative risk assessment of continuous liquid spill fires based on spread and burning behaviours. Applied Thermal Engineering, 2017. 126: p. 500-506. 60. Antonioni, G., et al., Quantitative assessment of risk due to NaTech scenarios caused by floods. Reliability Engineering & System Safety, 2015. 142: p. 334-345. 61. Cozzani, V., et al., Quantitative assessment of domino and NaTech scenarios in complex industrial areas. Journal of Loss Prevention in the Process Industries, 2014. 28: p. 10-22. 62. Gye, H.-R., et al., Quantitative risk assessment of an urban hydrogen refueling station. International Journal of Hydrogen Energy, 2019. 44(2): p. 1288-1298. 63. Ltd, G.P., Report for Boodarie Strategic Industrial Area - Concept Plan Quantitative Risk Assessment. 2012. 64. Vanorio, G. and J. Mera, Methodology for risk analysis in railway tunnels using Monte Carlo simulation. 2012. 673-683. 65. Meng, Q., et al., QRA Model-Based Risk Impact Analysis of Traffic Flow in Urban Road Tunnels. Risk Analysis, 2011. 31(12): p. 1872-1882. 66. Amundrud, Ø. and T. Aven, On how to understand and acknowledge risk. Reliability Engineering & System Safety, 2015. 142: p. 42-47. 67. Zio, E., The future of risk assessment. Reliability Engineering & System Safety, 2018. 177: p. 176-190. 68. Rausand, M. and K. Øien, The basic concepts of failure analysis. Reliability Engineering & System Safety, 1996. 53(1): p. 73-83. 69. (BSI), B.S.I., BS 7974:2019, Application of fire safety engineering principles to the design of buildings - Code of practice. 2019: BSI. 70. Goerlandt, F., N. Khakzad, and G. Reniers, Validity and validation of safety-related quantitative risk analysis: A review. Safety Science, 2016. 71. A. Rae, J.M., R. Alexander, The Science and Superstition of Quantitative Risk Assessment, in Proceedings of PSAM 11 & ESREL 2012. 2012, International Association of Probabilistic Safety Assessment and Management, IAPSAM. p. 2292- 2301. 72. Goerlandt, F. and G. Reniers, Evidence assessment schemes for semi-quantitative risk analyses: A response to Roger Flage and Terje Aven. Safety Science, 2017. 98: p. 12- 16.

315

Appendix 1. Uncertainty analysis options for risk assessment

73. Bjørnsen, K., J.T. Selvik, and T. Aven, A semi-quantitative assessment process for improved use of the expected value of information measure in safety management. Reliability Engineering & System Safety, 2019. 188: p. 494-502. 74. Flage, R. and T. Aven, Some brief concluding remarks in relation to the discussion with Floris Goerlandt and Genserik Reniers about strength of knowledge (strength of evidence) judgments in semi-quantitative risk analysis. Safety Science, 2018. 108: p. 237. 75. Khorsandi, J. and T. Aven, Incorporating Assumption Deviation Risk in Quantitative Risk Assessments: A Semi-Quantitative Approach. Vol. 163. 2017. 76. Askeland, T., R. Flage, and T. Aven, Moving beyond probabilities – Strength of knowledge characterisations applied to security. Reliability Engineering & System Safety, 2017. 159: p. 196-205. 77. Bjerga, T., T. Aven, and E. Zio, Uncertainty treatment in risk analysis of complex systems: The cases of STAMP and FRAM. Reliability Engineering & System Safety, 2016. 156: p. 203-209. 78. Aven, T., Supplementing quantitative risk assessments with a stage addressing the risk understanding of the decision maker. Reliability Engineering & System Safety, 2016. 152: p. 51-57. 79. Castino, G., Fire Risk Assessment. 1982: ASTM International. 80. Paté-Cornell, M.E., Uncertainties in risk analysis: Six levels of treatment. Reliability Engineering & System Safety, 1996. 54(2): p. 95-111. 81. Flage, R., et al., Concerns, Challenges, and Directions of Development for the Issue of Representing Uncertainty in Risk Assessment. Risk Analysis, 2014. 34(7): p. 1196- 1207. 82. Dubois, D., Representation, Propagation, and Decision Issues in Risk Analysis Under Incomplete Probabilistic Information. Risk Analysis, 2010. 30(3): p. 361-368. 83. Klir, G.J., Generalized information theory: aims, results, and open problems. Reliability Engineering & System Safety, 2004. 85(1): p. 21-38. 84. Beer, M., S. Ferson, and V. Kreinovich, Imprecise probabilities in engineering analyses. Mechanical Systems and Signal Processing, 2013. 37(1): p. 4-29. 85. Aven, T., A risk concept applicable for both probabilistic and non-probabilistic perspectives. Safety Science, 2011. 49(8): p. 1080-1086. 86. Abdo, H., J.M. Flaus, and F. Masse, Uncertainty quantification in risk assessment - Representation, propagation and treatment approaches: Application to atmospheric

316

Appendix 1. Uncertainty analysis options for risk assessment

dispersion modeling. Journal of Loss Prevention in the Process Industries, 2017. 49: p. 551-571. 87. Lempert, R.J., Robust Decision Making (RDM), in Decision Making under Deep Uncertainty: From Theory to Practice, V.A.W.J. Marchau, et al., Editors. 2019, Springer International Publishing: Cham. p. 23-51. 88. Y. Ben-Haim, Y.M., Info-gap Decision Theory For Engineering Design. Or: Why 'Good' is Preferable to 'Best', in Engineering Design Reliability Handbook, D.G.a.S.S. E. Nikolaides, Editor. 2005, CRC Press. 89. Bankes, S., Exploratory Modeling for Policy Analysis. Operations Research, 1993. 41: p. 435-449. 90. Takewaki, I. and Y. Ben-Haim, Info-gap robust design with load and model uncertainties. Journal of Sound and Vibration, 2005. 288(3): p. 551-570. 91. J. Cadena, J.H., C. Maluk, D. Lange, J. L. Torero, A. F. Osorio, Overcoming Risk Assessment Limitations for Potential Fires in a Multi-Occupancy Building. Chemical Engineering Transactions, 2019. 77. 92. Kabir, S. and Y. Papadopoulos, A review of applications of fuzzy sets to safety and reliability engineering. International Journal of Approximate Reasoning, 2018. 100. 93. Shi, H. A Fuzzy Approach to Building Fire Risk Assessment and Analysis. in 2009 Third International Symposium on Intelligent Information Technology Application. 2009. 94. Funtowicz, S.O., Uncertainty and quality in science for policy / by Silvio O. Funtowicz and Jerome R. Ravetz. Theory and decision library. Series A, Philosophy and methodology of the social sciences ; v. 15., ed. J.R. Ravetz. 1990, Dordrecht, the Netherlands ; Norwell, MA, U.S.A: Kluwer Academic Publishers. 95. Ellis, E.C., et al., LONG-TERM CHANGE IN VILLAGE-SCALE ECOSYSTEMS IN CHINA USING LANDSCAPE AND STATISTICAL METHODS. Ecological Applications, 2000. 10(4): p. 1057-1073. 96. Zadeh, L.A., A Note on Z-numbers. Information Sciences, 2011. 181(14): p. 2923-2932. 97. Young, C., Model Uncertainty and the Crisis in Science. Socius, 2018. 4: p. 2378023117737206. 98. HSE, Offshore Hydrocarbon Release Statistics 2001, in HID Statistics Report HSR. 2001, Health & Safety Executive. 99. Frank, M.V., Probabilistic Analysis of the Inadvertent Reentry of the Cassini Spacecraft's Radioisotope Thermoelectric Generators. Risk Analysis, 2000. 20(2): p. 251-260.

317

Appendix 1. Uncertainty analysis options for risk assessment

100. Laboratory, J.P., Cassini Earth Swingby Plan Supplement. 1997, Jet Propulsion Laboratory: Pasadena, CA, USA. 101. Apostolakis, G., The Concept of Probability if Safety Assessments of Technological Systems. Science (New York, N.Y.), 1991. 250: p. 1359-64. 102. Savage, L.J., The foundations of statistics. 2nd. rev. ed.. ed. 1972, New York: Dover Publications. 103. Barua, S., et al., Bayesian network based dynamic operational risk assessment. Journal of Loss Prevention in the Process Industries, 2016. 41: p. 399-410. 104. Hanea, D. and B. Ale, Risk of human fatality in building fires: A decision tool using Bayesian networks. Fire Safety Journal, 2009. 44(5): p. 704-710. 105. Traub, J.F., Interval Analysis. Science, 1967. 158(3799): p. 365-365. 106. Du, X. Uncertainty Analysis With Probability and Evidence Theories. in ASME 2006 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. 2006. 107. Dell'Orco, M. and S. Kikuchi, An alternative approach for choice models in transportation: Use of possibility theory for comparison of utilities. Yugoslav Journal of Operations Research, 2004. 14. 108. Darby, J.L., Estimating Terrorist Risk with Possibility Theory. 2004. 109. Ouazraoui, N., et al., Layers of protection analysis in the framework of possibility theory. Journal of Hazardous Materials, 2013. 262: p. 168-178. 110. Mandal, S. and J. Maiti, Risk analysis using FMEA: Fuzzy similarity value and possibility theory based approach. Expert Systems with Applications, 2014. 41(7): p. 3527-3537. 111. Ben-Haim, Y., Value-at-risk with info-gap uncertainty. The Journal of Risk Finance Incorporating Balance Sheet, 2005. 6(5): p. 388-403. 112. Aven, T., Knowledge in Risk Assessment and Management Fundamental Ideas, Principles and Approaches. 2017: p. 143-164. 113. Kwakkel, J.H., The Exploratory Modeling Workbench: An open source toolkit for exploratory modeling, scenario discovery, and (multi-objective) robust decision making. Environmental Modelling & Software, 2017. 96: p. 239-250. 114. D. Groves, E.B., J. R. Fischbach, D. Knopman, Adapting to a Changing Colorado River: Making Future Water Deliveries More Reliable Through Robust Management Strategies, in American Geophysical Union, Fall Meeting 2013. 2013. 115. Ocran, N., Fire loads and design fires for mid-rise buildings. 2012, Carleton University.

318

Appendix 1. Uncertainty analysis options for risk assessment

116. Bwalya, A., N. Benichou, and M. Sultan, Literature Review on Design Fires. 2003. 117. M. Nilsson, N.J., P. Van Hees, A New Method for Quantifying Fire Growth Rates Using Statistical and Empirical Data – Applied to Determine the Effect of Arson Fire Safety Science-Proceedings of the Eleventh International Symposium, 2014: p. 517-530. 118. Sam L. Savage, M.T., SIP Math. 2012. 119. (NCD-RisC), N.R.F.C. Height: Evolution of adult height over time. 2020 13/08/2020]; Available from: http://www.ncdrisc.org/height-mean-distribution.html. 120. Hopkin, C., et al., Design Fire Characteristics for Probabilistic Assessments of Dwellings in England. Fire Technology, 2019. 121. Yildiz, A.E., I. Dikmen, and M.T. Birgonul, Using Expert Opinion for Risk Assessment: A Case Study of a Construction Project Utilizing a Risk Mapping Tool. Procedia - Social and Behavioral Sciences, 2014. 119: p. 519-528. 122. Bergmans, H., et al., Working with Scenarios, Risk Assessment and Capabilities in the National Safety and Security Strategy of the Netherlands. 2009. 123. Ferson, S. and W.T. Tucker. Probability boxes as info-gap models. in NAFIPS 2008 - 2008 Annual Meeting of the North American Fuzzy Information Processing Society. 2008. 124. Gwynne, S.M.V. and K.E. Boyce, Engineering Data, in SFPE Handbook of Fire Protection Engineering, M.J. Hurley, et al., Editors. 2016, Springer New York: New York, NY. p. 2429-2551. 125. Peffers, K., et al., A design science research methodology for information systems research. Vol. 24. 2007. 45-77. 126. Prat, N., I. Wattiau, and J. Akoka, Artifact Evaluation in Information Systems Design Science Research ? A Holistic View. 2014. 127. Duijm, N.J., Recommendations on the use and design of risk matrices. Safety Science, 2015. 76: p. 21-31. 128. Zhao, X., R. Lovreglio, and D. Nilsson, Modelling and interpreting pre-evacuation decision-making using machine learning. Automation in Construction, 2020. 113: p. 103140. 129. Lattimer, B.Y., J.L. Hodges, and A.M. Lattimer, Using machine learning in physics- based simulation of fire. Fire Safety Journal, 2020. 114: p. 102991. 130. Worrell, C., et al., Machine learning of fire hazard model simulations for use in probabilistic safety assessments at nuclear power plants. Reliability Engineering & System Safety, 2019. 183: p. 128-142.

319

Appendix 1. Uncertainty analysis options for risk assessment

131. Johnson, P. and N. Lobel, Fire Safety Verification Method – The Australia Research Experience. Journal of Physics: Conference Series, 2018. 1107: p. 042033. 132. Johansson, U., B. Stratton, and P. Olsson, On the Current Quality and Depth of Fire Safety Engineering Analysis in Australia – A Statistical Review of Recent Designs. 2011. 133. Bjelland, H. and A. Borg, On the use of scenario analysis in combination with prescriptive fire safety design requirements. Environment Systems & Decisions, 2013. 33(1): p. 33-42. 134. Denœux, T., Inner and Outer Approximation of Belief Structures Using a Hierarchical Clustering Approach. International Journal of Uncertainty, Fuzziness and Knowledge- Based Systems, 2001. 9: p. 437-460. 135. Guyonnet, D., et al., Hybrid Approach for Addressing Uncertainty in Risk Assessments. Journal of Environmental Engineering, 2003. 129(1): p. 68-78.

320

Appendix 1. Uncertainty analysis options for risk assessment

Annex I -Compartment fire model

The model for the compartment fire begins defining the heat release rate of the burning fuel, which is given by:

= = ̇ 𝑐𝑐 𝑓𝑓 is the heat release rate of a𝐻𝐻 material𝐻𝐻𝐻𝐻𝐻𝐻 𝑅𝑅𝑅𝑅𝑅𝑅 𝑅𝑅𝑅𝑅-and𝑅𝑅𝑅𝑅 this𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 is measured𝑄𝑄 ∆𝐻𝐻 in ∙Watts𝑚𝑚̇ (W), is the ideal heat

𝑄𝑄oḟ combustion of the fuel measured in [J/kg] and is the burning rate ∆measured𝐻𝐻𝑐𝑐 in [kg/s]. The latter can also be computed as a function of the𝑚𝑚 areȧ𝑓𝑓 by defining is as:

𝑓𝑓 𝑓𝑓 Where A is the area of the fire and 𝑚𝑚 iṡ the𝐴𝐴 burning∙ 𝑚𝑚̇ ′′ rate per unit area. The fire growth can

be extremely complex to model due𝑚𝑚 tȯ 𝑓𝑓′′ solid fuel combustion processes. A key -but onerous- simplification is that a fire can grow radially at an exponential rate. This means that the area of the fire can be described as a function of a radial fire spread ( ):

𝑠𝑠 = = ( ) 𝑣𝑣 2 2 𝑠𝑠 Where t is the time step. Integrating these𝐴𝐴 𝜋𝜋 three𝑟𝑟 eq𝜋𝜋uations𝑣𝑣 ∙ 𝑡𝑡 the heat release rate can be expressed as a function of a fire growth rate parameter known as alpha ( ):

= = 𝛼𝛼 2 2 2 ̇ 𝑐𝑐 𝑠𝑠 𝑓𝑓 Performing an energy balance𝑄𝑄 based�∆ 𝐻𝐻on ∙the𝜋𝜋 ∙heat𝑣𝑣 ∙released𝑚𝑚̇ ′′� ∙ 𝑡𝑡by the𝛼𝛼 fire𝑡𝑡 and the heat transferred by radiation to the surroundings (taken as a 30% of the total released), the heat feedback into the fire and the heat transferred to the smoke, it can be found that the smoke temperature can be expressed as:

= + ̇𝑠𝑠 𝑆𝑆 𝐴𝐴 𝑄𝑄 𝑇𝑇 𝑇𝑇 𝐴𝐴 𝑝𝑝 With the air entrainment is produced by the fire𝑚𝑚̇ 𝐶𝐶and the smoke plume, is the specific

heat capacity𝑚𝑚̇ 𝐴𝐴 of the smoke, is the temperature of the smoke, and is the temperature𝐶𝐶𝑝𝑝 of the ambient air. Based on experimental𝑇𝑇𝑠𝑠 correlations, the air entrainment𝑇𝑇 𝐴𝐴can be estimated as:

= 2 1⁄3 𝑔𝑔𝜌𝜌𝐴𝐴 1⁄3 5⁄3 𝑚𝑚̇ 𝐴𝐴 𝐸𝐸 � � 𝑄𝑄̇ 𝐻𝐻 𝐶𝐶𝑝𝑝𝑇𝑇𝐴𝐴

321

Appendix 1. Uncertainty analysis options for risk assessment

With g the gravity of the earth, E the entrainment constant (taken as 0.2), the air’s density and H the height at which the air entrainment is estimated. Assuming that the𝜌𝜌𝐴𝐴 smoke produced is the same as air is entrained, it is possible to find the height of the smoke layer if its density is known ( ). To estimate it, the ideal gas law is used:

𝜌𝜌𝑆𝑆 = 𝐴𝐴 𝑆𝑆 𝐴𝐴 𝑇𝑇 𝜌𝜌 𝜌𝜌 𝑆𝑆 Using the compartment’s ceiling area and the volume𝑇𝑇 of the smoke layer, its height can be calculated:

= = 𝑚𝑚̇ 𝑆𝑆 𝑉𝑉𝑆𝑆 �𝜌𝜌𝑆𝑆 𝐻𝐻𝑆𝑆 𝐴𝐴𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 𝐴𝐴𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 Annex II -Detailed evaluation of each alternative

Probabilistic approach

Table A1-11. Evaluation -Probabilistic approach Dimension Criterion Evaluation Supporting comments The main strength of this approach is that it considers a range of possible values for a quantity of interest and distributes the probability along it. This allows fitting a distribution that best fits the experimental data Significant FA and obtaining its parameters, which allow for advantages future estimations of the 'a posteriori' probability of failure. In practical terms, this approach allows experts selecting a range of values associated with different probability of occurrence instead of a single point value. This approach assumes that the probability of an event can be estimated as the result of infinite Suitability similar (equal) trials in which conditions remain the same, but this is hardly ever the case in real engineering systems. This largely depends on the availability (data set) to determine the distribution that fits the behavior of a quantity of Insignificant interest. In practice, especially in complex NA limitations systems, this information is seldom available and if so, it might need periodical updating. When this approach is used with subjective expert criteria, engineers might introduce uncertainty in the estimations by pre selecting a 'shape' of the probability distribution based on their judgment instead of using the best data available; this can also happen when no data is

322

Appendix 1. Uncertainty analysis options for risk assessment

Dimension Criterion Evaluation Supporting comments available and the experts choose a distribution in a subjective manner. Given required inputs and a clear workflow, the approach is easy to implement. However, this is Easy to dependent on having technical experts to FA implement structure the probabilistic assumptions, as well as competent practitioners to execute the analysis. Outputs of numerical nature can be easily informed to stakeholders as in the case of individual risk indices, e.g. 1 x 10-6 Effectiveness fatalities/year; however such indices might not convey all the information associated to the Easy to source of probability distribution functions or to PA communicate the assumptions behind their selection. Given that this alternative has been in practice for decades, this criterion is judged as partially achieved as it is not ensured that the uncertainty involved is explicitly accounted for and communicated. • MATLAB (https://www.mathworks.com/products/matlab.html) • R (https://www.r-project.org/) Software • Pelican (https://www.vosesoftware.com/products/pelican/) available • CristalBall (https://www.oracle.com/au/middleware/technologies/crystalball.html) • SIPmath (https://www.probabilitymanagement.org/sipmath)

P-boxes

Table A1-12. Evaluation -P-box Dimension Criterion Evaluation Supporting comments Require fewer assumptions from the possible probability distribution functions that fit elicited data from experts, ranging from non-parametric, parametric to bounded Significant parametric p-boxes. The less assumptions, the wider the FA advantages bounds for the resulting p-box. This flexibility provides an alternative to Monte Carlo sampling methods, which require independency assumptions and therefore additional Suitability assumptions, which for complex systems might not hold. A p-box analysis requires information on the quartiles or key parameters of the distribution of the variables of interest and it does not solve the expert elicitation issues. Insignificant NA Furthermore, explaining the difference between a limitations parametric or non-parametric p-box and the technical details that define it might be a challenge when communicating it to decision-makers. Given required inputs and a clear workflow, the approach Easy to is easy to implement. However, this is dependent on having PA implement technical experts to structure the probabilistic assumptions, Effectiveness as well as competent practitioners to execute the analysis. The result of P-boxes -typically a range, as in the example Easy to PA of the compartment fire- can contribute significantly to the communicate communication of the knowledge limitations of the key

323

Appendix 1. Uncertainty analysis options for risk assessment

Dimension Criterion Evaluation Supporting comments inputs of the analysis, and therefore of the uncertainty involved. The margin between the P-box boundaries and the uncertainty it represents can be challenging to communicate to the stakeholders, as well as the implications of the upper bound on decision-making. Software RAMAS Risk Calc 4.0 (http://www.ramas.com/riskcalc), MATLAB, SIPmath available (https://www.probabilitymanagement.org/sipmath)

Evidence theory

Table A1-13. Evaluation -Evidence theory Dimension Criterion Evaluation Supporting comments Evidence theory does not try to describe uncertainty using a measurement, but it does provide a measure of the existing evidence that supports a particular subset of possibilities. A key advantage is allowing for non- mutually exclusive (e.g. overlapping subsets of a larger Significant fundamental set) to be computed, which is not allowed PA advantages by classical probability theory. Finally, the quantifiable gap between Belief and Plausibility is a measure of the uncertainty, which constitutes a valuable tool to evaluate Suitability different input sets for fire safety engineering such as the multiple -complex and unknown- design fire characteristics. This theory becomes less useful in cases where evidence is limited and where the assignment of basic probability assignments transform into a subjective exercise. As Insignificant NA Denœux [134] explains, "the complexity of aggregating limitations pieces of evidence increases exponentially with the number of sources", which leads to restrictions on the size of the problem to be handled. There are important challenges to implement evidence theory, beginning with the -often-subjective- definition of the ranges and basic probability assignments used for the quantities of interest. The combination of multiple Easy to inputs and then the processing can be challenging if the PA implement quantities are considered dependent. This implementation requires a subject expert guiding the process and ensuring the desired outcome is obtained, Effectiveness although the availability of tools such as IP Toolbox increases the ease of use. As an uncertainty function that maps into the [0, 1] range, the output is easy to communicate. However it presents similar challenges to the P-box’s outputs, as it is Easy to PA not a point-value but bounded distributions. communicate Furthermore, the use of the evidence measurements (Belief and Plausibility) increases the complexity of the information to be communicated. • MATLAB module DSI Toolbox (Auer, A Verified MATLAB Toolbox for the • Dempster-Shafer Theory) • IP Toolbox for MATLAB (Philipp Limbourg - https://www.mathworks.com/matlabcentral/fileexchange/9379-imprecise-probability- Software propagation-toolbox, https://www.uni-due.de/informationslogistik/ipptoolbox.php) available • R package 'EvCombR' (Alexander Karlsson, 2014) • R packace 'evclust' or Evidential Clustering (Thierry Denoeux, https://cran.r- project.org/web/packages/evclust/index.html) • IDRISI GIS Analysis in TerrSet (Clark University)

324

Appendix 1. Uncertainty analysis options for risk assessment

Dimension Criterion Evaluation Supporting comments • Orfeo Toolbox, Fusion of Classifications application (https://www.orfeo- toolbox.org/CookBook/Applications/app_FusionOfClassifications.html) • GRASS GIS program 'r.dst.combine' using Dempster's Rule of Combination (Benjamin Ducke, Gavin Powell, http://svn.osgeo.org/grass/grass- addons/grass6/dst/raster/r.dst.combine/description.html)

Possibility theory

Table A1-14. Evaluation -Possibility theory Dimension Criterion Evaluation Supporting comments Its output is a range of possible values for a variable of Significant PA interest, regardless of their probability. This adds a layer of advantages information to the risk picture [135]. There is the potential of obtaining over conservative Suitability solutions, as unlikely values can be found within the output. Insignificant Furthermore, the subsets representing the values of the NA limitations variable of interest should be nested; if this is not the case or cannot be ensured, evidence theory should be considered instead. There are important challenges to implement possibility theory, beginning with the -often-subjective- definition of the fuzzy sets used for the quantities of interest. The Easy to combination of multiple inputs and then the processing can NA implement be challenging depending on the sampling of the fuzzy Effectiveness inputs and also the defuzziphication technique selected. This implementation requires a subject expert guiding the process and ensuring the desired outcome is obtained. As in evidence theory, the outputs of possibility theory Easy to PA might be unfamiliar to the stakeholders, specially the communicate possibility and necessity measures. Software PossibleRisk, LEDTools, IP Toolbox for MATLAB available

Info-gap

Table A1-15. Evaluation -Info-gap Dimension Criterion Evaluation Supporting comments The outcomes of an info-gap analysis provides a quantification of uncertainties that cannot be assessed form a probabilistic approach and that are necessary to understand to Significant FA support decision-making. In the presence of unstructured advantages uncertainties such as in the case of fire safety concerns with new materials or assemblies, info-gap theory provides a Suitability functional alternative to support adequate decision-making. Most engineering analysis including those of fire safety engineering employ a large amount of models with Insignificant irreducible -and unstructured- uncertainties, both key PA limitations elements to apply the info-gap approach. Given the flexibility of this alternative, no significant limitations are identified except those related to its implementation. An info-gap analysis requires a detailed understanding of the system, the models involved and the inputs required, as well Easy to Effectiveness NA as an understanding of the uncertainty sources. Although implement these sources can be unstructured and unbounded, the info- gap is constructed based on those identified and an

325

Appendix 1. Uncertainty analysis options for risk assessment

Dimension Criterion Evaluation Supporting comments optimization scheme is used to estimate the robustness solution. This process can be mathematically demanding, requiring technical expertise for its implementation. Despite it does not use probabilities, the results of the robustness functions should be fairly easy to communicate to Easy to stakeholder, in particular when two or more competing PA communicate alternatives are compared. The significance of the robustness value can present significant challenges when taken out of a comparative analysis.

Strength of knowledge

Table A1-16. Evaluation -Strength of Knowledge Dimension Criterion Evaluation Supporting comments Adding the additional layer of strength of knowledge into the risk assessment, two same subjective probabilities can be judged in a very different way and therefore lead to different decision making. This allows explicitly stating the Significant FA uncertainty behind the assumptions and the expected advantages outcomes in the assessment. A significant advantage of this approach is that it can be used at all levels of risk assessment (e.g. ongoing operations, design considerations) and Suitability compatible with tools such as QRA or Risk Matrices. Agreement is required to define the criteria for the strength of knowledge ordinal levels, which can be a complicated task when the assessment involves personnel with very different Insignificant PA point of views or expertise. This constitutes the biggest limitations challenge of the alternative, as the criteria can be largely subjective and can diverge in the presence of stakeholders with extremely different points of view. Initially, SoK approach could be considered the easiest approach to account for uncertainty as it relies on a flexible scheme of qualitative levels that measure the supporting evidence and knowledge of the engineers. However, its actual Easy to PA implementation -as in the adequate use of risk analysis implement matrices- must be tailored to each system and to the objectives of the analysis. This increases the potential for Effectiveness misusing the alternative or requiring considerable additional resources for properly applying it. SoK yields a list of assumptions and limitations and the associated knowledge that supports them or questions them. Easy to FA This not only allows for ease of communication, but also communicate constitutes an additional information layer which can help formulate risk management actions.

Exploratory Model Analysis

Table A1-17. Evaluation -Exploratory Model Analysis Dimension Criterion Evaluation Supporting comments EMA is able to explore a vast set of possible scenarios including a large range of variables and variability in conditions. A statistical treatment of the resulting scenarios Significant Suitability FA allow identifying key conditions that would make the advantages decision alternatives inviable, hence finding robust strategies. Additionally, this methodology intakes new information, leading to adaptive strategies.

326

Appendix 1. Uncertainty analysis options for risk assessment

Detailed knowledge of the system and the internal and external variables that might influence its performance is needed. Furthermore, advanced modelling tools and data base management is required in order to host the scenarios discovery and its statistical treatment, limiting the range of users for whom this methodology is viable. Not only Insignificant specific competences in the field of computer experiments PA limitations is required, including potential knowledge and skills of machine-learning and artificial intelligence, but a deep knowledge of the structure of the problem is required. The latter refers to the need for competent fire engineers to guide the construction process of the computational experiments. This in sum shows significant limitations to the methodology. Depending on the amount of conditions that require exploring and the technique chosen to generate the possible universes, the implementation will require more planning and technical resources such as computing power. Given Easy to PA the existence of a well characterized model, the EMA implement Workbench [113] can be used to run it, given that a competent user leads the implementation. The latter Effectiveness constitutes the challenge for EMA implementation, as this technique is not typically used for fire risk assessments. The ease of communication directly depends on the defined outcomes, as well as the complexity of the model. However, Easy to the communication is based on the evaluated policies and NA communicate the scenario sets that yield either successful or unsuccessful results. For stakeholders used to probabilistic results, receiving such results could be challenging. Software R, EMA workbench (TU Delft) available

327

Appendix 2

Identifying uncertainties for field modelling of fully-developed compartment fires Journal Paper (submitted)

328

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Identifying uncertainties for field modelling of fully-developed compartment fires Jaime E. Cadena, The University of Queensland, Advanced Engineering Building, Australia Vinny Gupta, The University of Queensland, Advanced Engineering Building, Australia Agustin Majdalani, Cosmos Investments, United Kingdom & Argentina Jose L. Torero, University College London, Gower Street, United Kingdom Juan P. Hidalgo, The University of Queensland, Advanced Engineering Building, Australia

Abstract

When evaluating the fire effects on structural integrity, it is important to characterize the thermal boundary conditions within the fire compartments. Fire safety engineers use simple analytical and complex computational models to simulate the fire dynamics and estimate the thermal-boundary condition, which can then be used as an input to structural models. It is of particular relevance to model the thermal-boundary condition when the fire is fully developed. An important challenge arises as the thermal characteristics of fully-developed fires at larger length-scales are heterogeneous and are strongly dependent on the compartment geometry and fuel nature. Thus, modelling the behaviour of such fires is not a trivial exercise and requires field models. The experimental data for a highly instrumented medium-scale compartment is compared to the output of a CFD fire model. Qualitative and quantitative analyses were performed to identify uncertainty sources across the fire regimes. The ventilation-controlled regime exhibited uncertainties generated by the formulation of the combustion reaction under an oxygen-deprived environment. In the fuel-controlled regime, it is the definition of the fire source and its discrepancies with the real system that generate the uncertainty. The novelty stems from the fact that these findings emerge from a sensitivity analysis of the model and comparisons with the extensive data. Having identified the effect of input uncertainty on each end of the regimes allows to establish challenges inherent to a priori fire modelling and extract recommendations for fire safety practitioners engaged in the use of CFD models to quantify compartment fire behaviour.

Keywords

Fire safety; CFD; fire simulation; uncertainty

329

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

1 Introduction

Modelling of building fires is a key aspect of fire safety engineering, nevertheless, establishing the adequacy of model outputs is a very complex process [1, 2]. Fire is inherently complex and its interactions with the building pose equally severe challenges. Therefore, mathematical representations of many of the processes involved need to be strongly simplified. These simplifications introduce systemic uncertainties which are amplified by uncertainties in the quantification of all relevant input parameters, and the initial and environmental conditions. Despite the complexity of fire modelling and output uncertainties, if they are to be used appropriately, it remains necessary to demonstrate their value and trustworthiness. The objective of this paper is to establish a methodology to identify uncertainties of CFD fire models. It is not intended to qualify the adequacy of the model used but to enable the user to ascertain the importance of the different uncertainties. Beard [3] formulated the concept of the “Fire Design Triangle”. This concept provides a framework that serves to establish if a model is valuable for the purpose of design. The author establishes that, for a model to be valuable, it requires an adequate methodology of use, a competent user and mechanisms to establish the adequacy of the model. In the definition of value provided by Beard, it is clear that the author also includes trustworthiness. The author suggests that to establish the adequacy of the model and its limits of use, it is necessary to form Model Assessment Groups (MAG). These are independent groups that have no stakes on the model and that would scrutinise its development and use. Generic assessment procedures will enable to define, given a specific model, a methodology of use. A competency framework will scrutinise the user. Similar frameworks have been proposed by others [4]. However, given the number of models [5] and the uncoordinated nature of the fire safety engineering design process, frameworks of this nature have never been implemented. The consequences of this lack of framework have been demonstrated through a priori modelling round robins [6, 7] but it was only in 2016 that the first true MAG was established. This MAG has still to deliver any results that have practical implications for using fire models in design [8]. A significant limitation to the effectiveness of a MAG is the lack of experimental data at a scale, precision and density relevant to detailed fire models. Given the nature of fires and if all the model features are to be explored, the scale cannot be avoided [1]. Traditionally, the adequacy of field-scale models has been addressed by studies that compare model predictions against available experimental data. There exists a myriad of these comparison based studies, and some examples covering an extensive set of scenarios are listed in the following references

330

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

[9-21]. Among the large set of available field models [5], the Fire Dynamics Simulator (FDS, current version 6.7.2) contains the highest number of such comparisons. These studies are intended to contribute to understanding the model's capabilities when used to predict the behaviour of real fire scenarios. The information extracted could then be used for model improvements and to inform the assessment conducted by a MAG. Nevertheless, this so far has been rarely the case. The complexity of the combined phenomena in a fire and the limited measurements implemented in the experiments do not enable, by means of comparison, to establish which specific sub-models require further development or improvement [22]. In the meantime, the models continue to be modified and, in the case of FDS, the developers have chosen the approach of comparing the ‘model predictions to as many experiments as possible’ [10]. Most efforts to collect relevant data and to use it to assess the adequacy of fire models have focused on small isolated or interconnected compartments [2, 10, 21]. Nevertheless, the last two decades have been characterised by a strong shift in architecture towards large (and many times, interconnected) spaces; this is particularly common in tall building design [23]. Therefore, there is a practical need to conduct detailed experiments that enable a benchmarking process between adequately instrumented large open floor plan experiments and modelling results. Some of these experiments have appeared recently in the literature [24-26]. These experiments have been analysed and, in some cases, phenomena unique to large compartments have been identified and described [27]. Nevertheless, their analysis has never been focused on the discussion of model uncertainty. The use of the model also defines the nature of the uncertainties. In fires, model results can be used as inputs to assess different protective measures or the interactions between people and the fire. Smoke production modelling is associated with the definition of smoke management systems; toxic concentrations modelling provides insight into human/smoke interaction and defining compartmentation and evacuation; heat transfer modelling pertains the evaluation of structural performance. The uncertainties involved when modelling each one of these aspects of fire dynamics are very different. So are the variables that need to be evaluated. Most studies have focused on assessing smoke production in what relates the enabling of smoke management strategies [16, 28, 29] and very few have focused on the use of these tools to define the thermal boundary condition for structural performance assessment [30-33]. Given the large differences in resolution and time scales between the solid and gas phases [9], the variables of interest are very different from those associated with other applications [32, 33]. With no exception, all the structural performance studies have used a small compartment to

331

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires approach the assessment of the fire. Even in the analysis of the structural failure of the World Trade Centre [30], the tests were of a small sector of the floor plan (a cubicle), and it was only the model that extrapolated the behaviour. Thus, no uncertainty analysis was possible. It is tempting to start the process of evaluating uncertainties of a model by direct comparison with large compartment fire tests. Nevertheless, the complexity of the phenomena and the limited instrumentation will make it very difficult to establish links between uncertainties in modelling, modelling outputs and the differences with experimental results. This is of particular relevance when modelling fully-developed fires to estimate structural performance. Small to medium-scale experiments enable a high degree of control of the boundary and environmental conditions used to explore a wide parametric space. If such experiments are designed to explore phenomena occurring for large compartment fires, [34], then such experiments provide an advantage of complementing the limited database of large-scale experiments. This study attempts to characterize the different uncertainties in modelling a highly instrumented small compartment fire experiment using FDS (current version 6.7.2), focusing on the whole range of fully-developed fire behaviour; this is graphically illustrated in Figure A2-.

Figure A2-1. Graphic representation of the problem studied.

332

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

2 Reproducing Large Compartment Fire Dynamics with Small Compartment Experiments

Within the context of a compartment fire, the relation between compartment geometry, ventilation and the fire must be understood to establish the dynamics of the fire. In a subset of scenarios where fire dynamics are a function of ventilation and fire size, i.e. the compartment fire framework, the supply of air can be used to characterise the temperature and heat transfer coefficients (i.e. emissivity of the gas and convective heat transfer coefficients) and together with the mass of combustible materials, the duration of the fire until burnout (Ventilation- controlled or Regime I [35]). In all other scenarios, the complex fluid mechanics control the heat transfer to a structure and duration of the fire (Fuel-controlled or Regime II [35]). The latter scenario is most commonly the case for large open floor plan spaces while the former for small almost-cubic compartments with limited ventilation [36]. Majdalani [34] concluded that open architectural configurations challenge the fire safety engineer with understanding – and managing – the energy distribution across the compartment and flowing out of it. Thomas et al.’s [35] description of ventilation-controlled fires includes sufficiently small openings to enable a build-up of perfectly mixed hot gasses within the compartment. Therefore, the compartment experiences relatively uniform temperatures, the internal velocities are negligible, and pressures are purely hydrostatic. The exchange of flow through the opening is controlled only by the temperature difference between the interior and exterior of the compartment. Larger openings or compartments reduce the depth of the smoke layer and enable the fire to deliver the largest buoyancy force. The result is a complex flow field dominated by the fire that drives mass and energy in and out of the compartment (fuel-controlled fire). The characterisation of these flow fields requires full resolution of the mass, momentum, energy, and species transport in space and time. Majdalani et al. [28] reported a series of experiments conducted to understand the dynamics between the two regimes and the transitional behaviours. The experiments were conducted using a compartment of 0.82 m x 1.06 m x 0.82 m in dimension in which the front opening was adjusted to provide 100% to 20% of ventilation and the fire used propane gas burners across the base of the compartment at three different flows (0.5, 1.0 and 1.5 g/s). Instrumentation was placed to ensure a high level of resolution in the characterisation of the experimental conditions. Gas temperatures, IRHF (incident radiant heat flux) to the solid boundaries and gas velocities were measured. These experiments will be used here to perform an uncertainty

333

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

analysis pertaining to the modelling of fully-developed fires in compartments. While other experiments in the literature could supplement this data, none of these was developed with the level of control and instrumentation density of those by Majdalani et al. [34] (section 3 summarizes the experimental setup). The level of control and the sensor density within these experiments allow for the uncertainty characterisation across the range of fully-developed compartment fire regimes. The scale of the experiment introduces uncertainty, but it is a comprehensive case study to understand how the diverse inputs of compartment fire modelling are linked to uncertainty. The rest of this study consists of a description of the experimental setup (section 3), the modelling setup (section 4) and the methodology to account for uncertainty (section 5). Qualitative and quantitative uncertainty analyses are presented in section 6 and 7, respectivelly. Finally, the results are discussed in detail, focusing on the different uncertainty sources relevant across the range of fire regimes and the implications for design process (section 8).

3 Experimental setup

The experimental setup developed by Majdalani et al. [34] used a compartment of external dimensions 0.92 m wide, 1.16 m deep and with a height of 0.92 m high. The internal dimensions of the compartment were 0.82 x 1.06 x 0.82 m. The compartment boundaries consisted of 6 cm thick ceramic fibre sheets. The opening sizes were adjustable using two horizontal ceramic fibre sheets. The fire within the compartment was attained through a set of 128 small gas-fired burners (each with a 3.13 mm2 area) equally distributed across the floor area of the compartment using four thermally insulted pipes (Figure A2-2). The bottom of the pipes was positioned one centimetre above the floor level. The burner holes faced downward to neutralize the momentum of the gas fuel and induce buoyancy-driven flames [37]. The experimental matrix of experimental scenarios used in and modelled with FDS is presented in Table A2-. Table A2-1. Experimental matrix associated with the experiments [27] Ventilation Ventilation area [m2] Inverse Opening factor [m-½] 20% 0.672 70.1 40% 0.538 23.9 60% 0.403 12.5 80% 0.269 7.8 100% 0.134 5.4

The inverse opening factor, , presented is based on Eq. (1) [35], where AT is the total internal

area of the compartment (excluding𝜙𝜙′ the floor and openings), AW is the opening area, and H is

334

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires its height of the opening. This definition follows Thomas and Heselden [35]. The opening factor will be used as a reference to characterize the different experiments, and it is key to determining each experiment's fire regime. = (1) ′ 𝐴𝐴𝑇𝑇 𝜙𝜙 𝐴𝐴𝑊𝑊√𝐻𝐻𝑊𝑊

Figure A2-2. Graphic representation of the experimental burner system A fixed flow of 0.5, 1.0, or 1.5 g/s of propane was distributed through the burners and – using the ideal heat of combustion for propane – translate into fires of 25.2, 50.3 and 75.5 kW, respectively. These fires are denoted as small, medium and large fires throughout this study. In order to obtain the fire sizes for a scaled compartment of dimensions 2.46 m wide, 3.24 m deep and 2.46 m high, the non-dimensional heat release rate is calculated by applying Eq. (2) [38] and Eq. (3) [39]; the results are presented in Table A2-2. It must be noted that Eq. (2) is a non- dimensional grouping that only applies where the enthalpy (bottom term) depends on the ventilation parameter. In contrast, Eq.(3) applies when the fire is driving the flow.

= (2) ∗ 𝑄𝑄̇ 𝑜𝑜 𝑄𝑄 𝜌𝜌𝑐𝑐𝑝𝑝𝑇𝑇∞√𝑔𝑔𝐴𝐴𝑜𝑜√𝐻𝐻 = (3) ∗ 𝑄𝑄̇ 2 Table A2-2𝑄𝑄. Scaled𝜌𝜌𝑐𝑐 𝑝𝑝fire𝑇𝑇∞� size𝑔𝑔𝑔𝑔∙𝐷𝐷 calculation

C3H8 Flow Rate HRRPUA scaled (kW) Classification 2 (g/s) (kW/m ) ∗ ∗ 𝑸𝑸(kW)̇ 𝑸𝑸̇ 𝑸𝑸𝟎𝟎 𝑸𝑸 0.5 20.90 24.0 0.067 5.12 325 Small 1 41.81 48.1 0.134 10.24 650 Medium 1.5 62.71 72.1 0.201 15.36 1000 Large

335

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

A variable vertical ventilation area between 20% and 100% of the opening height allowed observing the characteristic fully-developed fire behaviours ranging between ventilation- to fuel-controlled. From the eighteen experiments conducted within the original scope, the configuration with both front and rear doors fully open of was not considered in the present work. This configuration represents an extreme case of fuel-controlled fire, and for this study, it was decided to limit the scope to those configurations with only one varying opening.

The compartment was instrumented with 54 gas temperature thermocouples (K-Type, 3 mm thickness) within the compartment (Figure A2-10), 45 thin-skin calorimeters (TSCs) at the exposed surfaces of all inner surfaces (Figure A2-7) following the design of Hidalgo et al. [40] and eight bi-directional velocity probes at the opening [41]. For more details of the original experimental set up the reader is directed to Majdalani et al. [34] and Majdalani [37].

Figure A2-3. Distribution of thermocouples for gas temperatures

336

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

LC5 B5 R5

LB4 LC4 LD4 B4 R4

LB3 LC3 LD3 B3 R3

LB2 LC2 LD2 B2 R2

LC1 B1 R1

Left wall Back wall Right wall

Ceiling Floor CE3 FE3

CD2 CD3 CD4 FD2 FD3 FD4

CC1 CC2 CC3 CC4 CC5 FC2 FC3 FC4

CB2 CB3 CB4 FB2 FB3 FB4

CA3 FA3 Front Front Figure A2-4. Distribution of incident radiant heat flux gauges

4 Simulation setup

The experiments were modelled using FDS v.6.5.3, and key specifications for the simulation setup are presented in Table A2-3. Due to computational limitations, the 3000 seconds of experimental behaviour could not be simulated; an alternative timeframe of 750 seconds was selected. Both compartment geometry and opening are identical to the experimental in the simulation setup. The walls of the compartment were given properties of ceramic fibre sheet with specific heat, cp = 0.9 kJ/kg.K, density = 500kg/m3, thermal conductivity as a linear function of temperature (0.07 W/m.K at 300°Cρ to 0.2 W/m.K at 1000 °C) based on information provided by the manufacturer. The reduced simulation time was deemed appropriate due to the low thermal inertia of the compartment boundaries. As the characteristic time to heat the solid boundaries is small due to the low thermal inertia, a thermal equilibrium in the gas-phase will be attained quickly. Table A2-3. Specifications of the simulation setup constructed in FDS v.6.5.3 Simulation Parameter Values

Domain Range [x0, xf, y0, yf, z0, zf] (m) Single Mesh [0.00, 2.00, 0.00, 2.00, 0.00, 2.00] Cell Size [Length, Width, Height] (m) 0.02 x 0.02 x 0.02 Number of cells 1,000,000 cells Simulation Time (s) 750s

Fuel Propane (C3H8)

337

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Simulation Parameter Values Heat of Combustion 46.45 kJ/g Combustion Efficiency 0.90 Soot Yield 0.01 [42] Number of radiation angles 100

Back of compartment

Simulated burner holes 2 x 2 cm

Front of compartment

Figure A2-5. Graphic representation of the simulated burner system As the experimental heat release rates from the gas burners spread across the floor were known, the heat release rate per unit of area (HRRPUA) was adjusted for the new number of burners. This enabled preserving the total heat release rate despite the simplification of the geometry of the burners. The experimental HRR of 20.9, 41.8, or 62.7 kW for each fire size required a HRRPUA in the model of 3,266, 6,532 and 9,709 kW/m2, respectively. Up until this point, most of the experimental features can be reproduced with a high degree of fidelity. However, one of the components of uncertainty in fire modelling is the fire source. In this case, the fire source is the burner system. The characteristic length-scale of the experimental gas burners is the diameter of holes in the pipe (2mm). Such a length-scale is incompatible with Large Eddy Simulation (LES) as the length-scales are too small. Instead, Direct Numerical Simulation would be required to model the 128 burners at a very small cell size. Due to the need to resolve the turbulent characteristics of the flow within the entirety of the compartment geometry, it is not feasible to model the burners using DNS. Thus, the modelling of the fire was simplified by approximating the burners as 16 square (4 cm2 each) upward-facing burners, distributed across the floor of the simulated compartment (Figure A2- 5). This simplification results in a fire source area that is 16 times larger than the holes in the experimental setup, and thus overall injection velocities to maintain the same HRRPUA will be smaller. The different orientation of holes and burners is intended to compensate for this

338

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

difference. If the flames are dominated by buoyancy, then it can be expected that the differences in the injection velocities fields will have a mild effect on the output of the simulation. Nevertheless, it is expected that changing the boundary condition of the fire will undoubtedly introduce uncertainty, however, such simplifications are often necessary for modelling fires at length-scales appropriate to real fires in buildings. It is then essential to verify that the simplifications applied to the modelled fire boundary condition maintains similarity to the buoyant flames reported within the experiment [34]. The buoyancy of the flames can be assessed either by quantifying the non-dimensional heat release rate ( ) or the Froude number. Based on the calculated values of from Table A2-2, the ∗ ∗ fires correspond𝑄𝑄̇𝑐𝑐 to Region III of Figure A2-6, which indicates a transitional𝑄𝑄̇𝑐𝑐 region of both buoyancy-driven and turbulent flames. To estimate the Froude number, the maximum and minimum flow velocities (Vmax and Vmin) at the burner source are extracted from numerical simulations by Majdalani [37]. Assuming that the characteristic length-scale of the simplified burners in the model are approximately 2cm, the Froude number is estimated, and the flames still correspond to Region III of Figure A2-6. The experimental burners correspond to Region I (Figure A2-6) and are purely buoyant, therefore uncertainty in the fire boundary condition is introduced when changing the boundary condition representing the burners. This uncertainty is associated to the reduced buoyancy of the flames.

339

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Figure A2-6. Identification of the type of flames expected from the simplified burners

Based on the non-dimensional heat release rate ( ; vertical lines for each fire size) and the ∗ Froude number (shaded areas; light red corresponds𝑄𝑄̇𝑐𝑐 to the range of Froude number for the 20% configuration, while the blue region corresponds to the Froude numbers for the 100% configuration). Plot taken from Drysdale [39]

A mesh size of 2 cm was selected as it provides a balance between adequately capturing the spatial distributions in the temperature and flow field, without compromising computational times. Using the user guidelines of FDS a characteristic fire diameter (D*) was estimated [43] using the heat release rate in each simulation. The smaller the mesh size is, the more cells will cover D* and therefore the resolution increases. The relation between the numbers of cells required and the mesh size (δx) is described in Figure A2-7, and from this, it can be observed that the selected 2 cm mesh implies D* will be covered by 9 to 13 cells. Comparing this range with those provided for different tests in the FDS validation guide [10], 3 to 9 for tests with similar height/D* ratio, justify the selected mesh size configuration.

340

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

60.0

50.0

40.0 x

δ Small 30.0

D*/ Medium 20.0 Large

10.0

0.0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 δx [m] Figure A2-7. Cell number comparison for different mesh sizes Taking into account the analysis of D*, a sensitivity analysis was conducted comparing the results for 1 cm and 2 cm mesh sizes. This required estimating the error between the simulation output and the experimental data, defined as in Eq. (4), assuming a quasi-steady state of about 400 seconds (see section 0 for further details on time averaging). The grid sensitivity study is selected for the 100% opening configuration with a large fire, with the sensitivity of all the individual thermocouples in space analysed. Other configurations were not considered as the model output would be limited by the small concentrations of oxygen associated with ventilation-controlled configurations. The output of the grid sensitivity study is presented in Figure A2-8. A finer mesh size of 1 cm mesh size yields an error reduction for the temperatures closer to the burners, with the most notable one the reduction for tree C1 of up to 50%. However, for the rest of the heights, there is no improvement in the error beyond 20% despite a significant increase in computational cost. Finally, based on the times measured for both meshes, the coarser 2 cm mesh requires approximately 1/8 of the computational time. The limited error reduction of the 1 cm mesh size along with the favourable computational times of the coarser mesh supported the decision of using it throughout this study. The results of Figure A2-8 are further detailed in Annex 1, where time-temperature curves for the experimental data and simulations with both mesh sizes are presented for specific thermocouples.

= 100% (4) 𝑇𝑇𝐹𝐹𝐹𝐹𝐹𝐹−𝑇𝑇𝐸𝐸𝐸𝐸𝐸𝐸 𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 𝑇𝑇𝐸𝐸𝐸𝐸𝐸𝐸 ∗

341

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Figure A2-8. Gas temperature error comparison between 1 cm (black markers) and 2 cm (grey markers) mesh size for 100% - large fire Top labels indicate the height of the thermocouple layer, with 0.1 m being closer to the floor and 0.75 m being 7 cm below the ceiling. Error bars represent one standard deviation of the errors estimated as in Eq. (4) for a period of 400 seconds. With both the mesh size and the burners defined, the simulation setup was used to obtain the desired quantities. To do so, the TSCs were modelled as being the output an incident radiant heat flux. Temperatures were modelled using thermocouples that incorporate the radiative contribution [44]. The velocity probes were modelled by taking the output of the overall magnitude, as well as capturing vertical and horizontal components.

5 Methodology

To discern the effect of the two distinct uncertainty sources (i.e. model and user decisions), qualitative and quantitative analyses on the output of the FDS model were conducted. The qualitative component uses point-to-point representations and additional model outputs (temperature and heat flux contours) to describe the degree to which the experimental behaviour is modelled. Departures from the experimental behaviour allow identifying compartment configurations for which uncertainty is relevant. These departures are later quantified using the model bias factor (δ), which is “a way to express by what percentage the model is over or under predicting on average” [10, 45, 46]. The bias factor is a useful

342

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

mathematical tool to quantify uncertainty, but as any statistic, its interpretation should be careful and based on a phenomenological understanding of the physical characteristics of the compartment fire. Using the results of the qualitative and quantitative analyses, the uncertainty arising at each end of the fire regime spectrum is discussed. In order to perform point-to-point comparisons and the calculation of the bias factor, a data procedure is implemented as described in detail in Annex 2.

6 Qualitative analysis

The behaviour of the flames, thermal fields and gas concentrations for the two limiting regimes of behaviour are fundamental elements to assess the bounds of applicability of the physical model. Contours of the temperatures and the HRRPUV in Figure A2-9 and Figure A2-10 are developed from the models to identify the characteristic fire behaviour at the limiting ventilation conditions. Figure A2-9 shows the thermal fields for the 40% and 100% configurations. The uniform temperature in Figure A2-9(a) which corresponds to the 40% opening configuration and illustrates how the lack of oxygen within the compartment yields a uniform distribution of temperatures and an egress of hot gases. This is in strike contrast with the 100% configuration (Figure A2-9 (b)). Figure A2-9 (a) shows that it will be required to accurately describe oxygen entry and consumption to resolve the fire dynamics and obtain the appropriate temperature fields for an underventilated compartment.

Figure A2-9. Gas temperature [°C] slice at the middle of the compartment (averaged at 70 s) for a medium-sized fire for the a) 40% and b) 100% configurations. Figure A2-10 presents the flames using the heat release rate per unit volume (HRRPUV), which is useful to show how the ventilation factor changes the flame shape. First, there is a clear difference in flame tilt, as seen between Figure A2-10a and Figure A2-10d. The former tilts

343

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

forwards while the latter tilts towards the back, also noting that in the latter there is a separation of the flames. Furthermore, the heat output is observed as a plume dispersing towards the opening in Figure A2-10a, while in the rest of configurations, the heat output is limited to the flames. The presence of blockages as the opening as in Figure A2-10a and Figure A2-10b flaten the shape of the areas with the highest HRRPUV (as seen in red in Figure A2-10). Once the restriction is minimum as in Figure A2-10c or none as in Figure A2-10d, these areas tend to elongate and tilt towards the back. These observations suggest that factors influencing flame shape could potentially impact the IRHF readings and therefore the measured uncertainty.

a) c)

c) d)

Figure A2-10. Heat release rate per unit volume [kW/m3] for a medium sized fire for the a) 20%, b) 40%, c) 80% and d) 100% configurations. Time-averaged results for a 30 second period.

6.1 Gas temperatures Following the qualitative description of the main phenomena observed in the compartment for different configurations, the results are presented and analysed in detail in the following sections. Scatter plots are used (typically used in FDS validations) with a diagonal line to define the limit between over and underprediction, and error bars accounting for one standard deviation of the experimental data and FDS outputs for the period analysed. Additional dotted

344

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires lines are incorporated in the scatter plots, which account for a deviation of ±10% from the diagonal. Figure A2-11 presents the scatter plots for all thermocouples at each one of the five configurations, excluding the thermocouples closest to the burners. These thermocouples are not directly comparable due to how the burner is modelled, so the data near the burner will be excluded in the quantitative analysis. The data indicates different levels of deviation between the simulation and the experiment, which can be observed by the distance of each marker (representing the time-averaged value for a single thermocouple) from the diagonal. It can be observed that the deviations tend to be significantly smaller and the grouping narrower for the small fire size. This is the product of the relative uniformity in the compartment temperatures. Hence the magnitude of the error bars is significantly smaller compared to those of the 100% configuration. Despite differences in how narrow the group of thermocouples are across the different configurations, as the fire size increases, so does the deviation from the diagonal representing a match between the experiment and the simulation values. Extreme representations of this trend are the 20% and 100% configurations, with underestimations of - 100% for the thermocouples located at the height of 0.23 m. This leads to a second observation concerning thermocouple height, as the deviation is largest for those closer to the burners.

Figure A2-11. Temperature scatter plots for all thermocouples at each configuration. The colour scale refers to the thermocouple (TC) height, with the darkest closer to the floor and

345

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

the white one closest to the ceiling of the compartment. The markers refer to the fire size as described in Table A2-2, with the triangle pointing up for small fires, triangle pointing down for medium fires and circle for large fires. The thermocouples located in the upper middle of the compartment display a more consistent behaviour as indicated by the results of Figure A2-12. Thermocouple temperatures vary significantly for the different configurations. At = 70.1 m-½ (20% configuration), ′ temperatures are underestimated for the medium and large𝜙𝜙 fires by a significant margin. The configuration with the largest proportion of thermocouples within the ±10% range is the 40% configuration, while those between 60% and 100% configurations display different levels of deviation from the diagonal, along with a wider spread in the temperature range captured in each fire size. The latter is particularly noticeable for fully open configuration (100% or an opening factor of 5.4 m-½), where the grouping of thermocouples by fire size in previous configurations is no longer easily distinguishable.

Figure A2-12. Temperature scatter plots for thermocouples from the middle of the compartment to the ceiling at each configuration. The colour scale refers to the thermocouple (TC) height, with the darkest closer to the middle of the compartment and the white one closest to the ceiling of the compartment. The markers refer to the fire size as described in

346

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Table A2-2, with the triangle pointing up for small fires, triangle pointing down for medium fires and circle for large fires. The spatial distribution of temperatures within the compartment corresponds to the description of the burning regimes by Thomas et al. [35] and by Majdalani et al. [34] as the configurations with small openings yield a hot uniform temperature distribution and the configurations with larger openings yields lower temperatures with a large spatial temperature distribution. Contrasting the thermocouple temperatures confirms the breakpoint between fire regimes postulated by Majdalani [37]. For the 20% configuration, the thermocouple temperatures are highly uniform. As the opening size is increased, the scatter of temperatures increases. The breakpoint in the model output occurs between the 20% to 40% configurations. The transition is identified by a significant increase in the model error. Higher HRR values correlate with a more significant underprediction of the data. This trend is consistent throughout all configurations and is more clearly observable for configuration 100% in Figure A2-12.

6.2 Incident radiant heat flux (IRHF) Gas temperatures are governed by the entrainment model given that the flames are resolved with the selected mesh size (see section 4), and there is sufficient oxidiser transport into the reaction zone to obtain the required mixture fraction to enable combustion. It was observed experimentally that the smoke layer was optically thin irrespective of the opening factor due to the low soot yield of propane gas. Therefore, the IRHF is governed only by the temperature of the hot surfaces and the flame shape, as the emissivity of the smoke layer is small [37]. An initial analysis was carried out quantitatively for IRHF presenting typical scatterplots for the time-averaged simulated and experimental results with ±10% error bar references [47]. The modelled IRHF for 40% and 100% configurations were consistent with Majdalani’s [34] experimental observations regarding the change in behaviour between a ventilation-controlled fire and a fuel-controlled one, and contribute the additional layer of information regarding FDS’ capability to reproduce the experimental measurements. This section further details the IRHF behaviour for the different configurations of the compartment and discusses the underlying modelling uncertainties. The spatial measurements of the IRHF on the back wall and the ceiling are considered here. The scatter plot for the ceiling is presented in Figure A2-13 where significant underpredictions are observed for the 20% configuration in the ceiling for medium and large fires, with a good agreement for small fire configurations. Similarly, this trend is also observed in all other walls and can be associated with the low oxygen concentrations that challenge the limits of the FDS

347

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires extinction model. In contrast to these significant underpredictions, configurations of 40% and 60% present and excellent agreement for the ceiling, regardless of the fire size. For 80% and 100% configurations, slight overpredictions are noticeable, particularly for the large fire.

Figure A2-13. Incident radiant heat flux scatter plots for the ceiling. The colour bar refers to the TSCs distributed throughout the ceiling as displayed on the left bottom corner. The back wall is analysed using four TSCs placed along the centreline as shown in Figure A2- 14 (the TSC closest to the burners is excluded). As in the ceiling results, a significant underprediction for the medium and large fires for the 20% configuration can be observed, as well as a general tendency to overpredict the IRHF as the fire size increases. The devices located closest to the ceiling (B4, B5) show an adequate fit and fall within the ±10% reference lines for most configurations.

348

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Figure A2-14. Incident radiant heat flux scatter plots for the back wall. The results for IRHF reinforce the observations from the gas temperature data regarding the significant underprediction for the underventilated configuration of 20% opening. The previously presented trend of increasing underprediction of gas temperatures as the compartment goes from underventilated to fuel-controlled, is not observed. In fact, the IRHF results show the opposite trend, with overpredictions increasing as the opening area and fire size increase.

6.3 Gas velocities The scatter plots for the gas velocities are presented in Figure A2-15. The results indicate that the devices located towards the mid-height of the opening tend to align with the diagonal, while those located either towards the upper or lower limit indicate errors for the inflows reaching as much as 243% and for outflows as much as 323%. As the configuration of the compartment goes from ventilation- to fuel-controlled, the experimental error bars representing the standard deviation enlarge significantly. These errors seem to be accentuated as the fire size increases. The only general trend is that of overprediction of flows close to 0 m/s and underprediction of both high inflow and outflow velocities.

349

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

20% 40% 60%

80% 100%

Figure A2-15. Gas velocity scatter plots It is important to note that the bi-directional probes used were coupled with a low-pressure range DPT to obtain the pressures to calculate the velocity. The coupling of this assembly has been shown to have significant errors; particularly at low Reynolds number flows typical of compartment fires [48]. Furthermore, the arrangement of the probes can induce error if they are not properly aligned with the bulk flow direction. When misalignment occurs, e.g. due to the presence of turbulent flows, the velocity cannot be accurately measured, and some studies report errors of around ±20% [49]. The presented results do not allow distinguishing whether the error is a product of the measurement uncertainty or in what proportion. The inflow velocity vectors have quite high vertical components as the flow is entrained into the compartment, with generally small magnitudes [36]. This is consistent with the observations of high vertical components in compartments with low ventilation area (40%- 60%) by Majdalani [37]. The result is that the direction of bulk flow in the experiment generally does not align with the probe, thus an incorrect velocity is measured. The simulation results in Figure A2-15 show the highest error for the lower probes in all cases (except for the 20% configuration due to well-stirred conditions).

7 Quantitative analysis

A bias factor less than the unity reflects the model's tendency to underpredict the quantity, potentially underestimating the hot gas layer temperature or the heat fluxes to the solid boundaries. Underestimating these quantities in the design could lead to a structure that cannot

350

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires withstand or with reduced resistance to real fire conditions. On the other hand, a bias factor larger than unit reflects a tendency of the model to overestimate the quantities and could lead to unrealistic conservative designs; these directly translate into an economic loss. To estimate the bias factor, the calculation starts from the combined experimental uncertainty ( ), a function of the measurement uncertainty and the propagated input uncertainty. Table

A2𝝈𝝈�𝑬𝑬-4 presents the selected quantities associated to this work (gas temperature, heat fluxes and velocity) as registered in the FDS validation guide [10], which are compared with the results of this work. McGrattan et al. [10, 45] provide a procedure to obtain the bias factor. Table A2-4. Bias factor, experimental and FDS uncertainties as reported in [10]. Quantity Device Bias

Hot gas layer temperature (Natural Ventilation) Thermocouple 0.07𝝈𝝈�𝑬𝑬 0.07𝝈𝝈�𝑴𝑴 1.04 Hot gas layer temperature (No Ventilation) Thermocouple 0.07 0.07 1.12 Velocity Bi-directional probe 0.08 0.09 0.99 Surface heat flux Thin skin calorimeter 0.11 0.23 1.02

The uncertainty values calculated for all experimental devices are presented in Table A2-5. The heat fluxes are calculated using a correction-factor method detailed by Hidalgo et al. [40]. All other measurement uncertainties have been obtained from the devices manufacturer’s information as recorded in [37]. Taking these measurement uncertainties, a combined measurement is obtained by calculating the square root of the squares of the measurement uncertainty and the propagation uncertainty [20].The resulting values (New ) are then used to estimate the model standard deviation. 𝝈𝝈�𝑬𝑬 Table A2-5. Adjustment of combined measuring uncertainty for experimental data *This is reported as a maximum uncertainty of ±3.5˚C and the percentage was calculated using the lowest steady-state averaged temperatures recorded in the experiment, corresponding to the 100% ventilation configuration with a 0.5 g/s of propane (~50˚C). Propagation Measurement Measurement New Device uncertainty (%) uncertainty (%) [37] uncertainty (%) [20] [20]𝑬𝑬 [20] 𝝈𝝈� 𝑬𝑬 Thermocouple 7* 5 5 0.07 0.09𝝈𝝈� Bi-directional 7 10 21 0.23 0.22 probe Thin skin 10 5 10 0.11 0.14 calorimeter

351 Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

A MATLAB script is used to estimate the bias factor for each of the variables of interest for each of the configurations, and the results are presented in Table A2-6. Gas flow velocities were excluded from the estimation of bias factors due to the numerical restriction arising from errors with opposite signs, i.e. positive and negative velocities. Heat flux sensors B1 and LC2 show faulty data and were removed from the analysis. The floor devices are omitted due to the significant differences between the modelled and the experimental burners (see sections 4 and 8.2).

Table A2-6. Tendency to over- or under-estimate gas temperatures (δTgas) and heat fluxes

(δIRHF) Qualitative analysis may indicate *opposite behaviour or **magnitude discrepancy.

-½ Ventilation Ao [m ] Fire size δTgas δIRHF

S 6% 28% 20% 70.1 M -26% -46% L -40% -72% S 0% 9% 40% 23.9 M -3% 4%* L -6% -3%** S 3% -1% 60% 12.5 M -3% 23%** L -8% 16% S 0% -4% 80% 7.8 M -10% 28% L -14% 30% S -12% 2% 100% 5.4 M -16% 25% L -18% 40% Figure A2-16 shows a comparison of the bias factors for gas temperatures with those reported in the FDS Validation & Verification Guide [10]. Agreement is shown for gas temperatures in small fire configurations (δT = [1 - 1.06]), with the exception for 100% configuration (δT = 0.88), while larger fire sizes present an increasing under-estimation of gas temperatures. In FDS, the state equation that describes the relationship between density, species mass fractions and temperature is used to compute the latter, which is done through the numerical solution of the Naver-Stokes equations [2]. Mismatches between the experimental and simulated species

352

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires mass fractions could account for the under predictions of temperature, implying limitations in the combustion modelling.

Small fires Medium fires Large fires Reported Bias 1.2 1.4

1 1.2 1 0.8 0.8 0.6 Bias Bias 0.6 0.4 0.4

0.2 0.2

0 0 20% 40% 60% 80% 100% 20% 40% 60% 80% 100% Configuration Configuration Figure A2-16. Comparison of reported and obtained FDS bias factor for temperature (left) and heat flux (right) For IRHF, the bias factors for the 20% and 40% configurations seem to closely match those of gas temperatures in magnitude and trend, noting a major underprediction for the larger fires of the 20% configuration. The over predictions closer to fuel-controlled fires seems to be inconsistent with the under prediction of gas temperatures. IRHF is a function of gas and solids temperature and the radiative fraction ( ), with a default value for propane of 0.3 [43]. FDS v6 integrates the heat term associated with𝜒𝜒𝑅𝑅 radiation in order to match the pre-defined value. This value in real fires varies as a function of the fire size, as demonstrated experimentally𝜒𝜒𝑅𝑅 by Hu [50]. It is feasible that the radiative fraction simplification does not allow accurately reproducing the IRHF at the solid surfaces, particularly at fuel-controlled configurations with higher fire sizes. In Figure A2-16, the over predictions of IRHF for 60-100% configurations and larger fire sizes are related to large availability of oxygen in the compartment as well as higher gas flows, which could hinder the correct estimation of the radiative fraction released from the flames into the compartment. This matches the qualitative description of how the flame shape varies between a ventilation- and a fuel-controlled fire. These bias factors cannot be directly compared to those reported by FDS Validation & Verification Guide [10], as the latter represents a large body of validations. The results indicate that ventilation and fire size have significant effects on the capability of the model to reproduce the experimental results. The results of the qualitative analysis indicate that some of the bias

353

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

factors do not accurately capture the discrepancies between experimental data and simulation outputs, reinforcing the need to interpret such factors with caution.

8 Discussion

8.1 Uncertainty arising from model parameters The quantitative and qualitative analyses of the model show a breakpoint in the model’s ability to estimate the fire dynamics once the opening sizes are reduced to an inverse opening factor of = 70.1 m-0.5. Under this ventilation condition, gas-phase temperatures and radiative heat ′ fluxes𝜙𝜙 significantly underpredict the experimentally measured quantities. The statistical analysis shows that the magnitude of this error corresponds to an increased heat release rate from the fire source. Contrasting the input and output HRR from the simulations in Figure A2- 17 shows that energy conservation is maintained ( = 0) for all configurations except -0.5 those at = 70.1 m for the medium and large fireΔ𝐻𝐻 sources.𝐻𝐻𝐻𝐻 ′ 𝜙𝜙

Figure A2-17. Delta HRR for the simulation timeframe for all configurations, with respect to the specified HRR (see Table A2-2) The inability in attaining energy conservation for the configurations with small openings is likely attributed to the extinction submodel suppressing combustion as the compartment burns within a ventilation-controlled regime. This is a particular issue as the combustion model is

354

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires necessary to establish the flame geometry and the thermal fields, thus spurious extinction of the fire within the compartment will have an effect on the output. The residence times of a ventilation-controlled compartment are very long due to the static flow fields [34]. Thus, there is innate importance on the model resolution as modelling the thermal environment of the compartment correctly requires an accurate approximation of the flame sheet; a requirement that cannot be delivered by typical LES solvers. The limitations of the extinction submodel on the capability of modelling low oxygen compartment fires using CFD is well known [10, 51]. Some recent attempts have focused on with recent studies focusing on refining existing extinction submodels [52-58]. Of particular relevance are the studies by Vilfaeu et al. [59] and White et al. [60], which have shown that the treatment of re-ignition upon local extinction is particularly important to modelling global extinction. While these studies represent the state of the art of extinction modelling of fires in low oxygen environments, selection of an appropriate extinction and re-ignition model is largely dependent on the scenario. This study does not intend to evaluate the performance of existing extinction submodels. Instead, it is argued that the errors in modelling underventilated compartment fires arise not due to the performance of the extinction submodels, but rather, due to the uncertainty in the model inputs, which are defined by the environmental conditions specific to the compartment fire. A novel technique to evaluating the uncertainty in the model inputs is proposed within the context of FDS and the existing extinction model. To demonstrate this, an overview of the extinction model currently implemented in FDS is first provided.

8.1.1 Extinction hypothesis FDS uses a mixing-controlled reaction model with infinitely fast chemistry that assumes a reacting flow regardless of local temperature, reactant concentration, or strain rate. The physical mechanisms controlling local extinction often occurs at the sub-grid scale, such as the flame temperature and strain rates reaching critical extinction values. In the VLES model (default option) used, the flame sheet is not resolved, and thus mixing rates are based on semi- empirical models. Similarly, FDS models local extinction using a semi-empirical approach based on the concept of a critical flame temperature, . A mixture in a grid cell below this critical temperature will not react, and local extinction𝑇𝑇CFT will be assumed by the model. If combustion within a cell raises the temperature of the cell above the critical flame temperature, combustion occurs. Thus, the only mechanism of extinction is by thermal quenching, with other forms such as aerodynamic and kinetic quenching neglected.

355

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

In FDS, the critical flame temperature is a pre-defined value. In this study, the default value for propane in FDS is employed, with = 1447°C. It is worth noting that the assumption

of a constant may not necessarily𝑇𝑇 beCFT correct, and it has been shown by White et al. [60] and Maragkos𝑇𝑇 CFTand Merci [58]. These studies show that the model outputs are sensitive to the treatment of . th Prior to the 6𝑇𝑇CFT version of FDS, the extinction model was formulated based on tracking the fuel and oxygen concentrations with constant specific heat capacity. The resulting criterion is a linear function of the limiting oxygen index (LOI) and the local temperature. The limitations of this model are documented by Vaari et al. [52] and were addressed in a revised extinction model for FDS6 onwards [43]. The current formulation of the FDS extinction model used in LES computes temperature-dependent sensible enthalpies of two gases, the fuel ( ) and the

local gas mixture excluding fuel ( ): Δℎ𝑓𝑓 = ( Δ)ℎ𝑔𝑔 ( ); = ( ) ( ) (5) with Δℎ𝑓𝑓 ℎ𝑓𝑓 𝑇𝑇𝐶𝐶𝐶𝐶𝐶𝐶 − ℎ𝑓𝑓 𝑇𝑇 Δℎ𝑔𝑔 ℎ𝑔𝑔 𝑇𝑇𝐶𝐶𝐶𝐶𝐶𝐶 − ℎ𝑔𝑔 𝑇𝑇

= ; ( ) = ( ) (6) 𝑇𝑇 , ′ ′ g 𝛼𝛼 𝛼𝛼 𝛼𝛼 𝑝𝑝 𝛼𝛼 ℎ � 𝑌𝑌 ℎ ℎ 𝑇𝑇 �0 𝑐𝑐 𝑇𝑇 𝑑𝑑𝑇𝑇 where is the cell temperature,𝛼𝛼 is an empirically𝑇𝑇 defined critical flame temperature, is the chemical𝑇𝑇 and sensible enthalpy𝑇𝑇𝐶𝐶𝐶𝐶𝐶𝐶 for a species , and , is the specific heat capacity ofℎ 𝛼𝛼a species at a given temperature determined from a𝛼𝛼 lookup𝑐𝑐𝑝𝑝 table.𝛼𝛼 The prime subscript in the integral refers to a dummy variable. The fuel mixture ( ) and gas mixtures ( ) are defined as:

F g 1 Δ𝑌𝑌 = min , Δ𝑌𝑌 ; = (7) − 𝑌𝑌𝐹𝐹 F 𝐹𝐹 O2 g 𝐹𝐹 Δ𝑌𝑌 �𝑌𝑌 𝑌𝑌 ⁄𝑠𝑠� Δ𝑌𝑌 𝑠𝑠Δ𝑌𝑌 2 where is dependent on the stoichiometric ratio of each grid cell,𝑌𝑌𝑂𝑂 and is the fraction of fuel Δunder𝑌𝑌g stoichiometric conditions. is defined as the stoichiometric 𝑌𝑌ratioO2⁄ 𝑠𝑠as a function of the molar ratio of oxygen-fuel in stoichiometric𝑠𝑠 conditions and the molecular weights of oxygen ( ) and fuel ( ), with the oxygen-fuel molar ratio in stoichiometric conditions:

𝑂𝑂2 F 𝑂𝑂2 𝑀𝑀 𝑀𝑀 𝑣𝑣 = (8) 𝑣𝑣𝑂𝑂2𝑀𝑀𝑂𝑂2 𝑠𝑠 The formulation of Eq. (7) defines as either𝑀𝑀 theF fuel concentration in fuel-lean conditions or by the stoichiometric reaction forΔ the𝑌𝑌F full consumption of oxygen. If the combustion occurs in an air vitiated environment and the compartment is fuel-rich such that > / , the

𝑌𝑌F 𝑌𝑌𝑂𝑂2 𝑠𝑠 356

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

combustion is modelled as stoichiometric. Changes in the mass fraction of the fuel ( ) during

combustion are: Δ𝑌𝑌F = (9)

The local mass fraction of the gas mixtureΔ𝑌𝑌 Fcan be𝑌𝑌O 2reduced⁄𝑠𝑠 to: 1 = (10) − 𝑌𝑌𝑂𝑂2 The formulation of the fuel and gas mixturesΔ𝑌𝑌g from Eqs. (8-10) do not allow for the possibility 𝑠𝑠 of fuel-rich combustion. Local extinction occurs if the enthalpy changes from raising a given cell temperature to the critical flame temperature are greater than the ideal heat of combustion of the fuel. Thus, combustion is suppressed if the following inequality is satisfied: + > (11)

where is the net heat of combustionΔ𝑌𝑌𝐹𝐹Δℎ𝐹𝐹 ofΔ the𝑌𝑌𝑔𝑔Δ fuel.ℎ𝑔𝑔 Δ𝑌𝑌𝐹𝐹Δ𝐻𝐻𝐶𝐶 As all excessΔ𝐻𝐻𝐶𝐶 air is consumed in the combustion, it is not taken into account in the right-hand side of the inequality in Eq. (11) that defines the extinction criterion. In this case, excess fuel acts as a diluent and does not react. Substituting the redefined definitions for the mass fractions of the fuel and gas mixture from Eqs. (9-10) into Eq. (11) produces the following inequality 1 + > (12) 𝑌𝑌O2 − 𝑌𝑌𝑂𝑂2 𝑌𝑌O2 The critical assumption of Eq. (12)Δℎ is𝐹𝐹 that the fuelΔℎ undergoes𝑔𝑔 Δ𝐻𝐻 complete𝐶𝐶 combustion, therefore 𝑠𝑠 𝑠𝑠 𝑠𝑠 the heat of combustion is considered a net quantity. In reduced oxygen environments, this assumption does not hold a physical basis as the mixture of the combustion gases is fuel-rich and complete combustion cannot occur. Products of incomplete combustion are generated and reduce the heat of combustion of the fuel to an effective quantity that is controlled by the completeness of the combustion. For fuel-rich combustion, specifying a net heat of combustion for the extinction inequality in Eq. (12) does not carry physical meaning, and therefore extinction is invoked incorrectly. Once the extinction model is invoked locally in a cell, the reactions are suppressed, and the species mass fractions are returned to their original values at the start of the time step, i.e. resulting conditions are the same as the initial ones.

The consequence of Eq. (11) is that in fuel-rich conditions, extinction is invoked in a cell that otherwise should still be sustaining heavily incomplete combustion. By introducing local extinction into a cell, energy is not released, thereby modifying the temperature field. In ventilation-controlled fires where the flow fields are hydrostatic, modifying the thermal fields is particularly significant as flow exchange through the opening is determined by the buoyancy

357

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

of the hot gases inside the compartment. Local changes in the thermal field alter the buoyancy, which alters the modelling of the fluid mechanics and ultimately the entrainment of oxygen into the reaction zone of the compartment. This leads to an incorrect model formulation and significant errors for compartments with small openings. As extinction is incorrectly invoked, modelled temperatures are significantly lower than the experimental values as the input parameter is now incorrect by virtue of the size of the openings. Errors increase with higher HRR inputs as the mixture becomes increasingly fuel- rich mixture. If the flow fields are inertia-driven due to larger openings, the effects of local extinction are dampened as the buoyancy forces are not dominant and the model matches the experiments with smaller errors.

8.1.2 Model diagnostic To demonstrate the role of the extinction submodel in highly ventilated-controlled compartment configurations, a comparison is made to a case where extinction is disabled (Figure A2-18). The comparison of the HRRPUV shows various time steps comparing where the reaction zone occurs. When the extinction model is disabled, it can initially be observed that fuel is concentrated at the location of the burners and is mixed with the oxygen within the enclosure. In contrast, when the extinction model is activated, the reaction zone is entirely confined to the area of the opening. Once complete oxygen depletion is attained within the compartment, the reaction zone is confined to the opening also, albeit with more heat released inside of the compartment. Variable Extinction model Extinction model = OFF

HRRPUV propane (kW/m3)

358

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Variable Extinction model Extinction model = OFF

Extinction criteria

Fuel mass fraction (kg/kg) - White corresponds

to YF = 0.4

Oxygen mass fraction (kg/kg)

Figure A2-18. Model diagnostic through comparison of selected quantities with and without the extinction model activated. All slices are taken in the 100s-150s range The second comparison shows the progression of how the extinction criterion is satisfied spatially inside the compartment. The model without extinction is shown for reference. Cells coloured white (1) correspond to cells where the extinction criterion has been met and combustion is suppressed. Cells coloured grey (0) correspond to cells where the extinction criterion is not met, and cells coloured black (-1) correspond to cells where there is no fuel or oxidiser present for combustion. The extinction criterion is met at approximately 15s, with regions around the burners and walls having combustion suppressed. At approximately 30s, extinction occurs in most of the compartment except at the opening. As combustion is suppressed in a very short timescale, the total heat released inside the compartment is small, therefore gas-phase temperatures are lower (refer Annex A temperatures). In the case where

359

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

the extinction model is disabled, complete oxygen depletion occurs in the compartment (Annex A). Oxygen depletion corresponds to the combustion of the reactants inside the control volume, thus gas-phase temperatures are higher as the heat losses are regulated by the hydrostatic flows through the opening. It is important to highlight that the combustion model in FDS is mixing- controlled and follows the Eddy Dissipation Concept (EDC) where the reaction rate source term is presented in Eq. (12) with the turbulent mixing timescale ( , / ) 𝜏𝜏=𝑚𝑚𝑚𝑚𝑚𝑚 (12) ′′′ 𝑚𝑚𝑚𝑚𝑚𝑚 𝑌𝑌𝐹𝐹 𝑌𝑌𝐴𝐴 𝑠𝑠 The result of this formulation of𝑚𝑚̇ the combustion−𝜌𝜌 𝜏𝜏𝑚𝑚𝑚𝑚𝑚𝑚 model is that the reaction zone corresponds to the mixing zone. Thus, combustion is localised for the most part to the opening. Comparing the oxygen mass fractions and temperatures in Figure A2-18 confirms the hypothesis that oxygen depletion does not occur for the default model due to the limiting factor of the extinction model. Careful attention should be focused on the comparison of the mass fractions of fuel. When the extinction model is activated, fuel mass concentrations are large close to the burners. Local extinction occurs as excess fuel dilutes the oxygen within the burner location.

8.1.3 Emulating incomplete combustion Extinction is invoked incorrectly for small opening sizes as a result of an incorrect definition of the heat of combustion of the fuel ( ) as the model input and the change in the mass -0.5 fraction of the fuel ( ). To test this hypothesis,Δ𝐻𝐻𝐶𝐶 models are run for = 70.1 m at the large -1 fire (1.5 g.s ) withΔ the𝑌𝑌𝐹𝐹 input specification for the fuel heat of combustion𝜙𝜙′ systematically reduced. Fractions of the net heat of combustion are specified at 0.8 , 0.6 , 0.4 , and

0.2 . The objective of these simulations is to artificially emulateΔ𝐻𝐻 incomplete𝐶𝐶 Δ𝐻𝐻𝐶𝐶 combustionΔ𝐻𝐻𝐶𝐶 throughΔ𝐻𝐻𝐶𝐶 a reduced heat of combustion. Figure A2-19 shows the temperature results of the plot of the simulation against the experimental data. Comparisons to the net heat of combustion ( ) simulation are included for reference.

Δ𝐻𝐻𝐶𝐶

360

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Figure A2-19. Comparisons of the modelled and experimental thermocouple measurements to artificially emulate incomplete combustion by reducing the heat of combustion of the fuel. Each plot shows a different specified heat of combustion compared to the same experimental data. Input values are percentages of the net heat of combustion The results of these simulations show that the best agreement to the experimental data occurs between 0.6 and 0.8 suggesting that the combustion is incomplete and the experimental Δheat𝐻𝐻𝐶𝐶 of combustionΔ𝐻𝐻𝐶𝐶 reduces below the net heat of combustion to an effective quantity. By reducing the heat of combustion to between 0.6 and 0.8 , the temperatures within the compartment are raised by approximately 100°CΔ𝐻𝐻 𝐶𝐶– 200°C.Δ 𝐻𝐻While𝐶𝐶 it is counterintuitive to expect that reducing the heat of combustion to result in larger compartment temperatures, this simple technique demonstrates that by lowering the enthalpy requirement for ignition (Eq. (12)), burning within the compartment can be sustained leading to a thermal environment more representative of the experimental data. It is important to note that the effective heat of combustion cannot be explicitly quantified using this approach as cannot be defined as a fuel-rich mixture. Instead, is assumed by the model as the stoichiometricΔ𝑌𝑌𝐹𝐹 quantity on the basis that all the air is consumedΔ𝑌𝑌𝐹𝐹 and excess fuel leftover acts as a diluent. Uncertainty in the model inputs is not solely limited to the efficiency of combustion. The incompleteness of the combustion and environmental conditions within the compartment are directly linked with the generation of the products of combustion (soot and CO in particular) and the radiative fraction. Both of these inputs significantly influence the radiative heat transfer, which is the dominant mechanism in ventilation-controlled

361

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires compartments [61]. The effect of uncertainty in these inputs is a lack of accuracy on the simulated gas temperatures and IRHF. While qualitative, the results highlight that a great deal of the uncertainty in modelling underventilated fires is associated to the ill-defined nature of the model inputs obtained as a function of specific environmental conditions within the compartment fire. Thus, under certain configurations, a source of model uncertainty arises within the inputs by virtue of the opening factor of the compartment. Under these conditions, the model uncertainty is systemic and can only be addressed by means of using experimental measurements of the same compartment to formulate the correct inputs or to validate a particular submodel for a specific context. Consequently, apriori modelling of this scenario remains outside the purview of ventilation- controlled compartment fires.

8.2 Uncertainty arising from fire source modelling Above the regime breakpoint, where the compartment is no longer ventilation-controlled and hence the model is able to replicate the experiments to varying extents, the simplifications made to the boundary condition of the fire are shown to have varying effects on the model output. All simulation devices are susceptible to the error introduced by the simplified burners, as this can result in different flame volume and height, as well as different entrainment behaviour. As recognized in section 4, the 128 burners with a diameter of 2 mm are replaced by 16 burners, with a total area of 64 cm2. The area of the original 128 burners has a combined area of about 4 cm2, which implies a difference in areas by a factor of 16. This has a direct effect on the flame velocity and therefore on its behaviour. Although section 4 also showed that this simplification does not yield jet flames, the differences in flame behaviour will introduce uncertainty. The change from a burner with small orifices for a set of square burners can result in different flame volume and height, as well as a different entrainment behaviour. These, in turn, can translate to inaccurate gas temperatures and IRHF at the walls, as their correlation with the non-dimensional heat release rate changes [62]. Figure A2-20 compares the flames recorded for the experiment in the 40% configuration, with those from the FDS model, using the simplified square burners. Although qualitative, this comparison indicates the potential for flame shape differences.

362

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Figure A2-20. Comparison of flame shape and volume for the configuration 40%, small fire for the experiment (left) and the FDS simulation (right) The sensitivity of the model to the use of a line burner (as in the experiment) was assessed by modelling the 100% configuration for the large fire size using four-line burners of 54 by 2 cm in area and adjusting the HRRPUA (Figure A2-21). Only this scenario was selected as the intention was to purely assess the effect of modifying the boundary condition of the fire on the model outputs with minimal influence from the compartment geometry. The results indicate that the line burner simulation yields a decrease in the bias factors of -6% and -5% for gas temperatures and IRHF. This supports the hypothesis that the line burners better represent the flames and therefore the radiative heat transfer, as the new bias factor indicates an overestimation of the IRHF of only 1.2%. The bias factor for the temperature in the line burner exhibits the opposite behaviour, with a resulting bias factor indicating an under prediction of almost 25% (the underprediction for the square burners was of 18%). The results indicate that line burner configuration is able to better model the entrainment of the flow, and thus output more accurate hot thermocouple temperatures.

Figure A2-21. Line burner configuration in FDS Each modelling decision represents a trade-off, such as more resolution (finer mesh) requires additional computing resources or time, less accurate geometrical representations introduces significant biases for the quantities of interest. However, in the spectrum of fire regimes within the compartment, these decisions become less relevant towards ventilation-controlled fires as long as the total heat input is comparable. On the other hand, fuel-controlled fires with large

363

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

openings require a detailed and accurate representation of the fire source. In this particular case, the effect of the burner system simplification results in a significant underprediction of gas temperatures and over prediction of the IRHF. Although this uncertainty indicates shortcomings of the modelling approach, it nevertheless allows characterizing the effect of the inputs on the resulting uncertainty, which is the objective of this research.

9 Conclusions

This research used robust experimental data and matching fire modelling outputs to characterize the effect of different uncertainty sources on the capability to model fully- developed fire regimes. The compartment configurations covered five ventilation settings (from 20% to 100% opening area) as well as three fire sizes (equivalent in full scale to 325, 650 and 1000 kW) and investigated the gas temperatures, IRHF to solid internal boundaries and the flow behaviour at the opening. The study used qualitative and quantitative analyses to provide a basis to characterize uncertainty. Both analyses allowed identifying areas where the effect of uncertainty limits the capability of modelling experimental behaviour. The result indicates that there are two mechanisms resulting in uncertainty within the context of fully developed compartment fires. These mechanisms bifurcate at the transition from ventilation- to fuel-controlled fires; this transition is controlled by the opening factor and the HRR and lays between the 40% and 60% configurations as shown by Majdalani et al. [34] and verified through the simulation outputs (section 6). On one of the ends of the fire regimes spectrum, the compartment is ventilation-controlled with hydrostatic flows and buoyancy dominating the thermal fields. Here, uncertainty is mainly driven by the inability to specify correct input parameters under these conditions (e.g. heat of combustion). The input parameters are determined as a function of the environmental conditions associated with the nature of the compartment fire, and thus cannot be specified a priori. While improvements in the extinction/re-ignition sub-models may offer improvements, any model will still require correctly defining the input parameters. Such limitation can only be resolved by modelling the detailed chemistry of the gas-phase to obtain the correct inputs and then be used in the CFD modelling. This approach relies on the resolution of very small length-scales (i.e. less than 10-1 m) that are incompatible with conventional fire problems (i.e. between 10-1 m to 102 m). On the other end of the fire regimes spectrum, the analysis of the momentum-driven regime at larger opening sizes (> 60%) shows that the model is generally able to resolve the

364

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

characteristics of the thermal and flow environments of the compartment quite well. However, uncertainty is driven by the decisions of the user in the model formulation and appropriation of the boundary conditions. In the compartment analysed in this study, the simplifications of the fire source have a direct effect on the resulting uncertainty for the configurations of 80% and 100%. This limitation can be managed by refining the mesh and setting up a fire source of higher fidelity, trading off computational time; this was not implemented in this study as accurately representing the fire source would have required implementing a DNS simulation, which is seldom an option in fire safety engineering applications; the results of the study are synthesized in Figure A2-22.

Figure A2-22. Graphic representation of the results It is of relevance that the results of the temperatures spatial distribution obtained from models like FDS are inputs for structural engineers to determine the response of the structural system [63, 64] and verify their reliability [65]. However, the results and observations of this work indicate that the accuracy of spatial temperature distributions is questionable under specific conditions such as ventilation-controlled fires, but also for the other end of the range (momentum-controlled fires, typically associated with opening plan compartments). When models such as FDS are used to estimate the thermal-boundary conditions, it becomes necessary to explicitly consider the uncertainty sources discussed in this study (this is particularly relevant for systems for which matching validation cases are not available). Based on the results of this study, the data from the centreline of the compartment could be more reliable when taken as thermal boundary conditions for the whole compartment, despite yielding a more conservative solution. Another alternative to mitigate uncertainty for

365

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

ventilation-controlled fires is using well-validated tools such as the compartment fire framework [66, 67]. The current spotlight on fire safety engineering shortcomings in the UK and Australia [68, 69] has made it a priority to generate awareness of the technical limitations, as the lack of awareness can lead to inappropriate use of models. This is particularly relevant for designers and engineers that model fires to ‘predict’ and impose spatial distributions of the thermal-boundary conditions. The uncertainty characterization in this study should aid practitioners when analysing the potential limitations of their results when modelling fully-developed fires and, therefore, enhancing fire safety engineering practice.

References

[1] Torero, J.L., Scaling-Up fire. Proceedings of the Combustion Institute, 2013. 34(1): p. 99-124. [2] K. McGrattan, B.K., S. Hostikka, J. Floyd, Fire dynamics simulator (version6), Technical Reference Guide, N.I.o.S.a.T.R. (NIST), Editor. 2012. [3] Beard, A.N., Requirements for acceptable model use. Fire Safety Journal, 2005. 40(5): p. 477-484. [4] Jutras, I. and B.J. Meacham, Development of objective-criteria-scenario triplets and design fires for performance-based Fire Safety Design. Journal of Building Engineering, 2016. 8: p. 269-284. [5] Olenick, S.M. and D.J. Carpenter, An Updated International Survey of Computer Models for Fire and Smoke. Journal of Fire Protection Engineering, 2003. 13(2): p. 87- 110. [6] Rein, G., et al., Round-robin study of a priori modelling predictions of the Dalmarnock Fire Test One. Fire Safety Journal, 2009. 44(4): p. 590-602. [7] Rengel, B., et al., A priori validation of CFD modelling of hydrocarbon pool fires. Journal of Loss Prevention in the Process Industries, 2018. 56: p. 18-31. [8] Merci, B., J.L. Torero, and A. Trouvé, Call for participation in the first workshop organized by the IAFSS Working Group on Measurement and Computation of Fire Phenomena. Fire Safety Journal, 2016. 82: p. 146-147. [9] McGrattan, K., Fire Modeling: Where Are We? Where Are We Going? . Fire Safety Science, 2005(8): p. 53-68.

366

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

[10] Kevin McGrattan, S.H., Randall McDermott, Jason Floyd, Craig Weinschenk, Kristopher Overholt, Fire Dynamics Simulator Technical Reference Guide, Volume 3: Validation, N.I.o.S.a. Technology, Editor. 2013. [11] Trévisan, N., et al., Experimental and numerical study of interactions between sprinklers and natural smoke vents. Fire and Materials, 2018. 42(3): p. 247-254. [12] Drean, V., et al., Numerical Simulation of Fire Exposed Façades Using LEPIR II Testing Facility. Fire Technology, 2018. [13] Dréan, V., et al., Numerical simulation of the fire behaviour of facade equipped with aluminium composite material-based claddings-Model validation at large scale. Fire and Materials, 2019. 43: p. 981-1002. [14] Nilsson, M., et al., A numerical comparison of protective measures against external fire spread. Fire and Materials, 2018. 42(5): p. 493-507. [15] Lu, L., et al., Large-scale test as the basis of investigating the fire-resistance of underground RC substructures. Engineering Structures, 2019. 178: p. 12-23. [16] Zhao, G., T. Beji, and B. Merci, Study of FDS simulations of buoyant fire-induced smoke movement in a high-rise building stairwell. Fire Safety Journal, 2017. [17] Zhao, G., T. Beji, and B. Merci, Application of FDS to Under-Ventilated Enclosure Fires with External Flaming. Fire Technology, 2016. 52(6): p. 2117-2142. [18] Li, L.J., et al., Experimental investigation on the characteristics of buoyant plume movement in a stairwell with multiple openings. Energy and Buildings, 2014. 68: p. 108-120. [19] Wahlqvist, J. and P. van Hees, Validation of FDS for large-scale well-confined mechanically ventilated fire scenarios with emphasis on predicting ventilation system behavior. Fire Safety Journal, 2013. 62: p. 102-114. [20] U.S. Nuclear Regulatory Commission, O.o.N.R.R.R., Electric Power Research Institute (EPRI), Verification and Validation of Selected Fire Models for Nuclear Power Plant Applications, Supplement 1. 2016: Rockville, MD, Palo Alto, CA. [21] U.S. Nuclear Regulatory Commission, O.o.N.R.R.R., Electric Power Research Institute (EPRI), Verification and Validation of Selected Fire Models for Nuclear Power Plant Applications, Volume 7: Fire Dynamics Simulator (FDS). 2007: Rockville, MD, Palo Alto, CA. [22] Brown, A., et al., Proceedings of the first workshop organized by the IAFSS Working Group on Measurement and Computation of Fire Phenomena (MaCFP). Fire safety journal, 2018. 101: p. 1-17.

367

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

[23] Cowlard, A., et al., Fire Safety Design for Tall Buildings. Procedia Engineering, 2013. 62: p. 169-181. [24] Joseph Su, P.-S.L., Matthew Hoehler, Matthew Bundy, Fire Safety Challenges of Tall Wood Buildings – Phase 2: Task 2 & 3 – Cross Laminated Timber Compartment Fire Tests. 2018, National Research Council of Canada: Ottawa, Ontario, Canada. [25] Hidalgo, J.P., et al., The Malveira fire test: Full-scale demonstration of fire modes in open-plan compartments. Fire Safety Journal, 2019. 108. [26] Welch, S., et al., BRE large compartment fire tests—Characterising post-flashover fires for model validation. Fire Safety Journal, 2007. 42(8): p. 548-567. [27] V. Gupta, A.F.O., J. L. Torero, J. P. Hidalgo, Mechanisms of flame spread and burnout in large enclosure fires. Proceedings of the Combustion Institute, 2020. [28] Li, J., et al., CFD study of fire-induced pressure variation in a mechanically-ventilated air-tight compartment. Fire Safety Journal, 2020: p. 103012. [29] Hadjisophocleous, G. and Q. Jia, Comparison of FDS Prediction of Smoke Movement in a 10-Storey Building with Experimental Data. Fire Technology, 2009. 45(2): p. 163- 177. [30] Disaster, F.B.a.F.S.I.o.t.W.T.C., Final Report on the Collapse of the World Trade Center Towers, N.N. 1, Editor. 2005, National Institute of Standards and Technology (NIST): Gaithersburg, MD, USA. [31] Meacham, B., Post-Earthquake Fire Performance of Buildings: Summary of a Large- Scale Experiment and Conceptual Framework for Integrated Performance-Based Seismic and Fire Design. Fire Technology, 2016. 52(4): p. 1133-1157. [32] Hoehler, M.S., et al., Behavior of steel-sheathed shear walls subjected to seismic and fire loads. Fire Safety Journal, 2017. 91: p. 524-531. [33] Jowsey, A., Fire imposed heat fluxes for structural analysis. 2006. [34] Majdalani, A.H., et al., Experimental characterisation of two fully-developed enclosure fire regimes. Fire Safety Journal, 2016. 79: p. 10-19. [35] Thomas, P.H., et al., Fully-developed Compartment Fires: Two Kinds of Behaviour. 1967: H.M. Stationery Office. [36] V. Gupta, J.P.H., A.Cowlard, C. Abecassis-Empis, A.H. Majdalani, C. Maluk, J.L. Torero, Ventilation effects on the thermal characteristics of fire spread modes in open- plan compartment fires. Fire Safety Journal, 2020. in Press. [37] Majdalani, A., Compartment Fire Analysis for Contemporary Architecture. 2014, The University of Edinburgh.

368

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

[38] Quintiere, J.G., Scaling and Dimensionless Groups, in Fundamentals of Fire Phenomena. 2006, John Wiley & Sons, Ltd. p. 377-408. [39] Drysdale, D., An introduction to fire dynamics. 1999: Wiley. [40] Hidalgo, J.P., et al., A Thin Skin Calorimeter (TSC) for quantifying irradiation during large-scale fire testing. International Journal of Thermal Sciences, 2017. 112(Supplement C): p. 383-394. [41] McCaffrey, B.J. and G. Heskestad, A robust bidirectional low-velocity probe for flame and fire application. Combustion and Flame, 1976. 26: p. 125-127. [42] A. P. Robbins, C.A.W., Soot yield values for modelling purposes - Residential occupancies. 2007, BRANZ Ltd: Porirua, New Zealand. [43] K. McGrattan, B.K., S. Hostikka, J. Floyd, Fire dynamics simulator (version6), Users' Guide, N.I.o.S.a.T.R. (NIST), Editor. 2012. [44] Welch, S. and P. Rubini, Three-dimensional simulation of a fire-resistance furnace. Fire Safety Science, 1997. 5: p. 1009-1020. [45] McGrattan, K. and B. Toman, Quantifying the predictive uncertainty of complex numerical models. Metrologia, 2011. 48(3): p. 173-180. [46] Cicione, A., et al., Full-Scale Informal Settlement Dwelling Fire Experiments and Development of Numerical Models. Fire Technology, 2019. [47] V. Gupta, J.C., J. P. Hidalgo, A. Osorio, C. Maluk, A. H. Majdalani, Accounting for Uncertainty in Computational Fluid Dynamics Modelling of Fully-Developed Compartment Fires, in 12th International Performance-Based Codes and Fire Safety Design Methods Conference (SFPE), S.o.F.P.E. (SFPE), Editor. 2018: Honolulu, Hawaii. [48] Gupta, V., et al. Analysis of Convective Heat Losses in a Full-scale Compartment Fire Experiment. in Proceedings of the Ninth International Seminar on Fire and Explosion Hazards (ISFEH9). 2019. St. Petersburg, Russia: St. Petersburg Polytechnic University Press. [49] Smardz, P., Validation of Fire Dynamics Simulator (FDS) for forced and natural convection flows. 2006, University of Ulster: Londonderry, Northern Ireland. p. 110. [50] Hu, L., et al., Flame radiation fraction behaviors of sooty buoyant turbulent jet diffusion flames in reduced- and normal atmospheric pressures and a global correlation with Reynolds number. Fuel, 2014. 116: p. 781-786. [51] Floyd, J. and K. McGrattan, Extending the mixture fraction concept to address underventilated fires. Fire Safety Journal, 2009. 44(3): p. 291-300.

369

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

[52] Jukka Vaari, J.F., Randall McDermott, CFD Simulations on Extinction of Co-Flow Diffusion Flames. Fire Safety Science, 2011: p. 781-794. [53] Hu, Z., et al., Towards large eddy simulations of flame extinction and carbon monoxide emission in compartment fires. Proceedings of the Combustion Institute, 2007. 31(2): p. 2537-2545. [54] Ren, N., et al., Modeling of flame extinction/re-ignition in oxygen-reduced environments. Proceedings of the Combustion Institute, 2019. 37(3): p. 3951-3958. [55] Snegirev, A.Y. and A. Tsoy, Treatment of local extinction in CFD fire modeling. Proceedings of the Combustion Institute, 2015. 35(3): p. 2519-2526. [56] Snegirev, A.Y., Perfectly stirred reactor model to evaluate extinction of diffusion flame. Combustion and Flame, 2015. 162(10): p. 3622-3631. [57] Vilfayeau, S., et al., Numerical simulation of under-ventilated liquid-fueled compartment fires with flame extinction and thermally-driven fuel evaporation. Proceedings of the Combustion Institute, 2015. 35(3): p. 2563-2571. [58] Maragkos, G. and B. Merci, Large eddy simulations of flame extinction in a turbulent line burner. Fire Safety Journal, 2019. 105: p. 216-226. [59] Vilfayeau, S., et al., Large eddy simulation of flame extinction in a turbulent line fire exposed to air-nitrogen co-flow. Fire Safety Journal, 2016. 86: p. 16-31. [60] White, J., et al., Modeling flame extinction and re-ignition in large eddy simulations with fast chemistry. Fire safety journal, 2017. 90: p. 72-85. [61] J.L. Torero, A.H.M., C. Abecassis-Empis, A. Cowlard, Revisiting the Compartment Fire. Fire Safety Science, 2014(11): p. 28-45. [62] O. Sugawa, H.S., Y. Oka. Flame Height From Rectangular Fire Sources Considering Mixing Factor. in Proceedings of the Third International Symposium on Fire. [63] Du, Y. and G.-q. Li, A new temperature–time curve for fire-resistance analysis of structures. Fire Safety Journal, 2012. 54: p. 113-120. [64] Zhang, C., et al., Simulation Methodology for Coupled Fire-Structure Analysis: Modeling Localized Fire Tests on a Steel Column. Fire Technology, 2016. 52(1): p. 239-262. [65] Van Coile, R., et al., The meaning of Beta: background and applicability of the target reliability index for normal conditions to structural fire engineering. Procedia Engineering, 2017. 210: p. 528-536. [66] Harmathy, T.Z., A new look at compartment fires, part I. Fire Technology, 1972. 8(3): p. 196-217.

370

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

[67] Harmathy, T.Z., A new look at compartment fires, part II. Fire Technology, 1972. 8(4): p. 326-351. [68] Hackitt, J., Building a Safer Future, Independent Review of Building Regulations and Fire Safety: Final Report. 2018: London, UK. [69] Peter Shergold, B.W., Building Confidence – Improving the effectiveness of compliance and enforcement systems for the building and construction industry across Australia. 2018, Building Ministers’ Forum (BMF).

371

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Annex 1 – Mesh sensitivity

The following are sample time-temperature comparisons between the experimental data and the FDS output. The FDS output is further divided into the results of the 2 cm mesh size (selected for the whole study) and those of the 1 cm mesh size. These results support the use of the coarser mesh size, as well as representing a quasi-steady state behaviour that supports the averaging of results in the 300-700 seconds timeframe.

Figure A2-23. Time-temperature profiles for the experimental data and for the FDS output with 1 cm and 2 cm mesh size at 0.23 m from the floor

372

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Figure A2-24. Time-temperature profiles for the experimental data and for the FDS output with 1 cm and 2 cm mesh size at 0.49 m from the floor

373

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Figure A2-25. Time-temperature profiles for the experimental data and for the FDS output with 1 cm and 2 cm mesh size at 0.75 m from the floor

Annex 2 – Data processing

The following describes the procedure used to process both the experimental data and the model outputs and how the datasets were arranged to enable point-to-point comparisons. 1. Comparison of timescales: the experimental data was recorded spanning different timeframes, usually of around three thousand (3000) seconds, while FDS simulations had a consistent timeframe of seven hundred (700) seconds. 2. Timeframe match: using the data processing script in MATLAB, the experimental data was trimmed to remove data before ignition and after the end of the FDS simulation.

374

Appendix 2. Identifying uncertainties for field modelling of fully-developed compartment fires

Experimental data was also trimmed after 700 seconds, which is the time at which the FDS simulation stops. 3. Timeframe for comparison: with the matching data sets, these were plotted to check for the timeframes in which the analysis should take place. As shown in Figure A2-26, there is an initial period of time in which the fire grows until the temperature peaks. The temperature then follows a slow decay before it stabilizes and slowly starts increasing again. Considering that the initial transient period is not representative of the fully developed fire, this is discarded. The timeframe for comparison is then limited to the 300-700 seconds (400 seconds in total), which is taken as a stable period for the experimental runs. This timeframe of stable measurements represents a key component of the fully developed stage of a fire (temperature distributions and gas flows), which was the scope of the experimental research and is relevant to determine the thermal load for structural analysis. This range is deemed acceptable as the timescales associated to reaching a thermal equilibrium between the gas and solid is relatively short due to the low thermal inertia of the compartment boundaries. 4. Matching data points: the ratio between the number of experimental data and FDS outputs is about 1:5 due to FDS’ adaptive time-step. In order to have matching sets of data points, the experimental data is interpolated linearly.

Figure A2-26. Sample experimental data for A1, B2 and C3 thermocouple trees at the middle height of the compartment. Error bars are omitted for clarity

375

Appendix 3

Comparative evaluation of Guidance for fire risk assessments - Full results

376

Appendix 3. Comparative evaluation of guidance for fire risk assessment – Full results

Evaluation Specific criteria NFPA 551 Comment SFPE Guide Comment PD 7974-7 (2019) Comment ISO 16732-1 (2012) Comment TR 951 (2019) Comment MAD Comment criteria Section 4.1.1 indicates the use of a PRA to explicitly calculate risk and demonstrate acceptability, enabling "a better In section 4.1.1 the guidance informed options appraisal". document puts forward that the Section 1 provides the This does not constrain the Section 4 and 4.1 describe role of a fire risk assessment is to Chapter 3 defines fire risk assessments as possibility of using PRA as a tool to guide design risk assessment as a tool to focus “attention on what is the technical basis for the decisions comparative analysis in order Section 6.2.2 defines the Role of risk decisions, but also offers the achieve safety in cases where important to fire safety” and as regarding fire risk management, to verify the fire safety scope of the methodology assessment: The possibility to use it as a low frequency- high one of the elements considered in including the safety measures affecting performance. Based on section and states that it is meant to guidance document verification tool. This is consequence scenarios are risk-informed decision-making. likelihood and consequences. 4.1, a fire risk assessment is 1.1 clearly states the role MC MC PC confirmed by section 4.1.3.1. PC present. Section 5 proposes DC MC provide a risk quantification used to verify the level of of the methodology in risk assessment can help that mediates between the Furthermore section 4.4.1.1 states Section 4.2 defines the step in which the safety of a design, which based supporting a specific Section 5.2.7 states that prioritizing design design goals and the fire the need for the specific purpose risk assessment goals are defined in on section 1 means to "use fire design philosophy inherently safe designs should alternatives. No clear role of safety objectives. of the fire risk assessment to be conjunction with the stakeholders and safety engineering methods in be prioritized, but in no way it the risk assessment is defined in each project (identify provides a list of typical ones. order to evaluate compliance is stated that the PRA process defined. risk level, methods for risk with an absolute criterion". can guide the construction of treatment, etc.). such design. There is a clear break between trial designs definition and the actual safety verification process. Section 3.5.4 and 6.1.1 define hazards Figure 1 excludes the Although section 4.4. seems to identification as the first step in scenario HAZID step from the fire Typical hazards identification refer to the process of hazards development. risk management process, techniques such as what-if identification as part of the while Figure 2 limits it to the analysis are treated by the initial design review step, Section 6.4 sets out the process, which identification of scenarios guidance (section 5.2.1) as particularly those not covered begins with past experience and (section 6.2.2). This section qualitative risk assessment by pre-accepted solutions (i.e. acknowledges all possible hazards will references the ISO 16733- methods. This is misleading as it deemed-to-satisfy), there is no not be captured. Future hazards are 1:2015, section 6.2, which in treats hazid techniques and Neither figure 1, 2 nor 3 consideration of the process by addressed (6.4.4). turn offer a) prescriptive quantitative methods as mutually indicate hazards identification which this is done. Section 6.2.4 introduces the scenarios, b) qualitative or Hazards identification: exclusive options, which could as a step in the process for use of scenarios in the Section 6.8 presents structured semi-quantitative approaches The role of hazards not be farther from reality. In demonstrating adequate safety Section 6.2 and 6.3 address the methodology and addresses techniques for hazard identification such to identify credible scenarios identification is reality, the qualitative (hazid) of a design or in the PRA construction of fire scenarios, 1.2 DC MC as HAZOP, FMEA, Event trees, failure DC PC (w/out guidance) or c) DC PC the need for a systematic defined and structured feeds onto the quantitative. process. In figure 3 the enquiry stating that their potential trees and what ifs. However, fails to selection of a comprehensive identification of hazards, but Design in the guidance "are all possible consequences number is infinite. Reference 1 describe how these can be applicable to set of scenarios using event no guidance is provided to do philosophy document Although the what if technique is tolerable" seems to imply a full is made to ISE/TS 16733-1 the FSE space (not self-evident). trees (give data existence). so. references, it is treated as a risk scenario discovery, though this and ISO 16732-1 to construct Reference is made to NFPA 550 for None of the previous assessment methodology. is not specified. fire scenarios and to use the guidance on fire safety concepts tree. constitutes a structure Furthermore, recognition of concept of scenario clusters. process to hazards hazards as the basis of the risk Section 6.3 addresses specific Section 7.3 provides a cross reference identification. Steps 2-4 assessments is provided in section fire dynamics components that between the outputs of the hazard touches on hazards 7.2.2.5, but no formal hazards can be associated to hazards, identification process and the questions identification in an identification process is but no mention of a specific required to formulate a scenario. This is a unstructured manner and described. hazard identification scheme is useful resource to link hazid and design relying on frequency data to provided. decisions to the scenario construction. classify types of fires. Section 4.1.2 treats deterministic analyses and their Section 4.1 implies in its title Both sections 6.2.2 and 6.2.4 related scenarios as that there are circumstances put forward the possibility of conservative and based on in which a probabilistic using the methodology to experience. Figure 1 presents a assessment can be more Section 4.1 mentions the role Section 7.1.1 addresses the need for an general framework for produce insight on inherent The guidance document presents advantageous than a of comparative analysis initial scenario basis which is then demonstrating adequate safety risk, which can then be used five types of risk assessments, deterministic one. Neither of (mainly through deterministic included in the quantification process. either through deterministic or as an input for the selection Gradual approach to with the first one oriented them are defined, neither the analysis) to verify compliance The fire risk assessment flow chart of probabilistic analyses, but of safety measures through risk assessment: towards hazards identification. so-called advantages. with safety. However, this is Figure 3.1 explicitly addresses the presents no direct flow from the use of risk assessments Guidance is provided However, these types are treated portrayed as a complementary possibility of refining the analysis in lieu one to the other. Although not that focus on variability such to implement a gradual in an independent manner, with ISO 23932-1:2018 makes approach and not a gradual of reducing risk, which is then discussed disconnected, the link between as PRAs. It is clearly stated transition towards the redaction implying mutual statements such as 'the one. 1.3 DC PC in sections 14.3-4. PC them is not clear. In a design DC DC MC quantitative exclusion. Table 5.1.2.1 provides qualitative analysis belongs in section 6.2.4 that the process (such as described in assessments, including an overview of the five types of to the comparative approach' Identification, prioritization methodology is to constitute Section 14.4.1 implies that a potential Chapter 4), there is a need to qualitative screening assessment, which is valuable to and others that have no and gradual management of the basis for designing out solution to risks that are not clearly gradually increase the level of of scenarios and use of the practitioner to understand scientific base and seem to scenarios is not addressed at hazards and risks in the acceptable or unacceptable is to increase sophistication, only to better deterministic analysis their differences, however no reflect the authors' own any point as the risk earliest design stages, while the sophistication of the selected method, understand increasingly connection is provided between taxonomy. Although this assessment is presented as a setting up the foundation for although it is not self-evident how this important and complex them. document provides a wider step in time, not an iterative will improve the result. scenarios. Such connection is further analyses in which perspective on risk analysis activity. not presented here, although variability is explored and approaches, there is no sense the starting point of the fire cost-benefit alternatives can of gradualism or scenario safety design is a Qualitative be ranked. classification. Design Review, as defined in the parent document. Both section 6.2.4 and 6.2.6 identify the responsibility of the 'building manager' towards implementing a fire No step of the guidance risk management action plan Stakeholder considers the stakeholders, resulting from the risk identification: Section 3.3. specifies the need for except for a reference in assessment and taking into Guidance is provided Section 4.2 provides guidance on stakeholders to be involved in specifying section 8 (Documentation) and account its assumptions. for a structured identifying the stakeholders, and Risk a fire risk assessment, while also No reference is found in the No reference is found in the in the introduction. Although 2 2.1 process to identify PC provides a list of typical MC DC DC DC PC ownership providing a list of typical stakeholders guidance document. guidance document. the introduction mentions that Section 6.2.1 identifies the stakeholders, their stakeholders. Section 4.3 and how their interests influence their selecting voluntary roles and describes in particular the AHJ. fire safety practitioners as the decision-making. performance criteria involves responsibilities users of the methodology, the stakeholders, the process is towards fire safety while section 6.1.4 discusses not specified. the responsibility these have with the arguments they make to support the results of a risk assessment.

377

Appendix 3. Comparative evaluation of guidance for fire risk assessment – Full results

Evaluation Specific criteria NFPA 551 Comment SFPE Guide Comment PD 7974-7 (2019) Comment ISO 16732-1 (2012) Comment TR 951 (2019) Comment MAD Comment criteria The role of other relevant stakeholders is not identified in the methodology.

No specific mention of the risk owner is made, despite the building owner is referenced. In particular section 4.4.1.1 considers that objectives are defined based on the owner's fire safety expectation over specific time periods, however does not No mention of the role of A single mention of the address the issue of risk risk owners is made in the building owner is identified in ownership continuity and how Section 3.4.3 defines the owner as the No reference is found in the document. It is worth noting section 8 (Documentation) this affects this step. stakeholder concerned with the full range guidance document. Annex C that ISO 23932-1:2019 calling for the need to clarify of risk issues. No responsibilities are Risk owner: Guidance describes a proposed hierarchy mentions affected parties or the distinction between Section 5.6.10.2 addresses the associated to the owner, nor to their role is provided to identify for acceptability criteria and stakeholders, but limits their mandatory and voluntary fire building owners' needs in the in the risk assessment process. An No mention of the role of risk the key responsibilities specifies the designer involvement in the fire risk safety objectives for the sake context of cost-benefit analyses, explicit definition of risk owner is not 2.2 of the risk owner in PC PC DC responsibility for each one of DC assessment process to that of DC of clear understanding of the DC owners is made in the but not their responsibilities or provided, although a risk manager is ensuring an adequate the five criteria presented. This being informed of deviations purpose of the fire safety document. specific role towards fire safety. introduced in section 3.4.11. This risk fire safety can be interpreted as the risk in order to document them measures. The guidance does This can lead to interpreting that manager is portrayed as someone performance ownership of the designer, but and seek their agreement. not refer to how the risk owner the guidance document has balancing costs vs. acceptable risks, fails to capture the requirement This diverges greatly from should participate in the risk implicit assumptions or which does not constitute the definition of the criterion. the concept of risk assessment process or the role presuppositions about the attitude of a risk owner. ownership and stakeholder of its outcomes in the decision of the building owner towards engagement. making process. risk ownership.

As a matter of ensuring the validity of the assessment or identify reassessment needs, the building owner is required to be educated regarding management of change (7.4.3.3). Section 4.5 requires assessing uncertainties in models and methodologies, including assumptions and data used. These do not address how to identify and manage epistemic uncertainties associated to Section 6.1.3 recognizes the scenario completeness and complexity of the fire safety description of performance. The Section 6.5.4 explicitly states requirement does point towards Section 6.5.2.1 defines problem and that all models that the use of engineering Despite mentions regarding the need for 'reasonable' parameter uncertainty and will be imperfect. This leads judgment for frequency and uncertainty due to lack of assurance of the acceptance within it epistemic to the introduction of a consequence estimations knowledge or imperfect criteria being met. uncertainties. In this context it damage model in section does not have to be knowledge in section 6, does not address performance 6.2.5. This is a conceptual individually contemplated, section 7 dealing with A listing of limitations and Section 13.3.2.2 addresses uncertainty modelling uncertainties or representation of the system Epistemic uncertainty but can be combined into a uncertainty does not explicitly (1): Guidance is provided assumptions are required as part sources which are not quantified in the limitations. and acts as a tool for the single quantity. address epistemic uncertainty 3.1 on the identification and MC of the documentation, specifically MC analyses and the need to justify them, as MC PC DC MC or its implications in the analysts to express the management of epistemic the operations and maintenance well as defining an approach to treat In general, section 6.5.5 Section 7 states that verification of safety. Section knowledge boundaries of the uncertainty. manual (section 7.4.2). This them. provides guidance on uncertainty is not only 7.3.3 refers to model assessment, including implies that assumptions have an uncertainties treatment through aleatory but can also be a uncertainty, addressing that epistemic uncertainties. impact on the operation and a five step process product of 'gaps or errors in form software packages and Section 6.3.3 detailedly future state of the building and a (identification, screening, the knowledge underlying indicating the need for provides guidance on the requirement to protect them is put characterization, analysis and the procedure of computing 'considerable margin of errors'. forward (section 7.4.2.2-3, 7.4.3 - reporting). possibilities to construct a the risk measure'. Management of Change). There is damage model, exemplified no particular emphasis on with a case study. epistemic uncertainty, although it is not needed. 3 Uncertainty

A relevant reference is introduced in A.4.5(1) to the Ch. 76 of the SFPE Handbook dealing with uncertainty. Section 6.5.3.2 described the Section 5.1.4.4 states the concept of limit state equation, requirement for clearly describing which serves to quantify the the assumptions of the model The document does not performance of the system. used. Section 5.6.4 also addresses provide guidance regarding However, no explanation of the need to clearly describe the formulation and of the the process to construct it Section 10.5.4.3 addresses the issue of assumptions in the model. model used for performance (modelling) is provided and it probability distribution selection and the quantification (whichever the is only exemplified for the Model uncertainty is not potential issues of using a 'default' Specifically for the quantitative Section 6.5.3 addresses model practitioner may choose). evacuation case (based on explicitly addressed by the Epistemic uncertainty distribution, although this is a particular and cost-benefit methodologies uncertainties, including ASET and RSET quantities). methodology, but the (2): Guidance is provided example. on the model uncertainty (7.3.3.4.4-6, 7.3.3.5.5-7)) there is assumptions supporting the Section 12.7 of ISO 23932- When describing the process assumptions of the models

3.2 identification and PC a consideration of explicitly DC MC analysis and model validity. It DC 1:2019 makes reference to PC to estimate the probability of MC Chapter 11 presents a simplistic employed for risk management, including stating (and documenting) the clearly states it cannot be the need to use engineering failure through reliability representation of what a consequence quantification (section 6.3.4) how it is defined, limitations of the models used in eliminated, although models judgement when calculation methods it is acknowledged analysis is in a fire risk assessment. It need to be explicitly stated quantified and bounded. the assessment, the scope of the can be improved. methods or data are not that the provided targets does not specify the need to formulate a and judged (section 6.3.6). analysis and also related to the available, however this is not (acceptability criteria) only model for performance and to define its selection of specific models extended to the formulation applies if the limit state boundaries. (calculation tools). of a model that represents equation distributes normally. the performance of the fire The document does not Section A.4.5(2) includes a brief, safety strategy. explicitly state that the non-generic, discussion about presented example of using the theory and model uncertainty. reliability index is only valid when both quantities (ASET, 378

Appendix 3. Comparative evaluation of guidance for fire risk assessment – Full results

Evaluation Specific criteria NFPA 551 Comment SFPE Guide Comment PD 7974-7 (2019) Comment ISO 16732-1 (2012) Comment TR 951 (2019) Comment MAD Comment criteria RSET) are normally distributed.

Guidance is also provided regarding Bayesian networks and the models these require to estimate the probabilities of a system. References are provided, but no direct consideration of the model that quantifies performance.

Section 7.3.3 provides guidance on 'model uncertainty' but limits its scope to software issues.

No explicit consideration is given regarding the understanding of the performance of the fire safety strategy through a model. The definition of "Fire scenario, representative" provided in Ch. 2 provides a first glance at the limitation of scenario discovery, as the representative fire scenario is assumed to have a consequence that adequately describes the consequences of a scenario cluster.

Section 8.3 addresses representative fire scenarios and a set of qualitative statements are made regarding how to structure them. These include the assumption that a single design fire calculation represents the average The guidance acknowledges in consequence of all scenarios in the section 6.2 and 6.3 that the cluster. Additional considerations are scenario space is virtually built upon this assumption. infinite and although

references the ISO document, Section 5.5.1.1 argues that Guidance 8.3.4 and 8.3.5 aim at assessing Section 6.2 provides it does provide additional unrealistically severe or maximum foreseeable loss and 'near' guidance on scenario guidance. Specifically it Section 6.2.4 introduces the sufficiently infrequent scenarios worst-case scenarios (defined by specification and selection, addresses the important fact use of scenarios in the can be excluded, but provides no percentages of consequences). However, Section 6.5.4 addresses echoing the information that each scenario should be methodology and the issues guidance or definitions to help these do not explicitly state the purpose completeness uncertainty, Epistemic uncertainty presented in ISO 16733- linked to the fire safety of discovery when multiple define them. of such an exercise or what the outcomes which in particular includes (3): Guidance is provided 1:2015 in sections 6.2 and objectives. There is a failure modes and unknown are of assuming a particular (or full) set hazards, initiating events, on the limitations of 6.3. These sections provide requirement to formally conditions exist. Mainly, the scenario discovery, Section 5.5.8 states that of safety measures fail. failure modes, operation 3.3 PC MC MC PC guidance on identifying PC document -along with MC including the subjective completeness and robustness of modes, behaviours, phenomena methodology focuses on scenarios and clustering references- those scenarios not nature of the process and quantitative methods depends on Section 8.6 identifies a relation between or other relevant factors used bounding the validity of the them, but fails to provide a selected. Section 7.1 indicates of the scenarios the correct selection of scenarios, the scenario base and the available for scenario construction results rather than on structured process to do so that the probabilistic analysis themselves urging to select sets "shown to resources. This is a key element to notice (although this is not explicitly perfecting the scenario and mainly resorts to helps describing uncertainty, represent all possible outcomes". from the guidance as it impacts the basis stated). discovery (sections 6.2.4, hypothetical examples. This except when the scenario This by definition poses a for the assessment. 6.3.3 and 6.3.4). in part reflects the inherent space is not complete. prohibitive restriction on the subjectivity of the process. problem. Sections 8.7.5.2 and 8.7.5.3 provide No explicit consideration is important cautions to practitioners given to the subjective nature regarding exclusion of scenarios and the of this process and to the need for documenting and implications of undiscovered communicating it to the relevant scenarios. stakeholders, particularly in the case of scenarios with high and 'unavoidable' consequences.

Sections 13.2.3.2-3 state that an adequate (comprehensive and systematic) hazards identification and scenario discovery allows removing it from uncertainty quantification steps. Although Ch. 8 provides guidance, an 'appropriate' approach is not formulated. Finally, the treatment for missing/overlooked/ignored scenarios is not addressed. Ch. 9 provides guidance and insight into Considering the probabilistic Section 7 recognizes potential Section 6.3.4.1 discusses the potential complication in acquiring data and quantitative nature of the limitations of the data, but little use of conservative values for and verifying it is applicable to the method being presented, the Sections 4.4.5.2-4 establish the guidance is provided on how to the variables involved in the system being analysed. It explicitly document is mainly requirement to document the treat them. There is no explicit addresses the 'near-miss’ and unreported qualitative and vague in Data uncertainty is addressed risk quantification and also scope and limitation of input data, mention of the particular issues data (Section 9.2.3). However, defining key elements. In in section 7.3.2, calling for the introduces the treatment of Data limitations: as well as clearly explaining any associated with the throughout section 9 some of the direct relation to the data practitioners to consider the the background knowledge Guidance is provided for assumption or default values. completeness of the data guidance on these issues refers back to there is a set of provisions in effect of sample size and other supporting any input used. analysis alternatives in recorded in the fire context. 3.4 the absence of, or poor MC MC industrial systems, where data gathering PC PC section 6.3.1 regarding the PC relevant parameters on the MC Section 6.3.6 then provides Section 6.2 establishes general quality, data to support differs drastically from that in the built selection of databases and final quantification. No guidance on how to address requirements on the data quality, Examples of data are provided probabilistic and environment (See chapter 4). the assignment of structured process or set of including availability, type of sprinkler reliability (based on uncertainty introduced by reliability calculations frequencies to non-reported requirements are provided for sources, lack of data, records USA data), fire start inputs and manage it in the Reliability data is discussed in section scenarios. No guidance is this. management, applicability and probability, fire growth rate context of bounding the 9.2.4 but no details are discussed, provided on how to manage uncertainty and variability. distributions, fire load density validity of the assessment. including the tools to estimate or this uncertainty besides per occupancy, system Although not exemplified, construct it, nor about the limitations of sensitivity analysis and effectiveness (reliability this includes data for using typically available data to infer it. 'careful selection' of 379

Appendix 3. Comparative evaluation of guidance for fire risk assessment – Full results

Evaluation Specific criteria NFPA 551 Comment SFPE Guide Comment PD 7974-7 (2019) Comment ISO 16732-1 (2012) Comment TR 951 (2019) Comment MAD Comment criteria ranges), damage extent (based probabilistic distribution as probabilistic (and Section 9.2.7 provides guidance on the on USA experience). stated in section 7. deterministic) analyses. use of engineering judgment in the absence of data and the biases and potential pitfalls it entails. Section 9.2.7 also addresses engineering judgement which is not limited to data use (see criterion on data limitations below). It addresses the issues associated with this subjective but necessary tool and suggests techniques such as ranging, bracketing, partitioning and iteration to improve judgement.

Section 9.3 addresses some limitations of the data sources, but if mainly focus on incorrect application. The quality of the data as well as the reporting issues discussed in Chapter 4 are not addressed. Section 7.2 introduces sensitivity analysis as a way to understand the impact of Section 13.3.6 describes step 5 in varying mathematical uncertainty analysis guidance and formulations (models) and includes switchover analysis. This Precision is explicitly inputs on the performance; analysis hints at understanding whether considered for the number of however, no mention is the assessment can err on the Monte Carlo simulations Definition 3.27 - Uncertainty Section 6.2.6 and Figure 7 provided of explicit Section 5.1.3.7 refers to unacceptable performance side. No required (Section 6.2.5). includes "failure to include a mention the need to define a Precision requirement: quantification of required uncertainties expressed both in a detailed guidance is provided to execute Although precision is not relevant element". This is the safety margin and Guidance is provided to precision. qualitative and quantitative it beyond reference to the SFPE directly linked to the difference only mention of the furthermore, to define the explicitly quantify the manner. It states that in problems Handbook. between the result of the performance quantification trustworthiness of the precision of the Section 8 makes a clear assessment results and where cumulative effects of analysis and the acceptance not capturing important 3.5 DC PC PC DC PC statement regarding the PC assessment's result. Section verify that in case of uncertainty the latter can be Section 14.4 also hints at the issue of criterion, it is formulated in aspects of the system. No conclusions reached by the 6.3.6 establishes the basis to invalid assumptions the particularly useful. erring on the unacceptable performance general terms and could be reference is made to the need analysis and the effect of bind the validity of the estimates do not err on side. It recommends (without supporting applied to it. Specifically, for a specific level of assumptions on them. It assessment, through the unacceptable There is not an explicit reference evidence) implementing detailed stakeholders could require a precision, nor the performance explicitly states that no analysis of the information to the precision requirement. analysis, when the basis for decision is specific precision requirement. implications of erring on the assumption or simplification registry. simple analysis. If this is not possible the This does not seem to be less safe side. should alter the conclusions guidance indicate that the risk should not communicated elsewhere to beyond the justifications and be considered acceptable and proposes inform decision-making. considerations provided. This reducing risk or revisiting the acceptance could be considered a clear criteria. requirement on the precision of the result, although it is not formulated in such clear terms. Section 8.7.2 contemplates using qualitative methods, which include worst-case evaluations and maximum Figure 1 acknowledges Probabilities are the Section 4.5 considers foreseeable loss. These can be classified Section 5.1.1.5 mentions the deterministic analyses. These representation of choice for deterministic analyses. These The methodology applies a as non-probabilistic, despite the Non-probabilistic possibility of conducting can be classified as non- uncertainty in this document can be classified as non- non-probabilistic approach by underlying assumptions to define the alternatives: Guidance is deterministic analyses in order to probabilistic, despite the (see introduction). Section probabilistic, despite the itself. It does not make direct scenarios are mainly probabilistic, though provided on non- select scenarios. Although can be underlying assumptions to 6.3.1.3 references the reader underlying assumptions to 3.6 probabilistic uncertainty DC PC binary. These alternatives are not PC DC PC PC reference to other deemed a non-probabilistic define the scenarios are mainly to the use of Delphi methods define the scenarios are mainly analysis alternatives portrayed as options to better understand alternatives, although the alternative, it is not used as a probabilistic, though binary. to reduce the bias product of probabilistic, though binary. when the application of a uncertainty. Although section 9.2.7 larger body of work does manner of accounting for These alternatives are not differing engineering These alternatives are not PRA is not granted suggests techniques to mitigate include them (Annex 1) uncertainty. portrayed as options to better judgment between portrayed as options to better subjective judgements closely related to understand uncertainty. practitioners. understand uncertainty. P-boxes, there is no mention of non- probabilistic alternatives for uncertainty analysis. Annex B.1 addressing the validation of fire statistics to Section 4.4.2 points out the need to be used in the probabilistic risk consider the effect of occupancy changes assessment requires on likelihoods. considering the changes in Section 6.2.6 mentions the Time dependency Section 7.4 explicitly addresses regulations and developments need to maintain the assumption: the guidance the need for constructing and Sections 15.4 and 15.5 address bounding No reference is made to the in fire safety strategies, as performance over time. explicitly recognizes that maintaining an operations and conditions and monitoring for change. changes in time to the key these influence data. This Section 6.3.6 describes the a snapshot of risk in time No reference is found in the 4.1 MC maintenance manual to identify MC These state limitations on the validity of DC DC variables affecting fire risk DC consideration calls for using MC is not sufficient to guidance document. use of the information effectively manage risk the conditions that must be kept the risk assessment, as well as the need to or their consideration in the the most recent available data. registry and trustworthiness throughout the building in order for the risk assessment to monitor change in the building, which uncertainty analysis. However neither, Annex C, information to manage life-cycle remain valid. should lead to an "operations and Annex B nor the main body of assumptions thorughout time. maintenance manual". SFPE Handbook ir the guidance refer explicitly to cited for further guidance on this the dependence of all figures Risk as a document. on time and therefore to the 4 function of need to consider their change time in time. Section 5.4.4.2 addresses the Section 6.4.1 addresses the limitation on availability of 'good' need to use available data and data, pointing out the need to Section 7 explicitly provides Section 6 fails to indicate the the assumptions involved in its replace it with assumptions or guidance on data collection, need for data gathering, interpretation. A few key inter/extrapolations. Reference is made to the NFIRS database pointing the user directly to including component, sub- criteria are provided to check Data gathering: Guidance in Chapter 9 and guidance is provided on data sourcing from other fields systems and systems failure the data is satisfactory and to No mention of data gathering is provided to construct, Section 5.4.6.2 mentions the need the role of data and how to apply it in the (industries) without mention of data under fire and normal discard unreliable data. 4.2 obtain or estimate DC DC PC DC MC DC of guidance for it is provided reliability data specific to to consider the reliability of data risk assessment, but not directly on how the potential biases and issues operation. It also fails to Examples are not provided, in the document. the system studied collection and the possible to construct it, the different sources to of doing so. No guidance is mention the characteristics although Annex B provides subjective biases that could be obtain it or the methods to estimate it. provided on how data should of the necessary data for a sample statistical data. Sample introduced with it. be collected and how its quality reliability analysis as part of size is addressed in section should be judged. a probabilistic assessment. 7.3.2. Section 6 addresses data and includes general guidance on Annex C addresses reliability 380

Appendix 3. Comparative evaluation of guidance for fire risk assessment – Full results

Evaluation Specific criteria NFPA 551 Comment SFPE Guide Comment PD 7974-7 (2019) Comment ISO 16732-1 (2012) Comment TR 951 (2019) Comment MAD Comment criteria records management. No specific specifically, providing guidance is provided on how to references for the data sources construct the records and ensure as well as resources to estimate data quality throughout life-cycle. probability of failure of selected safety measures. Furthermore, Annex C.9 provides an introduction to reliability analysis, which aids practitioners in estimating reliability figures. Section 5.4.4.2 addressing the use Section 4.2.2 assumes that a of assumptions in lieu of 'good' PRA is not based on the Section 7 specify the need to data mentions the need to Section 4.4 addresses the protection of experience of the profession state and manage key explicitly state them and treat assumptions and in 4.4.1 explicitly states nor a large level of uncertainties (mainly aleatory them through sensitivity, in order that "the results of the risk analysis hold conservativeness. These are ones). Section 8 puts forward to account for the uncertainty to the extent that those conditions and assumptions, not explicitly the need to reach conclusions Section 6.3.5 states that they introduce. There is no assumptions remain in force." No stated, and simply used as which are not affected by regardless of the result of the examples of guidance on how to guidance is provided on how to Key assumptions: Any gateways for the PRA process. unjustified assumptions or performance assessment, the do this and no link to any other implement this in practice. key assumptions of the The level of knowledge gained 4.1 states that reliability is simplifications. This is a clear information registry can be part of the risk assessment methodology including in a deterministic analysis inherently probabilistic, call to competent practitioners used to bound the validity of process. A key element, often described as a tool, tractability, application seems to be downplayed, while the estimation of to reflect on all uncertainty the result and to understand is the assumption of tractability (see e.g. boundaries and although no detailed frequency and probability sources and their effects on the 5.1 PC (See next criterion) There is a PC Section 9.2.7.4). The issues associated to PC PC MC MC the required design ambiguity are clearly explanations are provided. (6.3) depend on it. The final conclusions drawn from requirement for stating such assumption or the boundaries for its modifications to achieve a stated by the limitations on available data the analysis. This key element assumptions and limitations. application are not discussed. better performance. Section document, providing Importantly, Figure 6 and to obtain reliability measures is formulated only here and in However, the assumptions and 6.3.6 provides an example of guidance to manage Annex C presenting a hierarchy is not mentioned. general terms, limiting its limitations underlying the Section 15.4 of the documentation the use of this information to them of acceptance criteria hold potential benefits on the guidance and its definition of the section addresses bounding conditions bound the validity of the important assumptions which overall process. Source methodologies (which include and the need to explicitly state the are not addressed in the references like Wolski address assessment. both ambiguity and tractability) conditions or assumptions in the analysis. guidance and can have a significant assumptions and are not fully addressed and only No practical examples are included and significant impact on the way criticism, which are not specific (limited) elements of it no statement of the potential use of this practitioners use it and conduct included in the guidance and are presented in section 7.3 section is provided. themselves in relation to the can impact decision-making. associated with comprehensive desired result. project documentation. The methodology does not include a section specifying Section 7.3.3 describes the documentation requirements per the documentation methodology type, including an The guidance does not requirements. The three case 5 Assumptions explicit statement of limitations. include any documentation studies presented in Chapter Documentation: Specific -yet typical- limitations requirements, however its 6 and 7 provide an Guidance is provided per type of methodology are Section 15.3 addressing the full parent document ISO23932- exemplification of how to to document the key presented. documentation includes the justification 1:2019 in its section 14 document the risk Section 8 provides a detailed results and for scenario selection/exclusion, as well No reference is found in the presents guidace for the final 5.2 MC MC PC PC MC table of contents for the PC assessment, while Annex 4 assumptions of the risk As part of the operations and as the results of uncertainty analysis. No guidance document. report incluidng scope, documentation. provides the format and assessment to enable maintenance manual there is an explicit mention of particular uncertainty chosen risk analysis contents of the information peer review and explicit requirement to list the sources is made. approach, methods, input registry. Despite the auditing limitations and assumptions that data and justifications, and a previous, no explicit and support the risk assessment (these description of the quality specific guidance is provided include associated controls) (7.4). assurance. A list of topics is provided with regards to (7.4.2.3) documentation of the risk assessment.

The peer review process Section 8 provides detailed Section 5.5. establishes the established in section 7.2 is guidance on reviews, including need for a review process to described as a fail safe when direct review by the AHJ and "interrogate the underlying basis for quantification are Section 4.9 specifies the need Peer review: Guidance third party reviews. These include principles and assumptions". not sufficient. Although ISO for documentation to enable is provided for a the verification process, in which Although a non-exhaustive list 23932:2009 is referenced, review and control procedures, Section 6.3.6 explicitly states structured peer review the risk assessment is checked for Section 13.3.2.2 suggests the use of a is provided, these are mainly the updated ISO23932- but does not elaborate further. the use of the information process that checks the predictable and consistent results "quality review process" to ensure focused on the assumptions 1:2019 only mentions peer Section 8 provides a detailed 5.3 underlying key MC (8.2.1); this step includes MC PC PC PC MC registry as a basis for peer correctness and completeness and/or underlying the probabilistic reviews as a tool to achieve table of contents for the assumptions of the risk replication of results (by review, along with the addressing its effect in qualitative terms. model and distributions. agreement with AHJs. documentation, without assessment and alternative calculations), auditing damage model. Further documentation of Overall this is a confusing providing any additional therefore complement selected numerical results. uncertainties that can enable a element which does not details on the review/control its results Furthermore, a set of review detailed peer review is reflect the importance in the processes. questions are formulated to described in the parent decision-making process of established the appropriateness of document. peer reviewing the fire risk the assessment (8.3). assessment.

381 Appendix 3. Comparative evaluation of guidance for fire risk assessment – Full results

382

Appendix 4

Information registries for the case studies

383

Appendix 4. Information registries for the case studies

Case study 1 – Multi-occupancy office building – Objective: Life safety – Section 6.5

Uncertainty Strength of Variable Nature Justification Value Range Sensitivity Importance type knowledge This value defines the amount of heat released by the burning fuel, however it is deemed irrelevant Heat of combustion given that the alpha value of the fire growth includes it and allows covering a wide range of materials Input 27100 Aleatory High Low 1 [kJ/kg] including their associated heat of combustion. In order for the selected modelling tool to perform mass and energy balances, the recorded value for PU is taken. Landing length per Input Given by initial design 2.5 Epistemic High Low 1 level [m]

Stair riser [mm] Input Given by initial design 178 Epistemic High Low 1

Stair tread [mm] Input Given by initial design 279 Epistemic High Low 1

Polyurethane and polyurethane foam are materials easily found in many items in a typical office building, which has a high energy output when burning and yields toxic substances into the hot gas Fuel Input C H O N Epistemic High Medium 2 layer [13]. A fire involving this material is taken as the least conservative condition, given that all N 25 42 6 2 and Cl will be converted into HCl and HCN by the used modelling tool. Radiative fraction This variable determines how much heat is released in the form of radiation. A typical and conservative Input 0.35 Aleatory High Medium 2 (XR) [-] value for most materials found in an office building is 0.35; this value will be selected [REF needed]. The production of CO has no effect on the descent of the smoke layer, despite it being toxic and CO yield (Y ) harmful for occupants. Given that the damage will be assessed based on zero exposure of the occupants co Input 0.23 Aleatory High Medium 2 [kg/kg] to the effects of the fire (i.e. use of height free of smoke or temperature threshold [SFPE 14]), this variable is not relevant and its value will be fixed in a higher end of the ranges reported in literature. The duration of the fire is one of the key variables to determine the damage potential. From the Duration of the fire evacuation point of view, the worst possible condition is that the fire continues active during the whole Output Until burnout Epistemic High Medium 2 [s] evacuation, therefore it is considered not relevant and the value assigned is larger than the values of the largest ASETs obtained in preliminary tests. The Available Safe Egress Time can be defined through “simple criteria for tenability based on zero Smoke layer height Output exposure: …minimum clear layer height of 2.5 m above the floor" as stated in [14]. The height of most 2 Epistemic High Medium 2 [m] compartments is 2.5 m, therefore a 2 m criteria is taken. Given that the temperature of the smoke layer can also influence the tenability conditions, the 200˚C Maximum upper- criterion will also be taken into account when this occurs before than the 2 m criterion for smoke layer Output 200 Epistemic High Medium 2 layer temperature [C] height. Partial results show that the smoke layer height is more conservative than this variable; nevertheless, checks will be performed to ensure to take the most conservative one. Occupants displacement speed Input Based on egress statistics [SFPE] Variable Aleatory High Medium 2 (horizontal and vertical) [m/s]

Constant 'a' Input Based on egress statistics [SFPE] 0.266 Aleatory High Medium 2

384 Appendix 4. Information registries for the case studies

Uncertainty Strength of Variable Nature Justification Value Range Sensitivity Importance type knowledge

Door effective width Defined by initial design. The design includes 0.95 m width doors, but the effective width based on Input 0.65 Epistemic High Medium 2 [m] egress statistics [SFPE] is assumes to be 15 cm less in each side, resulting in the selected value. This constitutes the first element of most fire safety strategies, and so it does in this case. However, Heat, Detection Input detection can be achieved by heat or smoke detectors. Both will be tested in the case-study to assess Epistemic High High 3 smoke their impact on RSET and therefore on the damage potential. This represents how much heat is released per unit area of the fuel, therefore the lower this value is, the Heat release rate per larger the area will be to maintain the established Heat Release Rate (see Fire dynamics variables). unit area (HRRPUA) Input There is a physical limit to this value, since the area of the fire cannot be larger than the area of the [10, 500] Aleatory High High 3 [kW/m2] compartment. From previous tests, the range represents the worst condition at lower values and is still within measured ranges [SFPE]. This variable takes on a range given that a high soot yield can create worst conditions inside the Soot yield (Y ) [0.001, soot Input building during the fire (e.g. toxic concentrations), but if it is low enough the detection times could Aleatory High High 3 [kg/kg] 0.4] increase significantly. Number of stairwell Input Given by initial design 1 Epistemic High High 3 shafts

Type and number of Input Given by initial design See Table 4 Epistemic High High 3 occupants per level

RTI - heat detector Given by initial design and available technology. Lower values (i.e. higher sensitivity) might result in Input 30 Epistemic High High 3 [ms^1/2] higher costs, requirements for non-available technology or trigger false alarms. Activation temperature - heat Input Idem 57 Aleatory High High 3 detector [C] Activation obscuracy - smoke detector Input Idem 14 Aleatory High High 3 [%/m] Number of doors During evacuation, emergency doors will be open and some may fail to close. The least conservative open in stairwell Input All Epistemic Medium High 6 condition is that all fail to close, leading to have all doors open since the beginning of the fire. shafts Location of the fire in Given the relevance of this variable and its direct influence on results, a wide range of locations will be [B2, O1, Input Epistemic Medium High 6 the building explored, including one level per type of use: O4, O6] The available fuel mass defines the duration of the fire and therefore the duration of the smoke production. For assessing the damage for the evacuation stage, this variable is assumed to be irrelevant Fuel mass [kg] Output Maximum Epistemic Low High 9 and it will take a fixed value that ensures that the smoke production does not stop even at the highest available safe egress times. It is assumed that the growth of the fire will involve a radial flame spread following a time-squared Radial power- Type of fire growth Input function. This growth captures the combustion, flame spread speed and is only capped by the peak heat law (αtn) with Epistemic Medium High 6 release rate possible in the compartment. n=2 A low value of the fire growth rate may lead to a long smouldering fire, while a large value can Fire growth rate (α) [1x10-4, Input represent a very fast and short fire. Hence, a wide range of fire growth will be explored, taking as upper Aleatory Medium High 6 [kW/s2] 1x100] and lower limits orders of magnitude of known materials.

385

Appendix 4. Information registries for the case studies

Uncertainty Strength of Variable Nature Justification Value Range Sensitivity Importance type knowledge There is no justification for selecting a maxHRR value, given that during evacuation the worst possible Peak Heat Release Output condition is represented by an ever growing fire. The selected value allows the two-zone modelling tool Maximum Epistemic Medium High 6 Rate or HRR [MW] to properly function, given that it does not solve the physics behind flame spread and heat output.

Notification time [s] Input Based on egress statistics [SFPE] 30 Aleatory Medium High 6

Premovement time in Input Idem 30 Aleatory Medium High 6 fire location [s]

Premovement time in Input Idem 60 Aleatory Medium High 6 remote location [s] Density near flow constriction (door) Input Based on egress statistics [SFPE] 1.9 Aleatory Low Medium 8 [persons/m2]

Case study 2 – High-rise residential building with non-compliant façade – Objective: Life safety – Section 7.1

Uncertaint Importanc # Variable Value/Range Unit Description Justification Source of information Assumptions SoK OS y nature e

It is assumed tenability can be Smoke layer defined in function of the potential At this height occupants can start height - to breath the hot toxic gases of the 64 2 m Exposure criteria inhaling the toxic gases of the hot SFPE handbook Aleatory High Medium 2 Exposure smoke layer, and not in other gas layer threshold criteria like heat exposure or visibility. Burning fuels in CFAST are This time is based on tenability assumed to be hydrocarbon fuels criteria. Since the stairs cannot be that contain at least carbon and modelled using simple geometry hydrogen and optionally oxygen, and ventilation, it is modelled as a nitrogen, and chlorine. All box in which the smoke flow of the nitrogen and chlorine are Time for fire is received. This allows converted to HCN and HCl. CFAST7 Users guide, chapter 56 untenable Output s Time estimation tenability to be defined in function An important assumption is the Epistemic High Medium 2 7 stairs of the smoke's quality, i.e. toxicity. fact that reaching this To estimate the smoke's quality, concentration will lead to CFAST (version 7) one step untenable conditions at the stairs in combustion reaction model is used the level of origin as well as in the and its outputs compared to the level above, but not in the rest of thresholds presented below. the levels, due to mixing and reduction of concentration.

386

Appendix 4. Information registries for the case studies Uncertaint Importanc # Variable Value/Range Unit Description Justification Source of information Assumptions SoK OS y nature e

Tenability times in staircase given 70% of the 10000 ppm threshold by smoke quality. Toxic CO for which 60% COHb is reached in Literature Yves Alarie, Toxicity of Fire CFK modified model applies and 57 concentration 7000 ppm 6.2 minutes doing extreme exercise Aleatory High Medium 2 information Smoke, 2002 toxic effects come from CO. HCl threshold with a limiting traveling distance of and HCN disregarded as modelled 190 m. fuel does not contain Cl.

Toxic HCN Literature At this value humans can be SFPE Handbook, Ch. 62, pp. Tenability times in staircase given 58 concentration 150 ppm Aleatory High Medium 2 information incapacitated in a few minutes. 2223 by smoke quality. threshold Typical material found in dwellings PU - Representative PU is a representative fuel for the and which has grown popular due 31 Fuel C25H42O6N - material that will ***Check Case study 1 occupation class Aleatory High High 3 to its use in low cost furniture and 2 burn in the fire Single material burning stationary Source of combustible fuel that will Typical load associated with the 32 Fuel load 26 kg/m2 Range allow the fire to grow and Report Prof Torero Aleatory High High 3 dwelling occupation class propagate Fuel burning 33 [0.01, 0.014] kg/m2s Range Report Prof Torero Single material burning Aleatory High High 3 rate Fuel heat of 34 [40, 46] kJ/kg Range Report Prof Torero Single material burning Aleatory High High 3 combustion Hurley, M.J. and M.J. Hurley, SFPE handbook of fire protection engineering / Fuel 35 400 kW/m2 Point value Associated to the burning material Morgan J. Hurley, editor-in- Single material burning Aleatory High Low 1 HRRPUA chief. 5th ed.. ed. 2016, New York: New York : Springer; Table 26.15

This value is a high value that yields a sooty smoke, leading to lower visibility in shorter times. It 36 Soot yield 0.2 kg/kg Point value Associated to the burning material is not the extreme value of 0.4 Aleatory High Low 1 corresponding to crude oil, but is close to it to account for multiple materials burning.

Fire grows in an αt^2 rate, which is conservative but commonly This is a conservative approach to used in most fire literature, Fire growth [0.0029, fire growth which also encompasses SFPE, multiple references, including SFPE Handbook and 37 kW/s2 Range Aleatory Medium Medium 5 rate 0.1876] a reasonable range of behaviour, Report Prof Torero Holborn et al, An analysis of fire from slow to ultra fast growing fires sizes, fire growth rates and times between events using data from fire investigations, 2004

387

Appendix 4. Information registries for the case studies

Uncertaint Importanc # Variable Value/Range Unit Description Justification Source of information Assumptions SoK OS y nature e Every combustible material has a critical heat flux above which it will Fire will not spread before Fire spread ignite. Providing a threshold to Hadden, et al, Radiant reaching this critical heat flux, criteria - Heat allow for the fire to spread Ignition of Polyurethane 48 [5, 7] kW/m2 Expert criteria which corresponds to the critical Epistemic Medium High 6 flux to throughout the contiguous locations Foam: The Effect of Sample heat flux for the fuel defined in the ground of the fire origin is reasonable, Size, 2014 system (PU). despite not a physical representation of the spread. A fire will occur within the falts Location of Any A fire can start at any location and fires will not occur in the 38 - Range Aleatory Low High 9 initial fire compartment within the building corridors or stairs, as this yields a trivial case.

This material property is intended Pau, et al, Thermophysical System's to allow for fire spread within flat 48 PU thickeness 0.1 m properties of polyurethane Epistemic High Low 1 information compartments. It is a typical foams and their melts, 2013 combustible upholstery material.

This material property is intended Pau, et al, Thermophysical PU thermal System's to allow for fire spread within flat 49 0.05 W/mK properties of polyurethane Aleatory High Low 1 conductivity information compartments. It is a typical foams and their melts, 2013 combustible upholstery material.

This material property is intended Pau, et al, Thermophysical PU heat System's to allow for fire spread within flat 50 2996 J/kgK properties of polyurethane Aleatory High Low 1 capacity information compartments. It is a typical foams and their melts, 2013 combustible upholstery material.

This material property is intended https://www.engineeringtoolb System's to allow for fire spread within flat 51 PU emissivity 0.9 - ox.com/emissivity- Aleatory High Low 1 information compartments. It is a typical coefficients-d_447.html combustible upholstery material.

Staircase Doors are malfunctioning, not to Binary Defines ventilation condition and 39 doors Open - specifications, kept open by Aleatory Medium High 6 (open/close) smoke propagation through stairs condition occupants during evacuation

Zero implies worst condition in compartment of origin Time to open Defines ventilation condition and Occupants have a delay in their 40 fire to rest of 0 s Aleatory High Low 1 smoke propagation through flat reaction within the origin compartment Occupants do not detect fire immediately

388

Appendix 4. Information registries for the case studies Uncertaint Importanc # Variable Value/Range Unit Description Justification Source of information Assumptions SoK OS y nature e Zero implies worst condition in Time to open compartment of origin Defines ventilation condition and 41 flat door to 0 s Occupants attempt extinguishing Aleatory High Low 1 smoke propagation through level rest of level fire or prepare to evacuate before opening doors Smoke lobby System's 23 1.37 m Report Prof Bisby Epistemic High High 3 presence information Siaka Dembele*, Ricardo A.F. Rosario, Jennifer X. Typical commercial glass Glass Scientific Wen, Thermal breakage of Glass' temperature is estimated by characteristic; furthermore uPVC 43 breakage 80 C literature reported window glass in room fires the heat transfer model of CFAST Aleatory High Medium 2 loses 80% of its stiffness by this temperature data conditions - Analysis of some assuming a plate facing the fire. temperature value important parameters, 2012 Report Prof Torero, 3.4.1

System's Glass' temperature is estimated by Windows 42 1.33 x 1.6 m information; see Report Prof Bisby the heat transfer model of CFAST Aleatory High Medium 2 dimensions Layout assuming a plate facing the fire.

Siaka Dembele*, Ricardo A.F. Rosario, Jennifer X. Glass System's Typical commercial glass Wen, Thermal breakage of 44 0.03 m Epistemic High Low 1 thickeness information characteristic window glass in room fires conditions - Analysis of some important parameters, 2012 Typical commercial glass Glass thermal [0.7, 0.95, System's characteristic, used for glass P. Pagni, Thermal Glass 45 W/mK Aleatory High Low 1 conductivity 1.4] information breakage and fire spillage time Breakage , 2002 estimation Typical commercial glass Glass heat [750, 820, System's characteristic, used for glass P. Pagni, Thermal Glass 46 J/kgK Aleatory High Low 1 capacity 950] information breakage and fire spillage time Breakage , 2002 estimation Typical commercial glass Glass System's characteristic, used for glass P. Pagni, Thermal Glass 47 0.9 - Aleatory High Low 1 emissivity information breakage and fire spillage time Breakage , 2002 estimation

The time at which a heat flux is The window will break from impinging on the façade wall, Time for thermal stresses at relatively low which is the output from the façade P. Pagni, Thermal Glass temperatures (see entry 43) and the 59 Output s Time estimation compartment fire modelling, Epistemic High Medium 2 exposure to Breakage , 2002 heat flux from the fire is specifically the time at which the fire considered to be the one of a window breaks due to the developed fire, applying entry 50. temperature of the hot gas layer

389

Appendix 4. Information registries for the case studies

Uncertaint Importanc # Variable Value/Range Unit Description Justification Source of information Assumptions SoK OS y nature e

Time delay There is a delay between the The compartment fire given the between breakage of the windows and the Drysdale, An introduction to window configuration yields a window unburnt combustion gases actually fire dynamics (Ch. 10.2) ventilation controlled regime, 62 0 s Expert criteria Epistemic High Medium 2 breakage and flowing to the outside of the SFPE Handbook (Ch. 86, p.p. which implies the production of external building and effectively mixing 3245) unburn combustion gases that will flaming with air. react in the outside of the building.

Applicability of equation to the case-study. Opening factor varies between 19.5 and 23.8, with 20 being the applicability limit. Although this might restrict the Abecassis-Empis model, Eq. 6.3 No use, the opening factors do not External heat Heat flux Draught, Fuel controlled fire, 50 Output kW/m2 Report Prof Torero, Table 4 account for additional window Epistemic High Medium 2 flux estimate minimum height (0.05m) for area due to uPVC failure and for window at compartment of origin door openings. It is assumed the correlation is applicable, with No Draught and fuel controlled conditions given mfuel"(0.14)>mvent"(0.078). Critical heat Expert The known range for critical heat flux for ACP materials comparable to the 51 18 kw/m2 criteria/Literature fluxes in facades is of 20-60 Report Prof Torero, 4.2 Epistemic High High 3 façade ones in literature information kW/m2 ignition Façade heating is neglected and it Time delay The heat transfer from the external is assumed that if the heat flux for external flames to the façade system is a impinging is higher than the 52 façade 0 s Expert criteria complex mechanism which allow Entries: 50, 51 Epistemic Medium Medium 5 critical heat flux for ignition, it ignition after for its heating and potential will ignite immediately, i.e. no fire exposure ignition. delay.

Solving this phenomenon would be Upward Rates can be compared to the ones Expert extremely hard and full of vertical registered in external fire events criteria/Past unknowns, a simplified way of 53 external 4 m/min Report Prof Torero, Fig. 23 and takes the average rate of the Epistemic Low High 9 events approaching it is taking reference flame spread Reports for Grenfell information spread rates reported in previous rate No wind fires.

Solving this phenomenon would be Rates can be compared to the ones Horizontal extremely hard and full of registered in external fire events external unknowns, a simplified way of 54 0 m/min Expert criteria Report Prof Torero, Fig. 23 and takes the average rate of the Epistemic Medium High 6 flame spread approaching is omitting it, as the Reports for Grenfell rate result would be irrelevant to the No wind ASET times in the level of origin

390

Appendix 4. Information registries for the case studies Uncertaint Importanc # Variable Value/Range Unit Description Justification Source of information Assumptions SoK OS y nature e

Wind can influence external flaming and the heating/ignition process of the façade, as well as the horizontal and vertical spread of the It is assumed that no wind will fire. Once the façade is ignited, Horizontal influence the external flaming, External previous events have shown that Abecassis-Empis, 63 and vertical 0 m/s thus allowing for it to reach its Aleatory Low Medium 8 conditions wind can help increase the spread HeatFluxFacade, PhD2010 wind speed maximum possible height above rate. This parameter is key (as the window. presented by Abecassis-Empis) when dealing with the structural integrity under fire, as it can lead to structural members exposure.

Time delay for windows This delay requires detailed breakage knowledge (not available) for the 55 0 s Expert criteria Window heating time is negligible Epistemic Low Medium 8 from external heating of the windows by the fire and re- external flames. entry Given the safety goal of evacuation, Census is valid and no major Building No. System's 1 [240, 480] the main vulnerable element is the Report Barbara Lane deviation from it occurs at the time Aleatory High High 3 occupation people information occupants of the system. of a fire

Detection Typical value for ionization and CFAST Technical Reference 66 24 %/m Point value Aleatory High Medium 2 criterion photoelectric smoke detectors guide, 6.3 The notification system will be available when a fire is detected and will be promtly operated, but There is a delay between fire Notification Evacuation exercises, the uncertainty on the actual 10 30 s Point value detection and the notification to the Epistemic Low High 9 time statistics emergency management requires a occupants wide range of times to be explored using half and ten minutes as bounds.

This time can delay evacuation Pre- considerably and it is associated 11 movement [30, 60] s Point value with the type of occupation, e.g. Statistics (SFPE handbook) Occupation will be as expected Aleatory Medium High 6 time sleeping residents vs. fully aware and trained day office workers

Numeric value Vertical 29 Output m/s with an error Statistics (SFPE handbook) Occupation will be as expected Epistemic High Medium 2 speed margin Occupants 16 density in [1, 2] p/m2 Range Assumed value Tight occupation Aleatory Medium Medium 5 staircase

391

Appendix 4. Information registries for the case studies

Uncertaint Importanc # Variable Value/Range Unit Description Justification Source of information Assumptions SoK OS y nature e

The speed of occupants will vary based on their density (people/m2) People's movement can be Numeric value SFPE Handbook of Fire Horizontal and a model is used to estimate the described as a fluid flow through 17 Output m/s with an error Protection Engineering, 5th Aleatory High Medium 2 speed speed, taking a reference of 1.4 m/s the building margin edition, Ch. 59 and adjusting to the density of each -tdisplacemen=max(tque,thor) room using statistical fit People's movement can be SFPE Handbook of Fire Que time at described as a fluid flow through 18 Output s Time estimation Protection Engineering, 5th Aleatory Medium Medium 5 doors the building edition, Ch. 59 -tdisplacemen=max(tque,thor) Stairs door System's Determined the flow of occupants 6 effective 1.37 m Report Bisby, p30 Boundary layer of 30cm Epistemic High Low 1 information going into the staircase width Although the width of the stairs is Stairs 5m, it is assumed that occupants horizontal System's 12 4 m Report Bisby, p26 will have to walk horizontally Epistemic High High 3 displacement information maximum 4 m in landings and 2m distance entering the stairs

This defines the longest scape route that the occupants will need to Lobby cover to reach the doors of the The estimation is done based on horizontal System's Report Lane, 15.2.10 + 13 18 m emergency stairwell. It is taken occupants being as far away as Epistemic High High 3 displacement information 2.15.16 form the available blueprints possible from the exit distance assuming the longest travel distance possible within the level.

Stairs riser System's 14 2 - Typical stairs Epistemic Medium Low 4 type information This defines the longest scape route that the occupants will need to Flat cover to reach the exit of the flat. It The estimation is done based on horizontal System's 15 11 m is taken form the available Report Bisby, p26 occupants being as far away as Epistemic High High 3 displacement information blueprints assuming the longest possible from the exit distance travel distance possible within the level. Number of System's 19 emergency 1 - Report Prof Bisby Epistemic High High 3 information stairwells Number of doors to enter System's 20 1 - Report Prof Bisby Epistemic High High 3 stairwell per information level Width of System's 21 0.95 m Report Prof Bisby Epistemic High High 3 doors information

392

Appendix 4. Information registries for the case studies Uncertaint Importanc # Variable Value/Range Unit Description Justification Source of information Assumptions SoK OS y nature e Effective System's 22 width of 0.95 m Report Prof Bisby Epistemic High High 3 information doors Number of The active extraction systems will No smoke System's 24 doors to enter - Report (Multiple) not be available when smoke go Epistemic High Low 1 lobbies information smoke lobby into the hall Vertical System's 26 distance for 3 m Report Prof Bisby Epistemic High High 3 information one level Flight System's 27 2.5 m Report Prof Bisby Epistemic High High 3 distance information Number of System's 28 23 - Report Dr Barbara Lane Epistemic High High 3 levels information

-No major obstructions exist on the Directly related to the distance the ceiling that would impede the Height of smoke has to travel to reach the smoke reaching the detectors 5 2.6 m Precise measure Report Bisby, p30 Epistemic High Low 1 compartment detector and therefore with the -the type of detectors are adequate detection time for the type of fire considered (flaming combustion)

All occupants fall into the range t vertical 25 Output Time estimation for which the displacement Aleatory Low High 9 displacement parameters are taken (entry 29) The way in which Staged evacuation is the original occupants are Evacuation Staged strategy for this building and it is 65 - instructed to Approved Document B Occupants will follow the strategy Epistemic Low High 9 strategy evacuation also part of the Approved evacuate in case Document B of fire

-No major obstructions exist on the ceiling that would impede the smoke reaching the detectors -the type of detectors are adequate It is the time for the fire to be for the type of fire considered Detection CFAST7 Users guide, 4 Output s Time estimation detected either by the occupants or (flaming combustion) Epistemic High Low 1 time detection model by an automatic detection device -Detectors will work despite failures, given redundancies -Despite estimation, minimum time for detection is taken as 50 seconds

393

Appendix 4. Information registries for the case studies

Uncertaint Importanc # Variable Value/Range Unit Description Justification Source of information Assumptions SoK OS y nature e 8 per level in Distance to the corridor The availability of a detector is Post incident analysis for All detectors work detector 7 1 smoke m Precise measure directly related to the time of Lacrosse building fire, PIA report shows alarms have Epistemic High High 3 (distribution alarm per detection Section 2 been tampered with of detectors) apartment Depending on the technology associated to the detector, the Type of 8 Photo-optical - Precise measure variables associated to its operation Adequate detector type Epistemic High Low 1 detector differ, e.g. size of particle picked up by the detector

Obscuration in the compartment at The detector is connected to the Activation Provided by manufacturer 9 14 % /m Precise measure or above which the detector power mains, has battery and is Epistemic High Medium 2 obscuration CSIRO obscuration activation activates working during fire Only passive or Safety independently 0 barriers redundant N/A included barriers are taken into account The available time for Criterion used since the 1960s and S. L. Poon, A Dynamic occupants to currently used in PDB guidelines ASET/RSET Approach to ASET/RSET 2 MAD - egress safely (IFEG 2005, CoPFSB 2011) and it N/A > 1 Assessment in Performance from the building was developed for its use with fire Based Design, 2014 is larger than the zone modelling. required one.

Estimated based on t=d/v and a hydraulic flow model for occupants SFPE Handbook of Fire 3 RSET Output s Time estimation movement. Distance is retrieved Protection Engineering, 5th Aleatory N/A from blueprints and movement edition, Ch. 59 speed is based on statistical analysis

The available time for occupants to evacuate is taken as the time for the ASET for the smoke height layer to reach a 30 Output s Time estimation Epistemic N/A building critical height (entry 49), as this is the main source of potential disruptions in the evacuation ASET at 60 compartment Output s Time estimation Epistemic N/A of origin ASET at 61 Output s Time estimation Epistemic N/A other levels

394

Appendix 4. Information registries for the case studies Case study 3 – Chemical storage warehouse – Objective: Business continuity & Property protection – Section 7.2

Key # Variable Value/Range Unit Source of information Assumptions Uncertainty SoK OS Importance assumption

A1: A spill of one IBC with ethanol occurs and an 1 Initial pool fire fuel 1 Ethanol - Facility Aleatory High Low 1 ignition source is readily available

A1: A spill of one IBC with ethanol occurs and an 2 Initial pool fire volume 1 1 m3 IBC volume Aleatory High Low 1 ignition source is readily available

A1: A spill of one IBC with ethanol occurs and an 3 Time to ignition 1 0 s Assumption Aleatory Low Low 7 ignition source is readily available

Fig 65.1 SFPE 5th Edition, Fire Dynamics and Forensic Analysis of Liquid Fuel Fires, Mealy. It is important to notice that a 0.75 mm pool thickness would be expected in perfect ethanol-air equilibrium, however the constraints caused by the drainage system plus the transient nature of the spill cannot allow ensuring this value. Lauri (https://www.aidic.it/icheap13/ A2: The drainage is not blocked and its borders increase 4 Spill depth 2 0.1-0.5 mm program/44lauri.pdf) assumed Aleatory Medium High 6 the expected pool thickness an ethanol pool thickness of 1.5 cm. Benfer (https://drum.lib.umd.edu/bitstr eam/handle/1903/10468/Benfer _umd_0117N_11327.pdf?seque nce=1&isAllowed=y) found thicknesses for denaturalized alcohol of less than 1 mm for spills of less than 450 ml, while reporting previous research in which hydrocarbon spills on concrete had a thickness ranging from 1 to 2 mm.

A2: The drainage is not blocked and its borders increase 5 Drainage present 2 No - Facility Epistemic High High 3 the expected pool thickness

395

Appendix 4. Information registries for the case studies

Key # Variable Value/Range Unit Source of information Assumptions Uncertainty SoK OS Importance assumption

6 Ambient temperature 25 C Facility Aleatory High Medium 2

Burn-out time (initial 7 Output s US-NUREG tool Epistemic High Medium 2 spill)

https://www.feraxo.com/en/ibc- 8 IBC dimensions 1 x 1.2 x 1.2 m3 Epistemic High High 3 containers.html

High density 9 IBC material polyethylene - RR564 Epistemic High Low 1 (HDPE)

IBC material specific kg/kJ http://dx.doi.org/10.1016/j.fires 10 1.7-2 Aleatory High Low 1 heat K af.2012.08.002

W/m http://dx.doi.org/10.1016/j.fires 11 IBC material condutivity 0.35-0.43 Aleatory High Low 1 K af.2012.08.002

IBC material melting 12 150 C SDS Qenos Aleatory High Low 1 temperature

IBC material ignition http://dx.doi.org/10.1016/j.fires 13 340 C Aleatory High Low 1 temperature af.2012.08.002

Heat fluxes less than 60 kW/m2 might damage the IBCs, IBC material degradation kW/ 14 8 60 RR564 but only this value is considered for rupture. A8: at least Aleatory Medium High 6 heat flux m2 1.5 minute for the IBC to be considered degraded.

Day (open doors) - A3: The fire occurs during normal operation hours when 15 Time of fire occurrence 3 - Facility Epistemic High High 3 Night (closed the front door of the warehouse is fully open doors) 396

Appendix 4. Information registries for the case studies Key # Variable Value/Range Unit Source of information Assumptions Uncertainty SoK OS Importance assumption Understand the Risks of Composite Intermediate Bulk Containers, M. Snyder http://www.aiche- A5: The secondary pool fire will have a volume 16 Time for IBC failure 5 2-3 min cep.com/cepmagazine/february Epistemic High High 3 equivalent to the contents of the degraded IBCs _2019/MobilePagedArticle.acti on?articleId=1462241#articleId 1462241 RR564

0, 3x50, A6, the extraction system is available and effective on 17 Extraction rate 6 m3/s Assumption Epistemic Medium High 6 3x100 demand

0.5 m spill depth is based on an effective spill volume 18 Secondary spill area 4 38 x 7.6 m2 Facility equivalent to 36 ICBs; A4: The leak rate of affected IBCs Epistemic High Medium 2 is assumed to be consistent with full rupture

Empirical mass burning Kg/ Gottuk - Babrauskas, SFPE Pool D > 3 m; A4: The leak rate of affected IBCs is 19 4 0.015-0.029 Aleatory High Medium 2 rate m2s Handbook assumed to be consistent with full rupture

20 Steel protection width 10 0.5-1.0 cm Specified by analyst A10: Insulation is applied and maintained properly. Aleatory High High 3

Dyrsdale indicates that the heat transfer coefficient varies between 5 and 25 W/m2K for free convection, with Buchanan indicating that hydrocarbon fires reach 50 W/m2K. This coefficient significantly defines the heat Convective heat transfer W/m Recommended for hydrocarbon transfer process from the fire to the structure and depends 21 25-50 Aleatory High Low 1 coefficient 2K fires [53] on a large array of variables such as the geometry and angle of the convective flow and its thermodynamic properties. As uncertainty on this variable is significant, it is explored systematically and then verified using the estimate for hc estimated by the FDS simulation.

Typical value for carbon steel 22 Resultant emissivity 0.8 Aleatory High Low 1 beams [53]

kg/m Typical value for carbon steel 23 Steel density 7850 Aleatory High Low 1 3 beams [53]

397

Appendix 4. Information registries for the case studies

Key # Variable Value/Range Unit Source of information Assumptions Uncertainty SoK OS Importance assumption

A7: The structure damage is defined by the loss of plastic Typical value for carbon steel 24 Steel yield strength 7 355 MPa moment capacity of the simple-supported roof beams and Aleatory High Low 1 beams [53] due to critical member temperature

https://www.spanman.net/Mem 25 Roof sheet weight 10 kg/m bers/Technical/Weight-Of- Facility Aleatory High Low 1

Building-Materials

https://www.stratco.com.au/site 26 Purlin weight (c-section) 5.66 kg/m assets/pdfs/steel_framing_purli Facility Aleatory High Low 1

ns_girts_design.pdf

A9: uniform temperature throughout the beam and an insulation temperature equal to the mid-difference between the gas and the steel temperature. The effect of Steel member 27 9 Output ºC U. Wickstrom. difussion in the steel member is considered under the Epistemic High High 3 temperature assumption of lumped capacity, which means that this process occurs infinitely fast, allowing for the member to heat up uniformly and instantaneously. A7: The structure damage is defined by the loss of plastic moment capacity of the simple-supported roof beams and due to critical member temperature. The loading is 28 Loading 7 Output kNm estimated as a function of the weight of the roof sheets, Epistemic High Low 1 purlins and the self-weight of the beam. No other interactions or loads are considered, which also exlcude wind effects such as potential suction (lift). A9: uniform temperature throughout the beam and an insulation temperature equal to the mid-difference between the gas and the steel temperature. This assumption is more conservative than that recommended 29 Insultation temperature 9, 10 Output ºC U. Wickstrom. by Buchanan or the Eurocode, where Ti is taken 2/3 Epistemic High Medium 2 between the insulation and the steel member.

A10: The insulation is assumed to be correctly installed and maintained.

Number of IBCs with 30 25, 50, 100 - Facility Epistemic Medium High 6 fuel

398

Appendix 4. Information registries for the case studies

399