Final Copy 2020 01 23 Stand
Total Page:16
File Type:pdf, Size:1020Kb
This electronic thesis or dissertation has been downloaded from Explore Bristol Research, http://research-information.bristol.ac.uk Author: Standish, Michael Title: Enhancing Current Software Safety Assurance Practice to Increase System Mission Effectiveness General rights Access to the thesis is subject to the Creative Commons Attribution - NonCommercial-No Derivatives 4.0 International Public License. A copy of this may be found at https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode This license sets out your rights and the restrictions that apply to your access to the thesis so it is important you read this before proceeding. Take down policy Some pages of this thesis may have been removed for copyright restrictions prior to having it been deposited in Explore Bristol Research. However, if you have discovered material within the thesis that you consider to be unlawful e.g. breaches of copyright (either yours or that of a third party) or any other law, including but not limited to those relating to patent, trademark, confidentiality, data protection, obscenity, defamation, libel, then please contact [email protected] and include the following information in your message: •Your contact details •Bibliographic details for the item, including a URL •An outline nature of the complaint Your claim will be investigated and, where appropriate, the item in question will be removed from public view as soon as possible. This is a redacted version of the full dissertation, as agreed by the candidate, the supervisors and the Defence Science and Technology Laboratory (Dstl) as the industrial sponsor of this Engineering Doctorate studentship in the Faculty of Engineering. The redactions cover information that was deemed too sensitive to be published. The redactions have been kept to the minimum level necessary so that the dissertation can still show the research excellence of the candidate. DSTL/PUB121398. Content includes material subject to c Crown copyright (2020). This material is licensed under the terms of the Open Government Licence except where otherwise stated. To view this licence, visit http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: [email protected]. Disclaimer: This thesis is released for informational purposes only. The contents of this thesis should not be interpreted as representing the views of Dstl, nor should it be assumed that they reflect any current or future government policy. The information contained in this thesis cannot supersede any statutory or contractual requirements or liabilities and is offered without prejudice or commitment. [This page intentionally left blank] Enhancing Current Software Safety Assurance Practice to Increase System Mission Effectiveness Michael Standish A thesis submitted to the University of Bristol in accordance with the requirements of the degree of Engineering Doctorate in Systems in the Faculty of Engineering. Faculty of Engineering March 2020 Word Count: 86,648 DSTL/PUB121398. Content includes material subject to c Crown copyright (2020). This material is licensed under the terms of the Open Government Licence except where otherwise stated. To view this licence, visit http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: [email protected]. Disclaimer: This thesis is released for informational purposes only. The contents of this thesis should not be interpreted as representing the views of Dstl, nor should it be assumed that they reflect any current or future government policy. The information contained in this thesis cannot supersede any statutory or contractual requirements or liabilities and is offered without prejudice or commitment. [This page intentionally left blank] Abstract To deploy a safety-critical system it is imperative to have confidence in the system's under- pinning software. This is gained by performing software safety assurance. If there is not a sufficient level of confidence in the software then there is not a sufficient level of confidence in the system. Therefore, the system would not be able to be deployed in applications where safety is paramount. A traditional method to gain confidence in software is to develop it to a process centred on the life-cycle. This is subsequently judged against a set of predefined objectives and the judgement on the level of compliance to the objectives is taken to war- rant a degree of confidence in the software. However, if only certain types of evidence are accepted to demonstrate compliance, e.g. process-based evidence, then the solution space is reduced and some technical solutions potentially excluded. The aim of the thesis is to provide additional methods and success factors to potentially expand the scope of the current safety assurance processes. The research has explored how other domains, both safety and non-safety, judge confi- dence in evidence to make their informed decisions. An approach has been developed which allows attributes to be associated with any form of software safety assurance evidence. The chosen attributes allow the characteristics of the evidence to be understood as well as the relationship of the evidence to other forms of evidence. A Decision Support Framework (DSF) has been researched and implemented which is a usable end-to-end tool for decision makers to construct diverse evidence assurance arguments. The term `diversity' in this context is to have a variety of independent evidence which allows overall confidence to be gained for the software safety assurance. Judgements on the evidence characteristics are captured via Fuzzy Logic. The framework also allows the practical implications of adopting any evidence to be considered, such as cost and time. Tools have been devised to visualise confidence propagation across different collections of various evidence types. Efficient and effective evidence solutions can be identified by using the tool's Genetic Algorithm (GA) optimisation techniques. Diverse evidence can be challenging to measure when compared to traditional process- based approaches. However, the research has devised solutions to ameliorate such difficulties via outputs such as a stakeholder communication model. The DSF has been implemented on a number of defence case studies to understand the non-trivial way in which evidence attributes combine and how to construct a diverse evidence approach. This research has demonstrated how the use of diverse evidence can achieve an equivalent level of compliance to a full process-based approach and therefore that it can form part of a software safety assurance strategy. The research outputs have not previously been implemented within the software safety assurance domain prior to this research. DSTL/PUB121398. c Crown Copyright (2020). Page i of 328 [This page intentionally left blank] DSTL/PUB121398. c Crown Copyright (2020). Page ii of 328 Acknowledgements Dr John May, University of Bristol For the constructive discussions, insightful advice, and for guiding me through the academic writing process. Dr Mark Hadley, Dstl For the perceptive comments, continual motivation, and guidance during my studies and throughout the years. Dr Theo Tryfonas, University of Bristol For the valuable feedback and support. Dstl Colleagues For being supportive throughout my studies. Systems Centre Support Staff, University of Bristol For the pragmatic and timely help throughout the whole of my time undertaking the EngD. My Family For the endless encouragement and for being so tolerant of the amount of time I've been away. I owe you all much more of my time. DSTL/PUB121398. c Crown Copyright (2020). Page iii of 328 [This page intentionally left blank] DSTL/PUB121398. c Crown Copyright (2020). Page iv of 328 Author's Declaration I declare that the work in this dissertation was carried out in accordance with the require- ments of the University's Regulations and Code of Practice for Research Degree Programmes and that it has not been submitted for any other academic award. Except where indicated by specific reference in the text, the work is the candidate's own work. Work done in col- laboration with, or with the assistance of, others, is indicated as such. Any views expressed in the dissertation are those of the author. SIGNED: ............................................................. DATE:.......................... DSTL/PUB121398. c Crown Copyright (2020). Page v of 328 [This page intentionally left blank] DSTL/PUB121398. c Crown Copyright (2020). Page vi of 328 Research Publications/Conferences The research has resulted in a number of outputs. 1. External Conferences/Seminars. (a) Institution of Engineering and Technology (IET) System Safety and Cyber Se- curity (October 2014): Safety Case Development: A Process to Implement the Safety three-Layered Framework. Authors: M. Standish, H.J. Auld, P.R. Caseley, and M.J. Hadley. (b) IET System Safety and Cyber Security (October 2014): The Safety three-Layer Framework: A Case Study. Authors: M. Standish, H.J. Auld, P.R. Caseley, and M.J. Hadley. (c) Safety-Critical Systems Club Seminar: Use of Service History and Field Data - In Support of Safety Justifications (April 2016): Field Service Evidence: Prac- tical Examples and Potential Refinements to Support Qualification Arguments. Authors: M. Standish, M.J. Hadley, and E. Lennon. (d) High-Integrity Software Conference