Joint IT Software Cost Forum

Cost of Developing Secure Software

Elaine Venson Brad Clark Barry Boehm

September 16, 2020

Center for Systems and Software Engineering 2 Outline

➢Background • Previous studies on secure software development costs • Proposed rating scale • Data collection and Initial results of a Wide-band Delphi • Next steps

Center for Systems and Software Engineering 3

CyBoK Software Security Known categories of programming errors resulting in security bugs, & techniques for avoiding these errors—both through coding practice and improved language design—and tools, techniques, and methods for detection of such errors in existing systems.

Secure Software Lifecycle The application of security software engineering techniques in the whole systems development lifecycle resulting in software that is secure by default.

Source: https://www.cybok.org/

Center for Systems and Software Engineering 4 Secure Software Development (Touchpoints)

https://www.slideshare.net/blackducksoftware/managing-open-source-in-application-security-and-software-development-lifecycle

Center for Systems and Software Engineering 5 Software Security

Engineering software that continues working under malicious attack [McGraw, 2006]

Many issues faced in computer security today are rooted in our approach to developing software and systems [Heitzenrater, 2016]

Software defects have security ramifications [McGraw, 2006]

Security is an emergent property of a software system. There is no single addition of a feature that can make a software secure [McGraw, 2006]

Center for Systems and Software Engineering 6 Security Objectives and Requirements

• Security objectives establish confidentiality, integrity and availability requirements • Functional • Security Features • Security Objectives -> Security Functional Components (CC) • Non-functional (a quality requirement) • In practice most software systems do not have explicit security objective/requirement • Software security if often about avoiding known classes of bugs that enable attacks

Center for Systems and Software Engineering 7 Outline

• Background ➢Previous studies on secure software development costs • Proposed rating scale • Data collection and Initial results of a Wide-band Delphi • Next steps

Center for Systems and Software Engineering 8 Sources of Cost (extracted from literature)

Source Papers Source Papers Perform Security Review 21 Perform Security Training 6 Apply Threat Modeling 18 Improve Development Process 5 Perform Security Testing 16 Perform Penetration Testing 5 Apply Security Requirements 11 Achieve Security Level 3 Apply Security Tooling 11 Document Technical Stack 3 Implement Countermeasures 9 Security Experts, Security Groups, 3 Security Master Fix Vulnerabilities 9 Track Vulnerabilities 3 Apply Secure Coding Standards 8 Functional Features 2 Apply Data Classifications Scheme 7 Hardening Procedures 2 Publish Operations Guide 7 Security by Design Paradigm 1

SWSec Practices Other Sources

Center for Systems and Software Engineering 9 Approaches to Estimating Costs of SWSec

Productivity Approach Additional Cost Range Source Validation COCOMO II security extension 0.94 (Low) Expert estimation Not validated 1.02 (Nominal) 1.27 (High) 1.86 1.43 (Very High) 1.75 (Extra High) COSECMO 0% (Nominal) Expert estimation with Not validated 20% to 80% (EAL 3 - High) two inputs provided by 50 to 200% (EAL 4 - Very High) a Commercial 31.25 125% to 500% (EAL 5 - Extra High) Company 313% to 1250% (EAL 6 - Super High) 781% to 3125% (EAL 7 - Ultra High) Weapon systems development 1.0 (Low or Nominal) Expert estimation and Cross cost model (COCOMO II based) 1.87 (High) 1.87 73 data points validation

Secure OS software cost model 1 (Nominal) Expert estimation Case study (COCOMO II based) 1.25 to 1.5 (High) 1.75 to 2.0 (Very High) 3.75 2.0 to 2.75 (Extra High) 3.0 to 3.75 (Super High) FPA security extension (from 0 to 5% increase in the function points size Practices from survey Not validated 1.05 General System Characteristics) of the project with developers

Center for Systems and Software Engineering 10 Practices Usage (from practitioners)

Apply Secure Coding Standards

Apply Security Tooling

Track Vulnerabilities

Apply Security Requirements

Perform Security Testing

Improve Development Process

Document Technical Stack

Perform Security Review

Apply Threat Modeling

Apply Data Classification Scheme

Perform Penetration Testing

Publish Operations Guide

Perform Security Training

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Daily Weekly Monthly

Center for Systems and Software Engineering 11 State of Art versus Practice

• Sources of cost • Practices found in literature are applied in industry, i.e., literature and survey are consistent • Estimation methods • Academia: COCOMO-based models • Industry: expert estimation • Added costs for security • For rating = High: • COCOMO-based models -> 20% to 80% added effort • Survey results for New Development projects -> 31.6% average, 25% median (risk level 4) • Level 1: low likelihood of attack and low impact of attack • Level 5: high likelihood and high impact of attack

Center for Systems and Software Engineering 12 Outline

• Background • Previous studies on secure software development costs ➢Proposed rating scale • Data collection and Initial results of a Wide-band Delphi • Next steps

Center for Systems and Software Engineering 13 Security Driver for COCOMO III • Based on software security practices • Found in literature • Usage confirmed in survey • Degrees of applying security practices based on existing studies • Building Security In Maturity Model (BSIMM) v10 • Published in 2019 with data observed form 122 firms • OWASP Software Assurance Maturity Model (SAMM) v1.5 • An open framework to help organizations formulate and implement a strategy for software security • Published in 2017 with contributions from industry • Rating scale will provide a measure of the level of secure software development

Center for Systems and Software Engineering 14 Practices Consolidation

Tasks, Practices & Activities Characteristics for SECU ratings Degrees Basic (list of banned functions), moderate, extensive Address common and Address all Coding to address all Ad-hoc secure Address common Apply Secure Coding Standards Standards coverage (proper use of APIs, memory sanitization, off-nominal vulnerabilities and known vulnerabilities coding vulnerabilities cryptography). vulnerabilities some weakness and weaknesses Basic testing (simple edge cases and boundary conditions), basic testing derived from requirements Moderate adversarial Rigorous adversarial Extensive adversarial and security features, derived from risk analysis with Ad-hoc security Basic adversarial testing driven with testing driven by Perform Security Testing Testing rigour and coverage testing driven by high medium coverage, comprehensive tests derived from testing testing security requirements security risks and security risks. abuse cases, complete set of tests derived from abuse and security features. attack patterns. cases. Casual use of Routine use of static Rigorous use of static Basic use of static Extensive use of static standard static analysis and analysis, penetration Basic tool configuration, customized with tailored analysis tool to analysis, penetration Apply Security Tooling Tools usage analysis tool to penetration testing testing and black-box rules, able to detect malicious code. identify security testing and black-box identify security tools to identify security testing tools defects. security testing tools. defects. security defects. with tailored rules. Ad-hoc basic code review for high-risk code, Ad-hoc security Systematic extensive Systematic rigorous systematic code review for high-risk code, systematic Basic security Moderate security Perform Security Review Review rigour and coverage features code security code and security code and comprehensive code review, systematic extensive features code review. code review. review. design review. design review. code review. Ad-hoc Systematic Extensive Track Vulnerabilities Critical vulnerabilities, high risk vulnerabilities, Regular vulnerabilities Rigorous vulnerabilities Resolution coverage vulnerabilities vulnerabilities tracking vulnerabilities tracking (development time) moderate risk vulnerabilities, low risk vulnerabilities. tracking and fixing. tracking and fixing. tracking and fixing. and fixing. and fixing. Extreme security Complex security Moderate security requirements derived Basic security requirements derived requirements derived from business Generic, based on business functionality, based on Ad-hoc security requirements derived from business Apply Security Requirements Requirements specification from business functionality, known risks, based on project specific threat model. requirements. from business functionality, functionality and compliance drivers and functionality. compliance drivers and compliance drivers. application/domain known risks. specific security risks. Systematic and Ocasional Regular Systematic frequent Systematic and Improve Software Development improvements improvements driven improvements driven improvements driven rigorous improvements Improvement frequency End of project, each release, each iteration. Process driven by security by vulnerabilities by vulnerabilities by organizational driven by security incidents. resolution. resolution. security knowledge science team. Practices + BSIMM + SAMM = base. Basic penetration Routine penetration testing addressing Frequent penetration testing (each release) Deep-dive analysis and Ad-hoc penetration common testing (each Perform Penetration Testing Penetration testing frequency Before shipping, for each release, periodic. addressing common maximal penetration testing. vulnerabilities (for increment) based on and critical testing. sanity check before project artifacts. vulnerabilities. shipping). Exceptional technical Detailed technical Moderate technical stack documentation stack documentation Basic (identify and keep third-part components up to stack documentation with third-part Control security of thid-part Basic technical stack with third-part Document Technical Stack date on security patches), moderate (assess third-part None with explicit third-part components identified components documentation. components identified components risk). components and formally rigorously and assessed based on identification. assessed by a security security risks. science team. Based on generic attacker profiles, with specific Apply threat modeling Apply threat modeling Apply threat modeling attackers information, using organization's top N Ad-hoc threat using new attack Apply Threat Modeling Attack information None with generic attacker with specific attackers possible attacks, based on new attack methods modeling. methods developed profiles. information. developed by a science team. with a science team. Simple classification (low risk data), moderate Simple data Moderate data Complete data Maximal data Apply Data Classification Scheme Data classification scheme classification (medium risk data), complex None classification scheme. classification scheme. classification scheme. classification scheme. classification (high risk data). Material specific to Security on-demand company history is training and used in training. General awareness, role-specific, advanced role- advanced-role specific Progression on security Security awareness Vendors and Perform Security Training Training level and coverage specific, customized with company data/knowledge, None training are training curriculum is training is performed. outwourced workers security certification. performed. Security rewarded. are trained. Annual centralized reporting training required for knowledge is used. everyone. Moderate operations Thorough operations Very thorough Regular operations guide with critical guide with with operations guide with Basic (critical security information for deployment), guide with critical security instructions detailed security with maximal security Publish Operations Guide Guiding coverage moderate (procedures for typical application alerts), None security instructions and procedures for instructions and, instructions and, thorough (formal operational security guides). for deployment. typical application procedures for all procedures for all alerts. application alerts. application alerts.

Center for Systems and Software Engineering 15 SECU Rating Scale

Practices Practices Practices Consolidation Grouping Summarization

Characteristics for SECU Tasks, Practices & Activities Degrees Characteristics for ratings Task Practices Low Nominal High Very High Extra High SECU ratings Basic (list of banned functions), moderate, Address common Address all Coding to address all Apply Secure Coding Ad-hoc secure Address common Standards coverage extensive (proper use of APIs, memory and off-nominal vulnerabilities and known vulnerabilities Extreme Standards coding vulnerabilities sanitization, cryptography). vulnerabilities some weakness and weaknesses Complex security Moderate security requirements Basic testing (simple edge cases and boundary security Moderate Basic security requirements derived from conditions), basic testing derived from Extensive Rigorous adversarial adversarial testing requirements requirements and security features, derived from Ad-hoc security Basic adversarial adversarial testing testing driven by Ad-hoc requirements derived from business Perform Security Testing Testing rigour and coverage driven with security risk analysis with medium coverage, testing testing driven by high security risks and Apply Security Requirements derived from requirements and security derived from business functionality, comprehensive tests derived from abuse cases, security risks. attack patterns. Requirements specification business security features. requirements. business functionality, compliance complete set of tests derived from abuse cases. functionality functionality. compliance drivers and and compliance Extensive use of drivers and application/do Casual use of Routine use of static Rigorous use of static Basic use of static static analysis, drivers. standard static analysis and analysis, penetration known risks. main specific Basic tool configuration, customized with tailored analysis tool to penetration testing Apply Security Tooling Tools usage analysis tool to penetration testing testing and black-box rules, able to detect malicious code. identify security and black-box security risks. identify security tools to identify security testing tools defects. security testing defects. security defects. with tailored rules. Build secure-by- Build container- tools. Build additional design based Ad-hoc basic code review for high-risk code, security middleware for approaches for Ad-hoc security Basic security Systematic extensive Systematic rigorous systematic code review for high-risk code, Moderate security Perform Security Review Review rigour and coverage features code features code security code and security code and Requirements and features security security systematic comprehensive code review, code review. Build basic review. review. design review. design review. Design (authentication, features features systematic extensive code review. security role (authentication, (authentication, Ad-hoc Application Type Stand alone Network Connected Access to private data Hospitals, Banking Loss of life Critical vulnerabilities, high risk vulnerabilities, Regular Systematic Extensive Rigorous features Track Vulnerabilities vulnerabilities Security Features Scope and rigour None. management, role role General Characteristic None/Ad-hoc Basic Moderate Extensive Rigorous Resolution coverage moderate risk vulnerabilities, low risk vulnerabilities vulnerabilities vulnerabilities vulnerabilities (development time) tracking and (authentication, vulnerabilities. tracking and fixing. tracking and fixing. tracking and fixing. tracking and fixing. key management, management, Complex security requirements, fixing. Extreme security requirements, role Moderate security requirements advanced secure-by-design managemente, key key container-based approaches for Extreme security management). Requirements and Design Security requirements considered Basic security requirements and and additional security features security features middleware advanced security features requirements audit/log, managemente, managemente, Complex security Summary part of reliability. features. Basic threat modeling. (audit/log, cryptography). development. Threat modeling Moderate security derived from development. Rigorous threat Basic security requirements cryptography, audit/log, audit/log, Moderate threat modeling. with specific attackers requirements business modeling. Generic, based on business functionality, based requirements derived from Ad-hoc security derived from functionality, protocols). cryptography, cryptography, information. Apply Security Requirements Requirements specification on known risks, based on project specific threat derived from business requirements. business compliance drivers model. business functionality, protocols). protocols). Effort Multiplier 1 1.1 1.25 1.45 1.7 functionality and and functionality. compliance drivers compliance drivers. application/domain Apply threat Very extensive list of and known risks. Apply threat specific security Apply threat modeling using Extensive list of vulnerabilities vulnerabilities and weaknesses Known and critical vulnerabilities risks. modeling with Basic vulnerabilities applicable to and weaknesses applicable to the applicable to the software will be Apply Threat Ad-hoc threat modeling with new attack applicable to the software will be Attack information None. specific the software will be prevented software will be prevented with prevented with secure coding Systematic and Modeling modeling. generic attacker methods No secure coding and no use of prevented with secure coding Regular Systematic frequent Systematic and attackers Coding and Tools Summary with secure coding standards or secure coding standards or standards or detected through Ocasional static analysis tool. standards or detected through improvements improvements improvements rigorous profiles. developed with detected through basic use of detected through extensive use of rigorous use of static analysis, Improve Software improvements information. routine use of static analysis and Improvement frequency End of project, each release, each iteration. driven by driven by driven by improvements driven Development Process driven by security a science team. static analysis tools. static analysis, black-box and penetration testing and black-box vulnerabilities vulnerabilities organizational by security science penetration testing tool. incidents. penetration testing tools. security testing tools with tailored resolution. resolution. security knowledge team. Coding to Address Address all rules. base. Address address all Apply Secure Coding Ad-hoc secure common and vulnerabilities Standards coverage common known 1 1.1 1.2 1.35 1.6 Basic penetration Routine penetration Frequent Standards coding off-nominal and some Extensive adversarial testing and testing addressing vulnerabilities vulnerabilities Basic adversarial testing and Rigorous adversarial testing and Ad-hoc testing (each penetration testing Deep-dive analysis Moderate adversarial testing and security design/code review. Penetration testing common vulnerabilities weakness Perform Penetration Testing Before shipping, for each release, periodic. penetration release) addressing (each increment) and maximal and weaknesses Verification and Validation No security testing, review or security code review. Basic security design/code review. frequency vulnerabilities (for security code review. Routine Frequent and specialized testing. common and critical based on project penetration testing. Summary penetration testing. penetration testing before Exhaustive deep-dive analysis sanity check before Rigorous use of penetration testing each release. penetration testing each vulnerabilities. artifacts. Routine use of Extensive use of shipping. penetration testing. shipping). Coding Casual use of static analysis, increment. Basic use of static analysis static analysis, standard static penetration 1 1.2 1.4 1.65 2 static analysis and penetration Exceptional technical analysis tool to testing and Extreme security requirements Detailed technical stack documentation Moderate technical Apply Security Tooling Tools usage tool to identify penetration testing and stack documentation with third-part identify black-box Complex security requirements and threat modeling. Container- stack Moderate security-related Basic (identify and keep third-part components Basic technical with third-part components security testing tools to black-box and threat modeling. Advanced based approaches to advanced Control security of thid-part documentation with security security testing activities for requirements, Document Technical Stack up to date on security patches), moderate (assess None stack components identified and secure-by-design security security features. Exhaustive components explicit third-part defects. identify security security testing third-part components risk). documentation. identified and formally rigorously defects. tools with Basic security-related activities design, coding, and testing. components features. Extensive adversarial adversarial testing, security assessed based on assessed by a defects. tools. for requirements, coding, and Additional security features identification. tailored rules. Security-related activities for testing and security design/code design/code review, and deep- security risks. security science Security Activities One Line testing. Typical security functional (audit/log, cryptography). requirements, coding, and testing review. Security assessment of dive analysis penetration testing. team. Moderate Summary features. Regular use of static Identification and controlled Rigorous considered part of reliability. third-part components and timely Third-part components rigorously adversarial Extensive analysis tools to detect security update of third-part components' Apply threat adversarial security patches updates. assessed and updated by a Based on generic attacker profiles, with specific Apply threat Apply threat defects. security patches. Routine use of modeling using new Basic testing driven adversarial Thorough use of statics analysis, security science team. Maximal attackers information, using organization's top N Ad-hoc threat modeling with modeling with Perform Security Testing rigour and Ad-hoc testing driven static analysis and penetration Apply Threat Modeling Attack information None attack methods possible attacks, based on new attack methods modeling. generic attacker specific attackers adversarial with security testing driven black-box, and penetration testing use of tools for static analysis, developed with a Testing coverage security testing by security risks testing tools. developed by a science team. profiles. information. testing requirements by high security tools. penetration testing, and black-box science team. and attack and security risks. security testing. Simple classification (low risk data), moderate Simple data Moderate data Complete data Maximal data patterns. Apply Data Classification Data classification scheme classification (medium risk data), complex None classification classification classification classification features. Scheme classification (high risk data). scheme. scheme. scheme. scheme. Systematic Systematic Ad-hoc Basic security Moderate extensive rigorous Perform Security Review rigour and security Security on-demand Material specific to features code security code security code security code training and company history is Verification and Review coverage features code advanced-role used in training. Progression on General awareness, role-specific, advanced role- Security awareness review. review. and design and design specific training are Vendors and security training Validation review. Perform Security Training Training level and coverage specific, customized with company None training is performed. Security outwourced workers curriculum is review. review. data/knowledge, security certification. performed. centralized are trained. Annual rewarded. Basic Routine reporting training required for penetration Frequent knowledge is used. everyone. penetration testing penetration Deep-dive testing (each Ad-hoc addressing testing (each analysis and Moderate Thorough operations Very thorough Perform Penetration Penetration testing release) operations guide penetration common increment) maximal Basic (critical security information for Regular operations guide with with operations guide with critical security Testing frequency addressing deployment), moderate (procedures for typical guide with critical detailed security with with maximal testing. vulnerabilities based on penetration Publish Operations Guide Guiding coverage None instructions and application alerts), thorough (formal operational security instructions instructions and, security instructions common and procedures for (for sanity project testing. security guides). for deployment. procedures for all and, procedures for typical application critical application alerts. all application alerts. check before artifacts. alerts. vulnerabilities. shipping).

Center for Systems and Software Engineering 16 Summarized Practices General Characteristic Basic Moderate Extensive Rigorous Characteristics High Very High Extra High Ultra High Moderate security Complex security requirements, Extreme security requirements, Basic security requirements and advanced secure-by-design container-based approaches for Security Requirements and requirements and additional security security features middleware advanced security features Design Summary features. Basic threat features (audit/log, development. Threat modeling development. Rigorous threat modeling. cryptography). Moderate with specific attackers' modeling. threat modeling. information.

Basic vulnerabilities Very extensive list of vulnerabilities Known and critical applicable to the Extensive list of vulnerabilities and and weaknesses applicable to the vulnerabilities applicable software will be weaknesses applicable to the software will be prevented with to the software will be Secure Coding and Security prevented with secure software will be prevented with secure coding standards and/or prevented with secure Tools Summary coding standards and/or secure coding standards and/or detected through rigorous use of coding standards and/or detected through basic detected through extensive use of static analysis and black-box security detected through routine use of static analysis static analysis and black-box tools. testing tools with tailored rules. use of static analysis tools. tools. Employ formal methods in coding.

Rigorous adversarial testing and Moderate adversarial Extensive adversarial testing and Basic adversarial testing security design/code review. testing and security code security design/code review. and security code Exhaustive deep-dive analysis review. Routine Frequent and specialized Security Verification and review. Basic penetration penetration testing. Use of formal penetration testing. penetration testing. Security V&V Validation Summary testing. Security V&V verification and custom developed Security V&V activities activities conducted by an activities conducted V&V tools. Security V&V activities conducted by an independent group at the within the project. conducted by an outside certified independent group. organizational level. company.

Center for Systems and Software Engineering 17 One-Line Summary

None/Ad-hoc Basic Moderate Extensive Rigorous Nominal High Very High Extra High Ultra High Moderate security- Complex security Extreme security requirements and related activities for requirements and threat threat modeling. requirements, design, modeling. Container-based approaches to Basic security- coding, and testing. Advanced secure-by-design advanced security features. Exhaustive related activities for Additional security security features. adversarial testing, security requirements, features (audit/log, Extensive adversarial testing design/code review, deep-dive analysis Security-related coding, and testing. cryptography). and security design/code penetration testing, and use of formal activities for Typical security Identification and review. Security assessment methods throughout the lifecycle. requirements, functional features. controlled update of of third-part components Third-party components rigorously coding, and Regular use of static third-part components' and timely security patches assessed and updated by a security testing analysis tools to security patches. updates. science team. Maximal use of tools for nonexistent. detect security Routine use of static Thorough use of statics static analysis, penetration testing, and defects within the analysis and analysis, black-box, and black-box security testing. Use of project. penetration testing penetration testing tools. formal verification and custom tools. Security V&V V&V activities conducted by developed V&V tools. Security V&V activities conducted by an independent group at activities conducted by an outside an independent group. the organization level. certified company.

Center for Systems and Software Engineering 18 Outline

• Background • Previous studies on secure software development costs • Proposed rating scale ➢Data collection and Initial results of a Wide-band Delphi • Next steps

Center for Systems and Software Engineering 1921 Data Collection

Security experts estimates for the security parameter Wideband Delphi

Estimation experts estimates for the security parameter

Industry Projects’Projects’ Data Manual Data Collection Form

OSS Projects’Projects’ Data Automated Data Collection

Projects’Projects’ Data Survey

CenterCenter f orfor S Systemsystems a andnd SSoftwareoftware EngEngineeringineering 20 Delphi Results

• Productivity Range (PR) • PR is the degree of influence a model parameter has on the estimated effort. • The Delphi session asked participants to rate the PR for security • The result is an average of 2.17 (table below) • A range of 2.17 means the parameter ratings from Nominal to Ultra High can change the effort estimate up to 117%, i.e., it costs 117% more to implement extreme security requirements and threat modeling Security Parameter Average Standard Deviation Coefficient of Variation Characteristic (AVG) (SD) (SD/AVG) Security Requirements and Design 1.18 0.045 4% Security Coding and Tools 1.18 0.130 11% Security Verification & Validation 1.56 0.152 10% Total Productivity Range 2.17

Center for Systems and Software Engineering 21 Converting Delphi to a Rating Scale

• The Productivity Range is used to derive individual quantitative values, EMi, for each rating level based on exponential growth • Exponential growth is used in all COCOMO III parameters

Exponential Growth Security Rating Scale 2.5 푖 2.170 퐸푀푖 = (1 + 푟) 2 1.788 푟 = 푟푎푛푔푒(1/푛) − 1 1.473 1.5 1.214 • EM is the effort multiplier for level i i 1.0001 • i is the security level (Nominal is 0) 1 • r is the growth rate • range is the range of the effort MULTIPLIERCOST multiplier 0.5 • n is the index of the highest level (i=4)

0 Nominal High Very High Extra High Ultra High SECURITY LEVEL

Center for Systems and Software Engineering 22 Outline

• Background • Previous studies on secure software development costs • Proposed rating scale • Data collection and Initial results of a Wide-band Delphi ➢Next steps

Center for Systems and Software Engineering 23 Next Steps

• Collect project and OSS data • Calibrate the COCOMO III model • Functional size measures as well as traditional lines of code • Domain classification of software applications • All the model parameters • Validate the security rating scale and its impact on effort

Center for Systems and Software Engineering Joint IT Software Cost Forum

Thank you!

Elaine Venson [email protected] Brad Clark [email protected]

September 16, 2020

Center for Systems and Software Engineering 25 References

• Chad Heitzenrater, Rainer Bohme, Andrew Simpson, 2016. The Days Before Zero Day: Investment Models for Secure Software Engineering, in: Proceedings of the 15th Workshop on the Economics of Information Security (WEIS). Presented at the Workshop on the Economics of Information Security. • McGraw, G., 2006. Software Security: Building Security In, 1 edition. ed. Addison-Wesley Professional, Upper Saddle River, NJ. • OWASP SAMM Project, 2017. Software Assurance Maturity Model (SAMM): A guide to building security into software development - v1.5 (No. Version 1.5). • Rashid, A., Chivers, H., Danezis, G., Lupu, E., Martin, A., 2019. The Cyber Security Body of Knowledge. • Sammy Migues, John Steven, Mike Ware, 2019. Building Security in Maturity Model (BSIMM) - Version 10 (No. 10). Synopsys Software Integrity Group. • Tierney, J., Boswell, T., 2017. Common Criteria: Origins and Overview, in: Mayes, K., Markantonakis, K. (Eds.), Smart Cards, Tokens, Security and Applications. Springer International Publishing, Cham, pp. 193–216. https://doi.org/10.1007/978-3-319-50500-8_8 • Venson, E., Guo, X., Yan, Z., Boehm, B., 2019. Costing Secure Software Development: A Systematic Mapping Study, in: Proceedings of the 14th International Conference on Availability, Reliability and Security, ARES ’19. ACM, New York, NY, USA, pp. 9:1–9:11. https://doi.org/10.1145/3339252.3339263 • Venson, E., Alfayez, R., Marília M. F., G., Rejane M. C., F., Boehm, B., 2019. The Impact of Software Security Practices on Development Effort: An Initial Survey, in: 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). Presented at the 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp. 1–12. https://doi.org/10.1109/ESEM.2019.8870153

Center for Systems and Software Engineering