Test performance and security meas- urements in software development Mikko Sara Master’s Thesis June 2019 School of Technology, Communication and Transport Information and Communication Technology Degree Programme in Cyber Security Description Author(s) Type of publication Date Sara, Mikko Master’s thesis June 2019 Language of publication: English Number of pages Permission for web publi- 91 cation: Yes Title of publication Test performance and security measurements in software development Degree programme Information and Communication Technology, Master’s programme in Cyber Security Supervisor(s) Saarisilta, Juha; Saharinen, Karo Assigned by Finnish Defence Forces Logistics Command, Tikkanen, Raimo Abstract The main goal of the thesis was to study cyber security implementation in different soft- ware development frameworks with two different research methods. The first method was a survey aimed at industry supplemented by the second method, literary research answer- ing the research question. The thesis was assigned by the Finnish Defense Forces Logistics Command, which utilized the results of the work in the software procurements bidding competitions. The research project proceeded by first discovering the sources, followed by a literature research based on the sources. The content of the survey aimed at industry was designed against a certain ICT architecture to support the findings of the literature research. The survey was conducted anonymously for domestic and foreign industries. The aim of the study was to build a situational awareness of how the industry sees secure software development and find alternative models for the secure software development framework. The research achieved the goals set by the Finnish Defence Forces, and the research clearly showed the facts received from the industry and the facts produced by the literature research. In conclusion, there are different methods for performing safe software development; however, there is no standardized model or standard, which would be applicable to every software development framework and every company’s quality system and processes. Keywords/tags Software development, security tools, security testing, secure code development Miscellaneous Research permit AO20582 Kuvailulehti Tekijä(t) Julkaisun laji Päivämäärä Sara, Mikko Opinnäytetyö, ylempi AMK Kesäkuu 2019 Julkaisun kieli: Englanti Sivumäärä Verkkojulkaisulupa 91 myönnetty: Kyllä Työn nimi Test performance and security measurements in software development Tutkinto-ohjelma Information and Communication Technology, Master’s programme in Cyber Security Työn ohjaaja(t) Juha Saarisilta; Karo Saharinen Toimeksiantaja(t) Puolustusvoimien logistiikkalaitos, Raimo Tikkanen Tiivistelmä Opinnäytetyön päätavoitteena oli tutkia tietoturvan toteutumista eri ohjelmistokehitys- menetelmissä kahdella eri tutkimusmenetelmällä. Teollisuudelle suunnatulla kyselytutki- muksella tavoiteltiin vastausta tutkimuskysymykseen, jota täydennettiin kirjallisuustutki- muksella. Opinnäytetyön toimeksiantajana toimi Puolustusvoimien logistiikkalaitos, joka hyödynsi työn tuloksia ohjelmistohankintojen kilpailutuksissa. Työn toteutus eteni eri lähteiden kartoittamisella, jonka jälkeen toteutettiin kirjallisuustut- kimus perustuen lähteisiin. Teollisuudelle suunnatun kyselytutkimuksen sisältö suunnitel- tiin tietynlaista ICT -arkkitehtuuria vastaan sekä myös tukemaan kirjallisuustutkimuksessa tehtyjä havaintoja. Kyselytutkimus suunnattiin anonyymina sekä kotimaiselle että ulko- maalaiselle teollisuudelle. Tutkimuksen tavoitteena oli rakentaa tilannekuva siitä, miten teollisuus näkee ohjelmisto- kehityksen tietoturvan sekä kartoittaa vaihtoehtoisia malleja turvalliseen ohjelmistokehi- tykseen. Tutkimuksilla saavutettiin työn toimeksiantajan asettamat tavoitteet, joista oli selkeästi nähtävissä sekä teollisuuden näkemykset että kirjallisuustutkimuksen tuottamat faktat. Johtopäätöksenä voitiin todeta, että ohjelmistojen turvalliseen kehittämiseen on olemassa erilaisia menetelmiä, mutta sellaista vakioitua mallia tai standardia aiheeseen ei ole ole- massa, mikä olisi sovellettavissa jokaiseen ohjelmistokehitysmenetelmään, projektiin sekä eri yrityksissä toimiviin laatujärjestelmiin että prosesseihin. Avainsanat Ohjelmistokehitys, tietoturvatestaus, ohjelmiston turvallinen kehittäminen Miscellaneous Tutkimuslupa AO20582 1 Contents 1 Introduction ................................................................................................... 8 1.1 About cyber security ................................................................................... 8 1.2 Cyber threats to commercial enterprises.................................................. 10 1.3 Cyber threats in international military domain ......................................... 10 1.4 Cyber security in Finland ........................................................................... 12 1.5 The Finnish Defence Forces Logistics Command ...................................... 14 2 Examples of software development frameworks ........................................... 16 2.1 Scrum ......................................................................................................... 16 2.2 Extreme programming .............................................................................. 17 2.3 Waterfall model ......................................................................................... 18 3 Secure code development example tools ...................................................... 20 3.1 Wireshark .................................................................................................. 20 3.2 Nessus ........................................................................................................ 20 3.3 Nmap ......................................................................................................... 20 3.4 Threat modelling ....................................................................................... 21 3.5 Coding standards ....................................................................................... 22 4 Theory summary........................................................................................... 24 4.1 Cornerstones of Theory ............................................................................. 24 4.2 Linking theory to the research .................................................................. 24 5 Research methods ........................................................................................ 25 5.1 Main research questions ........................................................................... 25 5.2 Research principles applied ....................................................................... 25 5.2.1 Dividing the research ............................................................................ 25 5.2.2 Research methods and ethical questions ............................................. 26 5.2.3 Literature research ............................................................................... 28 2 5.2.4 Survey research and open feedback .................................................... 29 6 Literature research ....................................................................................... 31 6.1 Framework, processes and testing ............................................................ 31 6.2 Security testing tools ................................................................................. 49 7 Survey research ............................................................................................ 54 7.1 Multiple-choice questions ......................................................................... 54 7.2 Free feedback ............................................................................................ 54 8 Research results and analysis ........................................................................ 55 8.1 Literature research .................................................................................... 55 8.1.1 Answers to the research questions ...................................................... 55 8.1.2 Scoping the software module entities .................................................. 61 8.2 Survey research ......................................................................................... 62 8.2.1 Anwers to the research question ......................................................... 62 8.2.2 Open feedback ...................................................................................... 65 8.3 Assessment of the research results........................................................... 66 8.3.1 Literature research ............................................................................... 66 8.3.2 Survey research .................................................................................... 67 8.4 Connecting survey results to the theory and literature reseach .............. 71 8.5 Answers to the research questions ........................................................... 72 9 Conclusions .................................................................................................. 74 9.1 Literature research .................................................................................... 74 9.2 Survey research ......................................................................................... 77 9.2.1 Critical analysis of the survey questions ............................................... 78 3 10 Discussion ...................................................................................................