Multi-Criteria Decision-Making in Software Production
Total Page：16
File Type：pdf, Size：1020Kb
Multi-Criteria Decision-Making in Software Production Siamak Farshidi SIKS Dissertation Serious No. 2020-35 The research reported in this thesis has been carried out under auspices of SIKS, the Dutch Research School for Information and Knowledge Systems. © 2020, Siamak Farshidi Multi-Criteria Decision-Making in Software Production ISBN: 978-90-393-7350-7 Multi-Criteria Decision-Making in Software Production Multi-criteria Besluitvorming in Softwareproductie (met een samenvatting in het Nederlands) Proefschrift ter verkrijging van de graad van doctor aan de Universiteit Utrecht op gezag van de rector magnificus, prof.dr. H.R.B.M. Kummeling, ingevolge het besluit van het college voor promoties in het openbaar te verdedigen op woensdag 2 december 2020 des morgens te 11.00 uur door Siamak Farshidi geboren op 8 augustus 1988, te Teheran, Iran Promotoren: Prof. dr. S. Brinkkemper Prof. dr. ir. H.A. Reijers Copromotor: Dr. S.R.L. Jansen This thesis was accomplished with financial support from NWO in the AMUSE project [project code 628.006.001]. Acknowledgments Undertaking this Ph.D. has been a truly life-changing experience for me, and it would not have been possible to do without the support and guidance that I received from many people. The entire list of people who encouraged me during my Ph.D. is pro- longed, whether within the research itself, forming a suitable environment, or even giving personal support. Probably, I will have forgotten to recall all names. First and foremost, I would like to thank Slinger Jansen. To me, Slinger was more than a supervisor. He was an inspiring and enthusiastic co-promoter who showed me the right course of action in all situations. His encouragements and optimism were indispensable to keep me motivated. Slinger taught me how to do research and then report findings correctly to scientific communities. I would like to express my sincere gratitude to Sjaak Brinkkemper. He actively sup- ported my research and encouraged me to write down high-quality scientific papers. With a few questions, he always helped me to focus on contributions and structure my findings. Most of his remarks will be with me for my entire career as a researcher. I would like to say a special thank you to Rolf de Jong. His support, guidance, and comprehensive insights into the software engineering field have made an inspiring experience for me. And special thanks to Michiel Overeem, whose support as an experienced software engineer and an eager researcher allowed my studies to go the extra mile (sorry for all the additional work, Michiel!). Doing research and writing papers in collaboration with others was a pleasant ex- perience as it allowed me to gain more insight and knowledge regarding what I was working on. I am profoundly grateful for my co-authors’ hard work, who have not been mentioned above, and their substantial contribution to uplift the studies pre- sented in this dissertation. Thanks, Sergio España, Jan Martijn van der Werf, Rolf de Jong, and Jacco Verkleij. I am also thankful to my motivated students who assisted me in the research projects, especially Elena Baninemeh, Andrey Krupskiy, Baharak Bakhtiari, and Mahdi Deldar. I am delighted to have worked with you, and I look forward to working with you again. My research would have been impossible without the feedback from domain ex- perts and case study participants from various software producing organizations that participated in the research projects’ evaluation phases in this dissertation. The work presented in this dissertation has been critically assessed and approved by an outstanding reading committee to whom I am more than grateful: Patricia Lago, Krzysztof Wnuk, Gabriele Keller, Bedir Tekinerdogan, and Diomidis Spinellis. I also vi S Acknowledgments want to thank the numerous anonymous reviewers of the journals and conferences. Often I received constructive feedback that helped me to improve the quality of my publications. To conclude, I cannot forget to thank my parents, beloved sister, and friends for all the unconditional love and support during my Ph.D. journey. Siamak Farshidi, November 2020 Contents Acknowledgments v Page 1 Introduction 1 1.1 Decision-Making in Software Production............2 1.2 Multi-Criteria Decision-Making Problems............3 1.3 Multi-Criteria Decision-Making Techniques...........4 1.4 Decision Support Systems.................5 1.5 MCDM Problems in Software Production............8 1.6 Problem investigation................. 13 1.7 Research Approach.................. 16 1.8 Theory Development in Design Science Research........ 21 1.9 The MCDM Framework................. 24 1.10 Dissertation Outline................. 28 COTS Component Selection Problems 2 Database Management Systems 33 2.1 Introduction.................... 35 2.2 Research Method................... 36 2.3 Related work.................... 36 2.4 Multi-Criteria Decision-Making.............. 38 2.5 DBMS Selection................... 40 2.6 Results and Analysis.................. 43 2.7 Discussion..................... 44 2.8 Conclusion..................... 45 3 Cloud Service Providers 47 3.1 Introduction.................... 49 3.2 Research Method................... 50 3.3 Related Work.................... 51 3.4 Multi-Criteria Decision-Making.............. 53 3.5 Cloud Service Provider Selection.............. 59 3.6 Results and Analysis.................. 63 3.7 Discussion..................... 65 ii S 3.8 Conclusion..................... 66 4 Blockchain Platforms 67 4.1 Introduction.................... 69 4.2 Research Approach.................. 70 4.3 Related Work.................... 72 4.4 Multi-Criteria Decision-Making for Blockchain Platform Selection... 76 4.5 Empirical Evidence: The Case Studies............ 84 4.6 Results and Analysis.................. 87 4.7 Discussion..................... 91 4.8 Conclusion..................... 92 Software development technology selection problems 5 Programming Language Ecosystems 95 5.1 Introduction.................... 97 5.2 Research Method................... 99 5.3 Multi-Criteria Decision-Making for Programming Language Selection.. 103 5.4 Empirical Evidence: The Case Studies............ 110 5.5 Analysis of the Results................. 121 5.6 Discussion..................... 123 5.7 Related Work.................... 129 5.8 Conclusion..................... 132 6 Model-Driven Development Platforms 133 6.1 Introduction.................... 135 6.2 Background.................... 136 6.3 Research Approach.................. 138 6.4 MCDM for MDD Platform Selection............. 143 6.5 Empirical Evidence: the Case Studies............ 149 6.6 Discussion..................... 157 6.7 Related Work.................... 161 6.8 Conclusion..................... 166 Decision-Making in Pattern-Driven Design 7 Capturing Software Architecture Knowledge 169 7.1 Introduction.................... 171 7.2 Background.................... 173 7.3 Systematic Literature Review............... 178 7.4 Practitioner Evaluation................. 196 7.5 Conclusion..................... 199 8 Decision Support for Pattern-Driven Architecture 201 8.1 Introduction.................... 203 8.2 Decision Support System................ 204 8.3 A Practical Running Example............... 206 S iii 8.4 Related Work.................... 211 8.5 Evaluation..................... 212 8.6 Conclusion..................... 214 9 Design Decisions in Pattern-Driven Architecture 215 9.1 Introduction.................... 217 9.2 Research method................... 218 9.3 Architectural pattern selection as an MCDM.......... 220 9.4 Empirical Evidence.................. 223 9.5 Discussion and Analysis................ 230 9.6 Related work.................... 231 9.7 Conclusions.................... 232 Concluding the Research 10 Conclusion 235 10.1 Contributions and observations.............. 236 10.2 Threats to Validity.................. 249 10.3 Reflections.................... 252 10.4 Limitations and Future Work.............. 257 Backmatter Bibliography 259 Summary 281 Samenvatting 283 Publication List 285 Curriculum Vitae 287 SIKS Dissertation Series 289 CHAPTER 1 Introduction 2 S Introduction 1.1 Decision-Making in Software Production Decision-making is a continuous problem-solving process in our daily lives. Some people consider decision-making as an art, while others view it as practice. Decisions can be personal or professional, but, in each case, alternative solutions typically have permanent consequences. In other words, decision-making is one of the underlying cognitive processes of human behaviors by which a preferred alternative or a course of action is selected from among a set of options according to particular criteria (Wang et al., 2004). The decisions we make have the potential to affect ourselves and others in the short and long term. Decision theories are widely applied in many disciplines. One example is software engineering (Rus et al., 2003), which has been defined as a continuous decision- making process (Fitzgerald & Stol, 2014). For instance, software producing organiza- tions need to decide whether using their internal development resources (in-house), buying Commercial off-the-shelf (COTS) components, do subcontracting (outsourc- ing), or whether to use open-source software (Badampudi et al., 2018). In literature, decision-making is typically defined as a process or a set of ordered activities con- cerning stages of problem identifying, data collection, defining alternatives, selecting a shortlist of alternatives as feasible solutions with the ranked preferences. Fitzgerald et