EVALUATION AND SELECTION OF CASE TOOLS: A METHODOLOGY AND A CASE STUDY A THESIS SUBMITTED TO THE GRADUATE SCHOOL OF INFORMATICS OF MIDDLE EAST TECHNICAL UNIVERSITY BY KORAY OKŞAR IN PARTIAL FULLFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN THE DEPARTMENT OF INFORMATION SYSTEMS FEBRUARY 2010 Approval of the Graduate School of Informatics _______________ Prof. Dr. Nazife Baykal Director I certify that this thesis satisfies all the requirements as a thesis for the degree of Master of Science. _________________ Assist. Prof. Dr. Tuğba Taşkaya Temizel Head of Department This is to certify that we have read this thesis and that in our opinion it is fully adequate, in scope and quality, as a thesis for the degree of Master of Science. _________________ Assist. Prof. Dr. Altan Koçyiğit Supervisor Examining Committee Members Prof. Dr. Semih Bilgen (METU, EEE) _____________________ Assist. Prof. Dr. Altan Koçyiğit (METU, II) _____________________ Assoc. Prof. Dr. Onur Demirörs (METU, II) _____________________ Assist. Prof. Dr. Pekin Erhan Eren (METU, II) _____________________ Assist. Prof. Dr.Sevgi Özkan (METU, II) _____________________ I hereby declare that all information in this document has been obtained and presented in accordance with academic rules and ethical conduct. I also declare that, as required by these rules and conduct, I have fully cited and referenced all material and results that are not original to this work. Name, Last name : Koray OKŞAR Signature : ________________ iii ABSTRACT EVALUATION AND SELECTION OF CASE TOOLS: A METHODOLOGY AND A CASE STUDY Okşar, Koray M. S., Department of Information Systems Supervisor: Assist. Prof. Dr. Altan Koçyiğit February 2010, 224 pages Today’s Computer Aided Software Engineering (CASE) technology covers nearly all activities in software development ranging from requirement analysis to deployment. Organizations are evaluating CASE tool solutions to automate or ease their processes. While reducing human errors, these tools also increase control, visibility and auditability of the processes. However, to achieve these benefits, the right tool or tools should be selected for usage in the intended processes. This is not an easy task when the vast number of tools in the market is considered. Failure to select the right tool may impede project’s progress besides causing economic loss. In this thesis study, a methodology is proposed for CASE tool evaluation and selection among various candidates and the points that separate this work from similar studies in the literature are explained. Moreover, the methodology is performed on a case study. Keywords: CASE tool evaluation, CASE tool selection iv ÖZ CASE ARAÇLARININ DEĞERLENDİRİLMESİ VE SEÇİMİ: BİR METODOLOJİ VE BİR ÖRNEK İNCELEMESİ Okşar, Koray Yüksek Lisans, Bilişim Sistemleri Bölümü Tez yöneticisi: Assist. Prof. Dr. Altan Koçyiğit Şubat 2010, 224 sayfa Günümüz Bilgisayar Destekli Yazılım Mühendisliği (CASE) teknolojisi yazılım geliştirmede gereksinim analizinden dağıtıma kadar neredeyse her aktiviteyi kapsamaktadır. Organizasyonlar süreçlerini otomatikleştirmek ya da kolaylaştırmak amacıyla CASE aracı çözümlerini değerlendirmektedir. Bu araçlar insan kaynaklı hataları azaltırken süreçlerin kontrolünü, görünürlüğünü ve denetlenebilirliğini arttırmaktadır. Ne varki bu yararlara erişmek için, düşünülen süreçlerde kullanılmak üzere doğru araç ya da araçlar seçilmelidir. Piyasadaki araç sayısının fazlalığı göz önüne alınırsa bu kolay bir iş değildir. Doğru aracı şeçmede başarısızlık projenin gidişatına sekte vurmanın yanı sıra ekonomik kayba da sebep olur. Bu tez çalışmasında, birçok aday arasından CASE aracı değerlendirmesi ve seçimi için bir yöntemler dizisi sunulmuş ve bu çalışmayı literatürdeki benzer çalışmalardan ayıran noktalar açıklanmıştır. Ayrıca, yöntemler dizisi bir örnek olay çalışması üzerinde uygulanmıştır. Anahtar kelimeler: CASE aracı değerlendirmesi, CASE aracı seçimi v ACKNOWLEDGMENTS It is my privilege to express my sincere gratitude to my supervisor Assistant Professor Altan Koçyiğit for his guidance, stimulating ideas, criticism, encouragement and insight throughout the completion of this thesis. I must thank my managers Suat Gümüşlüol, Özlem Değer, Seyhan Halisipek at OYAK Headquarters because of their generosity at giving me permission to leave for thesis meetings and their support for my case study. I should also thank my colleague Alper Aslan for his valuable comments and to my other colleagues because of their kind tolerance to my absences. I am also grateful to my beloved one Emel Sarıkaya whose endless support encouraged me to proceed, to my sister Dilek and to everyone who have aided in one way or another. vi TABLE OF CONTENTS ABSTRACT ................................................................................................................................................ IV ÖZ ................................................................................................................................................................ V ACKNOWLEDGMENTS ........................................................................................................................... VI TABLE OF CONTENTS ........................................................................................................................... VII LIST OF TABLES ....................................................................................................................................... X LIST OF FIGURES .................................................................................................................................. XIV LIST OF ABBREVIATIONS ................................................................................................................... XV CHAPTER 1. INTRODUCTION ...................................................................................................................... 1 2. LITERATURE SURVEY ........................................................................................................... 5 2.1 SOFTWARE EVALUATION AND SELECTION IN GENERAL ......................................... 5 2.2 CASE SOFTWARE ............................................................................................................. 8 2.3 CASE SOFTWARE EVALUATION AND SELECTION .................................................... 11 2.4 CONTINUOUS INTEGRATION ...................................................................................... 15 3. PROPOSED METHODOLOGY FOR CASE SOFTWARE EVALUATION AND SELECTION ................................................................................................................................. 20 3.1 THE INITIATION ACTIVITY ................................................................................................ 25 3.1.1 Step1: Rationale Determination ......................................................................... 26 3.1.2 Step2: Commitment Determination .................................................................... 28 3.1.3 Step3: Methodological Constraints Determination ............................................ 30 3.1.4 Step4: Evaluation Team Formation: .................................................................. 31 3.2 THE HIGH LEVEL EVALUATION CRITERIA DEFINITION ACTIVITY .................................... 32 3.2.1 Step1: Constraints Determination ...................................................................... 33 3.2.2 Step2: Existing Toolset Examination (optional) ................................................. 40 3.2.3 Step3: Tool Area to Search Determination ........................................................ 41 3.2.4 Step4: High Level Criteria Determination ......................................................... 43 vii 3.3 THE PRESCREENING ACTIVITY ......................................................................................... 46 3.3.1 Step1: Tool Information Gathering for Each Tool Area .................................... 48 3.3.2 Step2: Criteria Matching for Each Tool Area: .................................................. 49 3.4 THE LOW LEVEL EVALUATION CRITERIA DEFINITION ACTIVITY ..................................... 52 3.4.1 Step1: Organizational Requirement Analysis for Each Tool Area ..................... 54 3.4.2 Step2: External Analysis for Each Tool Area ..................................................... 56 3.4.3 Step3: Criteria Formation .................................................................................. 57 3.4.4 Step4: Criteria Prioritization and Categorization ............................................. 59 3.5 THE SCREENING ACTIVITY ............................................................................................... 61 3.5.1 Step1: Tool Information Gathering for Each Tool Area .................................... 62 3.5.2 Step2: Candidate CASE Tool Selection for Each Tool Area .............................. 63 3.6 THE EVALUATION AND COMPARISON ACTIVITY ............................................................... 63 3.6.1 Step1: Assessment Method Determination for Each Criterion ........................... 64 3.6.2 Step 2: Assessment and Comparison for Each Criterion .................................. 69 3.7 THE RANKING AND SELECTION ACTIVITY ........................................................................ 72 3.7.1 Step1: AHP Application ..................................................................................... 73 3.7.2 Step2: Final Selection .......................................................................................