Doktora Tezi
Total Page:16
File Type:pdf, Size:1020Kb
A METRICS-BASED APPROACH TO THE TESTING PROCESS AND TESTABILITY OF OBJECT-ORIENTED SOFTWARE SYSTEMS A THESIS SUBMITTED TO THE GRADUATE SCHOOL OF INFORMATICS OF THE MIDDLE EAST TECHNICAL UNIVERSITY BY TOLGA YURGA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY IN THE DEPARTMENT OF INFORMATION SYSTEMS FEBRUARY 2009 Approval of the Graduate School of Informatics Prof. Dr. Nazife BAYKAL Director I certify that this thesis satisfies all the requirements as a thesis for the degree of Doctor of Philosophy. Prof. Dr. Yasemin YARDIMCI Head of Department This is to certify that we have read this thesis and that in our opinion it is fully adequate, in scope and quality, as a thesis for the degree of Doctor of Philosophy. Prof. Dr. Semih BİLGEN Assoc. Prof. Ali H. DOĞRU Co-Supervisor Supervisor Examining Committee Members Dr. Ali ARİFOĞLU (METU, II) Assoc. Prof. Ali H. DOĞRU (METU, CENG) Prof. Dr. Semih BİLGEN (METU, EEE) Assist. Prof. Dr. Aysu Betin CAN (METU, II) Dr. Sadık EŞMELİOĞLU (BİLGİ GRUBU) I hereby declare that all information in this document has been obtained and presented in accordance with academic rules and ethical conduct. I also declare that, as required by these rules and conduct, I have fully cited and referenced all material and results that are not original to this wok. Name, Last name : Tolga YURGA Signature : _________________ iii ABSTRACT A METRICS-BASED APPROACH TO THE TESTING PROCESS AND TESTABILITY OF OBJECT-ORIENTED SOFTWARE SYSTEMS Yurga, Tolga Ph.D., Department of Information Systems Supervisor: Assoc. Prof. Dr. Ali Hikmet DOĞRU Co-Supervisor: Prof. Dr. Semih BİLGEN February 2009, 195 pages This dissertation investigates the factors that affect testability and testing cost of object- oriented software systems. Developing a software program which eases the testing process by increasing testability is crucial. Also, to assess whether or not the testing effort and cost consumed or planned is adequate or not is another critical matter this dissertation aims to answer by composing a new way to evaluate the links between software design parameters iv and testing effort via source-based metrics. An automated metric plug-in is used as the primary tool for obtaining the metric measurements. Our study is based on the investigation of many open-source projects written in Java to achieve our goals. By the help of the statistical evaluation of project data, we both propose a new model to assess testing effort and testability, and find significant relations and associations between software design and testing effort and testability of object-oriented software systems via source-based metrics. Keywords: Software, Testability, Testing, Metric, Testcase v ÖZ NESNE-YÖNELİMLİ YAZILIM SİSTEMLERİNİN TEST SÜRECİNE VE TESTEDİLEBİLİRLİĞİNE METRİK TABANLI BİR YAKLAŞIM Yurga, Tolga Doktora, Bilişim Sistemleri Bölumu Tez Yoneticisi: Doç. Dr. Ali Hikmet DOĞRU Ortak Tez Yoneticisi: Prof. Dr. Semih BİLGEN Şubat 2009, 195 sayfa Bu tez, nesne-yönelimli yazılım sistemlerinin testedilebilirliklerini ve test maliyetini etkileyen faktörleri araştırmayı hedeflemektedir. Testedilebilirliği arttırarak test sürecini kolaylaştıran bir yazılım geliştirmek çok önemlidir. Ayrıca, kaynak kod temelli metrikler yoluyla, tasarım parametreleri ve test gayreti arasındaki bağları değerlendirip yeni bir model oluşturarak, harcanması planlanan ya da harcanmış olan test gayretinin uygunluğunun değerlendirilmesi, bu tezin diğer bir önemli hedefini oluşturmaktadır. Metrik ölçümlemeleri vi için ana araç olarak otomatikleştirilmiş bir metrik eklentisi kullanılmıştır. Çalışmamız, hedeflerimize ulaşmak için Java dili ile yazılmış birçok açık-kaynak kodlu projenin detaylı araştırmasına dayanmaktadır. Proje verilerinin istatistiksel olarak değerlendirilmesi sayesinde, kaynak-kod bazlı metrikler üzerinden, hem test gayretini ve testedilebilirliğini değerlendirebilmek için bir yeni model öneriyoruz, hem de yazılım tasarımı ve test gayreti ve yazılımın testedilebilirliği arasında kayda değer ilişki ve bağlantıları ortaya koyuyoruz. Anahtar Kelimeler: Yazılım, testedilebilirlik, test süreci, metrik, test vakası vii To Didem viii ACKNOWLEDGMENTS I express sincere appreciation to Assoc. Prof. Dr. Ali Doğru for his guidance and insight throughout the research. I am grateful to the other supervisory committee members, Prof. Dr. Semih Bilgen and Dr. Sadık Eşmelioğlu, for their supports, suggestions and comments. To my wife, Didem and to both of my parents, I present sincere thanks for their unshakable faith in me and their willingness to endure with me since the beginning of my doctoral program and dissertation. I could not reach the end without them. ix TABLE OF CONTENTS ABSTRACT ...........................................................................................................................iv ÖZ .......................................................................................................................................vi DEDICATION ...................................................................................................................... viii ACKNOWLEDGMENTS ......................................................................................................... ix TABLE OF CONTENTS ........................................................................................................... x LIST OF TABLES .................................................................................................................. xiii LIST OF FIGURES ................................................................................................................ xvi LIST OF ACRONYMS ......................................................................................................... xviii CHAPTER 1. INTRODUCTION ............................................................................................................... 1 1.1 Problem Statement .............................................................................................. 1 1.2 Purpose of This Study ........................................................................................... 2 1.3 Organization of the Dissertation ........................................................................... 2 2. TESTABILITY & TESTING EFFORT ....................................................................................... 4 2.1 Software Testing Process ..................................................................................... 4 2.2 Testability ............................................................................................................ 9 2.3 Design for Testability.......................................................................................... 11 2.4 The Testability Fish-Bone ................................................................................... 11 2.4.1 Degree of Validity and Testing Effort .......................................................... 12 2.4.2 Representation ........................................................................................... 13 2.4.3 Implementation.......................................................................................... 14 2.4.4 Built-in Test ................................................................................................ 15 2.4.5 Test Suite ................................................................................................... 15 2.4.6 Test Tools ................................................................................................... 16 2.4.7 Process Capability....................................................................................... 16 x 3. OBJECT-ORIENTED DESIGN AND QUALITY MODELS ........................................................ 18 3.1 Object-Oriented Programming ........................................................................... 18 3.2 Quality Factors & Design Parameters ................................................................. 19 3.3 Quality Models for Software Design Assessment ................................................ 21 3.3.1 The Factor-Criteria-Metrics Model .............................................................. 21 3.3.2 ISO 9126 Model .......................................................................................... 22 3.3.3 REBOOT Models ......................................................................................... 23 3.3.4 Dromey’s Quality Model ............................................................................. 24 3.3.5 QMOOD (Quality Model for Object-Oriented Design) ................................. 25 3.3.6 MQMOOD (Metrics Based Quality Model for OO Design) ........................... 27 3.4 Testability and Quality Models ........................................................................... 28 4. SOFTWARE METRICS...................................................................................................... 30 4.1 Definition of Software Metrics ........................................................................... 30 4.2 Importance of Software Metrics ......................................................................... 30 4.3 Method-Level Metrics ........................................................................................ 31 4.4 Class-Level Metrics ............................................................................................. 33 4.5 Package-Level Metrics ........................................................................................ 39 4.6 Project-Level