A CONCEPTUAL MODEL AND PROTOTYPE FOR A CASE-BASED ADAPTIVE ANALYST SUPPORT SYSTEM by WILLIAM H. GWINN, B.A., M.S. A DISSERTATION IN BUSINESS ADMINISTRATION Submitted to the Graduate Faculty of Texas Tech University in Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY Approved Accepted May, 1999 Copyright 1999, William H. Gwinn ACKNOWLEDGMENTS 1 would like to take this opportunity to thank my wife Amie G. Gwinn for her encouragement and support throughout this endeavor I express my gratitude to Dr. Surya B. Yadav, my dissertation chair for his guidance and extreme patience over the long haul. I also thank my dissertation committee. Dr. Ralph R. Bravoco, Dr. Glenn J. Browne, and Dr. Rich L. Sorenson for their pertinent comments and recommendations and Dr. Nirup Menon and Mr. Harold Webb T.A. for their comments on improving the clarity of the screens which comprise the user interface for the prototype system. A special thanks to Ms. Barbi Dickensheet, Thesis Coordinator of the Texas Tech Graduate School for her preliminary and final dissertation review and helpful tips along the way 11 TABLE OF CONTENTS ACKNOWLEDGMENTS ii ABSTRACT viii LIST OF TABLES ix LIST OF FIGURES x CHAPTER 1 INTRODUCTION 1 Background 1 Requirement Definition 1 Requirement Determination Process o Importance of Fact Gathering and Analysis Activities 3 The Nature of Fact Gathering Activities .1 Why an Analyst Needs Support 4 Nature of Fact Gathering and Analysis Support 5 Problem Statement 6 Research Questions 7 Research Objectives 7 Research Deliverables 8 Significance of the Research 9 Structure of the Dissertation 10 II. LITERATURE REVIEW 12 Introduction 12 Works on Requirement Determination Processes and Analyst Activities 13 G. B.Davis 13 Yadav 15 Alan M. Davis 15 iii Byrd, Cossick, and Zmud 16 Works on Expert Analyst Assistants 17 Systems Analysis Expert Aide (SYS-AIDE) 17 Analyst Assistant 18 Analyst Assist 19 Expert Modeling Support System 20 Dalal/Yadav EMSS Modification 20 Works on Case-Based Learning 23 CYRUS 24 PLEXUS 25 CABSYDD 25 Summary 25 HI RESEARCH METHODOLOGY 27 Introduction 27 Formulating the Problem 29 Constructing the Knowledge Level Principles 29 Constructing the Symbol Level Principles 30 Developing the Prototype System 31 Evaluating and Validating the System 32 A Validation Framework 34 Summary 3 8 IV. CONCEPTUAL DEVELOPMENT OF A CASE-BASED ADAPTIVE ANALYST SLT^PORT SYSTEM 39 Introduction 39 Fact Gathering Activities 39 System Behavior 47 Knowledge Level Concepts and Principles ^6 Knowledge Level Concepts 58 Knowledge Level Principles 62 iv Requirement Specification 63 Requirement Set 63 A Conceptual Model of a CAASS 64 The Dialog Management Subsystem (DMS) 65 The Fact Gathering Coordinator Subsystems (FGCS) 66 Organization Base Management Subsystem (OBMS) 66 The Case Base Management Subsystem (CBMS) 66 The Domain Knowledge Base Management Subsystem (DKBMS) 67 The Basic Knowledge Base Management Subsystem (BKBMS) 67 The Resource Base Management Subsystem (RBMS) 67 Summary 68 \' SYMBOL LEVEL ARCHITECTURE FOR A CASE-BASED ADAPTIVE ANALYST SUPPORT SYSTEM (CA.ASS) 69 Introduction 69 Symbol Level Concepts and Principles 69 Symbol Level Concepts 69 Symbol Level Principles 73 Symbol Level Architecture of CAASS 74 Case-based Reasoning and Learning 76 Case Knowledge Frame Base 1~! Knowledge Frame Base 78 Structure Design 80 Summary 87 VI. DESIGN .AND IMPLEMENTATION OF THE PROTOTYPE 88 Introduction 88 Logical Flow Design 88 Implementation Languages 97 Visual BASIC 5.0 M7 The Haley Enterprise Products ^;8 CAASS Prototype Components 99 User Interface Module 100 System Control Module 115 Knowledge Frame Base 116 Case Knowledge Frame Base 116 Executables 116 Summary 116 VII. VERIFICATION AND VALIDATION 117 Introduction 117 Conceptual Model Verification 117 Prototype Verification 119 Verifying the Input Capability and User Interface 119 Verifying the Knowledge Base 120 Verifying Case-based Learning 120 Verifying Case-based Retrieval 121 Verifying Template Modification 121 Conceptual Model Validation 121 Prototype Validation 121 Summary 122 Vin. EXPERIMENTAL DESIGN 123 Introduction 123 Experiment Design 123 Group Selection 124 Testing Instrument 125 Experiment Layout and Model 126 Hypothesis Testing 128 User Survey 129 Summary 132 IX. RESULTS AND CONCLUSIONS VI Introduction 133 Experimental Results 133 Analysis of Variance (ANOVA) 133 User Survey Results 136 Conclusions 139 Summary 139 X RESEARCH CONTRLBUTIONS, LIMITATIONS. AND FUTURE RESEARCH 142 Introduction 142 Research Deliverables 142 Research Contributions 143 Research Limitations 146 Future Work 147 REFERENCES 148 APPENDIX. ORGANIZATIONAL CASE FACT LISTS FOR MEDIA TECHNOLOGY SERVICES AND HOMEOU^^TRS OF AMERICA 152 \ii ABSTRACT ¥c\\ researchers have addressed the question of how information system requirements should be derived. The rapidly changing needs of increasingly complex organizations are pressuring the analyst to rapidly produce information requirements This means the analyst needs the capability to rapidly acquire, organize and analyze organizational facts from which information requirements are derived. This research concerns the development of an adaptive analyst support system to assist the analyst with the gathering and managing of organizational facts. A check-list for analyst fact gathering activities is suggested. The knowledge needs, conceptual model, and architecture for a case-based adaptive analyst support system are developed and a prototype support system is implemented. This tool provides a means for an analyst to recall facts and information requirements from previously analyzed organizations rapidlv and adapt the recalled information to current organizational needs. The prototype demonstrates the feasibility of a case-based approach to an adaptive support system. The implemented prototype's adaptability is demonstrated b\' the growth of its case-base with repeated use. The primary contribution of this research is to provide the MIS communitv with a new analysis tool concept. The research describes the toofs ability to gather, organize store, recall, and adapt organizational facts to a current situation rapidly and efficientK This enhances the analyst's ability to rapidly produce information requirements Vlll LIST OF TABLES 4.1 Fact Gathering Activities 41 4.2 System Functions 49 4 3 System Functions and Supported Activities 57 5.1 Mapping Conceptual Model to Symbol Level Components 74 9 1 Responses to CAASS End-User Survey 137 9 2 Question 12 Responses 138 IX LIST OF FIGURES 2.1 Dalai and Yadav EMSS 21 3 1 A Validation Framework for CAASS 35 4 1 Knowledge Level Concepts 60 4.2 CAASS Conceptual Model 65 5.1 Frame and Slots 71 5.2 Frame Network 72 5.3 Symbol Level Architecture for CAASS 75 5 4 Structure Chart Symbols 81 5.5 CAASS Structure Chart 82 5.6 Begin New Case 83 5.7 Adapt Check-List 84 5.8 Resume Existing Case 85 5.9 Learn Case, Perform Case Matciiing, and Analyze Case Facts 86 6.1 CAASS System Control Module Logical Diagram 89 6.2 System Control Module Case Comparison Process 90 6.3 System Control Module Inference Engine Processes 91 6.4 System Control Module Case-Based Reasoner and General Utility Processes 92 6.5 User Interface Module Check-List and Print Check-List Processes 93 6.6 User Interface Module Display Info and Begin New Case Processes *^)4 6.7 User Interface Module Build Current Case, Resume Case, and Display Current Case Processes 95 X 6.8 User Interface Module Modify Case Slots and Display Match Processes 96 6.9 Screen Hierarchy 101 6.10 Opening Screen 102 6.11 Information Screen 103 6.12 Begin New Case Screen 105 6.13 Check-List Screen 106 6.14 Parameters and Objectives Screen 108 6.15 Strategy and Properties Screen 109 6.16 Identify'Entities Screen 110 6.17 Functions and Processes Screen 111 6.18 Case Comparison Screen 114 6.19 Case Comparison Screen with Case Suggested 114 7.1 A VaUdation Framework 118 8.1 Experiment Layout 127 8.2 CAASS User Survey Questionnaire 131 XI CHAPTER I INTRODUCTION Background One of the most difficult tasks in developing information systems is the determination of an organization's information requirements. Requirement specifications should deal with three basic questions (Yadav and Chand, 1989): 1. What should requirements be'^ 2. How should requirements be derived? 3. How should requirements be stated? The first and third questions address the actual content of each requirement and the form used to state the content of each requirement. The second question is concerned with the processes used to determine the contents of the requirements. Although a number of professionals have discussed the first and third questions, few researchers have addressed the second question (Yadav and Chand, 1989). Requirement Definition Webster's Dictionary defines requirement as "something required; something demanded or needed" (Webster, 1997, p. 1141). IEEE standard 729 (1983) defines requirement as: "(1) a condition or capability needed by a user to solve a problem or achieve an objective; (2) a condition or capability that must be met or possessed by a system ... to satisfy a contract, standard, specification, or other formally imposed document" (p. 29). This research concerns the process by which the organization's information requirements are derived. Requirement Determination Process The requirements determination phase of system analysis begins with the recognition of an organizational problem that requires a solution and ends when there is a complete description of the external behavior of the system needed to solve the organization's problem. The process seeks to describe what the system should do to support the organization's goals, not how it will do it (Davis, 1990). The requirements determination process is complex because it spans multiple organizational domains. The organization's executive, middle manager and the user technician domains are intimately involved with the requirements determination process (Yadav and Chand, 1989). The process used to complete the requirements determination phase is composed of three steps (Dalai and Yadav, 1989). The first step is to obtain the raw requirements from the organization.