Explanation in Information Systems A Design Rationale Approach Steven R. Haynes Submitted for the Degree of Doctor of Philosophy The London School of Economics and Political Science Department of Information Systems and Department of Social Psychology November 2001 UMI Number: U615601 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. Dissertation Publishing UMI U615601 Published by ProQuest LLC 2014. Copyright in the Dissertation held by the Author. Microform Edition © ProQuest LLC. All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code. ProQuest LLC 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, Ml 48106-1346 Theses F 7*1 77 7 / Abstract This dissertation investigates the relationship between the information systems (IS) development context, and the context in which such systems are used. Misunderstandings and ambiguities emerge in the space between these contexts and often result in construction of systems that fail to meet the requirements and expectations of their intended users. This study explores this problem using an approach derived from three largely separate and distinct fields: explanation facilities in information systems, theories of explanation, and design rationale. Explanation facilities are typically included in knowledge-based information systems, where their purpose is to provide system users with the underlying reasons for why the system reaches a particular conclusion or makes a particular recommendation. Prior research suggests that the presence of an explanation facility leads to increased acceptance of these conclusions and recommendations, therefore enhancing system usability. Theory of explanation is a field of study in which philosophers attempt to describe the unique nature of explanation and to identify criteria for explanation evaluation. Design rationale research is concerned with the capture, representation, and use of the deep domain and artefact knowledge that emerges from the design process. The design rationale approach goes beyond specification and suggests that to understand a system requires knowledge of the arguments that led to its realisation. This study proposes a model of IS explanation structure and content derived from formal theories of explanation with a method for obtaining this content based on design rationale. The study has four goals: to derive a theory of explanation specific to the domain of information systems; to examine this definition empirically through a study involving IS development and management professionals; to investigate in a case study whether the information needed to populate the explanation model can be captured using design rationale techniques; and construction of prototype software to deliver explanations per the proposed framework. 2 Acknowledgements I would first like to thank my thesis supervisors at the London School of Economics: Dr. Edgar Whitley from the Information Systems Department and Dr. Andy Wells from Social Psychology. Thank you for all your advice and support, and especially for your perseverance. Dr. Chrisanthi Avgerou, Dr. Tony Comford, and Dr. Jonathan Liebenau (Information Systems), and Dr. Bradley Franks and Dr. Patrick Humphreys (Social Psychology) all provided input and feedback at important points in the project. The assistance of many other members of the LSE Information Systems and Social Psychology faculties, as well that of fellow Ph.D. students in both departments was invaluable. My new colleagues at the Pennsylvania State University School of Information Sciences and Technology have provided advice, support, and a lot of constructive prodding over the last year. I look forward to many future projects with them. The confidence shown by my parents, my brothers and sisters, and by my extended family provided a special kind of motivation to get this work done, as always I am grateful to them. Last and best thanks to Anne and Helena for their love and understanding. 3 Table of Contents Abstract............................................................................................................................................2 Acknowledgements.......................................................................................................................... 3 Table of Contents............................................................................................................................. 4 1.1 Background............................................................................................................................... 9 1.1.1 Theoretical Framework. .....................................................................................................10 1.2 Problem Domain......................................................................................................................13 1.2.1 Research Questions.......................................................................................................... 15 1.3 Significance of die Study..........................................................................................................17 1.4 Methodology............................................................................................................................19 1.4.1 Design Science.................................................................................................................20 1.4.2 Method Selection..............................................................................................................21 1.5 Thesis Structure .......................................................................................................................23 Explaining Information Systems 26 2.1 Explanation in Expert Systems................................................................................................ 26 2.1.1 Why Provide Explanations?..............................................................................................27 2.1.2 Foundations of Expert System Explanation...................................................................... 30 2.1.3 Aspects of Expert System Explanations............................................................................ 38 2.2 Intelligent Tutoring Systems.................................................................................................... 47 2.3 User Modeling, Explanation, and HCI.....................................................................................50 2.3.1 User Modeling..................................................................................................................53 2.4 User Manuals and Online Help.................................................................................................58 2.4.1 The Systems Approach............................................................................ — ...................60 2.4.2 The Task-oriented Approach.............................................................................................61 2.4.3 The Minimalist Approach................................................................................................. 62 2.5 Discussion ............................................................................................................................... 65 Theories of Explanation 72 3.1 Explanations & System Design............................................................................................... 73 3.2 Why Look at Formal Theories? ............................................................................................... 74 3.3 Aspects of Explanation............................................................................................................76 3.3.1 Causality .......................................................................................................................... 76 3.3.2 Generalisation & Unification............................................................................................ 78 3.3.3 Explanation, Prediction, and Description.......................................................................... 79 3.4 Theories of Explanation........................................................................................................... 81 3.4.1 Deductive-Nomological Explanation................................................................................ 81 3.4.2 The Pragmatic Theory of Explanation...............................................................................85 3.4.3 Functional Explanation..................................................................................................... 86 3.4.4 Rational Choice and Explanations of Action..................................................................... 88 3.5 Theories of Explanation for IS................................................................................................. 89 3.5.1 Causal Relations in IS...................................................................................................... 90 3.5.2 Unification & Generalisation............................................................................................ 92 3.5.3 D-N Explanations of IS.................................................................................................... 94 3.5.4 Pragmatic Explanations of IS..........................................................................................