Making Intelligent Systems Team Players: Case Studies and Design Jssues
Total Page:16
File Type:pdf, Size:1020Kb
i f - ) .'- ] NASA Technical Memorandum 104738 ! / Making Intelligent Systems Team Players: Case Studies and Design Jssues Volume 1: Human-Computer Interaction Design Jane T. Malin Debra L. Schreckenghost David D. Woods, Scott S. Potter, Leila Johannesen, Matthew Holloway Kenneth D. Forbus (" ,.",.,_ , .; :_ . C_Ct ].Z_ -'. > ./:_ ? September 1991 N/ A vJ _._.1 r _dJ NASA Technical Memorandum 104738 Making Intelligent Systems Team Players: Case Studies and Design Issues Volume 1" Human-Computer Interaction Design Jane T. Malin Lyndon B. Johnson Space Center Houston, Texas Debra L. Schreckenghost The MITRE Corporation Houston, Texas David D. Woods, Scott S. Potter, Leila Johannesen, Matthew Holloway Ohio State University Columbus, Ohio Kenneth D. Forbus Northwestern University Evanston, Illinois National Aeronautics and Space Administration Lyndon B. Johnson Space Center Houston, Texas September 1991 Request for Comments We are interested in making this document useful to the reader. We encourage comments about this document that can help us improve it in future releases. Please send comments to: [email protected] schreck@ aio.jsc.nasa.gov Abstract Initial results are reported from a multi-year, interdisciplinary effort to provide guidance and assistance for designers of intelligent systems and their user interfaces. The objective is to achieve more effective human-computer interaction (HCI) for systems with real-time fault management capabilities (i.e., process monitoring and control during anomalous situations). Intelligent fault management systems within the National Aeronautics and Space Administration (NASA) have been evaluated for insight into the design of systems with complex HCI. Preliminary results include (1) a description of real-time fault management in aerospace domains, (2) recommendations and examples for improving intelligent system design and user interface design, (3) identification of issues requiring further research, and (4) recommendations for a development methodology integrating HCI design into intelligent system design. Keywords: human-computer interaction, user interface, intelligent system, design guidance, development methodology, real-time fault management °.. 111 PRECEDING PAGE BLANK NOT FII, M_ r) Acknowledgements We would like to thank all the NASA and contractor personnel who took time to demonstrate and explain their systems to us. Their patience and willingness to share their problems and ideas fostered an ideal environment in which to undertake this work. We also appreciate the time expended by these personnel to review the case study reports for accuracy. We would like to extend particular thanks to Louis Lollar, David Weeks, and Bryan Walls at NASA Marshall Space Flight Center for their cooperation and encouragement in developing a workshop to present the problems and issues summarized in this paper. We are also grateful to Christine Kelly, Leslie Ambrose, and Kathryn Cooksey from The MITRE Corporation for their review of portions of this report. V PRECEDING P,,_,E BLANK NOT F'ILMED jJ Executive Summary This report documents the initial results from a multi-year, interdisciplinary effort to provide guidance and assistance for designers of intelligent systems and their user interfaces. The objective is to achieve more effective human-computer interaction (HCI) for systems with real- time fault management capabilities (i.e., process monitoring and control during anomalous situations). Intelligent fault management systems within NASA have been evaluated for insight into the design of systems with complex HCI. Fifteen intelligent systems were selected from a variety of aerospace programs, including Space Shuttle, Space Station, unmanned spacecraft, and military and advanced aircrafL Information was collected by interviewing intelligent system developers and users, observing intelligent system demonstrations, and referencing available intelligent system documentation. A review of pertinent literature was also conducted in parallel to the case study. This report represents work in progress. Preliminary results of this study include (1) a description of real-time fault management in aerospace domains, described in section 3, (2) recommendations and examples for improving intelligent system design and user interface design and issues requiring further research, provided in sections 4 and 5, and (3) recommendations for a development methodology integrating HCI design into intelligent system design, discussed in section 6. These preliminary results will be amplified and formalized in the report Making Intelligent Systems Team Players: Design Assistance and Guidelines to be published in Fiscal Year 1993. The results of the case study are documented in a two volume report. This document is Volume 1, which summarizes the recommendations and issues for HCI design guidance and development methodology. Volume 2 (Matin et al., 1991) provides the detailed analysis of each application in the ease study. Real-Time Fault Management with Intelligent Systems Fault management has traditionally been performed by teams of human operators (i.e., onboard crew members and ground flight controllers). The introduction of intelligent systems into real- time flight support has altered this traditional mode of operation. The team now consists of human and computer agents. Activities are distributed between these agents. This distribution of activities to software agents and the resulting joint human-computer tasking is a key change in operations. The availability of multiple agents requires allocation of tasks to specific agents and introduces the need for dynamic task allocation for situations where an agent's capabilities to perform a task axe altered in real time (e.g., intelligent system fails and the human must take over its task). Issues include managing interruptions in time-critical activities and determining appropriate task assignments. Inappropriate task assignments can adversely affect the operator, complicating performance of tasks and removing opportunities for on-the-job training and development of expertise. Inherent in the availability of multiple agents is the need to coordinate the activities of these agents, particularly when they are working jointly on a task. These coordination activities are independent of and in addition to the fault management activities. When multiple team members are working simultaneously on a fault management task, information from independent activities must be exchanged and related to system behavior. Such a joint effort may be described as collaboration between agents of the fault management team. The goal of such collaboration is to allow the operator to understand the intelligent system's point of view by sharing information and providing visibility into intelligent system reasoning. Retrospective explanation (in the form of static rationale and rule trace) is not sufficient for such collaboration and information sharing. vii PBI_CEDIi_u P.:_.;t_ t_l_Nr{ NOI' FILM]_D Effective control of the intelligent system also requires that the operator understand intelligent system processing. This includes providing visibility into the intelligent system reasoning and clearly identifying what levels of automation are available, when they axe used, and how task are allocated for each level. Intelligent systems for flight operations support add both enhanced capability and an additional source of error. The operator is responsible to make the final decisions in fault management situations and thus must be able to manage and direct the intelligent system, including the ability to compensate for erroneous behavior. Erroneous behavior can be due to failures of the intelligent software (e.g., inaccurate knowledge base, inadequate performance) or due to erroneous or unavailable input information. Typically, errors in the intelligent system result in shut down of the intelligent system. Error correction is performed off-line, after the operational support period. The intelligent system can be designed, however, to permit on-line error compensation by operator intervention into intelligent system processing. Methods for intervention into the intelligent system reasoning process include (1) altering information used by the intelligent system, (2) redirecting the reasoning process, (3) restarting the intelligent system, and (4) selectively overriding portions of the intelligent system processing. For situations when redirection is ineffectual, the operator can take over all intelligent system task responsibilities. To retain the capability to manage the monitored process (i.e., the domain process that the team is responsible to manage) and make decisions in failure situations, the operator must maintain an awareness of the overall state and behavior of the monitored process, even when significant portions of the fault management tasks are conducted by the intelligent system. The operator must be able to extract the important information from the barrage of alarms that typically accompany an anomaly and interpret and pdoritize these alarms by severity of impact and time- criticality. The operator must also have access to diagnostic information critical to understanding situations affecting the monitored process, including assessment of mission impacts, evaluation of consequences and alternatives, monitoring and execution of procedures, and assessment of functional capability remaining after a failure. Since not all fault management situations can be anticipated, the intelligent system