NIST Aiiios Dflfiibfi pecial Publication 500-237 PUBLICATIONS I' ^— Information Technology: Usability Engineering: Industry- Government Collaboration for System Effectiveness and Efficiency SYMPOSIUM TRANSCRIPTION Laura L. Downey and Sharon J. Laskowski, Editors U.S. Department of Commerce Technology Administration QQ National Institute of Standards and Technology 100 U57 Nisr NO. 500-237 1997 I rhe National Institute of Standards and Technology was established in 1988 by Congress to "assist industry in the development of technology . needed to improve product quality, to modernize manufacturing processes, to ensure product reliability . and to facilitate rapid commercialization ... of products based on new scientific discoveries." NIST, originally founded as the National Bureau of Standards in 1901, works to strengthen U.S. industry's competitiveness; advance science and engineering; and improve public health, safety, and the environment. One of the agency's basic functions is to develop, maintain, and retain custody of the national standards of measurement, and provide the means and methods for comparing standards used in science, engineering, manufacturing, commerce, industry, and education with the standards adopted or recognized by the Federal Government. As an agency of the U.S. Commerce Department's Technology Administration, NIST conducts basic and applied research in the physical sciences and engineering, and develops measurement techniques, test methods, standards, and related services. The Institute does generic and precompetitive work on new and advanced technologies. NIST's research facilities are located at Gaithersburg, MD 20899, and at Boulder, CO 80303. Major technical operating units and their principal activities are listed below. For more information contact the Publications and Program Inquiries Desk, 301-975-3058. Office of the Director Physics Laboratory • National Quality Program • Electron and Optical Physics • International and Academic Affairs • Atomic Physics • Optical Technology Technology Services • Ionizing Radiation • Standards Services • Time and Frequency' • Technology Partnerships • Quantum Physics' • Measurement Services • Technology Innovation Materials Science and Engineering • Information Services Laboratory • Intelligent Processing of Materials Advanced Technology Program • Ceramics • Economic Assessment • Materials Reliability' • Information Technology and Applications • Polymers • Chemical and Biomedical Technology • Metallurgy • Materials and Manufacturing Technology • NIST Center for Neutron Research • Electronics and Photonics Technology Manufacturing Engineering Manufacturing Extension Partnership Laboratory Program • Precision Engineering • Regional Programs • Automated Production Technology • National Programs • Intelligent Systems • Program Development • Fabrication Technology • Manufacturing Systems Integration Electronics and Electrical Engineering Laboratory Building and Fire Research • Microelectronics Laboratory • Law Enforcement Standards • Structures • Electricity • Building Materials • Semiconductor Electronics • Building Environment • Electromagnetic Fields' • Fire Safety Engineering • Electromagnetic Technology' • Fire Science • Optoelectronics' Information Technology Laboratory Chemical Science and Technology • Mathematical and Computational Sciences^ Laboratory • Advanced Network Technologies • Biotechnology • Computer Security • Physical and Chemical Properties^ • Information Access and User Interfaces • Analytical Chemistry • High Performance Systems and Services • Process Measurements • Distributed Computing and Information Services • Surface and Microanalysis Science • Software Diagnostics and Conformance Testing ' At Boulder, CO 80303. ^Some elements at Boulder, CO. NIST Special Publication 500-237 Information Technology: SYMPOSIUM TRANSCRIPTION Usability Engineering: Industry- Government Collaboration for System Effectiveness and Efficiency held February 26, 1996, at the National Institute of Standards and Technology Laura L. Downey and Sharon J. Laskowski, Editors Information Access and User Interfaces Division Information Technology Laboratory National Institute of Standards and Technology Gaithersburg, MD 20899-0001 July 1997 U.S. Department of Commerce William M. Daley, Secretary Technology Administration Gary R. Bachula, Acting Under Secretary for Technology National Institute of Standards and Technology Robert E. Hebner, Acting Director Reports on Information Technology The Information Technology Laboratory (ITL) at the National Institute of Standards and Technology (NIST) stimulates U.S. economic growth and industrial competitiveness through technical leadership and collaborative research in critical infrastructure technology, including tests, test methods, reference data, and forward-looking standards, to advance the development and productive use of information technology. To overcome barriers to usability, scalability, interoperability, and security in information systems and networks, ITL programs focus on a broad range of networking, security, and advanced information technologies, as well as the mathematical, statistical and computational sciences. This Special Publication 500 series reports on ITL's research in tests and test methods for information technology, and its collaborative activities with industry, government, and academic organizations. National Institute of Standards and Technology Special Publication 500-237 Natl. Inst. Stand. Technol. Spec. Publ. 500-237 261 pages (July 1997) CODEN: NSPUE2 U.S. Government Printing Office Washington: 1997 For sale by the Superintendent of Documents U.S. Government Printing Office, Washington, DC 20402-9325 FOREWORD On February 26, 1996, the National Institute of Standards and Technology (NIST) sponsored the symposium "Usability Engineering: Industry-Government Collaboration for System Effectiveness and Efficiency." The symposium brought together over 100 industry and government professionals to exchange information and strategies for achieving effectiveness, efficiency and satisfaction in computer-based government systems. It also provided a forum for raising awareness about usability engineering and its role in increasing productivity and decreasing costs. The symposium was the first of its kind to address the need to incorporate usability engineering practices into the design and development of computer-based government systems. With the mandate and fiscal necessity of employing commercial, off-the-shelf (COTS) products with legacy data, the complexity of government systems is only increased. It is critical that government and industry implement the best practices now available to ensure usability. This is especially applicable to goverment systems because of the large number (and high cost) of custom computer systems and legacy system integration and updates. This document provides a complete record of the workshop presentations in a conversational style based on the transcription of the symposium videotapes. Figures from the original proceedings were included and referenced to enhance the transcription of each speaker's presentation. For a brief overview of the symposium, including background information, attendee demographics, and feedback, see the symposium report in the SIGCHI Bulletin, October 1996, Volume 28, Number 4. Much positive feedback was received and a second symposium is planned for March 3, 1997. As a note of information, this version (August 1997) of the transcription supersedes the preliminary version dated February 27, 1997. Also, any mention of specific products in either version of the symposium transcription is for informational purposes only and does not constitute any recommendation and/or endorsement by NIST. Editors: Laura L. Downey, NIST (Symposixun Chair) Dr. Sharon J. Laskowski, NIST (Program Committee) (Any errors in the symposium record are a result ofthe transcription and editing done at NIST.) TABLE OF CONTENTS Presentation Page Foreword-- ~ iii Author Index vii Executive Overview of Usability Engineering 1 Rex Hartson, Virginia Tech Keynote - Usability Trends in Government 23 Bill Hefley, Software Engineering Institute Cost-Benefit Analysis of Usability Engineering 41 Clare-Marie Karat, IBM Making Sense ofHCI/GUI Standards and Guidelines ~ 61 Elizabeth Buie, Computer Sciences Corporation Success Stories: A Usability Success Story at the Social Security Administration (SSA) 77 Pat Stoos, OSDD at SSA Focusing on Usability at the Federal Intelligent Document Understanding Laboratory: A Success Story 89 Therasa O'Connell, PRC TIGERS — A Successful Usability Engineering Methodology 99 Daniel Wallace, Naval Surface Warfare Center Usability Engineering in Europe 125 Nigel Bevan, NPL Using Usability Testing to Manage Risk 133 Carolyn Snyder, User Interface Engineering Conduct Usability Testing! A Major Findingfrom a Study of Commercial Best Practices 149 Pascal Gambardella, Computer Sciences Corporation V TABLE OF CONTENTS (CON'T) Presentation Page Usability Issues in Complex Government Systems 169 Donna Cuomo and Jill Drury, MITRE Advanced Interface Design and Evaluation for Navy Applications 191 Astrid Schmidt-Nielsen, Naval Research Laboratory The Role of Users in the Development of Computer Systemsfor the National Airspace System 205 Allyn C. Dillman, Professional Airways System Specialists Needs Assessment/Open Forum 211 Laura Downey and Sharon Laskowski, NIST Closing Plenary - Making Usability Work in the Organization 219 George Casaday, DEC An European Perspective