Peering Inside the Operators' Toolbox

Total Page:16

File Type:pdf, Size:1020Kb

Peering Inside the Operators' Toolbox FINISWG THE DFcIGN: PEERING INSIDE THE OPERATORS' TOOLBOX Gemrd L. Torenvliet A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Mechanical & Indusuial Engineering University of Toronto O Gerard L. Torenvliet (1999) National Library Bibliothèque nationale du Canada Acquisitions and Acquisitions et Bibliographie Services services bibliographiques 395 Wellington Street 395. ma WellMgton OîîawaON KlAW OaawaO(J K1AW Canada Canada The author has granted a non- L'auteur a accordé melicence non exclusive Licence aiiowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or sell reproduire, prêter, distriiuer ou copies of this thesis in microform, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/film, de reproduction sur papier ou sur format électronique. The author retains ownership of the L'auteur conseme la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fiom it Ni la thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation. ..hstead of searching for mechanisms in the environrnent that tum organisms into trivial machines, we have to find the mechanism within the organisms that enable thern tu tztrn their environment into a rrivial machine. Finishing the Design 1 Torenvliet, Gerard Lawrens (1999). Finishine the desi-gn: Peering; inside the operators' toolbox. Master's Thesis, Graduate Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ontario, CANADA. ABSTRACT While interacting with complex systems, operators of these systems fiequentIy modify their interfaces in order to achieve task goals more effectively or more efficiently (what Rasmussen (1986) has called 'finishing the design). While there is a literature on operator modifications, this literature includes only field studies and theoretical treatments, but has not proceeded to experimentation. This research describes two prelirninary experiments designed to answer four questions related to operator modifications: (1) Why do operators modify their interfaces? (2) How do these modifications develop over time? (3) How does this activity affect performance? and (4) How does this activity affect understanding? The resuits of these experiments pointed to answers to most of these questions, and most significantly, pointed to a taxonomy of the purposes behind operator modifications in complex systems. .. Finishinz the Design II ACKNOWZEDGMENTS This research was supported by grants from the Naniral Sciences and Engineering Research Council of Canada and the Japan Atomic Energy Research Institute (Dr. Fwniya Tanabe, contract monitor). A number of people were instrumental in cornpleting this research. First thanks go to my supervisor, Dr. Kim Vicente, for supporting and guiding this research effort. Deeper than that, 1 would Iike to thank Kim both for introducing me to the field of cognitive engineering during my undergraduate education, for fostering my enthusiasm in the field, for putting up with my weaknesses. and for developing my research career. Thanks are also due to Dr. Cathy Burns. who hris helped me tu flesh out a number of the ideas presented in this thesis, and who has helped me out with other research leading up to this thesis. John Hajdukiewicz also deserves mention for putting up with my barging into his off~ceto unload some new idea or insight. and for encouraging me in my work. Finally, this thesis is dedicated to three important individuals: to my father who sparked my love of science. to my mother, who has nurtured it, and to my wife, who has penevered with it. 1 have been blessed to have each of you in my life. To the glory of God done. G.T., July 1999 Finishing the Design iii TABLE OF CONTENTS Abstract ............................................................................................................................................ i .. Acknowledgments ...........................................................................................................................11 ... Table of Contents ........................................................................................................................... 111 Table of Tables ........................................................................................................ ... .................... vi Table of Figures .............................................................................. ... ............................................. ix Introduction ..................................................................................................................................... 1 Bac kground and Motivation ............................................................................................................4 Generd Theoreticai Treatments of Tool Use ....................................................................... 4 Moving From Theory to the Field........................................................................................ 8 Field Studies of Tool Use .................................................................................................... -9 Motivation: Moving From the Field to the Laboratory ...................................................... 15 Research Context ............................................................................................................... 18 Pilot Study ..................... ..,..... ................................................................................................... 19 Purpose ............................................................................................................................... 19 Experimental Design ...................................................................................................... L9 Participants ......................................................................................................................... 19 Apparatus ........................................................................................................................... 21 Procedure ............................................................................................................................ 22 Data Sources ....................................................................................................................... 25 Results: Tool Use ............................................................ .... ......................... 26 Results: Control Recipes .................................................................................................... 35 Results: Normal Tnal Completion Time ................. ...... .......... ............................... 39 Results: Normal Triai Blowups .......................................................................................... 42 Finishing the Design Results: Fault trials ............................................................................................................. 43 Discussion: Implications for a more thorough study ......................................................... 45 Experiment .................................................................................................................................... 47 Purpose ............................................................................................................................... 47 Experimental Design ............... ... ........................................................................................ 47 Participants ......................................................................................................................... 49 Apparatus ........................................................................................................................... 52 Procedure ............................................................................................................................ 52 Data Sources....................................................................................................................... 54 Results: Tool Use ............................................................................................................... 54 Results: Normal Trials ....................................................................................................... 87 Results: Fault Trials ..................................................................................................... 97 Results: Control Recipes .................................................................................................... 97 Results - Abstraction Hierarchy Analyses ................................................................ 98 Discussion ................................................................................................................................... 101 Why do operators modify their interfaces? .................... ,.. ......................................... 101 How do operator modifications develop over time? ....................... ... ......................... 104 Do operator modifications affect task performance? ....................................................... 105 Do operator modifications affect understanding? ............................................................ 105 ConcIusions ................................................................................................................................
Recommended publications
  • University of Otago the Ecological Approach to Interface Design In
    University of Otago Te Whare Wananga o Otago Dunedin, New Zealand The Ecological Approach to Interface Design in Intentional Domains William B L Wong The Information Science Discussion Paper Series Number 97/12 December 1997 ISSN 1177-455X University of Otago Department of Information Science The Department of Information Science is one of six departments that make up the Division of Commerce at the University of Otago. The department offers courses of study leading to a major in Information Science within the BCom, BA and BSc degrees. In addition to undergraduate teaching, the department is also strongly involved in postgraduate research programmes leading to MCom, MA, MSc and PhD degrees. Research projects in software engineering and software development, information engineering and database, software metrics, knowledge-based systems, natural language processing, spatial information systems, and information systems security are particularly well supported. Discussion Paper Series Editors Every paper appearing in this Series has undergone editorial review within the Department of Information Science. Current members of the Editorial Board are: Assoc. Professor George Benwell Assoc. Professor Nikola Kasabov Dr Geoffrey Kennedy Dr Stephen MacDonell Dr Martin Purvis Professor Philip Sallis Dr Henry Wolfe The views expressed in this paper are not necessarily the same as those held by members of the editorial board. The accuracy of the information presented in this paper is the sole responsibility of the authors. Copyright Copyright remains with the authors. Permission to copy for research or teaching purposes is granted on the condition that the authors and the Series are given due acknowledgment. Reproduction in any form for purposes other than research or teaching is forbidden unless prior written permission has been obtained from the authors.
    [Show full text]
  • 19496 NAE Bridge V32n4
    Winter 2002 The BRIDGE LINKING ENGINEERING AND SOCIETY Scalable Quantum Computing Using Solid-State Devices Bruce Kane The Role of Computational Fluid Dynamics in Process Industries David Lee Davidson The Human Factor Kim J. Vicente Sustainable Energy from Nuclear Fission Power Marvin L. Adams Stretching the Boundaries of Nuclear Technology James P. Blanchard Licensing and Building New Nuclear Infrastructure Peter S. Hastings Promoting the technological welfare of the nation by marshalling the knowledge and insights of eminent members of the engineering profession. The BRIDGE NATIONAL ACADEMY OF ENGINEERING George M.C. Fisher, Chair Wm. A. Wulf, President Sheila E. Widnall, Vice President W. Dale Compton, Home Secretary Harold K. Forsen, Foreign Secretary William L. Friend, Treasurer Editor-in-Chief George Bugliarello (Interim) Managing Editor: Carol R. Arenberg Production Assistants: Penelope Gibbs, Kimberly West The Bridge (USPS 551-240) is published quarterly by the National Academy of Engineering, 2101 Constitution Avenue, N.W., Washington, DC 20418. Periodicals postage paid at Washington, D.C. Vol. 32 No. 4 Winter 2002 Postmaster: Send address changes to The Bridge, 2101 Constitution Avenue, N.W., Washington, DC 20418. Papers are presented in The Bridge on the basis of general interest and time- liness. They reflect the views of the authors and not necessarily the position of the National Academy of Engineering. The Bridge is printed on recycled paper. © 2002 by the National Academy of Sciences. All rights reserved. A complete copy of each issue of The Bridge is available in PDF format at http://www.nae.edu/TheBridge. Some of the articles in this issue are also available as HTML documents and may contain links to related sources of information, multimedia files, or other content.
    [Show full text]
  • Workshop Report
    Workshop on Guiding Public Health Policy in Areas of Scientific Uncertainty July 11-13, 2005. McLaughlin Centre for Population Health Risk Assessment Institute of Population Health University of Ottawa In partnership with the World Health Organization WORKSHOP REPORT Prepared by Lorraine Craig, M.H.Sc. [email protected] TABLE OF CONTENTS 1.0 Workshop Overview ........................................................................................................... 1 2.0 Introduction to the Framework ........................................................................................... 3 2.1 Overview of the Framework ........................................................................................... 3 2.2 European Perspective on the Protection of Human Health............................................. 5 2.3 Discussion of Framework Elements ............................................................................... 7 2.3.1 Putting Health Issues into Context.......................................................................... 7 2.3.2 Option Generation and Assessment........................................................................ 8 2.3.3 Integrating Public Perception in the Precautionary Principle................................. 9 2.3.4 Scientific Risk Assessment and Uncertainty ........................................................ 10 2.3.5 Action Implementation and Evaluation ................................................................ 10 2.3.6 Legal Aspects of Using Precaution......................................................................
    [Show full text]
  • DSTO-GD-0680 PR.Pdf
    UNCLASSIFIED Cognitive Work Analysis: Foundations, Extensions, and Challenges Neelam Naikar Air Operations Division Defence Science and Technology Organisation DSTO-GD-0680 ABSTRACT This essay, which reviews the foundations, extensions, and challenges of cognitive work analysis, is based on a keynote address delivered at the 10th International Naturalistic Decision Making Conference held in Orlando, Florida from 31 May to 3 June, 2011. It describes the origins of cognitive work analysis and the utility of this framework for designing ecological interfaces, as well as for tackling a variety of other design challenges, particularly, the design of teams or organisations. Also featured in this essay is the formulation of a methodological perspective of cognitive work analysis, which complements the conceptual accounts provided by Rasmussen (1986), Rasmussen, Pejtersen, and Goodstein (1994), and Vicente (1999). Finally, this essay highlights the latest shift in research emphasis from work domain analysis, the first dimension of cognitive work analysis, to the subsequent dimensions of this framework. RELEASE LIMITATION Approved for public release UNCLASSIFIED UNCLASSIFIED Published by Air Operations Division DSTO Defence Science and Technology Organisation 506 Lorimer St Fishermans Bend, Victoria 3207 Australia Telephone: (03) 9626 7000 Fax: (03) 9626 7999 © Commonwealth of Australia 2012 AR-015-298 November 2011 APPROVED FOR PUBLIC RELEASE UNCLASSIFIED UNCLASSIFIED Cognitive Work Analysis: Foundations, Extensions, and Challenges Executive Summary Workers in complex sociotechnical systems, such as military systems, have incredibly challenging jobs. To help them perform their jobs effectively, it is indisputable that first we need to understand the nature of their work. Only then is it possible to design interfaces, teams, or training systems, for instance, that will enable workers to meet their work demands successfully – in a way that is safe, productive, and healthy (Vicente, 1999).
    [Show full text]
  • Toward an Understanding of Field Studies
    The Qualitative Report Volume 8 Number 2 Article 9 6-1-2003 Dare I Embark On A Field Study? Toward An Understanding Of Field Studies Yan Xiao University of Maryland, [email protected] Paul Milgram University of Toronto, [email protected] Follow this and additional works at: https://nsuworks.nova.edu/tqr Part of the Quantitative, Qualitative, Comparative, and Historical Methodologies Commons, and the Social Statistics Commons Recommended APA Citation Xiao, Y., & Milgram, P. (2003). Dare I Embark On A Field Study? Toward An Understanding Of Field Studies. The Qualitative Report, 8(2), 306-313. https://doi.org/10.46743/2160-3715/2003.1893 This Article is brought to you for free and open access by the The Qualitative Report at NSUWorks. It has been accepted for inclusion in The Qualitative Report by an authorized administrator of NSUWorks. For more information, please contact [email protected]. Dare I Embark On A Field Study? Toward An Understanding Of Field Studies Abstract Field studies have frequently been advocated as a means for understanding cognitive activities in naturalistic settings. However, there are several fundamental obstacles that one has to overcome to conduct a field study. This paper discusses two of these obstacles in the context of studying problem solving in complex environments: defining goals of a field study and justifying methods used in data analysis. Based on our experience from a recently finished field study, we outline a framework for understanding the nature of field studies and suggest a specific approach to data analysis. We argue that the goal of field studies should not be limited ot hypothesis testing, and that the process of data analysis in field studies can be viewed as an inductive abstraction process.
    [Show full text]
  • Human Factors Health Technology Safety
    Human Factors For Health Technology Safety: Evaluating and Improving the Use of Health Technology In The Real World Andrea Cassano-Piché, Patricia Trbovich, Melissa Griffin, Ying Ling Lin and Tony Easty HumanEra @ UHN Global Centre for eHealth Innovation University Health Network International Federation of Medical and Biological Engineering Clinical Engineering Division Cover Design: Brent Bily - Envisioneering Foreword One characteristic of the healthcare field is its constant and extremely fast technological development. While in the past its progress relied on expertise coming mostly from medicine, today it receives contributions from many different areas of research such as electronics, chemistry, biology, physics, mathematics, mechanical engineering, materials engineering, and informatics. Such fast development has produced an immense variety of well-accepted equipment and medical procedures for diagnosis and therapy. However, it also requires the development, transference and adaptation of safety programs, formerly developed in other industries, to be implemented in medical procedures as well as equipment operation, equipment design and production, hospital interoperability, medical records, cloud storage, and so on. Clinical engineers, used to dealing mostly with medical equipment management techniques, found themselves overwhelmed by safety demands associated with the use of such technologies. Worse, few academics worldwide are familiar with safety programs applied to the healthcare area. As one of the participants in the implementation
    [Show full text]
  • Cognitive Work Analysis: an Influential Legacy Extending Beyond Human
    Applied Ergonomics xxx (2016) 1e13 Contents lists available at ScienceDirect Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo Cognitive work analysis: An influential legacy extending beyond human factors and engineering Neelam Naikar Centre for Cognitive Work and Safety Analysis, Joint and Operations Analysis Division, Defence Science and Technology Group, 506 Lorimer St, Fishermans Bend, Victoria 3207, Australia article info abstract Article history: Jens Rasmussen’s multifaceted legacy includes cognitive work analysis (CWA), a framework for the Received 1 April 2015 analysis, design, and evaluation of complex sociotechnical systems. After considering the framework’s Received in revised form origins, this paper reviews its progress, predictably covering experimental research on ecological 25 April 2016 interface design, case studies of the application of CWA to human factors and engineering problems in Accepted 7 June 2016 industry, and methods and modelling tools for CWA. Emphasis is placed, however, on studying the nexus Available online xxx between some of the recent results obtained with CWA and the original field studies of human problem- solving that motivated the framework’s development. Of particular interest is a case study of the use of Keywords: fi Cognitive work analysis CWA for military doctrine development, a problem commonly regarded as lying outside the elds of Problem-solving human factors and engineering. It is concluded that the value of CWA, even for such diverse problems, is Reasoning likely to result from its conceptual grounding in empirical observations of patterns of human reasoning Sociotechnical systems in complex systems. Jens Rasmussen Crown Copyright © 2016 Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
    [Show full text]
  • Shaftnews.Comoffers New Satirical Parody of Tech
    pageFOCUS 15 Technique • Friday, January 24, 2003 • 15 Aimee Mann has smooth rhythm Track teams whoop it up Opinions Editor Sara Cames reviews Aimee Mann’s The men’s and women’s track teams came away FOCUS newest CD, a collection of smooth rhythms and with big wins at the recent Kentucky thought-provoking lyrics. See what else is good about Invitational. The women are the defending ACC Technique • Friday, January 24, 2003 the album in the review. Page 24 Champions. Page 34 Shaftnews.com offers new satirical parody of Tech The ‘Nique’s Jennifer Lee’ conducts an exclusive interview with the two creators of Shaft News, “The South’s Most Awesomest College Newspaper.” The parody website pokes fun at campus events and personalities. By Jennifer Lee email to our friends. JB cleverly dis- with student leaders, and first of all, get a day? people have a sense of humor about Staff Writer guised it as, “Hey guys, there’s this our site got quoted in with the GT- JB: 10 a day or so. We’ve had themselves. new site, you should check it out.” Sux site, which is bound to happen. 150 sign up to join our mailing list. JB: We only hope, because we Pssst… have you heard about JB: I think I fooled everyone. But we like Georgia Tech! We’re TQ: Can you really buy The Shaft can’t afford lawyers. www.shaftnews.com? CA: I think you fooled no one. happy to go here. We’re just trying News T-shirts and thongs? CA: Yeah, unless you guys buy a The site, which parodies Geor- CA: We also hung up about 15 to be funny.
    [Show full text]
  • Catherine M. Burns
    FELLOW PROFILE Catherine M. Burns Degrees: • PhD Mechanical and Industrial Engineering, University of Toronto • MASc. Industrial Engineering, University of Toronto • BASc. Systems Design Engineering, University of Waterloo • PEng. Professional Engineers of Ontario Current status: Professor, Systems Design Engineering, University of Waterloo; Director, Centre for Bioengineering and Biotechnology, University of Waterloo Homepage: aidl.uwaterloo.ca Biography: As a systems design engineering student, we experienced a very broad ranging technical and design oriented program where one pillar of the program was ergonomics. I quickly gravitated to this elective direction, eventually completing a minor in psychology along with my engineering degree. While interviewing for graduate schools, I met Kim Vicente, a dynamic young professor who was just starting at the University of Toronto. His ideas and passion for cognitive work analysis and ecological interface design were inspiring and allowed me to combine my interests in both psychology and engineered systems. After my master's degree at the University of Toronto I worked at Westinghouse in Pittsburgh for a year. At Westinghouse I met a very strong and experienced team of control room and display designers and began to appreciate the challenges and the historical commitment of those who worked in nuclear human factors. It was a domain that had had some serious accidents in the past, had a tight regulatory environment, challenging complexity for the operator, and yet the people who worked in the field were still challenging the boundaries of design, and passionate about finding smart and effective new ways to help humans work. This environment encouraged me to return for a PhD degree at Toronto where I investigated ecological integration concepts in nuclear power displays.
    [Show full text]
  • Work Domain Analysis Methodology for Development of Operational Concepts for Advanced Reactors
    INL/EXT-15-34783 Revision 0 Work Domain Analysis Methodology for Development of Operational Concepts for Advanced Reactors Jacques Hugo May 2015 DISCLAIMER This information was prepared as an account of work sponsored by an agency of the U.S. Government. Neither the U.S. Government nor any agency thereof, nor any of their employees, makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness, of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. References herein to any specific commercial product, process, or service by trade name, trade mark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the U.S. Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the U.S. Government or any agency thereof. INL/EXT-15-34783 Revision 0 Work Domain Analysis Methodology for Development of Operational Concepts for Advanced Reactors Jacques Hugo May 2015 Idaho National Laboratory INL ART Program Idaho Falls, Idaho 83415 http://www.inl.gov Prepared for the U.S. Department of Energy Office of Nuclear Energy Under DOE Idaho Operations Office Contract DE-AC07-05ID14517 INL ART Program Work Domain Analysis Methodology for Development of Operational Concepts for Advanced Reactors INL/EXT-15-34783 Revision 0 May 2015 Author: Jacques Hugo April 30, 2015 Human Factors Engineer Approved by: Travis Mitchell, INL ART Relationship Manager Date Approved by: Kirk W. Bailey, INL ART TDO Quality Assurance Date ABSTRACT This report describes a methodology to conduct a work domain analysis in preparation for the development of operational concepts for new plants.
    [Show full text]
  • Nureg/Cr-6633 Bnl-Nureg-52563
    NUREG/CR-6633 BNL-NUREG-52563 Advanced Information Systems Design: Technical Basis and Human Factors Review Guidance Brookhaven National Laboratory U.S. Nuclear Regulatory Commission Office of Nuclear Regulatory Research Washington, DC 20555-0001 ON AVAILABILITY NOTICE Availability of Reference Materials Cited in NRC Publications NRC publications in the NUREG series, NRC regu <http://www.nrc.gov> lations, and Title 10, Energy, of the Code of Federal Regulations, may be purchased from one of the fol After January 1,2000, the public may electronically lowing sources: access NUREG-series publications and other NRC records in NRC's Agencywide Document Access 1. The Superintendent of Documents and Management System (ADAMS), through the U.S. Government Printing Office Public Electronic Reading Room (PERR), link PO. Box 37082 <http://www.nrc.gov/NRC/ADAMS/index.html>. Washington, DC 20402-9328 <http://www.access.gpo.gov/sudocs> Publicly released documents include, to name a 202-512-1800 few, NUREG-series reports; Federal Register no 2. The National Technical Information Service tices; applicant, licensee, and vendor documents Springfield, VA 22161-0002 and correspondence; NRC correspondence and <http://www.ntis.gov> internal memoranda; bulletins and information no 1-800-553-6847 or locally 703-605-6000 tices; inspection and investigation reports; licens ee event reports; and Commission papers and The NUREG series comprises (1) brochures their attachments. (NUREG/BR-XXXX), (2) proceedings of confer ences (NUREG/CP-XXXX), (3) reports resulting Documents available from public and special tech from international agreements (NUREG/IA-)XXXX, nical libraries include all open literature items, such (4) technical and administrative reports and books as books, journal articles, and transactions, Feder [(NUREG-XXXX) or (NUREG/CR-)000q], and (5) al Register notices, Federal and State legislation, compilations of legal decisions and orders of the and congressional reports.
    [Show full text]
  • Human Factors Themes from the Legacy of Jens Rasmussen
    Human Factors Themes from the Legacy of Jens Rasmussen HFC Forum meeting 15-16 October, Oslo Henning Boje Andersen Professor Management Engineering Inst. Technical University of Denmark / DTU 2 DTU Management Engineering, Technical University of Denmark 1 3 DTU Management Engineering, Technical University of Denmark Jens Rasmussen (b. 1926) 4 DTU Management Engineering, Technical University of Denmark 2 Powerful head of the Electronics Department Atomenergikomissionens Forsøgsanlæg Risø / Risø National Laboratory 5 DTU Management Engineering, Technical University of Denmark 6 DTU Management Engineering, Technical University of Denmark 3 The work of Jens Rasmussen over the course of the last half century represents some of the most influential contributions to the field of human factors, safety science, human error and accident research. His work has influenced researchers and practitioners in a number of fields including psychology, engineering, sociology and human factors . In more recent years, a large amount of research has been inspired by theoretical and practical aspects of Rasmussen’s work including his models of the boundaries of safe operation and performance and methods such as Cognitive Work Analysis and graphical support for accident investigation such as AcciMaps. E.g., his work on the Risk Management Framework has been cited over 1000 times since its original publication in 1997. In 2013, Jens Rasmussen was elected a member of the US National Academy of Engineering 7 DTU Management Engineering, AdapTechnicalted from University call of Denmark text of the Legacy conference: 8 DTU Management Engineering, Technical University of Denmark 4 9 DTU Management Engineering, Technical University of Denmark From Sanderson 2014 10 DTU Management Engineering, Technical University of Denmark 5 11 DTU Management Engineering, Technical University of Denmark 1950: M.Sc.
    [Show full text]