Simulator Platform Motion Requirements for Recurrent Airline Pilot Training and Evaluation Judith Bürki-Cohen U.S. Department of Transportation Research and Special Programs Administration John A. Volpe National Transportation Systems Center Cambridge, MA 02142 Tiauw H. Go Massachusetts Institute of Technology Cambridge, MA 02139 William W. Chung Science Applications International Corporation Lexington Park, MD 20653 Jeffery A. Schroeder NASA Ames Research Center Moffett Field, CA 94035 Final Report September 2004 1 ACKNOWLEDGEMENTS Many people contributed to this work, and we are very grateful to all of them. The work has been requested by the Federal Aviation Administration’s Flight Standards Service Voluntary Safety Programs Branch managed by Dr. Tom Longridge. We greatly appreciate his insights. In his office, we would also like to thank Dr. Douglas Farrow for his support. Dr. Eleana Edens is the perfect FAA Program Manager. We thank her for her encouragement and effective guidance. The Chief Scientific and Technical Advisor for Human Factors, Dr. Mark D. Rodgers, sponsored the work. We thank him and Dr. Tom McCloy in his office for their involvement. The discussions with Dr. Ed Cook and Paul Ray, the present and former managers of the National Simulator Program Office, were always enlightening. Members of other branches of the FAA’s Flight Standard Services that provided helpful suggestions are Jan Demuth and Archie Dillard. At the Volpe National Transportation Systems Center, we thank Dr. Donald Sussman, the Chief of the Operator Performance and Safety Division, for his direction. Young Jin Jo provided critical support for the First Study, and Sean Jacobs and Kristen Harmon for the Second Study. Dr. Nancy Soja of Battelle provided expert advice on the experimental design and analyses throughout. Dr. Shuang Wu also of Battelle performed the extraordinary feat of programming the laptop that allowed completely hands-off administration of the experiments in the First Study, which eliminated any experimenter effect and saved us innumerable trips to the experiment site. The first two authors are greatly indebted to the regional-airline officials and the training facility that made the First Study possible. This involved donating some of pilots' training time and volunteering instructor/evaluator expertise on part of the airline. The training facility liberally shared simulator-engineering expertise and time. This involved simulator calibration, extensive data collection from the simulator, and programming the simulator interface with the experiment laptop. We feel extremely lucky to have enjoyed the generosity and competence of these collaborations and would love to thank them personally, but our Memorandum of Understanding promised them anonymity. Dr. Vic Lebacqz, Lynda Haines, Dr. Mary Connors, Dr. Key Dismukes, and Julie Mikula made it possible for us to conduct the Second Study at NASA Ames Research Center by contributing not only their expertise, but also financial support. They provided the simulator facilities with a wonderful team of highly qualified professionals: Ghislain Saillant, Charley Ross, Jerry Jones, Jim Miller, Norm Gray and Tom Standifur, Gary Uyehara (and Carlos and Steve). We are also indebted to Bob Shipley, Diane Carpenter, Conrad Grabowski, and Dave Lambert. We received constructive feedback on the simulator-motion tuning from Terry Rager, Dan Renfroe, Mietek Steglinski, Bob Cornell, Gordon Hardy and Dick Bray. We thank you all very much, and it was great to work with you! But nothing would have been accomplished without the many regional-airline crews serving as experiment subjects in the First Study and B747-400 pilots serving as experiment subjects in the Second Study. We thank them for their tolerance to fly many very difficult maneuvers, and for sharing their expertise with us in the long questionnaires. For the Second Study, we thank Bill Edmunds of the Airline Pilots Association, Michael Brown of United Airlines, and Bill Bulfer of Bluecoat Forum. Our 2 desperate pleas always yielded new phone calls to Wendy Krikorian and Mary K. Tracey, our competent recruiters. In conclusion, we would like to remember Edward M. Boothe, who supported this work from the very beginning. He was a key member of the team for the First Study and participated in the design of the Second Study until a few days before his death. We miss him. 3 TABLE OF CONTENTS ACKNOWLEDGEMENTS 2 TABLE OF CONTENTSLIST OF FIGURES 4 LIST OF FIGURES 8 LIST OF TABLES 9 ACRONYMS AND ABBREVIATIONS 11 EXECUTIVE SUMMARY 13 SIMULATOR PLATFORM MOTION REQUIREMENTS FOR RECURRENT AIRLINE PILOT TRAINING AND EVALUATION 17 1. Background 17 2. Requirements Review: Initial Findings and Research Questions 18 2.1 SUBJECT MATTER EXPERT OPINION........................................................................18 2.2 LITERATURE REVIEW..............................................................................................18 3. Empirical Research 19 3.1 RESEARCH APPROACH............................................................................................19 3.2 RESEARCH STRATEGY.............................................................................................20 3.2.1 Magnify Any Existing Evidence For An Effect Of Motion............................20 3.2.2 Avoid Spurious Effects...................................................................................21 4. First Study 21 4.1 RESEARCH QUESTION.............................................................................................21 4.2 METHOD................................................................................................................21 4.3 PROCEDURE............................................................................................................22 4.4 RESULTS.................................................................................................................23 4.4.1 Test Simulator Motion Performance.............................................................23 4.4.2 Pilot Performance and Behavior...................................................................24 4.5 DISCUSSION............................................................................................................24 5. Second Study 25 5.1 INTRODUCTION.......................................................................................................25 5.2 METHOD................................................................................................................26 5.2.1 Experiment Design Overview........................................................................26 5.2.2 Environmental Variables and Maneuver Choice..........................................27 5.2.2.1 Airport, weather and airplane variables............................................................................................27 5.2.2.2 Continued takeoffs with engine failure............................................................................................27 5.2.2.3 Engine-out landing maneuvers.........................................................................................................28 5.2.3 Simulator.......................................................................................................29 5.2.3.1 Visual system...................................................................................................................................29 5.2.3.2 Sound system...................................................................................................................................30 4 5.2.3.3 Control loading................................................................................................................................30 5.2.3.4 Motion system.................................................................................................................................31 5.2.3.5 Motion tuning..................................................................................................................................36 5.2.3.6 Simulator calibration........................................................................................................................38 5.2.4 Participants...................................................................................................38 5.2.4.1 Pilots Flying (PF).............................................................................................................................38 5.2.4.2 Pilots Not Flying (PNF)...................................................................................................................39 5.2.4.3 Air Traffic Controller (ATC)...........................................................................................................39 5.2.5 Procedures.....................................................................................................39 5.2.6 Performance Feedback Displays...................................................................40 5.2.6.1 Takeoff feedback displays...............................................................................................................40 5.2.6.2 Approach and landing performance feedback displays....................................................................42 5.2.7 Data Collection.............................................................................................43 5.2.7.1 Simulator data..................................................................................................................................43 5.2.7.2 Questionnaires.................................................................................................................................44