Predicting Pilot Misperception of Runway Excursion Risk Through Machine Learning Algorithms of Recorded Flight Data
Total Page:16
File Type:pdf, Size:1020Kb
Dissertations and Theses 2-2020 Predicting Pilot Misperception of Runway Excursion Risk Through Machine Learning Algorithms of Recorded Flight Data Edwin Vincent Odisho II Follow this and additional works at: https://commons.erau.edu/edt Part of the Aviation Safety and Security Commons, and the Risk Analysis Commons This Dissertation - Open Access is brought to you for free and open access by Scholarly Commons. It has been accepted for inclusion in Dissertations and Theses by an authorized administrator of Scholarly Commons. For more information, please contact [email protected]. PREDICTING PILOT MISPERCEPTION OF RUNWAY EXCURSION RISK THROUGH MACHINE LEARNING ALGORITHMS OF RECORDED FLIGHT DATA By Edwin Vincent Odisho II A Dissertation Submitted to the College of Aviation in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Aviation Embry-Riddle Aeronautical University Daytona Beach, Florida February 2020 © 2020 Edwin Vincent Odisho II All Rights Reserved. ii ABSTRACT Researcher: Edwin Vincent Odisho II Title: PREDICTING PILOT MISPERCEPTION OF RUNWAY EXCURSION RISK THROUGH MACHINE LEARNING ALGORITHMS OF RECORDED FLIGHT DATA Institution: Embry-Riddle Aeronautical University Degree: Doctor of Philosophy in Aviation Year: 2020 The research used predictive models to determine pilot misperception of runway excursion risk associated with unstable approaches. The Federal Aviation Administration defined runway excursion as a veer-off or overrun of the runway surface. The Federal Aviation Administration also defined a stable approach as an aircraft meeting the following criteria: (a) on target approach airspeed, (b) correct attitude, (c) landing configuration, (d) nominal descent angle/rate, and (e) on a straight flight path to the runway touchdown zone. Continuing an unstable approach to landing was defined as Unstable Approach Risk Misperception in this research. A review of the literature revealed that an unstable approach followed by the failure to execute a rejected landing was a common contributing factor in runway excursions. Flight Data Recorder data were archived and made available by the National Aeronautics and Space Administration for public use. These data were collected over a four-year period from the flight data recorders of a fleet of 35 regional jets operating in the National Airspace System. The archived data were processed and explored for evidence of unstable approaches and to determine whether or not a rejected landing was iv executed. Once identified, those data revealing evidence of unstable approaches were processed for the purposes of building predictive models. SAS™ Enterprise Miner® was used to explore the data, as well as to build and assess predictive models. The advanced machine learning algorithms utilized included: (a) support vector machine, (b) random forest, (c) gradient boosting, (d) decision tree, (e) logistic regression, and (f) neural network. The models were evaluated and compared to determine the best prediction model. Based on the model comparison, the decision tree model was determined to have the highest predictive value. The Flight Data Recorder data were then analyzed to determine predictive accuracy of the target variable and to determine important predictors of the target variable, Unstable Approach Risk Misperception. Results of the study indicated that the predictive accuracy of the best performing model, decision tree, was 99%. Findings indicated that six variables stood out in the prediction of Unstable Approach Risk Misperception: (1) glideslope deviation, (2) selected approach speed deviation (3) localizer deviation, (4) flaps not extended, (5) drift angle, and (6) approach speed deviation. These variables were listed in order of importance based on results of the decision tree predictive model analysis. The results of the study are of interest to aviation researchers as well as airline pilot training managers. It is suggested that the ability to predict the probability of pilot misperception of runway excursion risk could influence the development of new pilot simulator training scenarios and strategies. The research aids avionics providers in the development of predictive runway excursion alerting display technologies. v DEDICATION I would like to dedicate this work to my parents and my wife. My parents, Edwin and DeAnn Odisho, instilled upon me the value of education and the pursuit of knowledge. The encouragement you provided in my formative years, to ask why and seek answers, laid the foundation for a lifelong journey in aviation. Without your support and love, this journey would not have been possible. Although I lost you in the middle of the program, your presence in my life persists and will be with me until we meet again. I would like to share this dedication with my wife, Bettina Odisho. Bettina was with me every step of the way on this journey. Without you I would not have had the intestinal fortitude to begin this endeavor, much less the shared responsibility of balancing family, full time flying, and pursuing a Ph.D. During the challenging times, your love and encouragement gave me the courage and motivation to succeed. vi ACKNOWLEDGEMENTS To successfully complete an endeavor of this magnitude is not an individual accomplishment. I received immeasurable support, encouragement, guidance, advice, and help. I would first like to acknowledge my family. The day to day effort required to complete this program would not have been sustainable without complete and comprehensive teamwork at home. The demands of the program necessitated sacrifice and selflessness on behalf of my wife, Bettina and daughters, Natalie and Sofia. I am forever grateful to you all. I have been fortunate to receive coaching, guidance, instruction, and mentorship from teachers, coaches, flight instructors, professors, and others in my lifelong journey of learning. No one has been more important or valuable to me than my mentor and committee chairman, Dr. Dothang Truong. His unique ability to constructively criticize, encourage improvement, and maintain high standards was crucial to my development on this journey of scholarship. I have never met a finer gentleman in my nearly 40 years in aviation. I feel fortunate and blessed to have had the opportunity to work with Dr. Truong. Dr. Dothang Truong was assisted by a very special group of scholars on my dissertation committee. External committee member, Dr. Robert Maxson, is Director, National Centers for Environmental Prediction, and is an expert in data mining and predictive modeling. Dr. David Esser is an acknowledged expert in flight data analysis programs such as FOQA. Dr. Robert “Buck” Joslin, a fellow Marine Corps aviator and test pilot, is an expert in governmental regulatory issues associated with flight deck technology integration. Semper Fidelis! vii A special acknowledgement is greatly warranted for several people whose importance and value to the program could not be stressed enough. Mrs. Susie Sprowl, Administrative Assistant, School of Graduate Studies, College of Aviation, and Ms. Katie Esguerra, Marketing and Admissions Coordinator, School of Graduate Studies, College of Aviation, greatly assisted me in my transition back to student status after many years out of academia. I would also like to acknowledge Mrs. Jan Neal, Lead Instructional Designer, School of Graduate Studies, College of Aviation, for her assistance in the production aspect of my dissertation document. I would be remised if did not acknowledge each of my professors in the program: Dr. Bruce Conway, (Management of Systems Engineering); Dr. Haydee Cuevas (Human Factors in Aviation, Quantitative Research Methods in Aviation, User-Centered Design in Aviation); Dr. Mark Friend, Academic Advisor (Aviation Safety Management Systems); Dr. Steven Hampton, Associate Dean of Research and Graduate Studies, College of Aviation, (Current Practices/Future Trends in Aviation); Dr. Kadie Mullins, (Instructional Design in Aviation); Dr. Alan Stolzer, Dean College of Aviation (Topics in Safety Management Systems); and Dr. Dothang Truong, Dissertation Committee Chairman (Advanced Quantitative Data Analysis, Research Methods, Quantitative and Qualitative Data Analysis, Operations Research and Decision Making). Although I was not fortunate enough to have taken a class from Dr. Scott Winter, I would like to acknowledge the important role he played in my development in the program. Thank you all for your encouragement and support. viii TABLE OF CONTENTS Page Signature Page .................................................................................................................. iii Abstract ............................................................................................................................. iv Dedication ......................................................................................................................... vi Acknowledgements .......................................................................................................... vii List of Tables .................................................................................................................. xiv List of Figures .................................................................................................................. xv Chapter I Introduction ................................................................................................ 1 Background .................................................................................. 11