Louisiana State University LSU Digital Commons LSU Doctoral Dissertations Graduate School 10-16-2018 Machine Learning Tools for Optimization of Fuel Consumption at Signalized Intersections in Connected/Automated Vehicles Environment Saleh Ragab Mousa Louisiana State University and Agricultural and Mechanical College, [email protected] Follow this and additional works at: https://digitalcommons.lsu.edu/gradschool_dissertations Part of the Civil Engineering Commons, and the Transportation Engineering Commons Recommended Citation Mousa, Saleh Ragab, "Machine Learning Tools for Optimization of Fuel Consumption at Signalized Intersections in Connected/ Automated Vehicles Environment" (2018). LSU Doctoral Dissertations. 4725. https://digitalcommons.lsu.edu/gradschool_dissertations/4725 This Dissertation is brought to you for free and open access by the Graduate School at LSU Digital Commons. It has been accepted for inclusion in LSU Doctoral Dissertations by an authorized graduate school editor of LSU Digital Commons. For more information, please [email protected]. MACHINE LEARNING TOOLS FOR OPTIMIZATION OF FUEL CONSUMPTION AT SIGNALIZED INTERSECTIONS IN CONNECTED/AUTOMATED VEHICLES ENVIRONMENT A Dissertation Submitted to the Graduate Faculty of the Louisiana State University and Agricultural and Mechanical College in partial fulfillment of the requirements for the degree of Doctor of Philosophy in The Department of Civil and Environmental Engineering by Saleh Ragab Mousa B.S., Cairo University, 2010 M.S., Cairo University, 2013 December 2018 ACKNOWLEDGMENTS First, I thank God for providing me with all the support and help needed to complete this trip in my career journey. A big part of this support was through my parents, advisors, committee members, supervisors, siblings, son, and wife. I am a lucky person to get that much support. I would like to thank my father and mother, Ragab Mousa and Horyra Mokhtar, who raised me with great love and support. They always believed in me and backed me whenever the odds were stacked against me. They taught me the trait of being a fighter to achieve success. Without them and without their support and prayers, literally, I would have never had the chance to join LSU and meet my committee chair and advisor, Prof. Sherif Ishak, and I could have never achieved where I have reached today. My deepest appreciation to my exceptional advisor, Prof. Ishak, for his wise guidance, academic wisdom counsel, constant encouragement and commitment to complete this dissertation. His dedication and sincere interest in the Intelligent Transportation Systems (ITS) area made it possible for me to work on a hot advanced topic that is of great interest to me and matches with the recent technologies and innovation in the ITS area. Special thanks and appreciation go to Prof. Chester Wilmot, my committee chair, and advisor. In addition to introducing me to the art of designing transportation questionnaires and surveys, his support, sincere dedication, advice and high standards helped me a lot to raise the bar. Thank you for the support, thorough review and constructive criticisms that helped me to deliver a high-quality dissertation. I also feel highly indebted to my committee member, Dr. Julius Codjoe, for his support and guidance. He has been a wise academic supervisor and awesome research friend and mentor. Thank you for the wise advice, support, thorough review, and belief in me, that helped me to bring a high- quality dissertation. ii Special thanks to Prof. Charles Monlezun who triggered my passion for comprehending and mastering parametric and non-parametric statistical methods. To my siblings, Momen, Youssef and Rowda, and to my son, Hamza, thanks for your kindness, support, and belief in me. To my soul mate, better half, and sweetheart, Amany, thank you for being my best friend. Your love, belief in me, and continuous support made this trip, regardless of the challenges, pleasant, successful, and very quick. We did it together and I owe you everything. iii TABLE OF CONTENTS ACKNOWLEDGMENTS .................................................................................................................. ii LIST OF TABLES ............................................................................................................................ vi LIST OF FIGURES .......................................................................................................................... vii LIST OF ABBREVIATIONS ......................................................................................................... viii ABSTRACT ...................................................................................................................................... ix CHAPTER 1 INTRODUCTION .................................................................................................. 1 1.1 Problem Statement ............................................................................................................. 1 1.2 Research Objectives ........................................................................................................... 4 1.3 Scope of Study ................................................................................................................... 5 1.4 Dissertation Outline............................................................................................................ 5 CHAPTER 2 LITERATURE REVIEW AND BACKGROUND ................................................ 7 2.1 Eco-driving Studies ............................................................................................................ 7 2.2 Microscopic Traffic Simulation Platforms ....................................................................... 10 2.3 CAV Interactions.............................................................................................................. 12 2.4 Semi-Actuated Signalized intersections ........................................................................... 13 2.5 Deep Reinforcement Learning ......................................................................................... 17 2.6 Tree-Based Ensemble Algorithms ................................................................................... 19 CHAPTER 3 MODELING ECO-DRIVING AND CAV INTERACTIONS ............................ 24 3.1 Eco-driving Algorithms for Signalized Intersections ...................................................... 24 3.2 Microscopic Fuel Consumption Model ............................................................................ 26 3.3 Modeling Eco-driving Environment and CAV Interactions ............................................ 27 CHAPTER 4 ECO-DRIVING SEMI-ACTUATED ALGORITHM.......................................... 31 4.1 Signal Timing Prediction Module .................................................................................... 32 4.2 Trajectory Optimization Module ...................................................................................... 35 4.3 Queue Estimation Module ................................................................................................ 38 4.4 Experimental Setup .......................................................................................................... 39 CHAPTER 5 ECO-DRIVING DRL ALGORITHM .................................................................. 42 5.1 State and Action Space ..................................................................................................... 42 5.2 Reward ............................................................................................................................. 43 5.3 Deep Neural Network....................................................................................................... 44 5.4 Experience Replay Memory ............................................................................................. 45 5.5 Target Network ................................................................................................................ 46 5.6 Action Selection Policy .................................................................................................... 46 5.7 Prioritized Experienced Replay ........................................................................................ 48 5.8 Double Learning ............................................................................................................... 48 5.9 Experimental Setup .......................................................................................................... 51 iv CHAPTER 6 MARKET PENETRATION ANALYSIS AND RESULTS ................................ 53 6.1 Microscopic Evaluation.................................................................................................... 53 6.2 Macroscopic Evaluation ................................................................................................... 60 CHAPTER 7 CONCLUSIONS AND FUTURE WORK ........................................................... 65 REFERENCES ................................................................................................................................. 69 APPENDIX A: VT-CPFM-1 PYTHON SCRIPT IN VISSIM-COM ............................................. 73 APPENDIX B: SAMPLE VISSIM-COM ECO-DRIVING SCRIPT .............................................. 74 APPENDIX C: ECO-SEMI-Q TRAJECTORY OPTIMIZATION MODULE SCRIPT ................ 80 VITA ...............................................................................................................................................