A FRAMEWORK FOR SMART TRAFFIC MANAGEMENT USING HETEROGENEOUS DATA SOURCES Angelica Milagros Salas Jones BEng, MSc, PGCert A thesis submitted in partial fulfilment of the requirements of the University of Wolverhampton for the degree of Doctor of Philosophy (PhD) March 2020 This work or any part thereof has not previously been presented in any form to the University or to any other body whether for the purposes of assessment, publication or for any other purpose (unless otherwise indicated). Save for any express acknowledgements, references and/or bibliographies cited in the work, I can confirm that the intellectual content of the work is the result of my own efforts and of no other person. The right of Angelica Milagros Salas Jones to be identified as author of this work is asserted in accordance with ss. 77 and 78 of the Copyright, Design and Patents Act 1988. At this date, the author owns copyright. Signature……………………………………………… Date……………………………………………………… Abstract Traffic congestion constitutes a social, economic and environmental issue to modern cities as it can negatively impact travel times, fuel consumption and carbon emissions. Traffic forecasting and incident detection systems are fundamental areas of Intelligent Transportation Systems (ITS) that have been widely researched in the last decade. These systems provide real time information about traffic congestion and other unexpected incidents that can support traffic management agencies to activate strategies and notify users accordingly. However, existing techniques suffer from high false alarm rate and incorrect traffic measurements. In recent years, there has been an increasing interest in integrating different types of data sources to achieve higher precision in traffic forecasting and incident detection techniques. In fact, a considerable amount of literature has grown around the influence of integrating data from heterogeneous data sources into existing traffic management systems. This thesis presents a Smart Traffic Management framework for future cities. The proposed framework fusions different data sources and technologies to improve traffic prediction and incident detection systems. It is composed of two components: social media and simulator component. The social media component consists of a text classification algorithm to identify traffic related tweets. These traffic messages are then geolocated using Natural Language Processing (NLP) techniques. Finally, with the purpose of further analysing user emotions within the tweet, stress and relaxation strength detection is performed. The proposed text classification algorithm outperformed similar studies in the literature and demonstrated to be more accurate than other machine learning algorithms in the same dataset. Results from the stress and relaxation analysis detected a significant amount of stress in 40% of the tweets, while the other portion did not show any emotions associated with them. This information can potentially be used for policy making in transportation, to understand the users’ perception of the transportation network. The simulator component proposes an optimisation procedure for determining missing roundabouts and urban roads flow distribution using constrained optimisation. Existing imputation methodologies have been developed on straight section of highways and their applicability for more complex networks have not been validated. This task presented a solution for the unavailability of roadway sensors in specific parts of the network and was able to successfully predict the missing values with very low percentage error. The proposed imputation methodology can serve as an aid for existing traffic forecasting and incident detection methodologies, as well as for the development of more realistic simulation networks. - i - Table of Contents Abstract ................................................................................................................ i Table of Contents ................................................................................................. ii List of tables ....................................................................................................... vii List of figures ....................................................................................................... ix List of abbreviations ............................................................................................ xii Dedication ......................................................................................................... xiv Acknowledgments ............................................................................................... xv List of publications.............................................................................................. xvi Chapter 1: Introduction ....................................................................................... 1 1.1 Background and motivation ..................................................................... 1 1.2 Research questions ................................................................................. 5 1.3 Aim and objectives .................................................................................. 5 1.4 Research methodology ............................................................................ 7 1.4.1 Phase 1: Problem identification and objectives of a solution .................. 7 1.4.2 Phase 2: Design and development ...................................................... 8 1.4.3 Phase 3: Demonstration ..................................................................... 8 1.4.4 Phase 4: Evaluation ............................................................................ 8 1.4.5 Phase 5: Communication .................................................................... 9 1.5 Research gaps and contribution to knowledge .......................................... 9 1.6 Thesis outline ....................................................................................... 10 Chapter 2: Overview of Traffic Management Systems ......................................... 13 2.1 Introduction .......................................................................................... 13 2.2 Traffic variables, sensors and algorithms ................................................ 14 2.2.1 Traffic variables ............................................................................... 14 - ii - 2.2.2 Sensor Technologies ........................................................................ 17 2.2.3 Algorithms ....................................................................................... 24 2.3 Survey of Traffic Management Systems .................................................. 28 2.3.1 Incident Detection Systems .............................................................. 28 2.3.2 Traffic forecasting systems ............................................................... 31 2.3.3 Traffic flow modelling ....................................................................... 34 2.4 Imputation methodologies for missing traffic data ................................... 37 2.5 Transportation for smart cities ............................................................... 40 2.5.1 The smart city concept ..................................................................... 40 2.5.2 Smart mobility applications ............................................................... 42 2.5.3 Challenges and future perspectives ................................................... 47 2.6 Research gaps ...................................................................................... 50 2.7 Summary .............................................................................................. 51 Chapter 3: Social media for transportation ......................................................... 53 3.1 Introduction .......................................................................................... 53 3.2 Twitter as a sensor ............................................................................... 54 3.2.1 Characterisation of Twitter ............................................................... 55 3.2.2 Accessing Twitter data ..................................................................... 57 3.2.3 Twitter event detection methodologies .............................................. 60 3.2.4 Detecting sentiment from Twitter streams ......................................... 66 3.3 Twitter based transportation research .................................................... 68 3.4 Challenges of using Twitter data for traffic event detection ...................... 72 3.5 Research gaps ...................................................................................... 75 3.6 Summary .............................................................................................. 75 Chapter 4: Smart traffic management framework ............................................... 77 4.1 Introduction .......................................................................................... 77 - iii - 4.2 Research methodology .......................................................................... 77 4.2.1 Design science research ................................................................... 77 4.2.2 Systems design methodologies ......................................................... 81 4.2.3 The study research design ................................................................ 84 4.3 Smart traffic management framework ...................................................