Using Bluetooth Detectors to Monitor Urban Traffic Flow with Applications to Traffic Management by Mohsen Hajsalehi Sichani A thesis submitted to the Victoria University of Wellington in fulfilment of the requirements for the degree of Doctor of Philosophy in Computer Science. Victoria University of Wellington 2020 Abstract A comprehensive traffic monitoring system can assist authorities in identifying parts of a road transportation network that exhibit poor performance. In addition to monitoring, it is essential to develop a localized and efficient analytical trans- portation model that reflects various network scenarios and conditions. A compre- hensive transportation model must consider various components such as vehicles and their different mechanical characteristics, human and their diverse behaviours, urban layouts and structures, and communication and transportation infrastructure and their limitations. Development of such a system requires a bringing together of ideas, tools, and techniques from multiple overlapping disciplines such as traf- fic and computer engineers, statistics, urban planning, and behavioural modelling. In addition to modelling of the urban traffic for a typical day, development of a large-scale emergency evacuation modelling is a critical task for an urban area as this assists traffic operation teams and local authorities to identify the limitations of traffic infrastructure during an evacuation process through examining various parameters such as evacuation time. In an evacuation, there may be severe and unpredictable damage to the infrastructure of a city such as the loss of power, telecommunications and transportation links. Traffic modelling of a large-scale evacuation is more challenging than modelling the traffic for a typical day as his- torical data is usually available for typical days, whereas each disaster and evacua- tion are typically one-off or rare events. Damage due to a disaster, combined with a sudden increase of demand due to the evacuation of people will likely result in increased pressure on the remaining, potentially fragmented, infrastructure. The lessons learnt from evacuation modelling can assist traffic operation teams and local authorities to provide safer and more efficient planning. The development of pervasive personal digital devices such as phones, watches, and headphones which can be interconnected with technologies such as Bluetooth, has led to a disruptive change in the ways in which local governments can monitor traffic flows within their cities. Moreover, modern vehicles and navigation systems can interconnect to the personal devices of drivers and passengers primarily via Bluetooth tech- nologies. By continuously monitoring such devices when they are discoverable and in range, traffic patterns can be estimated based on, not only the volume of detection, but also other characteristics of the devices that can be used to give more refined estimates of the real underlying traffic flows. This thesis examines Blue- tooth traffic data collected from Bluetooth Traffic Monitoring Systems (BTMS) for modelling and monitoring the urban traffic. BTMS can monitor and track individual detected vehicles through a city. Installation, processing, data trans- mission, and maintenance of BTMS are easier, quicker and cheaper than existing standard monitoring systems such as CCTV cameras and inductive loops. Induc- tive loops are typically point-wise traffic monitoring systems that are installed in the roads and can measure the traffic flow. However, the use of BTMS devices presents several challenges: not every vehicle has a detectable device, some have many, and there are devices carried by pedestrians and non-motor vehicles as well as stationary devices. This thesis enumerates and investigates these challenges through statistical modelling, various protocols for cleaning and data preparation, dynamic estimation of the detection rate, and simulation through the case study of the city of Wellington, New Zealand. The city of Wellington experienced damage from the 2016 Kaikoura earthquake (a magnitude 7.8 earthquake), which led to road closures and other infrastructure damage. As part of modelling, performance evaluation, and identifying impacted routes by the 2016 Kaikoura earthquake, this thesis analyses three weeks of BTMS data from the periods before and after the earthquake. Furthermore, this thesis proposes a multi-disciplinary dynamic traf- fic modelling (TFDA2M) framework and evaluates the performance of TFDA2M on various large-scale evacuation scenarios. These scenarios cover a wide range of real-world use cases which may occur during a disaster such as power failure, an abrupt increase in demand, and damage to the main transportation infrastruc- ture. The findings of this thesis highlight an immediate need for preparations of a large-scale evacuation planning for Wellington to mitigate the consequences of a large-scale evacuation due to a future disaster. Moreover, TFDA2M can assist traffic operation managers and authorities in making smarter decisions (both quantitative and spatially) through the simulation process. Since TFDA2M has a flexible schema, it can be set to monitor, assess, and manage the traffic flow on a daily basis and disaster occasions. iv Acknowledgments PhD is sometimes called for "Permanent Head Damage", which may emphasis on a genuinely life-changing experience, and I was not an exception. Finishing it would not have been possible without the support and guidance that I received from many people. I would like to express my sincere gratitude to my supervisors Prof. Richard Arnold and A/Prof. Kris Bubendorfer, who have always been there for me, pas- sionate, motivated, kind, and supportive. Richard is a fantastic teacher and scien- tist, and Kris is a real motivator (also motivated me to lose 20kg!). Their guidance helped me to develop my research spirit and be a newer version of me. I would like to thank the examination committee: FGR, ECS , Dr. Andrew Meads, A/Prof. Ian Welch, and Prof. Peter Komisarczuk for their insightful com- ments and encouragement. I appreciate Andrew and Ian for their detailed com- ments. I would like to thank my parents (Mohammadali and Parvin) who made many sacrifices and supported me in different stages of my life, especially at the begin- ning of this journey. I cannot thank enough my wife, Hanieh, for being supportive, patient, kind, and motivator. She was there with me during the most challenging moments of my life, whenever I needed her, as there were some occasions that I was lost (and will be!). I would also like to thank amazing people at the Student Learning Centre at my university who helped me a lot at the beginning of this journey. I would also like to thank the researchers who supported me during my re- v vi search: Alfonso Ariza Quintana , Jakob Erdmann, Mitra Pourabdollah, and Joanne Taylor. Alfonso and Jakob are the real heroes, helping thousands of young re- searchers like me every year (although I am not that young anymore!). Special thanks to HMI for providing me with access to their Bluetooth data. Last but not least, I am also very grateful to my friends who supported me at different stages of this journey: Mohammad Nekooei, Ihab Sinno, my friends at Harmonic Analytic (Ari Angelo, Lisa Chen, and Harel Lustiger), my friends and colleagues at Auckland and Wellington councils (Boris Kirov, Haydn Read, Pamela Brown, Ian Kloppers, and John Dunshea). Contents 1 Introduction3 1.1 Data collection...........................5 1.2 Dynamic traffic monitoring system.................6 1.3 Disaster traffic modelling......................7 1.4 Case study, Wellington, New Zealand...............8 1.5 Motivation..............................8 1.6 Research questions......................... 10 1.7 Research contributions....................... 12 1.8 Organization............................. 14 2 Wellington, an overview 15 2.1 Wellington.............................. 15 2.2 Wellington spatial layers...................... 16 2.2.1 Fault zones......................... 16 2.2.2 Tsunami layer........................ 16 2.2.3 Flood layer......................... 16 2.2.4 Building layer....................... 18 2.2.5 Power lines......................... 18 2.2.6 Road layer......................... 18 2.3 Conclusion, a combined view.................... 26 3 Bluetooth data 31 3.1 Sample data of Bluetooth Traffic Monitoring Devices....... 33 vii viii CONTENTS 3.2 Types of detected Bluetooth devices................ 34 3.3 Limitation of Bluetooth devices.................. 36 3.3.1 Only detect a proportion of passing vehicles........ 36 3.3.2 Uncertainties in the detection of the type of Bluetooth de- vices............................ 37 3.3.3 Route uncertainty...................... 37 3.3.4 Multi-tenancy detection.................. 37 3.3.5 Privacy issues........................ 39 3.3.6 Other issues......................... 39 3.4 Conclusion............................. 39 4 ExtoVT and MTDiBT 41 4.1 Introduction............................. 41 4.2 Literature review of the cleaning process.............. 42 4.3 Multi-tenancy detection problem.................. 45 4.3.1 High-level design of MTDiBT............... 45 4.3.2 MTDiBT algorithm..................... 47 4.3.3 Possible limitation..................... 60 4.3.4 Example from collected data................ 61 4.3.5 Outputs of MTDiBT.................... 62 4.4 ExtoVT............................... 63 4.5 Pre-processing and exploratory analysis, a use case study..... 67 4.6 Results, the use case study....................