A Dissertation entitled Addressing Challenges with Big Data for Maritime Navigation: AIS Data within the Great Lakes System by Samir K. Dhar Submitted to the Graduate Faculty as partial fulfillment of the requirements for the Doctor of Philosophy Degree in Spatially Integrated Social Science ________________________________________ Dr. Peter S. Lindquist, Committee Chair ________________________________________ Dr. Kevin P. Czajkowski, Committee Member ________________________________________ Dr. Neil Reid, Committee Member ________________________________________ Dr. Mark A. Vonderembse, Committee Member ________________________________________ Dr. Richard D. Stewart, Committee Member ________________________________________ Dr. Amanda Bryant-Friedrich, Dean College of Graduate Studies The University of Toledo December 2016 Copyright 2016, Samir K. Dhar This document is copyrighted material. Under copyright law, no parts of this document may be reproduced without the expressed permission of the author. An Abstract of Addressing Challenges with Big Data for Maritime Navigation: AIS Data within the Great Lakes System by Samir K. Dhar Submitted to the Graduate Faculty as partial fulfillment of the requirements for the Doctor of Philosophy Degree in Spatially Integrated Social Science The University of Toledo Expected May 2016 The study presented here deals with commercial vessel tracking in the Great Lakes using the Automatic Identification System (AIS). Specific objectives within this study include development of methods for data acquisition, data reduction, storage and management, and reporting of vessel activity within the Great Lakes using AIS. These data show considerable promise in tracking commodity flows through the system as well as documenting traffic volumes at key locations requiring infrastructure investment (particularly dredging). Other applications include detecting vessel calls at specific terminals, locks and other navigation points of interest. This study will document the techniques developed to acquire, reduce, aggregate and store AIS data at The University of Toledo. Specific topics of the paper include: data reducing techniques to reduce data volumes, vessel path tracking, estimate speed on waterway network, detection of vessel calls made at a dock, and a data analysis and mining for errors within AIS data. The study also revealed the importance of AIS technology in maritime safety, but the data is coupled with errors and inaccuracy. These errors within the AIS data will have to be addressed and rectified in future to make the data accurate and useful. The data reduction algorithm shows a 98% reduction in AIS data making it more manageable. In future similar data reduction techniques can possibly be used with traffic GPS data collected for highways and railways. I dedicate this dissertation to my best friend and wife Madhavi Dhar whose sacrificial care made it possible for me to complete it successfully. I would also like to remember my parents, as they would always say education is a kind of treasure that no one can take it away from you ever. Acknowledgements First, I would like to acknowledge my professor and committee chair, Dr. Peter S. Lindquist, for giving me the opportunity to work on projects that eventually led to this dissertation. I would also like to thank Dr. Richard Stewart (aka Captain Stewart) and Dr. Mark Vonderembse for their insight into the maritime transportation industry. I would like to thank my committee members, Dr. Kevin P. Czajkowski, Dr. Neil Reid, for their valuable input and guidance towards my dissertation. Besides all the people on screen, there are few people behind the screen that were very instrumental and helpful in collecting first-hand information and providing resources related to my dissertation. I would like to acknowledge Douglas McDonald (MARAD), Brian J. Tetreault (USACE), Kenneth Ned Mitchell, PhD (USACE), David Winkler, Krithica Kantharaj, and all the others from different government agencies whom I had the pleasure of meeting them during conferences and meetings. Finally, I would like to thank late Dr. Udayan Nandkeolyar, who was one of the original committee member. I like to thanks him for his wisdom and encouragement that he gave me every time with a smile on his face. vi Table of Contents Abstract .............................................................................................................................. iv Acknowledgements ............................................................................................................ vi Table of Contents .............................................................................................................. vii List of Tables ......................................................................................................................x List of Figures .................................................................................................................... xi List of Graphs .................................................................................................................. xiii 1 Introduction ..........................................................................................................1 1.1 Problem Statement .............................................................................................3 1.2 Objectives ..........................................................................................................4 2 GLNS & Vessel Traffic Service ..............................................................................6 2.1 Background and Context....................................................................................6 2.1.1 Great Lakes Economy .......................................................................10 2.2 Vessel Traffic Service ......................................................................................13 2.2.1 VTS History ......................................................................................14 2.2.2 Automatic Radar Plotting Aids .........................................................16 2.3 Automatic Identification System (AIS) ...........................................................17 3 AIS Technology .....................................................................................................18 3.1 AIS Regulations ...............................................................................................19 3.2 AIS Equipment Setup ......................................................................................20 3.3 Technical Model of AIS...................................................................................23 3.4 AIS Message ....................................................................................................27 3.5 AIS Working Concept......................................................................................32 vii 3.6 Long Range Identification System ...................................................................37 3.7 Satellite AIS .....................................................................................................40 3.8 e-Navigation .....................................................................................................46 3.9 Application(s) of AIS .......................................................................................49 4 AIS Data Analysis and Mining ..............................................................................55 4.1 Study Area .......................................................................................................56 4.2 AIS Data Acquisition .......................................................................................57 4.3 AIS Data and Data Analysis ............................................................................60 4.4 Geo-Visualization of AIS Data ........................................................................64 4.5 AIS Data-Mining for Errors .............................................................................65 5 AIS Data Reduction ...............................................................................................70 5.1 AIS Data Reduction Technique .......................................................................70 5.2 AIS Distill Pseudo Code ..................................................................................72 5.3 Results .................................................................................................…...75 6 Application of Distilled AIS Data..........................................................................80 6.1 AIS Application – Calculate Vessel Calls .......................................................80 6.1.1 Pseudo Code – Vessel Call ...............................................................82 6.2 AIS Application – Tracking Vessel Path .........................................................83 6.3 AIS Application – Estimating Speed on Waterway Network ..........................87 7 Conclusion and Discussions ..................................................................................91 References ..........................................................................................................................97 A AIS Message Tables ............................................................................................101 A.1 Navigation Status ..........................................................................................101 viii A.2 Rate of Turn (ROT).......................................................................................102