NATIONAL AND KAPODISTRIAN UNIVERSITY OF ATHENS SCHOOL OF SCIENCE DEPARTMENT OF INFORMATICS AND TELECOMMUNICATIONS BSc THESIS Composite event recognition for maritime monitoring Manolis N. Pitsikalis Supervisors: Alexander Artikis Researcher, NCSR “Demokritos” Assistant Professor, UNIPI Panagiotis Stamatopoulos Assistant Professor, NKUA ATHENS JUNE 2018 ΕΘΝΙΚΟ ΚΑΙ ΚΑΠΟΔΙΣΤΡΙΑΚΟ ΠΑΝΕΠΙΣΤΗΜΙΟ ΑΘΗΝΩΝ ΣΧΟΛΗ ΘΕΤΙΚΩΝ ΕΠΙΣΤΗΜΩΝ ΤΜΗΜΑ ΠΛΗΡΟΦΟΡΙΚΗΣ ΚΑΙ ΤΗΛΕΠΙΚΟΙΝΩΝΙΩΝ ΠΤΥΧΙΑΚΗ ΕΡΓΑΣΙΑ Αναγνώριση σύνθετων γεγονότων για την παρακολούθηση της ναυτιλιακής δραστηριότητας Μανώλης Ν. Πιτσικάλης Επιβλέποντες: Αλέξανδρος Αρτίκης, Ερευνητής, ΕΚΕΦΕ «∆ημόκριτος» Επίκουρος Καθηγητής, ΠΑΠΕΙ Παναγιώτης Σταματόπουλος, Επίκουρος Καθηγητής, ΕΚΠΑ ΑΘΗΝΑ ΙΟΥΝΙΟΣ 2018 BSc THESIS Composite event recognition for maritime monitoring Manolis N. Pitsikalis R.N.: 1115201300143 SUPERVISORS: Alexander Artikis Researcher, NCSR “Demokritos” Assistant Professor, UNIPI Panagiotis Stamatopoulos Assistant Professor, NKUA ΠΤΥΧΙΑΚΗ ΕΡΓΑΣΙΑ Αναγνώριση σύνθετων γεγονότων για την παρακολούθηση της ναυτιλιακής δραστηριότητας Μανώλης Ν. Πιτσικάλης Α.Μ.: 1115201300143 ΕΠΙΒΛΕΠΟΝΤΕΣ: Αλέξανδρος Αρτίκης, Ερευνητής, ΕΚΕΦΕ «∆ημόκριτος» Επίκουρος Καθηγητής, ΠΑΠΕΙ Παναγιώτης Σταματόπουλος, Επίκουρος Καθηγητής, ΕΚΠΑ ABSTRACT Maritime monitoring systems support safe shipping as they allow for the real-time detection of dangerous, suspicious and illegal vessel activities. The intent of this thesis was the development of a composite event recognition engine for maritime monitoring and the construction of a set of patterns expressing effectively maritime activities in the Event Calculus. In this work, we use the Run-Time Event Calculus, a modern Prolog implementation of the Event Calculus along with tools allowing the compression of data streams, and the spatio-temporal link discovery. Additionally, to further improve the performance of recognition engine we extended the Run-Time Event Calculus with a dynamic grounding mechanism. Moreover, to increase the accuracy of the proposed system, we have been collaborating with domain experts in order to construct effective patterns of maritime activity. We evaluated our system in terms of predictive accuracy and efficiency using real kinematic vessel data. SUBJECT AREA: Artificial Intelligence KEYWORDS: event recognition, pattern matching, event calculus, grounding, prolog ΠΕΡΙΛΗΨΗ Τα συστήματα θαλάσσιας επιτήρησης υποστηρίζουν την ασφαλέστερη ναυτιλία, καθώς επιτρέπουν την ανίχνευση σε πραγματικό χρόνο, επικίνδυνες, ύποπτες και παράνομες δραστηριοτήτες σκαφών. Η πρόθεση αυτής της πτυχιακής είναι η ανάπτυξη μίας αρχι- τεκτονικής συστημάτων εστιασμένη στην θαλάσσια επιτήρηση, καθώς και ενός συνόλου “μοτίβων”, ικανά να εφράσουν αποτελεσματικά ναυτιλιακές δραστηριότητες και συμβάντα. Σε αυτή την δουλεία χρησιμοποιούμε ως μήχανη αναγνωρίσης γεγονότων τον Λογισμό Γε- γονότων Πραγματικού Χρόνου, μία σύγχρονη υλοποιήση σε γλώσσα Λογικού Προγραμ- ματισμού, του Λογισμού Γεγονότων, καθώς επίσης ένα εργαλείο συμπίεσης τροχιών και ένα εργαλείο ευρέσης χωρικών σχέσεων. Για να βελτιώσουμε περαιτέρω την απόδοση της μηχανής αναγνωρίσης γεγονότων, δημιουργήσαμε ένα γενικό μηχανισμό δυναμικής θεμε- λίωσης ο οποίος φαίνεται να είναι αποτελεσματικός στα ναυτιλιακά δεδομένα. Επιπλεόν, μέσω της συνεργάσιας μας με τους ειδικούς του δημιουργήσαμε ένα σύνολο από μοτι- βά ναυτιλιακής δραστηριότητας, τα οποία και χρησιμοποιούμε στην πειραματική ανάλυση του συστήματος. Για την αξιολόγηση της προτεινόμενης αρχιτεκτονικής εστιάζουμε σε α- πόδοση και σε ακρίβεια, χρησιμοποιώντας δύο μορφές ροών πραγματικών δεδομένων πλοιών. ΘΕΜΑΤΙΚΗ ΠΕΡΙΟΧΗ: Τεχνητή Νοημοσύνη ΛΕΞΕΙΣ ΚΛΕΙΔΙΑ: αναγνώριση γεγονότων, αντιστοίχηση προτύπων, λογισμός γεγονό- των, θεμελίωση, λογικός προγραμματισμός To my family. ACKNOWLEDGMENTS Firstly, I would like to express my sincere gratitude to my supervisor, Dr. Alexander Artikis for giving me the opportunity to work on this subject, and most importantly his support and deep knowledge on the field, during my research for my B.Sc. thesis. I would also like to thank Cyril Ray, Richard Dreo, Elena Camossi and Anne-Laure Jousselme for their valuable comments during my research on the field. Moreover, I would like to thank Paul Delaunay and Jules-Edouard Pouessel for their insights regarding maritime activity. Finally I would like to thank, assistant Professor Panagiotis Stamatopoulos, for his contribution on my thesis and for being inspiration in my academic course. This work was supported by the project datACRON, which has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 687591. June 2018 CONTENTS PREFACE ......................................... 14 1. INTRODUCTION ................................... 15 1.1 Problem description .................................. 15 1.2 Contribution ...................................... 15 1.3 Thesis structure .................................... 16 2. RELATED WORK .................................. 17 3. BACKGROUND .................................... 20 3.1 Event Calculus ..................................... 20 3.2 Event Calculus for Run-Time reasoning ......................... 22 3.2.1 Fluents ...................................... 23 3.2.2 Window mechanism ................................ 25 3.2.3 Semantics ..................................... 26 3.2.4 Deadlines ..................................... 26 3.3 Maritime Monitoring .................................. 27 3.4 Automatic Identification System - AIS .......................... 28 3.5 Trajectory Synopses Generator ............................. 28 3.6 Data streams ...................................... 28 4. MARITIME ACTIVITY PATTERNS ......................... 30 4.1 Building blocks ..................................... 32 4.1.1 Communication gap ................................ 32 4.1.2 Stopped vessel .................................. 32 4.1.3 Vessel moving with low speed ........................... 33 4.1.4 Vessel changing Speed .............................. 34 4.1.5 Vessel in area of interest .............................. 34 4.2 Maritime Situational Indicators ............................. 35 4.2.1 Vessel under way ................................. 35 4.2.2 Vessel with high speed near coast ......................... 36 4.2.3 Vessel aground .................................. 37 4.2.4 Vessel at anchor or moored ............................ 38 4.2.5 Vessel with travelling speed ............................ 40 4.2.6 Vessel’s movement ability is affected ........................ 40 4.2.7 Vessel has speed incompatible with its type ..................... 42 4.2.8 Vessel drifting ................................... 43 4.2.9 Fishing ...................................... 44 4.2.10 Vessel engaged in search and rescue operation ................... 45 4.2.11 Vessels in rendezvous ............................... 47 4.2.12 Vessel loitering .................................. 48 4.2.13 Tugging ...................................... 48 5. DYNAMIC GROUNDING ............................... 51 5.1 Building Domains ................................... 51 5.2 Updating domains ................................... 51 5.3 Algorithm ........................................ 52 6. EMPIRICAL ANALYSIS ............................... 55 6.1 Experimental setup ................................... 55 6.2 Experimental results .................................. 56 6.2.1 Efficiency ..................................... 56 6.2.2 Accuracy ..................................... 59 7. SUMMARY AND FURTHER WORK ........................ 62 ACRONYMS AND ABBREVIATIONS .......................... 63 REFERENCES ....................................... 64 LIST OF FIGURES Figure 1: A visual illustration of the three interval manipulation constructs of RTEC. In this example, there are two input fluent streams, I1 and I2. The output of each interval manipulation construct is colored light blue. ................................... 25 Figure 2: An illustration of deadlines. Arrows pointing up denote initiation points caused by initiatedAt rules while arrows pointing down indicate ter- mination points caused by deadlines, lines ending in arrows indicate the intervals the erraticMovement fluent holds, lines ending in empty circles indicate that the deadline has been extended and lines end- ing in black circles denote that the deadline has been met. 27 Figure 3: Processing steps prior to CER. ..................... 29 Figure 4: A dependency graph of the recognised CEs. 30 Figure 5: Vessel under way leaving the port of Brest, France. Yellow circles denote AIS messages, marked circles indicate critical points and the line represents the vessel trajectory reconstructed by the trans- mitted AIS signals. The absence of position signals is not always classified as a communication gap, since it may not be long enough according to the threshold of the synopsis generator. 36 Figure 6: Vessel, near the port of Brest, France, with speed above the 5 knots limit near coast. ............................. 37 Figure 7: Anchored vessel. Yellow circles denote AIS messages, marked cir- cles indicate critical points, the line represents the vessel trajectory and the green dotted area is an anchorage area. 39 Figure 8: General cargo vessel travelling with speed ≈ 10 knots. As illustrated on this Figure, it is common for vessels to travel long distances with travelling speed without changing direction. 40 Figure 9: Container ship moving with speed 1 to 4 knots in the open sea, while