Combined ICT Technologies for Supervision of Complex Operations in Resilient Communities
Total Page：16
File Type：pdf, Size：1020Kb
Politecnico di Torino ICT for Smart Societies Master’s Thesis Combined ICT Technologies for Supervision of Complex Operations in Resilient Communities Academic Supervisor: Prof. Enrico Magli Supervisor: Paola Dal Zovo (Santer Reply) Author: Seyyed Sadegh Bibak Sareshkeh – s235986 I II Abstract Despite the promotion of technology in various fields and its emerging in people’s conventional life, natural and manmade disasters and incidents are still among the top external causes of injuries and mortality. Moreover, with the rise of the population and rise of megacities, management of emergency cases is becoming more and more complicated every day. These factors have forced Emergency Management (EM) organizations and related institutions to consider implementing various methods and policies to become more robust and efficient while conducting operations. Further, in recent years, Information and Communication Technology (ICT) has been identified as one of the most promising success factors to improve emergency management processes. In the last decades, advances in mobile computing technologies have had a fundamental impact on devising ICT systems supporting the daily work of people. Although the domain of EM has evolved consistently over time and several supporting technologies have been developed, there is a lack of suitable methods to realize the benefits of ICT by emergency managers. The importance of monitoring has led the IP camera market to develop considerably and has brought numerous solutions depending on the scenario. IP cameras are installed in various places in cities (e.g., big squares, stadiums, metros, crowded intersections, etc.) and along the roads for the means of monitoring. Furthermore, in recent years many portable IP camera devices have been developed which reduce the burden of installation considerably and provide service for many diverse locations. However, although mounting and installation of IP cameras have become very cheap, typically to obtain useful information there is still the need for the presence of a human being to observe their output. Thus, often most of the data is processed after a special event such as an emergency has already occurred, causing losing the benefit of having a real-time monitoring tool and as a result, acting instantaneously. In this thesis work, we aimed to design a monitoring system to manage the information obtained from the scene of the emergency field in order to help the operators and decision- makers to have a more complete view regarding the emergency field so as act properly and more quickly. Various solutions were evaluated for each element of the monitoring system including the Media Streaming Server, Media Streaming Protocol, and Video codecs and the best tool is selected for each part. Moreover, an object detection solution is implemented on top of the video elements in order to help the supervisor to identify the required information faster and more accurately. III IV V Table of Contents 1. Introduction ………………………………………………………………………..…. 1 2. Background information ……………………………………………………….…….. 7 2.1. HTML5 Video Playing ……………………………………….…………………. 7 2.2. Streaming Protocol ………………………………………………………………. 7 2.3. Container ………………………………………………………………………… 7 2.4. Codec ……………………………………………………………………………. 7 2.5. Adaptive Streaming …………………………………………………………….... 8 2.6. Media Source Extension (MSE) ……………………………………………….… 9 3. Assessment of technologies …………………………………………..…………….. 10 3.1. Transcoding ………………………………………………………...………. 10 3.1.1. Motion JPEG …………………………………………………………….. 10 3.1.2. MPEG4 …………………………………………………………………... 11 3.1.3. H.264 …………………………………………………………………..… 12 3.1.4. H.265 …………………………………………………………………….. 13 3.2. Encoders …………………………………………………………………….. 14 3.2.1. FFmpeg …………………………………………………………………... 14 3.2.2. Gstreamer …………...…………………………………………………… 14 3.3. Live Streaming from IP Cameras ……………………………………………... 14 3.4. Broadcasting Protocol ……………………………………...…………..……... 17 3.4.1. Media Streaming Protocols and Standards ……………………………….. 17 - Real-time Streaming Protocol (RTSP) ........................................................ 17 - Real-time Messaging Protocol (RTMP) ...................................................... 19 - Dynamic Adaptive Streaming over HTTP (DASH) .................................... 20 - HTTP Live Streaming (HLS) ...................................................................... 21 - Microsoft Smooth Streaming (MSS) .......................................................... 21 - Secure Reliable Transport (SRT) ................................................................ 21 - Web Real-Time Communication (WebRTC) ............................................. 21 VI 3.5. Streaming Servers …………………………………………..………………… 28 3.5.1. List of Available Streaming Servers ……………………....…...………… 28 - Red5 Open-Source Media Server ………………………………………… 28 - MistServer Open-Source ………………………………...………………. 28 - Kurento Media server ………...……………………………..…………… 29 - Janus WebRTC Server ………………………………………...…………. 29 - Nginx …………………………………………………………………….. 29 - Nimble Streamer ………………………………………….……………… 29 - Icecast …………………………………………………….……………… 29 3.6. Object Detection …………………...…………….…………………………. 30 3.6.1. Kurento media API ………………………………………………………. 30 3.6.1.1. Kurento media element toolbox ………………………………… 30 3.6.2. Integration of TensorFlow Object Detection API with Kurento ……..…... 33 3.6.2.1. TensorFlow …………………………………………...………… 33 3.7. Microservice-oriented Architecture ………………………………………… 33 3.7.1. Node.js …………………………………………………………………… 34 3.7.2. RESTful API …………………………………………………………….. 34 3.7.3. Express.js ………………………………………………………………… 35 3.7.4. PostreSQL Server ………………………………………………….…….. 35 3.7.5. Sequelize.js ………………………………………………………………. 36 4. Results and Discussion ……………………………………………………………… 37 4.1. Broadcasting Protocol Assessment and Selection ………………….……….. 37 4.2. Streaming Servers Assessment and Selection …………………………….… 40 - Red5 Open-source ……………………………..………………………… 40 - Nginx ……………………………………………..……………………… 40 - Kurento Media Server ………………………………...………………….. 41 4.3. Selected Architecture ……………………………………………………….. 44 5. Conclusion ………………………………………………………………………….. 46 6. References …………..……….……………………………………………………… 48 VII List of Figures 1. UIERS, Crew’s information presentation ………………………………..….……..…. 2 2. UIERS, Video Monitoring ……………………………………………….....……..…. 3 3. Example of fire prevention by detecting smoke ...……………………….….……..…. 5 4. Example of pedestrian detection using a vehicle onboard camera to prevent pedestrians from being run over ……………………...…………………….….………………….. 5 5. Example of drowsy driver detection …………………………….……......…….……. 5 6. Detecting Two very different scenarios such as sitting and falling …………….……. 6 7. Simplified schema of encoding a video file using video and audio codecs …….….… 7 8. Adaptive Streaming working mechanism ……………………………………………. 8 9. Simple Schema of MSE code elements …………………………………………….... 9 10. Video encoder and decoder …………………………………….…………..…..…… 10 11. General block diagram of MPEG-4 algorithm (encoder). ………...…….……….…. 12 12. Structure of a GOP used in H.264 ……………………..……………………………… 13 13. The general architecture of video streaming ……………………………………….. 14 14. Camera with the whole streaming service ………………………………………….. 15 15. IP Camera separated from the streaming system ……………………………………. 16 16. Separate solution for each section ………………………………………………….. 16 17. Selection Equilibrium ………………….…………………………………………… 17 18. Session establishment and termination in RTSP ……………………………….…… 18 19. RTMP Streaming Session ……………………………………………………….….. 19 20. Dash communication mechanism …………………...……………………………… 20 21. The WebRTC Triangle ………………………..…………………………….………. 22 22. The WebRTC Trapezoid ………………...………………………………………….. 22 23. WebRTC Communication Flow in a Browser ……………………….……………… 23 24. Basic WebRTC call flow ………………………………………………………….… 25 25. A simplified diagram of a RESTful API ……………………………………………. 35 26. Selected streaming architecture …………………………………………….………. 38 27. Incoming streaming quality cases …………………………………………..….…… 39 28. HLS and DASH delay using Nginx RTMP module and FFmpeg ……………..……. 41 VIII 29. Kurento Media Server implementation of a WebRTC gateway for IP cameras supporting both RTSP/H.264 and HTTP/MJPEG ………………………………….. 42 30. IP Camera and TensorFlow Object Detection API integration proposed architecture ……………………………………………………………………….……………… 43 31. Implementation of Object Detection over WebRTC stream provided by Kurento Media Server ……………………………………………………………….………………. 44 32. Selected Services Architecture ………….………………..………………………… 44 IX List of Tables 1. A comparison of H.264 and H.265 performance for different video resolutions ..… 13 2. WebRTC APIs regarding connection setup and management ................................... 26 3. WebRTC APIs regarding identity and security .......................................................... 27 4. WebRTC APIs regarding telephony ........................................................................... 28 5. Kurento Media Server Endpoints ………………………………………………...… 31 6. Kurento Media Server Filters …………………………………………….………… 31 7. Kurento Media Server Hubs …………………………………………………..……. 32 X XI Chapter I: Introduction The concept of “extreme events” encompasses both natural and man-made emergencies, catastrophes, and disasters of exceptional and unthinkable magnitudes. Extreme events hit with little or almost no warning, cause massive injuries, losses of life, and damage to property, displace populations, severely disrupt critical infrastructures including the information and communication infrastructures,