DOCSLIB.ORG

Surveillance and Video Analytics: Factors Influencing the Performance

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Surveillance and Video Analytics: Factors Influencing the Performance Surveillance and video analytics: factors influencing the performance ERNCIP thematic group video analytics and surveillance Jeroen van Rest, MSc., TNO 2015 The research leading to these results has received funding from the European Union as part of the European reference network for critical infrastructure protection project. EUR 27852 EN Surveillance and video analytics: factors influencing the performance This publication is a technical report by the Joint Research Centre, the European Commission’s in-house science service. It aims to provide evidence-based scientific support to the European policymaking process. The scientific output expressed does not imply a policy position of the European Commission. Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of this publication. JRC science hub https://ec.europa.eu/jrc JRC100399 EUR 27852 EN ISBN 978-92-79-57771-0 ISSN 1831-9424 doi:10.2788/945388 © European Union, 2015 Reproduction is authorised provided the source is acknowledged. All images © European Union 2015 Contents Contents ............................................................................................................ 3 Acknowledgements .............................................................................................. 7 Abstract ............................................................................................................. 8 1 Introduction ................................................................................................. 10 1.1 The challenge .......................................................................................... 10 1.2 The approach .......................................................................................... 10 1.3 Reading guide ......................................................................................... 10 2 Surveillance ................................................................................................. 12 2.1 Surveillance as a risk management measure ............................................... 12 2.2 Surveillance purpose and effectiveness ....................................................... 13 2.3 Situational awareness .............................................................................. 14 2.4 Ethics, privacy and invasiveness ................................................................ 15 2.5 Typical components of a surveillance system ............................................... 16 2.6 Compartmentalisation .............................................................................. 16 3 Method ........................................................................................................ 18 3.1 Problem characteristics ............................................................................. 18 3.2 Morphological analysis .............................................................................. 18 3.2.1 Notation of a configuration ................................................................. 19 3.3 Principles for modelling surveillance systems ............................................... 19 3.3.1 System theory ................................................................................. 19 3.3.2 Levels of abstraction ......................................................................... 19 3.3.3 Security as a risk management strategy .............................................. 20 3.3.4 Design basis threat ........................................................................... 20 3.3.5 Surveillance effectiveness .................................................................. 21 3.3.6 Readiness and maturity ..................................................................... 21 4 Results ........................................................................................................ 22 4.1 Relevant factors for surveillance systems .................................................... 22 4.1.1 Dependencies between surveillance factors .......................................... 23 4.2 Relevant factors for video analytics ............................................................ 25 4.2.1 Dependencies between video analytics factors ...................................... 25 5 Instructions for use ....................................................................................... 28 5.1 Describing a threat or an incident .............................................................. 28 5.1.1 Example incident: Boston bombings .................................................... 28 5.2 Describing use cases ................................................................................ 28 5.2.1 Example use case: crowd control from airborne platform ....................... 28 5.3 Describing data sets ................................................................................. 29 3 5.3.1 Example data set: i-LIDS sterile zone .................................................. 29 5.4 Describing scientific work .......................................................................... 29 5.4.1 Example: Visual surveillance for moving vehicles .................................. 30 5.4.2 Example: event detection and analysis from video streams .................... 30 6 Conclusions, discussion and next steps ............................................................ 32 6.1 Myths, partial trusts and misconceptions ..................................................... 32 6.2 Discussion .............................................................................................. 32 6.2.1 Descriptions ..................................................................................... 33 6.2.2 Specificity ........................................................................................ 33 6.2.3 Cross-correlation matrix .................................................................... 33 6.3 Next steps .............................................................................................. 33 6.3.1 Disclose and develop test data sets for relevant video analytics use cases 33 6.3.2 Joint innovation ................................................................................ 34 6.3.3 Prevent and stop crises with dynamically deployed surveillance capabilities 34 6.3.4 Develop large-scale auto-calibration for robust and scalable video analytics 35 6.3.5 Develop metadata standards that cover these factors ............................ 36 6.3.6 Other potential applications of the MAS and MAVA ................................ 36 6.3.7 Potential topics other than video analytics ............................................ 36 References ......................................................................................................... 37 List of abbreviations and definitions ...................................................................... 39 List of figures ..................................................................................................... 43 List of tables ...................................................................................................... 44 Appendix A Profiling ....................................................................................... 45 Appendix B Relevant factors for surveillance systems .......................................... 46 B.1 Context, environment, asset and risk ......................................................... 46 B.1.1 Context ........................................................................................... 46 B.1.1.1 Weather .................................................................................... 46 B.1.1.2 Weather dynamics ...................................................................... 46 B.1.1.3 Privacy awareness ...................................................................... 46 B.1.1.4 Security awareness ..................................................................... 46 B.1.1.5 Intent ....................................................................................... 46 B.1.1.6 Relation .................................................................................... 46 B.1.2 Environment .................................................................................... 47 B.1.2.1 Type of environment ................................................................... 47 B.1.2.2 Type of object ............................................................................ 47 B.1.2.3 Existing infrastructure ................................................................. 47 B.1.2.4 Compartments present ................................................................ 47 4 B.1.2.5 Closed compartments ................................................................. 47 B.1.2.6 People density ........................................................................... 47 B.1.3 Risk ................................................................................................ 47 B.1.3.1 Risk cause ................................................................................. 47 B.1.3.1.1 Asset to protect ....................................................................... 48 B.1.3.1.2 Threat .................................................................................... 48 B.1.3.1.3 Vulnerability ............................................................................ 49 B.1.3.2 Resulting risk ............................................................................. 49 B.1.3.2.1 Chance ..................................................................................
Load more

Recommended publications
  • E Dawn of Robot Surveillance AI, Video Analytics, and Privacy
    e Dawn of Robot Surveillance AI, Video Analytics, and Privacy June 2019 e Dawn of Robot Surveillance AI, Video Analytics, and Privacy By Jay Stanley © 2019 AMERICAN CIVIL LIBERTIES UNION Cover: Sources images, shutterstock.com “The robots are here. The robots control your warnings. They analyze and notify, following instructions you have set.” — Promotional web site for “Video Surveillance as a Service”1 “The overwhelming majority of images are now made by machines for other machines” 2 — Trevor Paglen 1 http://www.vsaas.com/. 2 https://thenewinquiry.com/invisible-images-your-pictures-are-looking-at-you/. 1 Table of Contents I. INTRODUCTION ...................................................................................................... 3 II. WHAT IS AI VIDEO ANALYTICS .......................................................................... 5 The rise of deep learning ........................................................................................... 6 Overcoming challenges with new sources of data .................................................. 8 III. CURRENT DEPLOYMENTS ................................................................................ 9 Government deployments ......................................................................................... 9 Commercial deployments ........................................................................................ 10 Analytics in the cloud .............................................................................................. 11 IV. HOW COMPUTERS WILL
    [Show full text]
  • Review of Existing Smart Video Surveillance Systems Capable of Being Integrated with ADDPRIV Project Deliverable 2.1
    Review of existing smart video surveillance systems capable of being integrated with ADDPRIV project Deliverable 2.1 GDANSK D 2.1 – Review of existing smart video surveillance systems capable of being integrated with ADDPRIV Automatic Data relevancy Discrimination for a PRIVacy-sensitive video surveillance SEC-2010.6.5-2 - Use of smart surveillance systems, data protection, integrity and sharing information within privacy rules D2.1 – Review of existing smart video surveillance systems capable of being integrated with ADDPRIV project Due date of deliverable: 31-07-2011 Actual submission date: 31-07-2011 Start of project: 01 February 2011 Duration: 36 Months Lead Contractor for this deliverable: GDANSK Revision: 1.0 Project co-funded by the European Commission within the Seventh Framework Programme Dissemination level PU Public + Confidential, only for members of the consortium (including the CO Commission Services) Page 2 Status: Final version Version: 1.1 Date: 28.07.2011 D 2.1 – Review of existing smart video surveillance systems capable of being integrated with ADDPRIV Revision History Deliverable Administration and summary Project Acronym: ADDPRIV Grant Agreement no: 261653 Document Identifier: ADDPRIV_20113107_WP2_GDANSK_Scoreboard_R11.pdf Leading partner: GDANSK Report version: 1.1 Report preparation date: 28-07-20011 Classification: Public Nature: Report Author(s) and contributors: GDANSK, KU Status Plan Draft Working + Final Submitted Approved The ADDPRIV consortium has addressed all comments received, making changes as necessary. Changes to the document are detailed in the change log table below. Date Edited by Status Changes made 20.07.2011 GDANSK Beta version 25.07.2011 KU Reviewed 28.07.2011 GDANSK Corrected 31.07.2011 ANOVA Quality check Page 3 Status: Final version Version: 1.1 Date: 28.07.2011 D 2.1 – Review of existing smart video surveillance systems capable of being integrated with ADDPRIV Copyright This report is © ADDPRIV Consortium 2011.
    [Show full text]
  • Multimedia Content Analysis: the Next Wave
    Multimedia Content Analysis: The Next Wave View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by CiteSeerX Nevenka Dimitrova Philips Research, 345 Scarborough Rd., Briarcliff Manor, NY 10510, USA [email protected] Abstract. We have witnessed a decade of exploding research interest in multimedia content analysis. The goal of content analysis has been to derive automatic methods for high-level description and annotation. In this paper we will summarize the main research topics in this area and state some assumptions that we have been using all along. We will also postulate the main future trends including usage of long term memory, context, dynamic processing, evolvable generalized detectors and user aspects. 1 Introduction After a decade of exploding interest in the multimedia content analysis and retrieval [1,6,8,15], there has been enough research momentum generated that we can finally reflect on the overall progress. The goal has been to develop automatic analysis tech- niques for deriving high level descriptions and annotations, as well as coming up with realistic applications in pursuit of the killer application. Meanwhile the MPEG-7 has standardized the description of metadata – data describing information in the content at various levels. Applications range from home media library organization that con- tains volumes of personal video, audio and images, multimedia lectures archive, con- tent navigation for broadcast TV and video on demand content. The tools have emerged from traditional image processing and computer vision, audio analysis and processing, and information retrieval. In this paper we will first present an overview of the active research areas in video content analysis in Section 2.
    [Show full text]
  • Big Data Analytics for Video Surveillance
    Multimedia Tools and Applications https://doi.org/10.1007/s11042-019-07793-w Big data analytics for video surveillance Badri Narayan Subudhi1 & Deepak Kumar Rout2 & Ashish Ghosh3 Received: 1 September 2018 /Revised: 19 March 2019 /Accepted: 15 May 2019 # Springer Science+Business Media, LLC, part of Springer Nature 2019 Abstract This article addresses the usage and scope of Big Data Analytics in video surveillance and its potential application areas. The current age of technology provides the users, ample opportunity to generate data at every instant of time. Thus in general, a tremendous amount of data is generated every instant throughout the world. Among them, amount of video data generated is having a major share. Education, healthcare, tours and travels, food and culture, geographical exploration, agriculture, safety and security, entertainment etc., are the key areas where a tremendous amount of video data is generated every day. A major share among it are taken by the daily used surveillance data captured from the security purpose camera and are recorded everyday. Storage, retrieval, processing, and analysis of such gigantic data require some specific platform. Big Data Analytics is such a platform, which eases this analysis task. The aim of this article is to investigate the current trends in video surveillance and its applications using Big Data Analytics. It also aims to focus on the research opportunities for visual surveillance in Big Data frameworks. We have reported here the state-of-the-art sur- veillance schemes for four different imaging modalities: conventional video scene, remotely sensed video, medical diagnostics, and underwater surveillance. Several works were reported in this research field over recent years and are categorized based on the challenges solved by the researchers.
    [Show full text]
  • Who, What, When, Where, Why and How in Video Analysis: an Application Centric View
    Who, What, When, Where, Why and How in Video Analysis: An Application Centric View Sadiye Guler Jay Silverstein, Ian Pushee, Xiang Ma, intuVision, Inc. Ashutosh Morde 10 Tower Office Park, Woburn, MA 01801 intuVision, Inc. [email protected] {jay,ian,sean,ashu}@intuvisiontech.com Abstract This paper presents an end-user application centric of video data, some of which is analyzed in real-time. In view of surveillance video analysis and describes a addition to traditional surveillance video, in recent years a flexible, extensible and modular approach to video new source of video data has surpassed any kind of content extraction. Various detection and extraction volume expectations by individuals recording video components including tracking of moving objects, during public and private events with their cameras and detection of text, faces, and face based soft biometric for smart phones and posting them on the websites such as gender, age and ethnicity classification are described YouTube for public consumption. These unconstrained within the general framework for real-time and post event video clips (i.e., “other people’s video” the term coined by analysis applications Panoptes and VideoRecall. Some [2]) may provide additional content and information that is end-user applications that are built on this framework are not available in typical surveillance camera views; hence discussed. they became part of any surveillance and investigation task. For example, surveillance video cameras around a stadium typically have views of entry/exit halls, snack courts etc. which may provide partial video coverage of 1. Introduction some excited fans being destructive with no means to get a good description of the individuals; but an inspection of Since their early deployment in mid-1960s [25], of the other fan videos posted on public sites soon after the video surveillance systems held the promise for increased event may make available crucial information that would security, safety and they facilitated forensic evidence otherwise be missed.
    [Show full text]
  • A Survey on Content-Aware Video Analysis for Sports
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 99, NO. 9, JANUARY 2017 1 A Survey on Content-aware Video Analysis for Sports Huang-Chia Shih, Member, IEEE Abstract—Sports data analysis is becoming increasingly 1) Panasonic incorporated SAP SE to develop a video-based large-scale, diversified, and shared, but difficulty persists in sports analytics and tracking system. Match Insights is an rapidly accessing the most crucial information. Previous surveys analytics prototype SAP developed with the German have focused on the methodologies of sports video analysis from Football Association for the soccer World Cup 2014 in the spatiotemporal viewpoint instead of a content-based viewpoint, Brazil [7]. and few of these studies have considered semantics. This study 2) Vizrt, short for VisualiZation (in) real time, is a Norwegian develops a deeper interpretation of content-aware sports video analysis by examining the insight offered by research into the company that provides content production, management, structure of content under different scenarios. On the basis of this and distribution tools for the digital media industry. Its insight, we provide an overview of the themes particularly products include applications for creating real-time 3D relevant to the research on content-aware systems for broadcast graphics and maps, visualizing sports analyses, managing sports. Specifically, we focus on the video content analysis media assets, and obtaining single workflow solutions for techniques applied in sportscasts over the past decade from the the digital broadcast industry. Vizrt has a customer base in perspectives of fundamentals and general review, a content more than 100 countries and approximately 600 employees hierarchical model, and trends and challenges.
    [Show full text]
  • Deep-Framework: a Distributed, Scalable, and Edge-Oriented Framework for Real-Time Analysis of Video Streams
    sensors Article Deep-Framework: A Distributed, Scalable, and Edge-Oriented Framework for Real-Time Analysis of Video Streams Alessandro Sassu * , Jose Francisco Saenz-Cogollo and Maurizio Agelli Center for Advanced Studies, Research and Development in Sardinia (CRS4), Località Pixina Manna, Edificio 1, 09010 Pula, CA, Italy; [email protected] (J.F.S.-C.); [email protected] (M.A.) * Correspondence: [email protected] Abstract: Edge computing is the best approach for meeting the exponential demand and the real-time requirements of many video analytics applications. Since most of the recent advances regarding the extraction of information from images and video rely on computation heavy deep learning algorithms, there is a growing need for solutions that allow the deployment and use of new models on scalable and flexible edge architectures. In this work, we present Deep-Framework, a novel open source framework for developing edge-oriented real-time video analytics applications based on deep learning. Deep-Framework has a scalable multi-stream architecture based on Docker and abstracts away from the user the complexity of cluster configuration, orchestration of services, and GPU resources allocation. It provides Python interfaces for integrating deep learning models developed with the most popular frameworks and also provides high-level APIs based on standard HTTP and WebRTC interfaces for consuming the extracted video data on clients running on browsers or any other web-based platform. Citation: Sassu, A.; Saenz-Cogollo, J.F.; Agelli, M. Keywords: real-time video analytics; edge computing; deep learning; distributed systems; software Deep-Framework: A Distributed, framework Scalable, and Edge-Oriented Framework for Real-Time Analysis of Video Streams.
    [Show full text]
  • Searching Surveillance Video Contents Using Convolutional Neural Network
    International Journal of Electrical and Computer Engineering (IJECE) Vol. 11, No. 2, April 2021, pp. 1656~1665 ISSN: 2088-8708, DOI: 10.11591/ijece.v11i2.pp1656-1665 1656 Searching surveillance video contents using convolutional neural network Duaa Mohammad, Inad Aljarrah, Moath Jarrah Department of Computer Engineering, Jordan University of Science and Technology (JUST), Jordan Article Info ABSTRACT Article history: Manual video inspection, searching, and analyzing is exhausting and inefficient. This paper presents an intelligent system to search surveillance Received Feb 24, 2020 video contents using deep learning. The proposed system reduced the amount Revised Aug 4, 2020 of work that is needed to perform video searching and improved the speed Accepted Nov 4, 2020 and accuracy. A pre-trained VGG-16 CNNs model is used for dataset training. In addition, key frames of videos were extracted in order to save space, reduce the amount of work, and reduce the execution time. The Keywords: extracted key frames were processed using the sobel operator edge detector and the max-pooling in order to eliminate redundancy. This increases Convolutional neural networks compaction and avoids similarities between extracted frames. A text file, that Key frames contains key frame index, time of occurrence, and the classification of the Sobel detector VGG-16 model is produced. The text file enables humans to easily search for VGG-16 objects of interest. VIRAT and IVY LAB datasets were used in the Video content analysis experiments. In addition, 128 different classes were identified in the datasets. The classes represent important objects for surveillance systems. However, users can identify other classes and utilize the proposed methodology.
    [Show full text]
  • Affective Video Content Analysis: a Multidisciplinary Insight Yoann Baveye, Christel Chamaret, Emmanuel Dellandréa, Liming Chen
    Affective Video Content Analysis: A Multidisciplinary Insight Yoann Baveye, Christel Chamaret, Emmanuel Dellandréa, Liming Chen To cite this version: Yoann Baveye, Christel Chamaret, Emmanuel Dellandréa, Liming Chen. Affective Video Content Analysis: A Multidisciplinary Insight. IEEE Transactions on Affective Computing, Institute of Elec- trical and Electronics Engineers, 2017, 10.1109/TAFFC.2017.2661284. hal-01489729 HAL Id: hal-01489729 https://hal.archives-ouvertes.fr/hal-01489729 Submitted on 14 Mar 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1 Affective Video Content Analysis: A Multidisciplinary Insight Yoann Baveye, Christel Chamaret, Emmanuel Dellandrea,´ and Liming Chen Abstract—In our present society, the cinema has become one of the major forms of entertainment providing unlimited contexts of emotion elicitation for the emotional needs of human beings. Since emotions are universal and shape all aspects of our interpersonal and intellectual experience, they have proved to be a highly multidisciplinary research field, ranging from psychology, sociology, neuroscience, etc., to computer science. However, affective multimedia content analysis work from the computer science community benefits but little from the progress achieved in other research fields.
    [Show full text]
  • Video Content Analysis for Assessment of the Advertisement
    DOI 10.4010/2016.647 ISSN 2321 3361 © 2016 IJESC Research Article Volume 6 Issue No. 3 Video Content Analysis for Assessment of the Advertisement Jigar Gajjar1, Aayush Khator2, Pooja Patil3 Department of Computer Engineering Mumbai University, Vidyalankar Institute of Technology, India. [email protected], [email protected], [email protected] Abstract: This paper is a literature survey paper of our system which will help advertisers to get some sort of feedback about their advertisement. Advertisement is a form communication which helps to promote a product in an attempt to induce consumers purchase that product. Advertisement are basically displayed to increase the awareness about brands that are introduced in the market or to demonstrate the differences between the products available and the competing products in order to increase their sell.Advertisement helps to get insight of the product. Now a days billboards are preferred to display advertisements in malls. Billboards are effective way of advertising as they are more attractive and noticeable. However it is not possible for advertisers to know the public response for those ads on the billboards thus creating a bottleneck. Our system helps in getting some chunk of that response from the people who are interested in the displayed ads and crunching that data to give advertisers a true feedback for their ads on the billboard. This system includes real-time detection of multiple faces and tracking of those detected faces which will be used to calculate the amount of time person spends in front of the given billboard. It also includes nose and eyes tracking to know whether the viewer's attention is at billboard or outside it.
    [Show full text]
  • Towards Efficient Video Detection Object Super-Resolution with Deep Fusion Network for Public Safety
    Hindawi Security and Communication Networks Volume 2021, Article ID 9999398, 14 pages https://doi.org/10.1155/2021/9999398 Research Article Towards Efficient Video Detection Object Super-Resolution with Deep Fusion Network for Public Safety Sheng Ren , Jianqi Li , Tianyi Tu , Yibo Peng , and Jian Jiang School of Computer and Electrical Engineering, Hunan University of Arts and Science, Changde 415000, China Correspondence should be addressed to Jianqi Li; [email protected] Received 22 March 2021; Revised 14 April 2021; Accepted 14 May 2021; Published 24 May 2021 Academic Editor: David Meg´ıas Copyright © 2021 Sheng Ren et al. *is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Video surveillance plays an increasingly important role in public security and is a technical foundation for constructing safe and smart cities. *e traditional video surveillance systems can only provide real-time monitoring or manually analyze cases by reviewing the surveillance video. So, it is difficult to use the data sampled from the surveillance video effectively. In this paper, we proposed an efficient video detection object super-resolution with a deep fusion network for public security. Firstly, we designed a super-resolution framework for video detection objects. By fusing object detection algorithms, video keyframe selection al- gorithms, and super-resolution reconstruction algorithms, we proposed a deep learning-based intelligent video detection object super-resolution (SR) method. Secondly, we designed a regression-based object detection algorithm and a key video frame selection algorithm.
    [Show full text]
  • Digital Video Quality Handbook Appendix
    Digital Video Quality Handbook Appendix First Responders Group January 2018 Digital Video Handbook Appendix January, 2018 Prepared for: Department of Homeland Security Science and Technology Directorate Washington, DC This document was prepared under funding provided by the U.S. Department of Homeland Security Science and Technology Directorate (Resilient Systems for Public Safety Communication). Points of view or opinions expressed in this document are those of the authors and do not necessarily represent the official position or policies of the U.S. Department of Homeland Security. The Johns Hopkins University Applied Physics Laboratory (JHU/APL) assumes no liability for this document’s content or use thereof. This document does not constitute a standard, specification or regulation. Additionally, JHU/APL does not endorse particular products or manufacturers. Trade and manufacturer’s names may appear in this report only because they are considered essential to the objective of this document. Principal Authors: Steve Surfaro, Dan Syed (JHU/APL), Steven Babin (JHU/APL), Jay Chang (JHU/APL) Contributing Author: John Contestabile (JHU/APL) The authors would like to express their appreciation to the Department of Homeland Security Science and Technology Directorate and our sponsor Mr. Cuong Luu. Please send comments to: Mr. John Contestabile Program Manager JHU/APL 11100 Johns Hopkins Road Laurel, MD 20723-6099 Phone: 443-220-8090 E-mail: [email protected] 1 Digital Video Handbook Appendix January, 2018 Publication Notice Disclaimer The views and opinions of authors expressed herein do not necessarily reflect those of the U.S. government. Reference herein to any specific commercial products, processes or services by trade name, trademark, manufacturer or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the U.S.
    [Show full text]