Rochester Institute of Technology RIT Scholar Works Theses Thesis/Dissertation Collections 2-1-2011 Real-time data acquisition, transmission and archival framework Piyush Agarwal Follow this and additional works at: http://scholarworks.rit.edu/theses Recommended Citation Agarwal, Piyush, "Real-time data acquisition, transmission and archival framework" (2011). Thesis. Rochester Institute of Technology. Accessed from This Thesis is brought to you for free and open access by the Thesis/Dissertation Collections at RIT Scholar Works. It has been accepted for inclusion in Theses by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected]. Real-time Data Acquisition, Transmission and Archival Framework by Piyush K. Agarwal A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Engineering Supervised by Associate Professor Dr. Juan Cockburn Department of Computer Engineering Kate Gleason College of Engineering Rochester Institute of Technology Rochester, New York February 2011 Approved by: Dr. Juan Cockburn, Associate Professor Thesis Advisor, Department of Computer Engineering Dr. Jeff Pelz, Professor Committee Member, Center for Imaging Science Dr. Andres Kwasinski, Assistant Professor Committee Member, Department of Computer Engineering Thesis Release Permission Form Rochester Institute of Technology Kate Gleason College of Engineering Title: Real-time Data Acquisition, Transmission and Archival Framework I, Piyush K. Agarwal, hereby grant permission to the Wallace Memorial Library to reproduce my thesis in whole or part. Piyush K. Agarwal Date iii Dedication To my family, Suresh, Saroj, and Neha Agarwal iv Acknowledgments I am grateful to Dr. Cockburn for helping me discover this topic and guiding me through the thesis process. I thank Dr. Pelz for providing the hardware used for development and testing, and for his continued guidance on this thesis. I also thank Dr. Kwasinski for being part of my committee and his help with the networking portions of the thesis. v Abstract Real-time Data Acquisition, Transmission and Archival Framework Piyush K. Agarwal Supervising Professor: Dr. Juan Cockburn Most human actions are a direct response to stimuli from their five senses. In the past few decades there has been a growing interest in capturing and storing the information that is obtained from the senses using analog and digital sensors. By storing this data it is possible to further analyze and bet- ter understand human perception. While many devices have been created for capturing and storing data, existing software and hardware architectures are aimed towards specialized devices and require expensive high-performance systems. This thesis aims to create a framework that supports capture and monitoring of a variety of sensors and can be scaled to run on low and high- performance systems such as netbooks, laptops and desktop systems. The proposed architecture was tested using aural and visual sensors due to their availability and higher bandwidth requirements compared to other sensors. Four different portable computing devices were used for testing with a varied set of hardware capabilities. On each of the systems the same suite of tests were run to benchmark and analyze CPU, memory, network, and storage usage statistics. From the results it was shown that on all of these platforms capturing data from multiple video, audio and other sensor sources was possible in real-time. Performance was shown to scale based on several factors, but the most important were CPU architecture, network topology and data interfaces used. vi Contents Dedication :::::::::::::::::::::::::::::: iii Acknowledgments ::::::::::::::::::::::::: iv Abstract ::::::::::::::::::::::::::::::: v 1 Introduction ::::::::::::::::::::::::::: 1 2 Overview :::::::::::::::::::::::::::: 3 3 Related Work :::::::::::::::::::::::::: 5 3.1 Video Acquisition Systems . .5 3.2 Wireless Sensor Networks . 12 4 Methodology :::::::::::::::::::::::::: 14 4.1 Initial Prototype . 15 4.2 PC Based Solution . 16 4.3 Data Capture - Hardware . 18 4.3.1 Computer Platform . 18 4.3.2 Data Sources . 22 4.3.2.1 Analog Video Sources . 23 4.3.2.2 Digital Video Sources . 23 4.3.2.3 Additional Sensor Sources . 24 4.4 Capture Devices - Software . 27 4.4.1 User Interface . 27 vii 4.4.1.1 Application Modes . 27 4.4.1.2 Controls . 28 4.4.1.3 PhidgetSpatial Interface . 28 4.4.1.4 System Status . 29 4.4.2 DirectShow - Filters . 29 4.4.2.1 Audio/Video Capture Filter . 30 4.4.2.2 Video Stacking Filter . 31 4.4.2.3 Image Processing Filter . 32 4.4.2.4 MJPEG Compression Filter . 32 4.4.2.5 PhidgetSpatial Capture Filter . 35 4.4.2.6 GPS Capture Filter . 36 4.4.2.7 Video Streaming Filter . 36 4.4.2.8 AVI Mux/File Writer Filters . 36 4.4.3 DirectShow - Filter Graphs . 37 4.5 Monitoring Device . 37 5 Results and Analysis :::::::::::::::::::::: 39 5.1 Hardware Benchmarks . 39 5.1.1 Synthetic Tests . 39 5.1.2 Storage I/O Benchmark . 42 5.1.3 Networking Throughput Benchmark . 43 5.1.4 Battery Life Benchmark . 44 5.2 Framework Benchmark . 44 5.2.1 Uncompressed Audio/Video Capture Pipeline . 45 5.2.2 Compressed Analog Media Capture Pipeline . 46 5.2.2.1 Single Video Source . 46 5.2.2.2 Single Video and Audio Source . 51 5.2.3 Software Compressed Digital Media Capture Pipeline 55 5.2.3.1 Single Video Source . 56 viii 5.2.3.2 Single Video and Audio Source . 60 5.2.3.3 Dual Video Source . 65 5.2.4 HW Compressed Digital Media Capture Pipeline . 68 5.2.4.1 Single Video Source . 69 5.2.4.2 Dual Video Source . 71 5.2.5 Digital Media Capture with GPS Sensor Pipeline . 73 5.2.6 Digital Media Capture with PhidgetSpatial Sensor Pipeline . 75 5.3 Monitoring/Streaming Benchmarks . 78 5.4 Memory Usage Benchmarks . 81 5.4.1 Single Video Source . 81 5.4.2 Single Video and Audio Source . 84 5.5 Overview . 87 6 Conclusions ::::::::::::::::::::::::::: 90 6.1 Future Work . 90 6.1.1 Cloud Computing . 91 6.1.2 Leverage USB 3.0 . 91 6.1.3 GPU Accelerated Filters . 92 6.1.4 Integration of Additional Sensors . 92 6.1.5 Media Server Implementation . 92 6.1.6 ARM-Processor Support . 92 6.1.7 Security/Encryption . 93 Bibliography :::::::::::::::::::::::::::: 94 ix List of Tables 3.1 Vidboard frame rate performance[1] . .6 3.2 Estimated Performance of Single PC Capture[32] . 10 3.3 Video Capture Solutions - Summary of Features . 12 4.1 Operating System Comparisons . 16 4.2 Technical Specifications Comparison . 21 4.3 Common PC data interfaces and bandwidths. 23 4.4 Comparison of Video Compression Codecs - Data Rate . 34 5.1 wPrime Benchmark Results . 41 5.2 Battery Life Benchmark . 44 5.3 Uncompressed Audio/Video Capture Results . 45 5.4 Streaming Video Results . 79 x List of Figures 3.1 A typical ViewStation system [1] . .6 3.2 Block diagram of 3D Room Capture System [30]. .7 3.3 Block diagram of the synchronous, multi-channel, video recording system [39] . .8 3.4 Portable laptop-based system along with camera and power source. [2] . .8 3.5 Lei, et al.’s Software Architecture Component Diagram [31]9 4.1 LeopardBoard Development Board[10] . 15 4.2 ExoPC[12] . 19 4.3 Fujitsu LifeBook TH700[15] . 20 4.4 13” MacBook Pro - Windows 7[43] . 21 4.5 HP Pavilion Elite HPE-250F[20] . 22 4.6 Microsoft LifeCam Cinema - 5 MP Sensor[36] . 24 4.7 Microsoft LifeCam HD-5000 - 4 MP Sensor[37] . 24 4.8 DirectShow Camera Properties (Page 1) Filter . 25 4.9 DirectShow Camera Properties (Page 2) Filter . 25 4.10 PhidgetSpatial 3/3/3 Sensor[21] . 26 4.11 GPS Receiver Specifications[45] . 26 4.12 Capture Application - Main Screen . 27 4.13 PhidgetSpatial - User Interface . 29 4.14 DirectShow API Overview [35] . 30 4.15 Audio Filter Properties . 31 4.16 Video Filter Properties . 32 xi 4.17 OpenCV Overview [7] . 33 4.18 MJPEG Compression Algorithm [47] . 33 4.19 H.264 Encoder Overview [19] . 34 4.20 Basic DirectShow Graph . 37 4.21 Monitoring Device User Interface . 38 5.1 Prime95 - 1 Thread - Benchmark Results . 40 5.2 Prime95 - 2 Threads - Benchmark Results . 41 5.3 Prime95 - 3 and 4 Threads (TH700 Only) - Benchmark Results 42 5.4 ATTO Storage I/O Benchmark . 43 5.5 Single Analog Video Source - Preview Mode - DirectShow Graph . 46 5.6 Single Analog Video Source - Record Mode - DirectShow Graph . 47 5.7 Single Analog Video Source - Record and Preview Mode - DirectShow Graph . 47 5.8 ExoPC - CPU Usage - Single Analog Video Source . 48 5.9 MacBook Pro - CPU Usage - Single Analog Video Source . 49 5.10 TH700 - CPU Usage - Single Analog Video Source . 51 5.11 Single Analog Video and Audio Source - Preview Mode - DirectShow Graph . 52 5.12 Single Analog Video and Audio Source - Record Mode - DirectShow Graph . 52 5.13 Single Analog Video and Audio Source - Record and Pre- view Mode - DirectShow Graph . 53 5.14 ExoPC - CPU Usage - Single Analog Video and Audio Source 54 5.15 Macbook Pro - CPU Usage - Single Analog Video and Au- dio Source . 55 5.16 TH700 - CPU Usage - Single Analog Video and Audio Source 56 xii 5.17 Single Digital Video Source - Preview Mode - DirectShow Graph . 56 5.18 Single Digital Video Source - Record Mode - DirectShow Graph . 57 5.19 Single Digital Video Source - Record and Preview Mode - DirectShow Graph . 57 5.20 ExoPC - CPU Usage - Single Digital Video Source . 58 5.21 Macbook Pro - CPU Usage - Single Digital Video Source . 59 5.22 TH700 - CPU Usage - Single Digital Video Source . 60 5.23 Single Digital Video and Audio Source - Preview Mode - DirectShow Graph . 61 5.24 Single Digital Video and Audio Source - Record Mode - DirectShow Graph .