Thesis work for the degree of Licentiate of Technology Sundsvall 2011 Investigation of Architectures for Wireless Visual Sensor Nodes Muhammad Imran Supervisors: Professor Mattias O’Nils Professor Bengt Oelmann Dr. Najeem Lawal Electronics Design Division, in the Department of Information Technology and Media Mid Sweden University, SE-851 70 Sundsvall, Sweden ISSN 1652-8948 Mid Sweden University Licentiate Thesis 66 ISBN 978-91-86694-45-6 Akademisk avhandling som med tillstånd av Mittuniversitetet i Sundsvall framläggs till offentlig granskning för avläggande av teknologie Licentiate examen i elektronik onsdagen den 10 Juni 2011, klockan 10:30 i sal O102, Mittuniversitetet Sundsvall. Seminariet kommer att hållas på engelska. Investigation of Architectures for Wireless Visual Sensor Nodes Muhammad Imran © Muhammad Imran, 2011 Electronics Design Division, in the Department of Information Technology and Media Mid Sweden University, SE-851 70 Sundsvall Sweden Telephone: +46 (0)60 148561 Printed by Kopieringen Mittuniversitetet, Sundsvall, Sweden, 2011 ABSTRACT Wireless visual sensor network is an emerging field which has proved useful in many applications, including industrial control and monitoring, surveillance, environmental monitoring, personal care and the virtual world. Traditional imaging systems used a wired link, centralized network, high processing capabilities, unlimited storage and power source. In many applications, the wired solution results in high installation and maintenance costs. However, a wireless solution is the preferred choice as it offers less maintenance, infrastructure costs and greater scalability. The technological developments in image sensors, wireless communication and processing platforms have paved the way for smart camera networks usually referred to as Wireless Visual Sensor Networks (WVSNs). WVSNs consist of a number of Visual Sensor Nodes (VSNs) deployed over a large geographical area. The smart cameras can perform complex vision tasks using limited resources such as batteries or alternative energy sources, embedded platforms, a wireless link and a small memory. Current research in WVSNs is focused on reducing the energy consumption of the node so as to maximise the life of the VSN. To meet this challenge, different software and hardware solutions are presented in the literature for the implementation of VSNs. The focus in this thesis is on the exploration of energy efficient reconfigurable architectures for VSNs by partitioning vision tasks on software, hardware platforms and locality. For any application, some of the vision tasks can be performed on the sensor node after which data is sent over the wireless link to the server where the remaining vision tasks are performed. Similarly, at the VSN, vision tasks can be partitioned on software and the hardware platforms. In the thesis, all possible strategies are explored, by partitioning vision tasks on the sensor node and on the server. The energy consumption of the sensor node is evaluated for different strategies on software platform. It is observed that performing some of the vision tasks on the sensor node and sending compressed images to the server where the remaining vision tasks are performed, will have lower energy consumption. In order to achieve better performance and low power consumption, Field Programmable Gate Arrays (FPGAs) are introduced for the implementation of the sensor node. The strategies with reasonable design times and costs are implemented on hardware-software platform. Based on the implementation of the VSN on the FPGA together with micro-controller, the lifetime of the VSN is predicted using the measured energy values of the platforms for different processing strategies. The implementation results prove our analysis that a VSN with such characteristics will result in a longer life time. iii ACKNOWLEDGEMENTS First of all I am thankful to the Allah Almighty, the most merciful for giving me the strength and perseverance during this work and for blessing me with many great people who have been the greatest support in both my personal and professional life. I would, in particular, express my deepest regards and gratitude to my supervisor Prof. Mattias O’Nils for his guidance, support and encouragement throughout this work. Without his constant guidance and support this work would not have been possible. Every meeting with him gave me a new inspiration for work. I am thankful to Dr. Najeem Lawal for his support and motivation during this work. I would also like to thank Fanny Burman and Christine Grafström for the administrative support and Magnus Ödling for the IT support. Further, I am thankful to my friends and colleagues Khursheed Khursheed, Naeem Ahmad, Abdul Waheed Malik, Khurram Shahzad, Abdul Majid, Jawad Saleem, Dr. Benny Thörnberg, Suliman Abdalla, Mohammad Anzar Alam, Cheng Peng, Kotte Hari Babu, Radhika Ambatipudi, Muhammad Amir Yousaf, Mazhar Hussain, Xiaozhou Meng, Mikael Bylund, David Krapohl, Sebastian Bader and all the colleagues in the department for their discussions and cooperation. I would also like to express my gratitude to Mid Sweden University (miun), Higher Education Commission (HEC) Pakistan, Swedish Institute (SI) and Knowledge Foundation (KK) for their financial and administrative support. I am thankful to my sisters and brothers for all the blessing and prayers. Moreover, I humbly thank my parents and teachers for motivating me and giving me the inspiration for higher education. Finally, I am thankful to my wife Farri and my lovely daughter Aleena Imran who have been a constant source of support and strength during this work. Sundsvall, May 2011 Muhammad Imran v TABLE OF CONTENTS ABSTRACT..............................................................................................................III ACKNOWLEDGEMENTS.........................................................................................V TABLE OF CONTENTS .........................................................................................VII ABBREVIATIONS AND ACRONYMS ....................................................................XI LIST OF FIGURES ................................................................................................XIII LIST OF TABLES .................................................................................................. XV 1 INTRODUCTION ...............................................................................................1 1.1 APPLICATION OF WIRELESS VISION SENSOR NETWORKS ................................. 2 1.1.1 Industrial control and monitoring ....................................................... 2 1.1.2 Surveillance....................................................................................... 2 1.1.3 Environmental monitoring ................................................................. 3 1.1.4 Personal care .................................................................................... 3 1.1.5 Virtual Reality .................................................................................... 3 1.2 PROCESSING PLATFORMS............................................................................. 4 1.2.1 Hardware platforms........................................................................... 4 1.2.2 Software platforms ............................................................................ 5 1.2.3 Comparison of ASICs and Programmable Devices .......................... 6 1.3 PROBLEM DESCRIPTION................................................................................ 7 1.4 MAIN CONTRIBUTIONS.................................................................................. 8 1.5 THESIS OUTLINE .......................................................................................... 9 2 WIRELESS VISUAL SENSOR NETWORKS .................................................11 2.1 CHARACTERISTIC OF WIRELESS VISUAL SENSOR NETWORKS ........................ 12 2.1.1 Energy, bandwidth and processing requirements........................... 12 2.1.2 Data storage requirement ............................................................... 12 2.1.3 Delay requirement........................................................................... 13 2.1.4 Self-configuration ............................................................................ 13 2.1.5 Scalability ........................................................................................ 13 2.2 WIRELESS VISUAL SENSOR NODE............................................................... 13 2.3 TYPE OF WIRELESS VISUAL SENSOR NETWORKS .......................................... 14 2.3.1 Homogenous Visual sensor network............................................... 14 2.3.2 Heterogeneous visual sensor network ............................................ 14 2.4 PROCESSING IN THE WIRELESS VISUAL SENSOR NETWORK .......................... 16 2.4.1 Centralized smart camera network ................................................. 16 2.4.2 Distributed smart camera network .................................................. 17 3 ARCHITECTURES FOR VISUAL SENSOR NODES.....................................21 3.1 VISUAL SENSOR NODE ON SOFTWARE PLATFORM........................................ 21 3.1.1 Light-weight camera........................................................................ 21 vii 3.1.2 SensEye ..........................................................................................21 3.1.3 Stanford’s MeshEye ........................................................................22