DOCSLIB.ORG

Energy-Efficient Foreground Object Detection on Embedded Smart

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Energy-Efficient Foreground Object Detection on Embedded Smart University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln Faculty Publications from the Department of Electrical & Computer Engineering, Department Electrical and Computer Engineering of 2011 Energy-efficientor F eground Object Detection on Embedded Smart Cameras by Hardware-level Operations Mauricio Casares University of Nebraska-Lincoln, [email protected] Paolo Santinelli University of Modena, [email protected] Senem Velipasalar University of Nebraska-Lincoln, [email protected] Andrea Prati University of Modena, [email protected] Rita Cucchiara University of Modena and Reggio Emilia Follow this and additional works at: https://digitalcommons.unl.edu/electricalengineeringfacpub Part of the Electrical and Computer Engineering Commons Casares, Mauricio; Santinelli, Paolo; Velipasalar, Senem; Prati, Andrea; and Cucchiara, Rita, "Energy- efficientor F eground Object Detection on Embedded Smart Cameras by Hardware-level Operations" (2011). Faculty Publications from the Department of Electrical and Computer Engineering. 202. https://digitalcommons.unl.edu/electricalengineeringfacpub/202 This Article is brought to you for free and open access by the Electrical & Computer Engineering, Department of at DigitalCommons@University of Nebraska - Lincoln. It has been accepted for inclusion in Faculty Publications from the Department of Electrical and Computer Engineering by an authorized administrator of DigitalCommons@University of Nebraska - Lincoln. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 doi: 10.1109/CVPRW.2011.5981838 Energy-efficient Foreground Object Detection on Embedded Smart Cameras by Hardware-level Operations Mauricio Casares1, Paolo Santinelli2, Senem Velipasalar1, Andrea Prati2 and Rita Cucchiara2 1University of Nebraska-Lincoln, Dept. of Electrical Engineering [email protected], [email protected] 2University of Modena and Reggio Emilia, Modena Italy [email protected], [email protected] Abstract sources, such as computational power, memory and energy. Since battery-life is limited, and video processing tasks Embedded smart cameras have limited processing consume considerable amount of energy, it is essential to power, memory and energy. In this paper, we introduce have lightweight algorithms, and methodologies to increase two methodologies to increase the energy-efficiency and the the energy-efficiency of each camera. With the advances battery-life of an embedded smart camera by hardware- in hardware technology, embedded devices are becoming level operations when performing foreground object detec- more sophisticated. Embedded smart cameras are being tion. We use the CITRIC platform as our embedded smart equipped with general purpose processing units that allow camera. We first perform down-sampling at hardware level implementing sophisticated vision algorithms on these plat- on the micro-controller of the image sensor rather than per- forms. forming software-level down-sampling at the main micro- Fleck et al. [4] present a network of smart cameras for processor of the camera board. In addition, we crop an im- tracking people. They use commercial IP-based cameras, age frame at hardware level by using the HREF and VSYNC which consist of a CCD image sensor, a Xilinx FPGA and signals at the micro-controller of the image sensor to per- a Motorola PowerPC. Cameras communicate via Ethernet form foreground object detection only in the cropped search connections. Quaritsch et al. [7] employ smart cameras region instead of the whole image. Thus, the amount of data with multiple DSP processors for data processing. Bram- that is moved from the image sensor to the main memory berger et al. [1] presented a smart camera architecture reach- at each frame, is greatly reduced. Thanks to reduced data ing a processing power of 9600 MIPS with onboard mem- transfer, better use of the memory resources and not occu- ory of 784 MB. While this high-end platform provides suf- pying the main microprocessor with image down-sampling ficient capabilities for image processing, it requires an aver- and cropping tasks, we obtain significant savings in energy age power consumption of 35 Watts. consumption and battery-life. Experimental results show Wired or IP-based cameras have powerful processing ca- that hardware-level down-sampling and cropping, and per- pabilities and relatively high bandwidth for communication. 54.14% forming detection in cropped regions provide de- However, they have high power consumption and are larger 121.25% crease in energy consumption, and increase in in size. Many embedded vision platforms have been devel- battery-life compared to performing software-level down- oped more recently. The Cyclops [9] and MeshEye [5] plat- sampling and processing whole frames. forms have 7.3-MHz and 55-MHz processors, respectively. Thus, in both of these platforms the processing power is 1. Introduction very limited. The camera mote introduced by Kleihorst et al. [6] has an 84MHz XETAL-II SIMD processor, and Wireless embedded smart cameras are stand-alone units uses a higher resolution. The CMUcam2 [11] is a low- that combine sensing, processing and communication on cost embedded camera with 75MHz RISC processor and a single embedded platform. Rather than transferring all 384KB SRAM. Due to the limited memory and processing the data to a back-end server, they can process images, power, only low-level image processing can be performed. and extract data locally. Even though battery-powered em- Panoptes platform [3] hosts a 206-MHz processor, but has bedded smart cameras provide a lot of flexibility in terms high energy consumption. SensEye [8] is a multi-tier net- of camera quantities and placement, they have limited re- work of heterogeneous wireless nodes and cameras. Rinner 150 et al. [10] present a comparison of various smart camera sensor, a microprocessor, external memories and other sup- platforms. Kerhet et al. [14] employ a hybrid architecture porting circuits. The microprocessor PXA270 is a fixed- FPGA-MCU, where the processing load is distributed to in- point processor with a maximum speed of 624 MHz and crease the efficiency of the camera mote. Chen et al.[2] 256 KB of internal SRAM. The PXA270 is connected to a introduced the CITRIC camera mote that provides more 64 MB of SDRAM and 16MB of NOR FLASH. Attached computing power and tighter integration of physical com- to the camera board is a TelosB mote from Crossbow Tech- ponents while still consuming relatively little power. An nology with a maximum data rate of 250 Kbps. Omnivision sensor OV9655 is employed to capture frames. The down-sampling of the images is performed by software using the CITRIC API libraries. Casares et al. [12] introduced an algorithm for resource- efficient foreground object detection, and presented the sav- ings in processing time and energy consumption on CITRIC cameras. They obtained the savings in software-level. In this paper, our goal is to increase the energy-efficiency and the battery-life further by hardware-level operations when performing object detection. We present two methods to Figure 1. CITRIC camera: the wireless embedded smart camera achieve this: (i) rather than performing down-sampling and platform employed in the experiments. image cropping at the main microprocessor on the camera board, we perform these operations at the micro-controller 2.1. The Image Sensor of the OV9655; (ii) we crop an image frame at hardware The image sensor on the CITRIC camera is a OmniVi- level by using the HREF and VSYNC signals at the micro- sion OV9655 [13], which is a low voltage SXGA CMOS controller of the image sensor to perform foreground ob- image sensor with an image micro-controller on board. It ject detection only in the cropped search region instead of supports image sizes SXGA (1280 × 1024), VGA (640 × the whole image. Performing these functions at hardware 480), CIF (352 × 288), and any size scaling down from CIF level provides multiple advantages including savings in pro- to 40 × 30, and provides 8-bit/10-bit data formats [2]. cessing time and significant decrease in energy consump- tion. The amount of data, which is moved from the im- age sensor to the main memory at each frame, is greatly reduced. This, in turn, leads to significant savings in en- ergy consumption thanks to the better use of the mem- ory controller and the memory resources and not occupy- ing the main microprocessor with performing image down- sampling and cropping at software-level. We compare performing down-sampling and cropping by software using the CITRIC API libraries, and by hard- ware using the micro-controller of the image sensor. We present the experimental results showing the savings in processing time and energy consumption, and the gain in Figure 2. Interconnection of OV9655 and the Intel Quick Capture the battery-life when performing down-sampling and crop- Interface on ARM PXA270. ping operations at hardware-level, and processing only the cropped region of an image frame. Due to lack of com- The image sensor offers the full functionality of a cam- patibility between the device driver containing the existing era and image micro-controller on a single chip. There is kernel and the actual camera sensor, the results and savings a complete control over image quality. Formatting, output are provided based on a single frame. Currently, a new ker- data transfer and all required image processing functions nel patch is under evaluation to capture successive cropped are also programmable. The architecture and hardware in- frames with adaptive window sizes. terfaces of the OmniVision OV9655 are depicted in Fig. 2. The Serial Camera Control Bus (SCCB) interface is used to 2. The Embedded Smart Camera Platform program the sensor behavior by setting all the control reg- isters in the device. It is an Inter-Integrated Circuit (I2C) The wireless embedded smart camera platform em- compatible hardware interface. The Digital Video Port pro- ployed in our experiments is a CITRIC mote [2]. It consists vides a connection between the sensor and the CITRIC cam- of a camera board and a wireless mote, and is shown in Fig- era main processor PXA270.
Load more

Recommended publications
  • Energy Efficient and Programmable Architecture for Wireless Vision Sensor Node
    Thesis work for the degree of Doctor of Technology Sundsvall 2013 Energy Efficient and Programmable Architecture for Wireless Vision Sensor Node Muhammad Imran Supervisors: Prof. Mattias O’Nils Dr. Najeem Lawal Prof. Bengt Oelmann Faculty of Science, Technology and Media, Mid Sweden University, SE-851 70 Sundsvall, Sweden ISSN 1652-893X Mid Sweden University Doctoral Thesis 167 ISBN 978-91-87557-12-5 Akademisk avhandling som med tillstånd av Mittuniversitetet i Sundsvall framläggs till offentlig granskning för avläggande av teknologie doktors examen i elektronik tisdagden 22 October 2013, klockan 13:15 i sal M108, Mittuniversitetet Sundsvall. Seminariet kommer att hållas på engelska. Energy Efficient and Programmable Architecture for Wireless Vision Sensor Node Muhammad Imran © Muhammad Imran, 2013 Faculty of Science, Technology and Media Mid Sweden University, SE-851 70 Sundsvall Sweden Telephone: +46 (0)60 148561 Printed by Kopieringen Mittuniversitetet, Sundsvall, Sweden, 2013 ABSTRACT Wireless Vision Sensor Networks (WVSNs) is an emerging field which has attracted a number of potential applications because of smaller per node cost, ease of deployment, scalability and low power stand alone solutions. WVSNs consist of a number of wireless Vision Sensor Nodes (VSNs). VSN has limited resources such as embedded processing platform, power supply, wireless radio and memory. In the presence of these limited resources, a VSN is expected to perform complex vision tasks for a long duration of time without battery replacement/recharging. Currently, reduction of processing and communication energy consumptions have been major challenges for battery operated VSNs. Another challenge is to propose generic solutions for a VSN so as to make these solutions suitable for a number of applications.
    [Show full text]
  • VLSI Architectures for Digital Signal Processing on Energy-Constrained Systems-On-Chip
    VLSI Architectures for Digital Signal Processing on Energy-Constrained Systems-on-Chip A Dissertation Presented to the Faculty of the School of Engineering and Applied Science University of Virginia In Partial Fulfillment of the requirements for the Degree Doctor of Philosophy (Electrical and Computer Engineering) by Alicia Klinefelter August 2015 c 2015 Alicia Klinefelter Abstract The design of ultra-low power (ULP) integrated circuits for Systems-On-Chip (SoCs) requires consideration of both flexibility and robustness during low-energy operation. Due to their quadratic relationship, the strongest design knob for reducing energy on-chip is the supply voltage. By operating digital circuits in the subthreshold region, using a supply voltage that falls below the threshold of the device, designers can minimize energy per operation at the cost of a performance penalty. However, there are many applications with low throughput requirements that can benefit from these energy savings. For example, systems that process biomedical data such as ECG, EEG, and EMG require sampling and processing rates on the order of kHz, making subthreshold operation feasible. The result of these substantial energy improvements is an emerging application space for these ULP SoCs in batteryless sensor nodes capable of running on energy harvested from the surrounding environment alone. This work presents two versions of a highly integrated, flexible SoC platform targeted for Internet-of-Things (IoT) applications. These SoCs support multiple sensing modalities, extract information from data flexibly across applications, harvest and deliver power efficiently, and communicate wirelessly. The first version of the chip acquired ECG data, extracted the heart-rate, and transmitted the raw signal operating off of harvested energy while consuming 19µW.
    [Show full text]
  • (2009 A-SSCC).Indd
    7-2 IEEE Asian Solid-State Circuits Conference November 16-18, 2009 / Taipei, Taiwan CRISP-DS: Dual-Stream Coarse-grained Reconfigurable Image Stream Processor for HD Digital Camcorders and Digital Still Cameras Tsung-Huang Chen, Jason C. Chen, Teng-Yuan Cheng, and Shao-Yi Chien Media IC and System Lab Graduate Institute of Electronics Engineering and Department of Electrical Engineering National Taiwan University BL-421, 1, Sec. 4, Roosevelt Rd., Taipei 106, Taiwan Abstract— A 329mW 600M-Pixels/s dual-stream coarse- In this paper, a dual-stream coarse-grained reconfigurable grained reconfigurable image stream processor is implemented in image stream processor (CRISP-DS) is designed to support μm 2 TSMC 0.13 CMOS technology with a core size of 4.84mm . HD image processing with high power efficiency. It is charac- The reconfigurable pipelined processing element array architec- ture makes a good balance between computing performance and terized as follows. First, the reconfigurable pipelined process- flexibility with only 10Kb on-chip memory. Moreover, a new dual- ing element (PE) array architecture can provide a good balance stream architecture is proposed to improve the flexibility and between computing performance and flexibility with low on- hardware efficiency by processing two independent image streams chip memory cost. Second, the new concept of dual-stream with two-layer context switching, and an isolation technique is processing is introduced to improve both the performance and also proposed to improve the power consumption. Implementa- tion results show that it achieves 1.52 times power efficiency than flexibility. With PE isolation technique, CRISP-DS can achieve previous works and can meet the requirements of high-definition better power efficiency than previous works.
    [Show full text]
  • Design of an FPGA-Based Smart Camera and Its Application Towards Object Tracking
    Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and private study only. The thesis may not be reproduced elsewhere without the permission of the Author. Design of an FPGA-Based Smart Camera and its Application Towards Object Tracking A thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Electronics and Computer Engineering at Massey University, Manawatu, New Zealand Miguel Contreras 2016 Abstract Smart cameras and hardware image processing are not new concepts, yet despite the fact both have existed several decades, not much literature has been presented on the design and development process of hardware based smart cameras. This thesis will examine and demonstrate the principles needed to develop a smart camera on hardware, based on the experiences from developing an FPGA-based smart camera. The smart camera is applied on a Terasic DE0 FPGA development board, using Terasic’s 5 megapixel GPIO camera. The algorithm operates at 120 frames per second at a resolution of 640x480 by utilising a modular streaming approach. Two case studies will be explored in order to demonstrate the development techniques established in this thesis. The first case study will develop the global vision system for a robot soccer implementation. The algorithm will identify and calculate the positions and orientations of each robot and the ball. Like many robot soccer implementations each robot has colour patches on top to identify each robot and aid finding its orientation. The ball is comprised of a single solid colour that is completely distinct from the colour patches.
    [Show full text]
  • CMOS-3D Smart Imager Architectures for Feature Detection M
    IEEE JETCAS-SPECIAL ISSUE ON HETEROGENEOUS NANO-CIRCUTS AND SYSTEMS 1 CMOS-3D Smart Imager Architectures for Feature Detection M. Suarez,´ V.M. Brea, J. Fernandez-Berni,´ R. Carmona-Galan,´ G. Lin˜an,´ D. Cabello and A. Rodr´ıguez-Vazquez´ Fellow, IEEE, Abstract—This paper reports a multi-layered smart image dominate the market of area imagers, with more than 90% of sensor architecture for feature extraction based on detection the total share [10]. of interest points. The architecture is conceived for 3D IC The most important asset of CISs is the incorporation of technologies consisting of two layers (tiers) plus memory. The top tier includes sensing and processing circuitry aimed to intelligence on-chip [1]. Different levels of intelligence can be perform Gaussian filtering and generate Gaussian pyramids in contemplated. The lowest involves basically readout, control fully concurrent way. The circuitry in this tier operates in mixed- and error correction, and is the only one yet exploited by signal domain. It embeds in-pixel Correlated Double Sampling industry [11], [12]. Higher intelligence levels, as required to (CDS), a switched-capacitor network for Gaussian pyramid analyzing, extracting and interpreting the information con- generation, analog memories and a comparator for in-pixel ADC (Analog to Digital Conversion). This tier can be further split tained into images have been explored for years at academia into two for improved resolution; one containing the sensors and [13] - [25], but with scarce industrial impact [11]. From now another containing a capacitor per sensor plus the mixed-signal on we will refer to CISs with high-level intelligence attributes processing circuitry.
    [Show full text]
  • Investigation of Architectures for Wireless Visual Sensor Nodes
    Thesis work for the degree of Licentiate of Technology Sundsvall 2011 Investigation of Architectures for Wireless Visual Sensor Nodes Muhammad Imran Supervisors: Professor Mattias O’Nils Professor Bengt Oelmann Dr. Najeem Lawal Electronics Design Division, in the Department of Information Technology and Media Mid Sweden University, SE-851 70 Sundsvall, Sweden ISSN 1652-8948 Mid Sweden University Licentiate Thesis 66 ISBN 978-91-86694-45-6 Akademisk avhandling som med tillstånd av Mittuniversitetet i Sundsvall framläggs till offentlig granskning för avläggande av teknologie Licentiate examen i elektronik onsdagen den 10 Juni 2011, klockan 10:30 i sal O102, Mittuniversitetet Sundsvall. Seminariet kommer att hållas på engelska. Investigation of Architectures for Wireless Visual Sensor Nodes Muhammad Imran © Muhammad Imran, 2011 Electronics Design Division, in the Department of Information Technology and Media Mid Sweden University, SE-851 70 Sundsvall Sweden Telephone: +46 (0)60 148561 Printed by Kopieringen Mittuniversitetet, Sundsvall, Sweden, 2011 ABSTRACT Wireless visual sensor network is an emerging field which has proved useful in many applications, including industrial control and monitoring, surveillance, environmental monitoring, personal care and the virtual world. Traditional imaging systems used a wired link, centralized network, high processing capabilities, unlimited storage and power source. In many applications, the wired solution results in high installation and maintenance costs. However, a wireless solution is the preferred choice as it offers less maintenance, infrastructure costs and greater scalability. The technological developments in image sensors, wireless communication and processing platforms have paved the way for smart camera networks usually referred to as Wireless Visual Sensor Networks (WVSNs). WVSNs consist of a number of Visual Sensor Nodes (VSNs) deployed over a large geographical area.
    [Show full text]