DOCSLIB.ORG

Hardware Architectures for Infrared Pedestrian Detection Systems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

School of Engineering and the Built Environment Hardware Architectures for Infrared Pedestrian Detection Systems by Robert Walczyk A thesis submitted in partial fulfilment of the requirements of Edinburgh Napier University, for the award of Doctor of Philosophy May 2013 Contact: [email protected] This document was set with the help of KOMA-Script and LATEX Abstract Infrared pedestrian detection systems struggle with real-time processing performance. Known solutions are limited to either low resolution systems with basic functionality running at full frame rate, or software based techniques featuring higher detection rates with full set of features, however running only in off-line mode for statistical analysis. Here, a comprehensive solution for real-time pedestrian detection is described. This research project includes investigation of possible solutions, design, development and implementation of a pedestrian detection system, processing data from infrared video source in real-time. Design requirements include processing at full frame rate as well as low memory and system resource consumption. The memory utilization is one of the major concerns since high demand for memory resources is a critical aspect in most image processing applications. For the purpose of this task, a number of general purpose image processing techniques were revised, taking into consideration the suitability for infrared pedestrian detection. These tasks include background separation, acquisition noise removal and object de- tection through connected component labelling. They are discussed and addressed in individual chapters. Various techniques for background segmentation are discussed. A chronological review of popular techniques is provided. The proposed architecture for background subtrac- tion is based on selective running average for adaptive background model, supported by adaptive thresholding based on histogram calculation. In order to remove acquisition noise, a dual decomposed architecture was introduced, based on mathematical morphol- ogy and basic set theory definitions. It includes both erosion and dilation performed in a pipeline. For the purpose of object detection and feature extraction, a connected component labelling technique was employed, based on a single pass approach to fulfil real-time processing requirement. The system was implemented, verified and tested on XUP FPGA Development Board with Virtex-II Pro XC2VP30 chip from Xilinx. Details and limitation of the specific implementation are discussed. An overview of experimental pedestrian detection results is provided. The thesis concludes with system analysis and suggestions for future work. iii Acknowledgements I would like to express my gratitude to all the people who have contributed towards this research or completion of this thesis. In particular, I would like to give thanks to the following: To my university supervisors Dr T. David Binnie and Dr Alistair Armitage. For your advice and continued encouragement throughout this research. I appreciate your feedback and value the time you have spent on reading and correcting this thesis. To my line manager from Cambridge Silicon Radio, Dr Tim Clapp, for generous study-leave allowance and flexible working hours during last months of my study. To Stefan Maagh for valuable advice and effort when creating this thesis template. To Piotr Wojtczuk for your contribution to this project in various aspects. Also, for our long night FPGA-whisky sessions, which I consider to be one of best experiences during my academic years. To Alexander Balazs, for your contribution while working on the implementation of tracking algorithms on embedded processor core. It was a great experience for me to act as a teacher and supervisor. I am glad this cooperation was concluded with joint research paper presented at the 2011 IET Irish Signals and Systems Conference (ISSC 2011) in Dublin. To my family, including my parents and sister in law - Krystyna, Janusz and Karolina Ko lodziejczak, for your continuous support. To my mother and father Lucyna and Waldemar, for your effort into my life-long education. I hope, after all these years, you can be proud of me. Special thanks to my wife Ania and our little baby girl Zosia. This research project has affected your life as much as mine. I believe only you know how much effort I have made towards the completion of this thesis. There is no words to express my gratitude for your support, understanding and patience. Robert Walczyk iv Acknowledgements v I acknowledge that some of the work presented in this thesis has been presented or published at or in the following places: [1] R. Walczyk, A. Balazs, A. Armitage, T.D. Binnie, "System architectures for infrared pedestrian tracking,"in: IET Irish Signals and Systems Conference (ISSC 2011), June 2011, Dublin, Ireland [2] R. Walczyk, A. Armitage, T.D. Binnie, "Comparative Study on Connected Component Labeling Algorithms for Embedded Video Processing Systems," in: Proceedings of the International Conference on Image Processing, Computer Vi- sion, and Pattern Recognition (IPCV'10), July 2010, Las Vegas, NV, USA [3] R. Walczyk, A. Armitage, T.D. Binnie, "FPGA Implementation of Hot Spot Detection in Infrared Video," in: IET Irish Signals and Systems Conference (ISSC 2010), June 2010, Cork, Ireland [4] R. Walczyk, A. Armitage, T.D. Binnie, "FPGA Implementation of Pedestrian Detection System using Infrared Image Sequences," in: Faculty of Engineering & Computing Postgraduate Research Conference (FECCI 2010), May 2010, Edin- burgh, UK [5] R. Walczyk, A. Armitage, T.D. Binnie, "An Embedded Real-Time Pedestrian Detection System Using an Infrared Camera," in: IET Irish Signals and Systems Conference (ISSC 2009), June 2009, Dublin, Ireland Contents Abstract iii Acknowledgements iv List of Figuresx List of Tables xiii List of Abbreviations xiv 1 Introduction 16 1.1 Motivation .................................. 17 1.2 Research Objective.............................. 18 1.3 Contribution to Knowledge ......................... 19 1.4 Outline of the Thesis............................. 19 2 Image Processing for Infrared Pedestrian Detection 21 2.1 Infrared Detectors .............................. 22 2.1.1 Infrared Radiation.......................... 22 2.1.2 Thermal Sensors........................... 23 2.1.3 Thermographic Cameras....................... 24 2.1.4 Applications ............................. 25 2.2 Digital Image Processing........................... 25 2.2.1 Digital Image............................. 26 2.2.2 Raster Scan.............................. 27 2.2.3 Pixel Neighbourhood......................... 27 2.2.4 Pixel Connectivity.......................... 28 2.2.5 Connected Components....................... 28 2.2.6 Scan Mask .............................. 29 2.2.7 Labelling ............................... 30 2.2.8 Run Length Encoding (RLE).................... 30 vi Contents vii 2.3 Hardware Development and Implementation................ 31 2.3.1 Processing Unit............................ 31 2.3.2 Design Flow.............................. 34 2.3.3 Development Platform........................ 36 2.4 Pedestrian Detection............................. 37 2.4.1 Segmentation............................. 38 2.4.2 Object detection........................... 39 2.4.3 Classification and Tracking ..................... 40 2.5 Conclusions.................................. 40 3 Background Segmentation 41 3.1 Introduction.................................. 41 3.2 Review of Background Subtraction Techniques .............. 43 3.2.1 Intensity Thresholding........................ 44 3.2.2 Temporal Difference......................... 46 3.2.3 Background Modelling........................ 48 3.3 Algorithm Overview ............................. 55 3.3.1 Background Modelling........................ 56 3.3.2 Histogram Calculation........................ 56 3.3.3 Feedback Loop Control ....................... 58 3.3.4 Algorithm Pseudo Code....................... 60 3.4 Hardware Implementation.......................... 61 3.4.1 Architecture ............................. 61 3.5 Conclusions.................................. 64 4 Morphology Filtering 67 4.1 Introduction.................................. 68 4.2 Mathematical Morphology.......................... 68 4.2.1 Set Theory .............................. 69 4.2.2 Structuring Element......................... 71 4.2.3 Erosion ................................ 72 4.2.4 Dilation................................ 72 4.2.5 Opening and Closing......................... 73 4.2.6 Duality ................................ 75 4.2.7 Decomposition ............................ 75 4.2.8 Summary............................... 76 4.3 Implementation................................ 77 4.3.1 Introduction ............................. 77 Contents viii 4.3.2 Previous Work ............................ 78 4.3.3 System Requirements ........................ 79 4.3.4 Architecture ............................. 80 4.3.5 Module Instantiation......................... 85 4.3.6 Results and Performance ...................... 86 4.4 Conclusions.................................. 91 5 Connected Component Labelling 94 5.1 Introduction.................................. 95 5.1.1 Labelling ............................... 96 5.1.2 Label Collision............................ 96 5.1.3 Feature Extraction.......................... 97 5.2 Literature Review .............................. 98 5.2.1 Labelling Algorithms......................... 98 5.2.2 Summary............................... 101 5.3 Algorithm
Recommended publications
  • Linear Algebra and Numerical Techniques

    Linear Algebra and Numerical Techniques

    Appendix A Linear algebra and numerical techniques A.1 Matrix decompositions .............................646 A.1.1 Singular value decomposition .....................646 A.1.2 Eigenvalue decomposition .......................647 A.1.3 QR factorization ............................649 A.1.4 Cholesky factorization .........................650 A.2 Linear least squares ...............................651 A.2.1 Total least squares ...........................653 A.3 Non-linear least squares .............................654 A.4 Direct sparse matrix techniques .........................655 A.4.1 Variable reordering ...........................656 A.5 Iterative techniques ...............................656 A.5.1 Conjugate gradient ...........................657 A.5.2 Preconditioning .............................659 A.5.3 Multigrid ................................660 646 A Linear algebra and numerical techniques In this appendix, we introduce some elements of linear algebra and numerical techniques that are used elsewhere in the book. We start with some basic decompositions in matrix algebra, including the singular value decomposition (SVD), eigenvalue decompositions, and other matrix decompositions (factorizations). Next, we look at the problem of linear least squares, which can be solved using either the QR decomposition or normal equations. This is followed by non-linear least squares, which arise when the measurement equations are not linear in the unknowns or when robust error functions are used. Such problems require iteration to find a solution. Next, we look at direct solution (factorization) techniques for sparse problems, where the ordering of the variables can have a large influence on the computation and memory requirements. Finally, we discuss iterative techniques for solving large linear (or linearized) least squares problems. Good general references for much of this material include the work by Bjorck¨ (1996), Golub and Van Loan (1996), Trefethen and Bau (1997), Meyer (2000), Nocedal and Wright (2006), and Bjorck¨ and Dahlquist (2010).
  • Optimal Transport for Computer Graphics and Temporal Coherence of Image Processing Algorithms

    Optimal Transport for Computer Graphics and Temporal Coherence of Image Processing Algorithms

    No d’ordre NNT : 041 2018 HABILITATION À DIRIGER DES RECHERCHES DE L’UNIVERSITÉ DE LYON opérée au sein de l’Université Claude Bernard Lyon 1 École Doctorale ED01 École Doctorale InfoMaths Soutenue publiquement le 09/11/2018, par : Nicolas Bonneel Optimal Transport for Computer Graphics and Temporal Coherence of Image Processing Algorithms Devant le jury composé de : Julie DELON, Professeur, Université Paris Descartes Rapporteure Quentin MÉRIGOT, Professeur, Université Paris-Sud Rapporteur Filippo SANTAMBROGIO, Professeur, Université de Lyon Rapporteur Nicolas COURTY, Professeur, Université de Bretagne Sud Président Justin SOLOMON, Assistant Professor, Massachusetts Institute of Technology Examinateur 2 Remerciements Je remercie d’abord les rapporteurs et examinateurs de mon jury pour leur lecture attentive du manuscrit, les questions pertinentes lors de ma soutenance, mais aussi les riches discussions qui s’en sont suivies par email. Je remercie mes co-auteurs (j’ai pu en dénombrer une soixantaine que je ne listerai pas ici!), étudiants et collègues du LIRIS. En particulier, je ne serais pas arrivé au CNRS sans l’accueil qui m’a été fourni auparavant en master, thèse et postdoctorats, par Matthias Paulin, George Drettakis, Michiel van de Panne, Bruno Lévy et Hanspeter Pfister. Je remercie finalement mon père Stéphane pour son intérêt continuel, et ma femme Lucie pour m’avoir soutenu et accompagné lors de plusieurs années de vadrouille post-doctoresques en attente du Graal du poste permanent. Et je dédie ce manuscrit à Evan, notre fils, né le 19 Novembre 2017, presqu’un an avant cette soutenance. 3 4 Contents 1 Background 13 1.1 The Optimal Transport Problem ..........................
  • Abstracts Sitis 2012 Abstracts

    Abstracts Sitis 2012 Abstracts

    WG 2.6 Ernesto Damiani, Giuseppe De Pietro, Luigi Gallo Albert Dipanda, Richard Chbeir, Kokou Yetongnon ABSTRACTS SITIS 2012 ABSTRACTS Track Signal and Image Technology (SIT) SIT-S1: Video Processing and Coding 33 Adapting the Streaming Video Based on the Estimated Position of the Region of Interest Hussein Muzahim Aziz, Markus Fiedler, Håkan Grahn, Lars Lundberg [email protected] Keywords: streaming video, region of interest, sum of absolute differences, mean opinion scores Abstract Streaming real time video over wireless networks cannot guarantee that all the frames could meet their deadlines. Wireless networks may suffer from bandwidth limitations. To reduce the streaming data over wireless networks, we propose a technique to identify, and extract the Region Of Interest (ROI), and drop the non-ROI from the frames that are between the reference frames. The Sum of Absolute Differences (SAD) will compute the consecutive video frames to identify the ROI as it considered the most motion and important region. The reconstruction mechanism to the non-ROI is performed on the mobile side by applying linear interpolation between the reference frames. We evaluate the proposed approach by using Mean Opinion Score (MOS) measurements. MOS are used to evaluate the two scenarios with equivalent encoding size, where the users observe the first scenario with a low bit rate for the original videos, while for the second scenario the users observe our proposed approach. The results show that our technique significantly reduces the amount of data, while
  • Higher Level Techniques for the Artistic Rendering of Images and Video Submitted by John Philip Collomosse

    Higher Level Techniques for the Artistic Rendering of Images and Video Submitted by John Philip Collomosse

    Higher Level Techniques for the Artistic Rendering of Images and Video submitted by John Philip Collomosse for the degree of Doctor of Philosophy of the University of Bath 2004 COPYRIGHT Attention is drawn to the fact that copyright of this thesis rests with its author. This copy of the thesis has been supplied on the condition that anyone who consults it is understood to recognise that its copyright rests with its author and that no quotation from the thesis and no information derived from it may be published without the prior written consent of the author. This thesis may be made available for consultation within the University Library and may be photocopied or lent to other libraries for the purposes of consultation. Signature of Author . John Philip Collomosse Higher Level Techniques for the Artistic Rendering of Images and Video John Philip Collomosse i SUMMARY This thesis investigates the problem of producing non-photorealistic renderings for the purpose of aesthetics; so called Artistic Rendering (AR). Specifically, we address the problem of image-space AR, proposing novel algorithms for the artistic rendering of real images and post-production video. Image analysis is a necessary component of image-space AR; information must be ex- tracted from two-dimensional content prior to its re-presentation in some artistic style. Existing image-space AR algorithms perform this analysis at a \low" spatiotemporal level of abstraction. In the case of static AR, the strokes that comprise a rendering are placed independently, and their visual attributes set as a function of only a small image region local to each stroke.
  • Expanded Cinema

    Expanded Cinema

    EXPANDED CINEMA ARTSCILAB 2001 2 Blank Page ARTSCILAB 2001 Gene Youngblood became a passenger of Spaceship Earth on May 30, 1942. He is a faculty member of the California Institute of the Arts, School of Critical Studies. Since 1961 he has worked in all aspects of communications media: for five years he was reporter, feature writer, and film critic for the Los Angeles Herald-Examiner; in 1965 he conducted a weekly program on film and the arts for KPFK, Pacifica Radio in Los Angeles; in 1967 he wrote, produced, directed, edited, and on-camera reported "human interest" filmed news features for KHJ-TV in Los Angeles; since 1967 his column “Intermedia” has appeared weekly in the Los Angeles Free Press on subjects ranging from film and the arts to science, technology, and the cultural revolution. Mr. Youngblood currently is working on two books: The Videosphere, about global television in the 1970s as a tool for conscious evolution, and Earth Nova, a philosophical novel and screenplay about the new consciousness, the new lifestyle, and their relation to technology. ARTSCILAB 2001 4 Blank Page ARTSCILAB 2001 EXPANDED CINEMA by Gene Youngblood Introduction by R. Buckminster Fuller A Dutton Paperback P. Dutton & Co., Inc., New York 1970 ARTSCILAB 2001 Copyright © 1970 by Gene Youngblood Introduction and poem, "Inexorable Evolution and Human Ecology," copyright © 1970 by R. Buckminster Fuller All rights reserved. Printed in the U.S.A. First Edition No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system now known or to be invented, without permission in writing from the publishers, except by a reviewer who wishes to quote brief passages in connection with a review written for inclusion in a magazine or newspaper or broadcast.
  • 2011 18Th IEEE International Conference on Image Processing

    2011 18Th IEEE International Conference on Image Processing

    2011 18th IEEE International Conference on Image Processing (ICIP 2011) Brussels, Belgium 11 – 14 September 2011 Pages 1-740 IEEE Catalog Number: CFP11CIP-PRT ISBN: 978-1-4577-1304-0 1/5 Monday, September 12 MA.L1: Modern Shape from Shading and Beyond (Special Session) Modern Shape From Shading and Beyond Michael Breuss (Saarland University, Germany); Oliver Vogel (Saarland University, Germany); Ariel Tankus (Technion - Israel Institute of Technology, Israel) pp. 1-4 Numerical Schemes for Advanced Reflectance Models for Shape From Shading Oliver Vogel (Saarland University, Germany); Emiliano Cristiani (SAPIENZA - Universita di Roma, Italy) pp. 5-8 Break Shape From Shading with Specular Highlights: Analysis of the Phong Model Michael Breuss (Saarland University, Germany); Yong Chul Ju (Saarland University, Germany) pp. 9-12 Shape-from-shading Under Complex Natural Illumination Rui Huang (University of York, United Kingdom); William Smith (University of York, United Kingdom) pp. 13-16 Reconstruction of non-Lambertian Surfaces by Fusion of Shape From Shading and Active Range Scanning Steffen Herbort (Dortmund University of Technology, Germany); Arne Grumpe (Dortmund University of Technology, Germany); Christian Wöhler (Dortmund University of Technology, Germany) pp. 17-20 Shape From Specular Reflection in Calibrated Environments and the Integration of Spatial Normal Fields Jonathan Balzer (King Abdullah University of Science and Technology, Saudi Arabia) pp. 21-24 MA.L2: Structural Models (Lecture) Joint Pose Estimation and Action Recognition in Image Graphs Raja Kumar (IRISA & INRIA Rennes, France); Ivan Laptev (INRIA Paris - Rocquencourt, France); Patrick Pérez (INRIA, France); Lionel Oisel (Thomson, France) pp. 25-28 Inferring 3D Body Pose Using Variational Semi-parametric Regression Yan Tian (Hikvision Digital Technology Co.
  • Digital Video Processing

    Digital Video Processing

    DIGITAL VIDEO PROCESSING Second Edition This page intentionally left blank DIGITAL VIDEO PROCESSING Second Edition A. Murat Tekalp New York • Boston • Indianapolis • San Francisco Toronto • Montreal • London • Munich • Paris • Madrid Capetown • Sydney • Tokyo • Singapore • Mexico City Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The author and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. For information about buying this title in bulk quantities, or for special sales opportunities (which may include electronic versions; custom cover designs; and content particular to your business, training goals, marketing focus, or branding interests), please contact our corporate sales department at [email protected] or (800) 382-3419. For government sales inquiries, please contact [email protected]. For questions about sales outside the United States, please contact [email protected]. Visit us on the Web: informit.com/ph Library of Congress Cataloging-in-Publication Data Tekalp, A. Murat. Digital video processing / A. Murat Tekalp.—Second edition. pages cm Includes bibliographical references and index. ISBN 978-0-13-399100-0 (hardcover : alk. paper)—ISBN 0-13-399100-8 (hardcover : alk. paper) 1.
  • What to Submit And

    What to Submit And

    Digital Video Processing, 2E Table of Contents ©A. Murat Tekalp January 2015 TABLE OF CONTENTS Preface 1. MULTI-DIMENSIONAL SIGNALS AND SYSTEMS 1.1 Multi-dimensional Signals 1.1.1. Finite Extent Signals and Periodic Signals 1.1.2. Symmetric Signals 1.1.3. Special Multi-dimensional Signals 1.2 Multi-dimensional Transforms 1.2.1 Fourier Transform of Continuous Signals 1.2.2 Fourier Transform of Discrete Signals 1.2.3 Discrete Fourier Transform 1.2.4 Discrete Cosine Transform 1.3 Multi-dimensional Systems 1.3.1 Impulse Response and 2D Convolution 1.3.2 Frequency Response 1.3.3 FIR Filters and Symmetry 1.3.4 IIR Filters and Partial Difference Equations 1.4 Multi-dimensional Sampling Theory 1.4.1 Sampling on a Lattice 1.4.2 Spectrum of Signals Sampled on a Lattice 1.4.3 Nyquist Criterion for Signals Sampled on a Lattice 1.4.4 Reconstruction from Samples on a Lattice 1.5 Sampling Lattice Conversion References Exercises 2. DIGITAL IMAGES AND VIDEO 2.1 Human Visual System and Color 2.1.1 Color Vision and Models 2.1.2 Contrast Sensitivity 2.1.3 Spatio-Temporal Frequency Response 2.1.4 Stereo/Depth Perception 2.2 Analog Video 2.2.1 Progressive vs. Interlaced Scanning 2.2.2 Analog Video Signal Formats 2.2.3 Analog-to-Digital Video Conversion 1 Digital Video Processing, 2E Table of Contents ©A. Murat Tekalp January 2015 2.3 Digital Video 2.3.1 Spatial Resolution and Frame Rate 2.3.2 Color, Dynamic Range and Bit Depth 2.3.3 Color Image Processing 2.3.4 Digital Video Standards 2.4 3D Video 2.4.1 3D Display Technologies 2.4.2 Stereoscopic Video 2.4.3 Multi-view Video 2.5 Digital Video Applications 2.5.1 Digital TV 2.5.2 Digital Cinema 2.5.3 Video Streaming over the Internet 2.5.4 Computer Vision and Scene/Activity Understanding 2.6 Image and Video Quality 2.6.1 Visual Artifacts 2.6.2 Subjective Quality Assessment 2.6.3 Objective Quality Assessment References 3.