Object Detection and Recognition in Digital Images THEORY AND PRACTICE Bogusław Cyganek OBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES OBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES THEORY AND PRACTICE Bogusław Cyganek AGH University of Science and Technology, Poland A John Wiley & Sons, Ltd., Publication This edition first published 2013 C 2013 John Wiley & Sons, Ltd Registered office John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com. The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. MATLABR is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This books use or discussion of MATLABR software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLABR software. Library of Congress Cataloging-in-Publication Data Cyganek, Boguslaw. Object detection and recognition in digital images : theory and practice / Boguslaw Cyganek. pages cm Includes bibliographical references and index. ISBN 978-0-470-97637-1 (cloth) 1. Pattern recognition systems. 2. Image processing–Digital techniques. 3. Computer vision. I. Title. TK7882.P3C94 2013 621.3994–dc23 2012050754 A catalogue record for this book is available from the British Library ISBN: 978-0-470-97637-1 Typeset in 10/12pt Times by Aptara Inc., New Delhi, India To my family with love Contents Preface xiii Acknowledgements xv Notations and Abbreviations xvii 1 Introduction 1 1.1 A Sample of Computer Vision 3 1.2 Overview of Book Contents 6 References 8 2 Tensor Methods in Computer Vision 9 2.1 Abstract 9 2.2 Tensor – A Mathematical Object 10 2.2.1 Main Properties of Linear Spaces 10 2.2.2 Concept of a Tensor 11 2.3 Tensor – A Data Object 13 2.4 Basic Properties of Tensors 15 2.4.1 Notation of Tensor Indices and Components 16 2.4.2 Tensor Products 18 2.5 Tensor Distance Measures 20 2.5.1 Overview of Tensor Distances 22 2.5.1.1 Computation of Matrix Exponent and Logarithm Functions 24 2.5.2 Euclidean Image Distance and Standardizing Transform 29 2.6 Filtering of Tensor Fields 33 2.6.1 Order Statistic Filtering of Tensor Data 33 2.6.2 Anisotropic Diffusion Filtering 36 2.6.3 IMPLEMENTATION of Diffusion Processes 40 2.7 Looking into Images with the Structural Tensor 44 2.7.1 Structural Tensor in Two-Dimensional Image Space 47 2.7.2 Spatio-Temporal Structural Tensor 50 2.7.3 Multichannel and Scale-Space Structural Tensor 52 2.7.4 Extended Structural Tensor 54 2.7.4.1 IMPLEMENTATION of the Linear and Nonlinear Structural Tensor 57 viii Contents 2.8 Object Representation with Tensor of Inertia and Moments 62 2.8.1 IMPLEMENTATION of Moments and their Invariants 65 2.9 Eigendecomposition and Representation of Tensors 68 2.10 Tensor Invariants 72 2.11 Geometry of Multiple Views: The Multifocal Tensor 72 2.12 Multilinear Tensor Methods 75 2.12.1 Basic Concepts of Multilinear Algebra 78 2.12.1.1 Tensor Flattening 78 2.12.1.2 IMPLEMENTATION Tensor Representation 84 2.12.1.3 The k-mode Product of a Tensor and a Matrix 95 2.12.1.4 Ranks of a Tensor 100 2.12.1.5 IMPLEMENTATION of Basic Operations on Tensors 101 2.12.2 Higher-Order Singular Value Decomposition (HOSVD) 112 2.12.3 Computation of the HOSVD 114 2.12.3.1 Implementation of the HOSVD Decomposition 119 2.12.4 HOSVD Induced Bases 121 2.12.5 Tensor Best Rank-1 Approximation 123 2.12.6 Rank-1 Decomposition of Tensors 126 2.12.7 Best Rank-(R1,R2, ... ,RP) Approximation 131 2.12.8 Computation of the Best Rank-(R1,R2, ... ,RP) Approximations 134 2.12.8.1 IMPLEMENTATION – Rank Tensor Decompositions 137 2.12.8.2 CASE STUDY – Data Dimensionality Reduction 145 2.12.9 Subspace Data Representation 149 2.12.10 Nonnegative Matrix Factorization 151 2.12.11 Computation of the Nonnegative Matrix Factorization 155 2.12.12 Image Representation with NMF 160 2.12.13 Implementation of the Nonnegative Matrix Factorization 162 2.12.14 Nonnegative Tensor Factorization 169 2.12.15 Multilinear Methods of Object Recognition 173 2.13 Closure 179 2.13.1 Chapter Summary 179 2.13.2 Further Reading 180 2.13.3 Problems and Exercises 181 References 182 3 Classification Methods and Algorithms 189 3.1 Abstract 189 3.2 Classification Framework 190 3.2.1 IMPLEMENTATION Computer Representation of Features 191 3.3 Subspace Methods for Object Recognition 194 3.3.1 Principal Component Analysis 195 3.3.1.1 Computation of the PCA 199 3.3.1.2 PCA for Multi-Channel Image Processing 210 3.3.1.3 PCA for Background Subtraction 214 3.3.2 Subspace Pattern Classification 215 Contents ix 3.4 Statistical Formulation of the Object Recognition 222 3.4.1 Parametric and Nonparametric Methods 222 3.4.2 Probabilistic Framework 222 3.4.3 Bayes Decision Rule 223 3.4.4 Maximum a posteriori Classification Scheme 224 3.4.5 Binary Classification Problem 226 3.5 Parametric Methods – Mixture of Gaussians 227 3.6 The Kalman Filter 233 3.7 Nonparametric Methods 236 3.7.1 Histogram Based Techniques 236 3.7.2 Comparing Histograms 239 3.7.3 IMPLEMENTATION – Multidimensional Histograms 243 3.7.4 Parzen Method 246 3.7.4.1 Kernel Based Methods 248 3.7.4.2 Nearest-Neighbor Method 250 3.8 The Mean Shift Method 251 3.8.1 Introduction to the Mean Shift 251 3.8.2 Continuously Adaptive Mean Shift Method (CamShift) 257 3.8.3 Algorithmic Aspects of the Mean Shift Tracking 259 3.8.3.1 Tracking of Multiple Features 259 3.8.3.2 Tracking of Multiple Objects 260 3.8.3.3 Fuzzy Approach to the CamShift 261 3.8.3.4 Discrimination with Background Information 262 3.8.3.5 Adaptive Update of the Classifiers 263 3.8.4 IMPLEMENTATION of the CamShift Method 264 3.9 Neural Networks 267 3.9.1 Probabilistic Neural Network 267 3.9.2 IMPLEMENTATION – Probabilistic Neural Network 270 3.9.3 Hamming Neural Network 274 3.9.4 IMPLEMENTATION of the Hamming Neural Network 278 3.9.5 Morphological Neural Network 282 3.9.5.1 IMPLEMENTATION of the Morphological Neural Network 285 3.10 Kernels in Vision Pattern Recognition 291 3.10.1 Kernel Functions 296 3.10.2 IMPLEMENTATION – Kernels 301 3.11 Data Clustering 306 3.11.1 The k-Means Algorithm 308 3.11.2 Fuzzy c-Means 311 3.11.3 Kernel Fuzzy c-Means 313 3.11.4 Measures of Cluster Quality 315 3.11.5 IMPLEMENTATION Issues 317 3.12 Support Vector Domain Description 327 3.12.1 Implementation of Support Vector Machines 333 3.12.2 Architecture of the Ensemble of One-Class Classifiers 334 3.13 Appendix – MATLABR and other Packages for Pattern Classification 336 3.14 Closure 336 x Contents 3.14.1 Chapter Summary 336 3.14.2 Further Reading 337 Problems and Exercises 338 References 339 4 Object Detection and Tracking 346 4.1 Introduction 346 4.2 Direct Pixel Classification 346 4.2.1 Ground-Truth Data Collection 347 4.2.2 CASE STUDY – Human Skin Detection 348 4.2.3 CASE STUDY – Pixel Based Road Signs Detection 352 4.2.3.1 Fuzzy Approach 353 4.2.3.2 SVM Based Approach 353 4.2.4 Pixel Based Image Segmentation with Ensemble of Classifiers 361 4.3 Detection of Basic Shapes 364 4.3.1 Detection of Line Segments 366 4.3.2 UpWrite Detection of Convex Shapes 367 4.4 Figure Detection 370 4.4.1 Detection of Regular Shapes from Characteristic Points 371 4.4.2 Clustering of the Salient Points 375 4.4.3 Adaptive Window Growing Method 376 4.4.4 Figure Verification 378 4.4.5 CASE STUDY – Road Signs Detection System 380 4.5 CASE STUDY – Road Signs Tracking and Recognition 385 4.6 CASE STUDY – Framework for Object Tracking 389 4.7 Pedestrian Detection 395 4.8 Closure 402 4.8.1 Chapter Summary 402 4.8.2 Further Reading 402 Problems and Exercises 403 References 403 5 Object Recognition 408 5.1 Abstract 408 5.2 Recognition from Tensor Phase Histograms and Morphological Scale Space 409 5.2.1 Computation of the Tensor Phase Histograms in Morphological Scale 411 5.2.2 Matching of the Tensor Phase Histograms 413 5.2.3 CASE STUDY – Object Recognition with Tensor Phase Histograms in Morphological Scale Space 415 5.3 Invariant Based Recognition 420 5.3.1 CASE STUDY – Pictogram Recognition with Affine Moment Invariants 421 5.4 Template Based Recognition 424 5.4.1 Template Matching for Road Signs Recognition