Omni-Directional Image Processing for Human Detection and Tracking
Total Page:16
File Type:pdf, Size:1020Kb
Brno University of Technology Faculty of Information Technology Department of Computer Graphics and Multimedia Omni-directional image processing for human detection and tracking A THESIS SUBMITTED IN FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY . Ing. Igor Potú ček Supervised by: Doc.Dr.Ing.Pavel Zem čík Submitted: June 2006 State doctoral exam: 14.June 2004 Availability: Library of the Faculty of Information Technology, Brno University of Technology, Czech Republic Omni-directional image processing for human detection and tracking Abstract This work investigates the setup and the set of methods of omni-directional system for human activity detection and tracking. Research in the area of the omni-directional systems is growing, the reason are many advantages such as low cost, portability and easy installation. The main advantage is capturing a large portion of a space angle, which is 360 degrees. On the other hand, a lot of problems exist and they must be solved. Images from omni-directional systems have different properties than standard perspective images as they have lower effective resolution and suffer adverse image distortion. If the resulting image is to be presented to a human or is further processed, transformation and suitable kind of corrections must be done. Techniques for image transformation into a perspective or panoramic view and geometrical corrections are suggested in this paper. The mobile catadioptric system is usually prone to vibrations, which cause the distortion in the transformed panoramic image. Therefore the novel approach for stabilization of the image from omni-directional system was proposed. Human face and hands detection plays an important role in applications such as video surveillance, human computer interface, face recognition, etc. The wide view angle is important for this kind of tasks and therefore the omni-directional system is suitable for these purposes. Two different tracking methods are compared on the various kinds of video sequences captured by omni-directional system in order to demonstrate the benefits and/or drawbacks of the omni-directional system and proposed methods. The evaluation scheme was developed for qualitative and quantitative algorithm description and comparison on different video sources. - 1 - Omni-directional image processing for human detection and tracking Keywords: Image Processing, Computer Vision, Catadioptric System, Omni-directional Image, Mirrors, Perspective Transformation, Panoramic Transformation, Edge Detection, Sub-pixel detection, Skin Color, Tracking, Human Body Parts Detection, Tracking Evaluation. - 2 - Omni-directional image processing for human detection and tracking Contents 1 INTRODUCTION ..............................................................................................................4 2 OMNI-DIRECTIONAL SYSTEM AND IMAGE PROCESSING................................8 2.1 IMAGE REPRESENTATION AND ACQUISITION .............................................................................. 8 2.2 PANORAMIC SENSORS ............................................................................................................... 10 2.3 THE HISTORY OF PANORAMIC SENSORS ................................................................................... 11 2.4 STATE OF THE ART OF THE OMNIDIRECTIONAL SYSTEMS ......................................................... 14 2.5 TYPES OF CENTRAL CATADIOPTRIC CAMERAS ......................................................................... 15 2.6 MIRROR DESIGN ....................................................................................................................... 16 2.7 CATADIOPTRIC SYSTEM DESCRIPTION ...................................................................................... 19 2.8 SYSTEM CALIBRATION ............................................................................................................. 21 2.9 PERSPECTIVE CAMERA CALIBRATION ...................................................................................... 21 2.10 SIMPLE UNWRAPPING ............................................................................................................... 24 2.11 GEOMETRIC IMAGE FORMATION .............................................................................................. 25 2.12 IMAGE QUALITY DESCRIPTION ................................................................................................. 33 2.13 OVERVIEW OF RELEVANT IMAGE PROCESSING METHODS ........................................................ 36 2.14 EDGE FINDING .......................................................................................................................... 37 2.15 MODIFIED HOUGH TRANSFORMATION FOR CIRCLE DETECTION .............................................. 40 2.16 RANSAC ................................................................................................................................. 41 3 PROPOSED METHODS FOR OMNIDIRECTIONAL VISION SYSTEM..............44 4 ACHIEVEMENT OF OBJECTIVES.............................................................................48 4.1 PARAMETER ESTIMATION OF THE MIRROR PROJECTION ........................................................... 48 4.2 ONE -DIRECTIONAL EDGE DETECTION ....................................................................................... 48 4.3 SUB -PIXEL EDGE DETECTION .................................................................................................... 51 4.4 PERSPECTIVE RECONSTRUCTION FROM NON -CENTRAL OMNI -DIRECTIONAL IMAGE ............... 53 5 EXPERIMENTS AND RESULTS..................................................................................56 5.1 SIMPLE VS . GEOMETRIC TRANSFORMATION ............................................................................. 57 5.2 DISTANCE INFLUENCE .............................................................................................................. 59 5.3 COMPARISONS BETWEEN PERSPECTIVE PROJECTION FROM DIFFERENT MIRROR TYPES .......... 60 5.4 STABILIZATION ALGORITHM - ONE DIRECTIONAL EDGE DETECTION ....................................... 63 5.5 STABILIZATION ALGORITHM - SUB -PIXEL EDGE DETECTION .................................................... 64 5.6 PROBLEMS OCCURRED IN DETECTION PROCESS ....................................................................... 66 5.7 OTHER DETECTION METHODS ................................................................................................... 67 6 APPLICATION OF PROPOSED METHODS .............................................................70 6.1 DATA COLLECTION ................................................................................................................... 70 6.2 EVALUATION PROCEDURE ........................................................................................................ 71 6.3 DETECTION AND TRACKING METHODS ..................................................................................... 74 6.4 EVALUATION RESULTS ............................................................................................................. 78 7 CONCLUSIONS...............................................................................................................87 APPENDIX A..........................................................................................................................94 APPENDIX B..........................................................................................................................97 - 3 - Omni-directional image processing for human detection and tracking 1 Introduction Seeing is not a simple process: it is just that vision has evolved over millions of years, and there was no particular advantage in evolution giving us any indication of the difficulties of the task. If anything, to have done so would have cluttered our minds with worthless information and quite probably slowed our reaction times in crucial situations. The humans are now trying to get machines to do much of their work. For simplest tasks there should be no particular difficulty in mechanization, but for more complex tasks the machine must be given our prime sense, that of vision. Efforts have been made to achieve this, sometimes in modest ways, for well over 30 years. There is in fact a great variety of applications for artificial vision systems – including, of course, all of those for which we employ our visual senses. The pace of acquiring information has been lately increasing exponentially. That is the reason, why they occur in such areas, where no one presumed their utilization. One of such categories are the meeting recognition systems, which can be included into multi-party interaction domain. Meetings play a critical role in the everyday life of organizations, work or research groups. In order to retain the salient points for later reference, meeting minutes are usually taken. In addition, people are often only peripherally interested in a meeting; they want to know what happened during it without actually attending. The ability to browse and skim these types of meetings could be quite valuable. Meeting records are intended to overcome problems such as poor attention and memory. This branch is mentioned with aim of capturing such situations. The monitoring of the meetings usually requires several cameras to capture the whole scene with each participant. Conventional cameras have a relatively narrow field of view. It could for instance use a pan-tilt-zoom mechanism to aim the camera in different