Ee 5359 Multimedia Processing

EE 5359 MULTIMEDIA PROCESSING

PROJECT PROPOSAL

Submitted by, Gokul Krishna Srinivasan,

UTA ID: 1000660817, email : [email protected]

PROJECT TOPIC:

Implementation, Comparison and Literature Review of Spatio-temporal and Compressed domains Object detection.

ABSTRACT:

Object detection in a moving video stream is playing a prominent role in every branch of science and research [1]. Objection detection or tracking is done by two different methods, namely, Spatio-temporal domain and Compression domain. This project will deal with both the domains in order to bring out the advantages and disadvantages of each and every method in terms of complexity in computations, efficiency etc. Along with that, a detailed literature survey will also be done on the same topic. Most image and video data are stored or transmitted after compression for efficiency. Processes like pattern detection and localization typically include the extra expense of decompressing the data since most image and video processing techniques require access to the original pixel values in the spatial domain. This project performs a statistical analysis of present object detection schemes in both spatio-temporal and compressed domains. The results of multiple object detection in spatio-temporal domain are compared to those of compressed domain object detection and various evaluation results are analyzed. The comparisons will be made in the context of complexity of algorithm, efficiency and application.

MOTIVATIONS:

Even though there are several research programs and papers that discuss object detection in both spatio-temporal [4 & 5] and compressed domains [1, 2, 3 & 6], there is not much work presented in evaluating the functionality of compressed domain object detection with that of a spatio-temporal detection. This project presents an evaluation of compressed domain object detection and spatio temporal object detection.

COMPRESSED DOMAIN OBJECT DETECTION:

The compressed domain approach exploits the encoded information like motion vectors, discrete cosine transform (DCT) [1] coefficients, and macroblock types which are generated as a compressed bit stream [1, 2, 3 & 6]. Compressed domain object detection can greatly reduce the computational complexity and make real-time or fast processing possible although the precision is not better than the spatial domain approach. The conventional compressed domain object detection algorithms uses motion vectors or DCT coefficients as resources in order to perform object detection and tracking [6]. These encoded data are not enough credible or insufficient to detect and track moving objects, but recent work uses an extra feature called vector-featured images that record moving regions and accumulate unmoving regions in which the moving objects are expected to exist after the current frame [6]. This method uses only motion vector estimate without any other information from the encoded bit stream. In the vector featured images there are five types of block regions [2]. Considering that the basic unit of a region in the vector-featured image is the same size as MPEG macroblocks.

The five different blocks are reference block Br, current block Bc, background block Bb, moving block Bm and unmoving block Bu. The algorithm consists of four different steps initial region extraction, moving region detection, unmoving region creation and updating, modification of vector featured regions [2]. The results obtained are comparable to real time spatial object detection algorithm. Figure 1 shows the initial region extracted from the motion vector values, in fig 2 the label fn-2, fn-1, and fn represent the corresponding frame and each block in the frame represents a macro block motion vector value. The initial regions are formed by one to one mapping of the previous and the next corresponding frame. INITIAL REGION EXTRACTION: Fig.1: Each block in the figure represents a macro block motion vector value, [2].

CREATING AND UPDATING UNMOVING REGION DURING OBJECT STOP:

Fig.2: Extraction of moving and unmoving regions, [2].

SPATIO-TEMPORAL OBJECT DETECTION: Spatio-temporal object detection is an integral part of many computer vision applications [4, 5 & 7]. A common approach is to perform background subtraction, which identifies moving objects from portion of a video frame that differs significantly from a background model. There are four main steps in a background subtraction algorithm [5], they are preprocessing, background modeling, foreground detection and data validation as shown in fig.3. Preprocessing consists of a collection of simple image processing tasks that change the raw input video into a format that can be processed by subsequent steps. Steps like intensity adjustments, smoothening are handled in preprocessing stage. In real-time systems frame size reduction is also done to speed up the process. The block diagram of a background detection algorithm in spatial domain is shown in fig. 3. Background modeling is at the heart of any background subtraction algorithm. Background modeling is broadly classified into recursive and non-recursive techniques [5].

Fig.3: Block diagram of spatio-temporal object detecion [4]

Non-recursive technique uses a sliding window method and it stores a buffer of the previous L video frames, and estimates the background image based on the temporal variation of each pixel within the buffer. Recursive technique is a non-adaptive technique and does not use past input frame information. Foreground detection just compares the input frame with the background model and uses a threshold value as in binary classification. Data validation is the process of improving the candidate foreground mask based on information obtained from outside the background model. All the background models have three main limitations, first, they ignore any correlation between neighboring pixels; second, the rate of adaption may not match the moving speed of the foreground objects; and third, non-stationary pixels from moving objects are easily mistaken as true foreground objects. These limitations of background model are reduced in foreground detection and the possibility of false detection is also reduced. Summary: The main objective of this project is to learn the concepts in object detection using compressed and spatio-temporal domains and to implement these concepts using MATLAB simulation. Based on the results obtained from the simulation, a comparison will be made between compression domain and spatio-temporal domain techniques. REFERENCES: 1. Qiya Z and Zhicheng L, “Moving object detection algorithm for H.264/AVC compressed video stream”, ISECS International Colloquium on Computing Communication, control and management, vol. 1, pp. 186-189, Sep, 2009.

2. Yokoyama T, Iwasaki T, and Watanabe T,” Motion vector based moving object detection and tracking in the MPEG Compressed Domain”, Seventh International Workshop on content based Multimedia Indexing, pp. 201-206, Digital Object Identifier: 10.1109/CBMI.2009.33, Aug, 2009

3. Kapotas K and Skodras A. N,” Moving object detection in the H.264 compressed domain”, International Conference on Imaging systems and techniques, pp. 325-328, , Digital Object Identifier: 10.1109/IST.2010.5548496, Aug, 2010

4. Sen-Ching S. C and Kamath C,” Robust techniques for background subtraction in urban traffic video” Center for Applied Scientific Computing, Lawrence Livermore National Laboratory, vol. 1, pp. 586-589, Jul 2004.

5. Elhabian S. Y, El-Sayed K. M,” Moving object detection in spatial domain using background removal techniques- state of the art”, Recent patents on computer science, Vol. 1, pp. 32-54, Apr, 2008.

6. Sukmarg O and Rao K. R,” Fast object detection and segmentation in MPEG compressed domain”, TENCON 2000, proceedings, vol. 3, pp. 364-368, Mar, 2000.

7. Thompson W. B and Ting-Chuen P,” Detecting moving objects”, International journal of computer vision, vol. 6, pp. 39-57, Jun, 1990.