From Single Image to List of Objects Based on Edge and Blob Detection 619
Total Page:16
File Type:pdf, Size:1020Kb
From Single Image to List of Objects BasedonEdgeandBlobDetection RafalGrycuk1, Marcin Gabryel1, Marcin Korytkowski1, Rafal Scherer1, and Sviatoslav Voloshynovskiy2 1 Institute of Computational Intelligence, Cz¸estochowa University of Technology Al. Armii Krajowej 36, 42-200 Cz¸estochowa, Poland {rafal.grycuk,marcin.gabryel,marcin.korytkowski, rafal.scherer}@iisi.pcz.pl http://iisi.pcz.pl 2 University of Geneva, Computer Science Department, 7 Route de Drize, Geneva, Switzerland http://sip.unige.ch Abstract. In this paper we present a new method for obtaining a list of interest objects from a single image. Our object extraction method works on two well known algorithms: the Canny edge detection method and the quadrilaterals detection. Our approach allows to select only the significant elements of the image. In addition, this method allows to filter out unnecessary key points in a simple way (for example obtained by the SIFT algorithm) from the background image. The effectiveness of the method is confirmed by experimental research. 1 Introduction Content-based image retrieval is one of the greatest challenges of present com- puter science. Effective browsing and retrieving images is required in many vari- ous fields of life e.g. medicine, architecture, forensic, publishing, fashion, archives and many others. In the process of image recognition users search through databases which consist of thousands, even millions of images. The aim can be the retrieval of a similar image or images containing certain objects [4][6]. Retrieving mechanisms use image recognition methods. This is a sophisticated process which requires the use of algorithms from many different areas such as computational intelligence [7][8][12][20][21][22][24][27][29][32], machine learning [23], mathematics [2][13] and image processing [11]. An interesting approach for CBIR, based on nonparametric estimates (see e.g. [14][25][26]) was presented in [10]. One of important aspects of content-based image retrieval is the ability to find specific objects in the image. This is particularly important in the case of large image databases which are searched for specific objects. Many ways to detect objects can be found in the literature and one of them is image segmentation. The goal of the image segmentation is to cluster pixels into salient image re- gions, i.e. regions corresponding to individual surfaces, objects, or natural parts L. Rutkowski et al. (Eds.): ICAISC 2014, Part II, LNAI 8468, pp. 617–627, 2014. c Springer International Publishing Switzerland 2014 618 R. Grycuk et al. of objects. There are many methods for image segmentation: thresholding [3], clustering methods [1], histogram-based methods [30], edge detection [28], stereo vision based methods [16] and many others. We propose a new method for ex- tracting objects from images. This method consists of two parts: the Canny edge detection method [5] and the method for quadrilaterals detection [17]. An im- portant element in our algorithm is the additional step which consists in filtering out the key points from the background. The result is an image with key points only in the most interesting areas of the image. Our method uses the SIFT (Scale-invariant feature transform) algorithm to detect and describe local features of an image. It was presented for the first time in [18] and is now patented by the University of British Columbia. For each key point which describes the local image feature, we generate a feature vector, that can be used for further processing. SIFT contains four main steps [15][19]: extraction of potential key points, selection of stable key points (resistant to change of scale and rotation), finding key point orientation immune to the image transformation, generating a vector describing the key point. SIFT key points contain 128 values. The paper consists of several sections. In the next sections we will present methods used in the proposed approach and a method for image segmentation algorithm. The last section presents the experimental results on an original soft- warewrittenin.NET. 2CannyEdgeDetection The Canny edge detector [5] is one of the most commonly used image processing methods for detecting edges. It takes as input a gray scale image, and produces as output an image showing the positions of tracked intensity discontinuities. The algorithm runs in 5 separate steps: 1. Noise reduction. The image is smoothed by applying an appropriate Gaus- sian filter. 2. Finding the intensity gradient of the image. During this step the edges should be marked where gradients of the image have large magnitudes. 3. Non-maxima suppression. If the gradient magnitude at a pixel is larger than those at its two neighbors in the gradient direction, mark the pixel as an edge. Otherwise, mark the pixel as the background. 4. Edge tracking by hysteresis. Final edges are determined by suppressing all edges that are not connected to genuine edges. The effect of the Canny operator is determined by parameters: – The width of the Gaussian filter used in the first stage directly affects the results of the Canny algorithm, – The thresholds used during edge tracking by hysteresis. It is difficult to give a generic threshold that works well on all images. From Single Image to List of Objects Based on Edge and Blob Detection 619 3 Blob Extraction Blob extraction is one of the basic methods of image processing. It allows to detect and extract a list of blobs (objects) in the image. Unfortunately, obtain- ing homogeneous objects from an image as a list of pixels is extremely com- plicated. Especially when we deal with a heterogeneous background, i.e. the objects containing multicolored background. There are many methods for ex- tracting objects (blobs) from images [9]. In this paper we use methods imple- mented in the AForge.NET library. These algorithms are described by Andrew Kirillov [17]. There are four types of the algorithms: Convex full, Left/Right Edges, Top/Bottom Edges, Quadrilateral. Fig.1 describes these blob detection methods. Figure 6A illustrates Quadrilaterals detection method. As can be seen, round edges of the objects are not detected correctly. Much better results are obtained by the Top/Bottom Edges algorithm (Figure 1C). Edges of objects are detected mostly correctly, with individual exceptions. The Left/Right Edges method behaves similarly (Fig. 1B). The last method has a problem with the detection of vertices inside figures, e.g. star type objects (Fig. 1D). Quadrilateral detection method can be described by the following steps: 1. Locate each separate object in the input image, 2. Find object edge pixels (methods: Top/Bottom, Left/Right), 3. Detect four corners of quadrilateral, Fig. 1. Comparison of methods for blob detection used in the AForge.NET library [17] 620 R. Grycuk et al. 4. Set distortionLimit. This value determines how different the object can be. Set 0 to detect perfect quadrilateral, 5. Check how well the analyzed shape fits into the quadrilateral with the as- sumed parameter (see Fig. 1D ). 6. Check each mean distance between given edge pixels and the edge of assumed quadrilateral, 7. If mean distance is not greater then distortionLimit, then we can assume that the shape is quadrilateral. 4 Proposed Method for Segmentation Algorithm In this section we present a new approach to segmentation. When we have a single image described by pixel matrix (e.g. RGB), we can segment it into many objects. This section describes a methodology of this process. The algorithm can be divided into several steps. First of all we need to create mathematical description of the input image. In this step we use the SIFT algorithm. In return we obtain a list of key points (whole image). The next step is to create edge detected image. In this step we use the Canny algorithm described in section 2. Another step detects blobs, with all their properties such as: center of gravity, area, mean color, blob rectangle, position. When we have detected these object contours as blobs, we can extract rectangles for these objects. The next step detects and saves key points in each rectangle. In other words, we check if the key point position lies in rectangle area, if so it is assigned to this rectangle. The last step extracts objects from the original (input) image and saves them as separate objects. This method allows to extract the list of objects from the image, not only as raw part of the image but also with a list of key points as- signed to these objects. In other words, on the output of our method we obtain the list of objects (raw part) with assigned key points (vectors) to each ob- ject [31]. Figure 2 describes steps of the algorithm, witch can be represented by the following pseudo code: The blob detection step generates lots of small blobs. Fig. 2. Block diagram of the proposed method From Single Image to List of Objects Based on Edge and Blob Detection 621 INPUT: Single image, singleInputImage OUTPUT: Segmented image, list of objects with descriptors assign to each object, descriptorsList, objectsList allDescriptors := Generate Descriptor(singleInputImage); edgedImage := Canny Edge Detector(singleInputImage, thesh, threshLinking); blobs := Detect Blobs(edgedImage); foreach blob ∈ blobs do if blob.Area >= minBlobArea then extractedBlob := Extract Blob(singleInputImage, startP osition, blob.W idth, blob.Heigth); objectsList.Add To List(extractedBlob); extractedDescriptor := Extract Descriptor(allDescriptors, startP osition, blob.W idth, blob.Heigth); descriptorsList.Add To List(extractedDescriptor); end end Algorithm 1. Segmentation steps The minBlobArea describes min area for extracted objects. All blobs with area equal or greater will be extracted. For blob detection process we use Quadrilater- als detection method (see section 3). The extraction process (blob and descriptor extraction) is realized simply by copying entire blob rectangle into newly created image (pixel by pixel, key point by key point). Algorithm stages were implemented in .NET C# using AForge and Emgu CV libraries.