SEGMENTATION AND HISTOGRAM GENERATION USING THE HSV COLOR SPACE FOR IMAGE RETRIEVAL Shamik Sural, Gang Qian and Sakti Pramanik Dept. of Computer Science and Engineering, 3115 Engineering Building, Michigan State University, East Lansing, MI 48824, USA. [email protected], {qiangang, pramanik}@cse.msu.edu ABSTRACT the Saturation. For each pixel we, therefore, choose either its Hue or the Intensity as the dominant feature based on We have analyzed the properties of the HSV (Hue, its Saturation. We then segment the image by grouping Saturation and Value) color space with emphasis on the pixels with similar features using the K-means clustering visual perception of the variation in Hue, Saturation and algorithm [3]. Intensity values of an image pixel. We extract pixel A standard way of generating a color histogram of an features by either choosing the Hue or the Intensity as the image is to concatenate ‘N’ higher order bits for the Red, dominant property based on the Saturation value of a Green and Blue values in the RGB space [11]. The pixel. The feature extraction method has been applied for histogram then has 23N bins, which accumulate the count both image segmentation as well as histogram generation of pixels with similar color. It is also possible to generate applications – two distinct approaches to content based three separate histograms, one for each channel, and image retrieval (CBIR). Segmentation using this method concatenate them into one [2]. Smith and Chang [8] have shows better identification of objects in an image. The used a color set approach to extract spatially localized histogram retains a uniform color transition that enables color information. Ortega et al [6] have used the HS co- us to do a window-based smoothing during retrieval. The ordinates to form a two-dimensional histogram where results have been compared with those generated using the each bin contains the percentage of pixels in the image RGB color space. that have corresponding H and S colors for that bin. We generate a one-dimensional histogram from the HSV space where a perceptually smooth transition of color is 1. INTRODUCTION obtained in the feature vector. This enables us to use a window-based smoothing of histograms so that similar We have done in-depth analysis of the visual properties of colors can be matched between a query and each of the the HSV color space and its usefulness in content based database images. image retrieval applications. In particular, we have We explain the HSV-based feature extraction and developed image segmentation and histogram generation image segmentation method in the next section and the applications using this color space – two important histogram generation technique in section 3. We then methods in CBIR [5,7]. present experimental results in section 4 and draw Segmentation is done to decompose an image into conclusions from our work in the last section of the paper. meaningful parts for further analysis, resulting in a higher- level representation of the image pixels like the 2. IMAGE SEGMENTATION USING FEATURES foreground objects and the background. In region-based FROM THE HSV COLOR SPACE CBIR applications, segmentation is essential for identifying objects present in a query image and each of 2.1. Analysis of the HSV Color Space the database images. Wang et al [12] have used the LUV A three dimensional representation of the HSV color values of a group of 4X4 pixels along with three features space is a hexacone, where the central vertical axis obtained by wavelet transform of the L component for represents the Intensity [9]. Hue is defined as an angle in determining regions of interest. Region-based retrieval has the range [0,2S] relative to the Red axis with red at angle also been used in the NeTra system [4] and the Blobworld 0, green at 2S/3, blue at 4S/3 and red again at 2S. system [1]. We segment color images using features Saturation is the depth or purity of the color and is extracted from the HSV space as a step in the region- measured as a radial distance from the central axis with based matching approach to CBIR. The HSV color space value between 0 at the center to 1 at the outer surface. For is fundamentally different from the widely known RGB S=0, as one moves higher along the Intensity axis, one color space since it separates out the Intensity (luminance) goes from Black to White through various shades of gray. from the color information (chromaticity). Again, of the On the other hand, for a given Intensity and Hue, if the two chromaticity axes, a difference in Hue of a pixel is Saturation is changed from 0 to 1, the perceived color found to be visually more prominent compared to that of changes from a shade of gray to the most pure form of the 0-7803-7622-6/02/$17.00 ©2002 IEEEII - 589 IEEE ICIP 2002 color represented by its Hue. Looked from a different based approximation can determine the intensity and angle, any color in the HSV space can be transformed to a shade variations near the edges of an object, thereby shade of gray by sufficiently lowering the Saturation. The sharpening the boundaries and retaining the color value of Intensity determines the particular gray shade to information of each pixel. This phenomenon is exhibited which this transformation converges. When Saturation is in detail in figure 3. Figure 3(a) shows a number of solid near 0, all pixels, even with different Hues, look alike and colors with varying intensities. Figure 3(b)-(c) shows the as we increase the Saturation towards 1, they tend to get result of approximation using the RGB color space taking separated and are visually perceived as the true colors the 2 higher order bits. It is seen that some of the colors represented by their Hues as shown in figure 1. Thus, for with high intensities cannot be recognized, as they are low values of Saturation, a color can be approximated by a inseparable from the background. Also, we see that the gray value specified by the Intensity level while for higher background of white and gray are considered equivalent Saturation, the color can be approximated by its Hue. The due to approximation. The HSV features used by us retain Saturation threshold that determines this transition is once the identity of the colors even at these intensity levels as again dependent on the Intensity. For low intensities, even seen in figure 3(d). This makes the HSV-based features for a high Saturation, a color is close to the gray value and very useful in running segmentation algorithms like vice versa. Saturation gives an idea about the depth of clustering on the approximated pixels. color and human eye is less sensitive to its variation compared to variation in Hue or Intensity. We, therefore, 2.3. Pixel Grouping by K-means Clustering Algorithm use the Saturation value of a pixel to determine whether The RGB value of a pixel is first transformed to the HSV the Hue or the Intensity is more pertinent to human visual value using a method suggested in [9]. The feature is next perception of the color of that pixel and ignore the actual extracted from each image pixel. After extraction, the value of the Saturation. It is observed that for higher pixel features are clustered using the K-Means clustering values of intensity, a saturation of 0.2 differentiates algorithm to group them into regions of similar color. between Hue and Intensity dominance. Assuming the Since the Hue and the Intensity values belong to the same maximum Intensity value to be 255, we use the following number space, the two sets of data are clustered separately threshold function to determine if a pixel should be so that the color and the gray value pixels are not represented by its Hue or its Intensity as its dominant considered in the same cluster. In the K-means clustering feature. algorithm, we start with K=2 and adaptively increase the 0.8V number of clusters till the improvement in error falls thsat(V) = 1.0 (1) 255 below a threshold or a maximum number of clusters is In the above equation, we see that for V=0, th(V) = reached. We set the maximum number of clusters to 12 1.0, meaning that all the colors are approximated as black and an error improvement threshold over number of whatever be the Hue or the Saturation. On the other hand, clusters to 5 %. with increasing values of the Intensity, Saturation threshold that separates Hue dominance from Intensity 3. HISTOGRAM GENERATION dominance goes down. We also use the HSV color space for histogram generation 2.2. Feature Generation using the HSV Color Space where each pixel contributes either its Hue or its Intensity We generate features by utilizing the above properties of as explained in the last section. We extract the color the HSV color space for clustering pixels into segmented histogram as the feature vector having two parts: (i) A regions. Figure 2(a) shows an image and figure 2(b) representation of the Hue between 0 and 2S quantized shows the same image using the approximated pixels after after a transformation and (ii) A quantized set of gray Saturation thresholding. Pixels with sub-threshold values as shown in figure 4(a). The number of Saturation have been represented by their gray values components in the feature vector generated based on Hue while the other pixels have been represented by their is given by: Hues. The feature generation used by us makes an Nh = ªº2ʌ MULT__FCTR approximation of the color of each pixel in the form of Here MULT_FCTR determines the quantization level thresholding.