Pattern Recognition 40 (2007) 1182–1194 www.elsevier.com/locate/pr Multiscale directional filter bank with applications to structured and random texture retrieval K.-O. Cheng∗, N.-F. Law, W.-C. Siu Centre for Multimedia Signal Processing, Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, Hung Hom, Hong Kong Received 15 July 2005; received in revised form 16 June 2006; accepted 31 July 2006 Abstract In this paper, multiscale directional filter bank (MDFB) is investigated for texture characterization and retrieval. First, the problem of aliasing in decimated bandpass images on directional decomposition is addressed. MDFB is then designed to suppress the aliasing effect as well as to minimize the reduction in frequency resolution. Second, an entropy-based measure on energy signatures is proposed to classify structured and random textures. With the use of this measure for texture pre-classification, an optimized retrieval performance can be achieved by selecting the MDFB-based method for retrieving structured textures and a statistical or model-based method for retrieving random textures. In addition, a feature reduction scheme and a rotation-invariant conversion method are developed. The former is developed so as to find the most representative features while the latter is developed to provide a set of rotation-invariant features for texture characterization. Experimental works confirm that they are effective for texture retrieval. ᭧ 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Texture characterization; Texture retrieval; Directional filter bank; Multiscale directional filter bank; Rotation-invariant features 1. Introduction filtering approaches including wavelet [10,11], Gabor fil- ters [1,12], steerable pyramid [13] and directional filter bank As texture is one of the basic attributes of natural images, (DFB) [14] characterize textures in the frequency domain. texture analysis has attracted much attention on areas such Among the three categories, MPEG-7 has adopted Gabor- as computer vision, content-based image retrieval (CBIR), like filtering for the texture description [15]. The rationale remote sensing, medical imaging and quality inspection, behind is that visual cortex is sensitive to localized fre- etc. In particular, much research [1–4] has been done on quency components [16]. It has been shown that the direc- texture description for CBIR so as to manage the continu- tion together with scale information is important for texture ously growing multimedia data. The commonly used meth- perception. ods for texture characterization can be divided into three The filtering schemes, such as Gabor filters and steerable categories; statistical, model-based and filtering approaches pyramid, are developed for image analysis in a multiple [5]. Statistical methods such as co-occurrence features [6,7] scale and direction manner. Although Gabor filters and describe the tonal distribution in textures. Model-based steerable pyramid provide higher angular resolution than methods such as Markov random field (MRF) [8] and si- the wavelet transform, they are overcomplete in both scale multaneous autoregressive (SAR) models [9] provide a and directional decomposition. This in turn implies that they description of texture in terms of spatial interaction while are less computationally efficient than the wavelet approach [17]. For directional decomposition, DFB [18–20] has been ∗ Corresponding author. Tel.: +852 2766 6201; fax: +852 2362 8439. proposed as a highly computationally efficient tool. DFB E-mail addresses: [email protected] (K.-O. Cheng), is maximally decimated and so is not overcomplete. One ennfl[email protected] (N.-F. Law), [email protected] (W.-C. Siu). of the main disadvantages of DFB is the lack of multiscale 0031-3203/$30.00 ᭧ 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2006.07.014 K.-O. Cheng et al. / Pattern Recognition 40 (2007) 1182–1194 1183 property. Recently, pyramidal directional filter banks (PDFB) or contourlet transform [21,22] have been pro- posed to solve this problem by combining the DFB with Laplacian pyramid (LP) [23]. Although the LP is somehow redundant, the combined approach is still computationally efficient while providing a high angular resolution. In Ref. [24], the PDFB is modified as multiscale direc- tional filter bank (MDFB) to have a fine high-frequency de- composition. In both the PDFB and MDFB, various lowpass filters can be used in the LP while still maintaining perfect reconstruction. Usually, the filters have the stopband edge greater than /2, e.g. quadrature mirror filter (QMF) and “9-7” biorthogonal filter. However, the use of these lowpass filters has a shortcoming that aliasing occurs in the pass- bands after downsampling, i.e. those at scales other than the first one. When the DFB is applied on the decimated low- pass image, the aliasing components will be decomposed at the same time. However, the orientations of the aliasing components can be very different from those of non-aliasing components for some directional subbands [25]. Therefore, further analysis should be performed to study and remove Fig. 1. Frequency partitioning in DFB. the aliasing effect so as to improve the use of MDFB for texture characterization. Besides the use of lowpass filters in the LP, there are still in Section 2. Then Section 3 addresses the aliasing problem many issues about the use of MDFB for texture character- associated with the directional decomposition of bandpass ization. In particular, the retrieval performance for random images in the LP. We will also describe a way to adapt the textures should be improved since most features commonly bandlimiting constraint in Refs. [17,25] to the MDFB so as extracted from filtering approaches lack statistical descrip- to alleviate the aliasing problem. In Section 4, a measure of tion. One way to enhance the retrieval performance of texture regularity based on the MDFB features is proposed. the MDFB is to unify it with statistical or model-based With the use of this measure, a hybrid system combining the approaches. It has been shown in Refs. [4,26] that the MDFB and the model-based algorithm is then developed. unified approaches can take the advantages of filtering A feature reduction method targeted for structured textures approaches for structured textures and statistical or model- is also developed so as to speed up the searching. Section 5 based approaches for random textures. Thus, we will first describes the development of the rotation-invariant MDFB study ways for characterizing structured and random tex- features. Finally, Section 6 concludes the paper. tures using MDFB. After this pre-classification, MDFB will be used for retrieval of structured textures while model- based approach will be utilized to retrieve random textures. 2. Background Furthermore, feature reduction and rotation-invariance are often concerned in practice. The retrieval time usually in- 2.1. Directional filter bank creases with the number of features so that features used for texture description should be as few as possible while The DFB [18–20] performs directional decomposition maintaining good retrieval accuracy. On the other hand, the with partitioning of a frequency plane in wedge-shaped query texture image provided by a user may have different regions as shown in Fig. 1. The DFB is computationally ef- orientations from the texture images stored in the databases ficient due to its tree structure. In the first two stages of the of the retrieval system. Therefore, the features representing tree structure, the DFB has a two-band filter bank structure the texture images should be insensitive to rotation. We stud- given in Fig. 2(a). The two-band filter bank consists of two ied ways to reduce the number of features as well as use of H 2-D() H 2-D() complementary fan-shaped filters 0 and 1 . rotation-invariant features for texture description. To split the input spectrum into two wedge-shaped regions, In this paper, we focus on various issues associated with the input signal is fed into the respective filters for de- the use of the MDFB for texture characterization and re- composition. Two directional subbands are obtained after trieval. This includes the aliasing problem, the combination decimation on a quincunx lattice using the downsampling of MDFB and model-based texture description algorithm, 1 −1 11 matrix, Q = or Q = . feature reduction for structured textures and the develop- 0 11 1 −11 ment of a rotation-invariant texture description. This paper For stages subsequent to the second stage, a two- is organized as follows. Background about MDFB is given band filter bank shown in Fig. 2(b) is used. This kind of 1184 K.-O. Cheng et al. / Pattern Recognition 40 (2007) 1182–1194 Fig. 3. System block diagram for construction of each level in LP, where 2-D hL [n] is the 2-D lowpass filter implemented using the 1-D filter hL[n] through separable filtering. Fig. 2. Two-channel fan-shaped filter bank (a) for the first two stages and (b) for the third and later stages. two bandpass images. The lowpass image is not consid- ered for texture characterization. This is due to the non-ideal two-band filter bank still uses two complementary fan- frequency responses in real implementation which causes shaped filters for spectrum splitting. Since the next level uneven distribution of low-frequency components into dif- directional components have a parallelogram-shaped sup- ferent directional subbands [14]. Another reason for omitting port, resampling using some kind of unimodular matrix the lowpass image is that textures are often well character- Ri is required to change the support into fan-shaped for ized by the high- and mid-frequency components [28]. h [n] decomposition. There are four kinds of unimodular matri- In the MDFB, a 1-D lowpass filter l of cut-off fre- 11 1 −1 10 quency l is firstly applied on the input image on rows and ces, R = , R = , R = and 0 01 1 01 2 11 columns separately.