GENERIC IMAGE SIMILARITY BASED ON KOLMOGOROV COMPLEXITY Nima Nikvand and Zhou Wang Dept. of Electrical & Computer Engineering, University of Waterloo, Waterloo, ON, Canada Email: [email protected], [email protected] ABSTRACT in the class [3]. A practically more useful metric, namely the normalized information distance (NID), was introduced in Image similarity measurement is a fundamental and com- [4]. To overcome the non-computable nature of Kolmogorov mon issue in a broad range of problems in image process- complexity and NID, an normalized compression distance ing, compression, communication, recognition and retrieval. (NCD) measure was proposed [4], which is an effective Existing image similarity measures are limited to restricted approximation of NID and has found many successful appli- application environments. The theory of Kolmogorov com- cations in the fields of bioinformatics, pattern recognition, plexity and the related normalized information distance (NID) and natural language processing. measure provide an attractive theoretic framework for generic Nevertheless, the application of NID for image similarity image similarity that is applicable to any scenario. While this measurement is still in its early stage. Several authors have is appealing, the difficulty lies in the implementation due to done pioneering work that applied the NID framework and the the non-computable nature of Kolmogorov complexity. In NCD algorithm to image clustering [5], image distinguisha- this paper, we propose a practical framework to approximate bility [6], content-based image retrieval [7] and video classifi- NID, where the key is to find the shortest program within a cation problems [8], but most of them reported only moderate set of potential transformations that convert one image to an- success. Moreover, due to their focuses on specific applica- other and vice versa. As one of the initial attempts in this new tions, the generic property of NID was not fully exploited. and promising research direction, our preliminary experimen- In this paper, we attempt to develop a practical framework tal work demonstrates the wider applicability of the proposed that can effectively approximate NID, where the most critical approach than existing methods. step is to find the shortest program within a list of available Index Terms— image similarity measurement, Kol- transformations that convert one image to another. Based on mogorov complexity, normalized information distance, com- the framework, generic image similarity measures can be built pression distance that have much wider applicability than existing image simi- larity measures. 1. INTRODUCTION 2. KOLMOGOROV COMPLEXITY AND Measuring the similarity between two images is a funda- NORMALIZED INFORMATION DISTANCE mental issue in many problems throughout the entire field of image processing and machine vision. These include image The Kolmogorov complexity [3] of an object is defined to be restoration, denoising, coding, communication, interpolation, the length of the shortest program that can produce that object registration, fusion, classification and retrieval, as well as on a universal Turing machine and halt: object detection, recognition and tracking. Many existing K(x) = min l(p) : (1) image similarity measures were proposed to work with very p:U(p)=x specific types of image distortions (e.g., JPEG compression) [1]. There are also methods such as the structural similarity In [4], the authors assume the existence of a general decom- (SSIM) index [2] that are applicable to a wider range of ap- pressor that can be used to decompress the presumably short- plications. However, even these “general-purpose” methods est program x∗ to the desired object x. However, they note [1] are still limited in their application scopes. For example, that due to the non-computability of this concept, a compres- SSIM does not apply or work properly when significant geo- sor that does the opposite does not have to exist. metric changes (e.g., enlargement by a factor of 2, or rotation The conditional Kolmogorov complexity of x relative to by 90 degrees) exist between the two images being compared. y is denoted by K(xjy). An information distance between x The theory of Kolmogorov complexity provides solid and y can then be defined as maxfK(xjy; K(yjx)g, which groundwork to build a universal and generic information dis- is the maximum of the length of the shortest program that tance metric between any objects that minorizes all metrics computes x from y and y from x. To convert it to a normalized p symmetric metric, a novel NID measure was introduced in associated with a parameter compressor, and Ci denotes the [4]: parameter compressor of the i-th transformation. We can then maxfK(xjy∗);K(yjx∗)g define our conditional compressor as NID(x; y) = : (2) maxfK(x);K(y)g p CT (yjx) = minfC[y − Ti(x)] + Ci [p(Ti; x)] + log2(N)g ; It was proved that NID is a valid distance metric that satisfies i (6) the identity and symmetry axioms and the triangular inequal- where C remains to be a practical image compressor which ity [4]. encodes the difference between y and the transformed image The real-world application of NID is difficult because T (x), and the log (N) term computes the number of bits Kolmogorov complexity is a non-computable quantity [3]. i 2 required to encode the selection of one out of N potential By using the fact that K(xy) = K(yjx∗) + K(x) = transformations. K(xjy∗) + K(y) (subject to a logarithmic term), and by The idea of finding the simplest transformation between approximating Kolmogorov complexity K using a practi- two images is sensible from the viewpoint of human visual cal data compressor C, a normalized compression distance perception, for which it has long been hypothesis that the bi- (NCD) was proposed in [4] as ological visual system is an efficient coder of the visual world C(xy) − minfC(x);C(y)g [9]. For example, given two images that are rotated copy of NCD(x; y) = : (3) maxfC(x);C(y)g each other, our visual system would not interpret the differ- ence between them by directly differencing their intensity val- NCD has been proved to be an effective approximation of ues (which requires a large number of bits to encode the resid- NID and achieves superior performance in bioinformatics ap- ual), but by estimating the amount of rotation (which can be plications such as the construction of phylogeny trees using coded very efficiently). DNA sequences [4]. 4. IMPLEMENTATION 3. THE PROPOSED FRAMEWORK An advantage of NCCD (as opposed to NCD) is that it pro- When NCD was used to quantify image similarities, it did vides a more flexible framework so that different types of not achieve the same level of success as in other application transformations can be included. The list of transformations fields. For example, it was reported in [6] that NCD works can also be incremental, in the sense that new transforma- well when parts are added or subtracted from an image, but tions, when available, can be easily added into the existing struggles when image variations involve form, material and list, and expanding the list always improves the approxima- structure. We believe that this is mainly due to the poor ap- tion of NCCD to NID. Of course, exhausting all possible proximation of K(xy) using C(xy), which is often imple- transformations is practically impossible. However, by go- mented by applying a regular image compressor to the con- ing through a handful of transformations, it may be sufficient catenation of two images. For example, when an image is a to appropriately cover most image distortions encountered in ninety-degree rotated copy of another, concatenating two im- real-world applications. ages would not facilitate any efficient compression. To avoid Our current implementation of NCCD are as follows. this problem, we propose to approximate the conditional Kol- First, we adopt the content adaptive lossless image compres- mogorov complexity in Eq. (2) directly by designing a condi- sion algorithm (CALIC) [10] as the base image compressor, tional image compressor denoted by CT , so that which achieves superior performance when compared with state-of-the-art algorithms. CALIC is employed in computing K(yjx) ≈ CT (yjx) and K(xjy) ≈ CT (xjy) : (4) the denominator of Eq. (5) as well as the first term in Eq. (6). Since y − Ti(x) in Eq. (6) can generate negative values and This leads to a normalized conditional compression distance CALIC applies to grayscale images with positive intensity (NCCD) measure given by values only, the mean intensity value of y − Ti(x) is shifted to mid-gray level before the application of CALIC. Second, maxfC (xjy);C (yjx)g NCCD(x; y) = T T : (5) the types of transformations involved in the computation of maxfC(x);C(y)g CT include It remains to define the conditional compressor CT . Here • Global contrast and luminance change. This is com- we propose a practical solution by making use of a set puted by a pointwise intensity transformation defined of transformations that convert one image to another. Let as s = α(r −r¯)+r ¯+β, where r and s are the intensity fTiji = 1; ··· ;Ng be the set of transformations, let Ti(x) values before and after the transformation, respectively, represent the transformed image when applying the i-th trans- r¯ is the average value of r, and α and β are the parame- form to image x, and let p(Ti; x) denote the parameters used ters that determine the degrees of contrast and mean lu- in the transformation. Each type of transformation is also minance changes, respectively. In a special case when NCCD = 0.8121, SSIM = 0.4649, NCCD = 0.2189, SSIM = 0.7059, NCCD = 0.3901, SSIM=0.8425, NCCD =0.0091, SSIM = 0.9094, MSE = 73.4 MSE = 49.8 MSE = 78.4 MSE = 105.5 Fig.