INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017
Recursive Adaptive Color KLT for image sequences
Roumen K. Kountchev and Roumiana A. Kountcheva
Abstract - The method Recursive color space transform for a RDgDb, HSV or HSL, CMY, PhotoYCC, Lab, Luv, etc. sequence of correlated RGB images is based on the algorithm [2,3,8,12,13], is the maximum decorrelation of the transformed Adaptive Color KLT (AC-KLT). The non-recursive AC-KLT for a color components L1L2L3. The main disadvantage of the color sequence of color images is not efficient enough in this case, because KLТ is that it has higher computational complexity than the each image should be processed individually. Depending on the kind deterministic transforms. To partially overcome the problem, of the images in the processed sequence (for example, color video, approximately optimal color KLT with fixed coefficients is multi-view, medical sequences, etc.), the pixels of same position, but from two neighbor images, have significant similarity which in most presented in [8]. Another approach for KLT complexity cases is over 90 %. In such case, strong possibility exists for high reduction is given in [14], where it is used for recursive coding reduction of the information redundancy of the color information. To of image blocks of small size. In this case, the full transform is this end, here is offered the method Recursive AC-KLT (RAC-KLT) executed for the first block only, and for the next blocks are whose basic idea is to execute the AC-KLT for the first image in the calculated the values of the difference parameters in respect to sequence, and for the next images to calculate the values of the the previous. This approach was further developed in [15] for difference parameters only, which are needed for the successful local adaptive color transform of blocks and was aimed at the restoration of the color information. The efficiency of the new image compression. In each block of the image the coefficients method grows up together with the mutual correlation between the of the color transform matrix are determined from the consecutive images in the sequence. The evaluation of the computational complexity of the new algorithm confirms its previously compressed four neighbor blocks by using the advantage towards the basic AC-KLT. In the paper are also given weighted sums of the RGB pixel values, which makes the some experimental results from the comparison between color transform block-specific. The efficiency of the block coding is transforms YCrCb, AC-KLT and RAC-KLT which show the higher related to the spatial inter-block correlation, which depends on efficiency of the new approach. Here are also commented the the image contents and in many cases could be not too high. possible application areas for efficient representation of correlated For a large part of the existing image sequences the image sequences. correlation between pixels of same spatial position in two neighbor images is much higher than the inter-block Keywords—Adaptive Color Space Transform, Karhunen-Loeve correlation inside the image. To enhance the efficiency of the Transform, Recursive Adaptive Color KLT, Sequence of Correlated Color Images. RGB components processing, for each pixel from a given image in the group here is offered to execute recursive I. INTRODUCTION calculation of the KLT transform matrix of size 3×3. This approach is the basics of the method called Recursive Adaptive The processing and analysis of sequences of correlated color images are recently the object of significant number of Color KLT (RAC-KLT), presented here. It is based on the scientific investigations [1-4] aimed at the reduction of the adaptive RGB space transform for a single image, called color information redundancy, noise filtration, segmentation Adaptive Color KLT (AC-KLT) [16], which has lower and objects recognition based on their color, tracing of moving computational complexity than the famous KLT algorithms objects in video sequences, analysis of the color characteristics based on iterative computational methods [5,6]. Together with in selected areas in a group of images, etc. One of the most the fast development of the contemporary computer efficient transforms of the initial colors space (RGB) is the technologies the color space transform based on the AC-KLT Karhunen-Loeve Transform (KLТ) [5-10] also known as becomes suitable for more application areas, where the Hotelling Transform (НТ), or Principal Component Analysis processing time satisfies the real-time implementation (PCA). Its most important advantage towards the well-known requirements. deterministic color transforms YCrCb (the most commonly The paper is arranged as follows: In Section II is given the used color space for compression), YUV, YIQ, RCT, YCoCg, short description of AC-KLT for color images; in Sections III and IV are correspondingly presented the RAC-KLT method Prof. D. Sc. PhD. R. K. Kountchev is with the Technical University of and algorithm; in Section V the computational complexity of Sofia, Department of Radio Communications and Video Technologies, Bul. RAC-KLT is compared with that of AC-KLT; in Section VI are Kl. Ohridsky 8, Sofia 1000, Bulgaria (corresponding author, phone: shown some experimental results; in Section VII are +899924284) e-mail: rkountch@ tu-sofia.bg). Dr. R. A. Kountcheva is with TK Engineering, Sofia, Bulgaria (e-mail: commented a part of the possible applications of the new [email protected]). algorithm, and Section VIII contains the conclusions.
ISSN: 1998-4464 454 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017
23 − λδ jiji =+λ+λ+λ= ,0cba|k|det (7)
II. ADAPTIVE COLOR KLT FOR THE IMAGE RGB COMPONENTS ,1 = j,i where for ji =δ is got: Here follows the brief mathematical description of the ,0 ≠ j.i RGB color space transform of the color image by using the a (k ++−= kk 321 ), AC-KLT, in accordance with [15]. The transform has two 2 2 2 basic forms: Arithmetic and Trigonometric. kkkkkkb (k4323121 5++−++= kk 6), (8) 2 2 2 A. Arithmetic form of AC-KLT 61 52 kkkkkkc 43 (k +−++= kkk2kk 654321 ), Direct AC-KLT for the R,G,B components of the image of The matrix [KC] is symmetrical and its eigenvalues are S pixels and of size m×n=S: always real numbers. They can be defined using the Cardano [L ]( −Φ= )mC or Φ= X][L for s=1,2,..,S. (1) relations for the “casus irreducibilis” (or the so-called s s s s “trigonometric solution”): T Here = ssss ]B,G,R[C is the color vector which represents 1 =λ ()−ϕ )3/a(3/cos3/p2 , the processed pixel s from the image of S pixels; (9) T 3,2 −=λ −πϕ )3/a(]3/)cos[(3/p2 . = s3s2s1s ]L,L,L[L - the transformed color vector of the pixel s; For these eigenvalues are satisfied the relations: T 321 ≥λ≥λ≥λ 0 and ++=λ+λ+λ 111321 .kkk = R(E[m s ), G(E ), B(E ss )] - the mean color vector, S In (9) are used the following quantities: 3 where s == ∑x)S/1()x(Ex s is the averaging operator for xs in +−= =1s 3/)ab()3(a/2q c, respect to s; -p (a 2 <+= ,0b)3/ (10) T =−= 3 ss s3s2s1 ]x,x,x[mCX - the modified color vector s, −=ϕ 2/q[arccos (| )3/|p ]. for which s = ;0)X(E The solution of the systems of equations (6) defines the T ΦΦΦ=Φ ],,[][ - the transposed matrix of size 3×3 elements Φik of the matrix Φ][ when i,k=1,2,3, and the rows 321 used for the adaptive color KLT; of the matrix comprise the eigen vectors Φ . k Φ ΦΦΦ= ],,[ T when k=1,2,3 are the eigen vectors of k3k2k1k ,P/A ,P/B =Φ=Φ=Φ P/D ii3iii2iii1i for i =1,2,3, (11) the color covariance matrix defined on the basis of the where T modified vectors = s3s2s1s :]x,x,x[X A (k −= λ (k)[k λ −− kk) 64i25i3i ], (12) S T 1 T k(B )[ −λ−λ−= kk)k(k 54i16i3i ], c ss )XX(E]K[ == ∑ ss .XX (2) −1S =1s 2 = k(kkk2[kD )] 5i16546i k(k λ−−λ−− i2 ), The elements kij of this matrix are symmetric in respect to (13) its main diagonal, and it is represented as: 2 2 2 ii i ++= i .DBAP kkk 131211 kkk 541 Then from (11) it follows that the direct AC-KLT could be = = c ]K[ kkk 232221 kkk 624 . (3) represented as given below: L P/DP/BP/A R )R(E kkk 333231 kkk 365 s1 111111 s s = × − In the relation above are used the substitutions: k1=k11, L s2 P/DP/BP/A 222222 Gs s )G(E . (14) k2=k22, k3=k33, k4=k12=k21, k5=k13=k31, k6=k23=k32. The L s3 P/DP/BP/A 333333 Bs s )B(E elements k1 ÷ k6 of the covariance matrix are calculated from the relations: The inverse AC-KLT is defined by the relations: 2 2 2 2 2 2 T +Φ= Φ= T s1 ,)R()R(Ek 2 s ,)G()G(Ek =−=−= s3 ;)B(-)B(Ek (4) s s mL][C or s s.L][X (15)
4 ss −= R()GR(Ek )(G), From (14) and (15) it follows that the inverse AC-KLT could be represented as: 5 ss −= R()BR(Ek )(B), (5) R s P/AP/AP/A 332211 L s1 s )R(E 6 ss −= G()BG(Ek )(B). = × + Gs P/BP/BP/B 332211 L s2 s )G(E . (16) The eigenvectors Φk of the matrix [KC] are the solution of B P/DP/DP/D L )B(E the set of equations: s 332211 s3 s 2 3 2 As a result of the direct AC-KLT, the energy of the first ]K[ Φλ=Φ kkkC and k ∑ ki =Φ=Φ 1 for k=1,2,3. (6) =1i component ( L s1 ) of the transformed vector Ls is maximum,
The eigenvalues ,, λλλ of the matrix [KC] are the 321 while the second ( L s2 ) is much smaller, and the last ( L s3 ) is solution of its characteristic equation: close to zero.
ISSN: 1998-4464 455 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017
B. Trigonometric form of AC-KLT the basis of the averaged range of the mutual correlation The matrix Φ][ could be represented through the Euler calculated for the images in the group. In the second case, the length is set in accordance with the value of the mutual rotation angles (α,β,γ): correlation between the couples of neighbor images in the ΦΦΦ 11 12 13 group, which should be higher than a pre-selected threshold. ][ =Φ ΦΦΦ = The method RAC-KLT is applied for the images from each 21 22 23 (17) group, got after separation into sub-groups with fixed, or 31 32 ΦΦΦ 33 adaptive changing length (number of images in the group). ([ )] ([ )][ ( )] ,,([ γβαΦ=γΦβΦαΦ )], 1 2 3 One example for a group of sequential video frames is shown where the angles α,β,γ define the position of the transform on Fig. 1. The group is of fixed length of 8 images (N=8). coordinate axes L,L,L 321 in respect to the original RGB color space; α−α 0sincos =αΦ αα 1([ )] 0cossin ; 100
β sin0cos β TV Frame 0 TV Frame 1 =βΦ (18) 2([ )] 010 ; cos0sin ββ− γ−γ 0sincos =γΦ γγ . 3 ([ )] 0cossin 100 From (17) and (18) it follows that the elements of the TV Frame 2 TV Frame 3 matrix Φ][ are defined by the relations: γα−γβα=Φ ;sinsincoscoscos 11 γα−γβα−=Φ ;cossinsincoscos βα=Φ ;sincos (19) 12 13 γα+γβα=Φ ;sincoscoscossin 21 γα+γβα−=Φ ;coscossincossin βα=Φ ;sinsin (20) 22 23 TV Frame 4 TV Frame 5
31 γβ−=Φ ;cossin 32 γβ=Φ ;sinsin 33 β=Φ .cos (21) The matrix of the inverse AC-KLT in this case is defined by the relation: − T1 ][ 1([][ )] 2 ([ )] 3 ([ α−Φβ−Φγ−Φ=Φ=Φ )] . (22) − Hence, to define Φ][ 1 are needed the Euler angles α, β and γ only. These angles are calculated using the elements of TV Frame 6 TV Frame 7 Fig. 1. A sequence of 8 TV frames, 25 fps. The images are of size the matrix Φ][ : 1920×1080 each, in format.bmp, 24 bpp 2 2 2 =α arcsin(Φ23 1 Φ33 =− 3232 +BAPPD(arcsin) 3 ); (23) The processing of one group of images through RAC-KLT arccos ()()33 =Φ=β 33 ;P/Darccos (24) comprises the following basic operations:
2 2 2 • Recursive calculation of the color covariance matrix [Kc,t] of arccos( 32 33 =Φ−Φ=γ 33 + 3 .)BABarccos()1 (25) the processed image with sequential number t, by using [Kc,t-1], −1 The elements of the matrix Φ][ are calculated by using calculated for the preceding image, with number (t-1): the angles α, β, γ. As a result, to the decoder are transferred t,c −1t,c ∆+= K[]K[]K[ t,c ]. (26) the values of angles α, β, γ only, instead of all 9 elements of − where the matrix Φ][ 1 , i.e. the number of the needed coefficients is ∆∆∆ kkk t,5t,4t,1 − − − − − −1t,5t,51t,4t,41t,1t,1 )kk()kk()kk( 3 times smaller. =∆ ∆∆∆ = − − − t,c ]K[ kkk t,6t,2t,4 − − −1t,6t,61t,2t,21t,4t,4 )kk()kk()kk( . (27) III. RECURSIVE ADAPTIVE COLOR KLT METHOD FOR A ∆∆∆ kkk t,3t,6t,5 − − − − − −1t,3t,31t,6t,61t,5t,5 )kk()kk()kk( SEQUENCE OF CORRELATED COLOR IMAGES It is supposed here that in the related equations all The method, called Recursive Adaptive Color KLT (RAC- quantities with index (t-1) are already calculated for the KLT) is aimed at the processing of a sequence of correlated previous image with sequential number (t-1) and are stored in images separated into groups with fixed, or adaptively the dynamic buffer memory. changing length. In the first case, the fixed length is defined on
ISSN: 1998-4464 456 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017
T • Recursive calculation of the eigen vectors Φ t,k for Here the inverse transform matrix Φ t ][ and the vectors 3,2,1 k = 3,2,1 : m,C,L are calculated recursively: tt,st,s − ∆+= ΦΦΦ t,k1t,kt,k . (28) T T T t −1t +Φ=Φ ∆Φ t ,][][][ (35) th where each difference k eigen vector −=∆ ΦΦΦ −1t,kt,kt,k is the − ∆+= t,s1t,st,s ,LLL solution of the system of linear equations: ∆+= ,CCC ]K[ Φ ∆λ∆=∆∆ Φ for k=1,2,3. (29) − t,s1t,st,s (36) c, t,kt,kt,kt ∆+= t1-tt .mmm The difference eigen values λ∆ t,k of the matrix ∆ t,c ]K[ ∆Φ T are defined by relations, similar to (9)-(10), and the matrix The difference matrix for the inverse transform t ][ is calculated by analogy with (22), using the inverse rotation Φ t][ - by relations, similar to (19)-(21). Each of the α∆−β∆−γ∆− participating quantities should be assigned the index t. angles ( t),( t),( t).
• Recursive calculation of the angles ,, γβα ttt for the IV. RAC-KLT ALGORITHM FOR A GROUP OF COLOR IMAGES processed image t: On the basis of the already presented method was − α∆+α=α t1tt , − β∆+β=β t1tt , − γ∆+γ=γ t1tt , (30) developed the algorithm RAC-KLT for processing a group of where the difference angles are defined by analogy with (23) - N images, where t = 0,1,..,N-1 is the number of the currently (25): processed image. 2 2 αt t,3t,2t,3t,2 ∆+∆∆∆∆=∆ BAPPD(arcsin t,3 ), A. Algorithm for direct RAC-KLT
t ()∆∆=β∆ t,3t,3 ,P/Darccos (31) For the first image with components [R0], [G0], [B0] in the
2 2 group (image with number t=0) is executed the algorithm t t,3 t,3 ∆+∆∆=γ∆ t,3 .)BABarccos( initialization, which comprises the following steps:
• Calculation of the direct RAC-KLT for the vectors Step 1. Calculation of the elements ÷ kk 6,00,1 of the initial −= mCX tt,st,s in accordance with the relations: color covariance matrix [Kc,0], in correspondence with (4)-(5); Step 2. Calculation of the parameters 000 ,,cb,a 0,q ϕ0 , ,,([L γβαΦ= )]X t,sttts , for s=1,2,..,S. (32) ,, , λλλ 0,30,20,1 ,, 2,03,00,3 ,D,B,A 0,2 ,P P 0,3 - in correspondence The matrix ,,([ γβαΦ )] is defined by the relations (33) ttt with (8) - (13); where in the last line the mutual correlation between the Step 3. Calculation of the trigonometric functions sin(α0), corresponding two sequential color images with sequential cos(α0), sin(β0), cos(β0), sin(γ0), cos(γ0) for the angles α0, numbers (t) and (t-1) is close to 100 %. This permits the β0, γ0 in correspondence с (23) - (25); calculations needed for AC-KLT to be reduced many times, Step 4. Calculation of the initial AC-KLT matrix Φ0 ][ as a because the matrix Φ t ][ is not necessary to be calculated product of the corresponding initial rotation matrices Φ (instead, the already calculated matrix −1t ][ is used). γΦβΦαΦ ([ )] ([, )], ([ 010101 )], defined in accordance with the ,,([ γβαΦ )] ,X if ,0 ,0 ≠γ∆≠β∆≠α∆ ;0 relations (17) - (21); tttt,sttt T Step 5. Calculation of 0= R(E[m 0,s ), G(E 0,s ), B(E 0,s )] - the ([ − )][ ( )] ([ )] ,X if tt,st3t21t1 tt ≠γ∆≠β∆≈α∆γΦβΦαΦ ;0,0,0 initial mean color vector; )([ ][ ( − )] ([ γΦβΦαΦ )] ,X if tttt,st31t2t1 ≠γ∆≈β∆≠α∆ ;0,0,0 Step 6. Calculation of the transformed color vectors L 0,s for )([ ][ ( )] ([ γΦβΦαΦ − )] ,X if tttt,s1t3t2t1 ≈γ∆≠β∆≠α∆ ;0,0,0 the image t=0 and s=1,2,..,S in correspondence с (1). L t,s = For the next images in the group (t=t+1 for t≤N-1) are − )([ ][ − ,X])([])( if tttt,st31t21t1 ≠γ∆≈β∆≈α∆γΦβΦαΦ ;0,0,0 executed the steps given below (similar to these for the image with number t=0): )([ ][ γΦβΦαΦ ,X])([])( if ≈γ∆≠β∆≈α∆ ;0,0,0 − − tttt,s1t3t21t1 γβα Step 7. Calculation of the rotation angles ,, ttt for the )([ ][ − γΦβΦαΦ − ,X])([])( if tttt,s1t31t2t1 ≈γ∆≈β∆≠α∆ ;0,0,0 image t=t+1 using relations, similar to these from Step 3, where the index 0 is replaced by t. − )([ ][ − γΦβΦαΦ − ,X])([])( if tttt,s1t31t21t1 ≈γ∆≈β∆≈α∆ ;0,0,0 Step 8. Calculation of the difference rotation angles for the (33) image t: α α−α=∆ ; β−β=β∆ ; γ−γ=γ∆ . (37) • Calculation of the inverse RAC-KLT (IRAC-KLT) by −1ttt −1ttt −1ttt analogy with (15): ˆ Step 9. Calculation of the approximated angles ˆ , ,γβα ˆ ttt for -1 T T tt,s Φ=Φ= tt,s L][L][X t,s or tt,s +Φ= tt,s .mL][C (34) the image t:
ISSN: 1998-4464 457 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017 α δ<α∆ ˆ −1t , if t ;|| ˆ =∆ ˆ ˆ −=∆ ˆ =α (38) L -L −1t,st,st,s ,L 1-ttt .mmm (46) t ˆ α∆+α − t1t in- cases,other Step 15. Calculation of the quantizated difference vectors βˆ , if δ<β∆ ;|| ˆ ∗∗∗∗ T ˆ −1t t = t,s,3t,s,2t,s,1t,s ]L,L,L[L : t =β (39) ˆ −β∆+β in cases,other − t1t ˆ θ<∆ ˆ∗ ,0 if | 1t,s,1 ;|L L t,s,1 =∆ (47) γˆ −1t , if t δ<γ∆ ;|| ∆Lˆ - in all cases,other ˆ =γ (40) t,s,1 t ˆ −γ∆+γ in cases,other − t1t ,0 if | ˆ θ<∆ ;|L Lˆ∗ =∆ 2t,s,2 (48) In the relations above xˆ means the approximation of x, and δ t,s,2 ˆ ∆L t,s,2 - in all cases,other is a preset threshold (a small number, for example, in the range 0,01-0,05), which defines the accuracy of the restored color ,0 if | ˆ θ<∆ ;|L Lˆ∗ =∆ 3t,s,3 (50) vectors after the inverse RAC-KLT, and the volume of data t,s,3 ˆ ∆L t,s,3 - in all cases,other which represents the difference angles. where θ , θ , θ are thresholds defined in accordance with the αˆ αˆ βˆ βˆ 1 2 3 Step 10. The values of functions sin t , cos t, sin t , cos t , mean noise value in the corresponding component of the sin γˆ t and cos γˆ t are set from the pre-calculated tables. After ˆ ˆ vector ∆L t,s . In the general case, each component ∆L t,s,i has ˆ that, the matrix ([ ˆ , ,γβαΦ ˆ tttt )] is calculated as the product of ˆ to be quantizated in corresponding nonlinear scale (Q ∆ t,s,ii )L , ˆ the three matrices ([ αΦ ˆ t1 )], ([ βΦ t2 )] and ([ γΦ ˆ t3 )] in whose levels are harmonized with the image histogram. correspondence with Step 4, where the index 0 is replaced by t. When t gets the value t=N, the execution of the direct RAC-KLT for the group of images is finished. As a result, the Step 11. Calculation of the mean color vector mt for the image t, in correspondence with Step 5, where the index 0 was values of the vectors' components of ,L ,m ∆ˆ ,L ∆m 0,s 0 t,s t −= ∗∗∗ replaced by t, and the vectors are mCX tt,st,s ; and of the angles γβα ,,, ,, γ∆β∆α∆ for t=0,1,2,..,N-1 000 ttt ˆ and s=1,2,..,S, are got. After that they could be efficiently Step 12. Calculation of the restored vectors L t,s for coded using various methods, selected in accordance with the s=1,2,..,S on the basis of (38) - (40) and the relations in (41): application of the algorithm RAC-KLT. ([ ˆ )][ (ˆ )] ([ γΦβΦαΦ ˆ )] ,X if | ˆ |,| ˆ ,| | ˆ δ≥γ∆δ≥β∆δ≥α∆ ;| B. Algorithm for inverse RAC-KLT tttt,st3t2t1 The input parameters, needed for the inverse RAC-KLT are: ˆ ˆ ([ ˆ − )][ ( )] ([ γΦβΦαΦ ˆ )] ,X if | ˆ |,| |,| ˆ tttt,st3t21t1 δ≥γ∆δ≥β∆δ<α∆ ;| - the initial angles ,, γβα 000 and the components of the ˆ ˆ ([ ˆ )][ ( − )] ([ γΦβΦαΦ ˆ )] ,X if | ˆ |,| |,| ˆ tttt,st31t2t1 δ≥γ∆δ<β∆δ≥α∆ ;| vectors L 0,s and m0 for the image with group number t=0, when s=1,2,..,S; ˆ ˆ γΦβΦαΦ ˆ ˆ ˆ ˆ δ<γ∆δ≥β∆δ≥α∆ ([ )][ ( )] ([ − )] ,X if | |,| |,| tttt,s1t3t2t1 ;| ∗∗∗ ˆ - the difference angles ,, γ∆β∆α∆ ttt and the components L t,s = ˆ ˆ ([ ˆ − )][ ( − ([]) ˆ ,X]) if | ˆ |,| |,| ˆ tttt,st31t21t1 δ≥γ∆δ<β∆δ<α∆γΦβΦαΦ ;| of the vectors ∆L and ∆m for each image with number t,s t t=1,2,..,N-1, for s=1,2,..,S. ([ ˆ )][ (ˆ ([]) γΦβΦαΦ ˆ ,X]) if | ˆ |,| ˆ |,| ˆ δ<γ∆δ≥β∆δ<α∆ ;| − − tttt,s1t3t21t1 For the first image (with number t=0) the inverse RAC-KLT ˆ ˆ ([ ˆ )][ ( − ([]) γΦβΦαΦ ˆ − ,X]) if | ˆ |,| |,| ˆ tttt,s1t31t2t1 δ<γ∆δ<β∆δ≥α∆ ;| comprises the following steps: ∆Φ ˆ ˆ Step 1. Calculation of the matrix 0 ][ on the basis of the ([ ˆ − )][ ( − ([]) γΦβΦαΦ ˆ − ,X]) if | ˆ |,| ,| ˆ tttt,s1t31t21t1 δ<γ∆δ<β∆δ<α∆ ;| values of ,, γβα 000 in correspondence with Step 4 of the (41) algorithm for direct RAC-KLT; Step 13. Calculation of the modified angles ,, γ∆β∆α∆ ∗∗∗ ttt Step 2. Calculation of the restored initial color vectors satisfying the rules: ˆ ,0 if δ<α∆ ;|| C 0,s for s=1,2,..,S through the inverse RAC-KLT in ∗=α∆ t t (42) correspondence with (15), where is added the index 0; α∆ t - in cases,other For each of the next images with number t=t+1 when t ≤ N-1, ,0 if δ<β∆ ;|| ∗ t the inverse RAC-KLT comprises the following steps: t =β∆ (43) β∆ t - in cases,other Step 3. Calculation of the difference matrix ∆Φ t ][ based on ∗∗∗ ∗ ,0 if t δ<γ∆ ;|| the values of ,, γ∆β∆α∆ ttt in correspondence with the t =γ∆ (44) γ∆ t - in cases.other relations below. Step 14. Calculation of the difference vectors:
ISSN: 1998-4464 458 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017
)([ ][ ( )] )([ ], if ∗∗∗ ≠γ∆≠β∆=α∆γ∆Φβ∆Φα∆Φ ;0,0,0 - for the RAC-KLT of the initial image with sequential − t3t21t1 ttt ∗∗∗ number t=0 - the parameters ,,,m,L γβα 00000,s . For each of − )([ ][ ( − )] t31t21t1 )([ ], if ttt ≠γ∆=β∆≠α∆γ∆Φβ∆Φα∆Φ ;0,0,0 the next images in the sequence with number t=1,2,..,N-1 - the )([ ][ ( )] )([ ], if ∗∗∗ =γ∆≠β∆=α∆γ∆Φβ∆Φα∆Φ ;0,0,0 − −1t3t21t1 ttt ˆ∗ ˆ parameters ,m,L ˆ , , γ∆β∆α∆∆∆ ˆ ttttt,s , which for the whole ∗∗∗ ≠γ∆=β∆=α∆γ∆Φβ∆Φα∆Φ − )([ ][ ( − )] t31t21t1 )([ ], if ttt ;0,0,0 group are represented by 3(S+2)+3(S+2)(N -1)=3(S+2)N ∆Φ ][ = , t ∗∗∗ numbers in total. − )([ ][ ( )] −1t3t21t1 )([ ], if ttt =γ∆≠β∆=α∆γ∆Φβ∆Φα∆Φ ;0,0,0 Hence, there is no difference between both algorithms in ∗∗∗ )([ ][ ( − )] −1t31t2t1 )([ ], if ttt =γ∆=β∆≠α∆γ∆Φβ∆Φα∆Φ ;0,0,0 respect of the needed color parameters number. However, due ∗∗∗ to high mutual correlation between the neighbor images in the )([ ][ ( )] )([ ], if ≠γ∆≠β∆≠α∆γ∆Φβ∆Φα∆Φ ;0,0,0 t3t2t1 ttt group, the values of large number of the direct RAC-KLT ∗∗∗ − )([ ][ ( − )] −1t31t21t1 )([ ], ttt =γ∆=β∆=α∆γ∆Φβ∆Φα∆Φ .0,0,0if parameters are close, or equal to zero, while this number is much lower for the direct AC-KLT, because they have no (51) T relation with the mutual correlation. This is the main Step 4. Calculation of the matrix Φ t ][ for the inverse difference between RAC-KLT and AC-KLT. RAC-KLT of the image t: T T T B. Computational complexity of algorithms AC-KLT and t −1t +Φ=Φ ∆Φ t .][][][ (52) RAC-KLT ˆ −1 ˆ Step 5. Dequantization of the components ∆ ′ it,s,i (QL ∆= t,s,i )L The computational complexity of both algorithms could be evaluated on the basis of the computational operations ˆ ′ ˆ ′ ˆ ′ ˆ ′ T of the vectors ∆ s, [L ∆= ,L ∆ ,L ∆ t,s,3t,s,2t,s,1t ]L and calculation of (additions and multiplications) needed for their execution. Here follow the generalized data, got as a result of the analysis ˆ ′ the corresponding vectors Ls,t and mt for the image t: of the operations for direct and inverse color transform for a ˆ ˆ ˆ group of N images, S pixels each.The number of operations is: L′ = L′ ∆+ ′ tr,1-tr,tr, ,L ∆+= mmm t1-tt . (53) - for the direct AC-KLT: (37S+124)N≈37SN; (55) Cˆ ′ Step 6. Restoration of the color vectors t,s through the - for the RAC-KLT the calculation of the first image in the inverse AC-KLT: group needs 37S+124 operations, and for the remaining T images in the group the needed number of operations is ˆ ′ Φ= ][C ˆ ′ + mL for s=1,2,..,S. (54) tt,s s, tt (37S+129)(N-1). In total, the needed operations for RAC-KLT The execution of the inverse RAC-KLT stops when t=N are: is satisfied. 37S+124+(37S+129)(N-1) ≈37SN; (56) ˆ Accordingly, The difference (error) between the restored vectors C′ t,s - for the inverse AC-KLT: (15S+23)N≈15SN; (57) and the originals C t,s is defined by the thresholds δ, θ1, θ2, θ3 - for the inverse RAC-KLT: (15S+47)N-24≈15SN. (58) Hence, the computational complexity of both algorithms is and the quantization scales Qi(.) used for the calculation of the practically same. ∆ˆ components of the difference vectors Ls,t . The main advantage of RAC-KLT towards AC-KLT is the The presented algorithm RAC-KLT is asymmetric, because ability to achieve higher reduction of the color parameters the number of calculations needed for the inverse transform is data. As a result of the correlation between the couples of much lower than that for the direct. sequential images, the values of the difference angles
,, γ∆β∆α∆ ttt and of the components of the difference vectors V. EVALUATION OF RAC-KLT EFFICIENCY ˆ∗ ∆L t,s for significant number of pixels in each image are close, The efficiency of RAC-KLT for a sequence of correlated or equal to zero. In this case, the use of various existing color images is evaluated here through its comparison with methods for lossless or visually lossless coding methods AC-KLT. For this are used the number of parameters needed permits high reduction of the volume of the color components for the inverse transform, and the computational complexity of data, which should be transferred to the decoder. However, for both algorithms. the execution of the algorithm RAC-KLT is necessary to use A. Parameters calculated by direct AC-KLT and RAC-KLT larger memory volume to store the color data for the previous To execute the inverse color transform for each pixel s of the image in the group with number (t-1). This problems could be processed image from the sequence, should be known the considered easily solved because of the fast development of values of the following parameters: contemporary computer technologies. The choice of the values for the thresholds δ and θ1, θ2, θ3 - for the AC-KLT - the parameters s γβα ,,,,m,L which (the quantization scales Qi(.), resp.) influences the restoration are represented by 3(S+2)N numbers in total (S is the number accuracy of the color vectors in the group for all images except of pixels in the single image, N - the number of the images in the initial one, and also on the needed volume of data that the sequence); represent the color parameters of the images. For lower
ISSN: 1998-4464 459 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017
threshold values the accuracy of the restored color vectors is Computer Graphics PSNR Results higher, but together with it grows up the needed volume of the 52 color data, and vice versa. This is why these values should be 51 50 selected depending on the application of RAC-KLT. 49 48 47 ACT 46 YCrCb
VI. EXPERIMENTAL RESULTS PSNR[dB] 45 44 Here are presented some experimental results of the 43 42 comparison of the algorithm AC-KLT (abbreviated as ACT) 41 and the standard deterministic color transform for static
cgraph1 cgraph2 cgraph3 cgraph4 cgraph5 cgraph6 cgraph7 cgraph8 cgraph9 cgraph10 images, YCrCb [16,17,18]. For the experiments were used 3 Image Name sets of images of different size – the “Кodak image set“ of size 512×768 or 768×512, plus images “Lena” (512×512) and “Barbara” (640×512) - 26 images in total ; the “cgraph image set“ which comprises 10 computer generated images of size 1024×768, and the “natural image set“ which comprises 10 images of size 1920×1024 pixels. All images were in the format .bmp, 24 bpp. Example groups from the three test sets are shown on Fig. 2. For the elements of the first component [L1] of ACT was used adaptive scalar quantization (ASQ) and the elements were represented as 8-bit numbers. On Fig. 3 are shown the results for the comparison of the PSNR (Peak Signal to Noise Ratio) in dB for all three image sets, generalized in Table 1. They show that the mean increase for Natural HD images PSNR Results the ACT compared to YСrCb is 2 dB, and for its next version, 50 the Enhanced ACT (ЕACT) [18], based on histogram 49 matching of the color components, the PSNR is up to 5 dB 48
47 higher. ACT 46 YCrCb PSNR[dB] 45
44
43
Image Name
Images from the “cgraph image set“ 1024×768, 24 bpp
Images from the „natural image set“ 1920×1024, 24 bpp
Fig 3. Results for the comparison of PSNR between ACT and ЕACT towards YCrCb
Images from the „kodak image set“ 768×512, 24 bpp The comparison for the algorithms ACT and YCrCb about the Fig. 2. Example images from 3 test image sets power distribution of the three components [L1],[L2],[L3] and [Y],[Cr],[Cb] for the group of 10 natural test images are shown
on Fig. 4. They indicate the higher power concentration on the Table1 first component [L ] and the monotonous decrease in the next Image set YCrCb ACT EAСT 1 two components [L ] and [L ], compared to the power Mean PSNR Mean PSNR Mean PSNR 2 3 distribution for the component [Y] and the next components, [dB] [dB] [dB] [Cr] and [Cb]. Besides, in the second case, for some images Kodak+2 46.57 48.44 50.79 could be easily noticed the higher power concentration in the Cgraph 46.12 48.97 51.24 third component [Cb] than in the second, [Cr]. The results Natural HD 46.01 48.59 50.17 confirm the higher efficiency of ACT for all test images.
ISSN: 1998-4464 460 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017
YCrCb Power Distribution natural Toolbox ACT Power Distribution natural Toolbox
25000 250000
20000 200000
15000 Y 150000 L1 Cr L2 Power Power 10000 Cb 100000 L3
5000 50000
0 0
natural1 natural2 natural3 natural4 natural5 natural6 natural7 natural8 natural9 natural10 natural1 natural2 natural3 natural4 natural5 natural6 natural7 natural8 natural9 natural10 Image Name Image Name
Fig. 4. Results for the comparison of the power distribution of the three color components for YCrCb and ACT An example, illustrating the transforms YCrCb and ACT of components power distribution for both transforms. Besides, the RGB image “Lena” and the corresponding three this distribution is decreasing monotonously only for ACT. components [Y],[Cr],[Cb] and [L1],[L2],[L3] is shown on Fig. 5. The obtained results confirm the conclusions about the
Original Image [R] [G] [B]
Original Image [Y] [Cr] [Cb]
Original Image [L ] [L ] [L3] 1 2 Fig. 5. Comparison of the RGB transform components obtained for YCrCb and ACT for the test image “Lena”
Some results about the application of algorithms AC-KLT when the standard JPEG 2000-LS for lossless compression and RAC-KLT are shown on Fig. 6. For the processing of the was used, are given in Table 2. On the basis of the averaged values from Table 2 were difference components ∆PC1 for this sequence was used calculated the compression coefficients and relations, given threshold value θ1 =16. The differences obtained after the below: threshold transform are noted as thr∆PC1. The difference Mean compression coefficients CR for i=1,2,3,4: frames that represent the components ∆PC1 and thr∆PC1 are o i given in second and third columns of Fig. 6, correspondingly. MeanCR1=Mean[PC1(bmp)/PC1(jp2)]=2.94; For better visualization these pictures are with enhanced MeanCR2=Mean[PC1(bmp)/∆PC1(jp2)]=3.36; contrast. MeanCR3=Mean[PC1(bmp)/thr∆PC1(jp2)]=19.45; The opportunities for lossless compression of the input and MeanCR4=Mean[∆PC1(jp2)/thr∆PC1(jp2)]=5.79. output (transformed) sequence of TV frames were also o Relations between the calculated compression investigated. The results obtained for the compression of the coefficients: components PC1(t) of frames with sequential number t=0,1,..,7 Mean (CR2/CR1)=1.19 - i.e., 19 % higher mean compression for ∆PC1(jp2) towards PC1(jp2);
ISSN: 1998-4464 461 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017
Mean (CR3/CR1)=39.06 - i.e., 39.06 times higher mean As it was shown, the power of the next components (PC2 compression for Thr∆PC1(jp2) towards PC1(jp2); and PC3) is much lower than that of PC1, and their RAC-KLT Mean (CR3/CR2)=32.82 - i.e., 32.82 times higher mean compression compared to AC-KLT grows faster, getting compression for Thr∆PC1(jp2) towards ∆PC1(jp2); highest values for PC3 (these results are not given here). Mean (CR3/CR4)=4.23 - i.e., 4.23 times higher mean This analysis illustrates the higher efficiency of RAC-KLT ∆ compression for PC1(jp2) towards PC1(bmp). compared to AC-KLТ in respect of compression ratio for the The presented results show that after applying the RAC-KLT investigated TV sequence. In future the algorithm RAC-KLT on the non-compressed (bmp) frames, their first principal will be investigated experimentally in various application areas components PC1 have average compression of 19.45. When aiming at its parameters optimization for each specific case. AC-KLT used for same frames, compression ratio of 2.94 was obtained. For same example, the compression of PC1 for RAC- KLT is 6.61 times higher than that for AC-KLT.
AC-KLT RAC-KLT
PC1(t=0)
PC1 (t=0) - Thr
PC1(t=1) - PC1(t=0) Thr ∆PC1(t=1)
PC1(t=1) - Thr
PC1(t=2) - PC1(t=1) Thr∆PC1(t=2)
PC1(t=2) - Thr
PC (t=3) - PC (t=2) 1 1 Thr∆PC1(t=3)
PC (t=3) 1 - thThr
PC1(t=4) - PC1(t=3) Thr ∆PC1 (t=4)
PC1(t=4)
Fig. 6. Example for the application of algorithms AC-KLT and RAC-KLT concerning the first principal component PC1 of the TV frames 0,1,2,3,4 from the sequence shown on Fig. 1
Table 2. Compression results for the PC1 components of the frames from Fig. 1, through the standard JP2-LS
Number t PC1 (bmp) PC1 ∆PC1= Thr∆PC1= CR1= CR2= CR3= CR4= of PC1(t) 1920×1080 (jp2) [PC1(t)- Thr[PC1(t)- PC1(bmp) PC1(bmp)/ PC1(bmp) ∆PC1(Jp2) 8 bpp, KB KB PC1(t-1)] PC1(t-1)] /PC1(jp2) ∆PC1(jp2) /Thr /Thr
(jp2), KB (jp2), KB ∆PC1(jp2) ∆PC1(jp2) 0 1970 684 (0) 684 (0) 684 2.88 2.88 2.88 1 1 1970 694 (1-0) 760 (1-0) 29.40 2.83 2.59 67.00 2.71 2 1970 652 (2-1) 757 (2-1) 30.80 3.02 2.60 51.84 19.93 3 1970 669 (3-2) 504 (3-2) 12.30 2.94 3.91 160.16 40.96 4 1970 654 (4-3) 547 (4-3) 16.20 3.01 3.60 121.60 33.77 5 1970 664 (5-4) 473 (5-4) 10.20 2.96 4.16 193.14 42.26 6 1970 651 (6-5) 431 (6-5) 9.31 3.02 4.57 211.60 46.30 7 1970 693 (7-6) 536 (7-6) 17.80 2.84 3.67 110.67 30.15 Mean 1970 670.12 586.5 101.25 2.94 1 3.50 2 114.86 3 27.13 4
ISSN: 1998-4464 462 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 11, 2017
REFERENCES VII. APPLICATION AREAS FOR RAC-KLT [1] T. Reed (Ed.), Digital Image Sequence Processing, Compression, and On the basis of the efficiency evaluation of RAC-KLT used Analysis, 2005, CRC Press. [2] R. Lucas, K. Plataniotis, Color Image Processing: Methods and for the representation of the color components for a group of Applications, 2007, Taylor & Francis. correlated images, here are defined the main future application [3] M. Celebi, M. Lecca, B. Smolka (Eds.), Color Image and Video Enhance- areas: ment, 2015, Springer. - coding of video sequences got from stationary TV [4] M. Drew, S. Bergner, "Spatio-Chromatic Decorrelation for Color Image Compression", Signal Processing: Image Communication, Vol. 23, No. cameras used in the surveillance systems; 8, Sept. 2008, pp. 599-609. - search of closest images in databases containing the [5] H. Abdi, L. Williams, "Principal Component Analysis", Wiley information got from surveillance TV cameras; Interdisciplinary Reviews: Computational Statistics, Vol. 2, No. 4, 2010, - segmentation and object recognition based on the color pp. 433-459. [6] C. Maccone, "A simple introduction to the KLT (Karhunen-Loève information in a group of images, etc. Transform) ", Chapter in Deep Space Flight and Communications”, C. In general, RAC-KLT could be also used in the digital TV Maccone (Ed.), Springer, 2009, pp. 151-179. systems, but in this case the methods for movement [7] P. Hao, Q. Shi, "Reversible integer KLT for progressive-to-lossless compression of multiple component images". In: Intern. Conference on compensation should be added to the processing and Image Processing (ICIP), Vol. 1, 2003, pp. 633–636. calculation of the difference color vectors for each pixel and of [8] Y. Chen, P. Hao, and A. Dang, "Optimal transform in perceptually the difference rotation angles for each frame, to overcome the uniform color space and its applications in image coding", In: ICIAR problems related to the movement of the objects and the 2004, LNCS 3211, 2004, pp. 269–276. [9] A. Abadpour, S. Kasaei, Color PCA Eigenimages and their Application to background. For real-time applications of RAC-KLT should Compression and Watermarking, Image & Vision Computing, Vol. 26, be developed its modification based on the algorithm Issue 7, July 2008, рр. 878-890. Reversible Integer KLT [7]. [10] M. Minervini, C. Rusu, S. Tsaftaris, "Unsupervised and Supervised Approaches to Color Space Transformation for Image Coding", Proc. of the 21st IEEE Intern. Conference on Image Processing (ICIP’14), Paris, VIII. CONCLUSIONS France, Oct. 27-30, 2014, pp. 5576–5580. A basic method for recursive color space transform of a group [11] R. Starosolski, "New simple and efficient color space transformations for lossless image compression", Journal of Visual Communication and of correlated images is developed, based on the AC-KLT. It Image Representation, Vol, 25, No. 5, 2014, pp. 1056-1063. could be used as a basis for the development of: [ 12] M. Fairchild, Color and Image Appearance Models. John Wiley & Sons, - New algorithms for parallel processing of the images in 2005. the group, based on the RAC-KLT, which will enhance the [13] S. Singh, S. Kumar, "Novel Adaptive Color Space Transform and Application to Image Compression", Signal Processing: Image calculation speed of the direct and inverse transforms; Communication, Vol. 26, No. 10, Nov. 2011, pp. 662-672. - RAC-KLT algorithm with higher efficiency, based on [14] P. Farrelle, Recursive. Block Coding for Image Data Compression, the adaptive division of the processed group of images into Springer, NY, 1990. groups of various length, where the correlation between the [15] A. Suhre, K. Kose, A. Cetin, M. Gurcan, "Content-adaptive color neighbor image couples is higher that a preset threshold. This transform for image compression", Optical Engineering, Vol. 50, No. 5, May 2011, pp. 057003-1 - 057003-7. approach will ensure higher reduction of the color components [16] R. Kountchev, R. Kountcheva, "Image Color Space Transform with number for each group; Enhanced KLT", Сhapter in: New Advances in Intelligent Decision - New algorithms, aimed at various application areas. Technologies, K. Nakamatsu, G. Philips-Wren, L. Jain, R. Howlett Maximum efficiency for each algorithm could be achieved (Eds.), Springer, 2009, pp. 171-182. [17] Р. Ivanov, R. Kountchev, "Comparative analysis of Adaptive Color KLT when the basic parameters of RAC-KLT are adapted and YCrCb for representation of color images", Intern. Scientific Conf. depending on the application requirements in respect of on Information, Communication and Energy Systems and Technologies accuracy, reversibility, execution time, implementation, (ICEST’10), Ohrid, Macedonia, June 2010, SP I, pp. 29-32. methods for color parameters coding, etc. [18] P. Ivanov, R. Kountchev, R. Mironov, "Algorithm for adaptive color KLT of images, based on histogram matching of the color components", Intern. Scientific Conf. on Information, Communication and Energy Systems and Technologies (ICEST’11), Nis, Serbia, July 2011, Proc. of Papers, Vol. 1, SP I, pp. 5-8. [19] R. Kountchev, R. Kountcheva, "Color Space Transform for Correlated Images Based on the Recursive Adaptive KLT", IARAS Intern. Journal of Signal Processing, Vol. 2, 2017, рр. 72-80.
ISSN: 1998-4464 463