Binary Polynomial Transforms for Nonlinear Signal

BINARY POLYNOMIAL TRANSFORMS FOR NONLINEAR SIGNAL

PROCESSING

Sos Agaian

Jaakko Astola, Karen Egiazarian, Rusen Oktem

Electrical Engineering Department

Signal Pro cessing Lab oratory

UniversityofTexas at San Antonio

Tamp ere UniversityofTechnology

San Antonio, Texas, USA

Tamp ere, FINLAND

ABSTRACT ing computers or systems suitable for simultaneous arith-

metical or logical op erations.

In this pap er weintro duce parametric binary Rademacher

functions of twotyp es. Based on them using di erent kinds

2. BINARY POLYNOMIAL FUNCTIONS AND

of arithmetical and logical op erations we generate a set of

MATRICES

binary p olynomial transforms (BPT). Some applications of

BPT in nonlinear ltering and in compression of binary

2.1. Rademacher Functions and Matrices

images are presented.

Let a and b b e arbitrary integers. We form the classes

r (t; a; b) and s (t; a; b) of real p erio dic functions in the

n n

1. INTRODUCTION

following way. Let

Sp ectral analysis is a p owerful to ol in communication, sig-

r (t; a; b) s (t; a; b) 1; (1)

0 0

nal/ image pro cessing and applied mathematics and there

are many reasons b ehind the usefulness of sp ectral analysis.

(

First, in many application s it is convenient to transform a

2 1

m 1 m

; + ); a; for t 2 [

n n

n+1

2 2

m=0

problem into another, easier problem; the sp ectral domain

r (t; a; b)=

S n

n+1

2 1

m 1 m+1

b; for t 2 [ + ; ):

n n is essentially a steady-state viewp oint; this not only gives

n+1

2 2

m=0 2

insight for analysis but also for design. Second, and more

(2)

imp ortant: the sp ectral approach allows us to treat entire

and

(

classes of signals whichhave similar prop erties in the sp ec-

a; for t 2 [0; );

n+1

tral domain; it has excellent prop erties such as fast algo-

1 1

s (t; a; b)= n =0;1;2; :::

n+1

b; for t 2 [ ; )

n+1

2 rithms, high energy compaction prop erty of the transform

0; otherwise for t 2 [0; 1);

data.

(3)

However, there are some problems in signal/ image pro-

Note that

cessing (esp ecially if the input data is binary) where they

cannot b e applied directly.

); a; for t 2 [0;

r (t; a; b)=s (t; a; b)= :

1 1

Wemay prop erly ask what is a basic transform for bi-

b; for t 2 [ ; 1)

nary signal/image pro cessing which plays the same role for

it as sp ectral transform for linear pro cessing.

DEFINITION 2.1 : The functions r (t; a; b) are called

Similarly wemay ask what is a basic transform for non-

Rademacher (a; b)-functions of I-type. The functions

linear signal/image pro cessing which plays the same role for

s (t; a; b) are called Rademacher (a; b)-functions of II-type.

it as sp ectral transform for linear pro cessing.

We will develop a compact representation of signals for

Note that when a = 1 and b = 1; the Rademacher

later use in digital logic and nonlinear signal analysis. Our

(a; b)-functions of I-typ e coincide with the Rademacher func-

approach will b e based on intro duction of parametric bi-

tions [5]. Let us consider some prop erties of the generalized

nary Rademacher functions (BRF); on generating a set of

Rademacher functions de ned ab ove.

transforms using Rademacher functions and di erent kinds

of arithmetical and logical op erations. Emphasis is placed

PROPERTY 2.1 : There are the following representa-

on the two classes of transforms, intro duced in this pap er:

tions of the Rademacher (a; b)-functions of the I-typ e:

transforms generated by BRFs using only one op eration

(arithmetical or logical) and transforms generated by BRFs

1) r (t; a; b)=a+(b a)c (t);t2[0; 1); (4)

n+1 n+1

using two op erations (arithmetical or logical). We present

where c are the co ecients from

n+1

the basic prop erties of these transforms.

1 1

This development is motivated also by p otential appli-

X X

n n

cations in areas such as nonlinear signal/image pro cessing,

t = c (t)2 = c 2 : (5)

n n

construction of new architectures of signal/image pro cess-

n=1 n=1

The discrete Rademacher (a; b)-matrices of I-typ e and

h i

I I-typ e have the following prop erties.

2) r (t; a; b)= (ab)sgn [sin (2 t)] + a + b ;

PROPERTY 2.2:

1; if y>0;

(n)

t 6= ;k =0; :::; 2 ; sgn (y )=

1) R (x; a; b)=a+(ba)x (x); k =1; :::; n;

1; if y<0: k

3) There is the following representation of the

where n is a natural numb er, and x,0x2 1is

Rademacher functions of I I-typ e:

written in the form

s (t; a; b)=[a+(ba)c (t)] c (t): (6)

n+1 n+1 i

nj 1

x = x (x) 2 ;j=0;1; :::; n 1: (10)

i=1

j =0

4) There is the following relation b etween the Rademacher

T n2 2

(a; b)-functions of I-typ e and I I-typ e:

2) R (n; a; b) R(n; a; b)=2 [(a b) I +(a +

(n)

b) J ];

(n)

2 1

where I is the identity matrix of order n and J is the

(n) (n)

r (t; a; b)= s (t ;a;b): (7)

n+1 n+1

(n n) matrix whose all entries are equal to 1:

m=0

In particular, for the classic Rademacher matrices

T n

R (n; 1; 1) R(n; 1; 1)=2 I :

(n)

For a natural number n and integers a and b we de ne the

following discrete functions:

3) Let n b e a natural numb er, 1 k n and supp ose that

2x+1

x,0x2 1 is written in the form (10). Then

(n)

;a;b); (8) R (x; a; b)=r (

n+1

k 1

(n)

k =0;1; :::; n 1; x =0;1; :::; 2 1; n 0:

S (x; a; b)=[a+(ba)x (x)] [1 x (x)]; k =1; :::; n:

k i

i=0

2x+1

(n)

(11)

S (x; a; b)=s ( ;a;b); (9)

n+1

(i)

T i

4) S (n; a; b) S(n; a; b)=a(a+b)[J + 2 P ]

(n)

i=1

k =0;1; :::; n 1; x =0;1; :::; 2 1; n 0:

+b(b a) D ;

(n)

where J is the (n n) matrix whose all the entries are

DEFINITION 2.2 : The rectangular (2 (n + 1))

(n)

(i)

(n)

equal to 1; P is the (n n) matrix whose leftmost (n

matrix R(n; a; b); whose (x; k ) elementisR (x; a; b),

(n)

x =0;1;;2 1, k =0;;n, is called Rademacher (a; b)

i) (n i) submatrix is J ;i=1; :::; n 2; and

(ni)

matrix of I-type (of order n). The rectangular (2 (n + 1))

n1 n2

(n)

D = diag (2 ; 2 ; :::; 2; 1):

matrix S(n; a; b); whose (x; k ) elementisS (x; a; b), x =

(n)

0; 1; ;2 1, k =0;;n, is called the Rademacher (a; b)

matrix of II-type (of order n).

REMARK 2.1 : For each pair of integers (n; k ), n>0,

2.2. (a; b; )-Polynomial Functions of I-Typ e

1 k n, the k column of R(n; a; b) de nes a func-

Let b e an arbitrary asso ciative arithmetical or logical op-

tion f0; 1; ;2 1g!fa; bg. This set of functions is

eration, (m ;m ; :::; m ) b e the binary representation

0 1 n1

called the system of discrete Rademacher (a; b) functions of

n1i n

of an integer m = m 2 ; 0 m 2 1. Let

I-type. Analogously, for each pair of integers (n; k ), n>0,

i=0

also H G b e the Kronecker pro duct of the matrix H and

1 k n, the k column of S(n; a; b) de nes a function

n th

the matrix G; and H b e the Kronecker n power of the

f0; 1; ;2 1g!fa; bg; the system of which is called the

. matrix H; i.e. H = H H ::: H

system of discrete Rademacher (a; b) functions of II-type.

{z } |

n times

EXAMPLE 2.1 : When n =4;the Rademacher (a; b)-

(n)

T T We form a class of functions (x; a; b; ) from the

matrices of I-typ e R (4;a;b) and I I-typ e S (4;a;b)have

system of discrete Rademacher (a; b)-functions of I typ e (i.e.

the following forms, resp ectively:

(n)

from fR (x; a; b)g ) in the following way. Let

1 0

m=0

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

(n) (n)

a a a a a a a a b b b b b b b b

C B

(x; a; b; )= (x; a; b; )

m m ;m ;:::;m

0 1 n 1

C B

a a a a b b b b a a a a b b b b

A @

m m

0 n1

(n) (n)

a a b b a a b b a a b b a a b b

=[R (x; a; b)] [R (x; a; b)] ; (12)

0 n1

a b a b a b a b a b a b a b a b

where is an arithmetical or logical op eration. If is a

1 0

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

logical op eration, then we naturally assume a; b 2f0;1g.

a a a a a a a a b b b b b b b b

C B

(n)

C B

a a a a b b b b 0 0 0 0 0 0 0 0

DEFINITION 2.3 : The functions (x; a; b; );m=

A @

a a b b 0 0 0 0 0 0 0 0 0 0 0 0

0; :::; n 1, are called (a; b; )-polynomial functions.If is

a b 0 0 0 0 0 0 0 0 0 0 0 0 0 0

an arithmetical (logical) op eration, the (a; b; )-p olynomial

where T denotes matrix transp osition. functions are called arithmetical (logical).

DEFINITION 2.4 : The square matrix (n; a; b; )= DEFINITION 2.6: The square matrix H(n; a; b)

(n) (n)

n n

[ (x; a; b; )];x;m=0;1; :::; 2 1; which has as columns =[h (x; a; b)];x;r=0;1; :::; 2 1; which has as columns

m r

the (a; b; )-p olynomial functions, is called the (a; b; )-poly- the (a; b)-p olynomial functions of I I-typ e, is called the (a; b)-

nomial matrix. polynomial matrix of II-type.

Note that the classical Haar functions (matrices) are the

Let be multipli cati on op eration. The (a; b; )-p olynomial

(n)

particular case of the (a; b)-p olynomial functions (matrices)

functions are formed via Rademacher (a; b)-functions

of I I-typ e where a =1;b = 1. The Haar-conjunctive func-

(n)

R (x; a; b; )by the following formula:

tions (matrices) will b e (0; 1)-p olynomial functions (matri-

ces) of I I typ e.

(n)

(n) m

(x; a; b; )= [R (x; a; b)] ;

EXAMPLE 2.2: The Haar-conjunctive H matrix is:

i=0

2 3

1 0 0 0 0 0 0 0

where m is the i bit in the binary representation of m

1 0 0 0 1 0 0 0

n1 6 7

n1i

m 2 : via m =

6 7

1 0 1 0 0 0 0 0

i=0

6 7

When a =1;b = 1; the (a; b)-p olynomial functions

6 1 0 1 0 0 1 0 0 7

H = :

6 7

coincide with the Walsh functions, and when a =0;b =1;

1 1 0 0 0 0 0 0

6 7

they coincide with the Reed-Muller (conjunctive) functions

1 1 0 0 0 0 1 0

4 5

[1].

1 1 0 1 0 0 0 0

Based on Prop erty 2.2, 1) wehave the following expres-

1 1 0 1 0 0 0 1

sions:

(n) m

(x; 1; 1) = (1 2x ) ; (13)

3. BINARY POLYNOMIAL TRANSFORMS

i=0

for the Walsh functions, and

3.1. (a; b)-Polynomial Transforms of I I-typ e as Bi-

nary Wavelet Transforms

(n) m

(x; 0; 1) = x ; (14)

Binary Wavelet Transform emerged from the application of

i=0

wavelet theory in nite eld with two elements GF(2) [4, 7 ].

It is highly advantageous in computational p oint of view,

for the conjunctive functions.

since the computations are p erformed by "exclusive or" and

Let now b e an arbitrary asso ciative binary logical op-

"and" op erations. Although smo othness and vanishing mo-

eration, a = 0 and b =1: Let m and x have binary repre-

ments prop erties of real discrete wavelet transforms [9] do

sentations (m ; ;m ) and (x ; ;x ) resp ectively.

0 n1 0 n1

not make sense in GF(2), BWT is useful to lo calize data in

Then, by Prop erty 2.2, the formula (12) b ecomes

time and frequency and separately enco de rapid and slow

changes across the data. In that sense, an example for a

(n) (n) n1 m

(x; 0; 1; )= (x; 0; 1; )= x :

m m ;:::;m

i=0

0 n 1

one stage BWT is de ned as

Examples of the -p olynomial logical functions are:

I 0 I 0

a) conjunctive, if is the op eration ^ (AND); denoted BW T = L (16)

0 I I I

(n)

by (x; ^);

b) disjunctive, if is the op eration _ (OR); denoted by

in [4], where L is a p ermutation matrix called "lazy wavelet

(n)

(x; _); transform" such that L(x ;x ;:::)=[x ;x ;: ::;x ;x ;:::].

m 1 2 2 4 1 3

c)antivalent, if is the op eration (XOR); denoted

Wavelet vectors resulting from (16) are exactly the same as

(n)

the columns of Haar-conjunctive matrix ((0; 1)-Polynomial

by (x; ):

Matrix of I I-typ e) intro duced in the previous Section.

2.3. (a; b)-Polynomial Functions of I I-typ e

3.2. (a; b)-Polynomial Functions of I-typ e as Dis-

crete Wavelet Packet Transforms

First de ne the shifted Rademacher (a; b)-functions of I I-

typ e, namely

Wavelet packet transform is a generalization of wavelet trans-

form which o ers a wider range of analysis p ossibili ti es for

(k ) (k)

s (x; a; b)=s (x ;a;b); (15)

the signal [9]. It is asso ciated with a b est basis selec-

tion algorithm which selects a sub decomp ositi on structure

among all p ossible decomp osition structures presented by

where m 2f0;1; :::; 2 1g:

the packet transform, sub ject to a criterion. Wavelet Packet

(n)

Transform can b e b etter visualized by a full binary tree,

DEFINITION 2.5: The functions h (x; a; b)

(n) (n)

where left and right branchings representlowpass and high-

(0) (1)

= s (x; a; b);h (x; a; b)=s (x; a; b); and h (x; a; b)

2 +m

pass lterings, resp ectively, and b est basis selection corre-

(k +1)

= s (x; a; b);m=0;1; :::; 2 1;k=1; :::; n 1; are

sp onds to extracting a subtree of the binary tree.

called (a; b)-polynomial functions of II-type, and they form

Consider the Walsh matrix W of order n in Hadamard

the system of (a; b)-polynomial functions of II-type.

ordering. It corresp onds to a Wavelet Packet Transform

where with [1 1] and [1 1] b eing the lowpass and highpass lters,

a 1

resp ectively. The conjunctive matrix K of order n is also

]; (24) I G =[I

k1 k k1

b 1

awavelet packet transform in GF(2) with [1 0] and [1 1]

acting as the lowpass and highpass lters, resp ectively.

where

1 a

H = G =

1 1

1 b

3.3. Ecient computation algorithms

and I is the identity matrix of order 2 :

3.3.1. Fast (0,1)-RademacherTransforms of I-type

LEMMA 3.1: The (0; 1)-Rademacher matrix R =

4. BINARY POLYNOMIAL TRANSFORMS IN

R(n; 0; 1) of I-typ e can b e represented as

NONLINEAR FILTERING

The use of BPT in nonlinear ltering was discussed in de-

(j )

R = R ; (17)

tails in [1]. Here we just showvery simple and very ef- n

cient realization for a threshold lter [3]. The threshold

j =0

lter (TF) S (X ) based on the M -variable discrete func-

where

tion f (): f0;1g !< maps the signal X into the output

I 0

(0) n1 n1

signal Y = fY (k );k =1;2; :::; Lg; where

= R ; (18)

I 1

n1 n1

Y (k )=S (X(k)) = f ( (X(k ))); (25)

f n

I 0 0

n1j n1

(j )

n=1

= I 1 0 R ;j=1;2; :::; n 2; (19)

n1j n1

0 0 I

(j )

(:) is the vectoral threshold function with the elements

1 if X n

0 1

de ned by (X )=

1 0 0

0 otherwise.

Denote x = (X));j=1; :::; M : De ne an indicator

B C

j j

B C

(n1)

R = ; (20)

B C

@ A

(x )=[ (x ); :::; (x )]; j =1; :::; M ; (26)

j 0 j j

2 1

. I

(n)

where

I (resp ectively, I ) is the identity matrix of order 2

(j )

1; if (k ; :::; k )=x ;

N 1 0 j

(x )= (27)

k j

I is an opp osite identity matrix (resp ectively, of order j ),

(n)

0; otherwise,

of order n; 0 (resp ectively, 1 ) are column-vectors of

j j

length 2 ; consisting of zeros (resp ectively, of ones).

j =1; :::; M ;(k ; :::; k ) is the binary vector represen-

N1 0

tation of the p ositiveinteger k; k =0; :::; 2 1:

Denote by h the histogram vector of x ;j=1; :::; M ;

3.3.2. Fast (a; b; )-Polynomial Transforms of I-Type

i.e.

LEMMA 3.2 : The (a; b; )-p olynomial matrices =

h = (x ): (28)

(n; a; b; ) can b e expressed by the following formula:

j =1

=(G ) ; (21)

n n

The matrix analogue of the formula (25) is the following

(index n is omitted)

where

1 0

y = f h ; (29)

1 a 0 0 ::: 0 0

C B 0 0 1 a ::: 0 0

C B

where h is the transp osed vector of the histogram vector

::: ::: ::: :::

C B

h (28). Let us now mo dify (29) to the form

0 0 0 0 ::: 1 a

C B

(22) G =

C B

1 b 0 0 ::: 0 0

C B

y = h H g ; (30)

0 0 1 b ::: 0 0

C B

A @

::: ::: ::: :::

with

0 0 0 0 ::: 1 b

T 1

g = H f ; (31)

where H is any nonsingula r matrix. In fact, realization of

threshold lter in [3] follows directly from (30)-(31) when

3.3.3. Fast (a; b)-Polynomial Transforms of II-Type

H = K is the conjunctive (Reed-Muller) matrix. Nev-

ertheless, there exists much simpler and ecient realiza-

LEMMA 3.3 : The (a; b)-p olynomia l matrices of I I-typ e

tion of this lter using Rademacher-sorting network without

H = H(n; a; b) can b e expressed by the following formula:

computation of the threshold histogram h: The owchart

structure of Rademacher-sorting network is similar to the

owchart of the fast Rademacher transform, with changing

H = G diag (G ; I ; :::; I ); (23)

n n nj j n1

of addition op erations to minimum op erations.

j =1

5. BINARY POLYNOMIAL TRANSFORMS IN

Table 1: Compression results for thresholded 256 256

COMPRESSION OF BINARY IMAGES

Lena, 128 128 Ball, 128 128 Bird and 512 512 Text

images

Binary image compression may nd applicatio ns in com-

pressing bi-level high resolution images such as fax pages,

org. Lenna S&P S BT

scanned images or segmentation data and bit-plane by bit-

no.nonzero 26924 3940 3688 4251

plane compression of multi-level images in some sp eci c

Entropy 0.98 0.39 0.37 0.34

cases. Recently,multiresolution analysis via wavelet trans-

form has found ecient applicati ons in multilevel image

org.Bird S S&P BT

compression ( [9]). The success of multilevel image com-

no.nonzero 12940 843 747 730

pression with wavelets has b een followed by applicati ons of

Entropy 0.74 0.35 0.32 0.26

wavelet theory over the nite eld GF(2) ( [4, 7 ]). Swanson

org. Ball S&P S BT

and Tew k develop ed a theory of binary wavelet transform

no.nonzero 56715 1727 1599 1531

in terms of two band p erfect reconstruction lter banks in

Entropy 0.75 0.06 0.06 0.05

( [7]). Their test results with binary image compression

org. Text S S&P BT

achieved promising results in terms of rst order entropy.

no.nonzero 3174 366 318 399

So on after Johnston ( [4]) used another approach (lifting

Entropy 0.71 0.19 0.17 0.16

scheme ( [8])) to construct a binary wavelet transform which

is a cascade of binary matrices comp osed of upp er and lower

unimo dular binary blo cks and a simple p ermutation matrix

collecting even samples to upp er half and o dd samples to

real wavelet transforms for lossy and lossless compression,

lower half (see (16)). Here we will presenthow to obtain

they is also suitable for binary image compression b ecause

similar binary transforms via binary p olynomial matrices.

they pro duce integer outputs in a very limited range for

Consider the Reed-Muller matrix K of order M

binary data. The prop osed binary transform p erforms b et-

1 0

ter than b oth S and S&P in terms of rst order entropyof

K = K = (32)

1 1

transformed co ecients. It satis es signi cant reduction of

entropy from the original image.

and a binary vector f of length 2 . The Reed-Muller

Transform of f can b e de ned as

6. REFERENCES

f = K f ;

RM M M

[1] Agaian, S.S., Astola, J., and Egiazarian, K. (1995)

Binary Polynomial Transforms and Nonlinear Digital

where the output f is computed in mo dulo 2 arithmetic.

Filters, Marcel Dekker Inc.

f corresp onds to a wavelet packet representation of f

RM M

which can b e obtained by a full wavelet tree decomp osi-

[2] Beauchamp, K. G. (1975). Walsh Functions and Their

tion. In this decomp osition rst and second rows of K

Applications, Academic Press.

act as lowpass and highpass lters, resp ectively. The ab ove

[3] Jeong, H., Song, J., and Lee, Y. H. (1995). Extended

transform can b e generalized into two-dimensional case for

threshold Bo olean lters: representation and design,

images as

Proc. of 1995 IEEE Workshop on Nonlinear signal pro-

T T

F = K F K ; (33)

RM M

cessing, ed. I.Pitas, vol. I I, pp. 911-914.

where F is a binary matrix representing pixel values of a bi-

[4] Johnston, C. P. (1996). The Lifting Scheme and Finite

nary image. Note that (33) is an invertible transform where

Precision Error Free Filter Banks, SPIE vol. 2825, pp.

the inverse of K in GF(2) is equal to itself. We can obtain

307-316.

the b est multiresoluti on decomp osition of F,by selecting a

[5] Rademacher, H. (1922). Einige Satze von Allgemainen

sub-decomp osition structure covered by K : The sub de-

Orthogonal Funktionen, Math. Annalen, 87, pp. 122-

comp osition structure can b e obtained by forming a matrix

138.

such that

A 0

M 1

[6] Said A. and Pearlman, W.A. (1996). An Image Mul-

; (34) K =

B C

tiresolution Representation for Lossless and Lossy

Compression, IEEE Transactions on Image Process-

M 1 M 1

where 0 is the zero matrix with size 2 2 , and

M 1

ing,vol. 5, no. 9, pp. 1303-1310.

A, B, C can b e either 0 or K with the restriction

M 1

[7] Swanson, M.D. and Tew k, A.H. (1996). A Bi-

that

nary Wavelet Decomp osition of Binary Images, IEEE

K = K :

Transactions on Image Processing,vol. 5, no. 12.

Some quantitative results of the ab ove binary transform

[8] Sweldens, W. (1994). The Lifting Scheme: A Custom-

are presented in Table 1. For all four test images, the log-

Design Construction of Biorthogonal Wavelets, Tech-

arithmically growing decomp osition tree is found to give

nical report IMI 1994:7, revised version.

the b est p erformance in terms of minimum numb er of ze-

ros. Hence, the only overhead information needed is the

[9] Vetterli, M. and Kovacevic, J. (1995). Wavelets and

depth of the tree. For comparison, the results of S and S&P

Subband Coding, Prentice Hall.

Transform [6] are also included. Although S and S&P are