ERROR CORRECTION BY MEANS OF ARITHMETIC CODES: AN APPLICATION TO RESILIENT IMAGE TRANSMISSION M. Grangetto, G. Olmo P. Cosman CERCOM, Center for Multimedia University of California, San Diego, Radio Communications Dept. of Electrical and Politecnico di Torino Computer Engineering, Dipartimento di Elettronica 9500 Gilman Drive, La Jolla, Corso Duca degli Abruzzi 24 CA 92093-0407, USA 10129 Torino - Italy E-mail: [email protected] E-mail: (grangetto,olmo)@polito.it ABSTRACT Much research effort has been devoted to the joint source chan- nel decoding of variable length codes, in particular Huffman codes In this paper, two novel maximum a posteriori (MAP) estimators [5, 6, 7]. In [5, 6], the residual redundancy in the source encoder for the decoding of arithmetic codes in the presence of transmis- output is represented with a Markov model, and is used as a form sion errors are presented. Trellis search techniques and a forbidden of implicit channel protection at the decoder side; exact and ap- symbol are employed to obtain forward error correction. The pro- proximate maximum a posteriori (MAP) sequence estimators are posed system is applied to lossless image compression and trans- proposed. Results are provided in the case of image transmis- mission across the BSC; the results are compared in terms of both sion across the binary symmetric channel (BSC). In [7], soft MAP performance and complexity with a traditional separated source source decoding is investigated and applied to the transmission of and channel coding approach based on convolutional codes. MPEG-4 bitstreams. In this paper, the JSCC approach is applied to Arithmetic Cod- 1. INTRODUCTION ing (AC). AC is nowadays the most powerful entropy coding tool [8], and is replacing Huffman coding in novel standards such as Universal access to multimedia data is one of the major objectives JPEG2000 and H.26L. On the other hand, AC is extremely fragile of emerging communication systems. The extension of services in the presence of transmission errors; unlike Huffman codes, AC offered to mobile users, from traditional voice traffic to complex has poor resynchronization capability, and a single bit error in the data sources such as web-pages, images, video and music along compressed stream can propagate all along the compressed block. with the constraints imposed by the tetherless environment are Moreover, the residual redundancy in the compressed stream is boosting a considerable amount of research in the field of wireless usually negligible, preventing any MAP decoding attempt. Never- multimedia communications. In particular, many interdisciplinary theless, it is possible to perturb the arithmetic coder source model solutions are being investigated, and the interfaces among system in order to keep some residual redundancy at the expense of com- layers are becoming richer in information content [1]. This ap- pression efficiency. This idea was first introduced by Boyd et al. proach is noticeable in a number of traditionally separated fields: in [9] and extended in [10, 11] to provide continuous error detec- novel compression standards, such as JPEG2000 [2] for still im- tion during arithmetic decoding. The presence of known residual ages and H.26L [3] for video sequences, are improving compres- redundancy can be exploited for error correction as well. Some sion efficiency, but at the same time they devote great attention preliminary work can be found in [12], where the error correction to transmission and error resilience; conversely CDMA2000 and is performed in the case of transmission over an AWGN channel; UMTS systems are being designed with multimedia traffic in mind. binary signalling with null zone soft decoding is employed. The This scenario is generating a great interest in the development performance is evaluated in terms of packet recovery rate for dif- of novel and efficient Joint Source/Channel Coding (JSCC) tech- ferentially encoded images. In [7], a JSCC concatenated scheme niques. Channel bandwidth and power constraints, delay limita- based on AC and trellis coded modulation is presented and applied tions and error protection required by the application are empha- to image transmission as well. sizing the practical shortages of Shannon’s source-channel sepa- In this paper, we address the problem of MAP decoding of AC ration theorem [4], when applied to mobile multimedia communi- in the presence of transmission errors; AC with a forbidden sym- cations. JSCC techniques are founded on the fact that in practi- bol and trellis search techniques are used. The decoding task is cal cases the source encoder is not able to exactly decorrelate the formulated in terms of classical MAP estimation. Some prelimi- input sequence; some implicit redundancy is still present in the nary research by the same authors was presented in [13] for lossy compressed stream and can be properly exploited by the decoder image compression. In the present paper, novel results are pre- for error control. As a consequence, it is possible to improve the sented in the case of lossless compression of grayscale images and decoder performance by considering source and channel coding transmission across the BSC. The performance is compared with jointly. a standard separated source and channel scheme based on conven- in the following). The input sequence is mapped onto the binary a b r ¡ ¢ £ ¤ ¥ a^ ¡ ¢ £ ¤ ¥ ! KLM O #"N$ string of variable length bits, and transmitted ¤ @QPR © ¤ © ¡ £ ¢ £ ¡ ¦ § ¨ © © § & SC £ ¢ £ ¡ ¡ across the channel with transition probability . ¦ § ¨ © © § P P(r/b) The received sequence , possibly affected by errors, is pro- cessed by the MAP estimator that selects the most probable se- @ RYP @ RYP @ RYP US& B)VXWT C/Z[& C]\^`_)ba & C quence T . repre- sents the so called decoding metric, and can be expressed as @QPR @ @ Fig. 1. Transmission system block diagram. @QPR @ RYP & NCH& fgC & dCH& eC @QP @QP ) Cc) & (1) & C & C tional AC and rate compatible convolutional codes. @ eC & represents the a priori probability of transmitting the @QP & C string . On the other hand, the term cannot be easily eval- 2. ARITHMETIC CODING WITH A FORBIDDEN uated and in the following it will be approximated by hji]k , where SYMBOL O is the length in bits of the received string. This approximation assumes that all the received sequences of equal length are equally The objective of AC is to map a sequence of input symbols onto a likely; the assumption is not satisfied by variable length AC; how- binary string representing the probability of the input sequence. ever the exact evaluation of this term would require as much effort This is accomplished according to the available source model, and as the MAP decoding itself and it is not feasible in practice. This the compression efficiency mainly depends on the accuracy of the simplification is proposed also in [5, 6] in the case of Huffman model. codes and it provides satisfactory results. In this paper, we consider the particular case of the binary The most direct approach to MAP decoding should be the memoryless source; bits generated by grouping outcomes of evaluation of metric (1) for the subset l , containing the code- the binary memoryless source constitute the fixed length frame to k O words N of length . However, for reasonable input string length ! be encoded, i.e., #"%$ . This source is fully described by , the exhaustive approach is infeasible, and it is essential to resort &($*),+.-/&%' the probability of the binary symbols, &%' and respec- to a suboptimal criterion in order to reduce the search space dimen- tively. sion. A number of techniques for visiting trees and trellises have The AC encoding is an iterative task, performed by progres- been proposed in the past, the most popular one being the Viterbi sively refining the probability interval to which the input frame algorithm; a complete survey can be found in [14]. These tech- belongs. The output sequence corresponds to the shortest binary niques usually require i) a trellis representation of the search space string, which represents a binary value contained in the interval; and ii) an additive branch metric. We can recast the search for the decoding follows the dual process. It is worth remarking that both O best N as a search among all possible binary strings of length encoding and decoding can be performed sequentially, by apply- !o !o mn l Mmr #"%$ #"N$qp . The can be represented by a tree that ing interval normalization strategies. The recursive decoding task k grows exponentially with O . The metric (1), in its logarithmic is extremely sensitive to bit errors. Even a single flipped bit in form, can be easily decomposed into additive branch terms. The the output string can cause irreversible desynchronization. Para- channel term is computed comparing the received P sequence and doxically, it is this poor resynchronization ability that allows pow- the explored branch. The source term is obtained attempting par- erful continuous error detection. In [9, 10], a Forbidden Symbol R sl tial arithmetic decoding of a given tree path; in the case m , k &%12)43 (FS) 0 with probability is introduced in the input alpha- the FS will be revealed with a certain delay, and the explored path bet, but it is never transmitted. The introduction of the FS im- will be pruned. plies a perturbation of the source model by a factor +5-63 , thus For tree exploration, we tested two well known techniques reducing compression efficiency. The rate redundancy amounts to @ 798 known as stack algorithm (SA) and the M-algorithm (MA) [14]. +A-B3DC )4-;:=<?> bits/symbol [10]. If the FS is decoded, this SA is a metric first technique: the best path selection is based means that transmission errors have occurred. In [10], it is shown on a greedy approach, extending at each iteration the best stored that the probability that the number of erroneously decoded bits @ path, i.e., the one with the best accumulated metric (1).