Chapter 3 Multimedia Systems Technology: Co ding and Compression 3.1 Intro duction In multimedia system design, storage and transport of information play a sig- ni cant role. Multimedia information is inherently voluminous and therefore requires very high storage capacity and very high bandwidth transmission capacity.For instance, the storage for a video frame with 640 480 pixel resolution is 7.3728 million bits, if we assume that 24 bits are used to en- co de the luminance and chrominance comp onents of each pixel. Assuming a frame rate of 30 frames p er second, the entire 7.3728 million bits should b e transferred in 33.3 milliseconds, which is equivalent to a bandwidth of 221.184 million bits p er second. That is, the transp ort of such large number of bits of information and in such a short time requires high bandwidth. Thereare two approaches that arepossible - one to develop technologies to provide higher bandwidth of the order of Gigabits per second or more and the other to nd ways and means by which the number of bits to betransferred can be reduced, without compromising the information content. It amounts to saying that we need a transformation of a string of characters in some representation such as ASCI I into a new string e.g., of bits that contains the same information but whose length must b e as small as p ossible; i.e., data compression. Data compression is often referred to as co ding, whereas co ding is a general term encompassing any sp ecial representation of data that achieves a given goal. For example, reducing the numb er of bits is by itself a goal. In 1948, Claude Shannon, the father of Information Theory,intro duced 77 78 Multimedia Systems Technology: Co ding and Compression Ch. 3 Source Destination Encoder Channel Decoder (Sender) (Receiver) Fig. 3.1. Shannon's Communication Systems Mo del a concept called entropy to measure the information content of a source[1]. Shannon prop osed a communication system mo del shown pictorially in Fig. 3.1 which addresses two basic issues: How can a communication system eciently transmit the information that a source pro duces? How can a communication system achieve reliable communication over a noisy channel? The rst issue relates to compression and the second relates to error control. In b oth cases, there is a transformation of number of bits that represent the information pro duced by a source. These two asp ects are also known as sourcecoding and channel coding. Information theory is the study of ecient co ding and its consequences in the form of sp eed of transmission and probability of error. In this chapter we fo cus on source co ding compression and standards for compression. Information co ded at the source end has to b e corresp ondingly deco ded at the receiving end. Co ding can b e done in suchaway that the information content is not lost; that means it can b e recovered fully on deco ding at the receiver. However, media such as image and video meant primarily for human consumption provide opp ortunities to enco de more eciently but with a loss. Co ding consequently, the compression of multimedia information is 1 sub ject to certain quality constraints. For example, the quality of a picture should b e the same when co ded and, later on, deco ded. Dealing with p er- ceptual quality it is p ossible for one to design a co ding metho d that maybe lossy. By lossy co ding we mean that there is information loss, but the loss do es not a ect the p erceptual quality. In practice, one or more parameters related to the source may b e used in designing co ding schemes that result in lossy compression. Also, the complexity and the execution time of the tech- niques used for co ding and deco ding should b e minimal. These constraints 1 By quality,we mean p erceptual quality. Sec. 3.2. What is Image Compression? 79 are related to the nature of the applicationlive or orchestrated as well. It should b e noted that all these requirements are based on the characteristics of human p erception of di erent media. For example, for interactive mul- timedia application such as video conferencing, the end-to-end delay should b e less than 150 milliseconds. In the context of current high-sp eed transmis- sion technologies, such an end-to-end delay requirementwould imply that compression and decompression, if any, should b e contained within 50 mil- liseconds. Similarly,multimedia applications that are retrieval based, such as on-demand video service, require that random access to single images and audio frames b e p ossible under 500 milliseconds. Co ding and compression techniques are critical to the viabilityofmul- timedia at b oth the storage level and at the communication level. Some of the multimedia information has to b e co ded in continuous time dep endent format and some in discrete time indep endent format. In multimedia con- text, the primary motive in co ding is compression. By nature, the audio, image, and video sources have built-in redundancy, which make it p ossible to achieve compression through appropriate co ding. As the image data and video data are voluminous and act as prime motivating factors for compres- sion, our references in this chapter will often b e to images even though other data typ es can b e co ded compressed. 3.2 What is Image Compression? Image data can b e compressed without signi cant degradation of the visual p erceptual quality b ecause images contain a high degree of: spatial redundancy, due to correlation b etween neighb oring pixels this is also referred to as statistical redundancy, sp ectral redundancy, due to correlation among color comp onents, and psycho-visual redundancy, due to p erceptual prop erties of the human visual system. The higher the redundancy, the higher the achievable compression. This is what we referred to as sourcecoding earlier in the communication system of Shannon. A typical image compression system or source enco der consists of a transformer, quantizer, and a co der, as illustrated in Fig. 3.2. Transformer applies a one-to-one transformation to the input image data. The output of the transformer is an image representation whichis 80 Multimedia Systems Technology: Co ding and Compression Ch. 3 Image Representation that is more amenable to compression Symbols Input Image Raw Image Binary Data Transformer Quantizer Coder BitStream Fig. 3.2. Typical Image Compression System more amenable to ecient compression than the raw image data. Unitary mappings such as Discrete Cosine Transform, which pack the energy of the signal to a small numb er of co ecients, is a p opular metho d with image compression standards. Quantizer generates a limited numb er of symb ols that can b e used in the representation of the compressed image. Quantization is a many-to- one mapping which is irreversible. It can b e p erformed by scalar or vector quantizers. Scalar quantization refers to element-by-element quantization of data and vector quantization refers to quantization of a blo ck at a time. Co der assigns a co de word, a binary bit-stream, to each symb ol at the output of the quantizer. The co der may employ xed-length or variable- length co des. Variable Length Co ding VLC, also known as entropy co ding, assigns co de words in suchaway as to minimize the average length of the binary representation of the symb ols. This is achieved by assigning shorter co de words to more probable symb ols, which is the fundamental principle of entropy co ding. Di erent image compression systems are based on di erent combinations of transformer, quantizer, and co der. Image compression systems can b e broadly classi ed as: Lossless: Compression systems, which aim at minimizing the bit-rate of the compressed output without any distortion of the image. The decompressed bit-stream is identical to the original bit-stream. This metho d is used in cases where accuracy of the information is essential. Examples of such situations are computer programs, data, medical imaging, and so on. Lossless compression systems are also referred to as bit-preserving or reversible compression systems. Lossy: Compression systems, which aim at obtaining the b est p ossible delity for a given bit-rate or minimizing the bit-rate to achieve a given Sec. 3.3. Taxonomy of Compression Techniques 81 delity measure. Such systems are suited for video and audio. The transformation and co ding stages are lossless.However, the quantiza- tion is lossy. 3.3 Taxonomy of Compression Techniques Based on the lossless or lossy compression, the enco ding is broadly classi ed as entropy encoding leading to lossless compression and source encoding leading to lossy compression. It is interesting to note that the term entropy encoding refers to all those co ding and compression techniques which do not takeinto account the na- ture of the information to b e compressed. In other words, entropy enco ding techniques simply treat all data as a sequence of bits and ignores the seman- tics of the information to b e compressed. On the contrary, the source encoding takes cognizance of the typ e of the original signal. For example, if the original signal is audio or video, source enco ding uses their inherentcharacteristics in achieving b etter compression ratio. Normally, it is exp ected that source enco ding will pro duce b etter compression compared to entropy enco ding techniques.