A Perceptually Adaptive JPEG Coder
Total Page:16
File Type:pdf, Size:1020Kb
A perceptually adaptive JPEG coder Henry Hoi-Yu Tong A thesis submitted in conformity with the requirernents for the degree of Master of Applied Science. Graduate Department of Electrical and Computer Engineering, University of Toronto @Copyright by Henry Tong 1997 National Library Bibliothéque nationale 1*1 of Canada du Canada Acquisitions and Acquisitions et Bibliographie Services services bibliographiques 395 Wellington Street 395. rue Wellington OttawaON K1AON4 OttawaON K1AON4 Canada Canada The author has granted a non- L'auteur a accordé une licence non exclusive licence aiiowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or seli reproduire, prêter, distribuer ou copies of this thesis in microfom, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/film, de reproduction sur papier ou sur format électronique. The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantid extracts fiom it Ni la thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation. A perceptually adaptive JPEG coder A thesis for the Dcgree of' 3Iaster of Applied Science, 1997 b'- Henry Hoi-Yu Tong Department of Electrical and Cornputer Engineering University of Toronto Abstract A pcrceptually adaptive -JPEG coder is irnplernented in this research. The major part of the implementation involves the design of a perceptual model that is based on the texture masking and luminance masking properties of the human visual system (HVS). The main objective is to compute a local multiplier map that can be used to scale the quantization matris (QM) so that fewer bits are used to represent the perceptually less important areas of an image. The texture masking model is based on a proposed block classification algorithm to differentiate between the plain. edge. and texture blocks. An adaptive luminance masking scherne is also proposed n-hich adaptively adjusts the luminance masking strategy depending on an image's mean luminance value. Experimental results show that the adaptive coder provides savings in bit-rate over baseline JPEG. with no overall loss in perceptual quality according to a subject test. Acknowledgment s It is my pleasure to acknowledge and thank al1 people who have helped rnc through- out this researcli. 1 would like to thank rny supervisor, Prof. .A. N. Venetsanopoulos for his continucd support and guidance. Special thanks are due to Kostas Plataniotis for providing invaluable suggestions and ideas that have helpctl so much. I am also grateful to Huawu Lino Ido Rabinovitch, Lowe11 Winger, and Xicos Herodotou for the estremelÿ useful advicc, suggestions, and the friendly conversations we had during the long hours in the DSP lab. I would like to thank Prof. Iiatzela, Prof. Lee, and Prof. Sousa for scrving in the oral examination conimittee. 1 would also likc to thank Sicos and Lyriral Peart for helping me prepare for the examinat,ion. and Dimitrios Androiitsos and Kostas for proofreading the t hesis. Special thanks are also owed to al1 my friends in the communications group and the university for offering encouragement, helpful suggestions. and friendly chats. I would also like to acknowledge the Independent JPEG Group for providing the .JPEG software that forms the bais source code for the research. 1 wish to thank the following individuals for participating in the subjective testing and providing me with feedback and useful comments that were invaluable in com- pleting the perceptual aspect of the thesis: Changshan Xiao. Sheldon Tarn, Wing Tarn, Lowell, Michael Yu , Kostas, Anestis Karasaridis, Peter Androutsos. Evan' Wai Ming Louie. Liiian Herodotou, Stanley Ma, Kevin Gior, Halim Yanikomeroglu. Chao Zeng, Yimen, Dimitrios, Karen Cheung, and Ido. Last but certainly not least, my special thanks to my family for their encourage- ment, patience and understanding, despite the many long distance phone calls that 1 missed during the course of this research. Contents 1 Introduction 1 1.1 Rcsearch objectives and thesis oiitline .................. 3. 2 Review of image compression and some human visual system prop- erties 4 2.1 Classical information t heory ....................... 1 2.2 Lossless and lossy image compression .................. 8 2.3 Relevant human viaual system properties ................ 12 2.3.1 Colour space and irnage coding ................. 12 23.2 The filtering and masking properties .............. 14 3 The JPEG image compression standard 19 3.1 The transform coding mode1 ....................... 21 3.1.1 Tracsformation .......................... 21 3.1.2 Quantization ........................... 37-- 3.1.3 Entropy coding .......................... 26 4 Design of the adaptive JPEG coder 28 4.1 Perceptually adaptive JPEG coding ................... 28 4.1. The two adaptive coding modes ................. 30 4.1.1.1 The JPEG compatibility mode ............ 30 4.1.1.2 The overhead mode .................. 31 4.1.1.3 The difference between the two adaptive modes ... 33 4.2 Design of the perceptrial mode1 ..................... 3.5 4.2.1 Linear threshold elevation modeling ............... 36 4.2.2 The testure rnasking mode1 ................ .. 36 4.3.2.1 Block classification ................... 37 4.2.2.2 Calculation of the texture masking factor ...... 45 -4.2.3 The luminance niasking mode1 .................. 17 423.1 The luminance sensitivity subjective esperiment ... 47 1.2.3.2 The influence of the viewing conditions ........ 19 4.2.3.3 Calculation of the luminance masking factor ..... 50 4.2.4 Processing for the subsampled chrominance channels ..... 34 - - 4.3 Coding of the overhead information ................... 35 5 Experimentd results 58 5.1 Introduction and set-up of the coder parameters ............ 58 5.2 -4 coding esample with the image lenna ................ 62 5.3 Compression results for a fised quality factor .............. 65 5.3.1 Objective results ......................... 65 3.3.2 Subjective results ......................... 70 5.3.2.1 Description of the subjective test ........... 70 5.3.2.2 Discussions ....................... 73 5.4 Compression results for varying quality factors ............. 53 5.4.1 Objective results ......................... 83 5-42 Subjective results ......................... 85 5.5 Computational complexity of the coder ................. 88 6 Conclusions 92 6 Future work ................................ 97 A Possible applications for adapt ive JPEG coding 101 Bibliography List of Figures 2.1 The rate-distortion function ....................... 6 2.2 Classification of image compression methods .............. 8 2.3 Loszless and lossy compression general frarnework ........... 3 '2.1 The modulation transfer function .................... 1.5 2.5 Background luminance and the visibilit- t hreshold ........... 16 2.6 Distortion risibilit~. YS background brïghtness ............. 17 7.7 Texture (spatial j masking effects .................... 17 3.1 .i baseline .lPEG compression sysrem ......-.--....-... 20 3-2 The .JPEG luminance quantization marris ............... 2.5 3.3 The .l PEG chrominance quantization marris .............. 2.5 3.4 The QII multipiying factor as a function of the .JPEC. qualit'- factor . 26 3.5 The zig-zag ordering of AC coefficients ................. 27 1.1 The baseline JPEG-compatible encoder ................. 31 1.1 The overhead mode encoder ....................... 3'2 1.3 JPEG compatibility mode rs overhead mode .............. 34 1.1 Linear threshold elevation modeling ................... 36 4.5 The block classification indicative areas ................. 38 4.6 8 x 8 blocks from the image sail .................... 39 4.7 The block classification algorithm .................... 40 4.8 Locally adaptive cclassification correction ................ 12 4.9 la) barbara . (b) block classification ................... 13 1.10 (a) peppers . (b) block classification ................... 13 4.1 1 (a) .si~sie. (11) hlock classification .................... 44 1.12 (a) horises . (b) block classificar ion ................... 14 4.13 The luminance masking sihjectiw eelprrirncnt ............. 47 4.11 The noise visibility thressholds vs . background luminance ....... 48 4.15 Luminance masking linear modeling ................... 52 4.16 -4 16 x 16 area: four Y biocks . one Cr block . one Ch block ...... 54 - - 4-17 Generation of the chrominancr multiplier ................ JJ 5 (a) lenna . (b) harham. (c) bouts. Id) gofdhill. (e) harbor ........ 61 5 .2 (a) bridge . (b) peppers . (cl mandrill. (ci) fIowes. (e) lad!/ ....... 61 5.3 (a) sail . ibj hats. (cj sirrdow.~. (ri! houses. le) pl........... 61 .J .4 la)nl;er.~h)motorhike.(c)lock.................... 61 - - 3.3 The original lenna image ......................... 63 5.6 Block classification of the lenna image ................. 63 - - 3.r The texture masking multiplier map for lenna ............. 61 5.8 The luminance masking multiplier map for lenna ........... 61 3.9 The combined multiplier rnap for lenna ................. 66 5.10 The reconsrructed lennn image . JPEG cornpatibility mode ...... 66 5-11 Bit-rate savings vs . badine JPEG bit-rate (colour images) ...... 68 -- 5-12 The baseline JPEG coded ienna image ................. 1, 5-13 The A-JPEG coded lenna image - JPEG comparibility mode .... 78 5-14