IMPLEMENTATION OF AN MPEG-1 VIDEO ENCODER AND DECODER By John Bateman November 1995 Submitted as partial fulfilment of the degree of Master of Engineering Science DECLARATION I hereby declare that this submission is my own work and that, to the best of my knowledge and belief, it contains no material previously published or written by another person nor material which to a substantial extent has been accepted for the award of any other degree or diploma of a university or other institute of higher learning, except where due acknowledgement is made in the text of the thesis. J. A. BATEMAN November 1995 ACKNOWLEDGEMENTS I would like to thank my supervisor, Dr John Arnold, for his assistance and guidance throughout this project. I would also like to thank Dr Michael Frater for his advice on variable length coding techniques and other issues, and Mr Mark Pickering for providing advice on a range of video coding issues. ABSTRACT The demand for multimedia applications which include digital video (sequences of images in time) as well as still images, sound and text is increasing rapidly. Processing digital video is problematic due to the high bit rates involved. For example, a standard television picture requires a bit rate of approximately 165 Mbps. To overcome this problem an international standard known as MPEG-1 has been developed to define a coded video bitstream at compressed bit rates of around 1.5 Mbps. The aim of this project was to develop a software program to encode and decode video in accordance with this standard for use in future standards research and teaching. Due to the complexity of the task, a structured approach was followed for the software design and development process. The program was implemented using the MATLAB progranmiing environment to make use of its powerful mathematical and image processing features, and a faster development time than traditional languages such as C and Pascal. The performance of the encoder and decoder was then analysed in terms of picture quality and processing speed. Picture quality was found to be good at MPEG-1 bit rates. Encoding and decoding is a computationally intensive and slow process. Some areas of the encoder and decoder were targeted for possible future optimisation. TABLE OF CONTENTS Page No CHAPTER 1 INTRODUCTION 1.1 MPEG-1 Origins 1 1.2 Related Standards 2 1.3 Project Aim 2 1.4 Thesis Overview 3 CHAPTER 2 MPEG-1 STANDARD 2.1 Structure of the Standard Document 4 2.2 Applications 5 2.3 Features 6 CHAPTER 3 MPEG-1 COMPRESSION ALGORITHM 3.1 Overview 7 3.2 Constrained Parameters 8 3.3 Source Video 9 3.4 Picture Types 10 3.5 Motion Compensation 11 3.6 Discrete Cosine Transform (DCT) 12 3.7 Quantisation 13 3.8 Variable Length Coding (VLC) 14 3.9 Layered Syntax 15 CHAPTER 4 SOFTWARE DEVELOPMENT METHODOLOGY 4.1 Classical Life Cycle 17 4.2 Structured Analysis and Design 18 4.3 Structured Analysis 18 4.4 Acceptance Test Generation 21 4.5 Structured Design 22 4.6 Coding and Testing 24 4.7 Program Documentation 24 4.8 Quality Assurance 24 4.9 Results 25 Page No CHAPTERS MPEG-1 ENCODER 5.1 Coded Bitstream Structure 26 5.1.1 Sequence Layer 26 5.1.2 Group of Pictures Layer 26 5.1.3 Picture / Slice / Macroblock / Block Layers 28 5.2 Encoding Process 29 5.2.1 Source Video 30 5.2.2 Motion Estimation 30 5.2.3 Motion Compensation 3 2 5.2.4 Discrete Cosine Transform 32 5.2.5 Quantisation 33 5.2.6 VLC Coding 34 5.2.7 Encoder Output 34 5.2.8 Reconstructed Pictures 34 5.2.9 Rate Control 35 5.2.10 Error Reporting 35 CHAPTER 6 MPEG-1 DECODER AND VIDEO PLAYBACK 6.1 Decoding Process 36 6.1.1 Error Reporting 3 8 6.2 Video Playback 38 CHAPTER? RESULTS 7.1 Coded Bitstream Bit Rate 39 7.2 Picture Quality at Various Bit Rates 40 7.3 Numerical Accuracy 42 7.4 Processing Speed 42 7.4.1 Efficient MATLAB Code 43 7.4.2 Efficient Design 44 7.4.3 Encoder and Decoder Execution Profile 44 CHAPTER 8 CONCLUSION 8.1 Summary of Findings 47 8.2 Suggestions for Future Work 48 BIBLIOGRAPHY APPENDIX A - SAMPLE ENCODER/DECODER OUTPUT APPENDIX B - CODER STRUCTURE CHART APPENDIX C - CODE I-PICTURE STRUCTURE CHART APPENDIX D - CODE P-PICTURE STRUCTURE CHART APPENDIX E - CODE B-PICTURE STRUCTURE CHART APPENDIX F - MOTION ESTIMATION STRUCTURE CHART APPENDIX G - DECODER STRUCTURE CHART APPENDIX H - DECODE I-PICTURE STRUCTURE CHART APPENDIX I - DECODE P-PICTURE STRUCTURE CHART APPENDIX J - DECODE-B PICTURE STRUCTURE CHART APPENDIX K - LIST OF MATLAB FUNCTIONS APPENDIX L - MATLAB CODE LISTING CHAPTER 1 INTRODUCTION l.lMPEG-1 ORIGINS MPEG (Moving Picture Expert Group) is a group formed by the International Standards Organisation (ISO) to create standards for digital video and audio. The official title for the group is: ISO/IEC JTCl SC29 WGll. ISO: International Standards Organisation lEC: International Electrotechnical Commission JTCl: Joint Technical Committee 1 SC29: Sub-committee 29 WGl 1: Working Group 11 (moving pictures with audio) MPEG meets several times a year and receives input from both academic researchers and industry. One of the products of MPEG is the MPEG-1 standard for video and audio compression. The full title of the standard is: "Information Technology - Coding of moving pictures and associated audio - For digital storage media at up to about 1.5 Mbits/s IS 11172" Work on MPEG-1 began in 1988 in response to an increasing need for a common format for representing compressed video on digital storage media such as CD-ROM, Digital Audio Tape (DAT), computer hard disks, writable optical disks as well as LANs and telecommunications links [1]. By making the format an international standard, advantages such as compatibility between different vendor equipment, reduced cost, and encouraging consumer confidence that products based on the standard have a reasonable lifetime, are realised [4]. The target bit rate is 1.5 Mbps because CD-ROM, DAT and other digital storage media transfer data at around this rate. Video is allocated approximately 1.15 Mbps of this bandwidth, with the remainder going to audio and the synchronisation and timing functions between the video and audio bit streams [1]. Expressing the bit rate as "about 1.5 Mbps" is deliberate as the overall bit rate can vary from one application to another. 1.2 RELATED STANDARDS MPEG-1 is primarily intended for displaying progressive video (as opposed to interiaced TV pictures) with formats up to 288 lines of 352 pels and picture rates of around 25 to 30 Hz. (A pel is an 8-bit sample of luminance or chrominance data and is a synonym for pixel. A picture is an image consisting of luminance and chrominance data, and is similar to a TV frame). The fact that MPEG-1 is optimised for these parameters differentiates it from other standards such as H.261, JPEG and MPEG-2. JPEG uses similar compression techniques, but is intended for still images only. H.261 is optimised for visual telephony at low-bit rates (multiples of 64 kbps) where motion is more Hmited. MPEG-1 shares many of the features of H.261, such as the entropy coding tables. MPEG-2 is a newer standard intended for higher quaUty video at higher bit rates than MPEG-1 to support services such as entertainment television and HDTV (high definition television). MPEG-2 is much more than simply a scaled up version of MPEG-1. It supports field based coding (to handle interlace) and adds more powerful techniques for processing video bit streams in terms of profiles and levels at a cost of much greater complexity. 1.3 PROJECT AIM The aim of this project was to develop an MPEG-1 encoder which produced a coded bitstream which complies with the standard at bit rates of approximately 1.15 Mbps, and to develop an MPEG-1 decoder which complies with the standard. The encoder and decoder have been developed in MATLAB - a fourth generation computer language with powerful toolbox functions for image processing, graphics and other applications. MATLAB was selected as the development environment for its powerful features and ease of programming. Computer files stored in a format known as MPEG-1 Source Input Format (SIF - see Chapter 3) were used as source video for the encoder. The encoder output bitstream was stored as a binary file. The decoder used a binary file bitstream as input, and stored output video in MPEG-1 SIF format to file. The performance of the encoder and decoder was assessed in terms of the picture quality at the target bit rate (1.15 Mbps) and the processing speed of the encoder and decoder. There is not a great deal of literature on MPEG-1 due to the fact that it is relatively new technology. Besides professional journal articles and the standard itself, a significant amount of information is available over the Internet. Internet sites such as www.crs4.it provide excellent descriptions of the algorithm, whilst other sites such as www.cs.tu- berlin.de/'-phade provide general information such as details on MPEG-1 chips such as the MCD250 by Motorola, and on software products such as the set of tools available from the company North Valley Research. The significance of this project in the context of these other products is that the MATLAB code created is very easy to modify and adapt to different appHcations. This is important given the intended application of project as a research tool for future standards development, and as a teaching aid for video coding.