REAL-TIME IMPLEMENTATION OF AN OBJECT-BASED CODEC by Fergal Connor B.Eng. A thesis submitted in partial fulfilment of the requirements for the degree of Masters in Electronic Engineering Supervisor: Dr. Thomas Curran School of Electronic Engineering Dublin City University September 1997 I hereby certify that this material, which I now submit for assessment on the programme of study leading to the award of Masters in Electronic Engineering is entirely my own work and has not been taken from the work of others save and to the extent that such work has been cited and acknowledged within the text of my own work. TABLE OF CONTENTS TABLE OF CONTENTS...............................................................................................iii LIST OF FIGURES.............................................................................................................v LIST OF TABLES..............................................................................................................vi ABSTRACT....................................................................................................................... viii ACKNOWLEDGMENTS..............................................................................................ix Chapter 1 INTRODUCTION.........................................................................................1 1.1 Background.........................................................................................................1 1.2 Objectives of Research....................................................................................3 1.3 Structure of Thesis............................................................................................4 Chapter 2 VERY LOW BITRATE VIDEO CODING............................................ 5 2.1 Introduction...................................................................................................... 5 2.2 Object-Based Analysis-Synthesis Coding............................. 8 2.3 The DCU Object-Based Codec...................................................................12 2.4 Profiling of the DCU Object-Based Analysis-Synthesis Coder 21 2.4.1 Execution Time Estimation...............................................................21 2.4.2 Other Properties................................. 29 Chapter 3 REAL TIME IMPLEMENTATION.......................................................32 3.1 Introduction....................................................................................................32 3.2 Detection of Parallelism...............................................................................34 3.2.1 Inter-Function Parallelism.................................................................35 3.2.2 Function-Level Parallelism................................................................39 3.3 Proposed Parallel Algorithm......................... 41 3.4 Hardware Structure 48 Chapter 4 SOFTWARE DEVELOPMENT..............................................................51 4.1 Introduction.................................................................................................... 51 4.2 The TMS320C30............................................................................................52 4.2.1 Main Architectural Features..............................................................53 4.2.2 Programming the TMS320C30........................................................ 55 4.2.3 Pipeline Operation.............................................................................. 60 4.3 Software Implementation............................................................................. 65 4.3.1 Change Detection...............................................................................67 4.3.2 Motion Analysis...................................................................................75 4.3.3 Shape Approximation............................................. 93 4.3.4 Motion Synthesis...................................................... 95 4.3.5 Model Failure Detection.....................................................................97 4.3.6 The Decoder................................................. 99 Chapter 5 PERFORMANCE ANALYSIS............................................................... 100 5.1 Introduction..................................................................................................100 5.2 Execution Time Requirements.............................................. 101 5.2.1 Sequential Execution.........................................................................101 5.2.2 Parallel Execution...............................................................................110 5.3 Discussion.......................................................................................................115 Chapter 6 CONCLUSION........................................................................................... 119 Appendix A ........................................................................................................................A-l Appendix B ........................................................................................................................ B-l REFERENCES IV LIST OF FIGURES Number Page Figure 2.1 A typical videophone scene.....................................................................6 Figure 2.2 Block diagram of an object-based analysis-synthesis coder............. 9 Figure 2.3 Output of various stages of SIMOC coding scheme.......................13 Figure 2.4 An example of coarse polygon approximation................................ 16 Figure 2.5 Bilinear interpolation scheme.............................................................. 26 Figure 3.1 Amdahl Fraction graphs......................................................... 42 Figure 3.2 Motion estimation pipeline configuration........................................ 44 Figure 3.3 Four-processor geometric implementation.......................................45 Figure 3.4 Basic structure of multiprocessor system...........................................49 Figure 4.1 Block Diagram of TMS320C30 Architecture...................................53 Figure 4.2 Operation of TMS320C30 pipeline....................................................61 Figure 4.3 Contour direction codes........................................................................74 Figure 4.4 Mean-value filter structure.................................................................... 76 Figure 4.5 The two possible measurement window cases.................................79 Figure 4.6 Pel-subsampling pattern......................................................................... 85 Figure 4.7 Interpolation block showing location of grid positions................89 Figure 5.1 Amdahl Fraction value graphs........................................................... 115 LIST OF TABLES Table 2.1 Summary of three-level hierarchical block-matching algorithm...15 Table 2.2 Simulation results for DCU coder - average bitrates........................20 Table 2.3 Simulation results for DCU coder - average PSNR (Peak SNR). .21 Table 2.4 Summary of processing requirements for change detection 24 Table 2.5 Summary of processing requirements for motion analysis 27 Table 2.6 Summary of processing requirements for model failure detection28 Table 3.1 List of functions with input and output requirements.....................36 Table 3.2 Flow of grid positions through pipeline.............................................. 44 Table 4.1 Cycle count for implementation of methods A & B........................83 Table 4.2 Summary of sub-optimal motion estimation simulation results ...86 Table 5.1 Computation costs for fixed steps of change detection.................102 Table 5.2 Cycle counts for non-fixed steps of change detection.....................102 Table 5.3 Computation costs for different motion estimation methods. ...104 Table 5.4 Computation costs for some motion analysis functions................104 Table 5.5 Summary of computation costs for shape approximation 106 Table 5.6 Summary of computation costs for motion synthesis....................106 Table 5.7 Cycle counts for non-fixed steps of model failure detection.........107 Table 5.8 Summary of cycle counts for the encoding of a single frame......109 Table 5.9 Cycle counts for sub-optimal motion estimation methods...........109 Table 5.10 Summary of cycle counts for the decoding of a single frame. ...109 Vll Abstract REAL-TIME IMPLEMENTATION OF AN OBJECT-BASED CODEC Fergal Connor Modern video coding algorithms are becoming increasingly complex with the result that single general purpose processors are incapable of meeting the computational power required for real time implementation. The coding algorithms are continuously evolving therefore, any multiprocessor solution must not only possess the necessary computational power but must also be flexible enough to adapt to any modifications in the algorithms. This report presents a possible multiprocessor solution with specific reference to the DCU object-based analysis-synthesis coder. Firstly, an abstract model of the multiprocessor system is defined. The model is based on the dual requirements of computational power and flexibility. An analysis of the DCU coding algorithm is performed in order to refine the basic model by identifying potential realisation options that optimise coder performance. A reciprocal relationship exists whereby hardware constraints require modification of the algorithm. Any modifications are outlined and their effect on overall