Design and Implementation of a Real-Time Video and Graphics Scaler by Erika Shu-Ching Chuang Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Master of Engineering in Electrical Engineering and Computer Science at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY May 20, 1997 © Massachusetts Institute of Technology, 1997. All Rights Reserved. Auth or ........................... ........................................... Department of Electricaýgineering and Computer Science // May 20, 1997 Certified by .............................................................. .. .................... .... V. Michael Bove Associate Profes~ofe dia Technology, MIT Media Laboratory Thesis Supervir Accepted by .......... ( A-i*hur* C..Smith Chairman, Department Committee on Graduate Theses 0T: 2 9e 1997 Design and Implementation of a Real-Time Video and Graphics Scaler by Erika Shu-Ching Chuang Submitted to the Department of Electrical Engineering and Com- puter Science on May 19, 1997, in partial fulfillment of the require- ments for the degree of Master of Engineering in Electrical Engineering and Computer Science Abstract Various methods for scaling video and graphics images were studied in depth. The evalua- tion and selection of the optimal algorithms involved comparing the quality of scaled images, analyzing the hardware constraints imposed by the video and graphics architec- ture, and estimating the complexity and cost of the implementations. A real-time hardware scaler was designed using the chosen algorithms: Linear Interpolation and Area Weighted Averaging. The behavioral and register transfer level description of the hardware design was created and simulated with test benches. Thesis Supervisor: V. Michael Bove Title: Associate Professor of Media Technology, MIT Media Laboratory Thesis Supervisor: Mark Kautzman Title: Staff Engineer, IBM Microelectronics Thesis Supervisor: Mark Merrill Title: Manager, IBM Microelectronics Acknowledgments I would like to thank all the members in the Display Subsystems Solutions group at IBM Micro- electronics from whom I have received assistance. I would like to thank Mark Kautzman for intro- ducing me the good graphics and image processing books and many other useful resources. I am grateful to the team of engineers, Tom Wilson, John Pike, Hiro Ando, Pete Casavant, Paul Schanely, and Bob Horton, who gave me a lot of good ideas in hardware design and answered my endless questions. I would like to thank Mike West for his awesome computer graphics notes. I would like to thank my manager Mark Merrill for helping me find my thesis topic and putting up with my strange working schedule. I am grateful to Professor V. Michael Bove for giving me useful pointers in my research. I would like to thank Ya-Chieh Lai, Theodore Tonchev, and Andrew Chen for helping me with my thesis writing. Foremost, I would like to thank my family for their love and support. Without them I would not have reach where I am today. Table of Contents 1 Introduction ............................................................................................................... 10 1.1 O utline of the T hesis ........................................................................................ 11 2 Background ...................................................................................... 12 2.1 DSP Basic and Sampling Rate Conversion .................................................. 12 2.2 Methods for Image Scaling .......................................... ................ 14 2.3 Comparison of the Software Methods ....................................... ..... 20 3 Design Constraints .............................................................................................. 26 3.1 Video/Graphics System Hardware Overview ................................................ 26 3.2 Real-time Scaler......................................................................................... 27 4 Implementation of Scaling Algorithm ................................................................ 34 4.1 Issues in Selecting Scaling Methods ............................................. 34 4.2 Algorithm Detail .......................................... 37 4.3 Correction for Precision Error ......................................................................... 42 5 Hardware Implementation .......................................................... ............................ 46 5.1 Design Methodology...................................................................................46 5.2 Video/Graphics Scaler................................................................................... 47 5 .3 T esting ................................................................. ........................................... 59 6 C onclusion ........................................................... .. .............................................. 62 Appendix A C code for Scaling Algorithms ........... ............................. 64 Appendix B Verilog Code for Video and Graphics Scaler .......................................... 82 B .1 M odules.............................................................. ............................................ 82 B .2 T est Drivers...................................................................................................... 94 Appendix C Simulation Results ................................................................................... 100 C. 1 Hmodule (Horizontal Scaler) ...................................................................... 100 C.2 Vmodule (Vertical Scaler) .......................................................................... 100 C.3 Scaler (Full Scaler) ........................ ................ 100 B ibliography .......................................................... ............................................. 112 List of Figures Figure 2.1: Frequency spectrum of bandlimited analog signal x(t) ............................ 12 Figure 2.2: Spectrum of sampled x(t) at ws, where os = 27t/T ....................................... 12 Figure 2.3: Interpolation by a factor of 2. (a) a section of original samples (b) sampling rate expansion (c) interpolation ........................................................................................ 13 Figure 2.4: Interpolation by a factor of 2- frequency domain representation. The figures correspond to the same steps in figure 2.3. (a) original spectrum (b) sampling rate expan- sion(c) interpolation ........................................................................................ ........... 13 Figure 2.5: Downsampling by a factor of 2. (a) a section of original samples (b) decima- tion ........ ............................................................................................... ..... ............. 14 Figure 2.6: Downsampling by a factor of 2 - frequency domain representation (a) original spectrum (b) decimation with high frequency aliasing ................................................... 14 Figure 2.7: Block diagram for anti-aliased resampling ([1],[2]) ................................. 15 Figure 2.8: Block diagram for simplified anti-aliased resampling ([1],[2]) ................ 15 Figure 2.9: Common functions for interpolators [3]: (a) bilinear (b) biquadratic (c) bicubic (d) Lanczos interpolator ............................................................................................... 19 Figure 2.10: Area Weighted Averaging (4:3 reduction) .................................... ... 20 Figure 2.11: Image enlargement: (a) Bresenham Algorithm (b) Bilinear Interpolation.21 Figure 2.12: Original Image of man in striped shirt .................................................... 23 Figure 2.13: Image of man in striped reduced by 20% with (a) Bresenham algorithm (b) Area Weighted Averaging ....................................................................................... 24 Figure 3.1: Example Architecture of a one-port Graphics System [8]. ....................... 27 Figure 3.2: Example Architecture of a two-port Graphics System [8] ........................ 27 Figure 3.3: Illustration of the data flow requirement ..................................... ... 30 Figure 3.4: A FIR filter implemented using tapped delay line with multipliers and adders. 31 Figure 4.1: Area Weighted Averaging........................................................................ 42 Figure 5.1: Design Methodology ............................................................................. 46 Figure 5.2: Abstract representation of video/graphics scaler............................. .. 48 Figure 5.3: Input U nit .............................................................................................. 49 Figure 5.4: YUV Data format ............................................................................... 50 Figure 5.5: Vertical Scaling Unit (control module is not shown in the figure for simplici- ty ) .................................................................................................................................... 5 1 Figure 5.6: Vertical Unit state diagram........................................................................52 Figure 5.7: Vertical Arithmetic Unit........................................................................ 55 Figure 5.8: Horizontal Unit state diagram ................................................................... 56 Figure 5.9: Horizontal Arithmetic Unit ..................................................................