Signal Processing for Communications Winter Semester 2007 - 2008 Instructor: Prof. Suhas Diggavi LICOS, EPFL based on lecture notes by Suhas Diggavi (LICOS), Paolo Prandoni and Martin Vetterli (LCAV), EPFL Contents 0 Mathematical Prerequisites 9 0.1 Complex Numbers ................................. 9 0.1.1 Operations on Complex Numbers ..................... 9 0.1.2 The Complex Number Field ........................ 10 0.1.3 The Complex Plane ............................ 10 0.1.4 Absolute Value, Conjugation and Distance ................ 12 0.2 Summations .................................... 13 0.3 Integration ..................................... 14 0.4 Linear Algebra ................................... 14 0.4.1 Matrices .................................. 14 0.4.2 Vectors ................................... 15 0.4.3 Determinants ............................... 16 0.4.4 Eigenvalues and Eigenvectors ....................... 16 0.5 Problems ...................................... 17 1 What Is Signal Processing ? 19 1.1 Introduction .................................... 19 1.2 Elementary Concepts ............................... 20 1.3 Examples of Signals ................................ 21 1.3.1 Speech ................................... 22 1.3.2 Music ................................... 23 1.3.3 Other One-Dimensional Signals ...................... 23 1.3.4 Images ................................... 26 1.3.5 Other Types of Images .......................... 27 1.3.6 Higher-Dimensional Signals ........................ 27 1.4 Systems ...................................... 30 1.4.1 Speech Recognition ............................ 30 1.4.2 Denoising ................................. 31 3 4 CONTENTS 1.4.3 Inverse Problems ............................. 31 1.4.4 Decision Systems ............................. 32 1.4.5 Compression Systems ........................... 33 1.4.6 A Communication Systems Example ................... 34 1.5 World Models ................................... 34 1.6 Analog and Digital Worlds ............................ 36 1.6.1 The Analog World (Continuous Time, Continuous Amplitude) ..... 36 1.6.2 Discrete-Time, Analog Worlds (Discrete Time, Continuous Amplitude) . 37 1.6.3 Digital Worlds (Discrete Time, Discrete Amplitude) .......... 37 1.6.4 Analog versus Digital ........................... 39 1.7 Overview of the Course .............................. 42 1.A Historical Notes .................................. 43 1.B Literature ..................................... 45 2 Discrete-Time Signals 47 2.1 Continuous and Discrete-time Signals ...................... 47 2.2 Informal description of sampling theorem ..................... 48 2.3 Discrete-time sequences .............................. 51 2.3.1 Basic Signals ............................... 52 2.3.2 Digital Frequency ............................. 54 2.3.3 Elementary Operators ........................... 55 2.3.4 The Reproducing Formula ......................... 56 2.3.5 Energy and power ............................. 57 2.4 Classes of Discrete-Time Signals ......................... 57 2.4.1 Finite-Length Signals ........................... 58 2.4.2 Infinite, Aperiodic Signals ......................... 59 2.4.3 Periodic Signals and Periodic Extensions ................. 59 2.4.4 Finite-Support Signals ........................... 61 2.5 Summary ...................................... 61 3 Representation of Discrete-Time Sequences (DFS, DFT) 65 3.1 Preliminaries .................................... 66 3.1.1 Terminology ................................ 66 3.1.2 Complex Oscillations? Negative Frequencies? .............. 67 3.1.3 Complex Exponentials ........................... 69 3.2 Representation of Periodic Sequences: The Discrete-Time Fourier Series (DFS) 69 3.2.1 Interpretation of the Fourier series .................... 74 3.3 The Discrete Fourier Transform (DFT) ...................... 77 3.4 Properties of the DFS ............................... 80 CONTENTS 5 3.5 Properties of the DFT ............................... 81 3.6 Summary ...................................... 82 3.7 Problems ...................................... 83 4 Signals and Hilbert Space 87 4.1 A Quick Review of Euclidean Geometry ..................... 88 4.2 From Vector Spaces to Hilbert Spaces ...................... 92 4.2.1 The Recipe for Hilbert Space ....................... 92 4.2.2 Examples of Hilbert Spaces ........................ 96 4.2.3 Inner Products and Distances ....................... 97 4.3 Subspaces, Bases, and Projections ........................ 98 4.3.1 Definitions ................................. 99 4.3.2 Properties of Orthonormal Bases ..................... 100 4.3.3 Examples of Bases ............................. 105 4.4 Signal Spaces Revisited .............................. 106 4.4.1 Finite-Length Signals ........................... 106 4.4.2 Periodic Signals .............................. 107 4.4.3 Inifinite Sequences ............................. 107 4.5 Summary ...................................... 109 4.6 Problems ...................................... 109 5 The DTFT (Discrete-Time Fourier Transform) 111 5.1 The DTFT as the Limit of a DFS ........................ 113 5.2 The DTFT as a Formal Change of Basis ..................... 115 5.3 Relationships Between Transforms ........................ 119 5.4 Properties of the DTFT .............................. 121 5.5 Summary ...................................... 125 5.6 Problems ...................................... 126 6 Fourier Analysis - Practice 129 6.1 The Transforms in Practice ............................ 129 6.1.1 Plotting Spectral Data .......................... 130 6.1.2 Computing the Transform: the FFT ................... 131 6.1.3 Cosmetics: Zero Padding ......................... 131 6.2 Spectral Analysis ................................. 133 6.3 Time-Frequency Analysis ............................. 135 6.3.1 The Spectrogram ............................. 135 6.3.2 The Uncertainty Principle ......................... 137 6.4 Digital Frequency vs. Real Frequency ....................... 137 6 CONTENTS 6.5 Problems ...................................... 140 7 Linear Systems 141 7.1 Definition and Properties ............................. 141 7.1.1 Properties of the convolution ....................... 143 7.1.2 The meaning of the convolution ..................... 144 7.1.3 Convolution of frequency spectrum .................... 144 7.2 Circular convolution ................................ 145 7.3 Stability ...................................... 145 7.3.1 Causality .................................. 147 7.4 Introduction to Filtering .............................. 151 7.4.1 FIR filtering ................................ 151 7.4.2 IIR filtering ................................ 152 7.5 Filtering in the Frequency Domain ........................ 155 7.5.1 Preliminaries ................................ 155 7.5.2 The Convolution and Modulation theorems ............... 156 7.6 The Frequency Response ............................. 157 7.6.1 Magnitude ................................. 158 7.6.2 Phase ................................... 158 7.7 Examples of Filters ................................ 160 7.7.1 Ideal Filters ................................ 160 7.7.2 Examples Revisited ............................ 164 7.8 Filtering and Signal Classes ............................ 166 7.8.1 Filtering of Finite-Length Signals ..................... 166 7.8.2 Filtering of Periodic Sequences ...................... 167 7.9 Summary ...................................... 169 7.10 Problems ...................................... 170 8 The Z-Transform 177 8.1 Region of convergence for the Z-transform .................... 179 8.2 The inverse Z-transform .............................. 181 8.3 Partial fraction expansion ............................. 181 8.4 Z-transform properties ............................... 189 8.5 Analysis and characterization of LTI systems using Z-transform ......... 192 8.5.1 Causality .................................. 192 8.5.2 Stability .................................. 192 8.5.3 LTI systems and linear constant-coefficient difference equations ..... 193 8.6 Problems ...................................... 194 CONTENTS 7 9 Filters and Filter Design 199 9.1 Realizable Filters: General Properties ....................... 199 9.1.1 Difference Equations & Initial Conditions ................ 200 9.1.2 Transfer Functions ............................. 202 9.1.3 Stability Analysis ............................. 203 9.2 Filter Design - Introduction ............................ 203 9.2.1 FIR versus IIR ............................... 204 9.2.2 Filter Specifications & Tradeoffs ..................... 205 9.3 FIR Filter Design ................................. 207 9.3.1 FIR Filter Design by Windowing ..................... 207 9.3.2 Minimax FIR Filter Design ........................ 214 9.4 IIR Filter Design .................................. 233 9.4.1 All-Time Classics ............................. 233 9.4.2 IIR Design by Bilinear Transformation .................. 235 9.5 Filter Structures .................................. 241 9.5.1 FIR Filter Structures ........................... 241 9.5.2 IIR filters structures ............................ 242 9.5.3 Some Remarks on Numerical Stability .................. 245 9.6 Problems ...................................... 246 10 Interpolation and Sampling 249 10.1 Preliminaries and Notation ............................ 250 10.2 Continuous-Time signals ............................. 251 10.3 Bandlimited Signals ...............................