Review of Analog Signals and Sampling • Discrete-Time

Digital Signal Processing Disclaimer Fundamentals with Hands-on Experiments These course notes cover the fundamentals and select applications by of Digital Signal Processing and are intended solely for education. No other use is intended or authorized. No warranty or implied Andreas Spanias, Ph.D. warranty is given that any of the material is fit for a particular purpose, application, or product. Although the author believes that June 24, 2009 the concepts, algorithms, software, and data presented are accurate, School of Electrical, Computer and Energy Engineering he provides no guarantee or implied guarantee that they are free of Arizona State University Tempe, AZ 85287-5706 error. The material presented should not be used without extensive 480 965 1837 verification. If you do not wish to be bound by the above then 480 965 8325 please do not use these notes.

DSP Primer – June 24, 2009 Sponsored in part by NSF 0817596

Contents • Introduction to DSP - Review of analog Digital Signal Processing (DSP) Introduction signals and sampling • Digital Signal Processing (DSP) is a branch of signal processing • Discrete-time systems and digital filters that emerged from the rapid development of VLSI technology • The z transform in DSP that made feasible real- time digital computation. • Design of FIR digital filters • DSP involves time and amplitude quantization of signals and • Design of IIR digital filters relies on the theory of discrete- time signals and systems. • The discrete and the fast Fourier • DSP emerged as a field in the 1960s.

transform • Early applications of off- line DSP include seismic data analysis, • FFT info and applications voice processing research.

Advantages of digital over analog signal processing: • Nyquist Theorem 1920's.

• flexibility via programmable DSP operations, • Statistical Time Series, PCM 1940's. • storage of signals without loss of fidelity, •off- line processing, • lower sensitivity to hardware tolerances, • Digital Filtering, FFT, Speech Analysis mid 1960s (MIT, Bell • rich media data processing capabilities, Labs, IBM). • opportunities for encryption in communications, • Multimode functionality and opportunities for software radio. • Adaptive Filters, Linear Prediction (Stanford, Bell Labs, Japan 1960s). -Disadvantages : • Digital Spectral Estimation, Speech Coding (1970s). • Large bandwidth and CPU demands

DSP Historical Perspective (2) DSP Applications

• Military Applications (target tracking, radar, sonar, secure communications, sensors, imagery) • First Generation DSP Chips (Intel microcontroler, TI, AT&T, Motorola, Analog Devices (early 1980s) • Telecommunications (cellular, channel equalization, vocoders, software radioetc) • Low-cost DSPs (late 1980s) • PC and Multimedia Applications (audio/video on demand, streaming data applications, voice synthesis/recognition) • Vocoder Standards for civilian applications (late 1980s) • Entertainment (digital audio/video compression, MPEG, CD, MD, • Migration of DSP technologies in general purpose CPU/Controllers DVD, MP3) "native" DSP (1990s) • Automotive (Active noise cancellation, hands-free communications, • High Complexity Rich Media Applications navigation-GPS, IVHS) • Manufacturing, instrumentation, biomedical, oil exploration, robotics • Low Power (Portable) Applications • Remote sensing, security

Nowdays LPF and A/D integrated DSP chip • DTMF (use of the FFT and digital oscillators) Digital • Adaptive echo cancellation (Hands-free telephony, Speakerphones) x(t) x`(t)sample x(nT) LPF Signal & A/D • Speech coding (speech coding in cellular phones) Processor f • Modem (data/computer connectivity) s y(nT) Reconstruction Antialiasing • Software radio (multi-mode/multi standard wireless communications) y(t) y`(t) • Channel estimation (equalization) LPF D/A

• Antenna beamforming (space division multiple access - SDMA) Nowdays LPF and D/A integrated

• CDMA (modulating with random sequences) Remarks: The diagram shows the sampling, processing, and reconstruction of an analog signal. There are applications where processing stops at the digital signal processor, e.g., speech recognition.

Symbols and Notation Continuous vs Discrete-time

Continuous-time (analog) Signal Discrete-time (digital) signal xa (t) |tnT  x(nT)  x(n) ;discrete  time input x(t) x(n) y(n) ;discrete  time output H (.) ;transfer and frequency responsefunctions

h(.) ;impulseresponse(systemfunction) 0T2T ... t n n ;discrete  timeindex x(t)Q x(n) Remarks: In general and unless otherwise stated lower case symbols will be used for time-domain signals and upper case symbols will be used for Remarks: A continuous-time signal is converted to discrete-time using sampling and transform domain signals. Bold face or underlined face symbols will be quantization. As a result aliasing and quantization noise is introduced. This noise Be generally used for vectors or matrices. can be controlled by properly designing the quantizer and anti-aliasing filter.

quantization noise Performance in terms of Signal to Noise Ratio (SNR) eq(t)

SNRPCM  6.02Rb  K1 sampling period xa(t) where Rb is the number of bits and the value of K1 T analog waveform depends on signal statistics. For telephone speech K1 = -10

Oversampling / or / Conversion Time vs Frequency Domain

tim e-dom ain frequency-domain • Integrated oversampling and 1-bit quantization x(t) |X(f)|

...

...... • Very compact and inexpensive circuitry (some low power applications as well) 00t f

x(t) |X(f)| • Lowers analog circuit complexity with a modest increase in software (DSP MIPS) complexity ...

......

00tf • Uses concepts from multirate signal processing and Delta Modulation Remarks: Slowly time-varying signals tend to have low-frequency content • Will be described in the context of multirate signal processing while signals with abrupt changes in their amplitudes have high frequency content. The frequency content of signals can be estimated using Fourier techniques.

1.0 50 Time domain speech segment fundamental TAPE TIME: 8014 frequency

Formant Structure

20 0.0 Discrete-time Impulse 1 Amplitude Magnitude (dB) 0

-1.0 -20 0 8 16 24 32 01234  (n)  .. .. Time (mS) Frequency (KHz) Periodic waveform gives harmonic spectra 0 n 1.0 40 Time domain speech segment TAPE TIME: 3840

0.0 0 Amplitude Magnitude (dB) Magnitude Think of signals as a weighted sum of impulses.

Some Important Signals (3) Some Important Signals (4) The sinusoid Period T The sinc function sidelobes mainlobe

2 sin(t) sin( t)  sin( t)  sinc(t)   ...... T { } t { 0 }

2 π 2π   2 f  units: ω(rad/s) f (Hz) T(s) T Sinc functions often appear in signal and filter analysis Sinusoids are used in analyzing or synthesizing acoustic and other signals particularly when considering frequency domain behavior

2009 Copyright 2009 ©Andreas Spanias I-19 2009 Copyright 2009 ©Andreas Spanias I-20 Some Important Signals (5) Representing Periodic Signals with Sinusoids Random noise Fourier series: Trigonometric form:   x(t)  a 0   a k cos( k o t )   bk sin( k o t ) k 1 k 1 Fourier series: Complex (magnitude/phase) form:  jk t Preferred in engineering-- >> o x(t)   X k e k 

Xk are complex F.S. coefficients and provide spectral magnitude and phase info Encountered in communication systems and other application Characterized by their mean and variance jkot and e  cos(kot)  jsin(kot)

Fourier Series Example (2) Fourier Series Analysis Example Harmonic Spectrum Representing a Periodic Pulse Train as a Sum of Harmonic Sinusoids

x(t) X k d d 1/ 4 2o T harmonics ...... 3o 0 T t

d / 2 1 d  k  d  ...... X  1e  jk  o t dt  sinc  o  k  0 T  d / 2 T  2   Remarks: A periodic pulse signal has a discrete F.S. spectrum described by samples that fall on a sinc (sinc(x)=sin(x)/x) function. As the period increases the F.S. components become more dense in frequency and weaker Fundamental frequency in amplitude. If T goes to infinity periodicity is lost and the F.S. vanishes. o 2009 Copyright 2009 ©Andreas Spanias I-23 2009 Copyright 2009 ©Andreas Spanias I-24 Use Sinusoids to synthesize a periodic pulse using the Fourier Series Example (3) Fourier series (only one period shown)

d/T=1/5

10 sinusoids

0 1 sinusoid

4π 50 sinusoids 2 sinusoids d/T=1/10

3 sinusoids 0 100 sinusoids

Fourier transform of a time-limited pulse The Continuous Fourier Transform (CFT) Equations (Represent a single pulse by sinusoids) Analysis Expression Given the signal The Fourier transform xt  X ( )  x (t )e  j t dt  d   ...... The inverse Fourier transform Synthesis Expression t  0 1 jt x()tXed ( )  d / 2  d 2   j t    X ( )   e dt  d sinc    d / 2  2  A Fourier transform pair is designated by: x (t )  X ( )

Remarks: Note that a time-limited signal has a non-bandlimited CFT spectrum. Remarks: Both time and frequency are continuous variables. The CFT can The sinc function has zero crossings at integer multiples of 2π/d. As the pulse handle non-periodic signals as long as they are integrable. Periodic signals can width increases the sinc function “shrinks”. In the limit, if T goes to infinity be handled using the impulse and CFT properties. (i.e., pulse becomes D.C. signal) the sinc function collapses to a unit impulse.

  d  X ( )  d sinc   if x (t )  X ( ) then X (t )  2x ( )  2 

(time-limited) x(t) (non band-limited) X(ω) ...... 0 ω ...... 0

2π /d 4π/d t  time domain (non time-limited) frequency domain X(t) (band-limited) 2π x(-ω) a transform pair pulse sinc ...... d 0

...... t 0 t 

The Time-Domain Convolution (Filtering) Property Important Fourier Transform Pairs x (t )  X ( ) h (t )  H ( ) cos( t)   ( (  )   (  )) h(t)* x(t)  H ()X ( ) 0 0 0

h (t ) * x (t )  h ( ) x (t   )d  Muliplication in frequency π(ω- ωO) is essentially a filtering operation

convolution in time is multiplication in frequency ω t -ωO 0 ωO DEMO ...... t * t = Example: Convolution of an exponential with a pulse 2009 Copyright 2009 ©Andreas Spanias I-31 2009 Copyright 2009 ©Andreas Spanias I-32 Truncating Signals with Tapered Windows Truncating a Cosine 13dB

S()  ((  )  (  )) st() cos(0 t ) 00 where 2 /T … … 00

1 CFT  .. .. wt() t B  0 T0 t   ω 0 0 0

CFT  jT /2 T 1; 0 tT 0 0  0 WTe()  0 sinc wt()  2 25dB 0; otherwise

 jT 0 /2 stw () stwt () () STe() sinc  T /2+sinc  T /2 w 00000 … …

1 CFT T0

Truncating Speech Truncating Speech (tapered window)

CFT CFT

Normalized frequency x rad/sec Normalized frequency x rad/sec

2009 Copyright 2009 ©Andreas Spanias I-35 2009 Copyright 2009 ©Andreas Spanias I-36 The Sampling Process Example: Audio - Bandwidth A bandlimited signal that has no spectral components at or above B can be uniquely represented by its sampled values spaced at uniform intervals that are not more than π/B seconds apart. 200 - 3200 Hz Basic Telephone Speech  Intelligible T  Preserves Speaker Identity B or a signal that is bandlimited to B must be sampled at a rate of 50 - 7000 Hz Wideband Speech AM-grade audio ωs where B 50 - 15000 Hz Near High Fidelity  s  2B or fs   FM-grade Audio

20 - 20000 Hz High-Fidelity x = CD/DAT Quality Voice analog signal sampling digital signal

Sampling and Periodic Spectra Example: Sampling of Audio Signals x(t) Format Bandwidth Sampling frequency X () Telephony 3.2 kHz 8 kHz

…… Wideband audio 7 kHz 16 kHz ...... High-fidelity, CD 20 kHz 44.1 kHz 0 t  B 0 B  Digital audio tape 20 kHz 48 kHz x (t) (DAT) s X s () Super audio CD 100 kHz 2.8224 MHz (SACD) 1/T DVD audio (DVD-A) 96 kHz 192 kHz ...... T2T3T   B B  2 0 t s 0 s s 

...... T2T3T   B B  2 0 t s 0 s s  ......  B B 2 x(t) X () 0 s s  aliasing

…… ...... the signal can not be recovered perfectly even with an ideal filter only a distorted version of the signal can be recovered 0 t  B 0 B 

Example Oversampling ωs>>2B R C Differential equation: i(t) x(t) y(t) dy() t 11 yt()xt () dt RC RC X s ()

Transfer function: Frequency response function: Guard bands 1 1 ... Hs() H ()  1 jRC  s  B B  s 2  1 sRC 0 s x(t )  sin(  t ) Oversampling relaxes the requirements on antialiasing filters if sinusoid in then sinusoid out It is used in / (/ ) A-to-D converters y ss (t)  H () sin(t  H ())

2009 Copyright 2009 ©Andreas Spanias I-43 2009 Copyright 2009 ©Andreas Spanias I-44 Example - Impulse Response Example - Convolve and obtain an output  t Consider the circuit below with R=1M, C=1x10-6 Consider the RC with impulse response h (t )  e u (t ) R C dh() t 11 and the input x()t  u ()t  u (t  1 ) i(t) .. .. x(t) y(t) ht() ()t t dt RC RC t y (t )   e   d   1 e  t for 0  t  1 t 0 1  t The solution: ht()eeRC for t > 0 t RC y (t )   e   d    e  t  e  ( t  1 ) for t  1 t  1

x(n) y(n) h(n)

The output is produced by convolving the input with the impulse response  Discrete-time Linear Systems y(n)   h(m)x(n  m)  h(n)* x(n) m

Digital Filters This operation can also involve a finite-length impulse response(FIR) sequence L y(n)   h(m)x(n  m) m0 An FIR filter is programmed using a multiply-accumulate instruction

Some Definitions Some More Definitions •A digital filter is linear if it has the property of x(n) x(n-1) x(n) x(n-1) generalized superposition T orz1 ;unit delay •A digital filter is causal if it non anticipatory, i.e., the present output does not depend on future x(n) b x(n) inputs. L ;signal scaling by a bL •All real-time systems are causal. filter coefficient x(n) • Non-causalities arise in image processing where  x(n)+x(n-1) ;signal addition the signal indexes are spatial instead of temporal. x(n-1) • Unless otherwise stated all systems in this . course will be assumed causal.

Jan. 2009 Copyright 2009 ©Andreas Spanias II-3 Jan. 2009 Copyright 2009 ©Andreas Spanias II-4 IIR Digital Filter Structure FIR Digital Filter Structure yn xn xn yn T ...b  ...... b  T L   - T T T T T L     -   . - a ...... 1 . . . .. b1 a2 b .. 1 . a b0 M b0

feedback L y(n)  b x(n  i) L M  i i 0 y(n)   bi x(n  i)   ai y(n  i) i 0 i 1

Two i/p- o/p Equations for Digital Filters Unit Impulse x(n) y(n) h(n) The analysis of digital filters in the frequency domain is facilitated using sinusoids. In the time domain a unique input signal is used One can compute the output using the convolution sum for analysis, namely the unit impulse. That is defined as:   y(n)   h(m)x(n  m)   x(m)h(n  m)  n m m or by using the difference equation L M 0 n y(n)   bi x(n  i)   ai y(n  i) i 0 i 1  n   1 for n0 Remark: The impulse response h(n) can be determined by solving 0 elsewhere the difference equation.

The response of a digital filter to a unit impulse is known as Any discrete-time signal may be represented by a linear combination impulse response and is given by of unit impulses xn

1.5 2 h(n)  b0 (n)  b1 (n 1) ... bL (n  L)  0.5

a1h(n 1)  a2h(n  2) ... aM h(n  M ) 0 1 2 3 4 5 n  0.5  1 is represented by:  2.5 (n) h(n) xn() .521512 ( n  )  . ( n  )   () n h(n) ().().()nn15 2 25  n  3

Impulses as input to source- system LPC Vocoders Finite-Length Impulse Response (FIR)

Vowels are typically synthesized by exciting a filter representing the If the digital filter has no feedback terms the impulse response is mouth and nasal (vocal tract) cavity with a train of periodic impulses finite length

pitch period h(n)  b0 (n)  b1 (n 1) ... bL (n  L) x(n)   (n  i ) Note that h(0)  b0 Remark: The filter has a finite-length i Time-varying impulse response and is called FIR. The h(1)  b1 values of the impulse response sequence digital filter are the coefficients themselves. The filter is always stable. VOCAL SYNTHETIC h(2)  b2 gain TRACT SPEECH .. FILTER ..

h(L)  bL

Jan. 2009 Copyright 2009 ©Andreas Spanias II-11 Jan. 2009 Copyright 2009 ©Andreas Spanias II-12 Example – The Moving Average Filter J-DSP Simulation of Averaging Filter 1 L y(n)   x(n  i) L  1 i 0 1 h(n)  { (n)   (n 1) ...  (n  L)} L 1 1 h(n)  0  n  L L 1

Remark: The moving average is essentially a low-pass (smoothing) filter. Later on we will see that this filter is also optimal in estimation problems. L=10 L=50 Jan. 2009 Copyright 2009 ©Andreas Spanias II-13 Jan. 2009 Copyright 2009 ©Andreas Spanias II-14

Infinite-Length Impulse Response (IIR) IIR – Another First Order Example If the digital filter has feedback terms then the impulse response is infinite hn length  L M  n 0.2  z1 h(n)   bi (n  i)   ai h(n  i)  i  0 i 1 0.8 Example: h(n)   (n)  a1h(n  1)

h(2)  a 2 .. n h(n)  0.2 (n)  0.8h(n  1) h(0) 1 h(1)  a1 1 .. h(n)  (a1) h(n)  0.2 (0.8)n n  0

Remark: Note that if the coefficient a1 has magnitude larger than one the impulse response will go to infinity and hence the filter Remark: This particular IIR filter is also a low-pass filter would be unstable. behaving in similar manner like the the averaging FIR filter.

n h(n)  0.2 (0.8) n  0 He()j He()j

0.2 dB dB 0.18

  0.16 1  z 1 1  z 1

0.14

0.12 00πΩ πΩ

0.1 dB dB 1 1 0.08 1 (1 0.9z ) 1 (1 0.9z )

0.06

0.04

0.02 00πΩ πΩ

0 0 5 10 15 20 25

Impulse Response and Stability An Unstable Filter For the causal digital filter  y(n)   h(m)x(n  m) m0

Bounded Input Bounded Output (BIBO) stability is defined as

  h ( k )   k  0

The condition above is guaranteed if

p i  1 for all i = 1, 2, . . . , M

 1 u(n)  1 n  0 un 0.2  z A special case of interest is the steady-state response to input sinusoids  and is formulated as follows. For the IIR filter

tr ss n y(n)  y (n)  y (n)  0.8(0.8) 1 0.8 L M y(n)  b x(n  i)  a y(n  i) 1  i  i 0.9 i 0 i 1 0.8 The frequency response function is given by: 0.7 0.6 0.5 b  b e j  b e j2 ... b e jL 0.4 H(e j )  0 1 2 L 0.3  j  j2  jM 0.2 1 a1e  a2e ... aM e 0.1 0 Jan. 20090 2 Copyright4 20096 8©Andreas10 12Spanias14 16 18 20 II-21 Jan. 2009 Copyright 2009 ©Andreas Spanias II-22

Steady-State Sinusoidal Response of Example of Steady State Sinusoidal Response

Linear Discrete Systems (Cont.) yn The frequency response function is periodic and an example of the  xn 0.2  z1 0.2 steady state sinusoidal response is given below  H (e j )  1 0.8e j if: 0.8 x(n)  sin(n) The filter is excited by a 500 Hz sinusoid and the then: Sampling rate is 2000Hz. ss j j  n y (n)  H (e ) sin(n  H (e )) x(n)  sin(2n500 / 2000)  sin( ) 2 2f 0.2  n 0.2   ;normalized frequency y ss (n) | | sin(  arg( )) f s 1 0.8 j 2 1 0.8 j

0.2 H (e j )  1 0.8e j

1 ``

0.5

0.1 0 1000 Hz

•The z-transform plays a similar role in DSP as the Laplace transform in analog circuits and systems.

•It provides intuition that is sometimes not evident in time- The Z-Transform in DSP domain analysis •Simplifies time-domain operations – time domain-convolution maps to Z-domain multiplication

•Used to define transfer functions

•Could be used to determine responses of systems using a table look-up process

From the Laplace Transform to the -TZ rans f o r m The -TZ ransform- Definition H (s)  H (z) a d Given the signal: x(n) s-domain transfer function z-domain transfer function   n its Z-transform is 1 bo Xz()  xnz () n  Ha (s)  Hd (z)  1 1 sRC 1 a1z For causal signals, i.e., x(n)  0 for n  0 R yn   C xn  z1  X ( z )   x ( n ) z  n n  0

Jan. 2009 Copyright 2009 ©Andreas Spanias III-3 Jan. 2009 Copyright 2009 ©Andreas Spanias III-4 Selected Properties of the -TZ ran s f o r m Delay of 7 Samples 7 Linearity: if: x()n  X ()z and yn() Y ()z x(7)nzXz () then xn() yn () X () z  Yz ()

m Shifting: x(n  m)  z X(z)

Convolution: x(n) * y(n)  X ( z)Y ( z)

n Scaling (bandwidth expansion): a x(n)  X ( z / a)

Selected -TZ rans f o r m P a i r s The Transfer Function Y (z) Unit-Impulse: X (z)  ()n  1 z1 ...z1 b  z1 z1 ... z1 L   -   1 . - a ...... -. 1  z sin( )  a Sinusoids: sin( n), n  0  , z  1 b1 2  1 2  .. 1  2z cos( )  z  . a b0 M L M 1  1  z cos(  )  y(n)  b x(n  i)  a y(n  i) cos(  n), n  0   1  2 , z  1  i  i 1  2 z cos(  )  z  i 0 i 1 To write the transfer function put the difference equation in the z-domain  1  Sampled Unit-Step: 10,,n  z  1 xn() X () z and yn() Yz () 1  z 1  L M  z  Y ( z)  b X ( z) z i  a Y ( z) z i Exponential Signals: ann ,,0  za   i  i  za  i 0 i 1 Jan. 2009 Copyright 2009 ©Andreas Spanias III-7 Jan. 2009 Copyright 2009 ©Andreas Spanias III-8 The Transfer Function (Cont.) The Transfer Function and the Impulse Response L M i i X ( z) ( bi z )  Y ( z) (1  ai z ) Convolution maps to multiplications in the z domain i 0 i1 The transfer function H(z) is defined as: y(n)  x(n) * h(n)  Y ( z)  X ( z)H ( z)

1  L Y (z) b0  b1z  ...  bL z It can be shown that a transfer function H(z) is related to the H (z)   1 M impulse response sequence h(n) by: X (z) 1 a1z  ...  aM z L  i H ( z )  h(n) z  n bi z   n  0 H (z)  i 0 M Note that feedback or 1  a z i  i terms are in the hn() H () z i 1 denominator

Example: Write the Transfer Function of a First Order IIR Filter Example: Transfer Function of a First Order IIR Filter (cont.)

yn Step 3: Since z-transform is a linear operation I can write  1 xn 0.2  z1 Y ( z)  0.2 X ( z)  0.8 z Y ( z)  Step 4: Form the ratio Y(z)/X(z) to get the transfer function 0.8 Step 1: Write the Difference Equation Y (z) 0.2 0.2z H (z)    y(n)  0.2 x(n)  0.8 y(n  1) X (z) 1 0.8z 1 z  0.8

Step 2: Transform all signals to the z domain Note below that the impulse response can be written by inspection of H(z) y(n)  Y (z) x(n)  X ( z) 0.2z h(n)  0.2 0.8n u(n)  H(z)  | z | 0.8 y(n  1)  z 1Y (z) z 0.8

x(n) + For the FIR filter the transfer function is: 1 + + L -1 -1 1  L i z + z H (z)  b  b z  ...  b z  b z Direct Form 1 0 1 L  i 2 1.3

i0 -1 -1 z z this is also in agreement with 3 0.4 L H ( z )  h ( n ) z  n  x(n) + y(n) n  0 1 + -1 Note that for FIR filters z +

1.3 2 Direct Form 2 h ( 0 ), h (1),..., h ( L )  b 0 , b1 ,...., b L z -1

Poles and Zeros of H(z) Example: Poles and Zeros of H(z)

In general the transfer function is rational; it has a numerator and a 0.2z • the zero (O) of this transfer function is at z=0 denominator polynomial. H (z)  z  0.8 • the pole (X) of this transfer function is at z=0.8 The roots of the numerator and denominator polynomials are called the zeros and the poles respectively. 1 Pole-zero decompositions of H(z) are quite useful and provide intuition in signal analysis and filter design. 0.5

L 0 O X (z  i ) (z 1)(z  2 )...(z  L ) i1 part Imaginary H(z)  G  G M -0.5 (z  p1)(z  p2 )...(z  pM ) (z  pi ) i1 -1

where G is a gain factor -1 -0.5 0 0.5 1 Real part

Jan. 2009 Copyright 2009 ©Andreas Spanias III-15 Jan. 2009 Copyright 2009 ©Andreas Spanias III-16 Example: Poles- Zeros of a Second Order System Poles and Zeros and Stability 1  1.3435 z 1  0.9025 z 2 H (z)  1  0.45 z 1  0.55 z 2 The poles are related to the stability of the filter since they are Note that the filter related to the impulse response of the system. In fact, the poles of oo (.ze95jj45 )(. ze 95  45 ) coefficients are For stability all the poles must be inside the unit circle, that is Hz() oo (.zeze7416jj72.. 34 )(. 7416  72 34 ) real valued and therefore poles lm and zeros occur in X  poles for all i = 1, 2, . . . , M complex pi 1 X 0  zeros 0 conjugate pairs. Re IIR filters may be all-pole or pole-zero and stability is always a concern. FIR or all-zero filters are always stable. 0 X

The Frequency Response Function The Frequency Response Function (Cont.)

The transfer function is The frequency response function and is a complex and periodic  1  L With period 2. The normalized frequencies  are associated to the b  b z  ...  b z sampling frequencies f by H ( z )  0 1 L s Sampling period 1  a z  1  ...  a z  M 1 M f    T  2 j by evaluating on the unit circle, i.e. for z  e f s

b  b e  j  ...  b e  jL rad rad/s Sampling frequency H (e j )  0 1 L  j  jM where fs is the sampling frequency and f is any frequency of 1 a1e  ...  aM e interest. In practice, one determines the frequency response up to half the sampling frequency (fold-over frequency).

The magnitude frequency response function

0 ππ2 L j Foldover Frequency  e   i j i 1 H (e )  G M j  e  p i 0 fs/2 fs i 1

The phase frequency response function •The frequency response is usually plotted w.r.t. normalized frequencies () L M j j j •The frequency response is periodic with period fs (2 π ) arg(H(e ))  arg(e  i )  arg(e  pi ) •Since frequencies of interest are up to the bandwidth of the analog signal i1 i1

the spectrum is usually plotted up to fs/2, ( π ) the foldover frequency

Remarks on Effects of Poles and Zeros on H(ej) Magnitude and Phase Response

•Poles tend to create peaks in the magnitude frequency response The magnitude and phase response of

Magnitude Response •Zeros tend to create valleys in the magnitude frequency response Im 2.0

•Very selective filters are designed efficiently by placing poles X 1.5 close to the unit circle Re 0

Magnitude H(z) 1.0 •Sharp notches are achieved efficiently with zeros very close to 0.5 the unit circle 0 X 0.0 0 32 64 96 128 •A sharp notch in the frequency response needs many poles (high Frequency Index (Theta=2*PI*Index/128) 0 2 order) if we are restricted to an all-pole filter (not efficient). 0zeros Foldover Frequency X  poles 0 fs/2 fs •A sharp peak in the frequency response needs many zeros (long or high order FIR) if an all-zero filter is to be used (not efficient).

o o X X o o X X foldover

As the poles move outwards we get sharper peaks and if the poles are on the unit circle we get an oscillator.

Bandwidth Expansion Stabilizing a Filter Using Pole Scaling Moving poles inwards stabilizes numerically a filter  n h(n)  H (z /  ) 0    1

example transient 1 n n H(z/) M i i 1  a z X  i X i1

X X

The scaling property of the z-transform is exploited for bandwidth expansion in LPC vocoders

Jan. 2009 Copyright 2009 ©Andreas Spanias III-27 Jan. 2009 Copyright 2009 ©Andreas Spanias III-28 Computing Filter Responses Using the Example: Find the Impulse Response of the Filter Inverse Z-Transform x(n) + + y(n)

- + Partial Fractions: Given a the transfer function + z-1 L L 1 b 0 z  b1 z  ...  b L 5/6 H ( z )  M M 1 -1 z  a 1 z  ...  a M z

1/6 for distinct poles write H(z) as: z2  z z z H z  H z  9  8 z2  5 z  1 1 1 z z 6 6 z - 2 z - 3 H (z)  c0  c1  ...  cM nn z  p z  p  1   1  1 M hn() 9    8   , n 0  2   3 

Use z transforms to change the filter structure xn  yn   Filter Configurations  z1  z1   = 0.8 0.4

Hz1 () Hz2 ()  Yz()  z 1  Xz() + x(n) .2008.8 y(n)  008.8  z 1 Hz3 () 

  z 1  =  x(n) y(n) Xz() Yz() + z 1 z 1 0.4 HzHz12() () Hz 3 () + - 1.2

0.32 Jan. 2009 Copyright 2009 ©Andreas Spanias III-31 Jan. 2009 Copyright 2009 ©Andreas Spanias III-32 Interesting Transfer Functions J-DSP Simulation Long Term Prediction Synthesis Filter 1 1 H (z)  Example p=10>> H (z)   p 1 0.9z 10 1 a p z

•Called long term because p is a long-term delay •Used in Vocoders

20 100

50 10

0 -50 Phase (degrees) Phase

-10 -100 Magnitude Response (dB) Magnitude 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1) Normalized frequency (Nyquist == 1)

LTP excited by a random signal J-DSP Simulation of LTP

1 1 0.95z 30 Frequency response 20 1 Impulse response 10 Hz() 10 0 10.9 z

Magnitude Response (dB) Response Magnitude -10 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1)

100

-50 Phase (degrees)

-100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1)

b0 bo H (z)  H (z)  Typically used in speech processing M M i i (vocoders) as well as in spectral 1  ai z 1  ai z estimation applications   i 1 i 1

y(n)

x(n) + b 0 T T T VOCAL SYNTHETIC - gain TRACT - - SPEECH FILTER a 1

a 2

Interesting Transfer Functions Interesting Transfer Functions Digital Oscillators Digital Oscillators (2)

If we realize the z-transform of a sinusoid as a transfer function and Note that H(z) has its poles on the unit circle we excite it with a unit impulse we get a sinusoidal output z1 sin() zsin()  1  z sin( ) H(z)  1 2  j  j sin( n), n  0   1 2 , z  1 1 2z cos()  z (z  e )(z  e ) 1  2z cos( )  z  If we excite H(z) below  X z1 sin() H(z)  1 2z1 cos()  z2

X with (n) then the output will be a sinusoid

Dual Tone Multi-frequency Encoder 1 z sin() Row Frequencies H(z)  1.209 kHz 1.336 kHz 1.477 kHz 1 2cos() z1 1 z2

0.697 kHz 1 2 3

b  0 (n) + 0 Sin(n) 0.77 kHz 4 5 6 + +

b  sin() -1 1 z a  2cos() 0.852 kHz 7 8 9

1 Column Frequencies

z -1 0.941 kHz * 0 # a2 1

DTMF Functionality

Generates dual-tone-multi- frequency (DTMF) tones used in landline telephony applications. DTMF DEMO

The tones can be played back using the J-DSP provided sound player, and used in a DSP simulation.

FFT yfnTfnTcos(2 12 ) cos(2 ) where f1 and f2 are chosen from the tone frequencies (697, 770, 852, 941, 1209, 1336, 1477 (Hz)). The sampling frequency is 8 KHz, i.e., T = 0.125ms

Advantages: Linear Phase Design Quite Efficient for designing notch filters Always Stable Design of FIR Digital Filters Disadvantages: Requires High Order for Narrowband Design

Applications: Speech Processing, Telecommunications Data Processing, Noise Suppression, Radar Adaptive Signal Processing, Noise Cancellation, Echo Cancellation, Multipath channels

FIR Digital Filters FIR Filter Frequency Response yn xn j  j  jL  ...  H (e )  b0  b1e  ...  b L e T T bL    . . .  L f . . . y(n)  b x(n  i)   2  b1  i i 0 f s b0

1  L o Y (z)  b0 X (z)  b1 X (z)z  ...  bL X (z)z

Y ( z )  1  L H ( z )   b 0  b1 z  ...  b L z X ( z ) foldover Jan. 2009 Copyright ©2009 Andreas Spanias IV-3 Jan. 2009 Copyright ©2009 Andreas Spanias IV-4 LINEAR PHASE DESIGN GROUP DELAY Linear Phase (constant time delay) FIR filter design is important in pulse transmission applications where pulse dispersion must be avoided. The frequency response function of the FIR filter is The time delay or group delay of a filter is defined as written as: d  (  ) j j j2 jL    H(e )b0 b1e b2 e ...bL e d 

where therefore if     is a linear function of  then  is a H(ej)M()e () constant.  is given in terms of samples M ()  H (e j ) , ()  arg( H (e j ))

LINEAR PHASE AND IMPULSE RESPONSE SYMMETRIC AND ANTI-SYMMETRIC SYMMETRIES LINEAR PHASE FILTERS

It can be shown that linear phase is achieved if Two Anti-symmetries for L=even or L=odd for h ( n )  h ( L  n )

where h(n) is the impulse response of the filter. For L = odd h(n)  h(L  n) L 1 2 H ( z )   h(n)( z  n  z  ( L  n ) ) Two Symmetries for L=even or L=odd for n  0 L1 h (n )  h ( L  n ) L  j 2 j 2   L  H (e )  e  2 h(n) cos   n  n0   2  Jan. 2009 Copyright ©2009 Andreas Spanias IV-7 Jan. 2009 Copyright ©2009 Andreas Spanias IV-8 EXAMPLES OF PHASE AND SYMMETRY IN h(n) EXAMPLES OF SYMMETRIES ) 50 L  4 L  3 L  4 0 hn hn -50 degrees -100

Phase ( -150

0 -200 0 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1)

100 hn hn L  4 L  3 L  4 (degrees) 0

-100

o o Phase 0 0 0 -200 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1)

Symmetries and Linear Phase Simulated with J-DSP Fourier Series Design Example

For the ideal low pass filter the impulse response sequence is an infinite length sampled sinc function. Lets say the sampling frequency is 8 KHz and we wish to have a cutoff frequency at 2 KHz. This results in

f 2000    2 c  2  c f 8000 2 That is s

H d z

    2 2

1  n   yn h d ( n )  sinc  , n  0 , 1, 2 ,...  2  2  1/ 5 1/ 3 1/ 1/ 2 For an FIR filter of 11 coefficients     1 1 1 1 1 1 1  h(n) ...0,0, ,0, ,0, , , ,0, ,0, ,0,0,... xn 1 1 1 1 1  5 3  2  3 5  ..z z z z . z .

z1 z1 . z1 z1 . z1 This impulse response is not causal, however a shift operator with 5 delays (z-5) will convert it into a causal DF.

Fourier Series Design Example (Cont.) Fourier Series Design Example L=32 1 1 1 1 1 1 1 1  (n  16)  b0  , b2  ,b4  ,b5  , b6  ,b8  , b10  bn  sinc  , n  0,1,.... 32 5 3  2  3 5 2  2 

20 20

0 0

-20 -20

-40 -40

-60 -60

Magnitude Response (dB) -80

Magnitude Response (dB) -80 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1) Normalized frequency (Nyquist == 1) 0 0

-500

-200 -1000

-400 -1500 Phase (degrees)

Phase (degrees) -2000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 -600 Normalized frequency (Nyquist == 1) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1) Jan. 2009 Copyright ©2009 Andreas Spanias IV-15 Jan. 2009 Copyright ©2009 Andreas Spanias IV-16 Fourier Series Design Example L=64 Truncating with Hamming Window L=64 1  (n  32)  1  (n  32)  bn  sinc  , n  0,1,...., 64 bn  sinc   x Hamming(L) 2  2  2  2 

50 50

0 0 -50

-50 -100

Magnitude Response (dB) -150 Magnitude Response (dB) -100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1) Normalized frequency (Nyquist == 1) 0 0

-1000 -1000 -2000

-2000 -3000 Phase (degrees) Phase (degrees)

-3000 -4000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1) Normalized frequency (Nyquist == 1) Jan. 2009 Copyright ©2009 Andreas Spanias IV-17 Jan. 2009 Copyright ©2009 Andreas Spanias IV-18

F.S. Design; Rectangular vs Hamming Window - L=64 Truncating in time- frequency convolution and F.S. design

Rectangular 50 Wide mainlobe = Narrow mainlobe= Ideal LPF Wide transition Narrower transition 0 ~13 dB

-50

Magnitude Response (dB) -100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1) Hamming

0 ~45 dB -50 •The main-lobe width determines transition characteristics

-100 •The sidelobe level determines rejection characteristics

Magnitude Response (dB) -150 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized frequency (Nyquist == 1) Jan. 2009 Copyright ©2009 Andreas Spanias IV-19 Jan. 2009 Copyright ©2009 Andreas Spanias IV-20 Truncating with Short/Long Windows and F.S. design Notes On Fourier Series Design

jΩ jΩ jΩ HLPF(e )*W(e ) |H(e )| Wide mainlobe Wide transition •The design performed in the previous example involved truncation of an ideal symmetric impulse response.A symmetric impulse response produces a linear phase design.

Ω •Truncation involves the use of a window function which is multiplied with the impulse response. Multiplication in the time H (ejΩ)*W(ejΩ) |H(ejΩ)| LPF domain maps into frequency domain convolution and the spectral Narrow mainlobe Narrow transition characteristics of the window function affect the design.

•The main-lobe width determines transition characteristics Ω •The sidelobe level determines rejection characteristics

DESIGN USING THE KAISER WINDOW DEFINING DESIGN SPECIFICATIONS The Kaiser window is parametric and its bandwidth as well as its sidelobe energy can be designed. Mainlobe bandwidth controls the transition 2 Hf Tolerance Regions characteristics and sidelobe energy affects the ripple characteristics.

 2 1 / 2  1 dp   n     I   1   1 0              1 dp w(n)    ,0  n  L  1 Ideal LPF I (  ) ds 0

f  = L/2 ; associated with the order of the filter fp fc fs Passband Transition band Stopband  is a design parameter that controls the shape of the window

I0(.) is a zeroth order modified Bessel function of the first kind

1 0

0.9

2 0.8  k 1x  0.7 0.6 I0(x)1       0.5 k1 k! 2     0.4 0.3

0.2

25 terms from the Bessel function are sufficient 0.1 5

0 0 20 40 60 80 100 120 140

KAISER WINDOW DESIGN EQUATIONS

Given fp, fs, T and dp, ds determine the FIR filter coefficients. DESIGN PROCEDURE

  min( ds , dp ) 1. Determine the cutoff frequency for the ideal Fourier Series A   20 log  method. 10 ffsp f c    2  ( f s  f p ) T 2 2. Design the ideal LPF using the Fourier Series. A  8 The filter order is L  (  2 ) 3. Design the Kaiser window 2 .285  4. Shift and truncate the ideal impulse response L and the kaiser parameter  is given by   h LPF ( n )  w ( n ) h d  n  , 0  n  L  2  0 .1102 ( A  8 .7 ), A  50   0 .4    0 .5842 ( A  21 )  0 .07886 ( A  21 ), 21  A  50  Note that this procedure can be generalized for the design of   BPF, HPF, and BSF. 0, A  21 

As zeros are placed towards the unit circle the frequency response magnitude decreases at and in the vicinity of the Ideal frequency response frequency of the zeros. (a)

. 0 π/4 π/2 π

Sampled ideal frequency response

N=16 samples (b) o 0 π/4 π

Interpolated frequency response o

Min-Max and Parks-McClellan Optimum FIR Design Frequency Sampling Methods for FIR Filter design (3) The Parks-McClellan design is based on Min-Max

Equiripple and linear phase design is possible

This class of methods involve minimizing the maximum error between the designed FIR filter frequency response and a prototype

min max E (e j )  h (i ),i  0 ,1,..., L where

j j j j E(e )  W (e )(H d (e )  H (e ))

IN THE MATLAB SP TOOLBOX Direct Realizations L  i H ( z )   b i z cremez - Complex and nonlinear phase equiripple FIR filter design. i  0 fir1 - Window based FIR filter design - low, high, band, stop, multi. xn fir2 - Window based FIR filter design - arbitrary response. 1 1 ... 1 fircls - Constrained Least Squares filter design - arbitrary response. z z z fircls1 - Constrained Least Squares FIR filter design - low and highpass firls - FIR filter design - arbitrary response with transition bands. b b firrcos - Raised cosine FIR filter design. 0 b1 2 bL1 bL intfilt - Interpolation FIR filter design. y n kaiserord - Window based filter order selection using Kaiser window.   remez - Parks-McClellan optimal FIR filter design.   ...   remezord - Parks-McClellan filter order selection. •Requires multiply accumulate instructions + MATLAB is registered trade mark of the MathWorks

FIR Filter Cascade Realizations Transform-Domain FIR Filter Realizations Xk(0) Cascade Realizations Bk(0)

x(n) x Xk(1) y q FFT IFFT 1  2 Bk(1) H ( z )   (b i 0  b i1 z  b i 2 z ) i 1 Xk(L) xn Bk(L) b b10  20  ... y(n) - z1 z1 Select Last N points b 11  b21  1 z1 z A Transform domain realization is possible using the overlap b and save and the FFT. This yields computational savings for high b12 22 order implementations. Input data is organized in 2N-point blocks and blocks are shifted N points at a time. The data blocks and •Reduced Effects from Coefficient Quantization and round-off N zero-padded coefficients are transformed and multiplied and the •In Fixed-Point implementation signal scaling must be results is inverse transformed. The last N-points are selected as the done carefully at each stage result. The blocks are updated and the process is repeated. Jan. 2009 Copyright ©2009 Andreas Spanias IV-35 Jan. 2009 Copyright ©2009 Andreas Spanias IV-36 Implementing Efficiently Digital Cross-Over Using Subtractive Operations

could also implement delay compensation z –L/2

+ - LPF  to tweeter

Linear Phase LPF to woofer With delay L/2 samples

Advantages: Efficient in terms of order • Poles create narrow-band peaks efficiently • Arbitrarily long impulse responses with few feedback DESIGN OF IIR DIGITAL FILTERS • coefficients Disadvantages: • Feedback and stability concerns • Sensitive to Finite Word Length Effects • Generally non-Linear Phase

Applications: • Speech Processing, Telecommunications • Data Processing, Noise Suppression, Radar

IIR FILTERS IIR FILTERS (Cont.)

The difference equation is: The transfer function:

L M 1 L y (n )   bi x (n  i)   a i y (n  i) Y (z) b0  b1z  ... bL z i  0 i 1 H (z)   1 M X (z) 1 a1z  ... aM z yn xn The frequency-response function : z1 ...z1  z1 z1 ... z1 bK   -   - bK 1 -  j  jL . . .. . a1 . . j b0  b1e  ...  bL e b1 a2 H (e )  ...  j  jM b0 a M 1 a1e  ...  aM e

The idea is to use many of the successful analog filter designs to design digital filters The impulse invariance method suffers from aliasing and is rarely used This can be done by either:

• by sampling the analog impulse response (impulse invariance) The bilinear transformation does not suffer from aliasing and is and then determining a digital transfer function by more popular than the impulse invariance method. The frequency relationship from the s-plane to the z-plane is or non-linear, and one needs to compensate by pre-processing the critical frequencies such that after the transformation the desired • by transforming directly the analog transfer function to a digital response is realized. Stability is maintained in this transformation filter transfer function using the bilinear transformation since the left-half s-plane maps onto the interior of the unit circle.

Impulse Invariance Impulse Invariance (2)

t 11 ht() eRC ut () and H ( j ) aaRCjRC1  11 1 Hsa ( ) ..... R s psp12 sp M C i x(t) y(t)

yn

 1 xn 0.2  z  TT T Hz( ) ..... 0.8 11 ezpT1211 ez p T 1 e  pM T z 1 nT T  (/TRC ) hn() eRC un () Hz() RC 1 ez(/TRC ) 1

Jan. 2009 Copyright © 2009 Andreas Spanias V-7 Jan. 2009 Copyright © 2009 Andreas Spanias V-8 The Bilinear Transformation The Bilinear Transformation (Cont.) 1  s Bilinear Transform  z  The bilinear transformation compresses the frequency axis 1  s S  plane z  plane  ,  , Im Im  

The non-linear frequency transformation (frequency warping function) is given by Re Re 15  10    5   tan 0 z  1   H ( z )  H ( s  )  2  -5 z  1 -10 -15-3 -2 -1 0 1 2 3 Jan. 2009 Copyright © 2009 Andreas Spanias V-9 Jan. 2009 Copyright © 2009 Andreas Spanias V-10

Applying the Bilinear Transformation Procedure for Analog Filter Approximation

specification

1. Consider Critical Frequencies  Prewarping 2. Pre-warp critical frequencies

3. Analog Filter Design ω Design 4. Bilinear Transformation

5. Realization ω bilinear transformation

Jan. 2009 Copyright © 2009 Andreas Spanias V-11 Jan. 2009 Copyright © 2009 Andreas Spanias V-12 EXAMPLE: TRANSFORMING AN RC CIRCUIT TO A DF TRANSFORMING AN RC CIRCUIT TO A DF (2) R Suppose we want a first-order C Step 4: Apply the Bilinear Transform (R-C LPF) appoximation i(t) x(t) z 1 1 1 H(z)  H(s  )   0.50.5z1 H ()  z 1 z 1 1 jRC 1 z 1 Say we have the following DF specs: Apply pre-warping

Step 5: Realization c  Step 1: c  / 2 Step 2:   tan( )  tan( ) 1 c 2 4 xn yn 0.5 

1 Step 3: Design the analog filter. z In this case the analog filter function 1 H(s)  0.5 is a first order LPF similar  1 s to an RC circuit Jan. 2009 Copyright © 2009 Andreas Spanias V-13 Jan. 2009 Copyright © 2009 Andreas Spanias V-14

TRANSFORMING AN RC CIRCUIT TO A DF (3) Analog Filter Designs Frequency Response H(ej)0.50.5ej

-20 1 •Butterworth - Maximally flat in passband

-40 0.5

Magnitude Response (dB) -60 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 •Chebyshev I - Equiripple in passband Normalized frequency (Nyquist == 1) 0

0 Imaginary part

-20 -0.5 -40 •Chebyshev II - Equiripple in stopband -1 -60

-1 -0.5 0 0.5 1 -80 Real part Phase (degrees)

-100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 •Elliptic - Equiripple in passband and stopband Normalized frequency (Nyquist == 1)

Notice that there is no aliasing effect with the bilinear transformation. Although in this simple R-C example the resultant digital filter is FIR,

moreJan. 2009 complex analog filters Copyright will © 2009 yield Andreas IIR Spanias digital filters. V-15 Jan. 2009 Copyright © 2009 Andreas Spanias V-16 Butterworth Max. Flat in Passband and Stopband Butterworth Filter Design

Butterworth frequency response - transition is steeper as order increase

• Maximally Flat in the Passband and Stopband 0.9

0.8

0.7

Butterworth Transfer Function Design Example - Butterworth

1 H (s)H (s)  s 2M • A Butterworth filter is designed by finding the poles of 1 ( ) H(s)H(-s) jC Poles on a circle of

radius c s • The poles fall on a circle with radius c ( ) 2 M  1 j C • The poles falling on the left hand s-plane (stable poles) are j ( 2 k  M 1) chosen to form H(s) 1 / 2 M 2 M s k  (1) j C   C e • H(s) is transformed to H(z) k  0,1,..., 2M  1

note that roots can not fall on imaginary axis Jan. 2009 Copyright © 2009 Andreas Spanias V-19 Jan. 2009 Copyright © 2009 Andreas Spanias V-20 Examples of IIR Filter Design Using MATLAB Butterworth Design in MATLAB

• % Design an IIR Butterworth filter FUNCTIONS IN THE SP TOOLBOX • clear • N=256; %for the computation of N discrete frequencies • Wp=0.4; %passband edge • Ws=0.6; %stopband edge IIR digital filter design. • Rp=1; % max dB deviation in passband butter - Butterworth filter design. • Rs=40; %min dB rejection in stopband cheby1 - Chebyshev type I filter design. • [M,Wn]=buttord(Wp,Ws,Rp,Rs); cheby2 - Chebyshev type II filter design. • [b,a]=butter(M,Wn); ellip - Elliptic filter design. • theta=[(2*pi/N).*[0:(N/2)-2]]; % precompute the set of discrete frequencies up maxflat - Generalized Butterworth lowpass filter design. to fs/2 • H=freqz(b,a,theta); % compute the frequency response yulewalk - Yule-Walker filter design. • plot(angle(H)) • pause IIR filter order selection. • H=(20*log10(abs(H))); % plot the magnitude of the frequency response • plot(H) buttord - Butterworth filter order selection. • title('frequency response') cheb1ord - Chebyshev type I filter order selection. • xlabel('discrete frequency index (N is the sampling freq.)') cheb2ord - Chebyshev type II filter order selection. • ylabel('magnitude (dB)') ellipord - Elliptic filter order selection.

Butterworth Design in MATLAB (2) Butterworth Design in MATLAB (3)

frequency response 1.4

1.2

• Wp=0.4; %passband edge 0.8

0.6 • Ws=0.6; %stopband edge magnitude • Rp=1; % max dB deviation in passband 0.4

• Rs=40; %min dB rejection in stopband 0.2

0 0 20 40 60 80 100 120 140 discrete frequency index (N is the sam pling freq.)

• b = 0.0021 0.0186 0.0745 0.1739 0.2609 0.2609 0.1739 1 • 0.0745 0.0186 0.0021 0.5

• a = 1.0000 -1.0893 1.6925 -1.0804 0.7329 -0.2722 0.0916 0

• -0.0174 0.0024 -0.0001 Imaginary part

-0.5

-1

frequency response 0 Chebyshev I - Equiripple in passband -10 -20

Chebyshev II - Equiripple in stopband -30

-40

-50

-60 magnitude (dB) magnitude

-70

% Design an IIR Chebyshev II filter - Ex 2.20 -80

-90

clear -100 0 20 40 60 80 100 120 140 N=256; %for the computation of N discrete frequencies discrete frequency index (N is the sampling freq.)

Wp=0.4; passband edge Ws=0.5; stopband edge 1 Rp=1; ripple in passband (dB) b = 0.0274 0.1065 0.2290 0.3252 0.3252 0.2290 0.1065

Rs=60; rejection (dB) 0.5 .0274 [M,Wn] = cheb2ord(Wp,Ws, Rp, Rs); % determine order 0

[b,a] = cheby2(M,58,Wn); %determine coefficients part Imaginary

size(a) a =1.0000 -0.7484 1.2644 -0.4555 0.3427 -0.0454 0.0186 -0.5 size(b) -0.0002 % use routines to plot frequency response -1

MATLAB Elliptic Design Example IIR Elliptic

% Design an IIR Elliptic filter frequency response 4 clear 0 N=256; %for the computation of N discrete frequencies -10 3

-20 2

Wp=0.4; %passband edge -30 Ws=0.6; %stopband edge 1 -40 Rp=2; % max dB deviation in passband 0 Rs=60; %min dB rejection in stopband -50 magnitude (dB) -1 [M,Wn] = ellipord(Wp,Ws,Rp,Rs); -60 -2 [b,a] = ellip(M,Rp,Rs,Wn); %design filter -70 size(a) -80 -3 size(b) -90 -4 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 discrete frequency index (N is the sampling freq.) theta=[(2*pi/N).*[0:(N/2)-2]]; % precompute the set of discrete frequencies up to fs/2 H=freqz(b,a,theta); % compute the frequency response plot(angle(H)) pause 1 H=(20*log10(abs(H))); % plot the magnitude of the frequency response plot(H) title('frequency response') b = 0.0181 0.0431 0.0675 0.0675 0.0431 0.0181 0.5 xlabel('discrete frequency index (N is the sampling freq.)') ylabel('magnitude (dB)') 0 pause zplane(b,a) ;% z plane plot a = 1.0000 -2.3214 3.3196 -2.8409 1.5154 -0.4151 part Imaginary -0.5

-1

-1 -0.5 0 0.5 1 Real part Jan. 2009 Copyright © 2009 Andreas Spanias V-27 Jan. 2009 Copyright © 2009 Andreas Spanias V-28 Shelving Filters Introduction to Special types of Digital Filters Shelving filters realize tone controls in audio systems. The frequency response of a low-pass (bass) and high-pass (treble) shelving filter is shown below. • Shelving Filter

• Peaking Filter

• Graphic Equalizer Frequency response of a low-pass and high-pass shelving filter.

Tone Control Block in J-DSP Peaking Filter Block in J-DSP

The low frequencies are A -DJ SP simulation using the Peaking Filter block is affected by bass adjustments with the shown below audio signal processed through low-pass shelving filters. The high frequencies are affected by treble adjustments with the audio signal processed through high- pass shelving filters. A J-DSP simulation using the Tone Control . Figure J-DSP simulation using the tone control block. Figure 9: J-DSP simulation using the peaking filter block. Jan. 2009 Copyright © 2009 Andreas Spanias V-31 Jan. 2009 Copyright © 2009 Andreas Spanias V-32 Graphic Equalizer Graphic Equalizer Block in J-DSP • A graphic equalizer uses a cascade of peaking filters • It alters the frequency response of each band by varying the A -DJ SP simulation using the Graphic Equalizer corresponding peaking filter’s gain. block is shown below. The sliders are a graphic representation of the frequency response applied to the input audio signal, hence the name “graphic” equalizer.

Input …

Peaking Peaking Peaking Peaking Filter Filter … Filter Filter #1 #2 #N-1 #N

+ … + + Output A diagram of a graphic equalizer with N bands of -DSP simulation using the graphic equalizer block. Jan. 2009control. Copyright © 2009 Andreas Spanias V-33 Jan. 2009 Copyright © 2009 Andreas Spanias V-34

FIR vs IIR Digital Filters FIR vs IIR Digital Filters (Cont.)

FIR IIR FIR IIR Always stable Not always stable Pole-zero efficient for both Transversal Recursive notches and peaks All-zero model All-pole or Pole-zero model Requires high order design Generally requires lower Moving Average(MA) model Autoregressive(AR) or order design Autoregressive Moving Less sensitive to finite word More sensitive to finite word Average (ARMA) model length implementation length implementation Inefficient for spectral peaks Efficient for spectral peaks (all- Linear phase design Generally non-linear phase pole, pole-zero) Efficient for spectral notches All pole inefficient for spectral notches

Echo Effects Reverberation Effects

The echo effect is obtained by Reverberation is obtained by mixing the input signal with its mixing the input signal with delayed version. the delayed versions of its feedback. The proportion of the delayed signal to the "clean" original The effect of the feedback signal determines how obvious results in multiple echoes. the echo is, and the delay signifies the echo period. Echo  y(n) = x(n) + b . y(n-R) Reverb  y(n)= x(n) + b . x(n-R) • R = feedback delay in samples. • R = the number of echo delay in samples. • b is the attenuation constant (|b| < 1). • In order to have a distinguishable echo, R should be relatively large. • b is the attenuation constant (|b| < 1).

Other Methods for Digital Filtering 2-D Filters for Image Applications Median Filters y (n )  median { x (n  i)} i  0 ,1, 2 ,..., L • FIR 2-D filter LM The median operation ranks the samples in the memory of the yn(, n ) hn (, n )( xn ln , m ) filter and picks the sample that falls in the middle of the 12 12 1 2 rank and assigns it to the output y(n) lm00 • IIR realizations also possible Used for impulsive noise. One application reported is scratch noise removal in vinyl record restoration • Theory very similar to 1-D and described in multidimensional signal processing books

Low-pass filtering of a natural High-pass filtering of a natural image imageJan. 2009 Copyright © 2009 Andreas Spanias V-41 The DTFT of a finite sequence if:: 10....., nN  1 xn()   0.....,elsewhere  then N 1 1  e  jN  THE DISCRETE AND THE FAST X (e j ) e  jn     j FOURIER TRANSFORM n 0 1  e or sin(N/2) X(e j)  e j(N1)/2 sin(/2) Remark: The sin(.)/sin(.) function is known as a digital sinc or a Dirichlet function. Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-1 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-2

The DTFT of a finite sequence (Cont.) The DTFT of a finite sequence (Cont.)  10....., n 15 xn()    10....., n 7  0 .....,elsewhere xn()    0 .....,elsewhere 16 8 14

7 12

1 The Discrete Fourier Transform (DFT) The DFT Matrix The DFT and the IDFT may be expressed in terms of matrices, i.e., N 1 Xk() xne () j 2/ kn N and k = 0, 1, . . . , N-1  X 0  1 1 1 ... 1   x0      -1 -2 -N -1    n 0 X 1 1   ...  x1        X 2  1  -2  -4 ...  -2N -1   x2  The inverse Discrete Fourier Transform (IDFT) of the sequence x(n)     .  ......  .         .   ......   . 

N 1    2    1 -N -1 -2 N -1 - N -1 x ( n )   X (k )e j 2  kn / N and n = 0, 1, …, N-1 X N 1  1   ...   xN - 1  N k  0 kjkN2/ 1 1 H where   e and F  F The DFT transform pair is denoted by N xn() Xk () 1   a more compact form X  F x and x  F X Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-5 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-6

The DFT Matrix (2) Selected Properties of the DFT

N=2, N=4, and N=8 Linearity: xn() yn ()  X () k   Yk () 

1 1  jkmN2  / F   Shifting: 1 -1 xn()mod m N e X () k

11 1 xn() hn () X () k H () k  Circular Convolution: F 1 0.5jj 0.866  0.5 0.866 1 0.5jj 0.866  0.5 0.866 N 1  where xn  h n   h m  x n - m mod N 11 1 1 m0  1 11j j xnwn() ()Xk () Wk () F  Freq. Circular Convolution: N 1111 N  1 N  1 2 1 2  xn() Xk () 11j j Parseval’s Theorem:   n  0 N k  0 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-7 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-8

2 Frequency resolution of the DFT Spectral Estimates over Finite-time Data windows

The frequency resolution of the N-point DFT is Frequency domain representations are appropriately defined by the f f  s Fourier Transform integrals over an infinite time span. r N The DFT, however, estimates the spectrum over finite time •The DFT can resolve exactly only the frequencies falling exactly at: k fs/N. There is spectral leakage for components The DFT essentially applies a window to truncate the data. falling between the DFT bins The simplest data window is the rectangular (boxcar). •Typically we use an FFT that is as large as we can afford Truncation in time is convolution in frequency •Zero-padding is often use to provide more resolution in the frequency components The frequency domain characteristics of the data window, namely its bandwidth and sidelobes, affect the DFT spectral estimate. •Zero padding is often combined with tapered windows Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-9 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-10

WINDOWS FFTs of Sinusoidal Signals (1) The spectral characteristics of the window affect the spectral estimates. The rectangular window has the narrowest mainlobe width but the wordt sidelobes. 256-point FFT of a 500 Hz sinusoid (fs=8 kHz). Notice Tapered windows have wider mainlobe width but better behaved bandwidth. that this sinusoid is resolved exactly

FFT N-point Mainlobe Sidelobe 50 width Window Level 0

Rectangular 4π/(N+1) -13 dB -50

Triangular 8 π /N - 25 dB -100 -150

Hamming 8 π /N - 41 dB magnitude (dB) -200

-250 Hanning 8 π /N - 31 dB -300

-350 Blackman 12 π /N - 57 dB 0 50 100 150 200 250 300 discrete frequency index (N is the sampling freq.) Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-11 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-12

3 FFTs of Sinusoidal Signals (2) Zero-padded FFTs of Sinusoidal Signals (2)

256-point FFT of a 510 Hz sinusoid (fs=8 kHz). Notice 16-point FFT of a 16-point 590 Hz sinusoid (fs=8 kHz). Vs that this sinusoid is NOT resolved exactly 256-point FFT of a 16-point 590 Hz sinusoid (fs=8 kHz). Notice that although this sinusoid is NOT resolved exactly the frequency of the peak in the FFT zero-padded case is closer to actual 45

FFT FFT 30 20 20

25 15 15

20 10 10

15 5 magnitude (dB) magnitude 5

10 (dB) magnitude

0 (dB) magnitude 0 5 -5 -5 0 -10 0 2 4 6 8 10 12 14 16 -10 -5 discrete frequency index (N is the sampling freq.) 0 50 100 150 200 250 300 0 50 100 150 200 250 300 discrete frequency index (N is the sampling freq.) discrete frequency index (N is the sampling freq.) Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-13 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-14

Windowed FFTs on Sinusoids Window Comparison with Closely Spaced Periodicities

hamming windowed FFT boxcar windowed FFT 20 20 boxcar windowed 512-point FFT of two 32 point sinusoids Hamming windowed 512-point FFT of two 32 point sinusoids 35 30 10

15 25 30 0 20 10 -10 25 15

5 10 -20 20 magnitude (dB) 5 magnitude (dB) 0 -30 15 0 magnitude (dB) magnitude (dB) -40 -5 -5 10

-50 -10 0 50 100 150 200 250 300 -10 discrete frequency index (N is the sampling freq.) 5 0 50 100 150 200 250 300 -15 discrete frequency index (N is the sampling freq.) hanning windowed FFT 20 0 -20 0 100 200 300 400 500 600 0 100 200 300 400 500 600 discrete frequency index (N is the sampling freq.) 10 discrete frequency index (N is the sampling freq.)

0 boxcar. hamming High Resolution" -10 boxcar. 60 hamming -20 50

-30 40

-40 magnitude (dB) 30

-50 20 -60 10 -70 magnitude (dB) magnitude 0 -80 0 50 100 150 200 250 300 discrete frequency index (N is the sampling freq.) -10

high resolution ----> -20

-30 hanning 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 discrete frequency index (N is the sampling freq.) Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-15 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-16

4 Window Comparison with Distant Periodicities The FFT-DIT Algorithm

Hamming windowed 512-point FFT of two 8192 point sinusoids Boxcar windowed 512-point FFT of two 32-point distant sinusoids 50 The FFT decimates the sequence and performs a DFT by processing results 50

40 of smaller size DFTs. This is done by decomposing the N-point DFT to 2- 30

30 20 point DFTs and using “butterfly” operations to obtain the result. For a

10 20

magnitude (dB) magnitude 0 Decimation in Time (DIT) FFT algorithm the following steps are taken: magnitude (dB) 10 -10

-20 0

-30 0 100 200 300 400 500 600 N  1 -10 discrete frequency index (N is the sampling freq.) 0 100 200 300 400 500 600 discrete frequency index (N is the sampling freq.) Xk() xne () jnkN2  / High Resolution"  80  boxcar. hamming n  0 60

40 20 By decimating x(n) we can write magnitude (dB) 0

-20

-40 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 (/)N 21 (/)N 21 discrete frequency index (N is the sampling freq.) Xk() x (221 ne ) jnkN22/ x ( n ) ejnkN221()/ High Resolution n0 n0

The FFT-DIT Algorithm (Cont.) The FFT-DIT Algorithm (Cont.) if we define nk  j2nk/ N x1(n)  x(2n) x2(n)  x(2n1) WN  e X 0 X 0 x0 1

X11 X 1 ( N / 2)1 ( N / 2 )1 x2 4  point nk nk X12 X 2 X 1 (k )   x1 (n)W N / 2 X 2 (k )   x2 (n)W N / 2 x4 DFT n 0 n 0 X 3 X 3 x6 1 and X 0 W 0 X 4 x1 2 8 1 k X 1 1 X 5 2   W 1   X (k)  X1(k) WN X 2 (k), k  0,1,..., N / 2 1 x3 4  point 8 2 X 22 1 X 6 k x5 DFT W8 3 X 7 X (k  N / 2)  X1(k) WN X 2 (k) X 2 3 1 x7 W8

Remarks: The N-point DFT is broken down to two N/2-point DFTs. We then write the N/2-point DFTs as a combination of two N/4-point DFTs and so forth.

5 The FFT-DIT Algorithm (Cont.) The DFT and the FFT Complexity

x(0) X'(0) X (0) 1 1 X(0) The N-point DFT requires N2 multiplications and N2 –1 additions to x(4) X'(1)1 X (1) X(1) -1 1 compute the discrete frequency spectrum. 0 x(2) X'(0) W8 X (2) 2 -1 1 X(2)

2 x(6) X'(1) W8 X (3) X(3) -1 2 -1 1 The complexity of the DFT is reduced using the FFT to

N/2 log2N multiplications and N log2N additions.

X (0) 0 x(1) X'(0)3 2 X(4) W8 -1

1 For example if N=4096 the DFT requires 16,777,216 x(5) X'(1)3 X2 (1) W8 -1 -1 X(5)

0 2 multiplications while the FFT requires 49,152 multiplications. X'(0) W8 X (2) W8 x(3) 4 -1 2 -1 X(6)

2 3 X'(1) W8 X (3) W8 x(7) -1 4 -1 2 -1 X(7)

FFT ALGORITHMS

IN FFT DECIMATION-IN-TIME -the frequency-domain (output) indices are in place while the time-domain (input) indices are bit-reversed

IN FFT DECIMATION-IN-FREQUENCY -the time-domain indices are in place while the frequency-domain indices are bit-reversed FFT Applications VARIANTS OF FFT ALGORITHMS: Low-Complexity "Prunned" FFTs - For computing fewer frequency bins - when time-domain values are systematically zero (ex: zero padded FFTs)

Radix 4 and Mixed-radix FFTs, Gortzel Algorithm (computes only one frq. bin), Rader, Prime Factor, Winograd, Zoom FFTs

Reference: E. Brigham, "The FFT and its Applications," Prentice Hall, NJ 1988 Links on FFT: http://www.fftw.org/links.html FFT laboratory: http://sepwww.stanford.edu/oldsep/hale/FftLab.html Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-23 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-24

6 FFT AND SPEECH ENHANCEMENT FFT SPEECH ENHANCEMENT IN CDMA

S (n) •If we have speech corrupted by background noise, the spectrum HP G(k) H(k) of background noise can be estimated during speech pauses. Overlap/add Window/FFT X •Speech is enhanced in the spectral domain by subtracting the IFFT estimated noise spectrum from the noisy speech spectrum

•FFT-based spectral subtraction is used in military applications Channel Energy Channel Gain Estimator Computation •Also used in CDMA cellular IS-127 telephony standard Spectral Deviation Ech(m) En(m) Estimator

Channel SNR Noise Channel SNR Etot(m) Estimator Estimator Modification

Voice Metric

Computation

v(m)

Noise Uodate

Decision Update Flag

FFT AND SINUSOIDAL TRANSFORM CODING Simple FFT Based Compression Output signal vector ENCODER PHASES Input signal TAN-1(.) DFT Redundancy IDFT ^ Speech vector x Analysis Removal Synthesis x FFT FREQUENCIES Peak . Picking AMPLITUDES DECODER

PHASES FRAME-TO-FRAME SYNTHETIC PHASE SINE WAVE SUM ALL GENERATOR x SINE WAVES UNWRAPPING & SPEECH INTERPOLATION FREQUENCIES

FRAME-TO-FRAME LINEAR AMPLITUDES INTERPOLATION Implemented by IFFT and R. McAulay and T. Quatieri, “Sinsoidal Coding,”Ch. 4, Speech overlap&add procedure Coding and Synthesis, W.B. Kleijn and K. Paliwal, eds, Elsevier, 1995. Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-27 Jan. 2009 Copyright (c) 2009 Andreas Spanias VII-28

7 Fast Convolution Using the FFT Fast N-point Circular Convolution

• Fast Circular Convolution x(n) X(k) FFT

Y(k) y(n) b. Fast Linear Convolution X IFFT

h(n) H(k) FFT FFTs are often used to compute efficiently convolutions of very long sequences. Such convolutions arise in adaptive filters that are used in noise and echo cancellation.

J-DSP and Fast Convolution Fast N-point Linear Convolution using the Overlap/Save

x(n) 2N X(k) FFT Y(k) 2N X IFFT

2N H(k) FFT 1 y(n) h(n) 1

0 0

8 Orthogonal Frequency Division Multiplexing (OFDM) and FFTs

zk (n)  H (2k / N ) sk (n)  wk (n) k  1,, N

z(n)  DH s(n)  v(n)