sign in sign up
<<
Home , Motion compensation

Overview: motion-compensated coding

n Motion-compensated prediction n Motion-compensated hybrid coding n Power spectral density of the motion-compensated prediction error n Rate-distortion analysis n Loop filter n Motion compensation with sub-pel accuracy

Bernd Girod: EE368b Image and Compression Motion Compensation no. 1

Motion-compensated prediction

previous frame stationary Dt background current frame x

time t y moving object

æ dx ö „Displacement vector“ ç ÷ shifted è dy ø object

Prediction for the luminance signal S(x,y,t) within the moving object: ˆ S (x, y,t) = S(x - dx , y - dy ,t - Dt)

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 2

1 Motion-compensated prediction: example

Previous frame Current frame

Current frame with Motion-compensated displacement vectors Prediction error Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 3

Motion-compensated hybrid coder

Control „side information“ Video Channel signal Intraframe S e DCT coder - Intraframe Decoder

Motion compensated predictor Decoder

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 4

2 Motion-compensated hybrid decoder

Channel reconstructed Intraframe video signal Decoder Motion compensated predictor

side information

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 5

Model for performance analysis of an MCP hybrid coder luminance signal S e R-D optimal intraframe - encoder

intraframe decoder

e‘

motion s‘ compensated predictor

displacement estimate

æ dx ö æ D xö true displacement ç ÷ + ç ÷ displacement error è dyø è D yø

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 6

3 Analysis of the motion-compensated prediction error

Motion-compensated signal

c(x) = s(x - Dx )- n(x) Prediction error Previous e(x) = s(x)- c(x) frame = s(x)- s(x - Dx )+ n(x)

x c(x) s(x) Current frame d Displacement x Displacement error Dx

x

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 7

Analysis of m.c. prediction error (cont.)

n Motion-compensated prediction error

e(x)= s(x)-c(x)=s(x)-s(x-Dx)+n(x)=(d(x)-d(x-Dx ))*s(x)+n(x)

n Power spectrum of prediction error, assuming constant displacement s n error D x, statistical independence of and

- jwD x jwD x F ee(w) = F ss(w)(1 - e )(1 - e )+ Fnn (w )

= 2F (w ) 1- Re e- jwD x + F (w ) ss ( { }) nn s, n n Random displacement error D x, statistically independent from

F ( ) = E 2F ( ) 1 - Re e- jwD x + F ( ) ee w { ss w ( { }) nn w }

= 2F ( ) 1- Re E e- jwD x + F ( ) ss w ( { { }}) nn w

= 2Fss (w )(1- Re{P(w )})+ Fnn (w)

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 8

4 Analysis of m.c. prediction error (cont.)

- jw D n What is P(w)? P(w ) = E{e x } ¥ - jw D = p (D )e dD = F p D x ò D x { D x ( )} -¥ of the displacement error pdf!! n Same as characteristic function of displacement error, except for sign n Extension to 2-d

F ee (w x ,w y ) = 2F ss (w x ,w y )(1- Re{P(w x ,w y )})+ F nn (w x ,w y )

Fourier transform of the noise spectrum displacement error pdf power spectrum of p(D , D ) luminance signal x y

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 9

Power spectrum of motion-compensated prediction error

0 p frequency w x

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 10

5 R-D function for MCP with integer- accuracy

Minimum -rate for given SNR T n (D x , D y ) assumed uniformly distributed between 1 D = ± pel x 2 1 D = ± line ~0.7 bpp y 2 n Gaussian signal model 3 2 2 - 2 æ w x +w y ö Fss (w x ,w y ) = Aç 1+ 2 ÷ è w 0 ø

n Typical parameters for CIF resolution (352 x 288 )

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 11

Required accuracy of motion compensation

n p(D x , D y ) isotropic Gaussian pdf with variance s 2 3 - 2 2 2 æ wx +w y ö n Fss (wx ,wy ) = Aç 1+ 2 ÷ è w0 ø n Typical parameters for CIF resolution (352 x 288 pixels) n Minimum bit-rate for SNR = 30 dB

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 12

6 Model of MCP hybrid coder with loop filter

e R-D optimal intraframe luminance - encoder signal S

intraframe decoder

spatial filter F

motion compensated predictor

displacement estimate

æ dx ö æ D x ö true displacement ç ÷ + ç ÷ displacement error è d y ø è D y ø

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 13

Motion-compensated prediction error with loop filter

Motion-compensated signal Prediction error

c(x) = s(x - Dx )- n(x) e(x) = s(x)- f(x)*c(x)

= s(x)- f(x)*s(x - Dx)+ f(x)*n(x) Previous frame Impulse response of loop filter

x c(x) s(x) Current frame d Displacement x Displacement error Dx

x

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 14

7 Spatial power spectrum of m.c. prediction error with loop filter

2 2 Fee(L) = Fss(L)(1+| F(L)| -2Re{F(L)P(L)})+Fnn(L)| F(L)|

P(L) 2 -D Fourier transform of displacement error pdf F(L) 2 -D Fourier transform of f (x, y)

Fuu spatial spectral power density of signal u

L vector of spatial frequencies (w x ,w y ) n(x, y) noise

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 15

Optimum loop filter

n Wiener filter minimizes prediction error variance F (L) F (L) = P*(L) × ss opt 12 3 F L + F L 1ss4 (4 4) 2 4 nn4 (43 )

accounts for accuracy of accounts for noise n To determine Wiener filter from measurements: cross spectrum between s(x,y) and the motion-compensated signal F (L) ˆ ˆ sc c(x, y) = r(x - d x , y - d y ) Fopt (L) = F cc (L)

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 16

8 Required accuracy of motion compensation with loop filter

2 n p(D x , D y ) isotropic Gaussian pdf with variance s n Minimum bit-rate for SNR = 30 dB

~0.8 bpp

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 17

Experimental evaluation of sub-pixel motion compensation n ITU-R 601 TV signals, 13.5 MHz sampling rate, interlaced, blockwise motion compensation with blocksize16x16

Zoom Voiture

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 18

9 Influence of noise on the performance of MCP

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 19

Motion Compensation Performance in H.263

500

400 Intra mode only 300 Integer-pel accuracy [kbps] 200 1/2-pel 100 accuracy 0 24 30 36

Simulation details: PSNR [dB] Foreman, QCIF, SKIP=2 Q=4,5,7,10,15,25

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 20

10 Summary: motion-compensated coding n Motion-compensated prediction exploits similarity of successive frames of a video sequence. n Hybrid coder combines motion compensation and spatial 2D coding. n Power spectral density of motion-compensated prediction error is flat. n Loop filter improves the prediction. n Maximum gain by motion compensation with integer-pel accuracy ~0.8 bit/sample. n Sub-pixel accuracy of motion compensation can improve prediction n “Noise” in the image signal limits the accuracy of motion compensation.

Bernd Girod: EE368b Image and Video Compression Motion Compensation no. 21

11