Spatio-Temporal Frequency Analysis of Motion Blur Reduction on Lcds
Total Page:16
File Type:pdf, Size:1020Kb
SPATIO-TEMPORAL FREQUENCY ANALYSIS OF MOTION BLUR REDUCTION ON LCDS F. H. van Heesch and M.A. Klompenhouwer Philips Research Laboratories, High Tech Campus 36, Eindhoven, The Netherlands ABSTRACT and MCU [8]. In this paper, the latter two motion blur reduction meth- Liquid Crystal Displays (LCDs) are known to suffer from mo- ods are analyzed and discussed in more detail, as they are tion blur. In this paper, motion blur on LCDs is analyzed by most suitable for TV applications [9]. A frequency analysis examining the spatio-temporal spectrum of the perceived pic- of motion blur on LCDs is given in Section 2. This analysis ture. This analysis is used to examine the performance of two is used to discuss the performance of SFI, in Section 3, and motion blur reduction techniques: Smooth Frame Insertion MCU, in Section 4. Advantages and disadvantages of both (SFI) and Motion Compensated Up-conversion (MCU). The methods are compared in Section 5 and final conclusions are performance of both techniques is compared for TV applica- drawn in Section 6. tions. Index Terms — Dynamic resolution, frame rate conver- 2. MOTION BLUR sion, LCDs, motion blur, video enhancement, video process- ing. In order to appreciate how the temporal response of video re- construction on a display causes a perceived spatial attenua- I (x, t) 1. INTRODUCTION tion of high frequencies, consider a video signal c that is sampled and reconstructed, as indicated in Eq. 1 and ex- The picture quality of motion pictures, shown on a display, plained in detail in [10]. is the product of processing a light signal through a cascade Id(x, t)=(Ic(x, t) · Λ(x, t)) ∗ A(x, t), (1) of signal capturing, storing, transmission, and reconstruction. The Human Visual System (HVS), however, determines the where Id(x, t) is the displayed (or reconstructed) video sig- perceived quality of the displayed motion pictures. For op- nal, Λ(x, t) the spatio-temporal sampling grid, and A(x, t) timal perception, i.e. indistinguishable from the original, the the reconstruction aperture of the display. Furthermore, video HVS should be the limiting factor in all process stages for all containing a motion v is denoted as: quality aspects such as resolution, brightness, and colorful- ness. Im(x, t)=Ic(x + vt, t), or (2) For TV applications, spatial (static) resolution has seen a f f Im(fx,ft)=Ic (fx,ft − v· fx) (3) recent jump in performance with the introduction of HDTV and the availability of large format LCDs at consumer price in the time and frequency domain, respectively. The pursuit of levels. LCDs, however, are known to perform poorly with this motion by the HVS, known as eye-tracking, corresponds respect to temporal resolution, causing visible spatial blur to a transformation of coordinates of the reconstruction sig- whenever video contains motion [1]. This low temporal reso- nal, yielding the video signal after eye-tracking, Ie(x, t),as lution is a result of the temporal response of the display, de- indicated in Eq. 4. fined by the LC material response, the hold characteristic of I (x, t)=I (x − vt, t) the active matrix, and the backlight. e d or (4) f f In order to reduce motion blur on LCDs its temporal res- Ie (fx,ft)=Id (fx,fv), (5) olution needs to be improved. This is achieved by reducing the response time of LCDs and by reducing the hold time of with fv = ft + v· fx. Combining the above, yields Eq. 6: the display. As such, in the past, improvements have been f f f f developed in the area of LC switching behavior [2], pixel lay- Ie (fx,ft)= Ic (fx,ft) ∗ Λ (fx,fv) A (fx,fv). (6) outs, and driving schemes [3], yielding a reduction of the re- sponse time to a few milliseconds. Also several hold time It illustrates that the temporal component of the aperture, Af , reduction methods have been reported in literature including is a function of motion v. scanning backlight [4], black frame insertion [5], gray frame Ie(x, t) equals the signal on the retina. The perceived insertion [6] (also known as pseudo impulse driving), SFI [7], video signal, however, is approximated by taking into account 1-4244-1437-7/07/$20.00 ©2007 IEEE IV - 401 ICIP 2007 that the HVS is only sensitive to low temporal and spatial fre- SFI alternates a blurred and a sharpened image fast enough to If (f ) quencies. As such, the perceived motion blur can be modeled perceive its average. The blurred image equals LP x . The as the spatial response on the retina (i.e. ft =0), yielding sharpened image is chosen such that the perceived average f f Ip (fx), as indicated in Eq. 7. equals Ic (fx), i.e.: f f f f f I (fx)=I (fx), Ip (fx)=Ic (fx)A (fx,v· fx), (7) blur LP f 1 f f Ic (fx)= I (fx)+I (fx) , and (11) f f 2 blur sharp with Ip (fx) the perceived spectrum and Ic (fx) a single im- f f f f f f I (fx)=2Ic (fx) − I (fx)=I (fx)+2· I (fx). age of the video signal Ic (fx,ft). sharp LP LP HP For LCDs the spatial and temporal component of the aper- f As I (fx) is displayed only every other sub-frame, its hold- ture is assumed independent, i.e. HP time is halved and the temporal aperture equals A(x, t)=As(x)At(t). (8) 1 Th Af (f )= π f . t,HP t 2sinc 2 v (12) Furthermore, by approximating the temporal response of the LCD by a ”hold”, which is true for infinitely fast respond- The temporal aperture for low spatial frequencies and the spa- ing LC-material, the perceived spectrum on such a display is tial display apertures are not altered. described by Eq. 9, in case the HVS tracks the motion v. Combining the above yields the perceived spectrum when If (f ) applying SFI, p,SFI x , as described by Eq. 13. f f f Ip (fx)=Ic (fx)As (fx)sinc(πThv· fx), (9) 1 If (f )= If (f )+If (f ) p,SFI x 2 p,blur x p,sharp x (13) sin(x) where Th is the hold-time and sinc(x)= . 1 x = 2 · If (f )Af (f )Af (vf )+ LP x s x t,LP x This attenuation is also indicated in Fig. 1 where the spa- 2 tial dimension is indicated as a 1D signal for clarity reasons. f f f 2 · I (fx)As (fx)At, (vfx) The spatio-temporal normalized frequency spectrum of a static HP HP image is depicted in Fig. 1a. The spectrum shears in case of If (f )Af (f ) (πT vf ), |f | <f = c x s x sinc h x x c motion (recall Eq. 2) as illustrated in Fig. 1b. Temporal sam- f f Th I (fx)A (fx)sinc(π vfx), |fx| >fc pling of the motion picture yields repeats of the spectrum, c s 2 while the spatio-temporal aperture results in an attenuation as Eq. 13 shows that the attenuation caused by the temporal indicated for an “ideal hold” LCD in Fig. 1c. The perceived aperture is reduced for frequencies above the spatial cut-off spectrum, in case of eye-tracking by the HVS, is illustrated in frequency fc. Therefore, by choosing fc sufficiently close Fig. 1d. In this figure, also the limits of the HVS are sketched, to fx =0, more of the image spectrum profits from a re- assuming TV applications. Comparing Fig. 1a with Fig. 1d duced hold time. This is also seen in Fig. 2, where the spatio- shows that the perceived image differs from the original: high temporal frequency spectrum of the perceived image is illus- spatial frequencies are attenuated, hence motion blur is visi- trated for the signal of Fig. 1b after applying SFI, again as- ble, and repeats are introduced, causing detail flicker. suming eye-tracking. It shows that high spatial frequencies In the next two sections this analysis is extended for the are attenuated less compared to Fig. 1d. Fig. 2 also shows motion blur reduction methods SFI and MCU. that the low spatial frequencies of the first repeats are still significantly reduced, preventing large area flicker. Although f 3. SMOOTH FRAME INSERTION choosing c too close to zero will increase this flicker. Sofar, the limited dynamic range for Isharp(x, t) has been I (x, t)+2· I (x, t) SFI doubles the frame rate of the video and redistributes spa- ignored. In practice, LP HP requires an ex- I (x, t) tial frequencies, such that high spatial frequencies are dis- tended dynamic range compared to c . For the upper ex- played for one of the two sub-frames, while low spatial fre- tension, this would be physically feasible by reserving head- quencies are displayed at both sub-frames. Therefore, SFI room, but the lower extension would require negative light I (x, t) alternates sharp and blurred images. This is done such that intensities. As such, sharp needs to be clipped. This re- the average of the sub-frame pair is identical to the original sults in a reduced contrast or a limited motion blur reduction. frame. In Section 5, the advantages and disadvantages of SFI will be To comprehend that this frequency split reduces motion compared with MCU. blur, let us investigate the Fourier transform of the perceived f image. Assume Ic (fx) is split at a spatial frequency fc into 4. MOTION COMPENSATED UP-CONVERSION low (LP) and high (HP) spatial frequencies, such that: In order to reduce motion blur, the hold time must be reduced.