A Review and Comparison on Different Video Deinterlacing

International Journal of Research ISSN NO:2236-6124

A Review and Comparison on Different Video

Deinterlacing Methodologies

1Boyapati Bharathidevi,2Kurangi Mary Sujana,3Ashok kumar Balijepalli 1,2,3 Asst.Professor,Universal College of Engg & Technology,Perecherla,Guntur,AP,India-522438 [email protected],[email protected],[email protected]

Abstract— Video deinterlacing is a key technique in Interlaced videos are generally preferred in video broadcast digital video processing, particularly with the widespread and transmission systems as they reduce the amount of data to usage of LCD and plasma TVs. Interlacing is a widely used be broadcast. Transmission of interlaced videos was widely technique, for television broadcast and video recording, to popular in various television broadcasting systems such as double the perceived frame rate without increasing the NTSC [2], PAL [3], SECAM. Many broadcasting agencies bandwidth. But it presents annoying visual artifacts, such as made huge profits with interlaced videos. Video acquiring flickering and silhouette "serration," during the playback. systems on many occasions naturally acquire interlaced video Existing state-of-the-art deinterlacing methods either ignore and since this also proved be an efficient way, the popularity the temporal information to provide real-time performance of interlaced videos escalated. but lower visual quality, or estimate the motion for better deinterlacing but with a trade-off of higher computational cost. The question `to interlace or not to interlace' divides the TV and the PC communities. A proper answer requires a common understanding of what is possible nowadays in deinterlacing video signals. This paper outlines the most relevant methods, and provides a relative comparison.

Keywords: Visual Quality, Frame rate, Video Signals, bandwidth, Interlacing, Deinterlacing.

I.INTRODUCTION

Interlacing is a video system dating from the 1940’s when various television systems were defined, all of which were naturally analog. The camera sweeps the uneven numbered lines first, generating the first field and then sweeps the even numbered lines to generate the second field. As a consequence each image frame is built from two fields that do not occupy the same instant of time. Various artifacts can appear in the image due to the sampling structure of interlaced systems. For Fig 1: The Basic Deinterlacing method. example, aliasing can occur in the presence of high vertical frequencies and a specific blinking known as flicker can Interlaced video however, complicates many tasks pertaining appear in the image. Figure 1 shows a graphical representation to image processing and has its advantages as well as of the sampling structure of an interlaced system. As early as disadvantages [4]. As scanning format conversion was 1935, it was well known that the human visual system is more necessary for international TV broadcasting, the first sensitive to large-area flicker than flickering detail [1]. proposals of deinterlacing were considered only for Television broadcasters in the early years of media boom used international programme exchange. With the advent of this idea to their advantage by transmitting interlaced videos modern technology such as high-definition television that reduced bandwidth usage. Interlaced videos are videos (HDTV), video streaming, and DTH the need for a stable and (that are rectangularly sampled) scanned in such a way that on a standard conversion between formats is increasing. Modern any given frame of ‘N’ rows, only ‘N/2’ alternate rows are display systems like LED and plasma displays work on scanned. The remaining rows are scanned on the next frame. progressive video. Progressive video unlike interlaced video

Volume VIII, Issue V, MAY/2019 Page No:2506 International Journal of Research ISSN NO:2236-6124

contain the entire frame in the videos’ original resolution. The Bob is the simpler of the two. Each frame of interlaced video widespread usage of displays that require progressive videos has only one half the lines. For example, the odd lines (1,3,5, has made deinterlacing an important process in the video … 479) would have pixels, and the even lines (2,4,6, … 480) processing arena. With the advent of the PC era, Haan and are blank. On the following frame, the even lines have pixels, Bellers raised the question of whether to interlace or not to but the odd lines are blank. The simplest bob interlacing is to interlace [5]. While the PC community believes that the just copy the pixels from the line above for blank even lines present-day technologies are powerful enough to produce, (copy line 1 to line 2), and copy the pixels from the line below transmit and display progressive video, the television for blank odd lines (copy line 2 to line 1). Another method community believes that it is advantageous to have interlaced would be to interpolate between the two adjacent lines to fill videos in service. The main argument put forth by the PC in a blank line. Both of these methods are shown in Fig. 2. community is that interlacing a video introduces a trade-off between vertical resolution and the time resolution. In the 1997 WinHec conference the cofounder of Microsoft Corporation, Bill Gates put forth a proposal to stabilize the picture rate of PCs to 60 Hertz progressive where it stands till date [6]. In [5], the authors studied alternate options to deinterlacing. Most importantly the question of whether present day technologies are powerful enough to deinterlace satisfactorily. The support towards an all progressive system is increasing as experiments show that an all progressive systems produces at least as good an image quality as an all interlaced system does [7]. This means that an interlaced system cannot produce any increase in quality but only an increase in performance. With the reasoning that the modern day technologies can support powerful deinterlacers, a combination of interlaced and deinterlaced systems are currently in use. In such systems, for broadcasting and Figure 2. Bob verses weave deinterlacing. transmission purposes, an interlaced system in used. The receiver has an in-built deinterlacer pre-display unit that This method can cause blurring of images, because the vertical deinterlaces the interlaced video. This also satisfied the TV resolution has been effectively halved. Weave deinterlacing community by removing the requirement for a new broadcast creates a full frame from the separate interlaced frames with protocol for progressive video. However, this now applies odd and even lines. It then copies this frame twice, to achieve additional pressure on the video processing community to the 60 fps rate. This method tends to work only if there is little come up with deinterlacing systems that are not only change in the odd and even interlaced frames, meaning there is computationally efficient to support real-time video little motion in the video. As the odd and even frame pixels deinterlacing but are also powerful enough in terms of belong to different instances in time (1/60th of a second deinterlacing performance. Deinterlacing is the task of difference), rapid motion can result in jagged edges in the converting a naturally or an artificially interlaced video into a images rather than smooth lines. This is shown in Fig.3. progressive video. Simply put, deinterlacing is the task of converting fields into frames. This process of deinterlacing is shown in Figure 1-2. The implementation methodology discussed in Figure 1-2 is the general process of deinterlacing, though the process of deinterlacing depends on the type of interlacing. It is reasonable to assume from deinterlacing literature that a video is interlaced while acquisition, owing to the virtue of the data acquisition system. In this assumption, videos can be considered to be interlaced in two ways.

II.DEINTERLACING

An interlaced video stream is usually converted to progressive for image processing, as well as to for display on nearly all computer monitors. Deinterlacing must be viewed as interpolation, for the result is twice the video bandwidth. There are several methods available for deinterlacing, which Figure 3. Deinterlacing effects. can result in different video qualities under different circumstances. The two basic methods are known as “bob” Both of these methods have drawbacks. A better method, and “weave.” which requires more sophisticated video processing, is to use motion adaptive deinterlacing.

Volume VIII, Issue V, MAY/2019 Page No:2507 International Journal of Research ISSN NO:2236-6124

Where there is motion on the image, the bob technique works The advantage of this algorithm is that it achieves the highest better, and slight blurring is not easily seen. In still areas of the resolution when there is little movement in the video. image, the weave method will result in crisper images. A However when there are large movements the field replication motion adaptive deinterlacer scans the whole image and method can generate artifacts that are highly visible. detects areas of motion, by comparing to previous frames. It will use the bob method in these areas of the frame and use the 3.2 Spatial Interpolation Deinterlacing weave method on the remaining areas of the frame. In this way, interlaced video can be converted to progressive with Another basic member of the deinterlacing algorithm family is little loss of quality. spatial interpolation, which consists of using the lines present in the field currently being displayed to estimate the missing III.LITERATURE SURVEY lines. The simplest of these algorithms is known as Line Average (LA). One descendant of Line Average that merits Deinterlacing is a well-studied topic. The last century has seen attention is an algorithm that performs a search for the borders a lot of deinterlacing algorithms being proposed. The literature of the image to perform spatial interpolation, Edge Line of deinterlacing can be studied under two broad categories, Average or ELA. These algorithms reduce the stepped effect Linear and Non-Linear deinterlacer. Deinterlacing requires the at the borders of the image and offer the best results for the display to buffer one or more fields and recombine them into spatial algorithms family. The greatest advantage of spatial full frames. In theory this would be as simple as capturing one algorithms is the lack of visible artifacts during movement in field and combining it with the next field to be received, the video, however their principal disadvantage is that because producing a single frame. However, the originally recorded they only use information from the field currently being signal was produced as a series of fields, and any motion of displayed they reduce the vertical resolution. the subjects during the short period between the fields is encoded into the display. When combined into a single frame, 3.3 Comparison of Spatial and Temporal Deinterlacing the slight differences between the two fields due to this motion results in a "combing" effect where alternate lines are slightly Figure 4 shows the results of temporal deinterlacing via displaced from each other. There are various methods to replication of the previous field and using Edge Line Average deinterlace video, each producing different problems or spatial deinterlacing side-by-side. As mentioned previously artifacts of its own. Some methods are much cleaner in the main problem with the previous field replication algorithm artifacts than other methods. Most deinterlacing techniques is the appearance of visible artifacts on moving objects, as can be broken up into three different groups all using their seen in the blue rectangle. On the other hand, when using own exact techniques. The first group are called field spatial interpolation deinterlacing a loss in vertical resolutions combination deinterlacers, because they take the even and odd results; this effect is especially noticeable in the eye, the fields and combine them into one frame which is then eyebrow and the definition of the hair. displayed. The second group are called field extension deinterlacers, because each field (with only half the lines) is extended to the entire screen to make a frame. The third type uses a combination of both and falls under the banner of motion compensation and a number of other names.Modern deinterlacing systems therefore buffer several fields and use techniques like edge detection in an attempt to find the motion between the fields. This is then used to interpolate the missing lines from the original field, reducing the combing effect.Deinterlacing consists of estimating the number of missing lines in each field of an interlaced video signal in order to produce a progressive scan signal. A great variety of algorithms exist to perform this process with varying levels of complexity[7]. This document will explore and compare deinterlacing algorithms based on temporal interpolation, spatial interpolation, Motion Adaptive and Motion Compensation techniques.

3.1 Temporal Interpolation Deinterlacing Figure 4: Comparison of temporal deinterlacing (top) and One of the simplest families of deinterlacing algorithms is that spatial deinterlacing (bottom). of temporal interpolation, based upon the information from adjacent frames to estimate the missing lines. The simplest of these methods is replication of the previous field.

3.4 Deinterlacing with Motion Adaptive Algorithms Volume VIII, Issue V, MAY/2019 Page No:2508 International Journal of Research ISSN NO:2236-6124

Since spatial algorithms are better for images with movement and temporal algorithms are better for static images, deinterlacing methods have evolved to combine the best of both types, giving rise to so-called Motion Adaptive algorithms.

Figure 6: Comparison of Motion Adaptive deinterlacing

(above) and Motion Compensation deinterlacing (below). Figure 5: Deinterlacing using a Motion Adaptive algorithm. This family of algorithms function exceptionally well when movement in the image is by only a few pixels and the movement vectors are well estimated, especially for textured Motion Adaptive deinterlacing detects the level of variation images with high vertical resolutions. However visible between the pixels of consecutive fields and performs a artifacts can appear if the movement vector calculations combination of spatial interpolation and temporal interpolation contain errors and the movement in the image is by a large based upon the amount of pixel variation observed. This number of pixels. For this reason the use of Motion family of algorithms is currently the most wide-spread. Figure Compensation algorithms needs to be combined with other 5 shows the results of deinterlacing the same video sequence deinterlacing methods so that erroneous calculation of with the static face and the blue rectangular moving across it movement vectors can be detected and combined with other in a diagonal pattern using a Motion Adaptive algorithm. No more precise estimation methods to avoid visible artifacts. artifacts are visible on the borders of the blue rectangle, and yet the resolution of the static face image is similar to that of 3.5 Comparison between Motion Adaptive and Motion the field replication temporal interpolation algorithm. Compensation Algorithms

The main problem with this algorithm is the loss of Figures 6 show the differences between deinterlacing with a vertical resolution of any moving objects in the video, Motion Adaptive algorithm and deinterlacing with a robust however this loss is lower than it would be with pure spatial method combining Motion Compensation with an adaptive interpolation because the loss only occurs in areas with algorithm. In both cases the video sequences were recorded movement. The result is a barely perceptible reduction in with the camera in motion. Each of the two comparisons resolution for the human eye during rapid movement, although demonstrate that the Motion Compensation algorithm achieves with relatively slow movements this loss is more noticeable. greater resolution in the deinterlaced frame. The river-and- Such slow movements can be produced when an object passes house sequence displays higher resolution, especially in the in front of a still background or during camera movements windows of the home, the leaves of the tree on the left and the such as pan and zoom, etc. oblique lines on the roof, where reduced aliasing is also observed. 3.5 Deinterlacing with Motion Compensation Algorithms IV.CONCLUSION Another family of algorithms that also merit attention are the Motion Compensation group. These algorithms attempt to Deinterlacing algorithms based on treatment of the entire estimate the movement between two consecutive fields by frame in the same manner regardless of the motion present in Calculating motion vectors. This is done by performing the video such as spatial interpolation and temporal temporal interpolation on the regions of two consecutive fields interpolation are relatively simple, but the results are lacking that are mostly alike. Searching for the regions with minimal when compared to methods that consider motion in the image change between consecutive fields requires significant to apply corrections. computing power.

Volume VIII, Issue V, MAY/2019 Page No:2509 International Journal of Research ISSN NO:2236-6124

Algorithms that analyze motion, whether partial or occupying the entire image are more efficient even though they require more mathematical computation and require faster and more powerful processors. Within this family of algorithms one group consists of the Motion Adaptive type, which applies either spatial or temporal interpolation to those areas of the video image that require one or the other, thereby adapting to any motion present in a given video sequence. The other group of algorithms is the more advanced Motion Compensation type, which requires higher computational power to achieve higher resolution in the final results.

REFERENCES

[1] E.W. Engstrom, `A study of Television Image Characteristics. Part Two. Determination of Frame Frequency for Television in terms of Flicker Characteristics', Proc. of the I.R.E., Vol. 23, no.4, 1935, pp. 295-310. [2] S. Pigeon and P. Guillotel, `Advantages and Drawbacks of Interlaced and Progressive Scanning Formats', HAMLET report, RACE 2110, 1996. [3] E. Dubois, G. de Haan and T Kurita, `Motion estimation and compensation technologies for standards conversion', Preface to special issue of Signal Processing: Image Communication, no. 6, 1994,pp.189-192. [4] G. de Haan, J. Kettenis, and B. De Loore, `IC for motion- compensated 100 Hz TV with natural motion movie-mode', IEEE Tr. on Cons. Electr.,May 1996, pp. 165-174. [5] Microsoft, 'Broadcast-Enabled Computer Hardware Requirements', WinHEC'97, Broadcast Techologies White Paper, 1997, pp. 11-12. [6] L.J. Thorpe and T. Hanabusa, `If Progressive Scanning is so good, how bad is Interlace?',SMPTE Journal, Dec. 1990, pp. 972- 986. [7] Deinterlacing Algorithms By Rebeca Sanz Serrano, 2016, Albalá Ingenieros S.A.

[8] A. Van den Enden and N. Verhoeckx, Discrete-time Signal Processing, Prentice Hall, 1989, pp. 223-. [9] G. Tonge, "Television motion potrayal," in Les Assises des Jeunes Chercheurs, Reennes(Fr.), 1985. [10] A. Tekalp, Digital Video Processing, Prentice hall, 1995, pp. 250-.. [11] M. Weston, Interpolating lines of video signals, US-patent 4,789,893, December 1988. [12] P. Guillotel and G. D'Agostino, "Towards the use of a progressive trasmission format," in International workshop on HDTV and the evolution of television, Taipei, 1995. [13] P. Anandan, J. Bergen, K. Hanna and R. Hingorani, "Hierarchial model-based motion estimation," Motion analysis and Image sequence processing, 1993. [14] C. Zwart and D. H. Frakes, "Soft adaptive gradient angle interpolation of grayscale images," in IEEE International conference on Acoustics, Speech and Signal Processing (ICASSP), 2012.

Volume VIII, Issue V, MAY/2019 Page No:2510