DOCSLIB.ORG

An Improved Algorithm for Deinterlacing Video Streams

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
An Improved Algorithm for Deinterlacing Video Streams UNLV Retrospective Theses & Dissertations 1-1-2006 An improved algorithm for deinterlacing video streams Christopher Weiss University of Nevada, Las Vegas Follow this and additional works at: https://digitalscholarship.unlv.edu/rtds Repository Citation Weiss, Christopher, "An improved algorithm for deinterlacing video streams" (2006). UNLV Retrospective Theses & Dissertations. 2082. http://dx.doi.org/10.25669/f2rv-104d This Thesis is protected by copyright and/or related rights. It has been brought to you by Digital Scholarship@UNLV with permission from the rights-holder(s). You are free to use this Thesis in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s) directly, unless additional rights are indicated by a Creative Commons license in the record and/ or on the work itself. This Thesis has been accepted for inclusion in UNLV Retrospective Theses & Dissertations by an authorized administrator of Digital Scholarship@UNLV. For more information, please contact [email protected]. AN IMPROVED ALGORITHM FOR DEINTERLACING VIDEO STREAMS by Christopher Weiss Bachelor of Science University of Nevada, Las Vegas 1997 A thesis submitted in partial fulfillment of the requirements for the Master of Science Degree in Computer Science School of Computer Science Howard R. Hughes College of Engineering Graduate College University of Nevada, Las Vegas December 2006 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UMI Number: 1441739 INFORMATION TO USERS The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleed-through, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. UMI UMI Microform 1441739 Copyright 2007 by ProQuest Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P.O. Box 1346 Ann Arbor, Ml 48106-1346 Reproduced witfi permission of tfie copyrigfit owner. Furtfier reproduction profiibited witfiout permission. Thesis Approval The Graduate College University of Nevada, Las Vegas November 30 . 2q06 The Thesis prepared by Christopher Allen Weiss Entitled An Improved Algorithm for Deinterlacing Video Streams is approved in partial fulfillment of the requirements for the degree of Master of Science in Computer Science Examination Committee Chair y I ' Dean of the Graduate College Emmination Committee Member Examination Committee Member Graduate College Faculty Representative 11 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE OF CONTENTS ABSTRACT...........................................................................................................................................iv LIST OF FIGURES..............................................................................................................................v ACKNOWLEDGEMENTS.............................................................................................................. vi CHAPTER 1 INTRODUCTION .................................................................................................... 1 Problems Arise With Interlacing............................................................................................... 3 CHAPTER 2 METHODS OF MEASURE..................................................................................5 CHAPTER 3 REVIEW OF PREVIOUS METHODS.............................................................. 8 Line Averaging............................................................................................................................. 10 Field Insertion............................................................................................................................... II Linear Vertical-Temporal Filtering..........................................................................................13 Vertical-Temporal Median Filtering........................................................................................14 Edge-Based Line Averaging....................................................................................................16 Motion-Compensated Vertical-Temporal Median Filtering..............................................19 CHAPTER 4 NEW METHOD ..................................................................................................... 22 Video Object Planes......................................................................................................................22 Difference Image Creation..........................................................................................................24 Canny Edge Detection.................................................................................................................26 Video Object Plane Bounding Box Extraction.................................................................... 31 Video Object Plane Based Deinterlacing................................................................................ 33 CHAPTER 5 COMPARATIVE RESULTS...................................... 39 CHAPTER 6 SUMMARY AND CONCLUSIONS ............................................................... 42 APPENDIX SOURCE CODE .................................................................................on CD-ROM REFERENCES....................................................................................................................................44 VITA ...................................................................................................................................................... 46 m Reproduced witti permission of ttie copyrigfit owner. Furtfier reproduction profiibited witfiout permission. ABSTRACT An Improved Algorithm For Deinterlacing Video Streams by Christopher Weiss Dr. Evangelos Yfantis. Examination Committee Chair Professor of Computer Seience University of Nevada, Las Vegas The MPEG-4 standard for computerized video incorporates the concept of a video object plane. While in the simplest case this can be the full rectangular frame, the standard supports a hierarchical set of arbitrary shaped planes, one for each content- sensitive video object. Herein is proposed a method for extracting arbitrary planes from video that does not already contain video object plane information. Deinterlacing is the process of taking two video fields, each at half the height of the finalized image frame, and combining them into that finalized frame. As the fields are not captured simultaneously, temporal artifacts may result. Herein is proposed a method to use the above mentioned video object planes to calculate the intra-field motion of objects in the video stream and correct for such motion leading to a higher quality deinterlaced output. IV Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. LIST OF FIGURES Eigure I General deinterlacing function....................................................................................... 8 Figure 2 Linear (non-motion compensation) deinterlacing..................................................... 9 Figure 3 Line-averaged deinterlaced frame.................................................................... 10 Figure 4 Artifacts introduced by field insertion........................................................................ 12 Figure 5 Vertical-temporal filtering weights...........................................................................13 Figure 6 Vertical-temporal filtered image ................................................................................. 14 Figure 7 Vertical-temporal median filter..................................................................................15 Figure 8 Vertical-temporal 3-tap median filtering....................................................................16 Figure 9 Edge line averaging edge detection........................................................................17 Figure 10 Edge line averaging pixel interpolation...................................................................17 Figure 11 Edge line averaged image......................................................................................... 18 Figure 12 Block-based sum of absolute error ...........................................................................19 Figure 13 M otion compensating vertical-temporal median filter........................................20 Figure 14 Motion compensated vertical-temporal median filtered frame........................ 21 Figure 15 Difference image function..........................................................................................24 Figure 16 Even field (n).................................................................................................................25 Figure 17 The next even field (n+2)...........................................................................................25 Figure 18 Difference image of fields n and n+2 (border added).........................................25
Load more

Recommended publications
  • Problems on Video Coding
    Problems on Video Coding Guan-Ju Peng Graduate Institute of Electronics Engineering, National Taiwan University 1 Problem 1 How to display digital video designed for TV industry on a computer screen with best quality? .. Hint: computer display is 4:3, 1280x1024, 72 Hz, pro- gressive. Supposing the digital TV display is 4:3 640x480, 30Hz, Interlaced. The problem becomes how to convert the video signal from 4:3 640x480, 60Hz, Interlaced to 4:3, 1280x1024, 72 Hz, progressive. First, we convert the video stream from interlaced video to progressive signal by a suitable de-interlacing algorithm. Thus, we introduce the de-interlacing algorithms ¯rst in the following. 1.1 De-Interlacing Generally speaking, there are three kinds of methods to perform de-interlacing. 1.1.1 Field Combination Deinterlacing ² Weaving is done by adding consecutive ¯elds together. This is ¯ne when the image hasn't changed between ¯elds, but any change will result in artifacts known as "combing", when the pixels in one frame do not line up with the pixels in the other, forming a jagged edge. This technique retains full vertical resolution at the expense of half the temporal resolution. ² Blending is done by blending, or averaging consecutive ¯elds to be displayed as one frame. Combing is avoided because both of the images are on top of each other. This instead leaves an artifact known as ghosting. The image loses vertical resolution and 1 temporal resolution. This is often combined with a vertical resize so that the output has no numerical loss in vertical resolution. The problem with this is that there is a quality loss, because the image has been downsized then upsized.
    [Show full text]
  • Cathode-Ray Tube Displays for Medical Imaging
    DIGITAL IMAGING BASICS Cathode-Ray Tube Displays for Medical Imaging Peter A. Keller This paper will discuss the principles of cathode-ray crease the velocity of the electron beam for tube displays in medical imaging and the parameters increased light output from the screen; essential to the selection of displays for specific 4. a focusing section to bring the electron requirements. A discussion of cathode-ray tube fun- beam to a sharp focus at the screen; damentals and medical requirements is included. 9 1990bu W.B. Saunders Company. 5. a deflection system to position the electron beam to a desired location on the screen or KEY WORDS: displays, cathode ray tube, medical scan the beam in a repetitive pattern; and irnaging, high resolution. 6. a phosphor screen to convert the invisible electron beam to visible light. he cathode-ray tube (CRT) is the heart of The assembly of electrodes or elements mounted T almost every medical display and its single within the neck of the CRT is commonly known most costly component. Brightness, resolution, as the "electron gun" (Fig 2). This is a good color, contrast, life, cost, and viewer comfort are analogy, because it is the function of the electron gun to "shoot" a beam of electrons toward the all strongly influenced by the selection of a screen or target. The velocity of the electron particular CRT by the display designer. These beam is a function of the overall accelerating factors are especially important for displays used voltage applied to the tube. For a CRT operating for medical diagnosis in which patient safety and at an accelerating voltage of 20,000 V, the comfort hinge on the ability of the display to electron velocity at the screen is about present easily readable, high-resolution images 250,000,000 mph, or about 37% of the velocity of accurately and rapidly.
    [Show full text]
  • Viarte Remastering of SD to HD/UHD & HDR Guide
    Page 1/3 Viarte SDR-to-HDR Up-conversion & Digital Remastering of SD/HD to HD/UHD Services 1. Introduction As trends move rapidly towards online content distribution and bigger and brighter progressive UHD/HDR displays, the need for high quality remastering of SD/HD and SDR to HDR up-conversion of valuable SD/HD/UHD assets becomes more relevant than ever. Various technical issues inherited in legacy content hinder the immersive viewing experience one might expect from these new HDR display technologies. In particular, interlaced content need to be properly deinterlaced, and frame rate converted in order to accommodate OTT or Blu-ray re-distribution. Equally important, film grain or various noise conditions need to be addressed, so as to avoid noise being further magnified during edge-enhanced upscaling, and to avoid further perturbing any future SDR to HDR up-conversion. Film grain should no longer be regarded as an aesthetic enhancement, but rather as a costly nuisance, as it not only degrades the viewing experience, especially on brighter HDR displays, but also significantly increases HEVC/H.264 compressed bit-rates, thereby increases online distribution and storage costs. 2. Digital Remastering and SDR to HDR Up-Conversion Process There are several steps required for a high quality SD/HD to HD/UHD remastering project. The very first step may be tape scan. The digital master forms the baseline for all further quality assessment. isovideo's SD/HD to HD/UHD digital remastering services use our proprietary, state-of-the-art award- winning Viarte technology. Viarte's proprietary motion processing technology is the best available.
    [Show full text]
  • BA(Prog)III Yr 14/04/2020 Displays Interlacing and Progressive Scan
    BA(prog)III yr 14/04/2020 Displays • Colored phosphors on a cathode ray tube (CRT) screen glow red, green, or blue when they are energized by an electron beam. • The intensity of the beam varies as it moves across the screen, some colors glow brighter than others. • Finely tuned magnets around the picture tube aim the electrons onto the phosphor screen, while the intensity of the beamis varied according to the video signal. This is why you needed to keep speakers (which have strong magnets in them) away from a CRT screen. • A strong external magnetic field can skew the electron beam to one area of the screen and sometimes caused a permanent blotch that cannot be fixed by degaussing—an electronic process that readjusts the magnets that guide the electrons. • If a computer displays a still image or words onto a CRT for a long time without changing, the phosphors will permanently change, and the image or words can become visible, even when the CRT is powered down. Screen savers were invented to prevent this from happening. • Flat screen displays are all-digital, using either liquid crystal display (LCD) or plasma technologies, and have replaced CRTs for computer use. • Some professional video producers and studios prefer CRTs to flat screen displays, claiming colors are brighter and more accurately reproduced. • Full integration of digital video in cameras and on computers eliminates the analog television form of video, from both the multimedia production and the delivery platform. • If your video camera generates a digital output signal, you can record your video direct-to-disk, where it is ready for editing.
    [Show full text]
  • A Review and Comparison on Different Video Deinterlacing
    International Journal of Research ISSN NO:2236-6124 A Review and Comparison on Different Video Deinterlacing Methodologies 1Boyapati Bharathidevi,2Kurangi Mary Sujana,3Ashok kumar Balijepalli 1,2,3 Asst.Professor,Universal College of Engg & Technology,Perecherla,Guntur,AP,India-522438 [email protected],[email protected],[email protected] Abstract— Video deinterlacing is a key technique in Interlaced videos are generally preferred in video broadcast digital video processing, particularly with the widespread and transmission systems as they reduce the amount of data to usage of LCD and plasma TVs. Interlacing is a widely used be broadcast. Transmission of interlaced videos was widely technique, for television broadcast and video recording, to popular in various television broadcasting systems such as double the perceived frame rate without increasing the NTSC [2], PAL [3], SECAM. Many broadcasting agencies bandwidth. But it presents annoying visual artifacts, such as made huge profits with interlaced videos. Video acquiring flickering and silhouette "serration," during the playback. systems on many occasions naturally acquire interlaced video Existing state-of-the-art deinterlacing methods either ignore and since this also proved be an efficient way, the popularity the temporal information to provide real-time performance of interlaced videos escalated. but lower visual quality, or estimate the motion for better deinterlacing but with a trade-off of higher computational cost. The question `to interlace or not to interlace' divides the TV and the PC communities. A proper answer requires a common understanding of what is possible nowadays in deinterlacing video signals. This paper outlines the most relevant methods, and provides a relative comparison.
    [Show full text]
  • EBU Tech 3315-2006 Archiving: Experiences with TK Transfer to Digital
    EBU – TECH 3315 Archiving: Experiences with telecine transfer of film to digital formats Source: P/HDTP Status: Report Geneva April 2006 1 Page intentionally left blank. This document is paginated for recto-verso printing Tech 3315 Archiving: Experiences with telecine transfer of film to digital formats Contents Introduction ......................................................................................................... 5 Decisions on Scanning Format .................................................................................... 5 Scanning tests ....................................................................................................... 6 The Results .......................................................................................................... 7 Observations of the influence of elements of film by specialists ........................................ 7 Observations on the results of the formal evaluations .................................................... 7 Overall conclusions .............................................................................................. 7 APPENDIX : Details of the Tests and Results ................................................................... 9 3 Archiving: Experiences with telecine transfer of film to digital formats Tech 3315 Page intentionally left blank. This document is paginated for recto-verso printing 4 Tech 3315 Archiving: Experiences with telecine transfer of film to digital formats Archiving: Experience with telecine transfer of film to digital formats
    [Show full text]
  • Motion Compensated Deinterlacer Analysis and Implementation
    2008:126 CIV MASTER'S THESIS Motion Compensated Deinterlacer Analysis and Implementation Johan Helander Luleå University of Technology MSc Programmes in Engineering Arena, Media, Music and Technology Department of Computer Science and Electrical Engineering Division of Signal Processing 2008:126 CIV - ISSN: 1402-1617 - ISRN: LTU-EX--08/126--SE Master’s Thesis Supervisor: Magnus Hoem Examiner: Magnus Lundberg Nordenvaad Telestream AB & Department of Computer Science and Electrical Engineering, Signal Processing Group, Luleå University of Technology Preface This Master’s Thesis was carried out by me during the autumn term 2007 and beginning of 2008 at Telestream AB’s office at Rådmansgatan 49, Stockholm. It is part of the Master of Science program Arena Media, Music and Technology at Luleå University of Technology (LTU). Because of my education and interest in signal processing in media applications, the proposed topic was very well suited. The reader is assumed having basic knowledge about signal processing, such as sampling, quantization, aliasing and so on. I would like to thank Telestream AB for their warm welcome and comfortable treatment during this period. I would especially like to thank Magnus Hoem, CEO Telestream AB, for the opportunity to carry this thesis through, Nils Andgren, Telestream AB, for help and support through important thoughts and discussions, Kennet Eriksson, Telestream AB, for supplying test video sequences. Finally, I would like to thank Maria Andersson, for great support by illustration of the majority of the figures contained in this Master’s Thesis. i ii Abstract In the early days of television as Cathode Ray Tube (CRT) screens became brighter, the level of flicker caused by progressive scanning became more noticeable.
    [Show full text]
  • Video Processor, Video Upconversion & Signal Switching
    81LumagenReprint 3/1/04 1:01 PM Page 1 Equipment Review Lumagen VisionPro™ Video Processor, Video Upconversion & Signal Switching G REG R OGERS Lumagen VisionPro™ Reviewer’s Choice The Lumagen VisionPro™ Video Processor is the type of product that I like to review. First and foremost, it delivers excep- tional performance. Second, it’s an out- standing value. It provides extremely flexi- ble scaling functions and valuable input switching that isn’t included on more expen- sive processors. Third, it works as adver- tised, without frustrating bugs or design errors that compromise video quality or ren- Specifications: der important features inoperable. Inputs: Eight Programmable Inputs (BNC); Composite (Up To 8), S-Video (Up To 8), Manufactured In The U.S.A. By: Component (Up To 4), Pass-Through (Up To 2), “...blends outstanding picture SDI (Optional) Lumagen, Inc. Outputs: YPbPr/RGB (BNC) 15075 SW Koll Parkway, Suite A quality with extremely Video Processing: 3:2 & 2:2 Pulldown Beaverton, Oregon 97006 Reconstruction, Per-Pixel Motion-Adaptive Video Tel: 866 888 3330 flexible scaling functions...” Deinterlacing, Detail-Enhancing Resolution www.lumagen.com Scaling Output Resolutions: 480p To 1080p In Scan Line Product Overview Increments, Plus 1080i Dimensions (WHD Inches): 17 x 3-1/2 x 10-1/4 Price: $1,895; SDI Input Option, $400 The VisionPro ($1,895) provides two important video functions—upconversion and source switching. The versatile video processing algorithms deliver extensive more to upconversion than scaling. Analog rithms to enhance edge sharpness while control over input and output formats. Video source signals must be digitized, and stan- virtually eliminating edge-outlining artifacts.
    [Show full text]
  • High-Quality Spatial Interpolation of Interlaced Video
    High-Quality Spatial Interpolation of Interlaced Video Alexey Lukin Laboratory of Mathematical Methods of Image Processing Department of Computational Mathematics and Cybernetics Moscow State University, Moscow, Russia [email protected] Abstract Deinterlacing is the process of converting of interlaced-scan video sequences into progressive scan format. It involves interpolating missing lines of video data. This paper presents a new algorithm of spatial interpolation that can be used as a part of more com- plex motion-adaptive or motion-compensated deinterlacing. It is based on edge-directional interpolation, but adds several features to improve quality and robustness: spatial averaging of directional derivatives, ”soft” mixing of interpolation directions, and use of several interpolation iterations. High quality of the proposed algo- rithm is demonstrated by visual comparison and PSNR measure- ments. Keywords: deinterlacing, edge-directional interpolation, intra- field interpolation. 1 INTRODUCTION (a) (b) Interlaced scan (or interlacing) is a technique invented in 1930-ies to improve smoothness of motion in video without increasing the bandwidth. It separates a video frame into 2 fields consisting of Figure 1: a: ”Bob” deinterlacing (line averaging), odd and even raster lines. Fields are updated on a screen in alter- b: ”weave” deinterlacing (field insertion). nating manner, which permits updating them twice as fast as when progressive scan is used, allowing capturing motion twice as often. Interlaced scan is still used in most television systems, including able, it is estimated from the video sequence. certain HDTV broadcast standards. In this paper, a new high-quality method of spatial interpolation However, many television and computer displays nowadays are of video frames in suggested.
    [Show full text]
  • Exposing Digital Forgeries in Interlaced and De-Interlaced Video Weihong Wang, Student Member, IEEE, and Hany Farid, Member, IEEE
    1 Exposing Digital Forgeries in Interlaced and De-Interlaced Video Weihong Wang, Student Member, IEEE, and Hany Farid, Member, IEEE Abstract With the advent of high-quality digital video cameras and sophisticated video editing software, it is becoming increasingly easier to tamper with digital video. A growing number of video surveillance cameras are also giving rise to an enormous amount of video data. The ability to ensure the integrity and authenticity of this data poses considerable challenges. We describe two techniques for detecting traces of tampering in de-interlaced and interlaced video. For de-interlaced video, we quantify the correlations introduced by the camera or software de-interlacing algorithms and show how tampering can disturb these correlations. For interlaced video, we show that the motion between fields of a single frame and across fields of neighboring frames should be equal. We propose an efficient way to measure these motions and show how tampering can disturb this relationship. I. INTRODUCTION Popular websites such as YouTube have given rise to a proliferation of digital video. Combined with increasingly sophisticated users equipped with cellphones and digital cameras that can record video, the Internet is awash with digital video. When coupled with sophisticated video editing software, we have begun to see an increase in the number and quality of doctored video. While the past few years has seen considerable progress in the area of digital image forensics (e.g., [1], [2], [3], [4], [5], [6], [7]), less attention has been payed to digital video forensics. In one of the only papers in this area, we previously described a technique for detecting double MPEG compression [8].
    [Show full text]
  • Deinterlacing Network for Early Interlaced Videos
    1 Rethinking deinterlacing for early interlaced videos Yang Zhao, Wei Jia, Ronggang Wang Abstract—In recent years, high-definition restoration of early videos have received much attention. Real-world interlaced videos usually contain various degradations mixed with interlacing artifacts, such as noises and compression artifacts. Unfortunately, traditional deinterlacing methods only focus on the inverse process of interlacing scanning, and cannot remove these complex and complicated artifacts. Hence, this paper proposes an image deinterlacing network (DIN), which is specifically designed for joint removal of interlacing mixed with other artifacts. The DIN is composed of two stages, i.e., a cooperative vertical interpolation stage for splitting and fully using the information of adjacent fields, and a field-merging stage to perceive movements and suppress ghost artifacts. Experimental results demonstrate the effectiveness of the proposed DIN on both synthetic and real- world test sets. Fig. 1. Illustration of the interlaced scanning mechanism. Index Terms—deinterlacing, early videos, interlacing artifacts this paper is to specifically design an effective deinterlacing network for the joint restoration tasks of interlaced frames. I. INTRODUCTION The traditional interlaced scanning mechanism can be de- Interlacing artifacts are commonly observed in many early fined as Y = S(X1; X2), where Y denotes the interlaced videos, which are caused by interlacing scanning in early frame, S(·) is the interlaced scanning function, and X1; X2 television systems, e.g., NTSC, PAL, and SECAM. As shown denote the odd and even fields. Traditional deinterlacing in Fig. 1, the odd lines and even lines of an interlaced frame are methods focus on the reversed process of S(·), which can scanned from two different half-frames, i.e., the odd/top/first be roughly divided into four categories, i.e., temporal inter- field and the even/bottom/second field.
    [Show full text]
  • Using Enhancement Data to Deinterlace 1080I HDTV
    Using enhancement data to deinterlace 1080i HDTV The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation Andy L. Lin and Jae S. Lim, "Using enhancement data to deinterlace 1080i HDTV", Proc. SPIE 7798, 77980B (2010). © 2010 Copyright SPIE--The International Society for Optical Engineering As Published http://dx.doi.org/10.1117/12.859876 Publisher SPIE Version Final published version Citable link http://hdl.handle.net/1721.1/72133 Terms of Use Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. Using Enhancement Data to Deinterlace 1080i HDTV Andy L. Lina, Jae S. Limb aStanford University, 450 Serra Mall Stanford, CA 94305 bResearch Lab of Electronics, Massachusetts Institute of Technology, 77 Massachusetts Avenue., Cambridge, MA USA 02139 ABSTRACT When interlaced scan (IS) is used for television transmission, the received video must be deinterlaced to be displayed on progressive scan (PS) displays. To achieve good performance, the deinterlacing operation is typically computationally expensive. We propose a receiver compatible approach which performs a deinterlacing operation inexpensively, with good performance. At the transmitter, the system analyzes the video and transmits an additional low bit-rate stream. Existing receivers ignore this information. New receivers utilize this stream and perform a deinterlacing operation inexpensively with good performance. Results indicate that this approach can improve the digital television standard in a receiver compatible manner. Keywords: Deinterlace, receiver compatible, 1080i, 1080/60/IS, HDTV, motion adaptive deinterlacing, enhancement data 1.
    [Show full text]