Patent Application Publication (10) Pub. No.: US 2007/0052846A1 Adams (43) Pub
Total Page:16
File Type:pdf, Size:1020Kb
US 20070052846A1 (19) United States (12) Patent Application Publication (10) Pub. No.: US 2007/0052846A1 Adams (43) Pub. Date: Mar. 8, 2007 (54) SOURCE-ADAPTIVE VIDEO Publication Classification DENTERLACER (51) Int. Cl. (76) Inventor: Dale Richard Adams, Gualala, CA H04N 7/01 (2006.01) (US) Correspondence Address: (52)52) U.S. Cl. .............................................................. 348/452 PERKINS COE LLP P.O. BOX 21.68 MENLO PARK, CA 94026 (US) (57) ABSTRACT (21) Appl. No.: 11/512,754 1-1. A method is described for deinterlacing an interlaced video (22) Filed: Aug. 29, 2006 stream. The method includes detecting an occurrence of Related U.S. Application Data groups of adjacent fields that are derived from a common original image frame source, and merging field pairs of the (60) Provisional application No. 60/715,711, filed on Sep. interlaced video stream to create a non-interlaced video 8, 2005. stream output. -1 4OO PERFORMA CORRELATION OPERATION BETWEEN FIELD PARS SAVE A HISTORY OF CORRELATION MEASUREMENTS 411 DETERMINE CORRRELATION MAXIMUM, MINIMUM, DYNAMIC RANGE, AND WHETHER 421. SCENE TRANSiTIONS HAVE OCCURRED CALCULATE A THRESHOLD VALUE 431 COMPARE 1-FIELD DIFFERENCE AND 2-FIELD 441 DFFERENCE TO RESPECTIVE THRESHOLDS EXAMINE COMPARSON HISTORY TO 451 DETERMINEF REPEATING PATTERNS EXIST ASSIGNA STATE VALUE TO THE CURRENT FELD 461 ACOUIRE LOCK ON SCNAL 471 DETERMINE HOW TO CONSTRUCT ASEQUENCE OF THE DENTERLACED OUTPUT FRAME 481 FROM THE SEQUENCE OF FIELDS OUTPUT THE SEOUENCE OF DENTERLACED FRAMES 491 Patent Application Publication Mar. 8, 2007 Sheet 1 of 11 US 2007/0052846 A1 101 FIG. 1A PRIOR ART 102 FIG. 1B PRIOR ART : 103 FIG. 1C PRIOR ART Patent Application Publication Mar. 8, 2007 Sheet 2 of 11 US 2007/005284.6 A1 CONSECUTIVE TEMPORAL INTERLACED FIELDS o H r rn vir Ln o n o H n m n Ln to n - - - - - - - - - - - - - - - - O O. O. O. O. O. O. O. O O. O. O. O. O. O. O U U U U U U U U U U U U U U U U LNEO P00 ero o so opsosovo LINE 0 ------|-- -T----T---- mum Hmmm LINE 1 LNE 1 PO1 P11P21 P31 iP41PS1P61 P71 milmmlum milmmell LINE 2 LCLCCLLCCP02P12 P22. P32 P42.P52P62P72. LINE 2 l-Hill LINE 3: LINE 3 iP03P13P23i P33 PPP LINE 4 P04 P14 P24 P34 P44P54P64P74 LINE 4 mmm Hm LINE 5 NES P05 P15. P25IP35 P45P55P65 P75 HHuman kmmHmmm LINE 6 P06 P16 26P36P46Psorse P76 LINE 6 . --------. LINE 7Poz Piz InP27p37P47576 n- in . P77. LINE 7 - 2O1T. y DEINTERLACING MODU LE 203 s s s LINE of --s T s s LNE 1 f up as LINE 2 . ------------L--. LINE 3: -------------- PROGRESSIVE LINE 4 i FRAME LINE 5 H. 204 on a m LINE 6 ----------------- LINE 7 ----------------- FIG. 2 Patent Application Publication Mar. 8, 2007 Sheet 3 of 11 US 2007/0052846A1 110 NTERLACED VIDEO SIGNAL VIDEO SIGNAL PROCESSOR DISPLAY SOURCE 60 DEVICE 80 70 DENTERLACING MODULE 2O3 DEINTERLACED VIDEO SIGNAL FIG. 3 Patent Application Publication Mar. 8, 2007 Sheet 4 of 11 US 2007/0052846A1 u1 COMPARE A SEQUENCE OF FIELD PARS 301 DETERMINEA MEASURE OF SIMLARITY BEWEEN THE FIELD PARS 311 321 ISA SEOUENCE OF DENTERLACE THE FIELD PARS FROM SEOUENCE OF FIELD 341 PARS USING OTHER THE SAME SOURCE2 TECHNIOUES MERGE FIELDS TO RECONSTRUCT A SEOUENCE OF ORIGINAL FRAMES FIG. 4 Patent Application Publication Mar. 8, 2007 Sheet 5 of 11 US 2007/0052846A1 u-400 PERFORMA CORRELATION OPERATION 401 BETWEEN FIELD PARS SAVE A HISTORY OF 411 CORRELATION MEASUREMENTS DETERMINE CORRRELATION MAXIMUM, MINIMUM, DYNAMIC RANGE, AND WHETHER 421. SCENE TRANSiTIONS HAVE OCCURRED CALCULATE ATHRESHOLD VALUE 431 COMPARE 1-FIELD DIFFERENCE AND 2-FIELD 441 DIFFERENCE TO RESPECTIVE THRESHOLDS EXAMINE COMPARISON HISTORY TO 451 DETERMINEF REPEATING PATTERNS EXIST ASSIGNA STATE VALUE TO THE CURRENT FIELD 461 ACOUIRE LOCK ON SIGNAL 471 DETERMINE HOW TO CONSTRUCT A SEOUENCE OF THE DEINTERLACED OUTPUT FRAME 481 FROM THE SEQUENCE OF FIELDS OUTPUT THE SEQUENCE OF DEINTERLACED FRAMES 491 FIG. 5 Patent Application Publication Mar. 8, 2007 Sheet 6 of 11 US 2007/0052846 A1 SCITHIAHVTIWISSGTBIHINBH3+HIGSCITBIHSGTBHINBHEBHIGSGITHIA(WIIWIS BWVHHBOHDOS TÕõ Patent Application Publication Mar. 8, 2007 Sheet 7 of 11 US 2007/0052846 A1 SGT3|?LNBH3+HICSCITBIHLNBH3+HIOLNBH3.J.HIGSOTB|+LNBYE-HIG] Z‘DI Patent Application Publication US 2007/0052846 A1 BWVH-IHOHDOS T?? ZS9S_LNHWNOISSVELVIS Patent Application Publication Mar. 8, 2007 Sheet 9 of 11 US 2007/0052846 A1 Z99SLNBWNDISSWELWLS SOTEII-ISO]TE|-3SOTEI-JSOTEI-SOTB||-||089SCITEI-829 Patent Application Publication Mar. 8, 2007 Sheet 10 of 11 US 2007/0052846A1 CORRELATION VALUE MAX 402 -- DYNAMIC RANGE 404 -- MIN 403 TIME CORRELATION VALUE MAX 405 MAX 402 MIN 403 TIME CORRELATION VALUE MAX 402 MIN 403 Patent Application Publication Mar. 8, 2007 Sheet 11 of 11 US 2007/0052846A1 SCENE TRANSITION MOMENTARY SPIKE CORRELATION VALUE MAX 402 MIN 403 TIME FIG. 13 US 2007/0052846 A1 Mar. 8, 2007 SOURCE-ADAPTIVE VIDEO DENTERLACER be detected. If the cadence is known, then the original film frame can be reconstructed by simply combining the correct CROSS-REFERENCE TO RELATED two fields in a weaving operation. However, cadence detec APPLICATION tion systems are inadequate because they are generally 0001. This Application claims the benefit of United States limited to 3:2 pulldown or 2:2 pulldown. Further cadence Provisional Application No. 60/715,711 entitled “Source detection systems incur a number of problems when the Adaptive Video Deinterlacer” filed on Sep. 8, 2005, which cadence pattern is broken for some reason, Such as video is incorporated by reference. edits. 0008. The foregoing examples of the related art and BACKGROUND limitations related therewith are intended to be illustrative and not exclusive. Other limitations of the related art will 0002 Interlaced scanning and progressive scanning are become apparent to those of skill in the art upon a reading typical scanning methods employed in a video display of the specification and a study of the drawings. device. Interlaced scanning has been employed in current National Television Systems Committee (“NTSC) televi SUMMARY sion (“TV) systems. For the video display shown in FIGS. 1A-1C, even and odd fields would be interlaced sequentially 0009. The following embodiments and aspects thereof and alternately on the screen when displaying a video image. are described and illustrated in conjunction with systems, For FIGS. 1A-1C, for interlaced scanning, the solid lines tools, and methods that are meant to be exemplary and would represent lines being scanned at present, and the illustrative, not limiting in scope. In various embodiments, dotted lines would represent lines Scanned for a preceding one or more of the above-described problems have been display. reduced or eliminated, while other embodiments are directed 0003 Deinterlacing an interlaced signal provides numer to other improvements. ous advantages for improving video quality. Specifically, 0010) A technique for deinterlacing an interlaced video deinterlacing can remove interlace motion artifacts, increase stream involves the detection and merging of fields. An apparent vertical resolution, and reduce flicker. Furthermore, example of a method according to the technique involves deinterlacing is often required because modern televisions detecting an occurrence of groups of adjacent fields that are are inherently progressive and the video feed is broadcast in derived from a common original image frame source. Field interlaced form. pairs of the interlaced video stream can be merged to form 0004 There are three common techniques for deinterlac a deinterlaced signal. ing an interlaced video signal. A first deinterlacing technique 0011. In certain embodiments, the detection of an occur is known as weaving. Weaving involves combining two rence of groups of adjacent fields that are derived from a adjacent fields into one frame. While this technique main common image frame source can involve determining if tains vertical resolution, it has the problem of creating fields pairs of the interlaced video stream are similar. If the interlace motion artifacts if motion is present. fields pairs are similar, the field pairs can be merged. If more than two adjacent field pairs are different, an alternate 0005. A second deinterlacing technique is known as deinterlacing technique can be implemented, such as motion vertical interpolation. Vertical interpolation involves aver adaptive deinterlacing. aging at least two scan lines to generate a new scan line. The technique is repeated for all scan lines and creates a full 0012. In some embodiments, determining if field pairs frame from a single video field. While vertical interpolation are similar can involve performing a correlation operation allows a progressive picture to be generated from one video between field pairs. In other embodiments, a difference field, half of the resolution of the video feed is lost. operation between field pairs can be performed. Additional embodiments can determine whether a scene transition 0006 Another deinterlacing technique is known as occurred in order to aid in the determination of similar and motion adaptive deinterlacing. For this technique, adjacent different field pairs. An alternate embodiment can involve fields are merged for still areas of the picture and vertical calculating a threshold value based on one or more factors. interpolation is used for areas of movement. To accomplish Following this embodiment, the factors can include a history this, motion, on a sample-by-sample basis, is detected over of correlation operation values between field pairs, a mini the entire picture in real time, requiring processing of several mum and maximum value in the history of correlation fields of a video signal. operation values, a minimum and maximum value for a 0007 To improve the results of the common techniques, range of correlation values, detecting scene transitions, and a cadence detection algorithm can be implemented. In order assigning State values. for the progressive source to be converted to an interlaced format, each frame from that Source must be represented as 0013 Another example a method according to the tech multiple fields in the interlaced format - i.e., a single frame nique involves detecting an occurrence of groups of con is converted to 20r more interlaced fields.