US 20070052846A1 (19) United States (12) Patent Application Publication (10) Pub. No.: US 2007/0052846A1 Adams (43) Pub. Date: Mar. 8, 2007

(54) SOURCE-ADAPTIVE Publication Classification DENTERLACER (51) Int. Cl. (76) Inventor: Dale Richard Adams, Gualala, CA H04N 7/01 (2006.01) (US) Correspondence Address: (52)52) U.S. Cl...... 348/452 PERKINS COE LLP P.O. BOX 21.68 MENLO PARK, CA 94026 (US) (57) ABSTRACT (21) Appl. No.: 11/512,754 1-1. A method is described for deinterlacing an (22) Filed: Aug. 29, 2006 stream. The method includes detecting an occurrence of Related U.S. Application Data groups of adjacent fields that are derived from a common original image frame source, and merging pairs of the (60) Provisional application No. 60/715,711, filed on Sep. interlaced video stream to create a non-interlaced video 8, 2005. stream output.

-1 4OO PERFORMA CORRELATION OPERATION BETWEEN FIELD PARS

SAVE A HISTORY OF CORRELATION MEASUREMENTS 411

DETERMINE CORRRELATION MAXIMUM, MINIMUM, DYNAMIC RANGE, AND WHETHER 421. SCENE TRANSiTIONS HAVE OCCURRED

CALCULATE A THRESHOLD VALUE 431

COMPARE 1-FIELD DIFFERENCE AND 2-FIELD 441 DFFERENCE TO RESPECTIVE THRESHOLDS

EXAMINE COMPARSON HISTORY TO 451 DETERMINEF REPEATING PATTERNS EXIST

ASSIGNA STATE VALUE TO THE CURRENT FELD 461

ACOUIRE LOCK ON SCNAL 471

DETERMINE HOW TO CONSTRUCT ASEQUENCE OF THE DENTERLACED OUTPUT FRAME 481 FROM THE SEQUENCE OF FIELDS

OUTPUT THE SEOUENCE OF DENTERLACED FRAMES 491 Patent Application Publication Mar. 8, 2007 Sheet 1 of 11 US 2007/0052846 A1

101

FIG. 1A PRIOR ART

102

FIG. 1B PRIOR ART : 103

FIG. 1C PRIOR ART Patent Application Publication Mar. 8, 2007 Sheet 2 of 11 US 2007/005284.6 A1

CONSECUTIVE TEMPORAL INTERLACED FIELDS

o H r rn vir Ln o n o H n m n Ln to n ------O O. O. O. O. O. O. O. O O. O. O. O. O. O. O U U U U U U U U U U U U U U U U LNEO P00 ero o so opsosovo LINE 0 ------|-- -T----T---- mum Hmmm LINE 1 LNE 1 PO1 P11P21 P31 iP41PS1P61 P71 milmmlum milmmell LINE 2 LCLCCLLCCP02P12 P22. P32 P42.P52P62P72. LINE 2 l-Hill LINE 3: LINE 3 iP03P13P23i P33 PPP LINE 4 P04 P14 P24 P34 P44P54P64P74 LINE 4 mmm Hm LINE 5 NES P05 P15. P25IP35 P45P55P65 P75 HHuman kmmHmmm LINE 6 P06 P16 26P36P46Psorse P76 LINE 6 . ------...... LINE 7Poz Piz InP27p37P47576 n- in . . P77. LINE 7 - 2O1T. y

DEINTERLACING MODU LE 203

s s s LINE of --s T s s LNE 1 f up as LINE 2 . ------L--. . LINE 3: ------PROGRESSIVE LINE 4 i FRAME LINE 5 H. 204 on a m LINE 6 ------LINE 7 ------FIG. 2 Patent Application Publication Mar. 8, 2007 Sheet 3 of 11 US 2007/0052846A1

110 NTERLACED VIDEO SIGNAL

VIDEO

SIGNAL PROCESSOR DISPLAY SOURCE 60

DEVICE 80 70

DENTERLACING MODULE

2O3

DEINTERLACED VIDEO SIGNAL

FIG. 3 Patent Application Publication Mar. 8, 2007 Sheet 4 of 11 US 2007/0052846A1 u1 COMPARE A SEQUENCE OF FIELD PARS 301

DETERMINEA MEASURE OF SIMLARITY BEWEEN THE FIELD PARS 311

321

ISA SEOUENCE OF DENTERLACE THE

FIELD PARS FROM SEOUENCE OF FIELD 341 PARS USING OTHER THE SAME SOURCE2 TECHNIOUES

MERGE FIELDS TO RECONSTRUCT A SEOUENCE OF ORIGINAL FRAMES

FIG. 4 Patent Application Publication Mar. 8, 2007 Sheet 5 of 11 US 2007/0052846A1 u-400 PERFORMA CORRELATION OPERATION 401 BETWEEN FIELD PARS

SAVE A HISTORY OF 411 CORRELATION MEASUREMENTS

DETERMINE CORRRELATION MAXIMUM, MINIMUM, DYNAMIC RANGE, AND WHETHER 421. SCENE TRANSiTIONS HAVE OCCURRED

CALCULATE ATHRESHOLD VALUE 431

COMPARE 1-FIELD DIFFERENCE AND 2-FIELD 441 DIFFERENCE TO RESPECTIVE THRESHOLDS

EXAMINE COMPARISON HISTORY TO 451 DETERMINEF REPEATING PATTERNS EXIST

ASSIGNA STATE VALUE TO THE CURRENT FIELD 461

ACOUIRE LOCK ON SIGNAL 471

DETERMINE HOW TO CONSTRUCT A SEOUENCE OF THE DEINTERLACED OUTPUT FRAME 481 FROM THE SEQUENCE OF FIELDS

OUTPUT THE SEQUENCE OF DEINTERLACED FRAMES 491 FIG. 5 Patent Application Publication Mar. 8, 2007 Sheet 6 of 11 US 2007/0052846 A1

SCITHIAHVTIWISSGTBIHINBH3+HIGSCITBIHSGTBHINBHEBHIGSGITHIA(WIIWIS

BWVHHBOHDOS TÕõ Patent Application Publication Mar. 8, 2007 Sheet 7 of 11 US 2007/0052846 A1

SGT3|?LNBH3+HICSCITBIHLNBH3+HIOLNBH3.J.HIGSOTB|+LNBYE-HIG] Z‘DI

Patent Application Publication US 2007/0052846 A1

BWVH-IHOHDOS T?? ZS9S_LNHWNOISSVELVIS

Patent Application Publication Mar. 8, 2007 Sheet 9 of 11 US 2007/0052846 A1

Z99SLNBWNDISSWELWLS SOTEII-ISO]TE|-3SOTEI-JSOTEI-SOTB||-||089SCITEI-829

Patent Application Publication Mar. 8, 2007 Sheet 10 of 11 US 2007/0052846A1

CORRELATION VALUE

MAX 402 -- DYNAMIC RANGE 404 -- MIN 403 TIME

CORRELATION VALUE MAX 405 MAX 402

MIN 403

TIME

CORRELATION VALUE

MAX 402

MIN 403 Patent Application Publication Mar. 8, 2007 Sheet 11 of 11 US 2007/0052846A1

SCENE TRANSITION MOMENTARY SPIKE

CORRELATION VALUE

MAX 402

MIN 403

TIME

FIG. 13 US 2007/0052846 A1 Mar. 8, 2007

SOURCE-ADAPTIVE VIDEO DENTERLACER be detected. If the cadence is known, then the original can be reconstructed by simply combining the correct CROSS-REFERENCE TO RELATED two fields in a weaving operation. However, cadence detec APPLICATION tion systems are inadequate because they are generally 0001. This Application claims the benefit of United States limited to 3:2 pulldown or 2:2 pulldown. Further cadence Provisional Application No. 60/715,711 entitled “Source detection systems incur a number of problems when the Adaptive Video Deinterlacer” filed on Sep. 8, 2005, which cadence pattern is broken for some reason, Such as video is incorporated by reference. edits. 0008. The foregoing examples of the related art and BACKGROUND limitations related therewith are intended to be illustrative and not exclusive. Other limitations of the related art will 0002 Interlaced scanning and progressive scanning are become apparent to those of skill in the art upon a reading typical scanning methods employed in a video display of the specification and a study of the drawings. device. Interlaced scanning has been employed in current National Television Systems Committee (“NTSC) televi SUMMARY sion (“TV) systems. For the video display shown in FIGS. 1A-1C, even and odd fields would be interlaced sequentially 0009. The following embodiments and aspects thereof and alternately on the screen when displaying a video image. are described and illustrated in conjunction with systems, For FIGS. 1A-1C, for interlaced scanning, the solid lines tools, and methods that are meant to be exemplary and would represent lines being scanned at present, and the illustrative, not limiting in scope. In various embodiments, dotted lines would represent lines Scanned for a preceding one or more of the above-described problems have been display. reduced or eliminated, while other embodiments are directed 0003 Deinterlacing an interlaced signal provides numer to other improvements. ous advantages for improving video quality. Specifically, 0010) A technique for deinterlacing an interlaced video deinterlacing can remove interlace motion artifacts, increase stream involves the detection and merging of fields. An apparent vertical resolution, and reduce . Furthermore, example of a method according to the technique involves deinterlacing is often required because modern televisions detecting an occurrence of groups of adjacent fields that are are inherently progressive and the video feed is broadcast in derived from a common original image frame source. Field interlaced form. pairs of the interlaced video stream can be merged to form 0004 There are three common techniques for deinterlac a deinterlaced signal. ing an interlaced video signal. A first deinterlacing technique 0011. In certain embodiments, the detection of an occur is known as weaving. Weaving involves combining two rence of groups of adjacent fields that are derived from a adjacent fields into one frame. While this technique main common image frame source can involve determining if tains vertical resolution, it has the problem of creating fields pairs of the interlaced video stream are similar. If the interlace motion artifacts if motion is present. fields pairs are similar, the field pairs can be merged. If more than two adjacent field pairs are different, an alternate 0005. A second deinterlacing technique is known as deinterlacing technique can be implemented, such as motion vertical interpolation. Vertical interpolation involves aver adaptive deinterlacing. aging at least two scan lines to generate a new scan line. The technique is repeated for all scan lines and creates a full 0012. In some embodiments, determining if field pairs frame from a single video field. While vertical interpolation are similar can involve performing a correlation operation allows a progressive picture to be generated from one video between field pairs. In other embodiments, a difference field, half of the resolution of the video feed is lost. operation between field pairs can be performed. Additional embodiments can determine whether a scene transition 0006 Another deinterlacing technique is known as occurred in order to aid in the determination of similar and motion adaptive deinterlacing. For this technique, adjacent different field pairs. An alternate embodiment can involve fields are merged for still areas of the picture and vertical calculating a threshold value based on one or more factors. interpolation is used for areas of movement. To accomplish Following this embodiment, the factors can include a history this, motion, on a sample-by-sample basis, is detected over of correlation operation values between field pairs, a mini the entire picture in real time, requiring processing of several mum and maximum value in the history of correlation fields of a video signal. operation values, a minimum and maximum value for a 0007 To improve the results of the common techniques, range of correlation values, detecting scene transitions, and a cadence detection algorithm can be implemented. In order assigning State values. for the progressive source to be converted to an interlaced format, each frame from that Source must be represented as 0013 Another example a method according to the tech multiple fields in the interlaced format - i.e., a single frame nique involves detecting an occurrence of groups of con is converted to 20r more interlaced fields. The conversion secutive temporal fields of interlaced video fields that are process (called ) typically results in a regular repeat derived from a common original image frame source. A ing pattern of fields taken from an original progressive repeating pattern in the groups can be detected and locked. frame. For example, to convert 24 frame/sec film to 60 Fields pairs can be combined based on the repeating pattern. field/sec interlaced video, a technique known as 3:2 pull 0014. The proposed method and device can offer, among down is used. The technique converts one film frame to 3 other advantages, improved quality of deinterlaced video. fields, the next frame to 2 fields, the next to 3, etc. The result This can be accomplished in an efficient and robust manner is a regular 3/2/3/2 repeating pattern, or cadence, which can compared to other deinterlacing techniques because a US 2007/0052846 A1 Mar. 8, 2007

cadence can be detected independent of the source. These sequence of two fields have a common source is to deter and other advantages of the present invention will become mine how similar the two fields are to each other. The more apparent to those skilled in the art upon a reading of the similar the two fields are to each other, the more likely it is following descriptions and a study of the several figures of that they have been taken from the same original source the drawings. frame. Once it has been determined that two fields come from the same original source frame, they can be merged to BRIEF DESCRIPTION reconstruct the original Source frame. 00.15 Embodiments of the inventions are illustrated in 0031. According to certain embodiments of the present the figures. However, the embodiments and figures are invention, the method of deinterlacing by determining if two illustrative rather than limiting; they provide examples of the fields have a common source can recognize and adapt to any invention. cadence or cadences in the video signal. Further, once it has been determined that two fields come from the same original 0016 FIGS. 1A-1C show a video display with screen Source frame, they can be merged without need of any lines. further processing, such as calculating pixels. 0017 FIG. 2 schematically illustrates the conversion of 0032 FIG. 2 illustrates a deinterlacing module 203 that two consecutive interlaced fields into a single progressive converts two temporal interlaced fields 201 and 202 into a frame. progressive frame 204. The interlaced field 201 is an even 0018 FIG. 3 illustrates a video system. field because it only includes lines 0, 2, 4,and 6. The interlaced field 202 is an odd field because it only includes 0.019 FIG. 4 is a flowchart of a method for converting a lines 1, 3, 5,and 7. sequence of consecutive interlaced fields into a sequence of progressive frames. 0033 For the example of FIG. 2 and subsequent figures, interlaced fields are shown as being relatively small for 0020 FIG. 5 is a flowchart of a method for converting a clarity. For example, the interlaced fields are illustrated as sequence of consecutive interlaced fields into a sequence of being composed of 4lines of 8 pixels, while the progressive progressive frames. frames are illustrated as being composed of 81ines of 8 0021 FIG. 6 illustrates the utility of using a 1-field pixels. Nevertheless, typical interlaced fields and progres difference for a 2:2 pulldown. sive frames will be many times larger than the example interlaced fields and progressive frames used in the figures 0022 FIG. 7 illustrates the lack of utility of using a and in this description. The smaller example fields and 2-field difference for a 2:2 pulldown. frames are used herein for clarity in describing the key 0023 FIG. 8 illustrates the utility of using a 1-field principles of embodiments of the present invention. difference for a 3:2 pulldown. Embodiments of the present invention are not limited to any particular size of interlaced field or progressive frame. 0024 FIG. 9 illustrates the utility of using a 2-field difference for a 3:2 pulldown. 0034) The deinterlacing module 203 of FIG. 2 may be part of a video system, such as the video system 110 of FIG. 0025 FIG. 10 is an illustration of correlation values of 3. Video system 110 includes a system 60,a temporally consecutive fields over time. video display device 70 and a signal source 80. Video 0026 FIG. 11 is an illustration of correlation values of processing system 60 may include one or more program temporally consecutive fields over time. mable general-purpose or special-purpose microprocessors, digital signal processors (“DSPs'), programmable control 0027 FIG. 12 is an illustration of correlation values of lers, field programmable gate array (“FPGAs), application temporally consecutive fields over time. specific integrated circuits (ASICs'), programmable logic 0028 FIG. 13 is an illustration of correlation values of device (“PLDs), or the like, or a combination of such temporally consecutive fields over time. devices. The deinterlacing module 203 may be, for example, implemented in a PLD, a FPGA, an ASIC or other type of DETAILED DESCRIPTION circuitry or device. Alternatively, or in addition, the dein terlacing module 203 may be one of the application pro 0029. In the following description, several specific details grams and/or other program modules which may be loaded are presented to provide a thorough understanding of into system memory of processing system 60 in preparation embodiments of the invention. One skilled in the relevant art for execution by processors of the processing system 60. For will recognize, however, that the invention can be practiced one embodiment, deinterlacing module 203 is part of video without one or more of the specific details, or in combination processor 60. with other components, etc. In other instances, well-known 0035) Processing system 60 includes a signal input that implementations or operations are not shown or described in receives a video signal from a signal source 80. Signal detail to avoid obscuring aspects of various embodiments, of Source 80, may be either a single channel signal. Source or a the invention. multiple channel signal Source. A single channel signal 0030. A method is described for source adaptive deinter Source provides programming from a recorded medium, lacing of an interlaced video signal to create a progressive Such as a videocassette, compact disc, DVD, etc. Examples video signal. As described further below, the method of a single channel signal source include a videocassette includes determining if a sequence of two or more fields recorder, a CD player, and a DVD player. A multiple channel from the interlaced source is derived from the same original signal Source includes any system or device that is capable progressive source frame. One method of determining if a of sending a signal that may be received by a satellite US 2007/0052846 A1 Mar. 8, 2007

receiver, a cable or optic connection, a terrestrial antenna, or 0042. According to embodiments of the present inven the like. Examples of a multiple channel signal Source tion, a correlation operation may be performed on two fields include Digital Satellite System (“DSS), Digital Video that are spaced “x' fields is apart, where x>0. However, Broadcasting (“DVB), a cable box, locally broadcast pro performing a correlation operation on fields that are in the gramming (i.e. programming broadcast using Ultra High vicinity of each other, for instance, are spaced 1 or 2 fields Frequency (“UHF) or Very High Frequency (“VHF'), and apart, is more useful and reliable, because it provides a more so forth. frequent indication of any changes in field data. 0036) The output of video processor 60 goes to a video 0043. The 2-field operation compares fields that are spa display device 70.such as an HDTV, a standard definition tially coincident. Thus, for an even-odd sequence of fields, TV, a computer monitor, etc. The display 70 can employ the fields being compared are both composed of either various display techniques, such a , a cathode even-numbered lines or odd-numbered lines. Accordingly, a ray tube (“CRT), a LCD display, a DLP display, and a 2-field difference. The magnitude of the pixel calculating a projector, for example. pixel-by-pixel difference to obtain a 2-field difference. The 0037. A video processor 60 employing a deinterlacing magnitude of the pixel difference values can be Summed module 203 receives an interlaced video signal from video over the entire field, with the resultant sum indicating how signal source 80 and provides a deinterlaced signal to different or similar the two fields are overall. A low resultant display device 70. sum value indicates very similar fields while a high resultant sum value indicates very different fields. The terms “low” 0038 For alternative embodiments, the deinterlacing and “high, as well as “similar and “different as used module 203 can be part of the video signal source apparatus herein, are, of course, relative. 80 or the video display device 70. For alternative embodi ments, the video processor 60 can be part of the video signal 0044) The 1-field operation involves comparing two source 80 or the video display device 70. fields that are not spatially coincident—i.e., for an even-odd sequence of fields, one field is composed of even-numbered 0039 FIG. 4 is a flowchart of an overall method 300 of lines while the other is composed of odd-numbered lines. converting a sequence of temporally consecutive fields of Thus, because the two fields are not spatially coincident, interlaced video signal (e.g., fields 201 and 202 of FIG. 2) they cannot be directly compared. In addition, there may be into a sequence of progressive frames of video data. The aliasing present in a single field due to the fact that taking method involves detecting in an interlaced video stream the only every other line of a source video image may not occurrence of groups of adjacent fields that are derived from generate a high enough vertical sampling rate to representall the same original video image frame so that the original the vertical high frequencies in the video signal. The aliases Video image frame may be reconstructed by merging the are different between the two fields being compared, causing fields. The detection is done by comparing a pair of fields at yet another difference between them. block 301, determining a measure of similarity or difference between the fields at block 311.and determining if the fields 0045. From each field a comparison field is created by are derived from a common source frame at block 321. If phase-shifting one field up and the other down. For one there is a high level of confidence that the fields are derived embodiment, one field is phase-shifted 4 line up and a field from a common source frame, then pairs of fields can be to be compared 4 line down. The phase shift can be merged to reconstruct the original source frame at block 331. performed for instance, using a Finite Impulse Response Otherwise, the two fields are deinterlaced using other tech (“FIR) filter approach, although other techniques such as niques at block 341. simple linear interpolation can also be used. For this embodiment, a comparison field is created from each field 0040 Method 300 utilizes the observation that the more by computing pixels from the pixels of each field and similar two or more fields are to each other, the more likely comparing the calculated pixels. Additionally, a comparison they are to have been taken from the same original video field can be created from one of the two fields by computing image. The method 300 also utilizes the fact that conversion pixels from the pixels of that field and the resultant calcu of a slower progressive source to a faster frame lated pixels can be compared with the pixels of the other rate interlaced format requires that multiple sequential fields field. The resulting field can be compared with an original be taken from the same original source frame. In other field in the same manner as used for the 2-field difference words, there will be a sequence of adjacent fields that are all described above. taken from the same source frame. 0041 FIG. 5 is a flowchart of a method 400 of converting 0046. In general, however, the 1-field difference has a a sequence of temporally consecutive fields of an interlaced higher baseline or noise level than the 2-field difference due video signal (e.g., fields 201 and 202 of FIG. 2) into a to the fact that the original two fields being compared are not sequence of a progressive video signal. At block 401.in spatially coincident. Therefore, one or more operations may order to determine how similar two fields are to each other, also be performed on each of the two original fields before a correlation operation is performed on a field pair. Accord the phase shift to reduce noise and aliasing artifacts. One ingly, a correlation operation may be performed on two Such operation is a vertical low-pass filtering operation. fields that are spaced one field apart. This operation is herein 0047 According to certain embodiments of the inven known as “1-field operation.” Alternatively, a correlation tion, both the 1-field difference and the 2-field difference operation may be performed on two fields that are spaced may be calculated. The 2-field difference provides useful two fields apart. This operation is herein known as "2-field measurements when used on certain cadences, like 3:2 operation.” According to certain embodiments of the inven pulldown, but is not very useful when used on a cadence that tion, both the 1-field operation and the 2-field operation may has only two fields taken from the same frame, such as 2:2 be calculated. pulldown. FIG. 6 illustrates a pattern 631 detected by using US 2007/0052846 A1 Mar. 8, 2007

a 1-field difference for a 2:2 pulldown. FIG. 7 illustrates that new correlation value is much larger than the maximum by using a 2-field difference for a 2:2 pulldown, a detected value currently in the history buffer, then the new value will pattern 641 actually shows no repeating pattern. For both be reduced in amplitude to avoid abnormally skewing the FIGS. 6 and 7. source frames 601, 611,and history data. A correlation value can be much higher than a 621 are converted respectively into interlaced video field maximum value due to noise, momentary changes in the pairs 603 and 605, 613 and 615,and 623 and 625. A 1-field image data, or a scene transition. If a long term increase in difference of the fields displays a detection pattern of 0-1- the correlation value is actually present, the history values 0-1-0, where a “0” represents that the “1'represents that the will adjust over time to match the incoming values. two fields being compared are different to each other. A 2-field difference of the fields displays a detection pattern of 0053) Once new correlation values have been added, the 1-1-1-1-1. Thus, a 2-field difference for a 2:2 pulldown updated history record is examined to determine the mini always conveys that the fields are different to each other, and mum and maximum correlation values. The sequence of therefore, in contrast to the 1-field difference does not history values is also examined to determine if the higher provide much valuable information about the fields or the values (which represent fields which are from different cadence. Source frames) have a decreasing trend, or if the lower values (which represent fields which are from the same 0048. On the other hand, as illustrated in FIGS. 8 and Source frame) have an increasing trend. Higher maximums 9.both 1-field difference and 2-field difference operations are and lower minimums are tracked by their very nature, but a suitable for use on a 3:2 pulldown video sequence. Fig. 8 downward maximum or upward minimum trend must be illustrates a pattern 651 detected by using a 1-field difference consciously tracked. For instance, as shown in FIG. 11, an for a 3:2 pulldown. FIG. 9 illustrates a pattern 661 detected upward maximum trend is tracked because a new local by using a 2-field difference for a 3:2 pulldown. For both maximum 405 is greater than the previous local maximum FIGS. 8 and 9.progressive scan source frames 601, 611,and 402. However, as shown in FIG. 12, a downward maximum 621 are converted respectively into interlaced video field trend is not tracked by its very nature, because the local groups 602, 604,and 606; 612 and 614; and 622, 624 and maximum 402 does not change. This tracking can be done 626. A 1-field difference of the fields displays a repeating for maximums by looking at the sequence of history values detection pattern of 0-0-1-0-1. A 2-field difference of the for which the threshold comparison (as described below) fields displays a repeating detection pattern of 0-1-1-1-1. yield a '1'(or “different') value, and is done for minimums Thus, both the 1-field and 2-field difference for a 3:2 by looking at only the history values for which the threshold pulldown provide useful information. comparison yields a “0”(or "similar') value. When such a 0049. In the description of FIG. 5, while both 1-field and trend is identified, the minimum and/or maximum values 2-field correlations are calculated and used, it is possible to provided as outputs for Subsequent processing steps can be perform the Source-type detection based on only one of these set to the most recent value in the trend as opposed to the measurements. Using both measurements together may absolute minimum and/or maximum in the history record. result in a much more reliable detection of similar fields. 0054 As discussed above, the 1-field difference measure 0050. At block 411 of FIG. 5, a history of the correlation ment has a higher baseline value than the 2-field difference measurements or values is saved. For one embodiment, the due to the fact that a pair of fields will almost never perfectly history of the correlation measurements is saved in a history correlate. According to certain embodiments of the present buffer. When a new pair of frame correlation values is invention, a method to compensate for this discrepancy is available at the end of each field period, the oldest values in provided. A multiple of the minimum value in the history the history record are discarded and the new values are record is subtracted from both the minimum and the maxi added. According to certain embodiments of the present mum values before they are provided as outputs. A larger invention, correlation history measurements are saved over multiple is used when the dynamic range value is high and time for “n” frame periods. Thus, for both the 1-field and a multiple of 1 is used when the dynamic range is low. One 2-field difference frame sums, a history is kept of the most net result of this is that the output minimum is always Zero. recent “n” frame periods. For one embodiment, the value of 0055. In addition, scene transition occurrences in a video n is chosen based on the longest cadence period that is sequence can also be detected. A scene transition normally desirable to detect. Thus, for 3:2 pulldown, a value of n can results in a momentary spike, as illustrated in FIG. 13, in be 5,which represents the maximum cadence period length. both the 1-field difference and the 2-field difference values. 0051 Based on the saved history, at block 421 of FIG. 5 Detecting the scene transition is important so as to prevent the following calculations can be made: local correlation the history values from being artificially skewed upwards, maximum and minimum values, the dynamic range of the and also to identify that a change in correlation level may be correlation values, and whether scene transitions occur occurring. A scene transition can result in significant within the vicinity of the current field or in the history range. changes in the overall video image content, and this can As illustrated in FIG. 10 a local correlation maximum value manifest as very different minimum, maximum, and 402 is the maximum value of the difference frame sums, dynamic range values. Furthermore, the scene transition while a local correlation minimum value 403 is the mini detection information is passed on to Subsequent processing mum value of the difference frame Sums. The dynamic range stages so that any Such changes can be taken into account. 404 of the correlations values is a difference between the 0056. At block 431 of FIG. 5, based on the correlation maximum and minimum correlation values in the history history measurements, a threshold value is calculated. If the range. correlation value is above the threshold, then the fields are 0.052 For one embodiment, new correlation values are different. If the correlation value is below the threshold the adapted before being saved in the history. For instance, if a fields are considered to be similar. The threshold value US 2007/0052846 A1 Mar. 8, 2007

calculation is based on a number of factors, such as the 3:2:3:2:2, 3:3, 5:5, 6:4 and 8:7. The 3:2 pulldown compari minimum and maximum correlation values from the corre son pattern repeats every five field periods for both the lation history, the presence or absence of a scene transition, 1-field difference and the 2-field difference. The 2:2 pull and the presence of a downward trend in the history maxi down comparison pattern repeats every two field periods, mum values. The threshold value is a dynamic value that but only for the 1-field difference, because the 2-field changes from field to field. difference fails in this case. Thus, for instance, a comparison 0057 Threshold values for 1-field operations and 2-field pattern of 0-1-0-1-0- ... using a 1-field difference indicates operations are calculated slightly differently. For one a 2:2 pulldown. According to certain embodiments of the embodiment, for both operations, the threshold value is a present invention, all cadences are tested from repeating fraction of the dynamic range and is calculated by using the every two field periods to repeating every fifteen field following equation: Dynamic Range/Scaling Factor+Mini periods. The detected cadence or cadences are presented as mum. The scaling factor varies depending on various fac output for Subsequent processing steps. tOrS. 0062). At block 461 of FIG. 5, based on the history of the comparison values, as well as the dynamic range value of the 0.058. The scaling factor for calculating the threshold for correlation values, a state value is assigned to the current the 1-field operation depends largely on the size of the field. This is done for both the 1-field and 2-field correlation maximum. Because the 1-field minimum is always Zero, the histories. For one embodiment, a state value includes the maximum is effectively the same as the dynamic range. values of “Start, “Middle, “End or “None. The “Start Larger maximum values cause a larger Scaling factor to be state indicates that a repeated sequence of fields from the used, with a nominal range of 6 to 64. The threshold is same frame is just beginning. The "End' state indicates that biased towards the minimum rather than the middle of the the repeated sequence is ending. The “Middle' state indi range for two primary reasons. First, the larger correlation cates that the current field is between the beginning and end values tend to vary more than the minimums, and secondly, of the repeated sequence. The “None” state indicates that no because it is better to incorrectly decide that two fields are repeated sequence is deemed to be present. As shown in FIG. different than to incorrectly decide that they are the same. In 8, the repeating state assignments 652 for 1-field difference the event of a scene transition, the threshold is decreased to for a 3:2 pulldown is Start/Middle/End/Start/End. prevent false detections when the transition is from a higher average motion level to a lower average motion level. In 0063. The basis of the state value assignment is the Such a case, a false detection would indicate that fields are observation that conversion of a slower frame rate progres similar when they are really not, resulting in interlace sive source to a faster frame rate interlaced format requires motion artifacts being present in the deinterlaced video that multiple sequential fields be taken from the same signal. If a downward maximum trend is present, then a original source frame. In other words, there will be a transition has no effect on the threshold calculation because sequence of adjacent fields that are all taken from the same the threshold level will already have been depressed by the Source frame. Such a sequence must have a first field and a decreasing maximum values. last field, and may have one or more middle fields. 0064. The possible state values are therefore “Start”, 0059. The 2-field threshold scaling factor is also calcu “Middle', and “End' when a repeating sequence exists. lated as a fraction of the 2-field difference dynamic range. When no repeating sequence exists a state value of “None” The calculation rules are a bit different, however, as the is used. Because the 3:2 pulldown technique is very com minimum is often not Zero and more truly represents the mon, and because a 2-field difference is a very reliable difference between the 2 fields. When the minimum is very indicator of Such a pattern, an additional state assignment Small, then the dynamic range scaling factor is chosen based can be made to cover the two field periods not handled by on the minimum amplitude. For larger minimums, the the other states for the 2-field difference pattern. This state threshold scaling factor is a fixed value. Like the 1-field assignment is termed “InBetween.” Thus, as shown in FIG. difference threshold, the 2-field threshold is depressed in the 9, the repeating state assignments 662 for a 2-field difference event of a scene transition. pattern with a 3:2 pulldown source would be Start / Middle/ 0060. At block 441, the 1-field difference and 2-field End/InBetween/InBetween. difference values are each compared to their respective 0065. The state assignments are based on the comparison thresholds. If the difference value is above the threshold, the history as well as the previous state values. The following is result is a “1”(or “different”) and if it is below the threshold, a non-exhaustive list of examples of State assignment: the result is a “0”(or “similar). A history of the comparison results for both 1-field and 2-field differences is maintained TABLE 1. for a set number “m” of previous field periods. For one embodiment, m is equal to at least two times n, because to Field Current Previous recognize a repeating pattern, at least two instances of the Operation Comparison Comparison Previous State State pattern need to be present. The comparison history may be 1-field O 1 End or None Start saved in a history buffer. 1-field 1 Start or Middle End 2-field O 1 None, InBetween, or End Start 0061. At block 451 of FIG. 5, the comparison history is examined to determine if repeating patterns exist. Many Such patterns are indicative of a transfer from a progressive 0066. The state assignments are used to determine which Source format to an interlaced one. The most common are fields to use to create the progressive video sequence, as known as 3:2 pulldown and 2:2 pulldown. Many other described below. cadences exist, although they are in less common use than 0067. According to certain embodiments of the present the two already cited. Examples include 2:3:3:2, 2:2:2:4, invention, other factors may be also used in determining the US 2007/0052846 A1 Mar. 8, 2007

presence or absence of a repeating field sequence. One Such remain constant and could, for example, change back and factor is a “pair quality metric.” This metric can be assigned forth between common patterns such as 3:2 or 2:2 pulldown when two sequential 2-field difference comparisons are without losing lock. made of fields that come from different original source 0071. Some factors which could cause loss of lock frames. In Such a case, one 2-field difference value is a include scene transitions which cause the 1-field state to comparison between the even field of original source frame become “None,” or both state values being “None,” or very “N” and the even field of original source frame "N+1, while poor quality metrics, or conflicting 1-field and 2-field State the other is a comparison of the odd fields of those two values. Recognized cadences that can cause lock to be source frames. For instance, referring to FIG. 9, the 2-field acquired are 2:2, 3:2, 3:3, 4:4, 5:5, 4:2, 2:2:2:4, 2:3:3:2. difference value 628 is a comparison between the odd field 3:2:3:2:2, 6:4 and 8:7. 604 of original source frame 601 and the odd field 612 of 0072 At block 481 of FIG. 5, method 400 determines original source frame 611, while the 2-field difference value how to construct the deinterlaced output frame from the 630 is a comparison between the even field 606 of original sequence of fields based on the State value assigned to the source frame 601 and the even field 614 of original source field. Accordingly, the current and previous fields are com frame 611. Because both 2-field differences 628 and 630 are bined if the current field is at the start or in the middle of a comparing the same two original Source frames (601 and repeated sequence. The previous and second previous fields 611), their values will be very similar. If they are not similar, are combined if the current field is at the end of a repeated then either the fields being compared are not from just two sequence. Furthermore, motion-adaptive deinterlacing pro source frames, or there is a problem of some type with the cedure can be performed on the current and previous field video signal. Based on the similarity of the two 2-field pair if there appears to be no valid combination of fields to difference values, a quality metric can be assigned to the reconstruct an original source frame. measurement. Thus, the method 400 of FIG. 5 determines, 0073. Once lock has been acquired, field pairs are com when performing two sequential 2-field correlations of fields bined to form a deinterlaced output stream. The lock state which come from different original source frames, if the alone does not solely enable this combination however. 2-field correlation values are consistent with even/even and Rather, multiple factors must be present for field pairs to be odd/odd field comparisons from 2 original source frames. combined to form the output. These factors include lock, 0068 The pair quality metric is only assigned when the current and previous states, dynamic range and field pair 1-field and/or 2-field state values indicate that the current quality, and cadence. and previous 2-field difference values were each comparing 0074 There are “strong” and “weak” detections for the at least one field from the same source frames. This second various field combination possibilities. A strong detection of the pair of 2-field difference values occurs when the state generally occurs when both the 1-field and 2-field measure is determined to be “End, and the first of the pair occurs in ments are in agreement and the quality metrics are not too the previous field period. The magnitude of the difference low. Weak detections occur when only one of the 1-field or between the pair of 2-field difference values is compared to 2-field measurements is valid and requires that the quality the larger of the two values. Based on the comparison result, metrics are high. There can be a prioritized sequence of a quality value (e.g., very good, good, medium, bad, very decisions regarding which fields to combine, with strong bad) is assigned to that pair. detections having precedence and the weak detections being valid only if the strong detections are not. This prevents a 0069. At block 471.based on the correlation history, state, weak detection decision criteria from being used when a any detected cadence, dynamic range quality, pair quality higher-confidence, strong detection is present. and scene transition values, the method determines if a lock can be acquired on the signal. A lock is considered to have 0075. As described above, when the current state is occurred on the signal when the indicators as a whole “Start” or “Middle, then the current and previous fields are provide a high confidence that a field sequence has come combined. When the state is “End,” the previous and second from a progressive source by extracting multiple fields from previous fields are combined. When the 2-field state is each original Source frame. In other words, the signal has “InBetween (and there is no conflicting 1-field state), the been recognized as having a valid repeating-field patternand first of the two "InBetween states is treated as a “Start” and field pairs can be combined to deinterlace the video signal. the second is treated as an "End,” When no valid overall A lock can be acquired on a sequence as long as a repeating system state is determined to exist, then the lock signal is de-asserted, combination of field pairs stops, and the system pattern is detected, regardless of what the pattern actually is. drops back to motion-adaptive deinterlacing. At block 491,a 0070. In order for lock to occur, a repeating pattern must deinterlaced frame is output. exist. Ideally, the 1-field and 2-field difference values both agree on the pattern type. In certain cases, only one of these 0076 All of the various measurements and quality met is required for lock as long as other quality metrics are rics, particularly as described in reference to FIG. 5, are not Sufficiently high. Once lock has initially occurred, the sys absolutely required. Indeed, a Subset of factors and mea tem stays in lock until Some other event causes lock to be Surements may be used. The more factors that are consid cleared. In general, it is harder to acquire lock than to lose ered, however, the more confidence that can be achieved in it. The basic idea is that there needs to be a high confidence the decision. in the signal before lock is set and field pairs are combined 0077. While FIG. 3 and the corresponding discussion to form the output, but that once lock has occurred, many above provide a general description of a suitable environ types of variations in the sequence pattern can be tolerated. ment in which embodiments of the invention may be imple Once the system is in lock the cadence does not need to mented, features of embodiments of the present invention US 2007/0052846 A1 Mar. 8, 2007

disclosed herein may be practiced in association with a 6. The method recited in claim 5, wherein performing a variety of different system configurations. Embodiments of correlation operation between field pairs further comprises: the invention may be implemented in hardware, Software, or any combination thereof, whether now existing or to be performing a difference operation between field pairs that developed in the future. Where desired, the operations of the are spaced one field apart. various embodiments of the present invention may be incor 7. The method recited in claim 5, wherein performing a porated into computer-executable instructions that may be correlation operation between field pairs comprises: stored on a computer-readable medium. performing a difference operation between field pairs that 0078 Embodiments of the invention can take the form of are spaced two fields apart. instructions, such as program modules, being executed by 8. The method recited in claim 2, wherein determining if processors. Generally, program modules include routines, field pairs of the interlaced video stream are similar further programs, objects, components, data structures, etc., that comprises: perform particular tasks or implement particular abstract determining whether a scene transition has occurred. data types. The sequences of instructions implemented in a 9. The method recited in claim 5, wherein determining if particular data structure or program module represent field pairs of the interlaced video stream are similar further examples of corresponding acts for implementing the func comprises: tions or steps described herein. calculating a threshold value based on one or more 0079. As used herein, the term "embodiment’ means an factors, wherein the factors comprise a history of embodiment that serves to illustrate by way of example but correlation operation values between field pairs, a mini not limitation. Also, reference has been made to an image mum and maximum value in the history of correlation represented by pixels. However, in other embodiments, the operation values, a minimum and maximum value for image can be represented by any convenient and/or known a range of correlation values, and detected Scene tran discrete component that forms the basic unit of the compo sitions. sition of an image. 10. The method recited in claim 9, wherein determining if 0080. It will be appreciated to those skilled in the art that field pairs of the interlaced video stream are similar further the preceding examples and embodiments are exemplary comprises: and not limiting to the scope of the present invention. It is comparing a correlation value for a field pair with the intended that all permutations, enhancements, equivalents, threshold value, wherein a correlation value smaller and improvements thereto that are apparent to those skilled than the threshold value indicates a field pair that is in the art upon a reading of the specification and a study of from a common source. the drawings are included within the true spirit and scope of 11. The method recited in claim 10, wherein determining the present invention. It is therefore intended that the fol if field pairs of the interlaced video stream are similar further lowing appended claims include all Such modifications, comprises: permutations and equivalents as fall within the true spirit examining a history of comparison values obtained from and scope of the present invention. comparing correlation values with the threshold value What is claimed is: to determine if a repeating pattern exists in the inter 1. A method of deinterlacing an interlaced video stream, laced video stream. comprising: 12. The method recited in claim 11, wherein determining if field pairs of the interlaced video stream are similar further detecting an occurrence of groups of adjacent fields that comprises: are derived from a common original image frame Source; and assigning a state value to a current field, wherein the state value is used to determine whether two fields can be merging field pairs of the interlaced video stream. properly merged. 2. The method recited in claim 1, wherein detecting an 13. The method recited in claim 11, wherein determining occurrence of groups of adjacent fields derived from a same if field pairs of the interlaced video stream are similar further original image frame in the interlaced video stream com comprises: prises: acquiring a lock on the repeating pattern. determining if field pairs of the interlaced video stream 14. The method recited in claim 3, wherein merging field are similar. pairs of the interlaced video stream further comprises: 3. The method recited in claim 2, further comprising merging field pairs of the interlaced video stream if the field determining how to construct the deinterlaced output pairs of the interlaced video stream are determined to be frame from the sequence of fields. similar. 15. The method recited in claim 14, wherein motion 4. The method recited in claim 2, further comprising adaptive deinterlacing is used if no repeating pattern exists. deinterlacing the interlaced video stream using motion adap 16. The method recited in claim 6, further comprising: tive deinterlacing if the field pairs of the interlaced video phase shifting field pairs to result in two spatially coin stream are determined to be different. cident calculated fields. 5. The method recited in claim 2, wherein determining if 17. The method recited in claim 6, further comprising: field pairs of the interlaced video stream are similar com performing a vertical low-pass filtering operation on each prises: field of the field pairs to remove noise and aliasing performing a correlation operation between field pairs. artifacts. US 2007/0052846 A1 Mar. 8, 2007

18. The method recited in claim 11, wherein determining detecting and locking onto a repeating pattern in the if a repeating pattern exists in the interlaced video stream groups; and comprises: combining field pairs based on the repeating pattern. assigning a quality value to a field pair representing a 20. A computer-readable medium having stored thereon level of similarity between correlation values of field computer-executable instructions to deinterlace image sig pairs. nals, the computer-executable instructions comprising: 19. A method of deinterlacing an interlaced video stream, detecting an occurrence of groups of adjacent fields that comprising: are derived from a common original image frame Source; and detecting an occurrence of groups of consecutive tempo merging field pairs of the interlaced video stream. ral fields of interlaced video fields that are derived from a common original image frame source: k k k k k