(12) United States Patent (10) Patent No.: US 8,300,701 B2 Flecchia (45) Date of Patent: Oct
Total Page:16
File Type:pdf, Size:1020Kb
US008300701 B2 (12) United States Patent (10) Patent No.: US 8,300,701 B2 Flecchia (45) Date of Patent: Oct. 30, 2012 (54) OFFSPEED PLAYBACK INAVIDEO EDITING 2006/0184684 A1* 8, 2006 Weiss et al. ................... TO9,231 SYSTEM OF VIDEO DATA COMPRESSED * cited by examiner USING LONG GROUPS OF PICTURES Primary Examiner — Twyler Haskins (75) Inventor: Mark Flecchia, Burlington, MA (US) Assistant Examiner — Brendan McCommas (73) Assignee: Avid Technology, Inc., Burlington, MA (74) Attorney, Agent, or Firm — Oliver Strimpel (US)US (57) ABSTRACT (*) Notice: Subject to any disclaimer, the term of this An editing system handles offspeed and full speed playback patent is extended or adjusted under 35 of long GOP compressed video information by having one U.S.C. 154(b) by 1471 days. decoder for full-speed or slower forward playback of long GOP compressed motion video information, and a second (21) Appl. No.: 11/298,866 single-frame, stateless decoder for use for off-speed play back. When compressed data is read from a file, the com (22) Filed: Dec. 9, 2005 pressed data for at least one complete group of pictures is read O O into a buffer, herein called a compressed data buffer. A data (65) Prior Publication Data structure, herein called a frame ring, is constructed to US 2007/O133951A1 Jun. 14, 2007 describe each image in the compressed data buffer. The single-frame, stateless decoder receives decompression (51) Int. Cl. parameters in addition to the compressed data for each image H04N 7/2 (2006.01) from this buffer. If the compressed data is a B-frame or (52) U.S. Cl. ......... 375/240.25:375/240.12; 375/240.26; P-frame, then uncompressed data for any other images (i.e., 375/240.28; 725/86; 725/91 reference frames) required by the decoder to decompress the (58) Field of Classification Search .............. 375/240.25 frame also is input to the single-frame, Stateless decoder. If See application file for complete search history. the uncompressed data for the reference frames is not already in a cache, these reference frames are decompressed and the (56) References Cited uncompressed data for the reference frames as used by the decoder is stored in the cache prior to the decompression of U.S. PATENT DOCUMENTS the B-frame or P-frame. 6,735,253 B1* 5/2004 Chang et al. ............. 375,24O16 2004/0220791 A1* 11/2004 Lamkin et al. .................. TO3/11 17 Claims, 4 Drawing Sheets Toi Compressed Compressed Long GOP Decompressed Doto From Frame Ring Dot Streaming Stream of Doto Fie co Decoder images Single Frame Decompressed 109-State Stoteless f16 Information Image Decoder Controller Decompressed Reference 114 Frome Coche Type of fts Operotionp Position of 1 Desired Picture U.S. Patent Oct. 30, 2012 Sheet 2 of 4 US 8,300,701 B2 -o 2O2 Yes Map Position of Streaming Desired Picture to Playback a Group of Pictures p and a Sample No 2O4 2O3 Map Position of the Intiate Streaming Desired Picture to o Playback GOP and a Somple 2O6 Load Group(s) of Pictures into Frame Ring 2O3 Set Toil Ond Hedd Postions -210 214 Reference Decompress Fromes in Reference Fromes Coche 216 Add Decompressed Decompress Desired Reference Fromes Picture to Coche F/G, 2 US 8,300,701 B2 1. 2 OFFSPEED PLAYBACK IN AVIDEO EDITING grams. Editing systems generally permit an editor to specify SYSTEM OF VIDEO DATA COMPRESSED a sequence of scenes, where each scene is defined by a refer USINGLONG GROUPS OF PICTURES ence to Video information (called a source) and a range within that video information (usually specified by a starting image BACKGROUND or mark-in point and an ending image or mark-out point). Generally, an editing system permits an editor to specify any There are several formats for compressing motion video image within a source as a start or stop point. When Such a information. Some compression formats use intraframe com sequence of scenes is played back, the editing system identi pression, while other compression formats use both inter fies the group of pictures within which each start point for a frame and intraframe compression. A compression format 10 that uses both interframe and intraframe compression is com scene is located, and decompresses at least part of that group monly called a long GOP compression format because a of pictures that precedes the desired start point. This decom sequence of images is broken into several groups of images (a pression adds complexity to an editing system that uses long group of pictures or GOP) that are compressed together. In GOP compressed motion video information. particular, in each group of pictures, at least one image is 15 There are other operations that an editor may perform intraframe encoded and is typically called an I-frame. Some while editing a video program that become more complex images are compressed based on a difference from a prior when the editing system uses long GOP compressed motion image. These images are referred to as being predictively Video information. These operations include reverse play, fast encoded or P-frames. Yet other images are compressed based forward play, jogging, shuttling, scrubbing, trimming and on a difference between two other images. These images are single frame playback. When playing back long GOP com referred to as being bidirectionally encoded or B-frames. pressed motion video information at a rate that is faster than I-frames and P-frames also are called reference frames. full-speed playback (such as for shuttling or jogging opera Some long GOP compression formats are included in stan tions), or in reverse, or by accessing a single frame (such as dards Such as, but not limited to, the standards commonly for scrubbing or trimming operations) each frame to be known as MPEG-2 (formally known as ISO/IEC 13818-2 or 25 played backis, in essence, randomly accessed from the stored ITU-T Rec. H.262) and H.264 (formally known as ISO/IEC compressed bitstream. These kinds of playback operations 14496-10 or ITU-T Rec. H.264). Other long GOP compres (faster than full-speed, reverse playback and single frame sion formats may borrow concepts from or use parts of these playback) are referred to as “off-speed playback. Because standards. For example, a format known as HDV encodes a most frames require compressed data from more than one high-definition image sequence using MPEG-2 encoding. 30 frame to be decompressed, random access of each individual Other example long GOP compression formats include frame during offspeed playback can require significant pro MPEG-4, AVC and VC-1. MPEG-2 permits groups of pic cessing. tures to be compressed using only I-frames, a combination of I-frames and P-frames, or a combination of I-frames, SUMMARY P-frames and B-frames. 35 The compressed data resulting from compressing motion Video information usually is stored in a data file. An example An editing system handles offspeed and full speed play data file format is the Material eXchange Format (MXF), back of long GOP compressed video information by having which is the subject of the standardization process of the one decoder for full-speed or slowerforward playback of long Society of Motion Picture and Television Engineers 40 GOP compressed motion video information, and a second (SMPTE), and is defined in documents including SMPTE single-frame, stateless decoder for use for off-speed play S377M through 394M, with SMPTE Engineering Guidelines back. When compressed data is read from a file, the com EG41 and 42. pressed data for at least one complete group of pictures is read Image index information for compressed motion video into a buffer, herein called a compressed data buffer. A data information is used to map the temporal position of an image 45 structure, herein called a frame ring, is constructed to in decompressed motion video information to a position of its describe each image in the compressed data buffer. The corresponding compressed data in the compressed bitstream. single-frame, stateless decoder receives decompression Temporal positions used with the image index information parameters in addition to the compressed data for each image correspond to the desired playback order. The image index from this buffer. If the compressed data is a B-frame or information may include a known amount of data for each 50 P-frame, then uncompressed data for any other images (i.e., image if the compression process produces the same amount reference frames) required by the decoder to decompress the of compressed data for each compressed image. If the com frame also is input to the single-frame, Stateless decoder. If pression process produces a different amount of data for each the uncompressed data for the reference frames is not already compressed image, which is typical for long GOP compres in a cache, these reference frames are decompressed and the sion formats, then an image index is created and used. 55 uncompressed data for the reference frames as used by the In some long GOP compression formats, such as MPEG-2, decoder is stored in the cache prior to the decompression of the temporal order of the compressed images is different from the B-frame or P-frame. the order in which the compressed data appears in the com The cache may use a least-recently-used replacement strat pressed bitstream. Thus the image index maps the temporal egy, and may hold uncompressed data for a number of refer order of images to the bitstream order of the compressed data 60 ence frames. For example, for HDV material, the cache may for the images and to the position of the compressed data in be size to store uncompressed data for about five to seven the compressed bitstream.