DECE Picture Formats

Kilroy Hughes 2009.01.12 Contents

• System model • Picture format variables • Proposal for consistent system of encoding and playback • Issues System Model

*Format conversions can be done and redone at both encoding and playback System Model

• Post production needs to understand what format conversion operations devices will handle, and what constraints they impose; and • Content constraints in addition to those in the h.264 Profiles and Levels should be added to the DECE Profile specs • Devices need to handle all variations of content they are expected to play (logo or practical marketing requirement if content type is popular) • There will always be differences in video rendering quality based on the type of device and its quality of implementation, but any content that meets encoding constraints should play on all devices rated for that Profile (PD, SD, or HD) • Less desirable alternative is to create additional separate content formats and devices (e.g. SD 50Hz, SD 60Hz, SD 4:3, SD 16:9, etc.) The Big System Question

• What format conversions will be required in post production for DECE compliant content? • What format conversions will be required in DECE compliant players? • The goal is that for all content to play on all devices with same or higher Video Profile Some Image format Variables

• Sample Aspect Ratio (NTSC, PAL, anamorphic, sq) • Picture Aspect Ratio • Overscan • Subsampling (4:2:0, 4:1:1, 1440h, etc.) • Interlaced temporal subsampling, filtering • Quantization range (e.g. 16-235, 0-255, +/- 128) • Color space (Rec 601, 709; negative coeficients) • Gamma curve, render intent, display adaptation • Frame rates 15.00p, 23.97p, 25.00i, 29.97i/p, 50.00p, 59.97i/p (1000/1001 video rates) Problem Areas

• Setup, color space, 3:2 pulldown, deinterlace, crop, p/s, letterbox, pillarbox, tiltscan, cylindrical stretch • Common aspect ratios: – 1.33 (4:3 TV), 1.78 (16:9 TV), 1.85 2.0, 2.33, 2.4(film), 16:9 anamorphic – 708x483 SMPTE D1 display aperature – 704x480 ATSC 4:3? – 720x480 DVD 4:3? • Overscan – CRT legacy. New model 1:1 sample to pixel (but reality usually involves scaling). Need to explicitly state SAR in video. “Hypothetical reference display”. • Output signal limitations: e.g. 1280x720 may require 1366x720, etc. connection to allow “overscan” cropping and 1:1 sample to pixel mapping (without scaling and filtering) • Playback of 25i content on 60Hz interfaces/displays PD

• Optimized for simple device (cell phone) and fast download with front end (post production) format conversion • Video format limited to 320x240, square pixel, 4:3 “exact scan” aspect, deinterlaced (if necessary), (601 color space and setup?), 15, 24, 30, 50, 60 frames/s. • Content often requires special editing, framing, processing, etc. for small screen anyway; studio can do better deinterlacing, pan/scan, etc. than device. SD • Optimized for wide range of devices, with post processing in device (high end cell phone, PMP, game device or console, computer, TV STB, internet TV, etc.) • Default is native format encoding (SAR, PAR, i/P, fps, color space 601 with setup). • Device will handle SAR, PAR, interlace, and frame rate conversion according to user preferences and display capabilities. • 2.35 PAR 24fps scope movie might be displayed on PC window or projection at 2.35 PAR 72Hz or 120Hz, flat panel 1.78 PAR LB or “full screen”, 4:3 CRT at 30i with 3:2 pulldown, etc. • Best system is to maintain original content format and encoding information, and let user preferences and device capabilities decide best presentation format. Will vary by device, user, content, and viewing circumstances. Most new devices already convert SAR, interlace, PAR, etc. for flat screens, so devices are capable of conversion. • Attempting to precorrect for limited devices at encoding (e.g. vertical resolution filtering to prevent “CRT flicker”, encoding letterbox bars, encoded pulldown, etc.) usually compromises high end devices (progressive displays, 120Hz, motion interpolation, automatic PAR adaptation, etc.) HD

• Optimized for wide range of content, moderate range of devices (more HD TV content shot for 1.78 PAR, but plenty of 1.33, 1.85, 2.X PAR content. More HD displays 1.78 DAR) • Native format encoding (same as SD, but 709 color space) e.g. 2.33 PAR > 1920x840 • Devices must convert SAR, PAR, interlace, frame rate • Should allow for negative YCrCb coefficients for wide color (e.g. xvYCC) Edge Cases

• Frame rates: – Any frame rate is OK on a self contained device with decoupled refresh or multisync; but NTSC, PAL, HDMI, etc. video connections have limited frame rate options. – STB can output 24Hz to new displays, or 30i to old (3:2 pulldown) – STB can output 24Hz with 3:2 or +4% to 50/60 Hz displays; problem is 25i content to 60Hz only displays. Devices can insert pulldown • Might recommend safe areas based on device cropping to 4:3 • Clarification of “exact scan” pixel dimensions to avoid unnecessary scaling (e.g. 704 or 720 = 4:3? = 16:9 anamorphic?) Display intent must be specified explicitly using correct SAR in h.264