Cinemadng Siggraph 2010 Presentation
Total Page:16
File Type:pdf, Size:1020Kb
The CinemaDNG Initiative Interoperability in Digital Camera Capture Presented at Siggraph 2010 Los Angeles, July 29, 2010 Copyright 2010 Adobe Systems Incorporated Today’s Agenda . Presentation - The CinemaDNG Initiative . Camera demo - ViewPLUS . Panel discussion - Interoperability on the Capture Side Copyright 2010 Adobe Systems Incorporated The CinemaDNG Initiative An industry-wide initiative for An open, raw image format for digital cinematography Lars Borg , Principal Scientist Adobe Systems Copyright 2010 Adobe Systems Incorporated Outline of Presentation . Raw image !le formats . Objectives for CinemaDNG . The road ahead . Slides posted at http://www.adobe.com/go/cinemadng . Questions welcome in the on-line user forum Copyright 2010 Adobe Systems Incorporated Background . Raw is the image !le format of choice for pro photography . Cinematographers now embracing raw . What makes raw so great? . Work"ow impacts? Copyright 2010 Adobe Systems Incorporated What is a RAW image !le? . Starting with Still Photography … . The RAW image captures the digital camera’s sensor data with no processing in the camera . Raw = unprocessed image sensor data . Raw opposite of baked (as in “look or white balance is baked in”) . All image processing is deferred to post . Look, white balance, black level, color grade, tone, … . No on-set commitment . Better !nal quality . 70 % of professional photographers shoot RAW (InfoTrends 2008) Copyright 2010 Adobe Systems Incorporated From scene to camera sensor to RAW image !le Original scene Bayer matrix Light on light on sensor optical color filter sensor pixels (magnified 100x) on the sensor Image processing JPEG image voltage 12-bit digital “No” processing Raw image Copyright 2010 Adobe Systems Incorporated From sensor data to !nal image Raw digital When processed in camera: sensor data - Early binding - JPEG or Video workflow (magnified 100x) Image processing When processed in post: pipeline - Late binding - RAW workflow Dead pixel removal Noise reduction Linearization Black level subtraction Demosaicing to RGB Cropping Final image White balance Still or Video Lens corrections Color conversion Color enhancements Toning curve “Digital Inverse” Gamma curve Bayer matrix Clip to 8-10 bits Lossy compression Colored items = Destructive! Copyright 2010 Adobe Systems Incorporated Destructive image processing . Examples . Black current subtraction . Out = Max ( 0, Image black - Reference black ) . Erroneously high reference black clips shadow details . Noise in image black causes positive bias after clip . White balance to bring neutral objects to R = G = B . Out = Min (1, RGB * RGB channel gains) . Overexposed areas are clipped to 1.0, 1.0, 1.0 to look white . Later rebalancing brings clipped values below 1.0, 1.0, 1.0 Copyright 2010 Adobe Systems Incorporated Quality Challenges in Image Processing . Process-intensive noise reduction & demosaicing . Camera processor is weak (to extend battery life) . With RAW capture - defer to post to get more compute power . Compare quality - noise, edge artifacts Case A Case B Copyright 2010 Adobe Systems Incorporated RAW — Too much of a good thing? .arw .bay . Proliferation of RAW formats .cap .cr2 . For example, 200+ formats supported in Adobe Photoshop, LightRoom .crw .dcr . Processing options coded uniquely for each camera model .dcs .drf . Proprietary formats .erf . Most formats are undocumented .fff .iiq . Several formats are encrypted .k25 . Google “RAW storm in a teacup?” .kdc .mef . The Digital Millennium Copyright Act may block legal access .mos .mrw .nef . A pro photographer’s nightmare .orf .pef . Can your client or publisher read your !le format? .ptx .pxn . An archivist’s nightmare .raf . Some early formats already dead, unsupported, unreadable, DMCA .raw .sr2 .srf .x3f … Copyright 2010 Adobe Systems Incorporated The Digital Negative (DNG) format “to the rescue” . A uni!ed RAW image format - superset of features in 200+ raw formats . Published 2004 by Adobe Systems . Free SDK & tools . Wide adoption - 20% of raw images archived in DNG (InfoTrends 2008) . See http://www.barrypearson.co.uk/articles/dng/products.htm for DNG products . ISO standard 12234-2 TIFF/EP - Next rev includes DNG Copyright 2010 Adobe Systems Incorporated DNG Format Overview . File structure is TIFF/EP (like most Raw !le formats) . File contains complete image & decoding description . Black level, white level, and linearization parameters . Color metadata for sensor colorimetry . Processing controls not tied to camera model . Options include . Single-chip (with color !lter array) or multi-chip (without CFA) image sensor . Arbitrary image dimensions (within 4 Gbyte !le size) . Sample size = 8-bit to 32-bit integer (typically 10 or 12) . Arbitrary-size CFA pattern (Bayer, striped, etc.) with up to seven !lter colors . Non-standard pixel placements (such as Fuji!lm Super CCD EXR) . Arbitrary byte order . Lossless compression (~ 2:1 ratio) . Look controls . TIFF/EP, EXIF and XMP metadata . Encryption is not an option Copyright 2010 Adobe Systems Incorporated RAW Increasingly Embraced in Cinematography .avi .mov . RAW suitable for cinematography .dpx . Set look on set, non-destructive .dng . For example, see Lightroom .r3d .ari . No commitment on set .siv . Deliver look + raw image data to post .cine . Better !nal quality .pgm . Raw images excellent source for green-screen work .wcr .fhgwc .fhgdp . Drawbacks with multiple, proprietary RAW formats .fhgip .fhgii . Already at 30 proprietary formats for digital cinema .fhgjp . Applications must be updated for each new format .fhgji . Some camera vendors already gone .fhgwr . Uncertain or limited interoperability .fhgwd .fhgwe . Expect an onslaught of DSLR raw video formats - 200? .fhgmp . Long-term archiving at risk Copyright 2010 Adobe Systems Incorporated Cinematography Work"ows with Multiple RAW Formats Are you fully connected? Vendor DPX Raw !les HD ACES fhgwc Camera 1 avi fhgdp mov fhgip dpx GRADING fhgii r3d fhgjp Camera... ari fhgji siv fhgwr cine NLE fhgwd pgm fhgwe Camera N wcr fhgmp … VFX Each application must read 30 (200) file formats Copyright 2010 Adobe Systems Incorporated Adding CinemaDNG to Cinematography Work"ows Capture in Deliver in Ingest Vendor Raw or CinemaDNG CinemaDNG CinemaDNG format directly DPX HD Transcode ACES Camera 1 vendor raw !le GRADE Camera... CinemaDNG NLE Camera N VFX Connect all products through one file format Copyright 2010 Adobe Systems Incorporated CinemaDNG format summary . Image essence in DNG format . Complete image description . Each clip is stored as . A folder with a DNG !le sequence or . A single MXF !le (OP1a or OP-Atom) . When wrapped as MXF, a clip can include . Single, Stereo (3D), or more image tracks . 0 - 16 audio tracks . Details at http://www.adobe.com/go/cinemadng Copyright 2010 Adobe Systems Incorporated Example Work"ow - Camera to Academy IIF Capture Deliver in Ingest Edit in RGB raw data CinemaDNG Universal Camera CinemaDNG DNG Input ACES transform CinemaDNG files are self-describing (Multiple) Look Settings On-set Color Look info separate from image Grade Grade in raw or CinemaDNG Copyright 2010 Adobe Systems Incorporated CinemaDNG Objectives . Facilitate source-material exchange between raw-format devices, storage systems, and applications . Leveraging existing (or extended) standards . Image essence = (ISO) TIFF/EP Pro!le 2 - built on Adobe DNG . Wrapping = (SMPTE) MXF Material Exchange Format . Metadata = XMP, TIFF, EXIF metadata sets . Creating an open, documented image format for RAW data from digital motion picture cameras into post-production . Building an open platform for raw image processing tools . Low threshold to entry . No IP encumbrances, no license fees, free SDKs . Not a goal: mastering, distribution stages Copyright 2010 Adobe Systems Incorporated CinemaDNG Status Adobe Systems Avid . 50+ companies participating Cine-tal . CinemaDNG 1.0 format spec released Sep 2009 Fraunhofer IIS . SMPTE Standard in FCD ballot Gamma & Density Ikonoskop . ST 2055 Mapping TIFF/EP Essence into MXF GC Image Engineering . ISO Standard in the works Indiecam . Revising ISO 12234-2 TIFF/EP image data format IRIDAS . IRIDAS SpeedGrade shipped CinemaDNG April 2009 MXF4mac RadiantGrid Technologies . Several cameras now available Silicon Imaging . In next major releases from Adobe & other vendors Synthetic Aperture The Foundry . SDKs, sample plug-ins, sample !les available ViewPLUS Vision Research Weisscam Copyright 2010 Adobe Systems Incorporated Ahead for CinemaDNG Initiative . Product announcements IBC 2010, NAB 2011 . Publish standards and next spec . Further work . Look management work"ow . Real-time playback on laptop . Better demosaicing of moving objects - consistent frame-to-frame . On-set camera (re-)calibration . Metadata set . High-quality “lossy” compression for raw . Further standardizations . Increased adoption . Contributions welcome . No IP encumbrances, no license fees, free SDKs . See http://www.adobe.com/go/cinemadng Copyright 2010 Adobe Systems Incorporated Summary - The Bene!ts of The CinemaDNG File Format . A uni!ed raw image format for motion picture capture . Can replace proprietary RAW formats . Retaining the user bene!ts found in proprietary raw formats . No on-set commitment, unaltered sensor data . Adding interoperability, metadata, standards, long-term viability What’s not to like about this? Copyright 2010 Adobe Systems Incorporated ViewPLUS Lumiere 4K digital camera Toru Nakamura, ViewPLUS Copyright 2010 Adobe Systems Incorporated Lumiere PCIe data camera