[standards in a NUTSHELL] Frédéric Dufaux, Gary J. Sullivan, and Touradj Ebrahimi The JPEG XR Image Coding Standard PEG XR is the newest image cod- baseline JPEG, but raw encoding has imaging and other flexible image inter- ing standard from the JPEG com- very high storage capacity requirements, action usage scenarios. mittee. It primarily targets the is generally camera specific, and typically JPEG XR’s architecture reflects the representation of continuous-tone lacks interoperability and published for- new requirements specific to high/ still images such as photographic mat documentation. extended dynamic range functionalities. Jimages and achieves high image quality, JPEG XR (ITU-T T.832 | ISO/IEC The traditional baseline JPEG coding on par with JPEG 2000, while requiring 29199-2) is a new image coding system format uses a bit depth of eight for each low computational resources and storage primarily targeting the representation of the three red green blue (RGB) color capacity. Moreover, it effectively address- of continuous-tone still images such as channels, resulting in 256 values per es the needs of emerging high dynamic photographic images. It is designed to channel or 16,777,216 color values. range imagery applications by including address the limitations of today’s for- However, more demanding applications support for a wide range of image repre- mats and to simultaneously achieve may require a bit depth of 16, providing sentation formats. high image quality while limiting com- 65,536 representable values for each putational resource and storage capacity channel or over 2.8 * 1014 color values BACKGROUND requirements. Moreover, it aims at pro- for a three-channel RGB image. viding a new set of useful image coding Additional scenarios may necessitate MOTIVATION capabilities focused around high/extend- even greater bit depths and sample rep- Fast-evolving digital imaging technolo- ed dynamic range imagery. The “XR” resentation formats. Conversely, when gies have made digital photography a part of the JPEG XR name evokes the memory or processing power is at a tremendous success with consumers and intent of the design to apply to an premium, as few as five or six bits per professionals. At the heart of this success “extended range” of applications beyond channel may be used. To fulfill these story, the baseline JPEG image coding the capabilities of the original baseline needs and others for a very broad vari- standard (ITU-T T.81 | ISO/IEC 10918-1) JPEG standard. JPEG XR is based on the ety of applications, JPEG XR has been has played a key enabling role. However, “HD Photo” technology developed by designed to include support for a wide while it has become one of the most Microsoft to address the demands of range of image representation pixel for- widely used standards in the world, the consumer and professional digital pho- mats, even including support for the nearly 20-year-old JPEG technology is tography. It is expected that JPEG XR floating-point and “radiance” formats reaching its limits and has begun to hin- will help pioneer inventive and ground- sometimes used in the most demanding der the development of innovative fea- breaking products and services in the HDR applications. tures and performance enhancements in marketplace, and will bring widespread Besides this special focus on HDR digital photography. More recently, the added value for consumers and profes- imagery applications, other major JPEG JPEG committee produced the sionals alike. XR design objectives include better com- JPEG 2000 standard (ITU-T T.800 | ISO/ pression for enhanced quality, a unified IEC 15444-1), introducing a number of OBJECTIVES system for lossless and lossy compres- novelties and new functionalities. JPEG XR is a new compression format sion, cost-effective computational per- However, it has notably higher computa- supporting high dynamic range (HDR) formance, and new progressive decod- tional resource requirements and has formats for a new generation of digital ing features for more powerful image not made a notable impact in mobile and cameras and other imaging appli- access and manipulation. embedded environments such as the dig- cations to bring a new level of success- Finally, the JPEG committee has en- ital photography market. Serious pho- ful user experience. It is designed to couraged patent holders to allow JPEG tographers have been switching to using give cost and compression benefits for XR to be implementable free of royalty “raw” image encoding for many purposes consumer, “pro-sumer,” and profes- and license fees to foster widespread to avoid the technology limitations of sional digital photography. On top of adoption of the specification and help en- that, it also has core code stream fea- sure that it can be implemented by the Digital Object Identifier 10.1109/MSP.2009.934187 tures that support interactive Web widest number of organizations. 1053-5888/09/$26.00©2009IEEE IEEE SIGNAL PROCESSING MAGAZINE [195] NOVEMBER 2009 standards in a NUTSHELL continued [ ] ISSUING BODY, STRUCTURE OF final standard in both ITU-T (where it TARGET APPLICATIONS THE STANDARD AND SCHEDULE is known as ITU-T Recommendation JPEG XR is intended for broad use in a The Joint Photographic Experts Group T.832) in March 2009 and in ISO/IEC very wide range of digital image han- (JPEG) is a working group that produces (as ISO/IEC 29199-2) in June 2009. dling and digital photography applica- joint standards of three major internatio- ■ Part 3: Motion JPEG XR: This part tions. Key application target areas nal standardization organizations: specifies the use of JPEG XR encod- include the following: the International Organization for ing for stored sequences of moving ■ Robust and high-fidelity image Standardization (ISO), the International images with associated timing infor- acquisition technologies, such as a Electrotechnical Commission (IEC), and mation. The Motion JPEG XR file wide range of camera applications. the International Telecommunication format is based on the ISO Base Using JPEG XR, a more accurate rep- Union Telecom Sector (ITU-T). JPEG is Media File Format standard. It is cur- resentation of the full range of cap- universally recognized as the leading rently a committee draft (CD) in the tured image signal fidelity can be committee for compressed image for- ISO/IEC approval process, and it is retained, avoiding the bottleneck mats, and it is responsible for the popular scheduled to reach final approval sta- introduced by the older baseline JPEG, JBIG, JPEG-LS, and JPEG 2000 tus in 2010. JPEG standard and avoiding the high families of imaging standards. ■ Part 4: Conformance testing: This storage capacity and interoperability The JPEG committee began the stan- part specifies a set of tests designed to difficulties associated with camera dardization of JPEG XR technology in verify whether code streams, files, raw image formats. July 2007. The initial design proposal encoders, and decoders meet the nor- ■ High dynamic range imaging was submitted by Microsoft, based on its mative requirements specified in Part workflows, including editing soft- HD photo technology, while the subse- 2. The tests specified provide methods ware suites and high-capability quent development and future evolution to (nonexhaustively) verify whether image processing pipelines. The abil- of JPEG XR as a standard has been the encoders and decoders meet these ity of JPEG XR to retain an extended responsibility of the JPEG committee. requirements. This part has currently dynamic range with a high degree of Under the general title “Information reached the status of final committee signal fidelity can help prevent loss Technology – JPEG XR Image Coding draft (FCD) in the ISO/IEC approval of quality in end-to-end workflow System,” the JPEG XR image coding sys- process and is on target to reach final environments. tem suite of standards (ISO/IEC 29199) approval in 2010. ■ Computationally constrained sig- consists of the following current and ■ Part 5: Reference software: This nal processing environments, such planned parts: part provides reference software for as mobile and embedded applica- ■ Part 1: System architecture: This Part 2. The reference software can aid tions. As the computational de - part is a nonnormative technical adoption of standard by providing an mands associated with JPEG XR report (TR), describing an overview of example implementation that can be images are substantially lower than different parts of the specifications used as a basis for making encoder with other coding systems such as and providing some guidelines on best and decoder products and can be used JPEG 2000 that also provide high encoding and decoding practices. It is to test conformance and interopera- quality capability. now a working draft (WD), and it is bility as well as to demonstrate the The use of JPEG XR for HDR imag- expected to become a formally pub- capabilities of the associated standard. ing is illustrated in Figure 1, as well as lished TR in 2010. It includes both encoder and decoder a comparison with JPEG and “raw” ■ Part 2: Image coding specification: functionality. This part is now at FCD encoding. When using JPEG, tone This part specifies the JPEG XR image status and is scheduled to reach final mapping or other image adjustments coding format. It was approved as a approval in 2010. must properly be applied prior to encoding, as the encoding results in a substantial irreversible loss of infor- Tone JPEG JPEG Low-Quality mation. If image adjustments are Mapping Encoding Decoding Image Rendering applied to an image after encoding and decoding it using the baseline JPEG format, serious quality degradation Tone High-Quality HDR Source Raw Mapping Image Rendering will often be evident. Raw encoding effectively supports HDR, but at the cost of high storage requirements and JPEG XR JPEG XR Tone High-Quality interoperability problems. JPEG XR Encoding Decoding Mapping Image Rendering successfully overcomes these short- comings, enabling HDR while keeping [FIG1] JPEG XR for high dynamic range imaging.