sign in sign up
<<
Home , FITS, JPEG 2000, Motion JPEG, Motion JPEG 2000

JPEG 2000

Abstract

The data collection capability of sensors for space science missions is rapidly growing. Similar high data rate and volume issues have been successfully addressed in earth observing and commercial missions over the last several years. These solutions are applicable to space science missions as well. Intelligent data management has been developed by ITT based on JPEG 2000 (J2K) as well as interactive streaming for multiple applications. ITT has incorporated this into ground-based image libraries, airborne platforms, and is currently incorporating it into a small satellite application. These developments along with JPEG 2000’s scalability make this the right time to explore it as a solution for an expanding space science need.

SUBMITTED BY Point of Contact: Bernard V. Brower ITT Corporation 1447 St. Paul Street P.O. Box 60488, Rochester, NY 14606-0488 Phone: 585.269.5079 Email: [email protected]

Page 1 of 11

Executive Summary ITT Space Systems Division (ITT) has over 50 years experience in design, development, and delivery of space and airborne remote sensing payloads and information systems. As in the commercial world, the sensors for space science mission are continuing to grow in capability. The sensor subsystems are including more sensors, with higher resolution, more radiometric resolution, higher frame rate, and more bands. The data collection capability of these sensors is growing faster than the ability to transmit the data to users. Therefore, a more intelligent data management is required. ITT has developed an intelligent data management system based on JPEG 2000 (J2K) and interactive streaming for multiple applications. We have incorporated this into ground-based image libraries, airborne platforms, and are currently incorporating it into a small satellite application. The intelligent data management system is based on the principle of only delivering the information required to achieve the goal of the user. JPEG 2000’s scalability and JPEG 2000 Interactive Protocol (JPIP) interactive streaming enable the delivery of imagery information based on resolution, quality, and region of interest (ROI). The intelligent data management system uses standard interfaces for the selection of the information that is important. The selection of what is important can be as simple as the user’s visual selection of regions or as complicated as onboard detection of targets. ITT is a world leader in JPEG 2000 technology with over 20 years experience in . JPEG 2000 compression out-performs most other compression in compression efficiency and in functionality. ITT has invested over $6 million in JPEG 2000 imaging solutions over the last 10 years. These JPEG2000 solutions are the cornerstones of ITT’s transformational intelligent data management architecture. This architecture has been used on the ground for everything from scientific applications and disaster relief to military applications. JPEG 2000 and JPIP enable full interactive streaming that is not available with any other standard compression technique. Another advantage of using open commercial standards instead of proprietary techniques and specialized standards (such as the Consultative Committee for Space Data Systems (CCSDS) recommended standards) is that the data can be read and displayed by commercial imaging systems such as ENVI Software and . This white paper discusses the JPEG 2000 compression standard and the possible development opportunities for taking the current technology to a level that could impact future NASA missions. The recommended future research and development would take advantage of ITT’s current investment to develop a space-qualified JPEG 2000 application-specific integrated circuit (ASIC) and/or add three-dimensional compression to the current intellectual property.

Page 2 of 11

Introduction As space science mission data collection capability continues to grow. the limiting factor will be the ability to communicate the data to the ground. Future systems are projected to be able to collect pixels at rates orders of magnitude greater than today. Such systems include the following: • Star Formation Camera (SFC) at over 3 billion pixels, • Joint Dark Energy Mission (JDEM) has over 0.5 billion pixels • European Space Agency (ESA) Gaia has over 1.0 billion pixels Any communication limitation will lead to tasking and collecting fewer missions than the systems are capable of providing. ITT proposes a more intelligent use of the system by including an intelligent data management system on board the spacecraft. The concept is to collect everything possible but deliver only the information that is important to the user. The selection of the “important” information can be performed by simple onboard processing or by user interaction from the ground. In contrast, suppose a mission is forced to decide to take one image over another to save on bandwidth. Information may be lost forever if that second image, which was not taken, contained critical mission information. With an intelligent data management, both images are taken but only the information that is critical from each image is transmitted to fit within the bandwidth. This may be as simple as sending lower resolution or lower quality imagery to the ground following which the users on the ground select regions of interest to increase quality or resolution that still within the bandwidth. ITT has demonstrated this technique for ground system and airborne systems with our JPEG 2000 technology. One of the enabling aspects of JPEG 2000 is this ability to collect and compress data numerically lossless and selectively stream data to the users at lower quality and lower resolution. Full quality and resolution of regions of interest may be streamed to users without re- transmitting the previous information. Other aspects of JPEG 2000 that meet the space science mission include the ability to compress data from one bit to 16 bit inputs and the extensibility to include multiple components (three dimensional data) such as spectral or temporal data. The three dimension compression defined in JPEG 2000 Part 2, can be performed numerically lossless or lossy with all of the same scalability available in the baseline of JPEG 2000. This white paper describes the JPEG 2000 standard capability, ITT’s JPEG 2000 technology, and possible research and development recommendations. Image Compression Image compression has been a critical JPEG 2000 Advantages component of all remote sensing and image • Improved compression efficiency delivery systems. Image compression is the • 5 – 15 % improvement art and science of reducing the number of bits required to represent the information in • Increased functionality an image. This is accomplished by • Resolution scalability reducing/eliminating the redundant • Quality scalability information used to describe an image. • Region of interest access Image compression is a relatively young • Optimized streaming discipline starting in the 1950s with Shannon’s information theory. Once the theory was established different compression schemes arose, including DPCM, DCT, , , and Arithmetic

Page 3 of 11

coding. The most successful ISO standard is the JPEG DCT standard currently used in every , cell phone, computer, DVD player, and most other electronics. The more recent standard developed by the same ISO group is the extremely powerful and versatile JPEG 2000. Still Imagery Still imagery is often compressed using one of several common compression techniques. Depending on the originator of the data, requirements, and the dissemination path a particular algorithm was chosen – perhaps for bandwidth reasons or image quality reasons. The current still imagery compression standards available include Rice algorithm, DPCM, Consultative Committee for Space Data Systems (CCSDS) recommended standard 121.0-B-1 (), ADDCPM (ITT downlink for all commercial remote sensing systems), JPEG-LS, JPEG-DCT, and CCSDS recommended standard 122.0-B-1 (Wavelet Compression). JPEG 2000 outperforms all of these compression algorithms for lossy data compression and has more functionality. JPEG 2000 JPEG 2000 has been shown to provide Improved compression decreases superior compression performance (i.e., storage and bandwidth requirements, superior image quality at equivalent bit-rates) which reduces cost. relative to all currently used still image compression techniques. Additionally, JPEG 2000 provides a fundamental change in how image compression is used. Figure 1 illustrates the traditional compression/decompression model where the decoder application could only restore an image to the exact bit rate and resolution chosen at the time of compression.

Encode

ENCODER CHOICES DECODER CHOICES • • No Decoder Choices • Quantization • Post-Processing • Entropy Encoder • Pre-Processing

Figure 1: Current compression paradigm Under the new JPEG 2000 paradigm, data is progressively layered within the compressed codestream. Several pointer mechanisms are available in JPEG 2000 that allows a server or client extremely fast access to the information of interest. An intelligent client or server application can instantly navigate through the compressed codestream to find exactly the data needed. Figure 2 shows the new compression paradigm in which a single

Page 4 of 11

JPEG 2000 encoded file can be access/decoded at any resolution, quality, or region of interest. The JPEG 2000 standard was developed to address common problems found among various disciplines. Remote sensing standards bodies such as the Open Geospatial Consortium (OPGC), National Imager Transmission Format Standards (NITFS) and NATO STANAG 4545 have adopted the JPEG 2000 standard. The medical community, through the standards body DICOM (Digital Imaging and Communications in Medicine), has also adopted the new compression technology. The U.S. motion picture industry, through its Initiatives, LLC has selected JPEG 2000 as the standard to distribute digital motion pictures to theaters (4K-by-2K, 30 bits/pixel, 24 frames per second).

Encode

ENCODER CHOICES DECODER CHOICES • Tiling •Image Resolution • Lossless/Lossy •Image Fidelity • Old Paradigm choices • Region-of-Interest •Fixed Size • Components

Figure 2: New compression paradigm JPEG 2000 Scalability advantage The true power of JPEG 2000 is the flexibility and scalability of the codestream. Scalability in JPEG 2000 comes in several different forms: resolution scalability and image quality scalability. The building blocks of JPEG 2000 are the , bit plane arithmetic encoding, and embedded bit stream. The wavelet transform pyramidal design nature eliminates the need for secondary reduced resolution data set generation. The bit plane arithmetic encoder of JPEG 2000 enables image quality

Page 5 of 11

scalability (also referred to as layering). Within a single file J2K can instantly have access to compression ratios from 2:1 lossless compression up to 100:1 (or greater) . The JPEG 2000 codestream syntax and embedded bit stream allows access to both the resolution and quality scalability while enabling fast region-of-interest (ROI) access at given quality and resolution. Enable Resolution Scalability Resolution Scalability enables fast Resolution scalability is important when access to multiple resolutions viewing very large images. High quality displays are typically limited to 2K × 2K pixels (quick zoom in and out) and monitors are commonly limited to High Definition (HD) resolution. Thus, displaying either still imagery or motion video larger than 2K × 2K cannot easily be accomplished. With a J2K stream, the built in reduced resolution data could be used for quick searching or of a scene (contextual overview). When needed, high-resolution data at a given ROI can be pulled up and displayed. A client application can simply (and smartly) choose which resolution should be used given a compressed JPEG 2000 image or Motion JPEG 2000 video codestream, without any additional processing costs. Enable Quality Scalability Quality scalability within JPEG 2000 allows the same image and motion image files to be Quality Scalability enables pre- and disseminated to multiple users with different post-compression tradeoff and exploitation or bandwidth needs. For example, if selection of data rate and quality an analyst is only concerned about receiving a “lay of the land” overview, the J2K architecture would simply parse out the pre-made reduced resolution version of the image at reduced quality. Another user may request the same image at full resolution for secondary visual evaluation. If this is done, the full resolution image at a visually lossless quality or even at high quality with some visual loss (i.e. about 8:1 compression ratio) could be delivered. Finally, a third user may request the image and demand radiometric accuracy (per pixel integrity). In this case, provided the image was compressed losslessly, the entire data set could be delivered to the user. The parsing of data sets also applies to the Motion JPEG 2000 standard as well. It should be pointed out that many features of these standards are dependent upon the methodology and storage of the initial compression criteria. In other words, one cannot recreate data that has been thrown away. Enable Region-of-Interest Access Region-of-interest access is useful for Region-of-Interest access enables transmission and decoding purposes. JPEG fast access to any region, at any 2000 enables access to regions of an image at a quality, and resolution. desired resolution and quality. This functionality allows for an encoder or library to send only the requested ROIs from an image without decoding any compressed data. The embedded bit stream also allows a client exploitation system to selectively decode the only display region of a compressed file. Both features can greatly reduce the memory and bandwidth requirements of an exploitation system.

Page 6 of 11

Intelligent use of bandwidth with JPEG 2000 Interactive Protocol (JPIP) JPIP enables the intelligent access to the scalability features through Another important part of the JPEG 2000 suite TCP/IP (client/server –web access) of standards is the JPIP standard (ISO/IEC 15444-9). JPIP, which operates over TCP/IP, is a client-server protocol that provides the capability for a client to interact with a large image or video sequence on a server. The JPIP client sends requests to a JPIP-enabled server to deliver a given region of interest in a streaming fashion to the client. The client may specify the desired spatial region, resolution, quality, and set of bands it wishes to receive. If a “stateful” session is initiated between the client and server, the server will mirror the client’s codestream cache to track what information the client has already received. Thus, the server need not continually resend data over and over as the user roams around an image or changes resolution (e.g. zooms). ITT has demonstrated the JPIP protocol along with a custom viewer application to browse large images over an 802.11b wireless link on PDA device. Implementation of JPIP-enabled solutions significantly extends the reach of the user to access very large imagery stored in central repositories. ITT JPEG 2000 Technology ITT has several products, intellectual property and expertise involving JPEG 2000 compression. Image Access Solutions (IAS) was developed for interactive dissemination of large images to users. This is currently being used by several customers and applications including the online delivery of HiRISE imagery1. ITT has invested in JPEG 2000 IP for future hardware development. This J2K IP has been targeted to an avionics field-programmable gate array (FPGA) board and successfully supports a flow in which the sensor was collecting 132 megapixels per second (over 1.5 gigabits per second) and streaming regions of interest to multiple users over a 48 megabits-per-second communication link. Image Access Solutions (IAS) The IAS products are commercial products that are supported by ITT Visual Information Solutions (VIS), a wholly owned subsidiary of ITT. All IAS products are commercial off- the-shelf products (COTS), fully documented and supported by VIS’s team of technical sales and support professionals. The conversion, management, and dissemination of accurate and timely geospatial information are growing needs for today’s intelligence and scientific communities. The advantages of the IAS platform include: • Faster Image Delivery: IAS delivers NITFS/JPEG 2000 imagery from server to client exponentially faster than conventional methods. Streaming data technology allows IAS to display multi-gigabyte sized images on demand, regardless of bandwidth constraints. • Immediate Image Access: The enhanced JPIP streaming capability of IAS lets a user begin viewing an image almost immediately, allowing him or her to focus on an area of interest, while the remainder of the image continues to load in the background. Image transmission can be stopped any time there is enough information to analyze.

1 HiRISE imagery can be streamed using IAS solution at http://hirise.lpl.arizona.edu/

Page 7 of 11

• SCOTS bases: All IAS components are standards-based commercial off-the-shelf (SCOTS) products, fully documented and certified for NITFS 2.1 with support for the JPEG 2000 NITFS Profiles (NPJE and EPJE). The IAS solution contains several components: • IAS Server/Converter (Windows, Solaris): ITT offers JPEG 2000 server and compression engine software for high-performance image libraries and archives. The server supplies management and dissemination capabilities for JPEG 2000 imagery. The compression engine provides the capability to convert images from multiple formats to standard JPEG 2000 profiles. The compression engine was designed to be multithreaded and works at about two megapixels per second per thread. • IAS Viewer (Java-webstart): Workstation-based, client software that enables users to receive and visualize streaming JPEG 2000 compressed images. JPEG 2000 IP ITT’s JPEG 2000 Compression Core, (internal codename - “CoreFire”), is a high performance, soft-core implementation of JPEG 2000 specifically designed to run on an FPGA or developed as a custom ASIC. Its intended use is real-time compression for airborne and space-based applications. It provides configurable, high-quality compression using a 9/7 two-dimensional discrete wavelet transform (2D DWT) in lossy mode, and 5/3 2D DWT in lossless mode. The synchronous, pipelined design consists of parameterized VHSIC (Very High Speed Integrated Circuits) Hardware Description Language (VHDL) source code, retargetable to common FPGA and ASIC technologies that support appropriate memory and logic resources. The CoreFire “Core” is a technology independent, JPEG 2000 soft IP core for use in applications for which quality is paramount at high compression ratios (low bit rates). The feature set of JPEG 2000 includes post compression bit rate modification, region of interest enhancement, multi-resolution data within the JPEG 2000 stream, progressive image enhancement and embedded thumbnails, in addition to other features. The CoreFire Core provides acceleration for the computationally intensive Tier I portion of the JPEG 2000 standard. The CoreFire Core supports compression of image data with a pixel depth of 1 to 16 bits with a minimum compression rate of 50 megapixels per second in any mode when implemented in the equivalent of a Xilinx Virtex 4 FPGA, or 0.25 micron ASIC technology. Maximum data rates are determined by maximum clock rates achievable for the target technology and the mode of operation selected. Achievable rates may be significantly higher when the core is targeted to later FPGA or ASIC technologies. Compression options include 9/7 filtering option in lossy mode, and 5/3 in lossless mode. Higher data rates can be achieved when operating in lossy modes, depending upon quantization parameters. Up to six levels of wavelet decomposition can be selected. The core supports up to 23 bit-planes per quantizer index, configurable per sub-band, for any supported tile or code-block size. Recommended Research and Development Areas The following recommendations expand on ITT’s current technology in which we have invested over $6 million in internal investments. The JPEG 2000 hardware development and demonstration is currently being used on airborne platforms and ground-processing systems. There are two logical next steps in the technology roadmap. First is the addition

Page 8 of 11

of multiple component compression to the current technology. Second is to take the VHDL design to a full space qualified ASIC design. Extension to 3D Compression ITT has demonstrated the 3-dimensional compression available in JPEG 2000 Part 2 for hyperspectral image cubes2. This project would evaluate the different 3D compression techniques available in Part 2 and how they impact the space science missions. This action would enable the hardware to take advantage of the correlation of information in the third dimension. This would include the lossless and lossy compression of three- dimensional data such as spectral images and large temporal images. Once a technique or techniques are selected, the impact on the hardware design would be evaluated. Three- dimensional wavelet transform could be implemented into the hardware or a more complicated linear transform technique. Optimized VHDL code would be developed and incorporated into the current CoreFire IP. The hardware would need to be optimized depending on the requirements for the compression ratio, number of components, and size of the images. Space Qualified ASIC Taking the current JPEG 2000 technology (with or without the addition of the 3D compression) to a space-qualified ASIC would require a design stage and a fabrication stage. The CCSDS recommended standard 122.0-B-1 was developed to take advantage of wavelet compression without the complexity of fully implementing JPEG 2000. Current space-qualified ASIC technology will allow JPEG 2000 to meet the throughput (pixels- per-second) power requirement ratio that was main driver in the development of the four- year-old CCSDS 122.0-B-1 standard. ITT has significantly invested in the development of our VHDL JPEG 2000 soft core and is continuing to invest for future applications. Taking this technology from its current status to a fully space-qualified ASIC could have a significant impact on how future space missions could be tasked and data collected. The intelligent data management would enable missions to collect more data while selectively disseminating only the mission-critical information.

2 T. S. Wilkinson, J. H. Kasner, B. V. Brower, and S. S. Shen, “Multi-component Compression Options in JPEG-2000 Part II,” SPIE Proceedings on Applications of Processing XIV Vol. 4472, July 2001 Brower, B. V., A. Lan, J. Kasner, S. Shen, “Multiple component compression within JPEG 2000 as compared to other techniques,” SPIE Proceedings on Applications of Digital Image Processing XXIII Vol. 4115, July 2000

Page 9 of 11

Annex A: JPEG 2000 Image Compression Standards There are several parts to the ISO/IEC 15444, of which the most important are the Core coding system (part 1), Extensions (part 2), Motion JPEG 2000 (part 3), and the JPIP interactive protocols (Part 9). The following is a complete list of current standards and standards being developed by the JPEG committee for the JPEG 2000 suite of standards. Current Standards • Part 1: Core coding system • Part 2: Extensions • Part 3: Motion JPEG 2000 • Part 4: Conformance • Part 5: Reference software • Part 6: Compound image • Part 7:has been abandoned • Part 8: JPSEC: Security • Part 9: JPIP: Interactive protocols and API • Part 10: JP3D: Volumetric imaging • Part 11: JPWL: Wireless • Part 12: Common format text (Motion J2K, MPEG 4) • Part 13: Standard encoder Annex B: ITT’s Compression Experience Compression ITT has been a leader in image compression for over 20 years, dating back to before the original JPEG DCT development. ITT has experience in the development, optimization, testing, and characterization of compression algorithms for all applications. ITT’s compression portfolio includes several patents, an Emmy, dozens of technical papers, and over 10 dedicated image compression experts. ITT has been supporting the Government with image compression since the start of digital imaging. Standards and Leadership ITT has been involved with the standardization of commercial compression standards since the early years of the development of the JPEG standard. Bernie Brower of ITT has been the chairman of the Inter-National Committee for Information Technology Standards (INCITS)/L3.2 “Still Image Coding” and United States Head of Delegation (HOD) to the Emmy-Award-winning ISO/JPEG Committee since 1997. ITT has been part of the US Government’s National Imagery Transmission Format Standards (NITFS) Technical Board (NTB) since the development of NITFS 1.1 in the 80s. Currently ITT is under contract to be the chairman of the NTB Bandwidth Compression (BWC) Working Group. ITT delivered the first NITFS JPEG DCT software that was used by the Navy during Desert Storm. ITT has been responsible for the development, documentation, and optimization of the current compression algorithms within the NITFS suit. ITT was responsible for the documentation of MIL-STD-188-198, MIL-STD-188-196, MIL-STD-188-197, and the Bandwidth Compression Standards and Guidelines Document (N-0106-97). ITT is currently responsible for the development of the JPEG 2000 migration plans for NITFS and the Distributed Common Ground Station (DCGS). ITT is currently developing and documenting the JPEG 2000 Profile for the NITFS and BIFF standards, which will be registered with ISO/IEC SC 24. ITT has also

Page 10 of 11

been involved with the inclusion of this profile into the NATO community for the STANAG 4545 and 7023 standards. ITT developed the down sampled JPEG (NIMA Method 4) compression algorithm, which produced compression efficiency close to that of JPEG 2000 at very low bit rates without adding significant computation to the JPEG standard. Compression Optimization ITT has a deep understanding of the users of the data and how this relates to image quality and compression. ITT has developed extensive tools in the optimization of all types of image compression algorithms (DPCM, DCT, Wavelet). Our thorough Human Visual System (HVS) models are incorporated into many of our optimization tools. ITT is responsible for the optimization of most of the current compression algorithms used in the NSGI architecture (1.3 DCT, 2.3 DCT, NITFS JPEG, NIMA Method-4, and the developing JPEG 2000 standard). Image Quality and Compression Characterization ITT’s understanding of image quality and what impacts it dates back to the original development of film. ITT has been evaluating image quality for the Government for over 50 years. One of the most important impacts in image quality since the use of digital imagery has been image compression. ITT has been involved with the evaluation of the current and past image compression techniques used with the NSGI architecture.

Page 11 of 11