Contents

Contributor Biographies xvii

Foreword xxix

Series Editor’s Preface xxxiii

Preface xxxvii

Acknowledgments xli

List of Acronyms xliii

Part A

1 JPEG 2000 Core Coding System (Part 1) 3 Majid Rabbani, Rajan L. Joshi, and Paul W. Jones 1.1 Introduction 3 1.2 JPEG 2000 Fundamental Building Blocks 5 1.2.1 Preprocessing 6 1.2.2 The Discrete Wavelet Transform (DWT) 7 1.2.3 Quantization 22 1.2.4 Entropy Coding 24 1.3 JPEG 2000 Bit-Stream Organization 32 1.3.1 Canvas Coordinate System 32 1.3.2 Resolution Grids 33 1.3.3 Precinct and Code-Block Partitioning 34 1.3.4 LayersCOPYRIGHTED and Packets MATERIAL 35 1.3.5 Packet Header 36 1.3.6 Progression Order 39 1.3.7 Code-Stream Organization and Syntax 43 1.4 JPEG 2000 Rate Control 46 1.4.1 Rate Control Using an Explicit q-Table 46 1.4.2 Rate Control Using the EBCOT Algorithm (PCRD-opt) 47 vi Contents

1.5 Performance Comparison of the JPEG 2000 Encoder Options 49 1.5.1 Lossy Results 51 1.5.2 Lossless Results 56 1.5.3 Bit-Plane Entropy Coding Results 59 1.6 Additional Features of JPEG 2000 Part 1 61 1.6.1 Region-of-Interest (ROI) Coding 61 1.6.2 Error Resilience 64 1.6.3 File Format 65 Acknowledgments 66 References 66

2 JPEG 2000 Extensions (Part 2) 71 Margaret Lepley, J. Scott Houchin, James Kasner, and Michael Marcellin 2.1 Introduction 71 2.2 Variable DC Offset 73 2.3 Variable Scalar Quantization 74 2.3.1 Theory 75 2.3.2 Signaling 75 2.4 Trellis-Coded Quantization 76 2.5 Precinct-Dependent Quantization 79 2.6 Extended Visual Masking 80 2.6.1 Theory 80 2.6.2 Signaling 81 2.6.3 Interactions 81 2.7 Arbitrary Decomposition 82 2.7.1 Theory 83 2.7.2 Implementation Hints 83 2.7.3 Signaling 84 2.8 Arbitrary Wavelet Transforms 84 2.8.1 Transform via Lifting 84 2.8.2 Boundary Extension 85 2.8.3 Signaling 85 2.9 Multiple-Component Transform Extensions 86 2.9.1 Interactions 87 2.9.2 MCT Framework 87 2.10 Nonlinear Point Transform 93 2.10.1 Relationship to Other Annexes 94 2.10.2 Nonlinear Transform 94 2.11 Geometric Manipulation via a Code-Block Anchor Point (CBAP) 94 2.12 Single-Sample Overlap 95 2.12.1 Theory 95 2.12.2 Comments 96 2.12.3 Signaling 97 2.13 Region of Interest 97 2.13.1 Theory 98 Contents vii

2.13.2 Implementation Details 99 2.13.3 Signaling 100 2.14 Extended File Format: JPX 100 2.14.1 Encoding versus Interpretation 100 2.14.2 File Format Scope 101 2.14.3 Packaging all this Extra Data 101 2.14.4 Specifying Color in JPX 102 2.14.5 Metadata 104 2.14.6 Other Features 104 2.14.7 Summary 104 2.15 Extended Capabilities Signaling 105 Acknowledgments 105 References 105

3 Motion JPEG 2000 and ISO Base Media File Format (Parts 3 and 12) 109 Joerg Mohr 3.1 Introduction 109 3.2 Motion JPEG 2000 and ISO Base Media File Format 110 3.3 ISO Base Media File Format 110 3.3.1 Boxes 110 3.3.2 File Structure 111 3.4 Motion JPEG 2000 112 3.4.1 Motion JPEG 2000 Samples 112 3.4.2 Profiles 114 3.4.3 Compliance Points and Testing 115 3.4.4 Using Motion JPEG 2000 116 References 118

4 Compound Image File Format (Part 6) 121 Frederik Temmermans, Tim Bruylants, Simon McPartlin, and Louis Sharpe 4.1 Introduction 121 4.2 The JPM File Format 123 4.3 Mixed Raster Content Model (MRC) 123 4.3.1 Introduction 123 4.3.2 Layout Object Generation 123 4.3.3 Layout Generation 125 4.3.4 Object Clipping and Positioning 126 4.3.5 Blending 128 4.3.6 Page Organization and Collections 128 4.4 Streaming JPM Files 129 4.5 Referencing JPM Files 130 4.6 Metadata 131 4.7 Boxes 131 4.8 Profiles 131 4.9 Conclusions 131 References 134 viii Contents

5 JPSEC: Securing JPEG 2000 Files (Part 8) 135 Susie Wee and Zhishou Zhang 5.1 Introduction 135 5.1.1 Overview 135 5.1.2 Media-Aware Security 135 5.1.3 Scalable Coding of Media: The Structure of JPEG 2000 Image Data 136 5.1.4 Example Application for Scalable Images 136 5.1.5 Applying Security to Media 137 5.1.6 JPSEC Media-Aware Security Tools 137 5.2 JPSEC Security Services 137 5.2.1 Overview 137 5.2.2 Confidentiality Service 138 5.2.3 Integrity Service 138 5.2.4 Authentication Service 138 5.3 JPSEC Architecture 139 5.3.1 What Security Service Is Provided? 139 5.3.2 Where Is the Security Tool Applied? 139 5.3.3 How Is the Security Tool Applied? 140 5.4 JPSEC Framework 140 5.4.1 A JPSEC System 140 5.4.2 JPSEC Stream 141 5.5 What: JPSEC Security Services 143 5.5.1 Overview 143 5.5.2 Confidentiality Methods 143 5.5.3 Integrity Methods 143 5.5.4 Authentication Methods 143 5.5.5 Key Template 143 5.6 Where: Zone of Influence (ZOI) 144 5.6.1 Description Classes 144 5.7 How: Processing Domain and Granularity 146 5.8 JPSEC Examples 148 5.8.1 Example: Encryption by Resolution 149 5.8.2 Example: Authentication by Resolution 151 5.8.3 Example: Combining Encryption and Authentication 153 5.9 Summary 156 References 157

6 JPIP – Interactivity Tools, APIs, and Protocols (Part 9) 159 Robert Prandolini 6.1 Introduction 159 6.2 Data-Bins 164 6.2.1 Streaming Data 164 6.2.2 Defining Data-Bins 165 6.2.3 Defining Metadata-Bins 166 Contents ix

6.3 JPIP Basics 168 6.3.1 Sessions, Channels and Cache Models 168 6.3.2 Behaviors 171 6.3.3 Compliance 173 6.4 Client Request–Server Response 174 6.4.1 Target, Session and Channel Identification 175 6.4.2 View-Window Region 177 6.4.3 View-Window Requests 179 6.4.4 Server Response Controls 180 6.4.5 Cache Management 183 6.4.6 Metadata Requests 187 6.5 Advanced Topics 189 6.5.1 JPIP Proxy Server 189 6.5.2 Upload and Version Control 190 6.5.3 JPIP on Multiple Code-Streams 191 6.5.4 Advanced Behaviors 192 6.5.5 JPIP Indexing 195 6.6 Conclusions 195 Acknowledgments 196 References 196

7 JP3D – Extensions for Three-Dimensional Data (Part 10) 199 Tim Bruylants, Peter Schelkens, and Alexis Tzannes 7.1 Introduction 199 7.2 JP3D: Going Volumetric 200 7.2.1 Preprocessing 201 7.2.2 The 3-D Discrete Wavelet Transform (3-D DWT) 202 7.2.3 Quantization 205 7.2.4 Bit Modeling and Entropy Coding 205 7.3 Bit-Stream Organization 207 7.3.1 The Three-Dimensional Canvas Coordinate System 207 7.3.2 Code-Stream 214 7.3.3 Rate Control 217 7.4 Additional Features of JP3D 218 7.4.1 Region of Interest 218 7.5 Compression performances: JPEG 2000 Part 1 versus JP3D 219 7.5.1 Test Setup 219 7.5.2 Lossless Compression 221 7.5.3 Lossy Compression 222 7.5.4 Time Complexity 223 7.5.5 Conclusions 225 7.6 Implications for Other Parts of JPEG 2000 225 7.6.1 Volumetric Extension to JPIP 226 Acknowledgments 226 References 226 x Contents

8 JPWL – JPEG 2000 Wireless (Part 11) 229 Fr´ed´eric Dufaux 8.1 Introduction 229 8.2 Background 230 8.2.1 Transmission Errors and Their Impact on Compressed Data 230 8.2.2 Error Detection, Resilience, Concealment, and Correction 231 8.2.3 Error Resilience Tools in JPEG 2000 Baseline 232 8.3 JPWL Overview 233 8.3.1 Scope 233 8.3.2 Main Functionalities 234 8.3.3 System Configuration 234 8.4 Normative Parts 236 8.4.1 Error Protection Capability (EPC) 237 8.4.2 Error Protection Block (EPB) 238 8.4.3 Error Sensitivity Descriptor (ESD) 240 8.4.4 Residual Error Descriptor (RED) 242 8.4.5 Registration Authority (RA) 242 8.5 Informative Parts 243 8.5.1 Error Resilient Entropy Coding 243 8.5.2 Unequal Error Protection (UEP) 246 8.6 Summary 246 Acknowledgments 247 References 247

Part B

9 JPEG 2000 for Digital Cinema 251 Siegfried Fo¨ßel 9.1 Introduction 251 9.2 General Requirements for Digital Cinema 253 9.2.1 General Requirements 253 9.2.2 Additional Requirements in the Acquisition Area 254 9.2.3 Additional Requirements in the Postproduction Area 254 9.2.4 Additional Requirements in the Archive Area 255 9.2.5 Summary 255 9.3 Distribution of Digital Cinema Content 255 9.3.1 Digital Cinema Initiatives, LLC (DCI) 255 9.3.2 System Concepts and Processing Steps 257 9.3.3 Digital Cinema Package (DCP) 260 9.3.4 Standardization of D-Cinema Distribution 261 9.3.5 JPEG 2000 D-Cinema Distribution Profiles 262 9.4 Archiving of Digital Movies 263 9.4.1 EDCine 264 9.4.2 Requirements for Digital Movie Archives 264 9.4.3 System Concepts and Archival Workflows 265 Contents xi

9.4.4 Archive Packages MAP and IAP 266 9.4.5 Standardization of Archive Profiles 267 9.4.6 JPEG 2000 Archive Profiles 267 9.5 Future Use of JPEG 2000 within Digital Cinema 269 9.5.1 Acquisition 269 9.5.2 Postproduction 269 9.5.3 Mastering with JPEG 2000 269 9.5.4 Enhanced Distribution Profiles 270 9.6 Conclusions 271 Acknowledgments 271 References 272

10 Security Applications for JPEG 2000 Imagery 273 John Apostolopoulos, Fr´ed´eric Dufaux, and Qibin Sun 10.1 Introduction 273 10.1.1 Media Security versus Data Security 273 10.1.2 Different Types of Security Services 274 10.1.3 Chapter Overview 274 10.2 Secure Transcoding and Secure Streaming 275 10.2.1 Motivation 275 10.2.2 Secure Transcoding for JPEG 2000 and JPSEC 276 10.2.3 Security Properties 277 10.2.4 Summary 278 10.3 Multilevel Access Control 278 10.4 Selective or Partial Encryption of Image Content 279 10.5 Image Authentication 280 10.5.1 Motivation 281 10.5.2 A Unified Digital Signature Scheme for JPEG 2000 282 10.5.3 Image Authentication for Surveillance 287 10.5.4 Summary for Image Authentication 289 10.6 Summary 289 Acknowledgments 289 References 289

11 Video Surveillance and Defense Imaging 291 Touradj Ebrahimi and Fr´ed´eric Dufaux 11.1 Introduction 291 11.2 Scrambling 292 11.2.1 Image-Domain Scrambling 293 11.2.2 Transform-Domain Scrambling 293 11.2.3 Code-Stream-Domain Scrambling 294 11.3 Overview of a Typical Video Surveillance System 294 11.4 Overview of a Video Surveillance System Based on JPEG 2000 and ROI Scrambling 296 11.4.1 Explicit Region of Interest (Max-shift) 296 11.4.2 Implicit Region of Interest 297 xii Contents

11.4.3 Scrambling of ROIs 298 11.4.4 Unscrambling of ROIs 299 11.4.5 Experimental Results 300

12 JPEG 2000 Application in GIS and Remote Sensing 305 Bernard Brower, Robert Fiete, and Roddy Shuler 12.1 Introduction 305 12.2 Geographic Information Systems 305 12.2.1 What Is Remote Sensing? 307 12.2.2 History of Remote Sensing 307 12.2.3 Sensor Types 310 12.2.4 Applications of Remote Sensing Data 315 12.2.5 JPEG 2000 for GIS and Remote Sensing Applications 317 12.2.6 JPEG 2000 Scalability and Access 317 12.3 Recommendations for JPEG 2000 Encoding 318 12.3.1 Wavelet Filters 318 12.3.2 Resolution Levels 319 12.3.3 Compression Ratio, Quality Layers, and Rate Control 320 12.3.4 Quantization 324 12.3.5 Tiles and Precincts 324 12.3.6 Code-Stream Organization 327 12.3.7 Pointer Marker Segments 329 12.3.8 Other Parameters and Implementation Considerations 330 12.4 Other JPEG 2000 Parts to Consider 334 12.4.1 ISO/IEC 15444-2: Extensions 334 12.4.2 ISO/IEC 15444-3: Motion JPEG 2000 and ISO/IEC 15444-12: ISO Base Media File Format 336 12.4.3 ISO/IEC 15444-4: Conformance Testing 336 12.4.4 ISO/IEC 15444-5: Reference Software 336 12.4.5 ISO/IEC 15444-6: Compound Documents 336 12.4.6 ISO/IEC 15444-8: JPSEC 337 12.4.7 ISO/IEC 15444-9: Interactivity Tools, APIs, and Protocols (JPIP) 337 12.4.8 ISO/IEC 15444-10: 3-D Volumetric Data 338 12.4.9 ISO/IEC 15444-11: Wireless 338 References 338

13 Medical Imaging 341 Alexis Tzannes and Ron Gut 13.1 Introduction 341 13.2 Background 341 13.3 DICOM and JPEG 2000 Part 1 343 13.3.1 Supplement 61 343 13.3.2 Color Transformations and Photometric Interpretations 343 13.3.3 Multiframe Imagery and Fragments 344 Contents xiii

13.4 DICOM and JPEG 2000 Part 2 344 13.4.1 Supplement 105 345 13.5 Example Results 346 13.5.1 Lossless Compression Results 346 13.5.2 Lossy Compression Results 347 13.6 Image Streaming, DICOM, and JPIP 348 13.6.1 Supplement 106 350 13.6.2 JPIP and DICOM Use Cases 350 References 351

14 Digital Culture Imaging 353 Greg Colyer, Robert Buckley, and Athanassios Skodras 14.1 Introduction 353 14.2 The Digital Culture Context 354 14.2.1 Requirements 354 14.2.2 Processes and File Formats 355 14.2.3 Technological Developments 356 14.2.4 Digitization 357 14.3 Digital Culture and JPEG 2000 358 14.3.1 Some Features that Match the Requirements of Archivists 358 14.3.2 Some Features that Match the Requirements of Curators 360 14.3.3 Integration of Preservation and Delivery 361 14.3.4 Some Technological Barriers 363 14.3.5 Exploitability of Designed-In Features 364 14.4 Application – National Digital Newspaper Program 367 14.4.1 JPEG 2000 Code-Stream 368 14.4.2 JP2 File Format 372 Acknowledgments 375 References 375

15 Broadcast Applications 377 Hans Hoffman, Adi Kouadio, and Luk Overmeire 15.1 Introduction – From Tape-Based to File-Based Production 377 15.1.1 The Advent of HDTV and Consequences for TV Production 378 15.2 Broadcast Production Chain Reference Model 380 15.2.1 System Building Blocks in File-Based Media Production 380 15.2.2 Required Functionality and Technical Guidelines 383 15.3 Codec Requirements for Broadcasting Applications 386 15.3.1 Broadcast Content Characteristics 386 15.3.2 Broadcast Applications 389 15.4 Overview of State-of-the-Art HD Compression Schemes 395 15.4.1 Technical Parameters 396 15.4.2 Legacy HD Compression Algorithms for Production 398 15.4.3 Advanced HD Compression Algorithms for Production 398 15.4.4 Summary 402 xiv Contents

15.5 JPEG 2000 Applications 402 15.5.1 Why is JPEG 2000 Interesting for Broadcasting? 402 15.5.2 JPEG 2000 for Production and Archiving 404 15.5.3 JPEG 2000 for Contribution 405 15.5.4 Issues with JPEG 2000 405 15.6 Multigeneration Production Processes 405 15.6.1 Test Setup 406 15.6.2 Results 407 15.7 JPEG 2000 Comparison with SVC 415 15.7.1 SVC Overview 415 15.7.2 JPEG 2000 versus SVC 417 15.8 Conclusion 417 References 418

16 JPEG 2000 in 3-D Graphics Terrain Rendering 421 Gauthier Lafruit, Wolfgang Van Raemdonck, Klaas Tack, and Eric Delfosse 16.1 Introduction 421 16.2 Tiling: The Straightforward Solution to Texture Streaming 424 16.3 View-Dependent JPEG 2000 Texture Streaming and Mipmapping 428 16.3.1 View-Dependent Streaming 428 16.3.2 Mipmaps and Spatial Scalability 428 16.3.3 Data Dependencies in JPEG 2000 and Mipmaps 432 16.3.4 Time-Dependent Mipmap/Clipmap Caching and JPEG 2000-JPIP Streaming 433 16.4 JPEG 2000 Quality and Decoding Time Scalability for Optimal Quality–Workload Tradeoff 435 16.5 Conclusion 439 References 439

17 Conformance Testing, Reference Software, and Implementations 441 Peter Schelkens, Yiannis Andreopoulos, and Joeri Barbarien 17.1 Introduction 441 17.2 Part 4 – Conformance Testing 442 17.2.1 Definition of Profiles 442 17.2.2 Definition of Compliance Classes 442 17.2.3 Encoder and Decoder Compliance Testing 442 17.3 Part 5 – Reference Software 443 17.4 Implementation of the Discrete Wavelet Transform as Suggested by the JPEG 2000 Standard 445 17.4.1 Introduction to the Discrete Wavelet Transform 446 17.4.2 Hardware Designs for the DWT 454 17.4.3 Pointers for Further Study on DWT Implementations 456 17.5 JPEG 2000 Hardware and Software Implementations 458 17.5.1 Kakadu 458 17.5.2 OpenJPEG 459 17.5.3 IRIS Codec 460 Contents xv

17.5.4 Wavescale™ Monolithic IC Implementations from Analog Devices, Inc. 460 17.5.5 Thomson JPEG 2000 Products 464 17.5.6 Barco Silex IP Cores for FPGA and ASIC 467 17.5.7 Alma Technologies JPEG 2k-E IP Core 470 17.5.8 Sesame and Falcon IP Cores by MegaChips 472 17.6 Conclusions 476 Acknowledgments 477 References 477

18 Ongoing Standardization Efforts 481 Touradj Ebrahimi, Athanassios Skodras, and Peter Schelkens 18.1 Introduction 481 18.2 JPSearch 481 18.2.1 Motivations 481 18.2.2 JPSearch Overview 482 18.2.3 JPSearch Architecture and Elements 483 18.2.4 JPSearch Specification 484 18.3 JPEG XR 485 18.4 Advanced Image Coding and Evaluation Methodologies (AIC) 487 References 489

Index 491