Layered Compression Technologies for Digital A Research Project in the 6th Framework of the European andCommunity Cross Media Conversion

15 Feb 07 1 Motivation

• Optimized workflow for the complete production chain for in 4k does not exist

• DCI specification does not deal with production (only deals with the backend of the chain)

• Digital Cinema is a chance for the European industry

• It requires an international consortium to solve the fundamental issues together A Research Project in the 6th Framework of the European Community

15 Feb 07 2 Project Overview WorldscreenTM

• Objective: Investigation of the use of scalable coding in digital cinema workflows • Duration: 09/2004 – 04/2007 • Partners: 6 + 2 advisory (, Warner Bros.)

• Project Coordinator:

• Budget: 6 Mio. €, Funding: 3 Mio. € • Funded by the European Community A Research Project in the 6th Framework of the European Community (6th Framework) • Website: http://www.worldscreen.org

15 Feb 07 3 Project Partners

A Research Project in the 6th Framework of the European Community

15 Feb 07 4 Scalable Video Coding

For acquisition, postproduction, distribution and archiving

Encoding Scalable Bitstream

SNR Spatial A Research Project in the 6th Framework of the European Community Scalability Scalability

15 Feb 07 5 Technical Workpackages

WP2 WP3 Simulation WP4 Implementation WP5 and Tests and Integration User Assess- Requirements, ment Specifications and and Concepts WP3.1 Compression WP4.1 Evaluation for Data Acquisition Compression for Data Acquisition

WP4.3 Integration and System Test

WP3.3 Transmission, WP4.2 Distribution Transmission, Distribution

WP3.2 Postproduction for Cross Media Distribution

A Research Project in the 6th Framework of the European Community

15 Feb 07 6 Project Details Layered scheme in Digital Cinema workflows

User Requirements Business Models

Acquisition Postproduction Distribution Exhibition

JPEG2000- Use of Scalability JPEG2000/MXF JPEG2000

A Research Project in the 6th Encoder for processing / Standardisation Decoder Framework of the European Fieldrecorder for output to target Community Transmission Quality tests formats

15 Feb 07 7 Layered scheme data compression in Digital Cinema workflows

Acquisition Postproduction Distribution Exhibition

JPEG2000- Use of Scalability JPEG2000/MXF JPEG2000

A Research Project in the 6th Encoder for processing / Standardisation Decoder Framework of the European Fieldrecorder for output to target Community Transmission Quality tests formats

15 Feb 07 8 Layered scheme data compression in Digital Cinema workflows

Acquisition Postproduction Distribution Exhibition

A Research Project in the 6th Framework of the European Community

15 Feb 07 9 Project Achievements

• Developed workflow and metadata models for the entire production chain with LSC • Developed a field recorder for providing a portable storage solution for future D-Cinema cameras including a JPEG2000 encoder • Developed postproduction and transcoding methods (e.g. for (HD-)DVD, TV, streaming formats) from LSC master • Developed scalable distribution and decoding systems (DCI compliant) • Developed a quality assessment methodology for Digital Cinema A Research Project in the 6th Framework of the European Community • Supported research and standardization of layered scheme data compression (both intra and inter) for D-Cinema

15 Feb 07 10 Layered Compression Technologies for andA Research Project Cross in the 6th Media Conversion Framework of the European Community

11 Goals – Methodology - Results

• The investigation of the use of layered scheme data compression throughout the entire DC chain and for rich media archives required: – Identification of user requirements – Specification of workflows and metadata models (current & future) – Evaluation of business models

• Methodology: – Interviews with 52 users and manufacturers – Structuring & Consolidation of market feedback

•Results – Deliverables with specifications for workflows and metadata

A Research Project in the 6th – Deliverables covering cost comparisons and business Framework of the European Community models for the value chain

12 Activity Diagrams

Digital Intermediate Post Production LAB Telecine Scan

roll get date rough cut EDL format "circled take" scene numbers numbers type generate pull list

negative developing film scan pull list

original selected film rolls develop clean negative takes

negative digital scans telecine scan

audio telecine & DAT sound synchronize tapes cc (P3) timing report

video tape 444 10bit log DCI compliant DC files HD & SD deliverables encoding for projection review e.g. for rough cut

A Research Project in the 6th Framework of the European UML activity diagrams were extracted from the information Community from the interviews – these formally describe the most typical workflows.

13 Acquisition with JPEG2000 (J2K)

Camera Fieldrecorder Postproduction-SW & with J2K Encoder Plugins

J2K-Import J2K-Export

Uncompressed Fieldrecorder (Megacine)

Display, Keyboad

VTR-Control Dual HD-SDI

J2K-Encoder Dual HD-SDI

Dual J2K- I/O-Modul DVI/Audio HD-SDI Realtime Fibrechannel Encoder (future (near IEEE1394 10G) lossless)

Storage- A Research Project in the 6th Framework of the European Modules Power Community (HDD or Flash)

14 JPEG2000 Workflow

Postproduction Mastering/Validation

DCP- DCP- Generator Analyser

J2K-Export DCP/KDM- DCP- Generator Validator

• Development of Softwaretools for JPEG2000 – Input/Output-Plugins for Digitalfusion – DCP/KDM Generation Tools – DCP/KDM Validation Tools

• Concepts for Archives based on JPEG2000 (will be continued in EU-Project EDCINE)

A Research Project in the 6th Framework of the European • Future Need: J2K-Hardwaresupport in Postproduction Community

15 What is Transcoding all about?

• Conversion of theatrical content into entertainment video formats, e.g.: – SDTV (PAL, NTSC) – HDTV Additional Sources – Streaming video (internet, mobile) of Revenue –etc.

A Research Project in the 6th Framework of the European Community Theatrical Content Entertainment Video

16 Transcoding Workflow – Example

Universal 2K Master (e.g., 2048x1080 pel, R‘G‘B‘ 4:4:4, 12 bit) Transcoding

A Research Project in the 6th Framework of the European Entertainment Video Master Community (1280x720 pel, YUV 4:2:0, 8 bit)

17 Digital Cinema Packages – DCI Model

JPEG 2000 Encoded < 250 Mbps Both Projections + = from the same AES Encrypted < 400 GBytes for DCP + a 3 hour Movie MXF Packaged

2K Projection Hard disk, Satellite... DCP Theater Storage 4K Projection

Decryption Keys sent in parallel

A Research Project in the 6th Key delivery messages protected with RSA Framework of the European Community Only the target projection system is able to decode the messages

18 Digital Cinema Packages – DCI Model

MXF Files

Reel 1

Chunks Image of Data AssetMap PackingList to Composition MXF to reconstruct verify PlayList integrity to put the each Track Audio file of all needed reels together files MXF

Theater Storage

Reel 1 Reel 2 Reel 3 Show Playlist ... Advertisement schedules presentation Reel 1 Reel 2 Reel 3 ... Trailer A Research Project in the 6th Framework of the European Community Reel 1 Reel 2 Reel 3 Projection ... Movie

19 Deutsche Telekom @ WorldScreen

T-Systems is a full service powered by: provider for Digital Cinema

Digital Cinema Factory™: Reliable delivery of Digital Cinema Packages via satellite

Digital Cinema ™: Server Solution for Multiplex Theatres

A Research Project in the 6th Framework of the European Community

20 Deutsche Telekom @ WorldScreen

Ingest-Archive Operational-Archive

Ingest-ServerIngest-Server DelDel i very-Server i very-Server

LAN

Communication-Communication- Database-ServerDatabase-Server VPN Internet GatGat eway eway

OperatiOperati on-Server on-Server

Digital Cinema Factory™

A Research Project in the 6th Framework of the European Within the WorldScreen project, software components for 4K Community DCP delivery have been developed, co-funded by the EC and the Deutsche Telekom Laboratories

21 J2K Decoder Architecture

Gigabit Ethernet Audio Connection Connector

Input “2k Module Audio data Board”

BUS PCI 32bits@66MHz

JP2 JP2 JP2 JP2 Module Module Module Module #1 #3 #5 #7

JP2 JP2 JP2 JP2 Module Module Module Module #2 #4 #6 #8 Output Module HD-SDI/GbE Serial rapid IO or Projector

This architecture must to be able to decode the 2K at

A Research Project in the 6th Framework of the European 24fps, 2K at 48fps and 4K at 24fps profiles; for the 4K, Community or future higher profile, some parallel 2K-Boards will be necessary.

22 DSP-FPGA Coprocessor in the Decoder Chain

• FPGA performs bit-oriented operation on Tier 1 (computationally expensive) and Inverse Quantization. • DSP completes the decoding chain, performing IDWT on large images. A Research Project in the 6th Framework of the European Community • FPGA acts as coprocessor for DSP, in a loosely coupled architecture.

23 Image Quality Assessment

Kodak Digital Cinema stands for highest image quality, analog AND digital – Multi-format Cineserver JMN3000 with 3D support – Digital Processing / Delivery of over 40 full feature – Comprehensive Theater Management System – Over 2100 pre-show advertising systems installed

• Motion Quality Ruler (MQR) is a new method for assessing image quality of motion sequences – Can be used to judge wide quality range efficiently and precisely

• MQR method was implemented and validated at a special Kodak test facility in Paris

• MQRs were created for a wide range of scene content and

A Research Project in the 6th origination sources that would be encountered in practice Framework of the European Community

24 Motion Quality Rulers

• MQRs were verified and subsequently applied to test scenes compressed with JPEG2000 at compression ratios of 6:1 to 100:1 • Results indicate feasibility to devise MQRs for individual scenes that can be used to judge quality degradation from compression over large quality range

Fire Engine - 1.4 JND sensitivity Bike - 1.4 JND sensitivity

6:1 12:1 25:1 50:1 75:1 100:1 6:1 12:1 25:1 50:1 75:1 100:1 1

0

-1

-2

-3

-4

JNDs -5

-6

-7

-8 A Research Project in the 6th Framework of the European Community -9

-10 Error bars are 95% Student's t-confidence intervals with respect to the actual number of judges

25 Layered scheme data compression in Digital Cinema workflows

User Requirements Business Models

Acquisition Postproduction Distribution Exhibition

JPEG2000- Use of Scalability JPEG2000/MXF JPEG2000 Encoder for processing / Standardisation Decoder Fieldrecorder for output to target Transmission Quality tests A Research Project in the 6th formats Framework of the European Community

26 Screening - Content

• Advertising Clips –ZKM-Spot (ZKM) – BMW X5 (BMW) –Rimowa (Rimowa)

• Trailer – Happy Feet (Warner Bros.) – Harry Potter (Warner Bros.) – We are Marshall (Warner Bros.)

• Excerpts from Feature Films/Testmovies – Cool Hand Luke* (Warner Bros.) – Ocean´s 11* (Warner Bros.) – As good as it gets (ARRI/Bill Benett) – Mystic India (BAPS/Keith Melton)

A Research Project in the 6th * available on DVD Framework of the European Community

27 Screening - Technology

• DCPs (Digital Cinema Package) – generated and tested by Fraunhofer IIS

• Players – T-Systems Digital Cinema Feature FilmTM – KODAK JMN3000 Digital Cinema Server

• 4k Projector – SRX-R110

A Research Project in the 6th Framework of the European Community

28