Layered Compression Technologies for Digital Cinematography A Research Project in the 6th Framework of the European andCommunity Cross Media Conversion
15 Feb 07 1 Motivation
• Optimized workflow for the complete production chain for Digital Cinema in 4k does not exist
• DCI specification does not deal with production (only deals with the backend of the chain)
• Digital Cinema is a chance for the European film industry
• It requires an international consortium to solve the fundamental issues together A Research Project in the 6th Framework of the European Community
15 Feb 07 2 Project Overview WorldscreenTM
• Objective: Investigation of the use of scalable video coding in digital cinema workflows • Duration: 09/2004 – 04/2007 • Partners: 6 + 2 advisory (ARRI, Warner Bros.)
• Project Coordinator:
• Budget: 6 Mio. €, Funding: 3 Mio. € • Funded by the European Community A Research Project in the 6th Framework of the European Community (6th Framework) • Website: http://www.worldscreen.org
15 Feb 07 3 Project Partners
A Research Project in the 6th Framework of the European Community
15 Feb 07 4 Scalable Video Coding
For acquisition, postproduction, distribution and archiving
Encoding Scalable Bitstream
SNR Spatial A Research Project in the 6th Framework of the European Community Scalability Scalability
15 Feb 07 5 Technical Workpackages
WP2 WP3 Simulation WP4 Implementation WP5 and Tests and Integration User Assess- Requirements, ment Specifications and and Concepts WP3.1 Compression WP4.1 Evaluation for Data Acquisition Compression for Data Acquisition
WP4.3 Integration and System Test
WP3.3 Transmission, WP4.2 Distribution Transmission, Distribution
WP3.2 Postproduction for Cross Media Distribution
A Research Project in the 6th Framework of the European Community
15 Feb 07 6 Project Details Layered scheme data compression in Digital Cinema workflows
User Requirements Business Models
Acquisition Postproduction Distribution Exhibition
JPEG2000- Use of Scalability JPEG2000/MXF JPEG2000
A Research Project in the 6th Encoder for processing / Standardisation Decoder Framework of the European Fieldrecorder for output to target Community Transmission Quality tests formats
15 Feb 07 7 Layered scheme data compression in Digital Cinema workflows
Acquisition Postproduction Distribution Exhibition
JPEG2000- Use of Scalability JPEG2000/MXF JPEG2000
A Research Project in the 6th Encoder for processing / Standardisation Decoder Framework of the European Fieldrecorder for output to target Community Transmission Quality tests formats
15 Feb 07 8 Layered scheme data compression in Digital Cinema workflows
Acquisition Postproduction Distribution Exhibition
A Research Project in the 6th Framework of the European Community
15 Feb 07 9 Project Achievements
• Developed workflow and metadata models for the entire production chain with LSC • Developed a field recorder for providing a portable storage solution for future D-Cinema cameras including a JPEG2000 encoder • Developed postproduction and transcoding methods (e.g. for (HD-)DVD, TV, streaming formats) from LSC master • Developed scalable distribution and decoding systems (DCI compliant) • Developed a quality assessment methodology for Digital Cinema A Research Project in the 6th Framework of the European Community • Supported research and standardization of layered scheme data compression (both intra and inter) for D-Cinema
15 Feb 07 10 Layered Compression Technologies for Digital Cinematography andA Research Project Cross in the 6th Media Conversion Framework of the European Community
11 Goals – Methodology - Results
• The investigation of the use of layered scheme data compression throughout the entire DC chain and for rich media archives required: – Identification of user requirements – Specification of workflows and metadata models (current & future) – Evaluation of business models
• Methodology: – Interviews with 52 users and manufacturers – Structuring & Consolidation of market feedback
•Results – Deliverables with specifications for workflows and metadata
A Research Project in the 6th – Deliverables covering cost comparisons and business Framework of the European Community models for the value chain
12 Activity Diagrams
Digital Intermediate Post Production LAB Telecine Scan
roll get date rough cut EDL format "circled take" scene numbers numbers type generate pull list
negative developing film scan pull list
original selected film rolls develop clean negative takes
negative digital scans telecine scan
audio telecine & DAT sound synchronize tapes cc (P3) timing report
video tape 444 10bit log DCI compliant DC files HD & SD deliverables encoding for projection review e.g. for rough cut
A Research Project in the 6th Framework of the European UML activity diagrams were extracted from the information Community from the interviews – these formally describe the most typical workflows.
13 Acquisition with JPEG2000 (J2K)
Camera Fieldrecorder Postproduction-SW & with J2K Encoder Plugins
J2K-Import J2K-Export
Uncompressed Fieldrecorder (Megacine)
Display, Keyboad
VTR-Control Dual HD-SDI
J2K-Encoder Dual HD-SDI
Dual J2K- I/O-Modul DVI/Audio HD-SDI Realtime Fibrechannel Encoder (future (near IEEE1394 10G) lossless)
Storage- A Research Project in the 6th Framework of the European Modules Power Community (HDD or Flash)
14 JPEG2000 Workflow
Postproduction Mastering/Validation
DCP- DCP- Generator Analyser
J2K-Export DCP/KDM- DCP- Generator Validator
• Development of Softwaretools for JPEG2000 – Input/Output-Plugins for Digitalfusion – DCP/KDM Generation Tools – DCP/KDM Validation Tools
• Concepts for Archives based on JPEG2000 (will be continued in EU-Project EDCINE)
A Research Project in the 6th Framework of the European • Future Need: J2K-Hardwaresupport in Postproduction Community
15 What is Transcoding all about?
• Conversion of theatrical content into entertainment video formats, e.g.: – SDTV (PAL, NTSC) – HDTV Additional Sources – Streaming video (internet, mobile) of Revenue –etc.
A Research Project in the 6th Framework of the European Community Theatrical Content Entertainment Video
16 Transcoding Workflow – Example
Universal 2K Master (e.g., 2048x1080 pel, R‘G‘B‘ 4:4:4, 12 bit) Transcoding
A Research Project in the 6th Framework of the European Entertainment Video Master Community (1280x720 pel, YUV 4:2:0, 8 bit)
17 Digital Cinema Packages – DCI Model
JPEG 2000 Encoded < 250 Mbps Both Projections + = from the same AES Encrypted < 400 GBytes for DCP + a 3 hour Movie MXF Packaged
2K Projection Hard disk, Satellite... DCP Theater Storage 4K Projection
Decryption Keys sent in parallel
A Research Project in the 6th Key delivery messages protected with RSA Framework of the European Community Only the target projection system is able to decode the messages
18 Digital Cinema Packages – DCI Model
MXF Files
Reel 1
Chunks Image of Data AssetMap PackingList to Composition MXF to reconstruct verify PlayList integrity to put the each Track Audio file of all needed reels together files MXF
Theater Storage
Reel 1 Reel 2 Reel 3 Show Playlist ... Advertisement schedules presentation Reel 1 Reel 2 Reel 3 ... Trailer A Research Project in the 6th Framework of the European Community Reel 1 Reel 2 Reel 3 Projection ... Movie
19 Deutsche Telekom @ WorldScreen
T-Systems is a full service powered by: provider for Digital Cinema
Digital Cinema Factory™: Reliable delivery of Digital Cinema Packages via satellite
Digital Cinema Feature Film™: Server Solution for Multiplex Theatres
A Research Project in the 6th Framework of the European Community
20 Deutsche Telekom @ WorldScreen
Ingest-Archive Operational-Archive
Ingest-ServerIngest-Server DelDel i very-Server i very-Server
LAN
Communication-Communication- Database-ServerDatabase-Server VPN Internet GatGat eway eway
OperatiOperati on-Server on-Server
Digital Cinema Factory™
A Research Project in the 6th Framework of the European Within the WorldScreen project, software components for 4K Community DCP delivery have been developed, co-funded by the EC and the Deutsche Telekom Laboratories
21 J2K Decoder Architecture
Gigabit Ethernet Audio Connection Connector
Input “2k Module Audio data Board”
BUS PCI 32bits@66MHz
JP2 JP2 JP2 JP2 Module Module Module Module #1 #3 #5 #7
JP2 JP2 JP2 JP2 Module Module Module Module #2 #4 #6 #8 Output Module HD-SDI/GbE Serial rapid IO or Projector
This architecture must to be able to decode the 2K at
A Research Project in the 6th Framework of the European 24fps, 2K at 48fps and 4K at 24fps profiles; for the 4K, Community or future higher profile, some parallel 2K-Boards will be necessary.
22 DSP-FPGA Coprocessor in the Decoder Chain
• FPGA performs bit-oriented operation on Tier 1 (computationally expensive) and Inverse Quantization. • DSP completes the decoding chain, performing IDWT on large images. A Research Project in the 6th Framework of the European Community • FPGA acts as coprocessor for DSP, in a loosely coupled architecture.
23 Image Quality Assessment
Kodak Digital Cinema stands for highest image quality, analog AND digital – Multi-format Cineserver JMN3000 with 3D support – Digital Processing / Delivery of over 40 full feature films – Comprehensive Theater Management System – Over 2100 pre-show advertising systems installed
• Motion Quality Ruler (MQR) is a new method for assessing image quality of motion sequences – Can be used to judge wide quality range efficiently and precisely
• MQR method was implemented and validated at a special Kodak test facility in Paris
• MQRs were created for a wide range of scene content and
A Research Project in the 6th origination sources that would be encountered in practice Framework of the European Community
24 Motion Quality Rulers
• MQRs were verified and subsequently applied to test scenes compressed with JPEG2000 at compression ratios of 6:1 to 100:1 • Results indicate feasibility to devise MQRs for individual scenes that can be used to judge quality degradation from compression over large quality range
Fire Engine - 1.4 JND sensitivity Bike - 1.4 JND sensitivity
6:1 12:1 25:1 50:1 75:1 100:1 6:1 12:1 25:1 50:1 75:1 100:1 1
0
-1
-2
-3
-4
JNDs -5
-6
-7
-8 A Research Project in the 6th Framework of the European Community -9
-10 Error bars are 95% Student's t-confidence intervals with respect to the actual number of judges
25 Layered scheme data compression in Digital Cinema workflows
User Requirements Business Models
Acquisition Postproduction Distribution Exhibition
JPEG2000- Use of Scalability JPEG2000/MXF JPEG2000 Encoder for processing / Standardisation Decoder Fieldrecorder for output to target Transmission Quality tests A Research Project in the 6th formats Framework of the European Community
26 Screening - Content
• Advertising Clips –ZKM-Spot (ZKM) – BMW X5 (BMW) –Rimowa (Rimowa)
• Trailer – Happy Feet (Warner Bros.) – Harry Potter (Warner Bros.) – We are Marshall (Warner Bros.)
• Excerpts from Feature Films/Testmovies – Cool Hand Luke* (Warner Bros.) – Ocean´s 11* (Warner Bros.) – As good as it gets (ARRI/Bill Benett) – Mystic India (BAPS/Keith Melton)
A Research Project in the 6th * available on DVD Framework of the European Community
27 Screening - Technology
• DCPs (Digital Cinema Package) – generated and tested by Fraunhofer IIS
• Players – T-Systems Digital Cinema Feature FilmTM – KODAK JMN3000 Digital Cinema Server
• 4k Projector – Sony SRX-R110
A Research Project in the 6th Framework of the European Community
28