MPEGMPEG--21:21: TheThe 21st21st CenturyCentury MultimediaMultimedia FrameworkFramework
Jan Bormans, Jean Gelissen, and Andrew Perkis
IEEE Signal Processing Magazine, March 2003 OutlineOutline z ContextContext andand motivationmotivation ofof MPEGMPEG--2121 z AnAn overviewoverview ofof MPEGMPEG--2121 specificationsspecifications z UMAUMA andand MPEGMPEG--2121 MultimediaMultimedia TechnologyTechnology z MultimediaMultimedia technologytechnology providesprovides contentcontent creatorscreators andand consumersconsumers withwith aa myriadmyriad ofof coding,coding, access,access, andand Content creators and consumers distributiondistribution possibilities.possibilities. z IndividualsIndividuals areare producingproducing moremore andand moremore digitaldigital mediamedia forfor bothboth professionalprofessional andand personalpersonal use.use. More and more digital media are created ContentContent AccessAccess andand DeliveryDelivery z CommunicationCommunication infrastructureinfrastructure isis beingbeing putput intointo placeplace toto enableenable accessaccess toto informationinformation andand multimediamultimedia servicesservices fromfrom anywhereanywhere atat anytimeanytime.. Communicating everywhereeverywhere at anytime z ExistingExisting businessbusiness modelsmodels forfor tradingtrading physicalphysical goodsgoods mustmust bebe extendedextended andand newnew modelsmodels forfor distributingdistributing andand tradingtrading digitaldigital contentscontents electronicallyelectronically areare
required.required. Traditional and new business models ProblemsProblems ofof MultimediaMultimedia z NoNo endend--toto--endend solutionssolutions existexist thatthat allowallow differentdifferent useruser communitiescommunities toto interactinteract inin anan interoperableinteroperable andand standardizedstandardized way,way, thusthus stallingstalling thethe deploymentdeployment ofof advancedadvanced multimediamultimedia packagingpackaging andand distributiondistribution applicationsapplications z UsersUsers areare notnot givengiven toolstools toto dealdeal efficientlyefficiently withwith thethe intricaciesintricacies ofof thisthis newnew multimediamultimedia usageusage contextcontext MPEGMPEG--2121 z In 2002 June, MPEG (ISO/IEC JTC1 SC29 WG11) started working on the definition of enabling normative technology for the multimedia applications of the 21st century: MPEG- 21 “MultimediaMultimedia FrameworkFramework”” z To enable transparent and augmented use of multimedia resources across a wide range of networks and devices z To support transparent and highly automated transactions, especially taking in account z Digital rights management (DRM) requirements z Multimedia access and delivery using heterogeneous networks and terminals DigitalDigital RightsRights ManagementManagement z TheThe desiredesire toto achieveachieve interoperabilityinteroperability maymay bebe inin violationviolation withwith thethe requirementrequirement toto protectprotect thethe valuevalue ofof thethe contentcontent andand thethe rightsrights ofof thethe rightsrights holdersholders z DRM systems can go against the very goal of interoperability if they use non-standardized protection mechanism z To realize an open multimedia infrastructure, more interoperability inin DRMDRM systemssystems are crucial z IPMPIPMP inin MPEGMPEG seriesseries z MPEG-4 describes a set of standard interfaces to proprietary intellectualintellectual propertyproperty managementmanagement and protection (IPMP) systems z IPMP is at the very core of the MPEG-21 specifications HeterogeneousHeterogeneous TerminalsTerminals andand NetworksNetworks
Terminals with different computation and rendering capability
z TheThe heterogeneousheterogeneous terminalsterminals andand networksnetworks makesmakes itit difficultdifficult forfor contentcontent creatorscreators andand serviceservice providersproviders toto ensureensure thatthat theirtheir contentcontent cancan bebe usedused andand renderedrendered inin aa meaningfulmeaningful wayway
Networks with differing bandwidth and characteristics DifferentDifferent PointsPoints ofof ViewsViews
• Content accessing and creating should be offered services with an a priori known subjective quality at a known/agreed price • Network and terminal installation/management/implementation issues should be shielded • “high-level” user parameters, subjective quality and price, need User to be mapped transparently to the underlying network and terminal parameters
• Application serving the user should be able to translate the user requirements into a network “QoS” contract • The contract is handled between the user and the network and Network guarantees the delivery of a given QoS network service • The contract is likely to have a dynamic nature
• The impact on the end-user perception of the variation in resource requirements associated to accessing dynamic, Terminal heterogeneous content should be kept hidden or minimum • The terminal should allow for trade-offs between the resource budgets and the end-user’s perception Media Scalability in MPEG-4 MPEGMPEG--2121 VisionVision
Consumption some metadata Delivery z MPEG-21 aims at defining a discarded
Delivery some metadata normative open framework discarded
Consumption Packaging z For multimedia delivery and consumption for use by all the players in the delivery and Interaction Composition
Library consumption chain z To enable transparent and Commission Synthesis augmented use of multimedia resource across a wide range of networks and devices used Elaboration Analysis by different communities Capture
Production Post-production CurrentCurrent StatusStatus ofof MPEGMPEG--2121
Part Current Status
1 Vision, Technology and Strategies TR
2 Digital Item Declaration (DID) FDIS
3 Digital Item Identification and Description (DII) FDIS
4 Intellectual Property Management and Protection (IPMP) CD
5 Rights Expression Language (REL) FCD
6 Rights Data Dictionary (RDD) FCD
7 Digital Item Adaptation (DIA) CD
8 Reference Software SW
9 File Format WD
10 Digital Item Processing (DIP) WD
11 Evaluation Methods for Persistent Association Technologies Request
12 Test Bed for MPEG-21 Resource Delivery WD TwoTwo EssentialEssential ConceptsConcepts ofof MPEGMPEG--2121 z DigitalDigital ItemItem (DI)(DI) z ““WhatWhat”” ofof thethe multimediamultimedia frameworkframework z UserUser (U)(U) z ““WhoWho”” ofof thethe multimediamultimedia frameworkframework
• Goal of MPEG-21 Defining the technology needed to support users to exchange, access, consume, trade, and otherwise manipulate digital items in an efficient, transparent, and interoperable way. DigitalDigital ItemsItems z DigitalDigital ItemsItems z AA structuredstructured digitaldigital objectobject withwith aa standardstandard representation,representation, identification,identification, andand associatedassociated metadatametadata withinwithin thethe MPEGMPEG--2121 frameworkframework z TheThe fundamentalfundamental unitunit ofof distributiondistribution andand transactiontransaction withinwithin thethe MPEGMPEG--2121 frameworkframework z MPEGMPEG--2121 definesdefines aa setset ofof abstractabstract termsterms andand conceptsconcepts toto formform aa usefuluseful modelmodel forfor definingdefining DIDI inin DIDDID z Digital representation of some work z The unit that is acted upon UsersUsers z UsersUsers z Any entity that interacts within the MPEG-21 environment or makes use of DI z Including individuals, consumers, communities, organizations, corporations, consortia, and governments z A content provider and a consumer are both users. z A user may assume specific or unique rights and responsibilities according to their interaction with other users within the framework
Transaction/Use/Relationship User A ÍDigital ItemÎ User B ÍAuthorization/Value ExchangeÎ
Users are identified specifically by their relationship to another User for a certain interaction InteractionsInteractions betweenbetween UU andand DIDI
Examples: What is the structure of the Examples: Digital Item fundamental unit of distribution •Unique identifiers •Container Declaration •Content Description and transaction? •Item Digital Item What content actually has been Identification and delivered? Description Examples: •Storage Management How is the content used Content Handling •Content Personalization and delivered? And Usage Transaction/Use/Relationship User A ÍDigital ItemÎ User B ÍAuthorization/Value ExchangeÎ Intellectual How are rights controlled in Property respect of each User? Management and Protection Examples: Is the content delivered Terminals and Examples: over a cable line or cell phone? •Resource Abstraction •Usage Permissions Networks •Resource Mgt. (QoS) •Authentication Content Is it natural or synthetic content? Examples: Representation How does it scale? •Error Resilience •Scalability What reportable event has Event Reporting happened and how is it described? Event Reporting provides metrics and interfaces for other six key elements PartPart 1:1: Vision,Vision, Technologies,Technologies, andand StrategyStrategy z FundamentalFundamental purposepurpose ofof thisthis technicaltechnical reportreport z Define a vision for multimedia framework to enable transparent and augmented use of multimedia resource across a wide range of networks and devices to meet the needs of all users z Achieve the integration of components and standards to facilitate harmonization of technologies for the creation, management, transport, manipulation, distributdistribution,ion, andand consumptionconsumption ofof DIs.DIs. z Define a strategy for achieving a multimedia framework by the development of specifications and standards based on well- defined functional requirements through collaboration with other bodies PartPart 2:2: DigitalDigital ItemItem DeclarationDeclaration z ThereThere areare manymany kindskinds ofof contentcontent andand probablyprobably justjust asas manymany possiblepossible waysways ofof describingdescribing itit toto reflectreflect itsits contextcontext ofof use.use. z AA powerfulpowerful andand flexibleflexible modelmodel forfor DIsDIs mustmust bebe ableable toto z AccommodateAccommodate thethe myriadmyriad formsforms contentcontent cancan taketake nownow andand inin thethe futurefuture z UsedUsed toto representrepresent anyany DIDI unambiguouslyunambiguously z UsedUsed toto communicatecommunicate DIsDIs successfullysuccessfully PartPart 2:2: DigitalDigital ItemItem DeclarationDeclaration (cont.)(cont.) z Two Examples of DI comprising a series of media resources
MPEG-21 Music Album Web Page HTML file
Elephant … Go!!Go!! Links Scripts Audio files (Track)
Text file Image Image Text file Image Image Video (Lyrics) (Cover) (Introduction) (Artwork) (GIF) (JPEG) ? PartPart 2:2: DigitalDigital ItemItem DeclarationDeclaration (cont.)(cont.) z RelationshipsRelationships betweenbetween thethe resourcesresources andand howhow theythey relaterelate toto thethe DIDI itselfitself isis defineddefined inin thethe DIDDID z DIDDID isis aa documentdocument thatthat specspecifiesifies thethe makeup,makeup, structure,structure, andand organizationorganization ofof DIDI z Three normative clauses z Model z Abstract terms and concepts to form a model for defining DIs z Representation z DescriptDescriptionion of the syntsyntaxax and semantics of each DI declaration elementslements ( represented in XML) z Schema z The XML schema comprising the entire grammar of the DID representation PartPart 3:3: DigitalDigital ItemItem IdentificationIdentification andand DescriptionDescription z BesidesBesides referencesreferences toto thethe resources,resources, thethe DIDDID cancan containcontain informationinformation aboutabout thethe itemitem itselfitself andand consistingconsisting partsparts.. z CurrentCurrent situationsituation z Proprietary identifying and description systems co-exist with standardized schemes z Some identifiers have been successfully implemented and commonly used, but they are specific to individual media types z The majority of content lacks identification and description. z There is no mechanism to ensure the identity and description informationinformation isis persistently associated with the content z Through file headers z Through digital watermarking Part 3: Digital Item Identification and Description (cont.)
z DIIDII providesprovides aa normativenormative wayway toto expressexpress howhow thethe identificationidentification cancan bebe expressedexpressed andand associatedassociated withwith DIs,DIs, containers,containers, components,components, andand fragmentsfragments byby includingincluding themthem inin aa specificspecific placeplace inin thethe IDID z DI and resources are identified by encapsulating Uniform Resource Identifier (URI) intointo thethe identifieridentifier elementelement z Likely identifiers include descriptive, control, revision tracking and identifying info. InteroperabilityInteroperability inin DRMDRM z IPMPIPMP inin MPEGMPEG--44 z Hooks z A set of standard interfaces to proprietary IPMP systems, deeply embedded in MPEG-4 systems z If you want to play some content, you only need to plug in the rightight IPMPIPMP system, and where to obtain it can be signaled in the bitstream z Interoperability loss z E.g. a portable music player cannot download a IPMP system z InteroperabilityInteroperability inin DRMDRM z A very difficult problem since standardized trust is needed z E.g. content owners must trust all the players that consume contents Part 4: Intellectual Property Management and Protection z MPEGMPEG--2121 partpart 44 definesdefines anan interoperableinteroperable frameworkframework forfor IPMP,IPMP, builtbuilt furtherfurther onon IPMPIPMP workwork inin MPEGMPEG--4,4, includingincluding z StandardizedStandardized waysways ofof z Retrieving IPMP tools from remote locations z Exchanging messages between IPMP tools z Exchanging messages between IPMP tools and terminals z AuthenticationAuthentication ofof IPMPIPMP toolstools z IntegratingIntegrating rightsrights expressionsexpressions accordingaccording toto RELREL andand RDDRDD PartPart 5:5: RightsRights ExpressionExpression LanguageLanguage z MPEGMPEG--2121 PartPart 4:4: RightsRights ExpressionExpression LanguageLanguage isis aa machinemachine--interpretableinterpretable languagelanguage intendedintended toto provideprovide mechanismsmechanisms toto supportsupport useuse ofof digitaldigital contentcontent inin aa wayway thatthat protectsprotects digitaldigital contentcontent andand honorshonors thethe rights,rights, conditions,conditions, andand feesfees specifiedspecified forfor digitaldigital contents.contents. z ItIt alsoalso supportssupports specificationspecification ofof accessaccess andand useuse controlscontrols andand exchangeexchange ofof privateprivate oror sensitivesensitive digitaldigital contentcontent PartPart 6:6: RightsRights DataData DictionaryDictionary z AA dictionarydictionary ofof keykey termsterms requiredrequired toto describedescribe rightsrights ofof thosethose whowho controlcontrol DIs,DIs, includingincluding intellectualintellectual propertyproperty rightsrights andand thethe permissionpermission theythey grants.grants. z PartPart 55 andand PartPart 66 togethertogether allowallow thethe expressionexpression ofof rightsrights inin anan interchangeableinterchangeable formform usingusing aa standardizedstandardized syntaxsyntax (REL)(REL) andand standardizedstandardized termsterms (RDD).(RDD). UniversalUniversal MultimediaMultimedia AccessAccess Mobile Phone z UMA deals with the Broadcast (3G terminals) delivery of media resources under different network conditions, User preferences, and capabilities of terminal devices. UMA Network z Wired and wireless
systems can access the Desktop same media resource Media Resource provider and receive media Provider resources enabled for their system capability z UMA will be a driving force PDA UMA will be a driving force Laptop behind the development of services in 3G systems The UMA concept PartPart 7:7: DigitalDigital ItemItem AdaptationAdaptation
Digital Item Adaptation Engine
Resource Adaptation Engine Adapted Digital Item Digital Item Description Adaptation Engine
•Adaptation engines are non- normative •Descriptions and format- DIA Tools independent mechanisms are normative
DIs are subject to a resource adaptation engine, as well as a descriptor adaptation engine, which together produce the adapted DI PartPart 7:7: DigitalDigital ItemItem AdaptationAdaptation (cont.)(cont.)
Requirements on Usage Environment Description
Terminal capabilities Network capabilities User info. •Delay characteristics •User preferences •Acquisition properties •Error characteristics •Demographic info. •Device type and profile •Bandwidth characteristics •Output properties •H/W and S/W properties Natural Env. •System properties Delivery capabilities •Location •IPMP capabilities •Type of location •Supported transport •Available access protocol types network •Supported connection •Velocity types Service capabilities •Illumination •User roles •Type of service Interactions and relations
Mechanisms •Taking usage environment changes into consideration Vocabularies •Dynamic updating of descriptions PartPart 7:7: DigitalDigital ItemItem AdaptationAdaptation (cont.)(cont.) z RequirementsRequirements onon mediamedia resourceresource adaptatibilityadaptatibility z FormatFormat descriptionsdescriptions independentindependent ofof actualactual contentcontent representationrepresentation formatsformats z DescriptionDescription ofof scalablescalable contentcontent representationrepresentation formatsformats z DescriptionsDescriptions thatthat cancan bebe automaticallyautomatically extractedextracted fromfrom thethe resourceresource inin aa formatformat--independentindependent wayway z DescriptionDescription ofof resourcesresources inin termsterms ofof perceivedperceived qualityquality andand associatedassociated processingprocessing complexitycomplexity z DescriptionDescription ofof metadatametadata inin termsterms ofof perceivedperceived importanceimportance andand associatedassociated processingprocessing complexitycomplexity RelatedRelated WorkWork z MediacomMediacom 20042004 z Established by ITU z A framework for the harmonized and coordinated development of global multimedia communication standards z CapacityCapacity--exchangeexchange relatedrelated standardsstandards z W3C z HTTP 1.1’’s content negotiating z Composite Capability/ Preferences Profile (CC/PP) for terminal to adapt content z Web Forum z IETF z ISO z DVBDVB--MHPMHP ConclusionConclusion z MPEGMPEG--2121 offersoffers excitingexciting solutionssolutions toto supportsupport interoperableinteroperable exchanging,exchanging, accessing,accessing, consuming,consuming, trading,trading, andand otherwiseotherwise manipulatingmanipulating DIsDIs betweenbetween UsersUsers inin anan efficient,efficient, transparent,transparent, andand interoperableinteroperable wayway z MPEGMPEG--2121 providesprovides DIDI AdaptationAdaptation toto enableenable UMAUMA scenarios.scenarios.